Is there any value in creating a system providing delivery guarantees for pub/sub EVM log events? I think i have an idea that i am ready to get stones thrown at. :P

Firstly thank you for reading/responding.

15yr+ Enterprise developer here.

I have an idea for a product offering (and a POC almost done) that i think Alchemy/Infura might be missing, but i am totally ready to have my bubble popped here. There might be a great reason no one has done this before, so im ready to hear that.

Just incase there is not a good reason: i think that I might see an issue with Alchemy's pub/sub that makes it kindof sub-par for a few important use-cases on the blockchain - especially when enterprise adoption is concerned. Here are my concerns:

• eth_subscribe is not fault-tolerant - in that there is no stateful buffer for your event. It's fire-and-forget over the websocket and if you're not there, you miss it.
• eth-newfilter + eth_getFilterChanges seems like a rad solution, but it has never worked for me thru alchemy. After about 30seconds the event filter deletes itself regardless of how often i call eth_getFilterChanges.

Either way - this seems to slightly miss the point of wanting a reliable queue as it relies on polling (likely 2 levels of polling - once internally by alchemy to gather logs between block-spans, and once from the "subscriber" to pull the logs over the wire)

I feel like there may be a product offering hidden in my learnings of using these systems at scale.

For example: I cant see a great way (and i might just be wrong, so please point me to solutions if you know of them) to use these Alchemy pub-sub systems to efficiently do these things at scale:

1. Give me all board-ape transfers since bored-apes were created, then keep watching forever.
2. Give me all transfers to the 0x00 address for contract X between block Y and Z.
3. Give me all the addresses which have EVER owned a bored ape regardless of current balance.

Im not promoting because its still just an idea, but I have a POC of a website that a subscriber could integrate with using like 5 lines of JS code. You could subscribe with at-least-once delivery of blockchain topic filters.

For those curious: the stack is NextJS+cognito+K8s+pg+rabbitMq

But having rabbitMq means that if your app goes down you will still get your events with a guarantee. It also means that :

• you (the subscriber) can parallelize the HECK out of processing of these logs.
• we (the service gathering the logs) can also parallelize the query/aggregation of the events.
• we (the service gathering the logs) can pull events from "latest" blocks while watching for you and re-queue them if they are involved in an re-org. That means you can safely operate closer to the head of the blockchain and possibly get events sooner than other systems could safely allow.

The question i have for this sub is...Is this a complete waste of time, or would someone find value in this?