Suitability of EventStoreDb For A system with a high number of streams

What you describe is a hard delete. When you hard-delete a stream, ESDB will tombstone it with a special marker, as you said, so they won’t be recreated again. However, for archiving scenarios it’s not really what you’d do. You’d rather soft-delete the stream, potentially archiving its content to something like an S3 bucket. When you’d need to restore the stream from the archive, you’d just put all the events back. That’s the strategy we use in production. I think when a stream is soft-deleted, we still keep its metadata (which is an event) so when you write to the same stream again, it will keep the numbering from the last known event number, etc. It should not really affect the database performance though as the metadata event is very small.

Concerning long streams, our product fits more to the scenario where one event stream represents one entity. In such a case, streams with millions or even thousands of events usually indicate a design smell, as the entity boundaries are probably identified incorrectly, and that particular entity takes too much responsibility, so it needs to be split up. A common case is something that we can call Product, where things like product metadata, stock level changes due to both sales (decrease) and filling up stock (increase), are all in the same stream. I would never design a system like that as it has multiple contexts missed together, along with all the different responsibilities.

Also, you mention reading all the events from a stream in one line with catch-up subscriptions. In EventStoreDB these are two different use cases. You’d read a stream to reconstruct an entity state as all the events represent individual state transitions. You subscribe to a stream, which contains events ($all) or links (like $ce) from multiple streams to compose use case-specific read models (materialised views), and those subscriptions run continuously, forever, until the read model demand is gone, and it’s replaced with another one.

1 Like