Side note 1: ES can also be embedded in your app (on the JVM).
Note 2: I actually used RocksDB to solve many use cases and it’s quite powerful and very performant. If anything from this post take this, it’s open source and a very solid building block.
Note 3: I would like to test drive quickwit as an ES replacement. Haven’t got the time yet.
1 - I think if we were sticking with the JVM, I do wonder if Lucene would be the right choice in that case
2 - It's a great tool with a lot of tuneability and support!
3 - We've been using it for K8s logs and OTEL (with Jaeger). Seems good so far, though I do wonder how the future of this will play out with the $DDOG acquisition.
I really enjoy embedding things in the vm. I run a discord bot with a few thousand users with embedded H2. Recently I’ve been looking at trying to embed keycloak (or something similar) for some other apps.
I did that with ES to squeeze performance but IIRC it didn’t really produce meaningful results. Otherwise, for most use cases an integration is better imho than embedding stuff that is when you have a full software service such as keycloack or ES. Rocksdb and h2 are tailor made as embedded libraries
