I recently completed an Endeca to Algolia migration. I spent quite a bit of time auditing the Endeca implementation, from the XML files to the Windows desktop application. It was pretty good for its era.
While the acquisition was good for me financially, I agree that Oracle didn't really invest in sustaining Endeca as a product. I am proud of what we achieved at Endeca, but search has come a long way since then.
> The big difference is that duplicate index entries can still occur in B-tree indexes. GIN indexes have to integrate new entries into the posting list every time, which makes data modifications less efficient (this is mitigated by the pending list). In contrast, PostgreSQL deduplicates B-tree entries only when it would otherwise have to split the index page. So it has to do the extra work only occasionally, and only when it would have to do extra work anyway. This extra work is balanced by the reduced need for page splits and the resulting smaller index size.
> Back in February, Lipsitch gave a very rough estimate that, absent intervention, herd immunity might happen after 40 to 70 percent of the population had been infected.
Which is actually a mischaracterization of what the previous article[1] said:
> Lipsitch predicts that within the coming year, some 40 to 70 percent of people around the world will be infected with the virus that causes COVID-19.
At no point does he mention herd immunity. So, no, he's not saying herd immunity at 40%.
I would be very hesitant to turn an ORM into a "smart" SQL generator. Depending on the type/distribution of data, there are sometimes very different paths to optimizing a query. The ORM as a "stupid" SQL generator (straightforward mapper) is a great way to allow for the flexibility to control how a query is generated.
The DB engine might be the place for those sort of improvements.
In this case the DB engine was not the place for those improvements, as the majority of the extra time was being spent in the "Compile" method which merely composes SQL. If you're worried about turning the ORM into a smart sql generator, having second thoughts about that, in Rails and in ActiveRecord, I'd have to say that ship has already sailed. The ORM is very smart and usually generates pretty good SQL for you. It's never a bad idea to review what it generated and double check based on your own DB knowledge, which might always be better informed than a robot's, but...
Early in the profiling part of the talk Aaron shows how he nearly got hoodwinked into thinking that differences in the query itself were what was causing the slowdown, and while in the end two different queries are still generated by the two different versions of ORM code, the bulk of the performance capture is reclaimed without any impact from the query difference, at least a ~30% performance discrepancy comes solely from the object side of the equation, in Ruby.
(Is it a string or integer parameter? That might make a difference in the query performance... is it one bound array parameter, or one parameter binding per array element? That could only make a difference on the Ruby side, as bound SQL params are always mapped into a query as individual values, at least in this example. These factors are all in play.)
It's a long talk but it's really interesting, (I set a time index in the link to get you past the most frivolous and off-topic parts, which you usually find in a tenderlove talk... that part is not for everyone) I think this talk probably has something for everyone, even if you're not a Rails dev.
I'm in a one bedroom Brooklyn apt. Any time I open up my laptop, my 2yr old daughter climbs onto my lap and repetitively hits the caps lock key (it lights up on my laptop). Luckily, I've found alternative places to work from, albeit with less than ideal setups.
