The first diagram in that article is incorrect/quite outdated. Modern computers have most PCIe lanes going directly into the CPU (IO Hub or "Uncore" area of the processor), not via a separate PCH like in the old days. That's an important development for both I/O throughput and latency.
Otherwise, great article, illustrating that it's queues all the way down!
Thanks for the comment, and you're right, modern computers do have a much better architecture! As I was laying out the story I was thinking about what it looked like when we started. I'll clarify that in the image caption that it's from that era.
I'd say the lack of flexible boards isn't really about lane count.
A B650(E) board has enough lanes to give you a fast GPU slot, four CPU x4 slots running at 8-16GB/s each, and four chipset x1 slots running at 2GB/s each.
The industry has made a market segment for this/largely ignored it, 'HEDT'. I'd say this is evidence to your point, GPUs being the motivator. I'm starting to think I'm the only one still using add-in NICs, sound cards, or bifurcation.
The ATX spec allows for 7 or fewer physical slots. That's not what's letting me down!
How many lanes are actually available and their division changes over time. Everything from USB to SATA has internally been moving to PCI-e, there's more 'internal consumption', too.
Gen5 is kind of a victim of success. Each individual lane is very fast, but providing them to a fair number of add-in cards/devices isn't practical. Simply physically, but also bandwidth between lanes.
Pretty much all of gen4 is nice, those systems are perfectly viable. More lanes that were slower offered me more utility. It's multi-faceted, I overstate it too. I like hyperbole :)
Thanks for a very informative article. It would be interesting have more detail about how EBS achieved a strong culture around quality and system resilience. Maybe a future post?
Otherwise, great article, illustrating that it's queues all the way down!