> In practice what I see fail most often is not premature optimization but premature abstraction
This matches my experience as well.
Someone here commented once that abstractions should be emergent, not speculative, and I loved that line so much I use it with my team all the time now when I see the craziness starting.
Similar to when someone said that the rule of thumb should not be DRY (don't repeat yourself) but WET (write everything twice) - that is, be happy to repeat similar code once or twice, and wait for the need for abstraction to become evident.
It's advice I like, as I'm prone to falling into design paralysis while trying to think of the One True Abstract Entity.
Some abstractions are obvious though. There is value in reusing them across projects and not reinventing the wheel each time. I don’t think this is counter to what you propose necessarily, just adding the nuance that abstractions serve another purpose: communicating intent to the readers of the code. Sometimes a concrete type is unnecessary detail.
Absolutely. I don't take that quote to mean we should all start from ground zero every time...if so experience wouldn't have much value.
I more think the point is that the abstractions now obvious to you emerged at some earlier point that you now recognize, and application makes sense. Speculative implies predicting, and if you've been around long enough in the same domain, it's likely you aren't guessing.
It's probably something that applies more to younger devs than older, but not exclusively of course.
But they should emerge right when they are needed, not some time later via a painful refactor.
I think every dev lives in fear of the situation where abstraction hasn't been done when it really should have. Maybe some junior dev came in after you and shoehorned in a bunch of features to your "simple" solution. Now you have to live with that but, guess what, there's no time to refactor it, in fact the business just wants even more features.
As usual these rules work best if everyone on the team understands them in the same way. If you work with people who can only see what is right in front of them (ie. the current feature), then it'll never work. You can always fit "one more feature" into the perfect codebase without thinking about the abstractions.
In a perfect world, sure. In my personal experience, refactoring to introduce an abstraction is far less painful than refactoring to remove or fix one.
Usually because intent is far clearer in the former case.
Premature abstraction is one of the worst pitfalls when writing software. It's paid for up front, it costs developer time to understand and work with, it adds complexity (tech debt), increases likeliness of bugs, and increases refactor time. All for someone to say "if we ever want to do X". If we wanted to do it, we'd do it now.
I truly believe this comes from devs who want to feel smart by "architecting" solutions to future problems before those problems have become well defined.
I think it mostly happens because a little bit of abstraction is nearly always uncontroversially good. If you want to print a line of text 5 times, you'll instinctively write a for loop to do it, instead of copy-pasting the original print() statement an extra 4 times. The cognitive overhead upon reading this code is near zero, since we're all so familiar with it. This is abstraction, nonetheless.
So a little bit is always good, and more is sometimes very good -- even memorably good. Together these cause many of us to extrapolate too far, but for understandable reasons.
Philip Wadler gave us a sound type system. Rob Pike gave us a programming language for stupid people. Each contributed in their own way, towards different goals.
This comment is fascinating to me, as it indicates an entirely different mindset than mine. I'm much more interested in code readability and maintainabilty (and simplicty and elegance) than performance, unless it's necessary. So I would start by saying everything flows from rule 4 or maybe 5. Rule 1 is a consequence of rule 4 for me.
Maybe it's because the comment you are replying to is from a new account posting paragraphs of LLMese in multiple comments in the same minute. It's unsurprising that soulless LLM output doesn't match your mindset!
I was just dealing with this is some home-grown configuration nightmare where every line of the file was pulled from some other namespace to the point where you end up with two dozen folders, each with half a dozen partial configuration files in them, and it takes half a hour to figure out where a single value is coming from.
I'm sure it's super flexible but the exactly same thing could have been achieved with 8 YAML files and 60% of the content between them would be identical.
As someone who believes strongly in type based programming and the importance of good data structure choice I'm not seeing how Rule 5 follows Rule 1. I think it's important to reinforce how impactful good data structure choice is compared to trying to solve everything through procedural logic since a well structured coordination of data interactions can end up greatly simplifying the amount of standalone logic.
Data cache issues is one case of something being surprising slow because of how data is organized. That said, Structure of Arrays vs Array of structures is an example where rule 4 and 5 somewhat contradict each other, if one confuses "simple" and "easy" - Structure of Array style is "harder" because we don't see it often; but then if it's harder, it is is likely more bug-prone.
But good data structure is not always evident from the get go. And if your types are too specific it would make future development hard if the specs change. This is what I struggle with
Professionally I'm a data architect. Modeling data in a way that is functional, performant and forward facing is not an easy problem so it's perfectly fine to struggle with it. We do our best job with what we've got within reasonable constraints - we can't do anything more than that.
I found that over time my senses have been honed to more quickly identify things that are important to deeply study and plan right now and areas where I can skimp more and fix it later if problems develop. I don't know if there was a short cut to honing those senses that didn't involve a lot of pain as I needed to pick apart and rework oversights.
Good strong (read specific) types encourage easier redactors.
Changing the function signature or the type then generated cascade of compiler errors that tells you exactly what you touched.
Weak non specific types does not have that property and even with tests you cannot be sure about the change and cannot even be sure you are upholding invariants
I only agree if you have a bounded dataset size that you know will never grow. If it can grow in future (and if you're not sure, you should assume it can), not only will many data structures and algorithms scale poorly along the way, but they will grow to dominate the bottleneck as well. By the time it no longer meets requirements and you get a trouble ticket, you're now under time pressure to develop, qualify, and deploy a new solution. You're much more likely to encounter regressions when doing this under time pressure.
If you've been monitoring properly, you buy yourself time before it becomes a problem as such, but in my experience most developers who don't anticipate load scaling also don't monitor properly.
I've seen a "senior software engineer with 20 years of industry experience" put code into production that ended up needing 30 minute timeouts for a HTTP response only 2 years after initial deployment. That is not a typo, 30 minutes. I had to take over and rewrite their "simple" code to stop the VP-level escalations our org received because of this engineering philosophy.
> You're much more likely to encounter regressions when doing this under time pressure.
There is nothing to suggest you should wait to optimize under pressure, only that you should optimize only after you have measured. Benchmark tests are still best written during the development cycle, not while running hot in production.
Starting with the naive solution helps quickly ensure that your API is sensible and that your testing/benchmarking is in good shape before you start poking at the hard bits where you are much more likely to screw things up, all while offering a baseline score to prove that your optimizations are actually necessary and an improvement.
I agree. My team usually tries to split functions into pieces that are too small, which makes debugging really hard. They try to cover for scenarios that may never happen
It would be easier if we could just block comments from green users. I get that it loses ~.1% of authors who might have made an account to comment on a blogpost of theirs that was posted here. I'd rather have that loss than have to deal with the 99.9% of spam.
You tend to see it a lot in "Enterprise" software (.Net and Java shops in particular). A lot of Enterprise Software Architects will reach for their favored abstractions out of practice as opposed to if they fit. Custom solution providers will build a bunch of the same out of practice.
This is something I tend to consider far, far worse than "AI Slop" in practice. I always hated Microsoft Enterprise Library's Data Access Application Block (DAAB) in practice. I've literally only ever seen one product that supported multiple database backends that necessitated that level of abstraction... but I've seen that library well over a dozen times in practice. Just as a specific example.
IMO, abstractions should generally serve to make the rest of the codebase reasonable more often than not... abstractions that hide complexity are useful... abstractions that add complexity much less so.
