too much confidence that we were building the right thing
SW is weird in that way, we can basically build houses that nobody wants to live in. Thing is you probably were building the right thing, structurally speaking.
My rule over time has become "structure follows complexity" i.e. Anticipate complexity where possible but don't apply the engineering structure until the complexity really exists. If you built an MVP house, don't add the second bathroom until you've got customers complaining about deadlocks on the first.
It's tough because that basically runs counter to the Spirit of an Engineer, which is just to imagine the grandest well-thought-out + organized thing and march toward creating it.
The bonus of having two decades+ in building SW though is you start to see the patterns and realize that the same fundamental needs drive everything, so the intersection of building something great with customer needs becomes less of a mystery and more of a "how fast can we juice this thing". At that point I think scalability prep is like compound interest.
There’s a ninety ten rule with structure that can be hard to communicate in code reviews. There’s a subtle difference between not adding unnecessary structure and writing code that is antagonistic to it being added later. Opening a PR where someone added a feature or fixed a bug exactly where I would have done so is one of the best feelings. Especially when they’re changing my code. Either they’re a very good coworker or I’ve laid the groundwork well. Either of these are good news.
Very recently I’ve come to think of my architecture style as Jazz. Like the old trope about listening to the notes he isn’t playing. But also the improvisational nature and the flexibility to adapt to new information.
> There’s a subtle difference between not adding unnecessary structure and writing code that is antagonistic to it being added later
This is such a wonderfully concise way of describing something that I try to teach people.
I have come to use the phrase "no regrets". Leave things out, but do it in a way that you won't regret it, if you need to add it later.
Another phrase that I read somewhere else was, "Weeks of programming can save you hours of planning" (pretty sure I read it on HN, and I wish I had kept the link). The point being that when you decide to leave something out, or when you decide to intentionally do a half-assed job of something, you should still think about how someone in the future would go about efficiently fixing it.
If you do it right, there's missing functionality, but little or no technical debt. If you do it wrong, you miss out on functionality and also take on technical debt. And I think that's the subtlety that you speak of.
Most people who push for early design understand that they have more control of the project at the beginning, before deadlines accelerate, before a bunch of bad hires and competing priorities. And they are definitely speaking from a place of regrets. This hurt so much last time I vowed never again.
There’s a list of things I won’t try to add on later. It’s smaller than it used to be, but the shiv I brandish at you for suggesting we cut those corners is also much scarier.
Jim Highsmith has the best theory I’ve heard in the subject. His opinion is that the reason we keeps having the same unwinnable arguments is because we think in terms of solving problems but the real issue in front of us is resolving paradoxes. There are no neat answers that fit in a short essay box on the test everything depends, and everything depends on everything, and something is always changing.
What the PTSD reactions come from is other people making decisions on behalf of the rest of the team without consent. But like most things in software it’s a 2 dimensional problem not a 1 dimensional one. The trick is to delay irreversible decisions as long as possible and get all responsible parties involved.
The flip side of the coin is that reversible decisions should be made as cheaply as possible. For instance my “Kobayashi Maru” solution to the bike shedding problem is that the team should have budgeted three colors of paint into the bike shed budget, picked green for the first coat, and moved on immediately. That $200 is far cheaper than the time and importantly energy that would go into discussing this at a meeting with senior staff members present. If you start thinking about all of the $5000 conversations you have to solve $2000 problems it’ll drive you mad. I almost recommend you don’t look at it, for your own safety. Almost.
> The trick is to delay irreversible decisions as long as possible and get all responsible parties involved
This is exactly right, but the irreversible decisions are often horizontal slices of software design that underlie other decisions - thing like authz, IPC, API design etc. If these questions can be identified and resolved up front, the cost of building vertical slices - features - goes way down. You're not tempted to reinvent the wheel, you can focus on the business logic.
What I'm saying is: it's worth spending time planning the foundations of a product. That includes the technology choices, but also the high level logical layout of the whole product. If you approach this foundational stuff like any other project, then if you're doing it right, you'll "get all responsible parties involved", like you should.
But defer the detailed planning of domain logic until you're actually ready to write it, and remain flexible with the plans you do make. "Plans are nothing; planning is everything".
Because like the GP said,
> the later you design something the more info at hand you have to make the design fit better
I find it remarkable how often engineers just leap in and start coding, and worry about the foundations later. Or never. I suspect this is one reason why rewrites are so common. If you start with bad foundations, it's often easier to blow up the whole building and start again.
> I find it remarkable how often engineers just leap in and start coding, and worry about the foundations later. Or never. I suspect this is one reason why rewrites are so common. If you start with bad foundations, it's often easier to blow up the whole building and start again.
To be fair, leaping in and starting coding allows you to discover the problem better. The problems start where you get stuck with your initial design and don't adjust/remake it once you explored some ideas, maybe found some new along the way and found out what doesn't fit.
That is of course problematic if you have deadline driven development because rewriting last 2 weeks of code might seem like total waste even if then-better design gonna save you much more even in near feature.
Not rewriting code when design flaws appear will lead to rewriting it eventually later and more expensive.
Of course, all of that depends on how much you know about domain and the problem. Exploring existing solutions and their flaws or having really detailed requirements might be much better approach that doesn't require starting to write code.
Some things, when you cut the corners off you also cut your fingers off.
We have all promised that we will “fix it later”. But whatever we do now is what’s done. Fixing it later is a promise whose resolution is often not in our control.
I suspect this dynamic is a big part of why Joel Spolsky jokes about transforming from a technical leader into an amateur psychologist. We are all lying to ourselves first and to others second.
Or as Feynman said, the trick is not to fool yourself, and you’re the easiest person to fool.
Or a personal favorite, unattributed: I know the guy who made this plan [me] and he’s full of shit.
I believe that quote is thought to have come from Frank Westheimer or at least he is associated with an earlier version of it.
“Why spend a day in the library when you can learn the same thing by working in the laboratory for a month?”
Jazz is an excellent analogy. Been in software dev for 30 years. It's hard to explain to hirers that one can only play jazz well (i.e. know what matters in a software project as you go) after a long time, so trust me :)
I realized at some point I got put in charge of things to shut me up. Someone would ask me why and I would launch into a treatise on all of the confounding factors and cognitive science reasons and and and OMG shut up already. Clearly you care about this more than I do, why don’t you do it?
Don’t put a hyperkinetic person in charge of something thinking that you’ll call their bluff.
>It's tough because that basically runs counter to the Spirit of an Engineer, which is just to imagine the grandest well-thought-out + organized thing and march toward creating it.
I'm not sure if Spirit of an Engineer is a book, essay, or some prior set of principles I'm unaware of but if it's not (and even if it is), I tend to disagree here.
The spirit of of an engineer, in my opinion, is to solve problems and problems for people--that's what I and most engineers I've spoken with have been drawn to (ignoring those purely seeking a high paying profession here since were talking about "spirit"). When I was youthful I sought out technical complexity and grandiose solutions (I admittedly still admire it when it's done correctly). None the less, to me, most of it is wasteful and at the end of the day I want to build the most useful solution for people to make their lives easier.
Some problems require high complexity and glass cathedrals in terms of technical solutions, most really don't, at least most of the problems I've been exposed to.
Actual engineers generally do try to solve problems for people. But the engineers understanding of the people that they are solving it for is generally rather flawed. And therefore their solutions frequently are difficult for the intended users for reasons that the engineer can't see.
And yes, engineers really do err on the side of designing for future problems and future users that often will never exist. This effect is worst on a rewrite of a system - look up the "second system effect". We tend to only truly get pragmatic tradeoffs the third and later times.
Maybe “spirit of an engineer” could be better phrased as “visionary mindset”. Lots of engineers I’ve worked with imagine potential problems and treat them as requirements to solve even though the problem may never actually materialize.
Figuring out which problems to solve before they happen and ignoring the ones that probably won't happen is the cosmic dance of software development. The Serenity Prayer states "... accept the things I cannot change, courage to change the things I can, and wisdom to know the difference." Except change "cannot"/"can" to "don't need to at least not yet"/"need to now because it's about to be an issue!".
Make a bare bones version and see if it sticks, and go from there. Both the Progressive Enhancement philosophy and the Lean Startup methodology demand the minimal before building further. Ex: Will anyone click this survey link? Who knows but you'll learn real quick if you offer them $50. If they still don't, then it probably ain't gonna get clicked ever. If they do, then you found at least _something_ viable.
I'm not sure if Spirit of an Engineer is a book, essay, or some prior set of principles I'm unaware of
Good idea, you are welcome to help me hone my later chapters. For me the engineering spirit - especially in software - is being a creator/builder. At the core, someone who takes the Legos in front of them and creates with those pieces.
A couple people took my "grandest" comment as a nod toward scale/size/narcissism, which is reasonable, and I meant it more in terms of total architectural completeness - i.e. a system perfectly fulfilling all the requirements that necessitated its creation. A blade of grass is not "grand" in the primary sense, but it still is grand + magnificent by its own perfection in design.
It's always a balance. You need some structure to avoid spaghetti, maybe a bit more to allow for plausible extensibility (microscopic pre-abstraction) but not too much you're adding fat.
"keeping an escape hatch". I like that. A former colleague said something similar that also stuck with me. Good software architecture allows you to delay decisions.
We talking Dandong, city at the border of China and North Korea? They built everything including a giant bridge, high-rise apartments, shopping centers, parks, and a ferris wheel. But there's no people and it's been ~10 years now. It is, however, still fully maintained. [0]
Yup. Literally first sentence of my citation "Dandong New City, a district of Dandong which was built from scratch to be a China-North Korean trade hub, now stands mostly empty." [0]
A large number of them being filled and another large number of them YET not being filled still makes dozens of millions of cities coming to being out of nowhere and providing people with affordable, well-planned urban environments instead of using the demand to skyrocket the real estate prices at the cost of causing homelessness and suffering.
Yours is an incredibly good comment. The comparison software/architecture is something that I have encountered here and there over the years, but you brought it to a new level. "don't add a second bathroom until you ve got customers complaing about deadlocks on the first". Beautiful. Thanks
SW is weird in that way, we can basically build houses that nobody wants to live in. Thing is you probably were building the right thing, structurally speaking.
My rule over time has become "structure follows complexity" i.e. Anticipate complexity where possible but don't apply the engineering structure until the complexity really exists. If you built an MVP house, don't add the second bathroom until you've got customers complaining about deadlocks on the first.
It's tough because that basically runs counter to the Spirit of an Engineer, which is just to imagine the grandest well-thought-out + organized thing and march toward creating it.
The bonus of having two decades+ in building SW though is you start to see the patterns and realize that the same fundamental needs drive everything, so the intersection of building something great with customer needs becomes less of a mystery and more of a "how fast can we juice this thing". At that point I think scalability prep is like compound interest.