I've been quite happy with multi module projects over the years. I.e. where your modules are a directed graph of dependencies. Whatever modular structure you have right now is never perfect, but refactoring them can be straightforward, which does require active pursuit of the ideal, yes. You start such a project with an educated guess about the module structure, then the team goes heads down developing stuff, then you come up for air and refactor the modules. If you don't do that part then things get increasingly crufty. I think a best practice in these environments is to not worship DRY too much, and instead encourage people to do non-scalable things in the leaf libraries, then later refactor re-usable stuff and move it to the branch libraries.
It helps very much to be in a language and build environment that has first class modularity support. I.e. good build tooling around them and good IDE support. And at the language level, good privacy constraints at the package level, so the deeper libraries aren't exposing too much by accident.
What modules patterns have I seen work over the years? Generally, having business logic and data access apis lower in the tree, and branching out for different things that need to be deployed differently, either because they will deploy to different platforms (say, you're running business logic in a web server vs a backend job), or because they are deployed on different schedules by different teams (services). A nice thing about the architecture is that when you have a bunch of services, your decision as to what code runs on what services becomes more natural and flexible, e.g. you might want to move computationally expensive code to another service that runs on specialized hardware.
But you need to refactor and budget time to that refactoring. Which I think is true in any architecture--it's just often more do-able to do in one pass with the multi module approach.
It helps very much to be in a language and build environment that has first class modularity support. I.e. good build tooling around them and good IDE support. And at the language level, good privacy constraints at the package level, so the deeper libraries aren't exposing too much by accident.
What modules patterns have I seen work over the years? Generally, having business logic and data access apis lower in the tree, and branching out for different things that need to be deployed differently, either because they will deploy to different platforms (say, you're running business logic in a web server vs a backend job), or because they are deployed on different schedules by different teams (services). A nice thing about the architecture is that when you have a bunch of services, your decision as to what code runs on what services becomes more natural and flexible, e.g. you might want to move computationally expensive code to another service that runs on specialized hardware.
But you need to refactor and budget time to that refactoring. Which I think is true in any architecture--it's just often more do-able to do in one pass with the multi module approach.