These are questions for wise men with skinny arms

Simplicity - the art of maximizing the amount of work not done, is essential. Most teams simply don’t spend enough time on this. A sense of urgency often overrides careful planning. The problem here is careful planning makes things get done faster. During the planning stage it feels like you’re not getting anywhere, but you are setting up for a quick sprint. This setup is often overlooked, and we end up with not only complicated software, but complicated development habits, complicated code, and generally poor software design. This slows down maintenance and new development, as we try to fit into poorly designed structures that become ingrained and impossible to improve. “This new feature is poorly designed because the application it integrates with is poorly designed” can go on forever.

Jasmine Adamson, Super friendly aviator

The last sentence is the scariest part of building big software. Getting something wrong from the start can scar all applications from then on. It’s very difficult to go back and re-work anything fundamental in a big system with the goal to make it simpler (removing some bad, but heavily used features in the process). The business case is tough when you already have something that “works”, so the value of future productivity gains is generally discarded in favor of just adding another abstraction layer on top to hide the ugly parts.

Good documentation and custom tools makes working with big balls of mud less painful, but no amount of paint will make a complex system look like a simple one. A through rewrite of the ‘most scary’ parts of a big code base just makes the less scary parts look worse!

When I see good software design, it appears to follow from a deep understanding of the problem domain such that the user’s understanding the programmer’s understanding of behavior aren’t at odds. Software design comes down to answering “What’s the best way to map the proposed solution into code”. Design simplicity is tough because there are now a plethora of ways to solve most any problem. The large number of tools available to a programmer makes it is very easy to pick a mapping that is familiar to the programmer, but that might not express the complexities of the domain accurately. The simplest domains are sometimes the worst because they can most easily fall to two extremes.

Over-engineering: A team of programmers break down the problem into it’s general counterparts and then create a much bigger and extensible solution to attempt to address the sum of the problems. This turns into either building a big framework with lots of unused functionality or it becomes gluing together lots of other big libraries where the glue is more complex than any feature the libraries provide.
Spaghetti coding: A program that isn’t yet interesting or important enough to warrant upfront design; probably initially built as a quick hack/script by a junior or non-technical programmer that grows out of control as it becomes useful. Features are added wherever they can fit, where the most important ones are the ugliest and the prettiest ones are unused.

These are both extremes because the addressed problem changed. Software monstrosities are blamed on over-specification or scope-creep respectively, but if agile preaches to embrace changing requirements, then how can these be avoided? (The source quote is from a rant on the hypocrisy of agile practitioners).

This is why everyone is singing praises for micro-services. Small programs are better because it’s easier to write the shortest solution to problem instead of everything you think you know about it. The simplicity comes from the direct mapping of problem space to solution space. Bottom up design gives the best insight into the root of the problem, such that it closely matches it’s implementation. Per problem, start with just the minimum solution and then plan to break it up instead of growing the original solution. This partially avoids the difficult practice of future-proofing and planning for extensibility when working out the implementation details. If the problem is indeed big, the original solution will have been broken up into multiple programs as each new problem is found and solved. And if the problems turns out to be very specific, then the solution will represent that.

Bottom up design suffers from a heavy dependance on the original implementation tool-set, so having a wide selection of tools and picking the most appropriate ones becomes the key design problem. If the programmer doesn’t have the appropriate tools, then even the first solution won’t map to the problem and it will be increasing difficult to logically break up the work as the program grows. The proliferation of languages (and frameworks where the language is fixed) is the expansion of tools to better model different domains, which is why it feeds so well into the micro-service movement.

The key point is that simplicity is a very tough and very important problem in software engineering, and requires just the right amount of planning, design, and implementation tools to get correct.