Connecting the Dots

Successful software products are hard to build and often fail, because of complexity. We can reduce risks, and increase our chances of success, by continuously evolving both the business model and the system architecture at the same time, using prototypes to drive learning and improvement.

Modern software delivery methodologies and UX design each hold half the picture of what it takes to achieve this. Neither, on its own, is enough to seamlessly sustain an evolutionary process under real conditions. Bringing both together requires keeping the system malleable to accommodate change, while the product evolves. Using self-similar patterns to hold technical (accidental) complexity at bay as long as possible is the key that unlocks the full potential.

That is the claim. It rests on four premises:

1. Software delivery, even with Agile, optimizes building solutions — and treats product development as if it were a solved problem.
2. UX design acknowledges that product development is complex — but typically explores and decides the product without the people who will have to build it.
3. Great system design needs the same try, observe, learn mechanisms UX uses — applied at the system level, with feedback from both perspectives integrated.
4. A process that brings both together has three requirements: focus on the right thing at the right time; optimize for feedback and learning; constantly apply good architecture and coding practices to accommodate change at any time.

The fourth premise is the one everybody underestimates. It is the one Domain Prototyping is built on.

Modern software delivery methodologies (Agile, DevOps, etc.) are single-mindedly focused on building and operating solutions, on time and on budget. Sprints, stories, velocity, acceptance criteria, retros, continuous delivery — the whole machinery presupposes that there is a thing to be built and that the job is to build it. They embrace change in detail, but not in essence. Consequently, developer teams may influence how it's being built, but not what, or why — and turn to stakeholders to answer those questions. The product owner becomes the single point of failure.¹
Inside their presuppositions, delivery methodologies are excellent. Outside them, they are a way to build the wrong thing reliably, and on cadence — which is ironic, because ‘build the right thing’ is one of the core promises Agile methods make.

Product development doesn't live inside those presuppositions. The Cynefin framework enables us to make the distinction that every team feels but rarely name: We often talk about best practices and act like software development belongs to the Clear domain - a solved problem. The most software development tools (user stories, architecture reviews, patterns, playbooks) belong in the Complicated domain, where cause and effect are knowable to experts. But product development really belongs in the Complex domain, where cause and effect are only clear in retrospect. To paraphrase Dave Snowden, the originator of Cynefin: we can't analyze our way to the right product. We have to probe, sense how customers react, and respond.

Agile proponents often state that — in principle — we could include product development with prototypes, experiments, and pivots in its structure. But this rarely happens in practice — not least because that structure is overhead, when we really just want to probe, sense, and respond as quickly as possible.

Agile, as commonly practiced, is mostly silent on this. It will faithfully deliver whichever product hypothesis we put into the backlog. If that hypothesis is wrong, sprint ceremonies will not tell us. Velocity will not tell us. Story points will not tell us. Sometimes it surfaces in sprint demos — and then it becomes a change-request argument with a committee. This is understandable because changing a solution already built for full scale is expensive — increasingly so, the later the change. When the cost of change exceeds the budget, cancellation of the project looms. We may be tempted to compromise and find a workaround. Or shelve the change until after the first release. In most cases, the actual design of the system will drift from the ideal state that fits customer needs.

And even more likely, we won't discover the drift until launch, or not at all.

The gap is not a criticism of Agile or DevOps. It is a statement about what they are for. Agile was designed to replace waterfall delivery, not to support product discovery. DevOps was designed to bring operations and delivery closer together. The people who built these tools were deeply involved in a broken process, and they made it so much better. The toolbox works as advertised — the problem is that the advertisement is narrower than most organizations would need it to be.

This is why teams that are otherwise doing everything right still ship products that fail for non-technical reasons. In fact, most software project failures are not technical at all. ² They are failures of matching the built thing to the real problem — a task that sits upstream of the delivery pipeline and is not part of what the pipeline was designed to solve.

UX design, done well, does everything product strategy is supposed to do. It accepts that the user's world is complex. It uses discovery, prototyping, and observation to learn what should exist. It iterates. It is, in Cynefin terms, a probe–sense–respond practice that happens to wear a product hat.

Unfortunately, in most organizations, That's also where it ends. The UX cycle produces a validated product concept and hands it over the fence to a delivery team that did not attend the research, did not see the prototypes, did not hear the user interviews, and often wasn't even staffed when the research happened.

The team that now has to build the solution inherits two problems at once:

The first is that the product concept was validated as a user experience, not as a system. It is not yet known whether the thing that makes users happy can be built in a way that is operable, secure, performant, integrable with the rest of the estate, and affordable to run. Usually some of those constraints collide with some of the design decisions, but we will not find out which ones until later in the process. When inconsistencies are discovered, revisiting the design decisions brings about a committee process, similar to the one discussed in section 1.

The second problem is the clock. The release date has been set based on when the concept was validated, not when the system can be understood. The team is expected to rebuild a throwaway prototype into production architecture while the product is already being sold.

This is the inverse failure mode of Agile.³ Agile is good at the how and silent on the what. UX-led development is good at the what and silent on the how. In both cases, the silent half often gets solved in panic mode later, and the Sunk Cost Fallacy drives the discussion.

The honest assessment is that we have two mature disciplines that each do half the job well, and an industry standard of running them separately.⁴That separation, the sequential arrangement, is what actually creates the crunch — it isn't inevitable.

Integrating UX design and systems design into a single feedback loop requires us to apply the same strategy to both disciplines (with different vantage points), and combine the collected feedback into shared understanding.

Cynefin gives us the general approach for complex domains: probe, sense, respond. That is the abstract pattern. It doesn't prescribe what the probes are, how the sensing happens, or what the response looks like. Software teams, when they implement probe–sense–respond, end up doing something more specific:

• Try. Put something real in front of reality. A prototype, a release, a running system — anything that produces an observable response from the world.
• Observe. Watch what actually happens. Not what was supposed to happen; what did.
• Learn. Update the next attempt based on the gap between the two. And repeat.

This is what UX research already does with users. The object is the product; we observe user behavior; the learning feeds back into the product design. Short cycles, real contact, small bets.

This is also what system design must do with a running system. The object is the current solution; we observe system behavior — under load, under failure, under integration with the estate, under evolution of the product it supports; the learning feeds back into the architecture. Same cycle. Same cadence. Different vantage points.
The mistake is to treat probe–sense–respond as a UX-only practice and then expect architecture to be produced by a different method — up-front analysis, best-practice selection, a governance committee. That is treating a complex problem with complicated-domain tools, and it fails in the way complex problems always fail when handled that way: through accumulating surprise.⁵

Architecture that can be evolved alongside the product needs the same discipline of put it in front of reality, watch, adjust. The loop is the same; only the vantage point is different. Calling it by one name (try, observe, learn) with two vantage points (user, system) is clearer than calling it by two different names and acting surprised when teams treat the two halves as unrelated.

Domain-Driven Design (DDD) — the practice of modeling software around the business domain it serves — implements try, observe, learn. The development team and subject-matter experts ‘crunch knowledge together’ (Eric Evans) to develop a shared understanding of the domain, build an executable model that represents that knowledge, and then use that model to reflect and shape the next iteration. It does for architecture what UX design does for the value proposition.

Additionally, it looks at the product itself from a different angle: UX focuses on the user, DDD tends to focus on the language used, and the business domain as a whole. The two disciplines are complementary, but they are not the same thing. Without DDD, UX research tends to improve how people already work, rather than imagine how they could work in a better system. Using DDD to find boundaries and structure for the architecture may well surface suboptimal choices in the product design.

Tightly integrating UX design and DDD significantly improves the chances of creating a better product. Together, they see the product and the system it embodies as one thing, not two. They share the same feedback loop, use the same prototypes, integrate their feedback, and grow a coherent product, with a cross-functional team.

If product and architecture are going to evolve together, three things have to be true at once:

a) Focus on the right thing at the right time. Early in a product's life, the expensive mistake is building out infrastructure for a proposition that hasn't been validated. Later, once the proposition is validated and demand is real, the expensive mistake is refusing to industrialize. Kent Beck's 3X Model (Explore, Expand, Extract) and Eric Ries' Lean Startup (Problem/Solution Fit, Product/Market Fit, Scaling) both name this: different phases demand different thinking. A team that scales prematurely wastes capital on a product nobody wants yet. A team that refuses to scale after product-market fit wastes the opportunity. Focus means knowing which phase you are in and spending effort accordingly. ⁶

b) Optimize for feedback and learning. Short cycles, tight feedback loops. Real users in contact with real things; real systems in contact with real usage, real integrations, real evolution of the surrounding estate. The feedback is worthless if it arrives after decisions have ossified, which is why the cycle length matters as much as the cycle's existence.

c) Architecture and coding practices that accommodate change — at any time. This is the one most teams underestimate, because it has no project phase of its own. It is a property the code has to always have. If architecture is not continuously malleable, the first two requirements collide with it: the team learns what the product should become, and then finds that the system can't become that without a rewrite.

(a) and (b) are broadly understood in the industry, even when not practiced. (c) is the open question. It is what this page has to answer before it can claim the gap is closable.

I was consulting at a bank, years before any of this was methodology. The enterprise architects showed me the diagram for their first serious microservices project — a BPM tool in a third-party cloud, connecting via Kafka into the internal data center, several downstream services. They had been working on it for months and were proud of it.

I looked at the diagram for a few minutes and immediately saw it was going to cause serious problems on go-live: circular dependency between the BPM tool and Kafka. Internal structure exposed. Interfaces missing where they mattered most. Service discovery, failure correction, and security would all hurt.

The team was not pleased. “How can you tell that so quickly? You've only been here a few hours!”

At the time I couldn't answer. It was a gut reaction — years of seeing the same shapes fail at the application level in the same ways. The words literally came out of my mouth before I had given it a close look. I knew I was right, but didn't know why. Then, going through the diagram slowly and item by item, I was able to explain each finding: arrows pointing back and forth showed the circular dependency. Missing interfaces at the boundaries were simply missing boxes (arrows going through environment boxes, no ‘doors’). The reasoning was sound.

But the question stayed with me because it implied something I had not thought through: If I could recognize a bad system architecture instantly, just from its shape, the way I could recognize badly structured application architecture from its shape, then whatever I recognized must have been patterned in roughly the same way at both scales.

The anti-patterns I named at the bank were the same ones I would have named in an object-oriented class diagram. Circular dependency between objects is a smell in code for the same reason it is a smell between services. Exposing internal structure is bad for objects because it couples callers to implementation and is bad for services for the same reason one layer up. Missing interfaces are a problem wherever two pieces of software need to talk across a boundary, because a change on one side requires a simultaneous change on the other.

That observation became the seed question: if design patterns and anti-patterns are self-similar across abstraction levels — if the same shapes mean the same things whether you're looking at a class diagram or a service topology — then the discipline we've built up for good object-oriented design might scale. Not as literal reuse, but as vocabulary. And if the vocabulary scales, the practice of evolving code by refactoring toward good patterns should scale with it — into a practice of evolving architecture by refactoring toward good architectural patterns.

That was the piece that made the other pieces fit. Try-observe-learn on the product was already a solved problem. Try-observe-learn on the architecture needed something that kept the architecture malleable between iterations. Self-similar patterns looked like that something.

I have applied this idea on every development project I've worked on since 2018 (with a few exceptions, where customers were already set on a different path). I've used it with teams of different sizes and across different industries. It can be used in greenfield and legacy projects. I have just recently been building a product of my own with Claude Code for six months, and it still holds up. I have yet to find a reason not to use it, other than ‘we don't want to’.

Self-similar patterns — the piece that lets architecture stay malleable

Building a product is like shaping a statue out of clay. Every round of user contact reshapes it — first, very broadly, then the details. The question that decides whether the work is possible at all is whether the clay stays malleable — whether, on the hundredth pass, we can still reshape the hand without the arm falling off.

Most architecture does not stay this malleable. It solidifies. Early prototypes become permanent; early abstractions outlive the assumptions they rested on; the system becomes something that can only be changed under duress. Once that has happened, try-observe-learn at the architecture level stops working, because the cost of respond has grown larger than the value of sense. The loop breaks.

Self-similar patterns are how the architecture stays malleable. The idea, stated simply:

1. We start with a small-scale domain prototype. No infrastructure dependencies. Just the business logic, expressed as code, with whatever the simplest possible representation of the surrounding world is. An in-memory list instead of a database. A function call instead of a service boundary. A local queue instead of a broker.
2. We refactor it with the rigor we'd apply to production code. Good patterns at the code level — clean interfaces, separated concerns, honest abstractions — are the same patterns we want at the architecture level. If we keep the code clear and well-tested, the system will already have the right shape — on a smaller canvas.
3. We extrapolate to larger-scale patterns only when the load demands it. When the prototype needs to accommodate multiple users, it's time to separate client and server. When user sessions need persistence, we go from in-memory to local storage. When users need undo/redo, we may want to introduce event sourcing. When the in-memory queue starts to need durability, we replace it with an external message queue. Whatever the driver — the code at a small scale already knows how to accommodate the mechanism at a larger scale; the shape is the same. When a function call needs to cross a process boundary, it becomes a service call, and the call site doesn't care that the implementation moved.

The simplest mapping to make the claim concrete: a plain queue in code and a message queue as an infrastructure component are the same pattern at two scales. A producer adds; a consumer takes; back-pressure and ordering are the same concerns on either side. Replacing one with the other is not a redesign. It is an upgrade of a single dimension (durability, cross-process delivery) of an already-correct shape. The same holds for other patterns, though the queue case is the simplest demonstration.

The payoff is that requirement (c) becomes satisfiable. Architecture can be evolved the same way the product is evolved — we try something small, see it behave, refactor toward a better pattern, extrapolate when the load demands it (and by then we know it's the right structure). The clay stays malleable. The statue can be reshaped on another pass.

That integrated practice is what the rest of this site calls Domain Prototyping. It has been applied in production since 2018, and so far it still holds up.

To continue from here:

• → Learn about the method
• → Catalogue of self-similar patterns