The days of imposing an Enterprise Architecture top down are gone. It doesn’t scale and it doesn’t create good architectures. Now it’s all about building the right culture and community, effecting architectural change through organisational change, and creating generative feedback mechanisms which cause sustainable architectures to grow. Instead of cost reduction we focus on maximising customer value and organisational resilience.

Squads

Agile cross-functional teams or ‘squads’ are designed to be fully empowered and self-organising delivery units with all the skills needed to deliver what the product owner wants. They typically include developers, testers, business analysts, UX experts and others. Forming your delivery units in this way drives a healthy architectural separation between products, allowing each to evolve freely.

If you are part of a central architecture team you might start to worry about an uncontrolled sprawl of new technologies and techniques emerging or that product focussed squads might not take resilience, performance, security or cost control seriously. What if the squads don’t co-ordinate and end up duplicating work? You might feel the need to fall back on means of exerting control such as Architecture Review Boards. But there might be a better way.

Building in Quality

For many of these concerns, the product owner has a direct interest in delivering quality, and just needs to understand the impact of design decisions. Squads and their product owners can take ownership of quality by talking openly about the trade-offs of alternative designs.

How sensitive are the customers to performance and availability? Experiment and find out, and prioritise performance optimisation and resilience when it’s justified. How many users do we expect to use the service? Instead of guessing use LEAN experiments and gradual rollouts to find out. Is it more important to be fast to market or meet stringent performance or scale requirements? Involve your product owner in the decision making and use real user monitoring and regular performance and capacity tests to validate assumptions.

And product owners are naturally motivated to keep costs in control. If they have clear visibility of the costs of implementation decisions and are liable for paying them from their budget then the squad can easily self regulate.

Healthy Diversity

Some diversity in technologies and techniques is healthy, and too little can reduce the resilience of your organisation. It can become hard to hire people with the right skills if you’re fixed on one technology which is in demand and it can be hard to retain staff if they have little prospect of learning new things. And embracing some degree of diversity can make technology selection more effective since it encourages a shift toward deciding through doing rather than deciding through guesswork. Instead of fixing on a single technology based on abstract research, try a few on real projects. When technologies don’t work out, reimplement with those that do. Since such a large proportion of software development is spent figuring out exactly what to build, reimplementing in a different technology is usually much quicker than the initial development effort. So the cost overhead of this approach is relatively small, usually much smaller than the cost of making a bad choice and sticking with it in the face of evidence it isn’t working.

But too much diversity can also reduce organisational resilience. It can be hard for people to move between squads, and hard to develop sufficient expertise except in a few people, meaning the impact when people leave is large.

So how to decide where the sweet spot is? The people best placed to decide on the healthy level of diversity for a particular type of technology are the people using it. So give the squads the tools and responsibility to make these decisions. The tools come in the form of feedback mechanisms. Encourage regular movement of people between squads so they can get direct experience of both the costs and benefits of diversity. Help product owners understand that the temporary slow-downs in delivery capacity when people move are a necessary part of maintaining healthy organisational resilience. Give squads full responsibility for hiring so that they understand the effect technology choices have. Above all, encourage active discussion between squads on whether the current level of diversity is too low or too high so that they can act to balance it. Use the Lean Coffee format to make sure these discussions stay relevant and aren’t dominated by the voices of HiPPOs (Highest Paid Person’s Opinions).

Duplicating Work

Some organisations go to great lengths to avoid situations where the same work may appear to be done twice, without considering the possible benefits. The most obvious and important benefit of allowing apparent duplication is that it lets squads or divisions move at their own speed and in their own direction. An over-emphasis on developing shared services can lead to bottlenecks in delivery, where squads spend significant time waiting for someone else to deliver a shared service they need to use. And building a service to be shared is much more difficult than delivering one for a single use case or consumer.

The needs of different consumers are rarely exactly the same and if they are forced to share a service then each may end up having to live with one which is a poor fit, whereas they could each have exactly what they need by building or choosing their own. And remember that the the cost of designing is a large part of the cost of building, so we want to avoid creating difficult design decisions by forcing services to cater for a multiple use cases.

Services fall into three categories:

1. Commodity requirements such as sending emails or SMSs, blob storage or DDOS protection. It’s clearly better to buy these off the shelf than build in house unless differentiation in this area is important for your organisation.
2. Services which are not specific to your business domain can be effectively shared between products and business divisions and are good candidates for open sourcing.
3. Services which are specific to your line of business such as calculating the risk of a loan tend to be hard to share effectively between products and divisions, since differentiation is likely to be important to your organisation. It’s better to avoid the temptation to share this kind of service.

The Price of Control

Some organisations have central architecture teams or design authorities. These come into being for the same reason as a centralised project management team or PMO function. They are there to control and coordinate the actions of specialist teams which work on their small part of a deliverable, without full ownership or visibility of the whole piece of work. But agile squads are designed to be fully empowered and self-organising delivery units, so they shouldn’t need to be controlled or coordinated. Every time an agile squad needs to stop to ask permission or wait for another team to do something they slow down and the predictability and lead time of their delivery increases, so we try hard to avoid this. Each squad ideally has its own codebase and can deploy independently of squads.

The self-contained nature of squads is essential to their effective functioning. But this doesn’t mean we don’t need to do architecture any more, we just need to do it differently. And we need to be aware that architectural and organisational structures have a strong influence on each other and can either work together or against each other.

Continuous Architecture

Like the move toward unifying development and operations, architecture needs to be a part of normal squad activity. Only the squad has enough context to make good architectural decisions about what they are building. And they need to continuously evolve the architecture as they learn, not need to stop to ask permission from outside architects.

As discussed above, we aim for an enterprise architecture which emphasises independence between divisions and squads to avoid burdening them with costly inter-dependencies. That is, we foster architectures and organisational structures which naturally avoid blocking dependencies. But squads can still benefit from each others’ experiences and learn from others’ mistakes, and we encourage this knowledge sharing through communities of practice, show and tell sessions and similar. The focus is on peer to peer community learning from which good practice naturally emerges.

Big Decisions

But what about the bigger decisions? Can squads be trusted to make decisions which can have far reaching consequences? The answer comes in two parts:

1. If a decision feels too big, ask a different question
2. The best way to know if a decision is good is to test it

Architects sometimes get hung up on trying to make optimal decisions, creating study phases which last months to make sure that a good decision is being made. But this is wasteful and ineffective as a way to make good decisions. And it ignores the cost of delay, focusing on perceived cost saving without appreciating the impact delaying can have on value delivered. A far better way to find good solutions is to encourage experimentation and diversity and grow good solutions. Instead of trying to design big systems top down, start building one or two pieces of it that have immediate business value as simply as possible and grow from there. Working this way, the architecture forms incrementally, its shape being found through practicing the craft like a stone or wood sculpture emerging from the material.

Summary

Build communities. Optimise for flow, not control. Decide by learning, and learn by doing. Good architectures grow from small and simple beginnings.