Is Agile Dead? - 1. What Is Agility?

Is Agile Dead?

Towards the end of 2024, the boss sent an email to all the consultants on the team asking “Is Agile Dead?”

Ivar

Team,

To quote Scott Ambler: “Agile isn’t dead, but the agile gold rush is, and it isn’t coming back.” Scott’s article is well worth reading, if you haven’t already done so: https://www.linkedin.com/pulse/agile-community-shat-bed-scott-ambler-1npwc/

Another article by Jürgen Apello: “Let’s celebrate twenty-five years of Agile and ten years of “Agile Is Dead” articles. Now, I won’t be one of those heroes trying to keep the brand alive. It’s time to move on.” https://www.linkedin.com/pulse/agile-undead-synthesis-jurgen-appelo-54wne/

Please, let us know your thoughts. Is it time for us to move on, and in that case, what should we do?

“Agile is Dead” headlines are largely LinkedIn content writers seeking clickbait for their own articles, but people are after all reading them, so there is definitely disquiet with the branded one-size-fits-all promises that did not really solve the client delivery issues. There are organisations that are laying off ScrumMasters and Agile Coaches, Is Agile Dead?

Of course, Agile, or some form of it, is not really dead, it is still very widely used but the gold-rush for “paint by numbers” transformations with inexperienced staff is over. I would go further and say that customers have “soured” to single approach selling as solutions to their problems.

Rather than add more click-bait to the situation, what I felt it needed was some sensible, reasoned thinking that gets beyond the observable symptoms and starts to ask “why?”

This series of posts covers:

• What Is Agility?
• What Has Changed?
• We Just Don’t See The Value
• Won’t AI Save Us?
• What Is The Problem?

Let’s start with something pretty fundamental…

What is Agility?

Figure 1: Agility: Johnson’s Dictionary, 1755 from Johnsons Dictionary Online

Let us look at the dictionary definition, the first dictionary definition, which predates automated computing by around 65 years¹ and the formalisation of the Manifesto for Agile Software Development by almost 250years.

Find your nearest executive, ask them which of these two options they would prefer?

Nimbleness Readiness to move Quickness Activity

Sluggish Unwilling to work Slow Idle

My suspicion would be the attributes on the left, Agility, rather than the right which is Johnson’s definition for Lazy.

Attributes that have existed for thousands of years and will continue to exist for the foreseeable future.

Attributes that are desirable to have in teams within organisations, now and for the foreseeable future.

Agility, as a concept, will continue. Instantiations of a concept can fall from favour, but the concept itself cannot die.

That’s great conceptually, but we need to explore why Agility is necessary…

The Necessity of Reactivity

My thoughts on this have been heavily influenced by Maarten Dalmijn’s buzz-word free explanation of Why Agile is needed. Maarten draws on Steven Bungay’s Art of Action, which itself is drawing upon General Carl von Clausewitz’s On War, it also references Complexity Domains, as described by David Snowdon in his Cynefin Framework. Other than trying to pronounce the Welsh name Cynefin, everything else is exceptionally approachable, even to those that don’t have a background in understanding the silly words used in the agile community. We don’t need Maarten’s full exploration of how the values in the Agile Manifesto tie back into the complexity domains, we just need to understand complexity itself, which is a lot less complex than it sounds…

Cynefin Complexity Domains

Figure 2: Cynefin Complexity Domains

The Cynefin framework is a conceptual framework created by Dave Snowden used to aid decision-making, the framework has been described as a “sense-making device”.

He outlines four different complexity domains:

	Clear: The cause and effect of an action is obvious, do the action and you get the result. Simpe instructions can be created for the action so that anyone can follow them and get the result. The situation is deterministic, action gives result.
Complicated: The cause and effect are not immediately obvious, but someone with experience would be able to assess the situation and understand what is happening and be able to determine an appropriate action to get the desired result. The expert, using their experience, is moving the situation clockwise around the domains towards the Clear complexity domain. The situation is complicated but deterministic, once an action has been determined, the action gives the result.
	Complex: The cause and effect are not obvious and even someone with experience is unsure what is happening without investigating. They need to perform targeted experiments to understand what is going on. Understanding the situation is moving the situation clockwise around the domains towards the Complicated complexity domain. The situation is non-deterministic, the results of the experiments and investigations cannot be determined in advance. Experimentation is required to determine what actions will produce the desired results.
Chaotic: There is no linkage between cause and effect, things just happen. There is no obvious way to target experiments, experiments just have to be attempted in order to see if anything works, if any insight can be gained from the results. Gaining enough insight about the situation to allow the experiments to become more targetted is moving the situation clockwise around the domains towards the Complex complexity domain. The situation is non-deterministic, the results of the experiments and investigations can’t be determined in advance. Try experiments, gather data, look for patterns, try again.

Example Situation

Imagine an automated production line, running smoothly, producing bread. The instructions are to put the ingredients of flour, yeast, water and salt in at the beginning, and take the product out at the end. The Clear complexity domain.

All hell breaks loose, the line has stopped, nobody knows what’s going on, it is chaos! How to resolve the Chaotic situation? Try things!

We know nothing at this point in time. All we can do is try things and gather information: inspect the machines. Information starts coming back, the dough mixers are working, the packaging machines report they are ready to go, it’s the oven that has gone wrong. Through the information received the situation starts to move from the Chaotic complexity domain to the Complex complexity domain. Why has the oven gone wrong?

The experimentation can start targeting the oven. The domains can be nested: from the perspective of the Production Line as a whole the situation is Complex now that we are targeting the oven, from the perspective of just the Oven the situation is Chaotic as we don’t know what’s gone wrong with the oven. For the oven we are just going to have to try different experiments until we gather information to move the situation towards the Complex complexity domain.

Investigations reveal that a bearing failed on the continuous conveyor that passes through the oven. The situation is moving towards the Complicated complexity domain and an expert on bearings can explain why the bearing has failed and issue instructions on how to fix the bearing, and how to prevent similar failures in the future. The instructions issued move the situation back to the Clear complexity domain, actions to perform that have obvious cause and effect, and once the bearing has been replaced production can resume.

The desire is to move clockwise through the domains towards the Clear complexity domain, although events could conspire against you and move you anticlockwise through the domains towards the Chaotic complexity domain. There is more to the diagram, the lines have meaning and purpose, but the presented simplification to just the domains is enough for the purposes of this conversation.

Deterministic Best Practice is not Non-Deterministic Best Practice

Figure 3: What to trust for each of the complexity domains

In The Plan We Trust

The lefthand complexity domains are deterministic, predictable, you can write instructions to handle the situation. As the situations becomes less clear and more complicated than you start to need experts to write the instructions. Better instructions, with less ambiguity, will make the situation easier because there is less chance of making a mistake when following the instructions. The goals are implicit, they are contained within the plan, the instructions. Deviation from the plan is likely to be costly, and the goals unlikely to be achieved.

In The Results We Trust

As you move from the Complicated complexity domain to the Complex complexity domain, the situation becomes non-deterministic, unpredictable. Writing more and more instructions, which was best practice in the deterministic space, won’t help here because we can’t determine the future up-front. A cyclic process of experimentation and then reacting becomes necessary. It is vitally important to have a clear goal, but the route to that goal cannot be determined in advance and has to be discovered…

The Impossibility of a Fixed Plan

Figure 4: Linear sequence of n actions

Imagine a simple action, perform it and you get a result. That result becomes the input for another simple action, and you get another result. Keep going and a simple linear chain builds up. Easy to understand, predictable, one thing leads easily to another. This is construction, do A then B, then C, one brick upon another. A plan of n steps leads to just one outcome.

Figure 5: Decision tree, n decisions results in 2ⁿ possible outcomes

Now, instead of a simple action we have a decision, there are two possible answers to the decision. This creates a branching point, we can plot that out, start creating a decision tree. What if those two answers are themselves just more decisions to be made? Now there are four possible outcomes. Keep going: 8, 16, 32 possible outcomes. The decision tree is starting to get difficult to create, the cost of creating a completely predictable plan is difficult to justify. A plan of n steps requires the creation of 2ⁿ⁺¹-1 decisions points on the plan. A Boeing 747 has 6 million parts, if every part required a yes/no decision then the calculation of how big the resulting decision tree would be breaks most calculators².

There is only one correct route through the tree, for every branch taken the other half is discarded, and the effort to elaborate the other half becomes waste. The bigger the tree, the more waste. Unless there is a compelling need to complete the whole tree, rather than trying to predict everything upfront by building the whole tree, let us make the first decision then use that to inform what the next decision should be. This is where iterative planning stems from, using one decision to inform the next. However it requires a leadership mindset that is comfortable with the uncertainty and capable of repeatedly evaluating whether progress is being made towards a clear goal.

Use The Right Tool for The Job

There is a spectrum of approaches to getting things done that runs from “Absolutely Deterministic” at one end to “Sense and Adapt” at the other end.

Figure 6: Process Spectrum

Production, repeatedly performing the same sequence of actions to produce a product is deterministic, therefore it lies towards that end of the spectrum. Engineering is problem solving, decision making, one decision leading to the next, lies towards the Sense and Adapt end of the spectrum.

Dave Snowdon always represents the complexity domains in a circular cycle because the Clear complexity domain can descend into the Chaotic complexity domain very easily through unexpected occurrences, as illustrated in the earlier example about production machinery failing. However, for the purpose of illustration the cycle has been unwound and laid out linearly in the above to diagram to represent the complexity domain’s relative positions on the spectrum.

In an organisation of any significant size, different areas of the organisation will require different ways of working and different ways of being managed. Managing a production line is not the same as managing the engineering teams that create the product the production line produces; they’re on different areas of the spectrum because the type of work that they do is different.

Even a single team can be on different areas of the spectrum because the work they do is varied, some work will be repetitive actions, other work will be problem solving. Production staff will be more towards the repetitive action end of the spectrum but might need to problem solve if problems occur. Engineers and creatives will be more towards the problem solving end of the spectrum, but there will sometimes be procedural work where they just have to follow the instructions.

The challenge for the central roles, the management and executive, is that the set of teams they are likely to be working with, will be using methods from across the whole spectrum. No single way-of-working is going to work for every team; therefore, the management have to understand all of the ways-of-working, at least to the extent where their actions aren’t conflicting with how the teams need to work.

No single way-of-working is going to work for every team; therefore, the management have to understand all of the ways-of-working, at least to the extent where their actions aren’t conflicting with how the teams need to work.

Next

Agility has not changed, it is what it always was: nimbleness, readiness to move, quickness; so what has changed?

^#1 Johnson’s Dictionary of the English Language, published 1755, https://en.wikipedia.org/wiki/A_Dictionary_of_the_English_Language to Babbage’s Difference Engine, designed circa 1820, https://en.wikipedia.org/wiki/Difference_engine. [Return]
^#2 I found one that suggested 10¹⁹⁸, but then couldn’t find it again to verify whether I’d remembered the number correctly. There is probably some maths I can do to change the base and exponents but I haven’t used those mathematical skills for a very, very, long time. Ultimately, the precise number is irrelevant, it is so big it is impossible to handle. [Return]

Without wanting to fan the flames of more click-bait, I hope the above has provided more insight into: Is Agile Dead?