Is Agile Dead?
Towards the end of 2024, the boss sent an email to all the consultants on the team asking “Is Agile Dead?”
| Ivar |
| Team, |
|
To quote Scott Ambler: “Agile isn’t dead, but the agile gold rush is, and it isn’t coming back.” Scott’s article is well worth reading, if you haven’t already done so: https://www.linkedin.com/pulse/agile-community-shat-bed-scott-ambler-1npwc/ |
|
Another article by Jürgen Apello: “Let’s celebrate twenty-five years of Agile and ten years of “Agile Is Dead” articles. Now, I won’t be one of those heroes trying to keep the brand alive. It’s time to move on.” https://www.linkedin.com/pulse/agile-undead-synthesis-jurgen-appelo-54wne/ |
| Please, let us know your thoughts. Is it time for us to move on, and in that case, what should we do? |
“Agile is Dead” headlines are largely LinkedIn content writers seeking clickbait for their own articles, but people are after all reading them, so there is definitely disquiet with the branded one-size-fits-all promises that did not really solve the client delivery issues. There are organisations that are laying off ScrumMasters and Agile Coaches, Is Agile Dead?
Of course, Agile, or some form of it, is not really dead, it is still very widely used but the gold-rush for “paint by numbers” transformations with inexperienced staff is over. I would go further and say that customers have “soured” to single approach selling as solutions to their problems.
Rather than add more click-bait to the situation, what I felt it needed was some sensible, reasoned thinking that gets beyond the observable symptoms and starts to ask “why?”
This series of posts covers:
- What Is Agility?
- What Has Changed?
- We Just Don’t See The Value
- Won’t AI Save Us?
- What Is The Problem?
Previously we explored Won’t AI Save Us? Dealing with AI is going to require industry to be more agile than ever because, once the AI is doing the mundane procedural information processing work, what is left is creativity and problem solving. In this post we loop right back to the beginning and ask…
What Is The Problem?
The problem is that the Late Majority Customers want a commoditised solution, something that just works, straight out of the box. The Agile industry has provided neatly packaged solutions that look like they are commoditised, but to make them work properly they need customisation, and that customisation needs the experience and innovation skills that Late Majority Consultants do not necessarily have1.
Let us return David Snowdon’s Complexity Domains from the Cynefin Framework:
Figure 1: Agile Adoption’s place within the Complexity Domains
If Agile adoption was in the Clear complexity domain, then organisations could do it themselves.
Typically, they cannot. Therefore, following the desire for the problem to be in the lowest domain possible, organisations want the solution to be in the Complicated complexity domain. In the Complicated complexity domain if an expert explains it to them, then everything will become clear. Following the instructions on a framework is supposed to be a substitute for the expert. The Agile industry making the frameworks look like a commoditised product gives that impression: just follow the instructions and you will be Agile.
The reality is that adopting any change in organisational behaviour, of which Agility is but one example, is a Complex problem and experience alone, as embodied in the Framework instructions, is not enough. It needs experimentation to determine what works best for this organisation. If organisations are following the instructions correctly, and implementing the sense and respond mechanisms properly, then they should be able to evolve the processes to cope with their complexity, the advantage of having Consultants with history and experience is that they can guide that experimentation and help organisations reach a conclusion quicker. If the consultants are blindly following instructions, then you might as well have just done it yourself.
Blame The Framework!
Except, the frameworks are not wrong.
Every Agile framework that I have studied has “Agile Principle #12: At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behaviour accordingly”, embedded within it somewhere. They all use slightly different words but the intent is there: every framework expects to be customised to suit the needs of this instance. So why isn’t that happening?
It is that desire to always be in a lower complexity domain, for things to be simpler. Customers assume things are simpler than they are, consultants assure customers that they are simpler, because they want the contract. Without realising it, the conditions have been set up for a dangerous death spiral.
What customers want, and reality, do not align. Put yourself in the shoes of the customer: you have one consultant that sucks air in through their teeth and goes “oooh, this is going to be difficult, we’ll only know once we get under the hood and start poking around.” And another consultant that goes “no problem, we’ll follow the process, and everything will be fine.” Which would you pick?
Organisations need help to become more effective but the “Agile” snake oil sales people have killed the easy route in their race to the bottom to undercut the competition. The race to get things done cheaper means quality suffers, which in turn is giving the industry as a whole a bad name. It is an almost impossible scenario: they do not want to be sold the harsh reality, therefore the lie that “it’s easy” will always win. There is no easy fix, but also this is no different to hundreds of other industries, there will be good contractors and bad contractors, buyer beware. The fix is to get customers to exhibit critical thinking, to really question what they want before they commit to buying it.
There have been a number of occasions over the years where the sales pitch presented by IJI has caused organisations to rethink their whole approach, because we’ve highlighted things that they don’t have the experience to have even considered. It has been annoying to the lose the opportunity, and it’s associated income, when the organisations go off and reconsider their approach to the transformation; but it is good to know that we’ve done the right thing, by getting them to reconsider what they’re asking for.
Change Management Blind Spot
Over the years I have worked with product companies that produce physical objects, and even some that produce the machinery for producing physical objects. They know that setting up a new production line is going to be hard. To minimise the risk, wherever possible they tend to use existing machinery, possibly with some customisation, and they know that they will have to create all the pipes, conveyors, etc… that link everything together. Usually, a team is assembled comprising of internal expertise about what the product being produced by the production line is, external expertise about what the machines going into the production line are capable of, and expertise around putting the whole production line system together.
Translate the above to the Engineering Development space, machines become processes, the glue is human. But there seems to be this particular blind-spot about the complexity of it all, perhaps because unlike its physical counterpart the development teams are infinitely malleable, the people can change what their doing and how they are doing it, at any time2. Except that changing the way that people behave is harder than changing machinery.
Whilst discussing this with my colleague, and co-trainer, Keith he observed that behavioural change in people is the Complex complexity domain, there is no clear link between cause and effect, people are non-deterministic. Compare that with the mechanical situation which lies in the Complicated or Clear complexity domains, the cause and effect is more easily observable, the machinery is deterministic. Organisations don’t want it to be in the complex domain, so they pretend it’s not! They try to manage the behavioural change through processes more suited to the Complicated and Clear complexity domains, and as we’ve seen before the processes suitable for the Complicated and Clear complexity domains don’t work in the Complex and Chaotic complexity domains. Instantiating Agility requires Agility because it requires an experimental mindset, a sense and adapt approach.
Agility is a human trait.
Agility is trying to help humans deal with complex situations.
Adopting Agility is a human problem.
Many Agile coaches come from the engineering space, when a background in psychology or sociology would be a better fit for dealing with the human issues and the change management.
Change Management is nothing new, it’s been extensively written about over the last few decades, and there are people who specialise in Change Management. The frameworks even incorporate Change Management into them, but if it’s mistakenly interpreted as a mechanistic process then problems will ensue. Change Management itself lies in the Complex complexity domain and benefits from using the Agile approach of sense and respond because the humans that you’re trying to persuade to change do not behave in a deterministic manner.
What Is the Problem We are Trying to Solve?
Agility itself, irrespective of the methodology, is a solution; therefore, we should be asking what is the problem that we are trying to solve?
A good example of the problem with “Agile” is present in the various frameworks’ implementation advice. The pattern is, teach framework, launch Practices, scale. I know that many will argue – rightly – that the implementation plan is not the start of the journey, but it does look that way to the inexperienced.
Agility is being adopted for a business reason: Better, Faster, Cheaper, Happier. Measure that underlying reason, track the true purpose. The frameworks often talk about measuring “Agility” because that is what they know how to measure, they can describe those measurements because that is the space the frameworks operate in. They cannot describe what your business should measure because they don’t know about your business, only you know about your business. Whilst the frameworks do suggest you should measure your real needs; because they can describe the “Agile” metrics they focus on those, rather than the business metrics which aren’t described in any detail. By example one training course that I looked at had 3 slides on the business outcomes, 6 slides on measuring the process, 6 slides on measuring skills and behaviours. It unfair to blame the training course because they can’t describe an organisations business outcomes, that is context specific, but to the inexperienced that outcomes has just 1/5th of the training material can de-emphasise them in favour of the other more prominent, but ultimately secondary, measures.
Industrialised Bad Practice
A good workman has the right tools for the job
A bad workman blames their tools
This does not mean that there is no value in frameworks – there clearly is, as has been described earlier in Why Do We Have Frameworks? It is not what you do, it is how you get there. To loop right back to the very beginning, Scott Ambler was right when he suggests that the Agile industry has done this to itself, but never ascribe to malice that which is more easily explained by ignorance…
The innovators at the start of the Agile Software revolution couldn’t transform the world on their own, the world is too big, they were too few. They needed to teach the next generation of people, who in turn taught the next set of people; another bifurcation tree. The problem is that if one of those people being taught picks up the wrong behaviour, they do not get the intent quite right, then that mistake can be communicated through the tree to a large number of people. If the misunderstanding occurs towards the top of the tree then huge swathes of the tree below that point have the misunderstanding communicated to them.
Figure 2: Failures Propagate Through The Tree
I am a SAFe trainer, I’ve probably taught more SAFe classes than anyone else on the planet including Dean Leffingwell, the creator of the Sacled Agile Framework. Whilst I teach SAFe, I know more than just SAFe; I understand Scrum@Scale, Nexus, LeSS, we even discuss these in the SAFe training course, not as the enemy but how each of the methodologies can support the others, remember they have more in common than they have differences. I try to impress my innovator mindset upon the attendees, that the framework, any framework, is just a starting point for customisation, adaptation and tailoring. Convincing people, empowering people, we try to highlight and show how the processes support that: Individuals and Interactions over Process and Tools. Does it all stick? I would like to think that most of it does, but not all of it will and it’s those mistakes being communicated down the tree, to the next generation, that can cause the problems.
Why Not What
If you only know “what to do”
without knowing “why you’re doing it”
you lack the context to adapt
My colleague Richard is currently sub-contracted through another organisation to a branch of the UK government. He regularly sees Agile Coaches brought in who start with “This is the Agile Software Manifesto, here is a process for developing Software in an Agile fashion” when they’re talking to a room of somewhat bemused librarians, historians and other non-software development roles. In fact, the number of software engineers within the this branch of the UK government can probably be counted on a single hand, out of a total workforce of around 40,000. They’ve jumped to the what without understanding the why.
Richard’s approach, honed over several years working with the government, is to ask “What are the problems your facing? What are your challenges?” Once he understands the problems and can empathise with the people he’s working with, then he can start to introduce agility, in it’s rawest form: Nimbleness, Readiness to move, Quickness, Activity and from there maybe it builds towards something that resembles one of the documented methodologies. He’s written about his experiences of Method Agnostic Agility in non-Development Environments and Nurturing an Agile Mindset Beyond Just Software Teams.
To some that might feel like going the long way around, as opposed to just jumping to the solution, but it’s the journey that matters. Understanding the problems is how you provide confidence and assurance that the absolute focus is on solving problems and making the organisation more effective. It also means the end result is tailored to the participants, it is not “an inappropriate pattern applied indiscriminately”.
As a trainer, when teaching, I try to impress upon the class why the process on the page exists, the purpose that it is trying to achieve, as much as I convey what needs to be done. Without the context of why things are being done it becomes impossible to customise the process, there is no way of judging whether a customisation is good, or not, if you can’t compare it to the underling reason for the process. It’s not uncommon for attendees to come out the far side of our training courses saying “that’s not what I expected”, or “that’s not what I was taught last time!” They’ve heard horror stories of SAFe being a top-down bureaucratic nightmare and we’ve demonstrated through the training that, done right, it can empower teams rather than impose upon them. Any tool can be abused, the trick is convincing people not to abuse it.
And if Design Thinking’s double-diamond isn’t trying to force it’s way out of your sub-conscious by the time you’ve finished reading the above, then you probably need to go work out how you’re going to instil the “Problems First” mentality in yourself.
Figure 13: You Need To Be Here, not Here!
Agile Needs Improvisational Comedy
Ha ha, very funny…
…but actually there is seriousness to my suggestion.
Back in the late 1980’s as young teenager not yet old enough to be out of the house on a Friday night, my Friday night ritual was at 9pm to tune my second hand, black and white, dial tuned television to Channel 4 and watch Who’s Line Is It Anyway? a comedy show where the performers would enact scenes to suggestions that were drawn live from the audience. There were rules to the games, which performers already knew and had practised, what they didn’t know until the moment was suggestions from the audience. If a performer turned up with a pre-written script, which might be the funniest script in the world, the audience wouldn’t appreciate it because the enjoyment is in the humour that comes from the spontaneity.
How is this relevant to Agility?
There are rules to the game, the frameworks, but you won’t know how to utilise the rules until the suggestions come from the audience. Turn up with a pre-written script, blindly following instructions, then the audience won’t be happy because the results aren’t what they wanted. To improvise you need to know the “why” of your chosen subject, to understand whether whatever you’re about to say and do fits within the rules. Improvisation is a learnt skill, my eighties comic performers were allegedly taught by the legendary Canadian comic Mike Myers, it’s also a skill that deteriorates if not exercised, to this day those same performers from the 1980’s still do Improvisational Comedy every Sunday evening at The Comedy Store in London.
The Innovator consultants, back at the beginning of Software Agility, were improvisers by default; much of the Late Majority Consultants are following a script. The commoditisation was inevitable, the incessant desire for things to be simpler than they are, which is a commercial advantage. Fighting it would be like trying to turn back the tide, instead shape it. Teach them to improvise rather than follow a script.
Do the current training materials teach the innovation, adaption and improvisation? Not really. They hint at it; it’s there but not at the forefront of the training.
It’s a chicken and egg situation, to allow the training material to be more improvisational you need to ensure the trainers can improvise and they’re following a script. I’m will raise my hand and admit that I’m as guilty as the rest, I have run the same training course repeatedly over the last 10 years and counting, there is a script. Not a word-by-word script, but the beats, the points to emphasise, the stories to illustrate, they all have their place within the material. Take away the slides, give me a white board and marker, and I can teach from a blank sheet of paper. Most people learn by doing, so let’s do it, simulate the entire system, discussing as we go. The points to emphasise will all still be in there somewhere, the stories for illustration, but now it goes where the audience want it to go, they learn what they want to learn, coupled with the odd nudge from me to ensure they learn what they need to learn. To improvise you don’t need a script but deep knowledge of the “why” of your chosen subject.
Improvisation is real-time problem solving. The discussion has looped back to Problem Solving again, the Complex Complexity Domain; all roads seem to lead here.
Caveat Emptor
Buyer beware
Need training?
The delivery of the course is often more important than the course material itself. The delivery should be providing the insight, how to utilise the knowledge within the material, not just the material itself; after all you could just read the material. Beyond the sales piece is there evidence that the trainers know what they’re talking about?
Need coaching?
Are the coaching providers that you’ve selected going to understand your problems, or are they going to impose a solution on you? How will they address the human factors? Can they cope with situations that lie outside of their preferred methodology or framework? Can they improvise to find workable solutions that address your challenges?
|
Conclusions
Agility cannot die, but the nature of that agility may change over time.
If the ability of an organisation to adapt so that it can navigate complexity is dead, then the organisation is also dead.
Whilst “Is Agile Dead?” is the question being asked; the underlying context might actually be “Will the brand of agile that I teach or coach die? Will I still have a job?” That is harder to answer, the nature of the customers is changing and therefore the nature of the players serving those customers is also changing.
There will always be a need for coaching and training because the workforce is always changing, new people that enter the workforce will need to learn but that is steady state and, as the Agile ways-of-working become endemic for engineering and creative teams, the transformational nature of the agile industry will pass.
Technology adoption, of which development processes are just a variant, is done by humans; what has happened is that somewhere along the way the human factors have been forgotten or politely ignored. Mechanical processes are easier to communicate, they don’t answer back, they don’t need convincing, they don’t have a religious devotion to another methodology. While we can perhaps understand a few rogue contractors, many consultancies have packaged a formula to industrialise bad practice and lost sight of why they are doing this and how it should be done, because all the frameworks themselves try to put “Individuals and Interactions over Processes and Tools”.
Acknowledgments
I hope I have managed to add some rational thought to the discussions of “Is Agile Dead?” rather than contributing more hysterical clickbait, to have peeled back a few more layers to try and find out what’s really happening rather than ranting at the observable symptoms. The opinions expressed are mine, from my observations, you may draw different conclusions from the data you available to you. I’m always happy to engage in reasoned debate
Thanks to all my fellow consultants at IJI, to Averil and Richard for their words, some of which have found their way into the above posts. To Keith for the conversations during the gaps in training courses, which regularly send me off exploring different areas of the vast landscape that we coach and teach. To Simon for essentialising the Cynefin framework and providing both insight and the card images. Ian Spence for giving me my initial grounding in Consultancy and, as an old punk, teaching me to improvise and barely follow the rules, and for explaining Scrum@Scale to me as part of his work with Jeff Sutherland.
|
#1 A hugely broad statement; there will be some Late Majority Consultants that are excellent innovators and are only late due to timing issues beyond their control, blame the parents! It’s not their fault that they weren’t even born when the Agile Manifesto was being signed![Return]
#2 Infinite malleability is a fallacy, not because the people can’t change but because the systems that support them need to change too. If I had a dollar for every Agile team that doesn’t inspect and adapt their own processes because their Jira tooling has been locked down centrally, I could probably by myself a decent dinner.
Before the Atlassian developers start shouting I need to acknowledge that Jira is highly customisable. Organisations choose to lock it down. Often in the erroneous belief that doing so means that the team is following a known process and therefore they comply with some regulations somewhere, possibly Sarbanes-Oxley.
Fixing the tooling does not fix the process, it usually causes the process to mutate to overcome the deficiencies of the fixed tooling. The mutations are rarely improvements on the original, base process.[Return]
|
|
Without wanting to fan the flames of more click-bait, I hope the above has provided more insight into: Is Agile Dead?