“Always show your working out!” was the mantra of my maths teacher in senior school. This series of blog posts “On the Nature of Lean Portfolios” is an exploration of Lean Portfolios. It is the thought processes running through my mind, exploring the possibilities so that I understand why things are happening rather than just doing those things blindly. It is not intended to be a fait-accompli presentation of the Solutions within Lean Portfolios but an exploration of the Problems to understand whether the Solutions make sense. There are no guarantees that these discussions are correct, but I am hopeful that the journey of exploration itself will prove educational as things are learnt on the way.
Portfolio Use of Weighted Shortest Job First
The standard advice within SAFe is that when it comes to prioritisation is use Weighted Shortest Job First (WSJF). It is a useful simplification to stop Cost-Of-Delay discussions getting unnecessarily complex, but there is more depth to the mechanic than people would assume from reading the SAFe webpage; depth that I’ve previously explored in a blog on The Subtleties of Weighted Shortest Job First.
Weighted Shortest Job First works at the Release Train level because the timeboxes, the Program Increments, exist. The challenge is that at the Portfolio level, where Epics can last for long periods of time, then I suspect that the “Shortest Job First” part of “Weighted Shortest Job First” is going to cause some problems. By its very nature going it’s to make it difficult to balance Short Term wins against Long Term investment, with the long term investment losing. Part of the reason for writing this blog series On The Nature Of Portfolios was to explore this very issue.
What follows over the next few postings are a series of “experiments” to explore the topic and look at the issues that arise. We’ll look at:
- WSJF using Total Epic Effort
- WSJF using Predicted Duration
- WSJF using Experimental Effort
- WSJF; what would Don do?
- WSJF considering Risk factors
- WSJF within the Investment Horizons
WSJF, Within the Investment Horizons
A few months ago I was chatting to my colleague, and SAFe Fellow, Ian Spence and he’d had an idea that might solve the challenge of Portfolio level WSJF, how to balance long term investments against the quick wins. A challenge that I’ve discussed extensively in earlier blogs.
The idea of “keeping WSJF simple by exploiting the Investment Horizons” was a core part of his 2022 SAFe SUMMIT presentation towards a Leaner Portfolio. As he was yet to write the idea up, and had referenced this blog series in his presentation, we decided that this would be a good place to publish and further explore the idea.
So, the author will be switching to “we” in this co-authored blog.
So what is the basic idea?
Weighted Shortest Job First works at the Release Train level because the timeboxes, the Program Increments, exist.Brian Tucker, SAFe Fellow
The challenge is that at the Portfolio level, where Epics can last for long periods of time, then the “Shortest Job First” part of “Weighted Shortest Job First” is going to cause some problems. By its very nature it’s going to make it difficult to balance Short Term wins against Long Term investment, with the long term investment losing. The challenges have been discussed in detail in earlier blogs in this series.
The basic idea is that we can simplify the application of WSJF to Epics, and address the challenge highlighted above, by treating the Investment Horizons as virtual time-boxes and running the WSJF algorithm to prioritize the Epics in each horizon against each other and then selecting the correct number of Epics to work on in the next PI by considering the Horizon Guardrail.
What are the Investment Horizons and How are They Used?
Figure 1: McKinsey Horizons Model
The idea references the McKinsey Horizons Model which is incorporated into SAFe as one of the four Portfolio Guardrails. The short explanation is that: a Portfolio should deliberately reserve some time for:
- Investing and exploiting the currently commercially available solutions (Horizon 1) to defend and extend your existing business. Investments within this horizon will produce a positive return on investment within a year from their approval.
- Emerging new and next generation solutions (Horizon 2) to grow your business into new emerging markets and renew your organization. Investments in this horizon will produce a positive return within 1 to 2 years (1 year < time to return < 3 if you want to be pernickety) and will typically require substantial organizational change to take the emerging solutions to market.
- Evaluating new ideas (Horizon 3) that will seed options for future growth. Investments within this Horizon will need see a return on investment for 3 years or more.
Capacity allocations will need to be made to ensure that there is investment in future growth and new transformative ideas otherwise short-term investment in the currently popular solutions will steal all the time and effort, because they are the current cash earners, and in doing so starve the Portfolio of the new solutions needed to replace the current ones when their time ends.
This Horizon Guardrail is one of the four key guardrails used in SAFe’s approach to Lean Portfolio Management.
| Note: |
|---|
| Care must be taken that when using the investment horizons they are treated in the way that they were originally intended – as investment horizons and not some kind of solution lifecycle. It is quite possible to bring a new product to market and achieve a positive return within a year. This would make the investment in this new product a horizon one investment. It is also implies that there will be minimal internal disruption when taking the new product to market (disrupting the competition is much more fun than disrupting yourself). If the new product requires new or significant changes to the operational value streams, although the development may take less than a year, to achieve a positive return on the investment is likely to take more than a year making it a Horizon 2 or 3 investment. |
The SAFe idea of mapping the current and proposed solutions to the investment horizons is a useful tool for understanding the current state of your portfolio but it only provides you with a snap-shot in-time (and is also more complicated than it first appears – the subject for a possible future blog).
It is much more powerful to relate the investment horizons to your significant portfolio investments, which in SAFe will typically take the form of Portfolio Epics.
Portfolio Epics and the Investment Horizons
As part of the Epic’s Lean Business Case you must forecast the return that you expect to achieve for the investment. This is what the Business Outcomes and Leading Indicators are for.
To understand how the Epics relate to the Investment Horizons it is first necessary to consider how investment in the creation and evolution of cyber and cyber-physical systems works. Any large investment will typically follow the profile shown below:
Figure 2: Value Accrued against Time
In SAFe we develop all of our solutions iteratively and incrementally to minimize the period before we have broken even (see: The Case for Incremental Development, Scott Ambler). This means that the solution is launched before the break-even point (we need to be bringing money in to start climbing the curve) and that development costs continue to be expended throughout the lifetime of the solution up until its successful retirement.
This general pattern will hold for whatever horizon the Epic is in – if you calculate the benefits and cost from the point of time that the Epic starts.
| Horizon | Focus | Return on Investment | Break-even Point |
|---|---|---|---|
| Horizon 3: Evaluating |
Wide ranging and exploratory – seed options for future growth | Over 2 years | Uncertain but not within sight |
| Horizon 2: Emerging |
Next generation offerings – growth and organizational renewal | Within 2 years | Within 2 years but hopefully earlier |
| Horizon 1: Extracting and Investing |
Next generation offerings – growth and organizational renewal | Current year | Within the year possibly within a PI. |
| Horizon 0: Retiring |
Decommission existing products (even if they’re H2 or H3 products) | Current year | Hard to say |
| Note: |
|---|
| As an addition to the original 3 Horizons identified by McKinsey, Geoffery Moore1 added a 4th Horizon, Horizon 0 – retiring existing solution. This represents the fact that specific investment is often needed to de-commission existing solutions that have run their course and are now making a negative contribution to our outcomes. Some organisations struggle with turning things off, nobody wants to use the budget for “The Replacement Solution” to turn of the “Old Solution”, so the “Old Solution” never gets turned off. Thus, many organisations are awash with old “Zombie” solutions that are running in the background and slowly consuming the operational budget of the organisation. Geoffrey Moore asserts that the authority required to get people to spend money on turning stuff off is so senior that sometimes it needs to come from the CEO themselves, hence instigating Horizon 0 to make it visible, and to allow formal allocation of budget to that Horizon in order to get old systems properly decommissioned.
If your organization has set a Horizon 0 guardrail (an unusual but occasionally necessary thing to do) then you would WSJF the Horizon 0 Epics against each other to understand the bast way to invest this allocation. In most cases Horizon 0 Epics take less than a year and can be treated as Horizon 1 Epics. |
The Horizon that an Epic lies within is determined by the predicted time to reach a Return-On-Investment.
The beauty of this approach is that we’re no longer trying to compare apples with oranges; we are no longer trying to compare long-term and short-term investments or relatively estimate and compare incredibly risky and unpredictable things (our Horizon 3 Epics) against lower risk, highly predictable things (our Horizon 1 Epics).
The Epic’s Horizon will need to be reassessed every Program Increment before you rerun the WSJF (remember you need to do it every PI) because the Epic should be getting closer to its Return-On-Investment and have better forecasts of the potential benefit and costs. Remember that the Horizons aren’t a lifecycle, some Epics will instantly start in Horizon 1 because they are forecast to produce a Return-On-Investment in under a year.
| Note: |
|---|
It may turn out that an Epic has been put in the wrong horizon and the Return On Investment may be closer than we think or alternatively even further away – we handle this in two ways:
|
Wouldn’t Slicing be a better option?
You may think that maintaining 3 prioritized lists sounds like a lot of bureaucratic nonsense and overhead.
As it says in SAFe, one of the benefits to using WSJF is that “this model encourages splitting large jobs into multiple smaller ones to compete against other smaller jobs”. But, slicing things up can run into challenges with the value being lost, as previously described in Slicing Epics.
So, what happens when we have a Horizon 3 Epic that is going to take 3 years of development to deliver the expected returns (we know that successful solutions will continue to generate benefits without major investment but we can ignore this as we are just considering the initial Epic in this case). Now our graph will look more like this:
Figure 3: Value Accrued against Time showing Years and Quarters
So, what would happen if we tried to slice up this Epic into 1 year slices so that it could compete with the more prevalent Horizon 1 Epics.
Well, we’d have to estimate the WSJF parameters for the first slice of the Epic:
- Business Value – Well this is negative at the end of the first year. So, we’d need to have a negative estimate for the business value
- Time criticality – Well it’s not going to be more than 1….
- Opportunity and Risk Reduction – well this gets even more difficult. Do we consider the promised 3 years out future benefits as an Opportunity? Even relative estimating is going to get very difficult here because of the huge margin of error involved when predicting the far future. There’s a strong probability that this will lose money overall so the Fibonacci sequence will not cater for the margin of error on the estimate. Also as we’ve not finished anything of any significance we’re not really opening up any new opportunities.
And as for our proxy for duration we’re really going to struggle as this is meant to represent the delay not the effort or the cost (these are just proxies).
Now the further out in time the greater the margin of error on our forecasts but the people who build the business cases are generally optimistic not pessimistic so we should always work under the assumption that if anything the development will cost more than expected and deliver less benefit than predicted.
What should we use as the Duration parameter in WSJF?
From the previous blogs we’ve seen that there are a number of different ways in which the Duration parameter of the WSJF could be formed.
Observation: Separate Requested Effort, Planned Effort and Priority
Requested Effort is the amount of Effort that the Epic is requesting from the Development Value Streams in this timebox.
Planned Effort is the amount of Effort that the Development Value Streams have given the Epic once they’ve been through their PI Planning Events.
Priority is the relative priority to the other Epics within the Portfolio, which WSJF can provide.
The relationship is that if the Requested Effort and the Planned Effort don’t match and the Priority is high then this might be a trigger that there is a problem. Either the Development Value Streams haven’t honoured the priority and are doing something else, or the Epic is requesting more Effort than the Development Value Streams have available.
However to obtain a Prioritisation it can’t be done on just the Requested Effort, that leads to the “As Below, so Above” and “Don’t Lose Focus” issues described in WSJF using Experimental Effort.
From the above observation the use of Experimental Effort, the effort within the timebox, would be wrong since we want a priority for the Epic as a whole. Therefore Total Effort or Predicted Duration would be better options, of which Predicted Duration is probably the best as it can account for Development Value Streams swarming to complete Epics and extended Durations if only a small amount of effort can be allocated per-timebox.
The idea is to use the Investment Horizons to keep Portfolio WSJF simple by running the WSJF within the Horizons. This means that you are comparing like with like; Horizon 3 (blue sky ideas) with long lead times are being compared against other long lead time items. This simple separation negates the challenges identified in earlier discussions with using Total Effort or Predicted Duration.
If Total Effort is being used then, drawing upon previous discussions that explored the relationship between Cost and Effort, it would be possible to use Estimated Cost as a substitute for the duration parameter.
Cost is also a better option at this level because it can factor in other costs over and above the pure development costs. Additionally, it is not uncommon for some of the work within Horizon 3 to be sub-contracted out or solutions bought in, since Horizon 3 is often setting up new lines of business, new Operational Value Streams where the supporting Development Value Stream do not yet exist therefore there is no historic data with which to establish a duration
WSJF, Within Horizons
The idea is to use the Investment Horizons to keep Portfolio WSJF simple by running the WSJF within the Horizons. This means that you are comparing like with like; Horizon 3 (Seed options for future growth) with long lead times for the Return-On-Investment are being compared against other long lead time items. This simple separation negates the challenges identified in earlier discussions with using Total Effort or Predicted Duration that were trying to balance the long term investments against quick wins in a single monolithic Weighted Shortest Job First.
The cyclic nature of the process deals with any inaccuracies. It is purely providing insight into the prioritisations of the set of Epics for the next Program Increment. Insight gained over the course of the Program Increment will correct mistakes when the process is repeated ready for the next Program Increment. It will also allow the Epic itself to progress through the horizons as the break even and Return On Investment points become closer.
Once separate the low cost Horizon 3 experiments will be competing for priority against other low cast Horizon 3 experiments, the bigger costs of work in Horizon 1 will be competing against other Horizon 1 work with similar costs. The capacities that the portfolio allocates to each Horizon then dictates how many, or how much, of the Epics can be done within each Program Increment.
With all of the setup covered it should be possible to run the set of test data through this variant of algorithm.
Figure 4: The test set used for the WSJF experiments updated to include horizons
When the test set was initially drawn up almost 2 years ago, it wasn’t foreseen that the experiments around WSJF would lead towards horizons, therefore the test set needs updating. The logic behind how the entries in the test set map to the Horizons needs to be expanded and explained:
New Product Small – Whilst this has just a small amount of physical effort the product is creating a new market, a new revenue stream, which will take some time to build up. It’s predicted Return-On-Investment is 3 years away; hence it’s Horizon 3 categorisation.
New Product Large – The early experimentation has already been done which is why the business knows that there is a demand and that that demand will translate into profits. It’s predicted Return-On-Investment is just under 2 years away; hence it’s Horizon 2 categorisation.
Regulatory Demand – Whilst the fixed date in the future isn’t immediately urgent, i.e. the next PI, the Return-On-Investment for this will still happen within a year, therefore Horizon 1.
Technological Change – Will provide a Return-On-Investment within the next year, therefore Horizon 1.
The Epics can be divided into the horizons and WSJF performed on each Horizon separately.
| Horizon 3 | ||||||
|---|---|---|---|---|---|---|
| Epic | BV | TC | RR|OE | CoD | Cost | WSJF |
| Product S | 13 | 1 | 1 | 15 | 1 | 15 |
| Horizon 2 | ||||||
|---|---|---|---|---|---|---|
| Epic | BV | TC | RR|OE | CoD | Cost | WSJF |
| Product L | 8 |
1 | 1 | 10 | 5 | 2 |
| Horizon 1 | ||||||
|---|---|---|---|---|---|---|
| Epic | BV | TC | RR|OE | CoD | Cost | WSJF |
| Regulatory (Swarm) |
21 | 1 | 1 | 23 | 2 | 11.5 |
| Enabler | 1 | 1 | 5 | 7 | 5 |
~1.3 |
For the purposes of the following discussions we’ll say that the portfolio has a Total Effort of 10 and that the balance of effort between the horizons is H3 10%, H2 20%, H1 70%.
Figure 5: Capacity Allocations for the Horizons
Product S is in Horizon 3 on its own. It can’t suck up all the effort with the promise of high rewards but “allegedly” small effort because the effort is constrained by the capacity available for Horizon 3. Its effort fits within the available capacity. Product L is in Horizon 2 on its own. The long lead time isn’t a problem for prioritisation because it’s not competing against anything else. Since there is a capacity allocation on the Horizon of 20%, which results in an available effort per Program Increment of 2, the Epic will take at 2.5 Program Increments to complete. Regulatory and Enabler are both competing for space in Horizon 1; however since that horizon gets most of the available capacity they both be fit. The regulatory piece has the higher priority.
Arguably the test set is too small to demonstrate this variant of the WSJF since there aren’t enough entries to see the prioritisation occurring within each of the horizons. It could be extended to include more within each horizon to show WSJF being run over each sub-set, but once separated into the horizons it just becomes a demonstration of running WSJF which isn’t that exciting. What happens to the prioritised Horizons once the prioritisation has been completed is more interesting.
Figure 6: Using capacity cut-offs to combine the horizons
Once each horizon has been prioritized the cut-off lines can be drawn to show where the available capacity ends. The Epics above the cut-off are the priorities for the next PI. This provides a priority within that horizon, but doesn’t give an absolute priority for the entire set. Is an Epic in Horizon 1 a higher priority than an Epic in Horizon 3? The lists need to be merged.
There is no algorithmic way to perform the merge without looping back into some of the convoluted discussions about WSJF that we’ve already been through; the situation has to be resolved through discussion, but there are options for resolving the merge:
- Seed the final list with the Horizon 1 epics (typically the largest capacity)
- Starting from the top insert (in order) the Horizon 2 epics at the appropriate spot
- Repeat for Horizon 3
Or
- For each Horizon 2 and Horizon 3 Epic, identify a slice of the Epic that’s less than a year in duration
- Add the slice into the Horizon 1 List
- Run WSJF across the Horizon 1 list that has been extended to contain the Horizon 2 and Horizon 3 slices
The former is probably easier, the latter becomes a variant of WSJF with Experimental Effort.
The key thing is we’ve sized the list and are just looking to rank the Epics in play, the ones above the cut-off, therefore a simple ranking is good enough; the Release Trains pulling Features and prioritising them in their backlog will take care of the rest…
Whilst not having am absolute priority could make Feature prioritisation harder, in reality it probably won’t. Each feature has to be discussed on it’s own merits because a “nice to have” Feature from the highest priority Epic might not be as important as the “Must Have” feature from a lower priority Epic. The interleaving always has to take place in the Feature backlogs at the Release Train level. The set of Epic priorities are useful inputs to the Feature prioritisation discussions but they are just one influence of many; Features from two Epics that have not been prioritised alongside each other, because they are from different horizons, can still be prioritised, because it’s done on an individual Feature’s merits; the conversations to do this might take longer because they don’t share a common reference point which the absolute priority provide, but it can still be done.
Consider Risks?
Probably not; trying to factor Risks into the algorithm as described in WSJF considering Risk Factors might have been a mistake. It was a useful detour as part of this series of articles exploring WSJF, because it enabled us to discuss risks, but not something that should be incorporated into WSJF.
It’s the job of the approvals process to consider the risks. If the risks are calculated into the WSJF then you don’t need approvals you just follow the algorithm2. However the algorithm isn’t capable of balancing the risk across the portfolio, it doesn’t account for the portfolio’s appetite for risk.
Observation: Appetite For Risk is separate from Priorities
WSJF is prioritisation for the set under construction, Approvals is about taking the set and considering “which one do we want to take the risk on and develop”.
Horizon 3 Epics in particular are risky, they’re pure experimentation and not all will succeed. The effort involved will not produce a return on investment, the exploration is pure expense. How much surplus profits does the portfolio have to fund that exploration? How much risk is it willing to take? No Prioritisation mechanism will help with these decisions, these are the decisions that the Lean Portfolio Management need to make. They must take some risks, to take no risks would be to always remain in Horizon 1 and there is always the possibility that the current Horizon 1 solutions will end, no operational value stream lasts forever3.
Horizon 2 Epics might fail to become productive lines of business. This happens all the time, products are there one minute and then gone; and it happens with physical products as much as does with IT products. The product, the Solution, hasn’t proved to be profitable so the organisation stops funding it, Google Betas that are discontinued, grocery lines that are in the supermarkets one month then gone. How much of an appetite does the Portfolio have for taking the risk that this Solution might not be profitable? Google’s advertising revenue from the search engine can fund an awful lot of risk, not all organisations are lucky enough to have a goose that lays golden eggs4.
The capacity allocation between the Horizons is there to ensure that some risk is taken for the sake of the future organisation.
WSJF, Conclusions
All models are wrong, but some are usefulGeorge E. P. Box
WSJF, Within Horizons solves some of the challenges of balancing the Short Term wins against Long Term investment by using the Horizons to separate them.
It works; Ian has been using this technique within the Portfolios of a large multi-billion dollar international organisation. It works extremely well for Horizon 1 and Horizon 2. It’s fairly good for Horizon 3 but there is always the challenge that there is no upper limit on duration for the return-on-investment in Horizon 3, any return that is too far into the future is always going to struggle to be prioritised.
It avoids premature slicing of Epics and the race to the bottom that can result in the Portfolio prioritising big Features rather than true Business Change, all because small slices are needed to “game” the algorithm.
The Capacity Allocation maintains the balance across the Horizons and the process honours that by utilising it rather than trying to cheat the inputs to honour the capacity allocation after the process has been run, which is what tends to happen with simple WSJF.
But, it only works with a true Product Mentality where Epics are ideas that could be pursued rather than the simplistic “work to be done”. Without the freedom to pick and choose ideas the Portfolio is always going to be constrained and inflexible.
The challenge is most organisations have a surplus of Project Managers who care about work being done, and a lack of Product Managers that care about the Product they own and making it successful. Hence most portfolios lapse into a “work to be done” mentality rather than “success”.
Next Steps
If you’re reading this chronologically as the articles are published, then the next step is to return to Epics and look at Enabler Epics, or perhaps the role of Epic Owner.
If you’re reading this sequentially through the topics, the next step is to explore the feedback cycle of Epics and put the case for and against splitting large Epics.
#1 Geoffrey A. Moore, Zone To Win, ISBN: 978-1-68230-211-8
#2 If you don’t need approvals, you don’t need the decision makers, the Executives, that make up the Lean Portfolio Management. Just get rid of them… viva-la-revolution! Let us know how it all turns out, we’ll be watching from afar.
#3 Except for the tax office… we once did a Value Stream analysis workshop at a tax office, we traced the Operational Value Stream, “Taxing The Population”, back over 400years, it went back further still but we had better things to do… I did ask if they thought the Operation Value Stream was going stop anytime soon, the answer was a resounding no, people are still going to be paying taxes in the future. Sorry.
#4 The Goose That Laid The Golden Eggs: one of Aesop’s Fables.