Don’t Stand on (Scrum) Ceremony
Are sprints really working for your team?
Intro
Lately I’ve been contemplating whether the Scrum process and the ceremonies that accompany it are actually serving a purpose for our team anymore.
When added together, the time spent in sprint planning, story refinement, and sprint review takes up a large chunk of time from a two-week sprint and often interrupts the team’s flow. It’s worth reflecting on whether this is time being well spent.
When thinking about these meetings collectively, it dawned on me that the majority of energy put into them is focused on organising work into sprints. This got me thinking — why are we doing that? Why are spending all this time organising work into sprints? Is the pay-off worth it? Does it (still) add value?
Sprints have become meaningless
I used to think sprints made a lot of sense — a time-boxed period where the team have complete focus and dedication to a goal. During this time, there should be no distractions for the team and the work for the sprint is “locked-in” — no changes are made ad hoc. This offers the team a level of protection, and in return, the team make a commitment to deliver the sprint. On paper this sounds like a good idea, but in my experience this doesn’t match reality.
When I look back, I’ve rarely experienced coming to the end of a sprint where the team have completed all we set out to achieve from the start. There have been some good periods where we’ve got into a reasonable rhythm and cadence, but for the most part, what tends to happen is a mixture of the following:
- We discover certain stories are more complicated than we thought (meaning we don’t have enough time to complete other stories).
- Priorities have changed mid-sprint, and we swap stories.
- Unforeseen circumstances arise, such as team illness.
- A story has a dependency on another team to complete (i.e., infrastructure/networking/approvals).
The issue of dependencies is not a problem with sprints per se, but rather the organisation structure and culture. Working in sprints is always going to be less effective for teams that are not fully empowered to deliver changes by themselves. If a team depend on external actors, then any commitment to complete the sprint becomes hollow when they are beholden to other people who will have different priorities.
The other issues are just par for the course when building software — but I’m starting to feel like sprints make dealing with them a burden. Agile is all about embracing change, and these issues are just that, changes to the situation.
Realising something is more complicated than originally thought is simply learning and gaining a better understanding, yet it feels punishing to do so because it could mean we don’t achieve our sprint goals and the burn-down looks bad.
For the most part, we end up starting a new sprint with a bunch of the stories we weren’t able to complete from the previous sprint carried over. Given this is so frequent, I feel that the “boundary” of a sprint has become somewhat of a blur.
Sprints as a concept
The more I think about the word “sprint,” the more I dislike it. It implies a period of time where we should be working as hard and fast as possible. Of course, this isn’t the intention of the creators of Scrum; sprints are intended as a means to achieve regular cadence and flow. But I think the word can subconsciously play on the psychology of people, making them feel compelled to rush in order to get everything done by the end of the sprint. I’ve found burn-down charts to have a similar effect: people worrying about maintaining a downward graph rather than delivering quality and value.
It also feels rather relentless and unsustainable that after “sprinting” you immediately start another sprint. If you start a new sprint immediately after one finishes, are you even sprinting?
If you’re going straight into a new sprint immediately after finishing one, you end up jogging in a never-ending marathon. That’s not necessarily a bad thing; jogging over a long distance is sustainable. But it begs the question: If we are in fact jogging through the endless “sprints,” what is the purpose of sprint as a concept?
My issue with sprints is not just semantics over the name. Naming them “iterations” would lose these negative connotations, but there are other issues I’ve experienced with them regardless of name.
Estimating for the process
I’ve used story points and the Fibonacci sequence as a means of estimating for years. I’ve always thought it was a great idea — it’s much quicker to estimate things using a scale of relativity rather than trying to guess in actual time. It also makes it harder for management to hold you to an estimate that they have confused as some sort of SLA / absolute certainty.
But now I’m wondering what is the point in trying to estimate at all? The main reason we estimate in Scrum is to try and gauge how many stories from the backlog we can commit to in the next sprint. Using story points and previous sprint velocities is a sensible mechanism to figure that out, but doing so, even with relativity rather than time, can be time-consuming.
I know of teams in my organisation that spend hours in meetings playing Planning Poker to decide how many story points should be attributed to stories — debating at length whether a story should be two or three points. I must admit I thought this was a fun way to do estimates when I first came across it, but is this really a good use of time? Is there really business value being realised through this activity? I don’t think so. Given that our sprint plans tend to go up in smoke anyway, what’s the point of devoting time to estimating solely for the purpose of planning sprints? This could be time spent on solving actual business problems.
“Individuals and interactions over processes and tools” — Agile manifesto
It seems like we’re having meetings purely to serve the process.
I think a better approach is to just continually prioritise the work. When finishing a piece of work, the team could just re-evaluate what item in the backlog is considered the highest value to the business and start work on it next. Taking this approach without sprints means there’s no need to estimate because you don’t need to plan. This provides more agility than organising work into fixed sprints, especially if you’re able to ship each item before moving on to the next.
But sometimes we need to provide estimates?
There are occasions where estimates may be required for reasons other than sprint planning, such as a genuine deadline, for example, the date a new government regulation comes into effect or the date agreed in a contract with a third-party business.
In the case of genuine deadlines, we do need to think about how long the work will take. But rather than guessing with estimates, we could use projections instead. Allen Holub, in his No Estimates talk, describes the research conducted by Vasco Duarte on how projections can be made simply by counting the number of stories being completed. It’s reasonable to wonder what the difference is between projecting with a count of completed stories versus projecting with a team’s velocity of story points. Both are using historical “outcome” data, but using velocity requires all the time and energy that comes with estimating, whereas counting requires no effort at all. So if, as Vasco’s research suggests, simply counting the number of stories completed gives you the same projection as you’d get from story point velocity, why go to the effort of attempting to estimate, even when you have a deadline?
Sprints are an enabler for bogus metrics/KPIs
Another downside of sprints is how they enable bogus metrics to be used in unhelpful ways. It’s common for management to look at the velocity of teams and compare/judge them, taking insights such as these: “Team 1 are delivering twice as many points as Team 2, therefore Team 1 are more productive.” This is, of course, nonsense.
Firstly, story points and velocity only serve meaning for the given team producing them. Their purpose is to help plan the next sprint, nothing more. They are not a measurement of success. Further, the scale of relativity is likely to be different between teams. A point is likely to be worth more in effort for a team that generally work on more complex items versus a team that work in a simple domain. Trying to compare productivity in this way is comparing apples and pears.
Secondly, there is no insight to be gained from measuring the number of story points completed. What if Team 1 are delivering lots of low-value or low-quality items that aren’t useful to customers, whereas Team 2 are delivering fewer items but of high value/quality, that are useful to customers?
Really, the most important metric managers need to be concerned with is business outcome — i.e., are the teams frequently delivering on the company’s strategic goals? Of course, this is much harder to measure and it’s much easier to simply look at velocity — which Scrum gives you out of the box.
If teams are being judged on story point velocity, a side effect can be that teams start schooling the system by inflating their points estimates. This is basically Goodhart’s law:
“Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes.”
— Charles Goodhart
And Strathern’s observation:
“When a measure becomes a target, it ceases to be a good measure.”
— Marilyn Strathern
Morale
In an environment where it’s difficult for the team to commit and complete all of the planned items each sprint, team morale can take a hit. I’ve felt the feeling of failure and futility because of it. You go into a sprint review and the burn-down chart looks like a roller coaster and the stakeholders have an “I’m embarrassed for you” look on their face. Then you picture management looking at your team’s “numbers.”
If it feels like completing sprints is a futile endeavour for whatever reason (and there could be many) I think you should question whether Scrum/sprints work for you. It can otherwise be somewhat self-punishing in terms of team morale.
Should perfect Scrum be a team’s final aspiration?
Even if a team has mastered Scrum in a proper agile environment, I can’t help but think the amount of time and energy going into the process and ceremonies will start to work against a good team’s true potential. From a Lean perspective, there is just so much waste going into the ceremonies to organise sprints.
If you’ve switched from waterfall and have been successfully running with Scrum for a while, you’ve no doubt grown to become more adept at managing requirements and priorities in a more “just in time” fashion. Perhaps it’s time to crank it up a notch and aim for “continuous flow” rather than the fixed periods of cadence that comes with Scrum? I feel like good teams that have mastered Scrum would do even better by eliminating the ceremony associated with planning sprints.
Scrum alone is not Agile
This post is not intended as a take-down of Scrum or sprints; these are just my experiences with it which have occurred under particular conditions and contexts. I’m well aware that some of the negative aspects I’ve detailed in this post are more to do with the poor application of Scrum than Scrum itself.
The problem is though that Scrum seems to be particularly susceptible to poor application, which I’m not sure can be said of less prescriptive methods like XP. Of course any methodology will only be as good as the people applying it, but I think a big contributor to the malpractice of Scrum is the commercialisation of Scrum certifications where apparently anyone can become an Agile™ guru after completing a two-day course.
Managers go on these courses and come back telling all teams in the company they must now use sprints, story points, and ceremonies, and hey presto! We’re officially an Agile™ transformed company!
Sprints on sprints
I mentioned earlier that sometimes we fail to complete sprints due to dependencies on other teams. One bemusing aspect of this type of Scrum roll-out that I’ve come across is the combination of retaining skill-specific teams (as opposed to cross-functional) together with enforcing sprints. For example, having a frontend development (FED) team responsible for all CSS/HTML work and a platform team responsible for provisioning/managing any infrastructure required to run the applications. In order for feature teams to get stuff done, they now need to schedule their requirements into another team’s backlog and wait until it’s completed. Need an S3 bucket provisioned? That’ll be three weeks.
The length of delays is dependant on priorities; three weeks is a best-case scenario and assumes it can be added into the next two-week sprint. If you have a centralised FED team serving multiple feature teams, there’s a limit to how much they can do, and they will become a bottleneck. If one team’s work is considered a strategic priority for the organisation, that’s going to be their priority and will result in the other teams grinding to a halt.
This ludicrous chain of dependencies promotes a ticket culture and results in stories either remaining in the backlog waiting to be considered “definition of ready” whilst waiting on other teams, or stories get moved into a “blocked/paused” state. This is obviously the antithesis of anything resembling agile or even Scrum itself, which is pretty clear:
“Scrum Teams are cross-functional, meaning the members have all the skills necessary to create value each Sprint.”
— The Scrum Guide
Enforcing Scrum on teams like this is just paying lip service to agility. What’s actually been rolled out is known as Fake Scrum — where the organisation retains its waterfall mindset and hierarchies but with a Scrum sticky plaster over the top. This is why the Scrum process alone does not give you agility. Given my experience with it, I’ve come to agree with Allen Holub’s view:
“Scrum is not Agile. Scrum is a wrapper around agile that makes it more palatable to people who don’t actually want to be agile. It provides a structured and controlled environment that appeals to companies that are used to structured and controlled environments, but it has nothing to do with agility.”
Allen Holub, “War is Peace, Freedom is Slavery, Ignorance is Strength, Scrum is Agile” GOTO conference 2020
Conclusion
Given the right environment, Scrum and sprints can obviously work just fine. I can see how Scrum’s easy-to-understand process can help an organisation transition from an old-school waterfall approach to dividing up usually big projects into smaller batches using sprints.
But applying the Scrum process/ceremonies alone won’t yield agility and will likely just end up frustrating people who want to work in a truly agile environment, especially if that’s what’s been sold to them.
Even where a company has a good agile environment and teams within it have mastered Scrum, I can’t help but feel it’s a stepping stone to achieving maximum agility — a less prescriptive approach that just focuses on agile values and principles would be more efficient.
Teams should regularly reflect on whether the process they follow is really helping them deliver maximum value to the business, rather than following Scrum processes by rote. That said, teams in organisations running Fake Scrum will probably not be empowered to change their approach to managing their work.
For me, I’ve grown tired of the effort required to organise work into sprints.
