Over the last couple of weeks, there have been a lot of conversations in my LinkedIn feed around story points and estimation in general. Many people believe that story points are essential for team planning and delivery, while others argue that they are a waste of time and a source of frustration.
I am not gonna enter into the debate of story points versus hours versus t-shirt size versus no estimation. I believe it is just a distraction from the real issues.
Estimating is natural…but we are bad at it
My take on estimation is that estimating is natural and gives us a sense of certainty and control which our brain craves. As Humans, we always estimate, whether consciously or unconsciously.
- “How much time do I need to get my laundry done?”
- “How much time do I have to prepare myself before going to the restaurant?”
- “How far is the grocery store so I can plan when to go depending on my day?”
These are natural questions we ask ourselves daily.
Even if we choose not to use story points or any specific estimation technique, we are still making assumptions about how long a product, a feature, a project, a fix, etc will take us before delivery… and we will be bad at it.
Researchers have long been studying human estimation abilities and have found that, on average, we tend to be bad at it.
The planning fallacy is a term used by psychologists to describe our tendency to underestimate the amount of time it will take to complete a task. The term was first coined in 1977 by psychologists Daniel Kahneman and Amos Tversky.
Another study conducted by Roger Buehler, Dale Griffin, and Michael Ross found that people consistently underestimate the amount of time it takes to complete tasks, regardless of their level of expertise or experience in a particular domain. Other studies have shown that we are prone to overconfidence in our estimates, leading us to make incorrect assumptions about timelines and resource allocation.
Though, size matters
There is research that suggests that people tend to be more accurate when estimating short time intervals compared to longer ones.
For example, a study published in the Journal Of Experimental Social Psychology in 2004 found that people tend to be more accurate when unpacking or listing small tasks needed to achieve an overall goal, especially when the goal involves more complexity and multi-facets. Hence by unpacking goals or bigger tasks, we can reduce the planning fallacy. Another study published in the Journal of Software: Evolution and Process in 2010 found that developers tended to be more accurate when estimating tasks that took less than a day.
These studies suggest that estimation accuracy can vary depending on the time scale of the task being estimated.
In the end, estimating or not estimating is a false debate.
Even the “no estimates” movement is still estimating. Saying a team has been able to deliver 8 items last time therefore it can deliver 8 items this time is still an estimation.
We need to talk about the real things.
How much time do we want to spend estimating
It is fundamental for a team and its stakeholders to have an open and honest conversation about how much time they want to spend estimating versus achieving the work at hand. Estimation can be time-consuming, and it’s essential to strike a balance between getting estimates that we judge “accurate” (which is an oxymoron or better said – a rather beautiful cognitive dissonance of our human brain) and actual work delivering value to customers.
Therefore we need to strike a balance between estimating and getting the work done.
The conversation should be focused on the team goals and the value it should deliver over a short period (whether you work in sprints, iterations, or on an item-done basis).
The team’s goal needs to be also tied up to the overall business goals – where does it fit into the big picture? Does it make sense?
Estimation is only one piece of the puzzle, and it’s critical to keep in mind the big picture and what the team is trying to achieve.
By defining the goal and measuring its value delivered, the team can get a clear understanding of what’s working and what’s not and adjust their approach accordingly.
This approach can also help identify areas of waste and reduce the overall cycle time of the team, leading to increased efficiency and better outcomes.
Dealing with “when will it be done?”
One question that always comes up is “when will it be done?”. The answer to this question is never straightforward. We can use probabilistic forecast such as Monte Carlo to provide a range of possible outcomes based on yesterday’s weather and retrospectively analyze the outliers of the distribution to get the necessary lessons on what happened.
Monte Carlo is a statistical technique that allows us to model the uncertainty and risk inherent in any project. When using a probabilistic forecast like the Monte Carlo simulation to estimate completion dates, it’s important to focus on the probabilities themselves rather than the accuracy of the estimate, and that’s the beauty of it! We start talking about probabilities rather than accuracy.
By using the beauty of probabilities, teams can communicate more clearly and effectively with stakeholders about the level of uncertainty and risk involved. This helps to manage expectations and reduce the likelihood of disappointment or frustration when the estimated completion date is not met. Additionally, using probabilities allows teams to make more informed decisions about how to allocate resources and manage risk since they have a clearer understanding of the potential outcomes and their likelihood. Overall, using probabilistic forecasts can help teams to make better decisions, manage risk more effectively, and communicate more clearly with stakeholders and their leaders.
This last part is key.
I believe it is where as agile coaches, scrum masters, change agents, or agilist, we need to bring our expertise, coaching, facilitating, and mentoring.
Leveraging the use of probabilities to shift the vocabulary used hence shifting the expectations.
What about “New development”?
One of the comments I received on LinkedIn was that we can’t rely on yesterday’s weather for “new development” and therefore probabilistic forecasting being a reference-based estimation cannot be used there.
Yes, but I’d like to bring some nuance.
First, isn’t it the case with all estimations techniques to rely on yesterday’s weather or yesterday’s experience?
More, isn’t it the base for empiricism?
When I work with teams that want to estimate with story points, we always take references from the past. If we don’t have available references from the past, we make new references that we retrospectively analyze therefore they become yesterday’s weather.
And, I do the same with teams that work with hours or t-shirt sizes.
Second, what do we mean by “New development”?
Are we referring to a complicated or complex “New development”?
If we refer to the Cynefin framework, in the case of complicated “new development”, yesterday’s weather still can help. We can leverage our experience to “Sense” what should we do based on our experience and then “Analyze” and “Respond”.
Complex “New development”, however, are often unpredictable, and the best approach is to “Probe”, then “Sense” and “Respond.” We can’t use yesterday’s weather in those cases as our experience doesn’t help us make “Sense”.
Rather than trying to control the situation or insisting on a plan of action, it’s often best to be patient, look for patterns, and encourage a solution to emerge.
So, we need to have a conversation about what are we willing to invest in for budget, people & time.
We need to move forward in small steps, assess the results or non-results and focus on reducing uncertainty but without being assured to yield results.
In Complex “New development”, we are going outside of the estimation field.
My learnings & conclusion
In short, a team should be empowered to use whatever they feel is good for them when it comes to estimation. Estimating is after all an help in capacity planning and not in delivery. We should also strive to reduce team dependencies (this will be part of another blog post).
I have been guilty of pushing story points on teams, believing that they were the key to the team’s successes and reassuring stakeholders or leaders. However, over time, I have come to realize that story points or whatever form of estimation does not matter much in the end.
In conclusion, while the debate over story points and estimation will probably continue, we should remember that what really matters is how we work together as a team to deliver value to our customers.
As I recently wrote in a comment on LinkedIn, I could not care less about what a team is nowadays using to estimate.
I’d rather coach the team, its stakeholders, and leaders in shifting their vocabulary using probabilities to manage expectations.
I would also teach and facilitate the use of the Cynefin framework for a team, its stakeholders, and leaders to understand the type of situation they are dealing with.
Diane Gombart
The writer of this article is Diane Gombart, an experienced Agile coach with a passion for helping individuals and organizations reach their full potential through Agile methodologies.
