This is the first of a series of posts I’m working on related to the big idea of Story Points, tracking project progress, judging value, and a myriad of related issues. For instance, I’m aware of three active agile government projects that each has a strong desire to use story point production (or a similar valuation) as a measure of progress and base payment on that progress per sprint or product increment (release). This isn’t to say that is good or bad, contractually right or wrong; rather, I want to explore this need of the customer to evaluate progress and value. But first, an introduction to Story Points:
Why are story points used in Agile/Scrum?
The notion of Planning Poker, getting the team together to debate the relative size of a story is to serve two primary goals. 1) To create communication among the team in discovering what the meaning, purpose, and understanding of the story. E.g. I think it is a ‘3’, you think it is a ‘13’, let’s discuss why we think that and perhaps discover some new insights. 2) it is intended to be a quick method of estimating. The intent is to not spend a lot of time trying to ‘perfect’ the estimation. It is to build general consensus and understanding with the team, and do it quickly. (Based on an estimating technique called Wide Band Delphi, developed in the 50’s or 60’s)
How are story points useful?
They are useful for quick estimations and long-term empirical projections. The only reason they can provide long-term ‘accuracy’ is because of the Law of Large Numbers. Essentially, the accuracy comes into focus after many sizing attempts of stories, and begin to average out and represent a forecasting value we call velocity. Some will be sized too low, some too high, but that is reality. If you roll a die enough times the average converges on 3.5.
Wikipedia: An illustration of the law of large numbers using a particular run of rolls of a single dice.
Why story points are not a good measure per sprint?
Based on this information, story points really are not a good measure for short-term projections. Let’s describe this in different terms. If the Washington Wizards are half-way through their season and currently have a scoring average of 95 points, we can say that for the remaining season they will likely average 95 points a game. A good long term projection. What we cannot say is that the next game they will score 95 points. They may never score 95 points in any of the remaining games. Imagine the team trying to play the game to score a particular number of points instead of winning the game? When we start to measure the teams to ‘score’ a certain number of points in the next sprint, with all the variables involved, we are going to start to make different choices to meet the score instead of delivering value (winning the game). Do we shoot a 3 pointer, or 2? Should I try to miss but get fouled so I can score 1? When applied to an agile team, we have many impactful scenarios like: Team members that are out (sick/vacation), poorly developed user stories, dev environments not available, dependencies, etc. You would not expect that basketball team to score their average if some of the team members were missing, or the rules changed in the game, etc. By measuring performance by meeting a quota, we start to focus on local optimizations like ‘scoring’ instead of looking at the whole system and ‘winning’.
Question: Do you have projects that are looking to establish Story Point quotas as a part of an incentive plan to encourage production? Or to use points as the primary method for contractual progress? You can leave a comment by clicking here.