10.4 Creating Meaningful Overall Best Case and Worst Case Estimates

10.4 Creating Meaningful Overall Best Case and Worst Case Estimates

If you can't use the sum of the best cases and worst cases to produce overall Best Case and Worst Case estimates, what do you do? A common approximation in statistics is to assume that 1/6 of the range between a minimum and a maximum approximately equals one standard deviation. This is based on the assumption that the minimum is only 0.135% likely and the assumption that the maximum includes 99.86% of all possible values.

Computing Aggregate Best and Worst Cases for Small Numbers of Tasks (Simple Standard Deviation Formula)

For a small number of tasks (about 10 or fewer), you can base the best and worst cases on a simple standard deviation calculation. First, you add the best cases together and add the worst cases together. Then you compute the standard deviation using this formula:

(#4)

If you take 1/6 of the range between 20.0 and 38.6 in Table 10-5, that will be 1 standard deviation of the distribution of project outcomes for that project. One–sixth of that difference is 3.1. You can then use a table of standard deviations to compute a percentage likelihood. In a business context, this is often referred to as percentage confident. Table 10-6 provides the standard deviation numbers.

Table 10-6: Percentage Confident Based on Use of Standard Deviation

Percentage Confident	Calculation
2%	Expected case - (2 x StandardDeviation)
10%	Expected case - (1.28 x StandardDeviation)
16%	Expected case - (1 x StandardDeviation)
20%	Expected case - (0.84 x StandardDeviation)
25%	Expected case - (0.67 x StandardDeviation)
30%	Expected case - (0.52 x StandardDeviation)
40%	Expected case - (0.25 x StandardDeviation)
50%	Expected case
60%	Expected case + (0.25 x StandardDeviation)
70%	Expected case + (0.52 x StandardDeviation)
75%	Expected case + (0.67 x StandardDeviation)
80%	Expected case + (0.84 x StandardDeviation)
84%	Expected case + (1 x StandardDeviation)
90%	Expected case + (1.28 x StandardDeviation)
98%	Expected case + (2 x StandardDeviation)

Using this approach, a statistically valid 75%-likely estimate would be the Expected case (29 weeks) plus 0.67 x StandardDeviation, which is 29 + (0.67 x 3.1), which equals 31 weeks.

Why do I say the answer is 31 weeks instead of 31.1? Because the garbage in, garbage out principle applies. The underlying task estimates are not accurate to more than 2 significant digits, much less 3, so be humble about the results. In this example, presenting an estimate of 31 weeks probably overstates the accuracy of the result, and 30 might be a more meaningful number.

Tip #49

Use the simple standard deviation formula to compute meaningful aggregate Best Case and Worst Case estimates for estimates containing 10 tasks or fewer.

Computing Aggregate Best and Worst Cases for Large Numbers of Tasks (Complex Standard Deviation Formula)

If you have more than about 10 tasks, the formula for standard deviation in the previous section isn't valid, and you have to use a more complicated approach. A science-of-estimation approach begins by applying the standard deviation formula to each of the individual estimates (Stutzke 2005):

(#5)

You use this formula to compute the Standard Deviation column in Table 10-7. You then go through some fairly complicated math to compute the standard deviation of the aggregate estimate.

Table 10-7: Example of Complex Standard Deviation Calculations

Weeks to Complete
Feature	Best Case	Worst Case	Standard Deviation	Variance (Standard Deviation Squared)
Feature 1	1.6	3.0	0.233	0.054
Feature 2	1.8	4.0	0.367	0.134
Feature 3	2.0	4.2	0.367	0.134
Feature 4	0.8	1.6	0.133	0.018
Feature 5	3.8	5.2	0.233	0.054
Feature 6	3.8	6.0	0.367	0.134
Feature 7	2.2	3.4	0.200	0.040
Feature 8	0.8	2.2	0.233	0.054
Feature 9	1.6	3.0	0.233	0.054
Feature 10	1.6	6.0	0.733	0.538
TOTAL	20.0	38.6	-	1.22
Standard Deviation	-	-	-	1.1

1. 
   

   Compute the standard deviation of each task or feature using the preceding formula.

   
   


2. 
   

   Compute the square of each task's standard deviation, which is known as the variance. This is shown in the right-most column of Table 10-7.

   
   


3. 
   

   Total the variances.

   
   


4. 
   

   Take the square root of the total.

   
   

In the table, the sum of the variances is 1.22, and the square root of that is 1.1, so that's the standard deviation of the aggregate estimate.

Tip #50

Use the complex standard deviation formula to compute meaningful aggregate Best Case and Worst Case estimates when you have about 10 tasks or more.

If you recall that the standard deviation produced by the preceding approach was 3.1, you'll realize that this approach produces an answer of 1.10 from the same data, which is quite a discrepancy! How can that be?

This turns out to be a case of the difference between precision and accuracy. The problem with using the formula (WorstCaseEstimate — BestCaseEstimate) / 6, is that, statistically speaking, you're assuming that the person who created the Best Case and Worst Case estimates included a 6 standard deviation range from best case to worst case. For that to be true, the estimation range would have to account for 99.7% of all possible outcomes. In other words, out of 1000 estimates, only 3 actual outcomes could fall outside their estimated ranges!

Of course, this is a ridiculous assumption. In the example, 2 outcomes out of 10 fell outside the estimation range. As Chapter 1 illustrated, most people's sense of 90% confident is really closer to 30% confident. With practice, people might be able to estimate an all-inclusive range 70% of the time, but estimators don't have a ghost of a chance of estimating a 99.7% confidence interval.

A realistic approach to computing standard deviation from best and worst cases is to divide each individual range by a number that's closer to 2 than 6. Statistically, dividing by 2 implies that the estimator's ranges will include the actual outcome 68% of the time, which is a goal that can be achieved with practice.

Table 10-8 lists the number you should divide by based on the percentage of your actual outcomes that are falling within your estimated ranges.

Table 10-8: Divisor to Use for the Complex Standard Deviation Calculation

If this percentage of your actual outcomes fall within your estimation range…	…use this number as the divisor in the standard deviation calculation for individual estimates
10%	0.25
20%	0.51
30%	0.77
40%	1.0
50%	1.4
60%	1.7
70%	2.1
80%	2.6
90%	3.3
99.7%	6.0

You would then plug the appropriate number from this table into the complex standard deviation formula:

(#6)

Tip #51

Don't divide the range from best case to worst case by 6 to obtain standard deviations for individual task estimates. Choose a divisor based on the accuracy of your estimation ranges.

Creating the Aggregate Best and Worst Case Estimates

In the case study, the team's actual results fell within its best case—worst case ranges 8 out of 10 times. Table 10-8 indicates that teams hitting the actual result 80% of the time should use a divisor of 2.6. Table 10-9 shows the results of recomputing the standard deviations, variances, and aggregate standard deviation based on dividing the ranges by 2.6 instead of 6.

Table 10-9: Example of Computing Standard Deviation Using a Divisor Other Than 6

Weeks to Complete
Feature	Best Case	Worst Case	Standard Deviation	Variance (Standard Deviation Squared)
Feature 1	1.6	3.0	0.538	0.290
Feature 2	1.8	4.0	0.846	0.716
Feature 3	2.0	4.2	0.846	0.716
Feature 4	0.8	1.6	0.308	0.095
Feature 5	3.8	5.2	0.538	0.290
Feature 6	3.8	6.0	0.846	0.716
Feature 7	2.2	3.4	0.462	0.213
Feature 8	0.8	2.2	0.538	0.290
Feature 9	1.6	3.0	0.538	0.290
Feature 10	1.6	6.0	1.692	2.864
TOTAL	20.0	38.6	-	6.48
Standard Deviation	-	-	-	2.55

This approach produces a standard deviation for the aggregate estimate of 2.55 weeks. To compute percentage-confident estimates, you would then use the Expected Case estimate of 28.6 weeks from Table 10-5 and the multipliers from Table 10-6. This would produce a set of percentage-confident estimates such as the ones shown in Table 10-10.

Table 10-10: Example of Percentage-Confident Estimates Computed From Standard Deviation

Percentage Confident	Effort Estimate
2%	23.5
10%	25.4
16%	26.1
20%	26.5
25%	26.9
30%	27.3
40%	28.0
50%	28.6
60%	29.3
70%	30.0
75%	30.3
80%	30.8
84%	31.2
90%	31.8
98%	33.7

Depending on the audience for these estimates, you might heavily edit the entries in this table before you present them. In some circumstances, however, it might be quite useful to point out that although totaling the Best Case estimates yields a total of 20 staff weeks, it's only 2% likely that you'll beat 23.5 weeks and only 25% likely that you'll beat 26.9 weeks.

As always, you should consider the precision of the estimates before you present them—I would normally present 24 weeks instead of 23.5 and 27 weeks instead of 26.9.

Cautions About Percentage Confident Estimates

One general pitfall with the approach I just described is that the Expected Case estimates need to be accurate—that is, they need to be truly 50% likely. You should underrun those estimates just as often as you overrun them. If you find that you're overrunning them more often than you're underrunning them, they aren't really 50% likely and you shouldn't use them as your expected cases. If the expected cases aren't accurate, then the sum of the expected cases won't be accurate either.

Chapter 9 provides suggestions for making the individual estimates more accurate.

Tip #52

Focus on making your Expected Case estimates accurate. If the individual estimates are accurate, aggregation will not create problems. If the individual estimates are not accurate, aggregation will be problematic until you find a way to make them accurate.