Estimating the cost of developing software is difficult. Very difficult. That’s because with each new project there are always unknowns, mis-interpretations and assumptions that are not communicated. If the client knew exactly what they wanted then it would just be a matter of typing it up. They hire you because they don’t know how to do it themselves.
Agile development realizes how difficult estimations are by attempting to avoid them entirely. Instead opting to push clients to time and materials which they have a hard time fitting into a fixed annual budgets.
Creating an estimate that accounts for uncertainty while fitting in with businesses’ existing processes is non-trivial.
It all starts by taking the project scope and breaking it into smaller pieces that are easier to estimate effort on. There are risks even at this early first step that your estimate may be off. Even with a great deal of thought you will likely overlook things.
Agile approaches would have you break a project up into ‘stories’ and estimate on those. I have found that stories generally have some overlap, making it difficult to understand as a line item in an estimate. I prefer to get down to the level of code structure.
The second thing to consider in an estimate is that with every guess you make there is some understanding of uncertainty that is not often portrayed. Creating a login page is fairly concrete, you’ve done it 50 times and are very sure how long it will take. Creating a custom machine learning algorithm for the application may be a bit more fuzzy. Providing a measure of uncertainty to your estimate turns your fixed estimate into a range of estimates with different probabilities.
Actually, a machine learning algorithm is useful example to explain my next point. A good ML algorithm can take time to train and tune and a bad one can be quick. You can take more or less time and still get an algorithm that works. There is not only an uncertainty but also some elasticity since it is unlikely that an RFP (Request for Proposals) included stipulations on the accuracy of the ML feature. Elastic features should exist as 2 options on an estimate. 1 for basic implementation and an other that provides a better implementation. That way the advanced option can be explicitly accepted by the client.
Next thing to consider is that not everyone’s estimates are equally good. Some people are overly optimistic others tend to assume worst case, sometimes you have better knowledge that can help cut down the estimate while another person considers things more difficult. By collecting estimates from many people you can average out their biases and with an open dialog hopefully team members can learn from each other.
This brings up a couple of major factors that can influence a person’s estimate:
- Anchoring bias – A person’s estimate can be influenced if they see someone else’s estimate first. You mitigate this by having estimates done blind.
- Optimism bias – People tend to want to impress. It may be to stand out from the team as better, or because they haven’t thought of some of the complexity in a solution. There is a tendency to under estimate.
- Lack of information – All estimates are made without complete knowledge. The client likely doesn’t have all the information either. When you don’t have all the information there is more uncertainty in an estimate.
Anchoring bias can be dealt with through procedures that make estimates independent of each other. Games like Planning Poker have rules to prevent anchoring bias (everyone turns over their card at the same time).
Optimism bias is more difficult to account for. Perhaps the best (though as of yet untested in software development) approach is to use reference class forecasting to provide some insight given historical realized costs on similar tasks. If setting up a new server has for the last 20 projects taken an average of 2 days, but the team has estimated it will take 1 day then they may be some optimism bias in the estimate.
Lack of information is accounted for by adding uncertainty to the estimate. Although we don’t always know what we don’t know so some amount of unknowns should usually be added as a buffer in the final estimate.
Using this information hopefully you can create better more realistic estimates for your clients. If you are a client accepting bids on an RFP you should be looking for bids that detail where there is uncertainty.
