003: The Estimation Conundrum

Software Estimation is hard. Do it anyway.1

It's about predicting the future of a project based on incomplete information, a task that is as challenging as it is crucial. Whether you're an engineer struggling with estimation or looking to improve this skill, remember: it's a learnable skill2 that evolves with experience and practice.

As I’ve seen my teams go through the estimation scenarios, I’ve observed several different reasons why it has been hard.

Understanding the Estimation Challenges

1. Ignoring the full-picture of tasks: Often, estimations focus solely on the core development work, neglecting documentation, unit tests, and code reviews. These tasks, although not directly contributing to the product's functionality, are vital for quality and long term success.

   

2. Poor Task Breakdown:3 Large tasks need to be broken into smaller, manageable parts. Without this, we miss out on understanding the true complexity and necessary time.

3. Misunderstanding dates:4 Is it a deadline, a target, or an estimate? Confusing these leads to unrealistic planning and unnecessary pressure.

4. Estimates as Deadlines: Treating estimates as deadlines is a common mistake. Estimates are, by nature, best guesses and should be treated as flexible.

5. Estimating without Full Details: Estimating tasks without all the necessary information often leads to inaccurate projections. The less you know, the more you're guessing.

   

6. Over-Optimism or Pessimism:5 Human nature often sways estimations. Developers may be overly optimistic, underestimating the complexity, or overly pessimistic, inflating estimates due to fear of under-delivering. Balance is key. We should try avoid leaning too heavily towards either optimism or pessimism in our estimates.

7. Failing to Account for Dependencies: Not considering the time and effort needed for integrating with other systems or waiting on dependencies can lead to significant underestimation.

8. Ignoring Non-Development Tasks: Don't overlook the time required for meetings, coordination, and other essential non-coding tasks.

9. Lack of Domain Knowledge: Without a clear understanding of the business domain, estimators might not fully grasp the complexity or nuances of the tasks, leading to inaccurate estimates.

10. Poor specifications: Unclear specifications lead to guesswork and inaccurate estimations.

    

11. Estimating Under Pressure: Rushed estimates under time pressure are almost always unrealistic. Take your time to think through.

Strategies for Better Estimation

Once you understand the root of what makes estimations feel difficult for you, you can use that to get better. Here are some strategies to get better at the estimation-conundrum for the reasons I outlined above:

1. Talk to your Engineering Lead: Map their expectations with your expectations of the task that is being estimated.

2. Talk to your Product Lead: Ensure you have complete and clear specifications. Clarify ambiguities and fill in gaps.

3. Incorporate All Elements: Always include time for documentation, testing, and reviews in your estimates. These are not just add-ons but integral parts of the development process.

4. Break Down Tasks6: Use tools like JIRA epics to break larger tasks into smaller units. This helps in understanding the scope and intricacies of the work, and allows for parallelization.

   

5. Clarify the Nature of Dates: Make sure everyone is on the same page regarding what a given date represents – is it a deadline or just an estimate?

6. Estimate with Adequate Information: If forced to estimate early, be clear about the assumptions and uncertainties involved. Highlight if a design is missing, a PRD is incomplete, or if there’s pending technical debt resolution.

7. Iterative Refinement: Treat estimates as living entities. As more information becomes available or tasks evolve, refine your estimates. And call them out.

8. Include Buffer Time: Always include some buffer time to account for unforeseen complexities or interruptions.

9. Leverage Historical Data: Use data from past projects to inform your estimates. This historical perspective can provide insights into how long similar tasks have taken.

10. Collective Estimation: Involve multiple team members in the estimation process. Diverse perspectives can balance optimism and pessimism, leading to more accurate estimates.

Conclusion

Estimation is a little more than just number crunching. It is nuanced. Once you acknowledge these common pitfalls and figure out effective strategies, you can start to enhance your estimation accuracy, leading to better project planning and execution.