Sunk Costs and Handling Bugs

When software quality is broadly defined in an organization (as it should be), the software testing team is likely to generate large quantities and varieties of bugs. There is a tendency among some organizations to close too many of these bugs through additional development. While this is an intuitive approach, handling bugs is subject to the same cost-benefit analysis of any other business initiative. When this cost-benefit is assessed subconsciously, qualitatively, and/or without proper assessment, the analysis can be distorted by the sunk cost bias/fallacy/trap.

The sunk cost bias is a pervasive failure in reasoning, based on an inaccurate accounting of costs. A typical example is that of attending a movie. In this example, you decide to attend a movie, since the enjoyment of the movie outweighs the cost to watch the movie (let’s say 15 USD for a ticket). You buy the movie ticket, and then walk around the surrounding mall while you wait for the movie. Unfortunately, the ticket is lost during your stroll through the mall. With the movie now about to start, you realize that the ticket has been lost and are faced with a choice. You can buy another ticket - for another 15 USD - or abandon the movie altogether. It may be tempting to see this cost-benefit analysis as “movie enjoyment” vs. 30 USD. However, the initial movie ticket is a sunk cost, and is already spent whether or not you see the movie. As such, a logical analysis would put the cost-benefit at “movie enjoyment” vs. 15 USD. This is the same cost-benefit you were faced with in the purchase of the initial ticket, so unless the mall stroll has substantially changed your expectations for the movie or belief about of the intrinsic worth of 15 USD (both being unlikely), the logical choice would be the purchase of another ticket for an additional 15 USD.

If this example didn’t hit the mark for you, imagine the experience once you are inside the theater (and forget the previous ticket pricing discussion). I am not aware of any theaters that give refunds to customers who didn’t enjoy the movie. As such, your ticket price is a sunk cost once you are in the theater. Your cost-benefit analysis is then “movie enjoyment” from the remainder of the movie vs. 0 USD (or perhaps enjoyment of the next best activity, such as the mall’s lazer tag, if you consider opportunity costs). If you don’t enjoy the movie, or would enjoy lazer tag more, the logical choice would be to stand up and leave the theater mid-film; the earlier ticket price costs are irrelevant sunk costs, at this point in your activities.

In software quality assurance, bug reports are a sunk cost. There is no way to recover the time and financial investment involved in the creation of the report. The cost is incurred whether or not the bug is handled. As such, deciding whether to invest in a bug fix should not take the bug report creation costs into account. It can be difficult to fight this bias. You may hear things like “we’ve already spent so much time in writing the report and collecting the associated environment details, stack traces, logs, screenshots….” While pointing out someone’s cognitive bias and irrational thinking is not recommended, being able to separate the sunk costs from the real costs can help you to lead the organization to better, more logical decisions about development priorities.