A Scientific Approach to Debugging


This list, taken from John Regehr’s How to Debug (via), aligns nicely with my process for debugging software.

  1. Verify the Bug and Determine Correct Behaviour.
  2. Stabilize, Isolate, and [Reproduce].
  3. Estimate [Likelihood of Probable Causes].
  4. Devise and Run an Experiment.
  5. Iterate Until the Bug is Found.
  6. Fix the Bug and Verify the Fix.
  7. Undo [Unwanted] Changes.
  8. Create a Regression Test.
  9. Find the Bug’s Friends and Relatives.

My changes/clarifications are shown within square braces. Trust me, read the full post. Each step is explained in detail.

I’ve just finished re-reading Robert Pirsig’s Zen and the Art of Motorcycle Maintenance, where a similar debugging approach (the scientific method) is explained.

“For this you keep a lab notebook. Everything gets written down, formally, so that you know at all times where you are, where you’ve been, where you’re going and where you want to get. […] Sometimes just the act of writing down the problems straightens out your head as to what they really are.

The logical statements entered into the notebook are broken down into six categories:

  1. Statement of the problem.
  2. Hypotheses as to the cause of the problem.
  3. Experiments designed to test each hypothesis.
  4. Predicted results of the experiments.
  5. Observed results of the experiments
  6. Conclusions from the results of the experiments.

“The real purpose of scientific method is to make sure Nature hasn’t misled you into thinking you know something you don’t actually know. […] One logical slip and an entire scientific edifice comes tumbling down. One false deduction […] and you can get hung up indefinitely.

As a programming instructor I help students when things go wrong with the applications they’re writing. Because of my experience with the types of apps they are coding, I’m usually quick at spotting the source of the bug. Pointing out these bugs may fix their immediate problem, but it does little to correct the error in logic that led to the bug. Perhaps before coming to me, students with buggy source code should work through steps 1 through 4 from either of these lists.

If I were to mandate this for my classes, I’d have to add a step zero:

Be Prepared to Learn From Your Mistake.

New programmers are often quick to blame the language or framework they are using when their code doesn’t work. I still catch myself thinking “IT isn’t working” when I encounter troublesome bugs in my code. However, experience has shown me two things:

  1. My tools are rarely the source of my problem.
  2. If I’m not prepared to accept responsibility for my bugs, I’m doomed to repeat them.


Recommended reading:

2013-03-09 12:58:00


I first read this book thirteen years ago while backpacking in Fiji. I just finished a re-read, this time from the perspective of an educator rather than that of a new grad.

Next up is a re-read of the sequel Lila. When I first read Lila I got trapped for hours (days!) thinking about the evolution of consciousness. The end result was the essay From Instincts to Consciousness I wrote in November 2003.