W19: Finding Bugs with Pi: Reproduce First

A good doctor examines you and runs a test before treating. A bad doctor hears "my stomach hurts" and hands out the first pill that comes to mind. AI models, left alone, are often the bad doctor: they read a bug report, guess a likely cause, and write a confident fix for it. This lesson teaches the good doctor's order, reproduce first, and how to make Pi follow it.

Understand Before You Fix

R03: Problem Solving says most software work is thinking, not typing: understand, plan, then implement. For a bug, "understand" has a very concrete meaning: see it happen, on purpose, every time. Until you can make the bug appear when you want, you do not know its cause. You only have a guess.

A True Story

This happened to Pi Workbench itself, in October 2026. A person reported: "Coming back to Chat from Activity leaves the view lifted, white at the bottom."

Pi read the code and guessed a cause: the chat's scroll position. It changed two functions and added a test. Its answer sounded finished.

The guess was wrong. The real cause was elsewhere: only between 761 and 1100 pixels wide, a hidden line of text for screen readers was placed at the very bottom of the whole page, which made the background scrollable. Any scroll then lifted every window. And the test Pi wrote clicked a button that did not exist on that screen, so it waited forever, with or without the fix.

The real cause was found by looking, not reasoning: open the workbench at 1024 pixels wide, send a message, see the window lift, measure which element had scrolled and what it was placed against. Four measurements.

The harness's own lesson file (L-008) lists what Pi lacked: a browser it could drive, the conditions (the width), a rule to reproduce first, and facts instead of guesses.

Report the Conditions

The report did not mention the width. People rarely think to. A good bug report answers:

Reproduce, Then Fix

Ask for the order explicitly:

"Bug: on my phone, the Delete button on a note deletes the note below it. Reproduce it first with a failing test, show me the test failing, then fix it."

What happens then is the harness's Red and Green (W03), used for its best purpose:

flowchart LR R[Report with conditions] --> T[A test that does what the report says] T --> F[It fails: the bug is real and caught] F --> C[Fix the cause] C --> P[The same test passes] P --> V[Verified and committed]

A test that reproduces the bug is a small proof: this was broken, now it is not, and it will stay fixed, because the test runs every time from now on.

Watch for the trap from the story: a test that fails for the wrong reason (a timeout, a missing button) still counts as "failed". Read the test. Does it do what the report says?

The Help the Harness Gives

When You Are Stuck

This Week's Exercise

  1. Read R03. Write its three steps in your own words, for bugs.
  2. Break something small in your app on purpose (or find a real bug). Write a report with all five answers above.
  3. Send it to Pi with "reproduce it first with a failing test". Read the test before the fix: does it do what your report says?
  4. Find the task file and check it has a Red line before the Green line.

Key Takeaways

  • Reproduce first: until you can make the bug appear on purpose, its cause is only a guess
  • Report the conditions: steps, expected, actual, where (device, browser, width), every time or sometimes
  • A failing test that does what the report says is the proof; the same test passing is the fix
  • A test can fail for the wrong reason: read it
  • The harness adds a bug guide, gives Pi a browser for page problems, and proves Red itself when Pi skipped it