Initech's support team files bugs straight into the coding agent's queue, most of them a single line saying something is broken. Your call: do reporters fill in steps, the error and the expected result before the agent starts, or does the agent ask?
Key idea: Told to keep going, a coding agent given a vague bug report picks the likeliest fix and ships it, and that fix sometimes hides the symptom instead of fixing the cause. Steps, the error message and the expected result make the cause findable, so the bug report is the agent's spec.
Finish a run to see what a PM takes away from this lesson.
Bugs fixed vs. words in the reports
Fix the bugs to plot the run here
Things to try
- Run it as it starts: Smart, It's broken. How many of the 8 bugs did the agent fix, and how many of its fixes only hide what the reporter saw?
- Switch to Steps to reproduce and run it again. Which bugs did the steps settle?
- Switch to Full report (steps, the error, and what should have happened). What changed?
- Go back to It's broken and turn on Let the agent ask first. Read its questions in the log. Which ones paid off?
- Turn off Let the agent ask first, switch the agent to Fast and run It's broken. Does the smaller model guess better?