From a failed run to a lesson that holds.

The pipeline

1. Collect

   Coldfix imports failed CI runs, code review findings, and agent transcripts into a private store on your machine. Rerunning collection is safe: anything already stored is skipped.

2. Find the mistakes that repeat

   Failures are grouped by what went wrong, with timestamps, paths and numbers normalized away. A group becomes a candidate when it failed on at least three different commits, or in at least two repositories. Flaky failures are dropped first. A single incident can also qualify when a real repair, such as a merged pull request, already shows the fix.

3. Draft a check

   Once a day, a model drafts a rule and a check for the top new candidates from their failure evidence. A second, different model reviews each draft. A draft the reviewer rejects goes no further.

4. Prove it by replay

   The check runs on clean Git copies of the broken commit and the fixed commit. It must fail on the first with the expected failure and pass on the second. For a fix that came from a real repair, it also runs on the fixed commit with the repair's code change reversed, and must fail there too. Each run is saved in a receipt.

   An invented receipt, trimmed:

   {
     "lesson": "check-row-count",
     "expected_failure": "row-count",
     "before":   { "commit": "3f9a2c1", "status": "FAIL" },
     "after":    { "commit": "b81e04d", "status": "PASS" },
     "reversed": { "commit": "b81e04d", "status": "FAIL" },
     "gates":    { "privacy": "PASS" }
   }

5. Deliver and enforce

   A proven lesson passes a privacy screen, then lands in the repository as a rule with its check. The pre-push hook and the CI step run every delivered check, so a push that brings the mistake back is blocked and names the rule.

One repo or every repo

Proof sets a lesson's scope, not labels.

• One repository. A lesson proven in one repository applies to that repository.
• All repositories. A lesson applies everywhere only after its check is proven in at least two unrelated repositories, ones that share no root commit. Seeing the same failure in many repos is not enough on its own.

Rechecked every week

Code moves on, and a lesson that was true last month may not describe the repository today. A weekly timer reruns each delivered lesson's check against the repository's current code.

• Passes: the lesson stays.
• Reports the mistake: the lesson is withdrawn from delivery.
• Times out or breaks unexpectedly: it's reported as unchecked, not withdrawn, so a bad day on the runner can't delete good lessons.

Each recheck covers one repository. Set one up for every repository that receives lessons.

Weekly report

After each recheck, Coldfix prints the week's numbers: failures collected, how many were flaky, candidates waiting, drafts by review verdict, lessons proven, and replays rejected with their reasons. Nothing tunes itself from those numbers. They are there for you to read.