#9. History and Lessons

#The timeline

#How the plans changed

The matcher's rules. The spec's priority list put "album version over single/EP" second. Real data flipped that for new releases (+500 for a self-titled single, #3), made popularity a real tiebreaker (#7), and then put a verification gate in front of everything (#13). The rules weren't wrong as a guess; they were a guess. Data replaced them.

Month targeting changed three times.

1. It started as the date the script ran, so songs published on 30 March landed in April.
2. Then the BBC's published date (#6), which was never actually read: the <time> element doesn't exist (#14).
3. Now it's the state's currentMonth, paired with the playlist ID (#10).

The lesson: a fix that "worked" was really only working because the fallback happened to give the right answer.

Source of truth. At first, our JSON was the truth and Spotify was output. Now Spotify is the truth and our JSON is a cache, plus tombstones. This is the biggest conceptual change in the project.

Deployment. The original Phase 13 was a hand-run deploy to a plain Ubuntu server. The new one is automated deploys on merge, to the real Plesk server, with release folders, a health check and rollback. Inside #21, the plan was refined again: workflow_call became workflow_run, two scripts became one release.sh, and the pm2 jlist check became a fresh log line plus a stable PID.

Logging. "pm2-logrotate will handle it" became "the app caps its own files" (#17, plus an ADR amendment).

Docs became history. The ADRs were written before the code, and several claims didn't survive. The Spotify library isn't official and doesn't refresh tokens. "Manual testing is adequate" was disproved by #9. "Zero-downtime reloads with PM2" doesn't apply to a single fork-mode process. The ADR was amended in places, and these guides note the rest.

#The lessons

These are the ones worth carrying to other projects.

1. Search results are not evidence. If a system always returns something, an empty-handed result looks exactly like a weak match. Verify the candidates before ranking them. (#13)

2. If a human can change it, don't cache it as truth. Read it back. Keep only what only you know: what you did and what was deliberately undone. (#10)

3. Compare on a key both sides agree on. Names drift between systems, but IDs don't. (#12)

4. A fallback that's usually right hides a bug that's always there. Log fallbacks loudly, or fail. The date "worked" for months while never being read. (#14)

5. Mocks don't expire, and time breaks things. Long-running processes fail in ways short scripts never do: tokens expire, logs fill disks, clocks change. Test with the clock as an input. (#16, #17)

6. Check what your dependencies actually do. One false line in an ADR ("handles token refresh automatically") was a production outage waiting to happen. (#16)

7. Test the real code path. An E2E script that reimplements the logic tests the copy, not the product. (#9)

8. Prefer "I don't know" to a confident wrong answer, and make "I don't know" cheap: a review email and 30 seconds of your time.

9. Restarts are normal, so design for them. With deploy-on-merge, every merge is a restart. That needs startup catch-up, graceful shutdown, a startup alert, and crash-loop backoff. (#19)

10. Exercise your alert paths before you need them. An untested SMTP config fails silently, so every other alert fails with it. (#25)

11. A gate that's always red isn't a gate. Fix lint before adding CI, or CI just teaches you to ignore it. (#18)

12. Environment differences are bugs waiting to happen. Locale, timezone, line endings (.gitattributes keeps *.sh as LF so Bash on Linux can run it), and the Node version (pinned in .nvmrc and engines). (#30)

13. Write the plan, then let reality edit it. Docs-first gave this project a shape and a build order. Most of the hard work came after, from real runs. Both are needed.

#On building it with an AI

This is worth having a straight answer for, because people will ask.

What worked: writing the spec, ADRs and plan first gave the AI a contract to build against. Strict TDD meant every generated change came with proof that it worked. Small PRs tied to issues with Gherkin criteria kept each change reviewable.

What to watch: the AI-written ADRs stated library features with total confidence, and some were false (#16). Code review before go-live found four blockers that hundreds of passing tests hadn't. The fix wasn't to trust it less in general. It was to verify claims about the outside world and to review with production in mind: what happens after an hour, a month, a restart, a clock change.

If you're asked "how much of this do you understand?", these guides are the answer. Go through the "Questions you might get asked" sections until you can answer them without notes.

#Questions you might get asked

"What was the hardest bug?" Duplicate and removal detection: four fixes (#5, #8, #9, #12) before it held. It was hard because it sits between two systems that name things differently, and a human edits one of them.

"What would you do differently?" I'd treat the Spotify playlist as the source of truth from day one, verify match candidates from day one, and test long-running behaviour (token expiry, log growth) before the first deploy, not after review.

"What's next?" Merge CD, bootstrap the server, cut over, and validate for a month. Then fix #15 (outage recovery), #14 (date parsing) and the flaws in guide 10.