1. The Big Picture
The 30-second version
One long-running Node.js process. Inside it, three cron jobs:
| Job | When (London time) | What it does |
|---|---|---|
| Monthly creator | 00:00 on the 1st | Creates "BBC 6 Music |
| Weekly updater | 00:00 every Tuesday | Scrapes the BBC, matches songs, adds new ones |
| Daily checker | 01:00 every day | Notices missed runs and re-runs them |
Under the jobs sit five services, each with one responsibility:
flowchart TD
subgraph Process["One Node process (PM2: bbc-6music-sync)"]
S[Scheduler<br/>node-cron + a job queue]
S --> M[MonthlyPlaylistCreator]
S --> W[WeeklyPlaylistUpdater]
S --> D[DailyStatusChecker]
D -.re-runs.-> M
D -.re-runs.-> W
W -.creates month first if needed.-> M
W --> B[BBCScraper<br/>axios + cheerio]
W --> SM[SongMatcher<br/>scoring + verification]
SM --> SP[SpotifyService<br/>spotify-web-api-node]
W --> SP
M --> SP
W --> ST[StateManager<br/>JSON files]
M --> ST
W --> N[NotificationService<br/>nodemailer]
M --> N
end
B -->|HTTPS| BBC[(bbc.co.uk playlist page)]
SP -->|HTTPS| SPOT[(Spotify Web API)]
ST --> FS[(data/*.json)]
N -->|SMTP| MAIL[(Plesk mailbox)]
Everything is wired together by hand in src/index.ts:
new each service, pass it into the jobs that need it, start the scheduler.
There's no dependency-injection framework, and it doesn't need one. That style
is called constructor injection, and it's why every class can be unit tested
with fakes.
One Tuesday, end to end
Here's what happens at 00:00 on a Tuesday, in order (weeklyPlaylistUpdater.ts):
- Is there a playlist for this month? If
processing-state.jsonstill says last month, create this month's playlist first. (This exists because of a real race. See #11.) - Scrape the BBC page. You get back about 30–40
{artist, title}pairs. - Find out what's already in the playlist. Read the current playlist
live from Spotify, plus our own records for this month and last month.
This yields three sets: URIs we've seen, song identities we've seen
(
artist|title), and tombstones (songs you deleted by hand, which must never come back). - For each BBC song:
- tombstoned → skip
- identity already seen (this month or last) → skip as a duplicate
- otherwise → ask the matcher. On success, queue the URI. On ambiguity, add it to the review list.
- Add the queued URIs to the Spotify playlist, in batches of 100.
- Write the records back: month metadata (now mirroring Spotify), the week's song batch file, and the processing state.
- Email a review list if anything needs a human.
- Clean up song batch files older than 28 days.
All the thinking happens in steps 3 and 4. Everything else is plumbing.
Shape of the design, and why
| Choice | Instead of | Because |
|---|---|---|
| One process with in-process cron | System cron starting a fresh process per job | Shared state and auth in one place. One thing to deploy and watch |
| JSON files | SQLite or Postgres | A few kilobytes, one writer, written weekly. Files are readable with cat and get backed up with the folder |
| Spotify playlist as the source of truth | Our JSON as the source of truth | You edit playlists by hand. Learned the hard way (#10) |
| Flag for review when unsure | Always pick something | A wrong song in your playlist is worse than a missing one (#13) |
| Email on failure | Dashboards or monitoring | One user. Email reaches you, and a dashboard nobody looks at doesn't |
| Services behind small classes | One big script | Each part can be tested alone. The original Python version was one script (History) |
The ADRs call this "appropriate tools for the scale". The point is that boring is a feature: every dependency is something that can break at 00:00 on a Tuesday while you're asleep.
Where things live
src/
index.ts wiring, startup, graceful shutdown
config/config.ts load + validate settings.json
schedulers/ the three jobs + the Scheduler that runs them
services/ BBC, Spotify, matcher, state, notifications
utils/ dates (Luxon), logger (Winston), title/artist cleaners
tools/sendTestEmail.ts one-off "does SMTP actually work?" command
scripts/ manual harnesses (test-matcher, inspect-playlist, …)
scripts/deploy/release.sh † runs on the server: install, health check, roll back
ecosystem.config.js † PM2's definition of the process
(† = on the feat/21 branch until it merges)
.github/workflows/ ci.yml and deploy.yml
tests/unit, tests/integration, tests/deploy, tests/fixtures
data/ state (gitignored): state/, playlists/, songs/
config/settings.json secrets (gitignored, chmod 600)
Questions you might get asked
"Walk me through the architecture." It's a single Node/TypeScript daemon run by PM2 on a VPS. Inside it, node-cron triggers three jobs on London time. The jobs are thin orchestrators over five services: scraping, Spotify, matching, state and notifications, and every service is constructor-injected so it can be tested on its own. State is JSON on disk. Spotify is the real record of what's in a playlist, and we read it back every run.
"Why not serverless, like a Lambda on a schedule?" It would work, and it would be a reasonable choice. I already had a VPS, and the job needs persistent state on disk and a long-lived refresh token. A serverless version would need a bucket or a table for state and a secrets store, which means more moving parts for a job that runs once a week. If I didn't have the VPS, I'd seriously consider it.
"Why does one weekly job need a daily job?" Because the weekly job can fail: the BBC is down, Spotify is down, or the server was rebooting at midnight. The daily check is a cheap safety net. It looks at state, sees "this week's run didn't complete", and runs it again. It also runs once at startup, so a deploy at 00:00 doesn't cost a week.
"What's the riskiest part?" The matcher. Everything else either works or fails loudly. The matcher can fail quietly, by confidently picking the wrong track. Most of the bug history is about that.