The day my tests started running on every pull request was the day they actually started catching things. A suite you have to remember to run is a suite you forget to run.
The Minimum Useful Pipeline
Three stages, in order, fail fast:
- Lint — cheap, catches typos and obvious mistakes.
- Unit — fast, runs in seconds.
- E2E — slower, runs on a real browser.
If lint fails, there's no point spending three minutes on E2E.
A Real Workflow
name: tests
on: [push, pull_request]
jobs:
e2e:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
- run: npm ci
- run: npx playwright install --with-deps
- run: npx playwright test
- uses: actions/upload-artifact@v4
if: failure()
with:
name: playwright-report
path: playwright-report/
The if: failure() Line Is the Important One
When a test fails in CI, you can't open the browser to see what happened. Uploading the report, trace, and screenshots as artifacts on failure means you can download exactly what the runner saw. Without this, CI failures become guesswork.
Make It a Required Check
A pipeline that's allowed to be red is decoration. I mark the test job as a required status check so a PR can't merge while tests fail. That's the line between "we have automation" and "automation actually protects the branch."
Keep It Fast or People Route Around It
If the pipeline takes 20 minutes, developers start merging without waiting. I parallelize across shards and run only smoke tests on every push, saving the full suite for the merge to main. Speed is what keeps the gate respected.