Skip to content

Unbreak CI: PHPStan false positive and matrix fail-fast - #974

Merged
masteradhoc merged 3 commits into
masterfrom
fix/unbreak-ci
Sep 8, 2026
Merged

masteradhoc merged 3 commits into
masterfrom
fix/unbreak-ci

Conversation

@georgestephanis

@georgestephanis georgestephanis commented Sep 8, 2026 •

Copy link
Copy Markdown
Collaborator

Why

Every pull request currently looks broken, and for reasons that have nothing to do with the code being proposed. This fixes the workflow itself.

PHPStan was the fourth cause, but #972 handles that one properly — by upgrading PHPStan and dropping includes/ from the analysed paths, matching the */includes/* exclusion that already exists in phpcs.xml.dist. That's a better fix than the scoped ignore this PR originally carried, so it has been removed here. This PR is now purely workflow changes; #972 keeps the PHPStan fix.

The two are complementary. #972 can't go green on its own — its only failing leg is Test PHP 7.4, with the other 27 cancelled, which is exactly what's fixed below. Landing this first should unblock it.

What

1. One failing leg no longer cancels the other 27

The matrix ran with the default fail-fast: true. A single failure cancelled every other job, so 28 jobs reported as 1 failure + 27 cancelled and nothing usable came back. Now fail-fast: false.

2. PHP 7.4 and 8.0 removed from the matrix

These cannot be run any more, and it isn't a transient breakage.

wp-env builds its container from wordpress:php<version>, and:

Tag Last rebuilt Base
wordpress:php7.4-fpm 2022-11-16 bullseye
wordpress:php8.0-fpm 2023-11-22 bullseye
wordpress:php8.1-fpm 2026-01-01 bookworm
wordpress:php8.4-fpm 2026-08-31 bookworm

Debian bullseye LTS ended 2026-08-31. Its security Release file is now expired, so the apt-get -qy update in wp-env's generated Dockerfile exits 100 before a single test runs. No bookworm image exists for 7.4 or 8.0 — both reached EOL before bookworm shipped — and wp-env hardcodes both the base image (lib/runtime/docker/docker-config.js:83) and that apt call (line 167), with no configuration hook for either. Verified against wp-env 11.14.0; this repo pins ^10.30.0.

8.1 is bookworm and passes, so it becomes the lowest version actually exercised.

Note: PHP 7.4 is still the declared minimum (Requires PHP in two-factor.php, >=7.4 in composer.json). It stays covered statically by PHPCompatibilityWP via testVersion 7.4-, but it is no longer exercised at runtime. Whether to raise the floor to 8.1 is a support-policy decision and deliberately isn't made here.

3. The suite ran twice for every pull request

on: [push, pull_request] fired both events for any push to a branch with an open PR. The concurrency group can't collapse them:

group: ${{ github.workflow }}-${{ github.event_name == 'pull_request' && github.head_ref || github.sha }}

Pull requests key on branch name, everything else on commit SHA — so the two runs land in different groups and neither cancels the other, and the push key is unique per push so it never matches anything at all. The result was two full 20-job matrices per push.

push is now scoped to master, leaving pull requests covered by pull_request alone while still running the suite on merges. Manual re-runs are unaffected.

It demonstrated itself on its own commit:

Commit Events fired
0b87b39 (matrix change) push + pull_request
b835d49 (trigger fix) pull_request only

Testing

Full matrix green before the PHPStan commit was dropped: 23/23 jobs passing — 20 test legs (PHP 8.1–8.5 × 4 WordPress versions), Lint PHP & PHP Compatibility, Lint JS & CSS, Build.

This revision only removes a phpstan.dist.neon hunk; git diff origin/master..HEAD now touches .github/workflows/test.yml and nothing else.

@github-actions

github-actions Bot commented Sep 8, 2026 •

Copy link
Copy Markdown

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message.

Co-authored-by: georgestephanis <georgestephanis@git.wordpress.org>
Co-authored-by: masteradhoc <masteradhoc@git.wordpress.org>

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

@jeffpaul jeffpaul added this to the 0.17.0 milestone Sep 8, 2026
@jeffpaul
jeffpaul requested a review from kasparsd September 8, 2026 15:49
georgestephanis and others added 3 commits September 8, 2026 13:06
The test matrix runs with the default fail-fast, so a single failure
cancels every other job in the run. Right now the PHP 7.4 and 8.0 images
are Debian bullseye based, and bullseye's security suite reached end of
life on 2026-08-31, so `apt-get update` fails while building the wp-env
container. One dead leg was taking the whole matrix with it and making
every pull request look broken.

Turning fail-fast off means a genuine failure on one PHP version no
longer hides the results for the other nineteen.
These two legs cannot be run any more, and the cause is permanent rather
than a transient upstream hiccup.

wp-env builds from `wordpress:php<version>`. The php7.4 tag was last
rebuilt in November 2022 and php8.0 in November 2023, both permanently
pinned to Debian bullseye. Bullseye LTS ended 2026-08-31, so its
security Release file has now expired, and the `apt-get -qy update` that
wp-env puts in its generated Dockerfile exits 100 before a single test
runs. There is no bookworm image for either PHP version -- both were
already end-of-life before bookworm shipped -- and wp-env hardcodes the
base image and the apt call with no configuration hook, so there is
nothing to work around from this repository.

PHP 7.4 remains the declared minimum and is still covered statically by
PHPCompatibilityWP through `testVersion 7.4-`, but it is no longer
exercised at runtime; 8.1 is now the lowest version the suite runs
against. That gap is called out in a comment beside the matrix so it is
visible to whoever revisits the supported-versions question.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`on: [push, pull_request]` meant a push to a branch with an open PR
triggered the whole matrix twice. The concurrency group cannot collapse
the pair: it keys pull requests on the branch name and everything else
on the commit hash, so the two runs land in different groups and both
run to completion. A push group keyed on the commit hash also never
matches a previous run, since every push has a new hash.

Limiting `push` to master leaves pull requests covered by the
`pull_request` event alone and still runs the suite on merges into the
default branch. Halves the CI minutes per PR, and halves the exposure to
transient runner failures -- one of which (a codeload.github.com fetch
of the setup-php action) turned up on this very branch, failing one leg
in one of the duplicate runs while the identical leg passed in the
other.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@georgestephanis

Copy link
Copy Markdown
Collaborator Author

Dropped the PHPStan commit — #972 fixes that error better than this did.

Brian's approach removes includes/ from PHPStan's paths: entirely, which lines up with the */includes/* exclusion already in phpcs.xml.dist:45. My scoped ignoreErrors entry cited that same precedent as a reason to go narrower, which reads backwards now — the precedent supports the directory-level exclusion. #972 also bumps PHPStan 1.12.34 → 2.2.13, removing the stale-version warning rather than working around it.

For the record, I checked whether the two would have conflicted if both landed: git merge-tree auto-merges cleanly, and running the merged config produced [OK] No errors rather than an unmatched-ignore failure. So they'd have coexisted — my entry would just have been dead config pointing at an unanalysed file. Cleaner to drop it.

This PR is now workflow-only: fail-fast: false, the EOL matrix legs, and the duplicate-run trigger. git diff origin/master..HEAD touches .github/workflows/test.yml and nothing else.

@masteradhoc masteradhoc left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@masteradhoc
masteradhoc merged commit e3546ac into master Sep 8, 2026
45 of 48 checks passed
@masteradhoc
masteradhoc deleted the fix/unbreak-ci branch September 8, 2026 17:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants