docs: add preflight tests concept and built-in check reference - #572
Conversation
Introduces preflight tests as a first-class concept in game-ci: - New page covering concept, 13 built-in checks, configuration, CLI usage, extensibility, and continuous improvement cycle - Cross-links from test-workflow-engine and build-reliability pages Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
Caution Review failedThe pull request is closed. ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (2)
📝 WalkthroughWalkthroughAdds a new "Preflight Tests" documentation page describing the pre-build, fail-fast validation stage (configuration, built-in/custom checks, CLI usage, design principles, and inputs), and adds cross-references from Test Workflow Engine and Build Reliability to the new page. ChangesPreflight Tests Documentation
Estimated Code Review Effort🎯 2 (Simple) | ⏱️ ~10 minutes Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Warning There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure. 🔧 ESLint
docs/03-github-orchestrator/07-advanced-topics/11-test-workflow-engine.mdxOops! Something went wrong! :( ESLint: 10.4.0 ESLint couldn't find an eslint.config.(js|mjs|cjs) file. From ESLint v9.0.0, the default configuration file is now eslint.config.js. https://eslint.org/docs/latest/use/configure/migration-guide If you still have problems after following the migration guide, please stop by docs/03-github-orchestrator/07-advanced-topics/13-preflight-tests.mdxOops! Something went wrong! :( ESLint: 10.4.0 ESLint couldn't find an eslint.config.(js|mjs|cjs) file. From ESLint v9.0.0, the default configuration file is now eslint.config.js. https://eslint.org/docs/latest/use/configure/migration-guide If you still have problems after following the migration guide, please stop by Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
docs/03-github-orchestrator/07-advanced-topics/13-preflight-tests.mdx (1)
276-279: 💤 Low valueMinor: Design principle timing may need clarification.
Line 278 states each check completes "in under 60 seconds," but line 125 shows compilation checks can take "30–60s," which technically allows checks to hit exactly 60 seconds. Consider revising to "in 60 seconds or less" for precision, or "well under 60 seconds" if the intent is stricter.
📝 Optional wording clarification
-- **Fast.** Each check completes in under 60 seconds. The full suite completes in under 5 minutes. +- **Fast.** Each check completes in 60 seconds or less. The full suite completes in under 5 minutes.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/03-github-orchestrator/07-advanced-topics/13-preflight-tests.mdx` around lines 276 - 279, The wording under the "Design Principles" heading currently says "Each check completes in under 60 seconds" which conflicts with the later note that compilation checks can take "30–60s"; update the phrase to be precise and consistent—e.g., change "in under 60 seconds" to "in 60 seconds or less" or to "well under 60 seconds" depending on the intended strictness, and ensure the change aligns with the "compilation checks" line so both statements convey the same timing expectation.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/03-github-orchestrator/07-advanced-topics/13-preflight-tests.mdx`:
- Around line 306-313: The Inputs Reference table documents inputs
(preflightSuite, preflightEnabled, preflightFailFast, preflightTimeout) that are
not present in the orchestrator/action schema; either remove this table or
update it to match the real implementation. Inspect the orchestrator and
unity-builder sources (search for the exact input names preflightSuite,
preflightEnabled, preflightFailFast, preflightTimeout) to confirm whether these
inputs exist, were removed, or are planned; if they don't exist, delete the
table and replace with a note stating preflight inputs are not supported, or if
they are planned/legacy, add a clear remark indicating their status and link to
the correct implementation or roadmap.
---
Nitpick comments:
In `@docs/03-github-orchestrator/07-advanced-topics/13-preflight-tests.mdx`:
- Around line 276-279: The wording under the "Design Principles" heading
currently says "Each check completes in under 60 seconds" which conflicts with
the later note that compilation checks can take "30–60s"; update the phrase to
be precise and consistent—e.g., change "in under 60 seconds" to "in 60 seconds
or less" or to "well under 60 seconds" depending on the intended strictness, and
ensure the change aligns with the "compilation checks" line so both statements
convey the same timing expectation.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 73907a6f-9c01-47a3-90a4-99f0401d7ba8
📒 Files selected for processing (3)
docs/03-github-orchestrator/07-advanced-topics/11-test-workflow-engine.mdxdocs/03-github-orchestrator/07-advanced-topics/13-preflight-tests.mdxdocs/03-github-orchestrator/07-advanced-topics/17-build-reliability.mdx
| ## Inputs Reference | ||
|
|
||
| | Input | Description | Default | | ||
| | --- | --- | --- | | ||
| | `preflightSuite` | Path to preflight suite YAML file | `''` | | ||
| | `preflightEnabled` | Enable the preflight stage | `'true'` when suite is set | | ||
| | `preflightFailFast` | Abort on first check failure | `'true'` | | ||
| | `preflightTimeout` | Maximum total preflight duration in seconds | `300` | |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Description: Find the orchestrator configuration schema or input definitions
# Search for orchestrator input/config schema
echo "=== Searching for preflight configuration schema ==="
rg -n --type=yaml --type=ts --type=json -C5 'preflightSuite|preflightEnabled|preflightFailFast|preflightTimeout'
# Look for JSON schema or TypeScript interfaces defining config
echo -e "\n=== Searching for config type definitions ==="
ast-grep --pattern 'interface $_ {
$$$
preflight$_: $_
$$$
}'
# Search for default values
echo -e "\n=== Searching for default value definitions ==="
rg -n --type=ts --type=js 'preflightTimeout.*300|preflightFailFast.*true'Repository: game-ci/documentation
Length of output: 214
🏁 Script executed:
#!/bin/bash
# Find references to the orchestrator or its repository
echo "=== Searching for orchestrator repository references ==="
rg -n "orchestrator" docs/ | head -20
echo -e "\n=== Checking git submodules ==="
cat .gitmodules 2>/dev/null || echo "No .gitmodules file"
echo -e "\n=== Looking for orchestrator links or URLs in docs ==="
rg -n "github\.com.*orchestrator" docs/ | head -10
echo -e "\n=== Checking directory structure ==="
ls -la | grep -i orchestrator
find . -maxdepth 3 -type d -name "*orchestrator*" 2>/dev/nullRepository: game-ci/documentation
Length of output: 4585
🏁 Script executed:
#!/bin/bash
# Examine the file under review
echo "=== Full content of preflight-tests.mdx ==="
cat -n docs/03-github-orchestrator/07-advanced-topics/13-preflight-tests.mdx | head -350 | tail -100
echo -e "\n=== Searching for examples of preflight inputs in docs ==="
rg -n "preflightSuite|preflightEnabled|preflightFailFast|preflightTimeout" docs/ -B2 -A2Repository: game-ci/documentation
Length of output: 5458
🏁 Script executed:
#!/bin/bash
# Look for examples showing these inputs in use
echo "=== Searching for configuration examples with preflight inputs ==="
rg -n "preflightEnabled|preflightSuite|preflightFailFast|preflightTimeout" docs/ --context 10 | head -60
# Search for workflow files that might use orchestrator
echo -e "\n=== Looking for orchestrator workflow examples ==="
find docs -name "*.yml" -o -name "*.yaml" | xargs grep -l "preflight" 2>/dev/nullRepository: game-ci/documentation
Length of output: 1870
🌐 Web query:
game-ci orchestrator preflight configuration inputs schema
💡 Result:
GameCI's Orchestrator does not have a published JSON schema or formal schema for preflight configuration inputs. Instead, configuration is provided via CLI flags, environment variables, GitHub Action with inputs, or overrides.[1][2][3] The primary configuration schema is defined in the unity-builder GitHub Action's action.yml file, which Orchestrator uses.[2][4] Key inputs include: - targetPlatform (required): Platform to build (e.g., StandaloneLinux64, WebGL).[5][6] - unityVersion (default: 'auto'): Unity version from ProjectSettings/ProjectVersion.txt or specified.[5] - projectPath: Relative path to Unity project.[6] - buildMethod: Namespace.Class.StaticMethod for custom builds.[5][6] - customParameters: Additional build parameters.[5][6] - versioning (default: 'None'): Semantic, etc.[5][6] - providerStrategy: For Orchestrator (aws, k8s, local-docker, local).[1][7] Orchestrator-specific CLI parameters (from API Reference):[1] - orchestratorBranch (default: main): main, orchestrator-develop. - cloneDepth - providerStrategy [as above] - readInputOverrideCommand: For secrets (e.g., gcp-secret-manager).[8] - readInputFromOverrideList: Comma-separated params to override.[8] - Modes: cli-build (default), remote-cli-pre-build, remote-cli-post-build, cache commands.[1][9] Full list in https://github.com/game-ci/unity-builder/blob/main/action.yml [2] and docs API reference [1]. No explicit "preflight" mode/schema found; likely refers to pre-build setup or license validation in workflows.[9] Use yarn run cli --help for CLI param descriptions.[3]
Citations:
- 1: https://game.ci/docs/github-orchestrator/api-reference
- 2: https://github.com/game-ci/unity-builder/blob/main/action.yml
- 3: https://game.ci/docs/github-orchestrator/examples/command-line
- 4: https://github.com/game-ci/unity-builder
- 5: https://game.ci/docs/github/builder
- 6: https://github.com/Cysharp/Actions/blob/main/.github/actions/unity-builder/action.yaml
- 7: https://game.ci/docs/github-orchestrator/getting-started
- 8: https://game.ci/docs/github-orchestrator/advanced-topics/configuration-override
- 9: https://game.ci/docs/github-orchestrator/cli/other-commands/
🌐 Web query:
site:github.com game-ci/unity-builder action.yml preflightSuite preflightEnabled
💡 Result:
No evidence found in the game-ci/unity-builder action.yml or official documentation for inputs named "preflightSuite" or "preflightEnabled". [1][2] The action.yml defines standard inputs like targetPlatform, unityVersion, projectPath, buildMethod, customParameters, versioning, buildProfile (added recently [3]), and recent orchestrator options like runnerCheckEnabled, providerStrategy, fallbackProviderStrategy (from PRs [4][5]). [1][2] "Preflight" appears in unrelated third-party projects (e.g., unity-cross-platform-local-build-orchestrator [6], unityctl [7]) and recent commits mention testSuitePath/testSuiteEvent for test workflows in orchestrator mode [5], but not matching the exact terms in unity-builder. GameCI docs recommend checking https://game.ci/docs/github/builder/ for full input list, which lists common options without preflightSuite/preflightEnabled. [2][8]
Citations:
- 1: https://github.com/game-ci/unity-builder/blob/main/action.yml
- 2: https://github.com/game-ci/documentation/blob/main/docs/03-github/04-builder.mdx
- 3: feat: add
buildProfileparameter unity-builder#685 - 4: LTS 2.0.0 -- Combined Release unity-builder#815
- 5: game-ci/unity-builder@564f590
- 6: https://github.com/sinanata/unity-cross-platform-local-build-orchestrator
- 7: https://github.com/Jason-hub-star/unityctl
- 8: https://github.com/game-ci/unity-builder/blob/main/README.md
🌐 Web query:
site:github.com game-ci/orchestrator preflight configuration inputs
💡 Result:
game-ci/orchestrator is a tool for remote build orchestration of game projects (primarily Unity, with plugins for others like Godot/Unreal), supporting providers like AWS Fargate (--providerStrategy aws), Kubernetes (k8s), local Docker (local-docker), GitHub Actions (github-actions), and more [1][2]. Configuration inputs are primarily CLI flags for the game-ci build or game-ci orchestrate commands, environment variables, or game-ci.yml files [3]. No explicit "preflight" configuration inputs or dedicated preflight mode are documented in the official README or related sources [1][2]. Builds likely include implicit validation (e.g., project path, credentials for cloud providers like AWS_ACCESS_KEY_ID [4]), but no specific preflight flags like --preflight or preflight config section are mentioned. Key configuration inputs from examples: - --providerStrategy (or --provider-strategy): e.g., aws, local-docker [1][2] - --projectPath: Path to the project [1] - --targetPlatform: e.g., StandaloneLinux64 [1] - --engine: e.g., godot (for plugins) [1] - --custom-job: For custom workflows [1] - Provider-specific: e.g., orchestratorCpu, orchestratorMemory for AWS Fargate (in YAML for GitHub Actions) [4] - Env vars: GAME_CI_VERSION, AWS credentials [1][4] For full CLI usage, run game-ci --help after installation [1]. Official docs at game-ci/documentation cover orchestrator setup, but no preflight details found [5][4][6]. "Preflight" appears in unrelated projects' contexts (e.g., validation checks) [7].
Citations:
- 1: https://github.com/game-ci/orchestrator/blob/main/README.md
- 2: https://github.com/game-ci/orchestrator
- 3: Official CLI Support — game-ci unity-builder#812
- 4: https://github.com/game-ci/documentation/blob/main/docs/03-github-orchestrator/03-examples/02-github-examples/02-aws.mdx
- 5: https://www.github.com/game-ci/documentation
- 6: https://github.com/game-ci/documentation/blob/main/versioned_docs/version-3/03-github-orchestrator/01-introduction.mdx
- 7: https://github.com/sinanata/unity-cross-platform-local-build-orchestrator
Remove inputs reference table or verify orchestrator implementation for preflight inputs.
The documented inputs (preflightSuite, preflightEnabled, preflightFailFast, preflightTimeout) do not appear in the official orchestrator schema. Web search of the unity-builder action.yml and orchestrator README found no evidence of these inputs being implemented. The table at lines 306-313 documents configuration that users cannot actually use.
Clarify whether:
- These inputs are planned but not yet implemented
- They were removed in a recent version
- The documentation refers to a different tool or legacy implementation
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/03-github-orchestrator/07-advanced-topics/13-preflight-tests.mdx` around
lines 306 - 313, The Inputs Reference table documents inputs (preflightSuite,
preflightEnabled, preflightFailFast, preflightTimeout) that are not present in
the orchestrator/action schema; either remove this table or update it to match
the real implementation. Inspect the orchestrator and unity-builder sources
(search for the exact input names preflightSuite, preflightEnabled,
preflightFailFast, preflightTimeout) to confirm whether these inputs exist, were
removed, or are planned; if they don't exist, delete the table and replace with
a note stating preflight inputs are not supported, or if they are
planned/legacy, add a clear remark indicating their status and link to the
correct implementation or roadmap.
|
|
|
Visit the preview URL for this PR (updated for commit 121882e): https://game-ci-5559f--pr572-docs-preflight-tests-ox7whc26.web.app (expires Mon, 25 May 2026 22:44:10 GMT) 🔥 via Firebase Hosting GitHub Action 🌎 Sign: 1f0574f15f83e11bfc148eae8646486a6d0e078b |
Summary
Test plan
🤖 Generated with Claude Code
Summary by CodeRabbit