Skip to content

test(orchestrator): further unit test coverage - #784

Closed
frostebite wants to merge 5 commits into
mainfrom
feature/orchestrator-unit-tests
Closed

test(orchestrator): further unit test coverage#784
frostebite wants to merge 5 commits into
mainfrom
feature/orchestrator-unit-tests

Conversation

@frostebite

@frostebite frostebite commented Mar 5, 2026

Copy link
Copy Markdown
Member

Summary

Adds 64 new mock-based unit tests for orchestrator services that previously had zero test coverage. These tests run without any external infrastructure and expand the safety net for PRs.

New test files

File Tests What it covers
task-parameter-serializer.test.ts 16 Env var format conversion (ToEnvVarFormat, UndoEnvVarFormat), round-trip fidelity, uniqBy deduplication, blocked parameter filtering, default secret reading
follow-log-stream-service.test.ts 15 Build output parsing — end-of-transmission detection, Build succeeded/failed, Library rebuild, error accumulation patterns (error , error:, command failed:, invalid, cannot be found)
orchestrator-guid.test.ts 8 GUID generation format, standalone prefix stripping, platform lowercasing, nanoid uniqueness and character set
orchestrator-folders.test.ts 25 All path computation getters, ToLinuxFolder slash conversion, repo URL generation, purgeRemoteCaching env flag, cache folder paths

Why these files

These four source files are the most critical untested orchestrator components:

  • TaskParameterSerializer — serialization backbone for all providers
  • FollowLogStreamService — build output parsing; silent bugs here cause misleading build results
  • OrchestratorNamespace (guid) — GUID format affects K8s job naming and caching
  • OrchestratorFolders — wrong paths = wrong build artifacts

Test characteristics

  • All tests are pure mock-based — no LocalStack, K8s, Docker, or AWS needed
  • Automatically picked up by yarn test:ci on every PR
  • 458 total tests pass (up from 394), 0 regressions

Test plan

  • All 64 new tests pass individually
  • Full test suite passes (yarn test — 458 passed, 0 failures)
  • No changes to source code — tests only

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Tests
    • Added comprehensive unit tests for folder/path handling, GUID generation, log-stream processing (including error accumulation and lifecycle events), and task-parameter serialization/normalization to improve reliability.
  • Chores
    • Added a fast unit-test CI step for quicker feedback.
    • CI/workflow updates: mac build job now continues on error; orchestration workflows and end-to-end tests now prefer the main branch and use a simpler clone fallback.

Tracking:

Adds 64 new mock-based unit tests covering orchestrator services that
previously had zero test coverage:

- TaskParameterSerializer: env var format conversion, round-trip,
  uniqBy deduplication, blocked params, default secrets
- FollowLogStreamService: build output message parsing — end of
  transmission, build success/failure detection, error accumulation,
  Library rebuild detection
- OrchestratorNamespace (guid): GUID generation format, platform
  name normalization, nanoid uniqueness
- OrchestratorFolders: path computation for all folder getters,
  ToLinuxFolder conversion, repo URL generation, purge flag detection

All tests are pure mock-based and run without any external
infrastructure (no LocalStack, K8s, Docker, or AWS).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Mar 5, 2026

Copy link
Copy Markdown
📝 Walkthrough

Walkthrough

Adds multiple Jest test suites for orchestrator components (folders, GUIDs, log streaming, parameter serialization) and updates CI/workflow behavior and repository-clone fallbacks across orchestrator workflows and an end-to-end test flag.

Changes

Cohort / File(s) Summary
Orchestrator Options Tests
src/model/orchestrator/options/orchestrator-folders.test.ts, src/model/orchestrator/options/orchestrator-guid.test.ts
New unit tests: path normalization and computed folder paths, repo/builder URL composition, PURGE_REMOTE_BUILDER_CACHE handling, and GUID formatting/uniqueness rules.
Orchestrator Services Tests
src/model/orchestrator/services/core/follow-log-stream-service.test.ts, src/model/orchestrator/services/core/task-parameter-serializer.test.ts
New tests for FollowLogStreamService (EOT detection, error aggregation, GitHub check updates, outputs) and TaskParameterSerializer (env var format/undo, uniqBy, blocked names, secret reads).
CI Workflows
.github/workflows/orchestrator-integrity.yml, .github/workflows/orchestrator-async-checks.yml, .github/workflows/build-tests-mac.yml
Adds a fast unit-test step to orchestrator-integrity.yml; switches clone targets from orchestrator-develop to main in async checks; sets continue-on-error: true on macOS build job.
Workflow Logic / Build Automation
src/model/orchestrator/workflows/async-workflow.ts, src/model/orchestrator/workflows/build-automation-workflow.ts
Reorders/simplifies repository-clone fallback to prefer main, then plain clone if that fails; removes previous orchestrator-develop fallback.
End-to-end Test Param
src/model/orchestrator/tests/e2e/orchestrator-end2end-caching.test.ts
Updates test parameter to use main branch instead of orchestrator-develop.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Suggested labels

codex

Suggested reviewers

  • webbertakken
  • cloudymax
  • davidmfinol

Poem

🐰 I hopped through tests and CI streams bright,
GUIDs that shimmered in the night,
Logs I chased and parameters spun,
Clone fallbacks changed — then back to fun,
A carrot-coder's hop of pure delight! 🥕

🚥 Pre-merge checks | ✅ 3
✅ Passed checks (3 passed)
Check name Status Explanation
Title check ✅ Passed The title 'test(orchestrator): further unit test coverage' accurately describes the primary change: adding 64 new unit tests for orchestrator components with zero prior coverage.
Description check ✅ Passed The description is comprehensive and well-structured, covering new test files with detailed coverage tables, rationale for selected components, test characteristics, and test plan validation results.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
  • 📝 Generate docstrings (stacked PR)
  • 📝 Generate docstrings (commit on current branch)
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch feature/orchestrator-unit-tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Adds a fast-fail unit test step at the top of orchestrator-integrity,
right after yarn install and before any infrastructure setup (k3d,
LocalStack). Runs 113 mock-based orchestrator tests in ~5 seconds.

If serialization, path computation, log parsing, or provider loading
is broken, the workflow fails immediately instead of spending 30+
minutes setting up LocalStack and k3d clusters.

Tests included: orchestrator-guid, orchestrator-folders,
task-parameter-serializer, follow-log-stream-service,
runner-availability-service, provider-url-parser, provider-loader,
provider-git-manager, orchestrator-image, orchestrator-hooks,
orchestrator-github-checks.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (2)
src/model/orchestrator/options/orchestrator-folders.test.ts (1)

146-161: Use try/finally around PURGE_REMOTE_BUILDER_CACHE mutations.

These tests restore env manually, but not in a failure-safe way. try/finally avoids cross-test contamination when assertions fail.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/model/orchestrator/options/orchestrator-folders.test.ts` around lines 146
- 161, The two tests that mutate process.env.PURGE_REMOTE_BUILDER_CACHE should
be wrapped in try/finally to guarantee restoration on failure: in the 'returns
false when env var is not set' and 'returns true when env var is set' specs,
capture the original value in a local variable, perform the delete/set and call
expect(OrchestratorFolders.purgeRemoteCaching) inside a try block, and then
restore process.env.PURGE_REMOTE_BUILDER_CACHE to the original value (or delete
it if originally undefined) in the finally block so the environment is always
reset even if assertions throw.
src/model/orchestrator/services/core/task-parameter-serializer.test.ts (1)

156-163: Expand assertions to cover all declared blocked/default-secret keys.

Current checks miss keys declared by implementation (CACHE_UNITY_INSTALLATION_ON_MAC, RUNNER_TEMP_PATH, NAME) and additional default secrets (UNITY_EMAIL, UNITY_PASSWORD, GIT_PRIVATE_TOKEN) from src/model/orchestrator/services/core/task-parameter-serializer.ts (Line [11]-[21], Line [169]-[179]). Adding those assertions will better protect against regressions.

Also applies to: 172-206

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/model/orchestrator/services/core/task-parameter-serializer.test.ts`
around lines 156 - 163, The test only asserts some blocked keys; update the test
in TaskParameterSerializer.unit tests to assert all keys declared on
TaskParameterSerializer.blockedParameterNames and
TaskParameterSerializer.defaultSecretParameterNames: add
expect(...has('CACHE_UNITY_INSTALLATION_ON_MAC')),
expect(...has('RUNNER_TEMP_PATH')), expect(...has('NAME')), and for default
secrets add expects for 'UNITY_EMAIL', 'UNITY_PASSWORD', 'GIT_PRIVATE_TOKEN'
(referencing TaskParameterSerializer.blockedParameterNames and
TaskParameterSerializer.defaultSecretParameterNames to find where to add the
assertions).
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@src/model/orchestrator/options/orchestrator-folders.test.ts`:
- Line 2: Remove the unused import statement "import path from 'node:path';"
from the test file (it isn't referenced anywhere), update any related imports if
you expected to use path elsewhere, and then run the linter/tests to confirm the
unused-import lint error is resolved.

In `@src/model/orchestrator/services/core/task-parameter-serializer.test.ts`:
- Around line 178-192: The test that mutates process.env.UNITY_SERIAL should
guard the mutation with try/finally: store the original value in originalSerial,
set process.env.UNITY_SERIAL = 'test-serial', run the assertions that call
TaskParameterSerializer.readDefaultSecrets() and inspect serialSecret, and then
restore or delete process.env.UNITY_SERIAL in a finally block to ensure cleanup
even on assertion failure; apply the same try/finally pattern to the other
related test(s) that mutate process.env (the block around the second test at
lines noted in the review) so no env leak occurs between tests.

---

Nitpick comments:
In `@src/model/orchestrator/options/orchestrator-folders.test.ts`:
- Around line 146-161: The two tests that mutate
process.env.PURGE_REMOTE_BUILDER_CACHE should be wrapped in try/finally to
guarantee restoration on failure: in the 'returns false when env var is not set'
and 'returns true when env var is set' specs, capture the original value in a
local variable, perform the delete/set and call
expect(OrchestratorFolders.purgeRemoteCaching) inside a try block, and then
restore process.env.PURGE_REMOTE_BUILDER_CACHE to the original value (or delete
it if originally undefined) in the finally block so the environment is always
reset even if assertions throw.

In `@src/model/orchestrator/services/core/task-parameter-serializer.test.ts`:
- Around line 156-163: The test only asserts some blocked keys; update the test
in TaskParameterSerializer.unit tests to assert all keys declared on
TaskParameterSerializer.blockedParameterNames and
TaskParameterSerializer.defaultSecretParameterNames: add
expect(...has('CACHE_UNITY_INSTALLATION_ON_MAC')),
expect(...has('RUNNER_TEMP_PATH')), expect(...has('NAME')), and for default
secrets add expects for 'UNITY_EMAIL', 'UNITY_PASSWORD', 'GIT_PRIVATE_TOKEN'
(referencing TaskParameterSerializer.blockedParameterNames and
TaskParameterSerializer.defaultSecretParameterNames to find where to add the
assertions).

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 857b6179-ac5c-472d-986d-168b507ceeae

📥 Commits

Reviewing files that changed from the base of the PR and between 9d47543 and 17a0ea3.

📒 Files selected for processing (4)
  • src/model/orchestrator/options/orchestrator-folders.test.ts
  • src/model/orchestrator/options/orchestrator-guid.test.ts
  • src/model/orchestrator/services/core/follow-log-stream-service.test.ts
  • src/model/orchestrator/services/core/task-parameter-serializer.test.ts

Comment thread src/model/orchestrator/options/orchestrator-folders.test.ts Outdated
Comment on lines +178 to +192
it('includes secrets from environment when present', () => {
const originalSerial = process.env.UNITY_SERIAL;
process.env.UNITY_SERIAL = 'test-serial';

const secrets = TaskParameterSerializer.readDefaultSecrets();
const serialSecret = secrets.find((s) => s.ParameterKey === 'UNITY_SERIAL');
expect(serialSecret).toBeDefined();
expect(serialSecret?.ParameterValue).toBe('test-serial');

if (originalSerial !== undefined) {
process.env.UNITY_SERIAL = originalSerial;
} else {
delete process.env.UNITY_SERIAL;
}
});

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Guard process.env mutations with try/finally to prevent test bleed.

If an assertion fails before cleanup, UNITY_SERIAL leaks into later tests and creates order-dependent failures. Wrap mutation and assertions in try/finally.

Proposed fix
 it('includes secrets from environment when present', () => {
   const originalSerial = process.env.UNITY_SERIAL;
-  process.env.UNITY_SERIAL = 'test-serial';
-
-  const secrets = TaskParameterSerializer.readDefaultSecrets();
-  const serialSecret = secrets.find((s) => s.ParameterKey === 'UNITY_SERIAL');
-  expect(serialSecret).toBeDefined();
-  expect(serialSecret?.ParameterValue).toBe('test-serial');
-
-  if (originalSerial !== undefined) {
-    process.env.UNITY_SERIAL = originalSerial;
-  } else {
-    delete process.env.UNITY_SERIAL;
-  }
+  try {
+    process.env.UNITY_SERIAL = 'test-serial';
+    const secrets = TaskParameterSerializer.readDefaultSecrets();
+    const serialSecret = secrets.find((s) => s.ParameterKey === 'UNITY_SERIAL');
+    expect(serialSecret).toBeDefined();
+    expect(serialSecret?.ParameterValue).toBe('test-serial');
+  } finally {
+    if (originalSerial !== undefined) {
+      process.env.UNITY_SERIAL = originalSerial;
+    } else {
+      delete process.env.UNITY_SERIAL;
+    }
+  }
 });

 it('excludes secrets not in environment', () => {
   const originalSerial = process.env.UNITY_SERIAL;
-  delete process.env.UNITY_SERIAL;
-
-  const secrets = TaskParameterSerializer.readDefaultSecrets();
-  const serialSecret = secrets.find((s) => s.ParameterKey === 'UNITY_SERIAL');
-  expect(serialSecret).toBeUndefined();
-
-  if (originalSerial !== undefined) {
-    process.env.UNITY_SERIAL = originalSerial;
-  }
+  try {
+    delete process.env.UNITY_SERIAL;
+    const secrets = TaskParameterSerializer.readDefaultSecrets();
+    const serialSecret = secrets.find((s) => s.ParameterKey === 'UNITY_SERIAL');
+    expect(serialSecret).toBeUndefined();
+  } finally {
+    if (originalSerial !== undefined) {
+      process.env.UNITY_SERIAL = originalSerial;
+    } else {
+      delete process.env.UNITY_SERIAL;
+    }
+  }
 });

Also applies to: 194-205

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/model/orchestrator/services/core/task-parameter-serializer.test.ts`
around lines 178 - 192, The test that mutates process.env.UNITY_SERIAL should
guard the mutation with try/finally: store the original value in originalSerial,
set process.env.UNITY_SERIAL = 'test-serial', run the assertions that call
TaskParameterSerializer.readDefaultSecrets() and inspect serialSecret, and then
restore or delete process.env.UNITY_SERIAL in a finally block to ensure cleanup
even on assertion failure; apply the same try/finally pattern to the other
related test(s) that mutate process.env (the block around the second test at
lines noted in the review) so no env leak occurs between tests.

@frostebite frostebite changed the title test(orchestrator): unit tests for untested core services test(orchestrator): additional test coverage Mar 5, 2026
@frostebite frostebite changed the title test(orchestrator): additional test coverage test(orchestrator): further unit test coverage Mar 5, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
.github/workflows/orchestrator-integrity.yml (1)

205-205: Test selector is too broad for a deterministic “fast” gate.

Line 205 matches by substring only, so future tests with similar names can be pulled in unintentionally. Anchor to test filenames (or switch to --runTestsByPath) to keep runtime stable.

🎯 Suggested pattern hardening
-          --testPathPattern="orchestrator-guid|orchestrator-folders|task-parameter-serializer|follow-log-stream-service|runner-availability-service|provider-url-parser|provider-loader|provider-git-manager|orchestrator-image|orchestrator-hooks|orchestrator-github-checks"
+          --testPathPattern="(orchestrator-guid|orchestrator-folders|task-parameter-serializer|follow-log-stream-service|runner-availability-service|provider-url-parser|provider-loader|provider-git-manager|orchestrator-image|orchestrator-hooks|orchestrator-github-checks)\\.test\\.ts$"
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In @.github/workflows/orchestrator-integrity.yml at line 205, The
--testPathPattern entry uses an unanchored substring list (the string containing
"orchestrator-guid|orchestrator-folders|...|orchestrator-github-checks"), which
can accidentally match future test names; fix by anchoring each alternative
(e.g., wrap each filename with ^ and $ in the regex) or replace this flag with
--runTestsByPath and supply the explicit test file paths for deterministic
selection. Update the line that sets --testPathPattern to either an anchored
regex of the exact test filenames or switch to --runTestsByPath with the
specific filenames to prevent accidental matches.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In @.github/workflows/orchestrator-integrity.yml:
- Around line 199-206: The "Run orchestrator unit tests (fast, no infra)" step
currently runs after heavy infra prep (LocalStack/AWS) which defeats the
fast-fail purpose; move the entire step (the job step named "Run orchestrator
unit tests (fast, no infra)" that runs yarn run test with --testPathPattern for
orchestrator-* and flags --verbose --detectOpenHandles --forceExit --runInBand)
to run immediately after checkout/setup and before any LocalStack/AWS
preparation steps so the infra-free fast gate executes first and can fail
quickly.

---

Nitpick comments:
In @.github/workflows/orchestrator-integrity.yml:
- Line 205: The --testPathPattern entry uses an unanchored substring list (the
string containing
"orchestrator-guid|orchestrator-folders|...|orchestrator-github-checks"), which
can accidentally match future test names; fix by anchoring each alternative
(e.g., wrap each filename with ^ and $ in the regex) or replace this flag with
--runTestsByPath and supply the explicit test file paths for deterministic
selection. Update the line that sets --testPathPattern to either an anchored
regex of the exact test filenames or switch to --runTestsByPath with the
specific filenames to prevent accidental matches.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 2ff29477-1ae3-4712-84ae-9140d388fdde

📥 Commits

Reviewing files that changed from the base of the PR and between 17a0ea3 and f445106.

📒 Files selected for processing (1)
  • .github/workflows/orchestrator-integrity.yml

Comment on lines +199 to +206
# FAST UNIT TESTS (no infra required, fast-fail gate)
# ==========================================
- name: Run orchestrator unit tests (fast, no infra)
timeout-minutes: 2
run: >-
yarn run test
--testPathPattern="orchestrator-guid|orchestrator-folders|task-parameter-serializer|follow-log-stream-service|runner-availability-service|provider-url-parser|provider-loader|provider-git-manager|orchestrator-image|orchestrator-hooks|orchestrator-github-checks"
--verbose --detectOpenHandles --forceExit --runInBand

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Fast-fail gate is still positioned after heavy infra setup.

Line 199 says this gate is infra-free, but it currently runs after LocalStack/AWS prep earlier in the job. That defeats the fast-fail intent and still spends setup time before failing.

🔧 Suggested reorder
-      - name: Set up kubectl
-        uses: azure/setup-kubectl@v4
-        with:
-          version: 'v1.34.1'
-      - name: Install k3d
-        run: |
-          curl -s https://raw.githubusercontent.com/k3d-io/k3d/main/install.sh | bash
-          k3d version | cat
-      ...
       - run: yarn install --frozen-lockfile
+      - name: Run orchestrator unit tests (fast, no infra)
+        timeout-minutes: 2
+        run: >-
+          yarn run test
+          --testPathPattern="orchestrator-guid|orchestrator-folders|task-parameter-serializer|follow-log-stream-service|runner-availability-service|provider-url-parser|provider-loader|provider-git-manager|orchestrator-image|orchestrator-hooks|orchestrator-github-checks"
+          --verbose --detectOpenHandles --forceExit --runInBand
+      - name: Set up kubectl
+        uses: azure/setup-kubectl@v4
+        with:
+          version: 'v1.34.1'
+      - name: Install k3d
+        run: |
+          curl -s https://raw.githubusercontent.com/k3d-io/k3d/main/install.sh | bash
+          k3d version | cat
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In @.github/workflows/orchestrator-integrity.yml around lines 199 - 206, The
"Run orchestrator unit tests (fast, no infra)" step currently runs after heavy
infra prep (LocalStack/AWS) which defeats the fast-fail purpose; move the entire
step (the job step named "Run orchestrator unit tests (fast, no infra)" that
runs yarn run test with --testPathPattern for orchestrator-* and flags --verbose
--detectOpenHandles --forceExit --runInBand) to run immediately after
checkout/setup and before any LocalStack/AWS preparation steps so the infra-free
fast gate executes first and can fail quickly.

@github-actions

github-actions Bot commented Mar 5, 2026

Copy link
Copy Markdown

Cat Gif

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@frostebite frostebite added enhancement New feature or request orchestrator Orchestrator module LTS 2.0 Orchestrator LTS v2.0 milestone labels Mar 5, 2026
@codecov

codecov Bot commented Mar 5, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 32.96%. Comparing base (9d47543) to head (e3c87cc).

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #784      +/-   ##
==========================================
+ Coverage   31.25%   32.96%   +1.70%     
==========================================
  Files          84       84              
  Lines        4563     4563              
  Branches     1103     1103              
==========================================
+ Hits         1426     1504      +78     
+ Misses       3137     3058      -79     
- Partials        0        1       +1     
Files with missing lines Coverage Δ
src/model/orchestrator/workflows/async-workflow.ts 27.77% <ø> (ø)
...rchestrator/workflows/build-automation-workflow.ts 10.44% <ø> (ø)

... and 3 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

frostebite and others added 2 commits March 5, 2026 23:33
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The orchestrator-develop branch no longer exists. Update all fallback
clone commands and test fixtures to use main instead.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In @.github/workflows/build-tests-mac.yml:
- Line 15: Remove the job-level continue-on-error: true (the top-level
continue-on-error key) so the macOS matrix remains blocking, and if specific
matrix legs are flaky scope tolerance to them by creating a separate job (or
using strategy.matrix.include to isolate those entries) that runs only the flaky
macOS configurations and sets continue-on-error: true for that job; ensure the
original job or primary macOS job has no continue-on-error so failures surface
as CI failures.

In @.github/workflows/orchestrator-async-checks.yml:
- Line 57: The workflow currently clones unity-builder using a hard-coded branch
("git clone -b main https://github.com/game-ci/unity-builder"); change that to
use the dispatched branch ref so checks run against the branch that triggered
the dispatch (replace "main" with the runtime ref variable, e.g. use "${{
github.ref_name }}" in the git clone command). Update the line containing "git
clone -b main https://github.com/game-ci/unity-builder" to "git clone -b ${{
github.ref_name }} https://github.com/game-ci/unity-builder".

In `@src/model/orchestrator/tests/e2e/orchestrator-end2end-caching.test.ts`:
- Line 33: The test is forcibly setting orchestratorBranch: `main`, which
bypasses the branch under review; remove the hard-coded orchestratorBranch
property in the orchestrator-end2end-caching.test.ts diff so the test uses the
existing orchestratorBranch input/environment resolution instead (i.e., delete
the line `orchestratorBranch: \`main\`,` or stop passing that override where the
test constructs its inputs).

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 8596660c-df88-4671-a36b-1c4f820734d1

📥 Commits

Reviewing files that changed from the base of the PR and between 7db70a7 and e3c87cc.

⛔ Files ignored due to path filters (2)
  • dist/index.js is excluded by !**/dist/**
  • dist/index.js.map is excluded by !**/dist/**, !**/*.map
📒 Files selected for processing (5)
  • .github/workflows/build-tests-mac.yml
  • .github/workflows/orchestrator-async-checks.yml
  • src/model/orchestrator/tests/e2e/orchestrator-end2end-caching.test.ts
  • src/model/orchestrator/workflows/async-workflow.ts
  • src/model/orchestrator/workflows/build-automation-workflow.ts

buildForAllPlatformsMacOS:
name: ${{ matrix.targetPlatform }} on ${{ matrix.unityVersion }}
runs-on: macos-latest
continue-on-error: true

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

cat -n .github/workflows/build-tests-mac.yml

Repository: game-ci/unity-builder

Length of output: 3713


Keep the macOS matrix blocking.

Job-level continue-on-error: true at line 15 marks the entire job as successful even when matrix legs fail, removing CI signal for macOS/iOS regressions across all Unity versions and platforms. If only specific configurations are flaky, scope the tolerance to those matrix entries instead of the whole job.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In @.github/workflows/build-tests-mac.yml at line 15, Remove the job-level
continue-on-error: true (the top-level continue-on-error key) so the macOS
matrix remains blocking, and if specific matrix legs are flaky scope tolerance
to them by creating a separate job (or using strategy.matrix.include to isolate
those entries) that runs only the flaky macOS configurations and sets
continue-on-error: true for that job; ensure the original job or primary macOS
job has no continue-on-error so failures surface as CI failures.

CHECKS_UPDATE: ${{ github.event.inputs.checksObject }}
run: |
git clone -b orchestrator-develop https://github.com/game-ci/unity-builder
git clone -b main https://github.com/game-ci/unity-builder

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

cat -n .github/workflows/orchestrator-async-checks.yml

Repository: game-ci/unity-builder

Length of output: 2359


🌐 Web query:

github.ref_name github actions variable documentation

💡 Result:

github.ref_name is a GitHub Actions context property (used in expressions like ${{ github.ref_name }}) that contains the short ref name of the branch or tag that triggered the workflow run (for example, feature-branch-1). For unmerged pull_request events, it is formatted as <pr_number>/merge. (docs.github.com)

The corresponding default runner environment variable is GITHUB_REF_NAME (used in shell as $GITHUB_REF_NAME on Linux/macOS or $env:GITHUB_REF_NAME on Windows), with the same meaning and PR formatting note. (docs.github.com)

Citations:


Use the dispatched ref here, not a hard-coded main.

This makes Async Checks run checks-update against main even when the workflow is dispatched from another branch, so branch-specific regressions can be missed.

Proposed fix
-          git clone -b main https://github.com/game-ci/unity-builder
+          git clone -b "${{ github.ref_name }}" https://github.com/game-ci/unity-builder \
+            || git clone -b main https://github.com/game-ci/unity-builder
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
git clone -b main https://github.com/game-ci/unity-builder
git clone -b "${{ github.ref_name }}" https://github.com/game-ci/unity-builder \
|| git clone -b main https://github.com/game-ci/unity-builder
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In @.github/workflows/orchestrator-async-checks.yml at line 57, The workflow
currently clones unity-builder using a hard-coded branch ("git clone -b main
https://github.com/game-ci/unity-builder"); change that to use the dispatched
branch ref so checks run against the branch that triggered the dispatch (replace
"main" with the runtime ref variable, e.g. use "${{ github.ref_name }}" in the
git clone command). Update the line containing "git clone -b main
https://github.com/game-ci/unity-builder" to "git clone -b ${{ github.ref_name
}} https://github.com/game-ci/unity-builder".

cacheKey: `test-case-${uuidv4()}`,
containerHookFiles: `debug-cache`,
orchestratorBranch: `orchestrator-develop`,
orchestratorBranch: `main`,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Don't pin this e2e run to main.

With this override, the caching test can go green without ever exercising the branch under review inside the orchestrator. Let the existing orchestratorBranch input/env resolution supply the branch instead.

Proposed fix
-        orchestratorBranch: `main`,
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
orchestratorBranch: `main`,
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/model/orchestrator/tests/e2e/orchestrator-end2end-caching.test.ts` at
line 33, The test is forcibly setting orchestratorBranch: `main`, which bypasses
the branch under review; remove the hard-coded orchestratorBranch property in
the orchestrator-end2end-caching.test.ts diff so the test uses the existing
orchestratorBranch input/environment resolution instead (i.e., delete the line
`orchestratorBranch: \`main\`,` or stop passing that override where the test
constructs its inputs).

@frostebite

Copy link
Copy Markdown
Member Author

Closing — all orchestrator code has been extracted to the standalone game-ci/orchestrator repository.

Content from this PR (unit test coverage for folders, guid, follow-log-stream, task-parameter-serializer) is fully present in the orchestrator repo. See PR #819 for the extraction.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request LTS 2.0 Orchestrator LTS v2.0 milestone orchestrator Orchestrator module

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant