You’re watching a YouTube video in 1080p.

Suddenly, your internet connection slows down.

The picture becomes slightly blurry.

The quality drops to 720p.

A few seconds later, your connection recovers.

The video returns to 1080p.

You didn’t press anything.

The video didn’t restart.

There was no page reload.

So, how did YouTube change the video quality while you were watching it?

The answer is Adaptive Bitrate Streaming (ABR).

And although YouTube’s web playback primarily uses MPEG-DASH rather than HLS, HLS is an excellent way to understand the underlying concept because both technologies use the same fundamental idea:

Provide multiple versions of the same video and allow the player to dynamically select the most appropriate one.

What Is Adaptive Bitrate Streaming?

Imagine the same video is available in several quality levels:

| Quality | Illustrative Bitrate |
|---------|----------------------|
| 360p    | 800 Kbps             |
| 480p    | 1.5 Mbps             |
| 720p    | 3 Mbps               |
| 1080p   | 6 Mbps               |

These are not four different videos.

They are different encoded representations of the same video.

Each representation may have a different:

• Resolution
• Bitrate
• Frame rate
• Codec
• Encoding profile

If your network is fast and stable, the player can choose 1080p.

If your network becomes slower, it can switch to 720p or 480p.

The goal isn’t simply to provide the highest possible quality.

The real goal is:

Deliver the highest sustainable quality while avoiding buffering.

Why Can’t the Player Just Change the Video Quality?

This is where adaptive streaming gets interesting.

Suppose the server has a single file:

movie.mp4

The browser downloads and plays it.

Now imagine the network suddenly becomes slower.

The player can’t simply say:

“I’m switching this same file from 1080p to 720p.”

Instead, adaptive streaming prepares multiple versions of the content in advance.

More importantly, each version is divided into small pieces called segments.

For example:

360p
[Segment 1] [Segment 2] [Segment 3] [Segment 4]

720p
[Segment 1] [Segment 2] [Segment 3] [Segment 4]

1080p
[Segment 1] [Segment 2] [Segment 3] [Segment 4]

All three versions represent the same timeline.

The player can therefore do something like this:

Segment 1 → 360p
Segment 2 → 360p
Segment 3 → 720p
Segment 4 → 720p
Segment 5 → 1080p

The video keeps playing.

Only the representation used for upcoming segments changes.

That’s the fundamental trick behind adaptive streaming.

How HLS Works

HLS (HTTP Live Streaming) is an adaptive streaming protocol originally developed by Apple.

HLS uses playlists to describe available streams and media segments.

A simplified HLS architecture looks like this:

                 master.m3u8
                      |
          +-----------+-----------+
          |           |           |
         360p        720p        1080p
          |           |           |
      playlist     playlist     playlist
          |           |           |
      segments     segments     segments

The top-level playlist tells the player which quality levels are available.

For example:

#EXTM3U

#EXT-X-STREAM-INF:BANDWIDTH=800000,RESOLUTION=640x360
360p/playlist.m3u8

#EXT-X-STREAM-INF:BANDWIDTH=3000000,RESOLUTION=1280x720
720p/playlist.m3u8

#EXT-X-STREAM-INF:BANDWIDTH=6000000,RESOLUTION=1920x1080
1080p/playlist.m3u8

The player now knows:

360p  → ~800 Kbps
720p  → ~3 Mbps
1080p → ~6 Mbps

It can choose whichever representation it believes is appropriate.

What Is Inside a Variant Playlist?

A variant playlist points to the actual media segments.

For example:

#EXTM3U

#EXTINF:6,
720p/segment001.ts

#EXTINF:6,
720p/segment002.ts

#EXTINF:6,
720p/segment003.ts

#EXTINF:6,
720p/segment004.ts

Each segment represents a small portion of the video.

If the segment duration is six seconds, the player might have:

0–6 sec
6–12 sec
12–18 sec
18–24 sec
...

The exact segment duration can vary depending on the streaming system.

The important point is that the player doesn’t need to download the entire video before making a quality decision.

It can make decisions continuously as playback progresses.

So How Does the Player Know When to Switch?

This is where the ABR algorithm comes in.

The player continuously monitors several signals.

The two most important ones are:

1. Available bandwidth

How quickly can the player download the next segment?

2. Buffer health

How much video has already been downloaded and is waiting to be played?

Conceptually:

              +------------------+
              |  Network Speed   |
              +--------+---------+
                       |
                       v
              +------------------+
              |   Buffer Level   |
              +--------+---------+
                       |
                       v
              +------------------+
              |   ABR Algorithm  |
              +--------+---------+
                       |
                       v
              +----------------------+
              | Next Quality Level   |
              | 360p → 480p → 720p   |
              |        → 1080p       |
              +----------------------+

The algorithm uses these signals to decide what should be downloaded next.

Bandwidth Is Not the Same as Internet Speed

Suppose your internet connection is capable of 100 Mbps.

That does not necessarily mean your video player should immediately choose the highest-quality stream.

Why?

Because available bandwidth can fluctuate.

You might see:

20 Mbps
↓
15 Mbps
↓
8 Mbps
↓
5 Mbps
↓
12 Mbps

If the player reacts aggressively to every measurement, quality could constantly oscillate:

1080p
↓
720p
↓
1080p
↓
720p
↓
1080p

That would be a terrible viewing experience.

Therefore, ABR algorithms generally use some combination of:

• Recent throughput measurements
• Throughput history
• Buffer level
• Safety margins
• Current representation
• Device capabilities
• Playback conditions

The exact algorithm depends on the player implementation.

The Buffer Is Your Safety Net

The second major concept is the playback buffer.

Imagine the player has already downloaded 20 seconds of video:

Buffer

████████████████████
        20 sec

If the network temporarily slows down, playback can continue because the player already has video waiting.

Now consider:

Buffer

██
2 sec

A slowdown is much more dangerous.

The player needs to become conservative.

This leads to a simple principle:

Healthy buffer + good bandwidth
            ↓
       Higher quality


Low buffer + poor bandwidth
            ↓
       Lower quality

The player isn’t simply asking:

“How fast is my internet?”

It’s effectively asking:

“Can I continue playing this quality without running out of buffered video?”

A Real Example: 1080p → 720p → 1080p

Let’s walk through a realistic scenario.

You’re watching a video at 1080p.

1080p
6 Mbps

Your network is stable.

The player continues downloading 1080p segments:

1080p → 1080p → 1080p → 1080p

Then your network slows down.

The player observes:

Throughput ↓
Buffer ↓

Continuing to download 1080p becomes risky.

Instead of waiting for the buffer to reach zero, the player proactively switches:

1080p
   ↓
720p

Now the required bitrate is lower.

The buffer stabilises.

A few seconds later, your network recovers:

Throughput ↑
Buffer healthy

The player determines that 1080p is sustainable again.

So it switches back:

720p
   ↓
1080p

From your perspective, it looks like YouTube simply changed the quality.

Underneath, something more interesting happened:

1080p segment
1080p segment
720p segment
720p segment
1080p segment
1080p segment

The timeline never stopped.

Why Doesn’t Quality Change Every Second?

Because that would create quality oscillation.

Imagine bandwidth fluctuates around the boundary between 720p and 1080p.

Without any protection, the player could repeatedly switch:

720p → 1080p → 720p → 1080p

A good ABR implementation therefore doesn’t necessarily switch immediately after every bandwidth change.

It can use:

• Hysteresis
• Switching thresholds
• Moving averages
• Buffer thresholds
• Safety margins

Conceptually:

Switch DOWN
when the current quality becomes unsafe.

Switch UP
only when the higher quality is comfortably sustainable.

This is why quality often drops relatively quickly when the network deteriorates but takes a little longer to increase again.

The player is trying to avoid making a decision it will immediately have to reverse.

HLS vs MPEG-DASH: An Important Distinction

There’s a common misconception:

“YouTube uses HLS to switch video quality.”

That’s not technically accurate for YouTube’s primary web playback architecture.

YouTube has historically used MPEG-DASH together with Media Source Extensions (MSE) for web video playback.

HLS and MPEG-DASH are different technologies.

HLS uses concepts such as:

.m3u8
Variant Streams
Media Segments

MPEG-DASH uses:

.mpd
Adaptation Sets
Representations
Media Segments

But the underlying idea is very similar:

Multiple qualities
       ↓
Segmented video
       ↓
Player measures conditions
       ↓
ABR algorithm
       ↓
Select next representation
       ↓
Continue playback

So if you understand adaptive streaming through HLS, you’ve already understood a large part of the conceptual foundation behind DASH-based streaming as well.

Where Does the CDN Come In?

There is another important component:

The Content Delivery Network (CDN).

Imagine millions of users watching videos simultaneously.

You wouldn’t want every request to go directly to one central origin server.

Instead, video segments are distributed through CDN infrastructure.

A simplified architecture looks like this:

                 Video Origin
                       |
                       v
                  CDN Network
                /      |      \
               /       |       \
          Edge A    Edge B    Edge C
             |         |         |
           Users     Users     Users

When your player requests a video segment, the request can be served by a nearby CDN edge.

For example:

Player
   |
   v
CDN Edge
   |
   v
Video Segment

This reduces latency and allows streaming platforms to serve enormous numbers of concurrent viewers.

What Happens Inside the Browser?

For modern web playback, the architecture can be simplified to:

Network
   |
   v
Streaming Player
   |
   v
ABR Algorithm
   |
   v
Media Buffer
   |
   v
Browser Media Pipeline
   |
   v
Decoder
   |
   v
GPU
   |
   v
Screen

With technologies such as Media Source Extensions, a web application can control how media segments are supplied to the browser’s media pipeline.

The browser then handles the heavy lifting of decoding and rendering the video.

For a frontend engineer, this is an important distinction.

The <video> element may look simple:

<video controls></video>

But behind that simple UI is an entire streaming system involving:

Manifest
↓
ABR
↓
HTTP requests
↓
CDN
↓
Segments
↓
Buffer management
↓
Media Source Extensions
↓
Codec
↓
Decoder
↓
GPU

The Complete Adaptive Streaming Flow

Putting everything together:

                 Original Video
                       |
                       v
                    Encoder
                       |
          +------------+------------+
          |            |            |
         360p         720p        1080p
          |            |            |
          +------------+------------+
                       |
                  Segmentation
                       |
                       v
                      CDN
                       |
                       v
                 Player/Browser
                       |
              +--------+--------+
              |                 |
        Bandwidth            Buffer
              |                 |
              +--------+--------+
                       |
                       v
                  ABR Algorithm
                       |
                       v
               Select Next Quality
                       |
                       v
                 Download Segment
                       |
                       v
                    Buffer
                       |
                       v
                    Decode
                       |
                       v
                    Display

This cycle repeats throughout playback.

The player continuously evaluates the situation and decides what should happen next.

Why Adaptive Streaming Is So Important

Without adaptive streaming, a video player would have to choose between two bad options.

Option 1: Always use high quality

Great picture.

But when the network slows down:

Buffer ↓
Buffer ↓
Buffer ↓

BUFFERING

Option 2: Always use low quality

Playback is more reliable.

But users with fast connections receive unnecessarily poor video quality.

Adaptive streaming provides a middle ground:

Fast network
     ↓
Higher quality

Slow network
     ↓
Lower quality

Network recovers
     ↓
Higher quality again

The player continuously adapts rather than committing to one bitrate for the entire session.

The Bigger Picture

The next time YouTube changes from 1080p to 720p, it isn’t necessarily “changing the video.”

It’s selecting a different representation of the same content.

The complete system looks something like this:

Video Encoding
      ↓
Multiple Representations
      ↓
Segmentation
      ↓
Manifest / Playlist
      ↓
CDN
      ↓
Streaming Player
      ↓
Bandwidth + Buffer Monitoring
      ↓
ABR Decision
      ↓
Next Segment
      ↓
Browser Decoder
      ↓
Screen

What looks like a tiny quality change in the YouTube UI is actually the result of multiple distributed systems working together.

Final Takeaway

Adaptive streaming is not about making one video dynamically change its resolution.

It’s about creating multiple representations of the same video, dividing them into manageable segments, and allowing the player to intelligently choose which representation to download next.

HLS does this through playlists and variants.

MPEG-DASH does it through MPDs, adaptation sets, and representations.

The ABR algorithm sits at the centre of the experience, continuously balancing:

Bandwidth

Buffer health

Video quality

Re-buffering risk

And that leads to the most important principle in adaptive streaming:

The best streaming experience isn’t the highest possible quality. It’s the highest sustainable quality without interrupting playback.

That’s why YouTube can move from 1080p to 720p and back again without you ever seeing the video stop.

The quality changed.

The playback didn’t.

Adaptive Streaming: How YouTube Changes Video Quality Without Stopping Playback was originally published in Level Up Coding on Medium, where people are continuing the conversation by highlighting and responding to this story.

I co-build a browser extension that exports AI conversations, so people write to us about this a lot: they want a chat on paper. A transcript for a lawyer, a research log for a supervisor, a record of a diagnosis conversation to take to an appointment.

Last week I went looking for the Print button in ChatGPT, Claude and Grok. There isn’t one. Not in any of them, and not documented anywhere by any of the three vendors.

So I pressed Cmd+P. Chrome prepared a 17-page print job, and every page was blank.

This is the story of what the apps do give you instead, why the browser’s own print capture fails, and the one-line Express configuration mistake that made our replacement for it fail in exactly the same way, for completely different reasons.

Nobody Ships a Print Button

Start with what is actually documented, because the gap is wider than it looks.

ChatGPT has no print option. What it added instead, in a February 13, 2026 release note, is a change to text selection: Cmd+A “now selects only the conversation content, excluding surrounding interface elements.” That is a copy-to-clipboard improvement, shipped by a team that clearly knows people are trying to get the text out. It is not printing. Deep research reports download as PDF, Word or Markdown, but ordinary conversations do not.

Claude has no print option either. Public share links are view-only. Artifacts created from a template export properly (Documents to Word, PDF, Markdown or Google Docs; Decks to PPTX or PDF), and Claude can create files directly. But the conversation around the artifact is not covered by any of it.

Grok has a share link, revocable at grok.com/share-links, and a full account data download in Settings under Data Controls whose format is not documented anywhere. There is no single-chat export.

Three products that people use for hours a day, and the only documented way to get one conversation out as a document is to make it into something else first.

Why Ctrl+P Comes Out Blank

The browser can always print a page, so why 17 blank sheets?

Because the transcript is not there. ChatGPT renders messages as you scroll, and the print capture takes the page in its unrendered state. The DOM the printer serialises contains the scroll container and almost nothing else.

This is not a bug in the usual sense. Virtualising a long transcript is the correct call for scroll performance, and every chat UI of any size does it. But virtualisation makes an implicit trade that nobody writes down: the document only exists in the places you have looked at. Print, Find in page, Reader mode and “save page as” all assume the opposite.

The workaround everyone discovers is to scroll from the first message to the last, slowly, so the whole thing is realised, and then print. That works. It is also an instruction you cannot give a non-technical user with a straight face.

Worth saying plainly: I tested this on ChatGPT. I have not run the same test on claude.ai or grok.com, so treat the print preview as the proof on any app. If the pages are blank or the end is missing, the capture failed.

So We Built the Export Instead

Our extension renders a conversation to PDF server-side, which sidesteps the virtualisation problem entirely: the messages come from the API, not the DOM, so there is nothing to scroll.

That worked until a paying customer told us it did not. They had bought the plan specifically to export one long chat as a PDF, and got this instead:

“This chat is too long to build as a PDF in your browser”

Three Limits, Three Units

The reason that message appeared turned out to be three separate ceilings, none of which knows about the others.

A Chrome extension’s background worker gets about 30 seconds for a fetch. A server render of a very long conversation takes longer than that, so the request died in flight. The export then fell back to the in-browser rasteriser, which refuses anything above 200 pages. The customer saw the 200-page refusal and reasonably concluded the product could not do the thing they had paid for.

The Fix: Render It as a Job

The fix was to stop trying to do it in one request. Above a threshold, the conversation goes to the backend as a job: start it, poll, download.

/**
 * Content characters above which a conversation is rendered as a job.
 *
 * Equal to the backend's PART_BUDGET_CHARS: below it the backend would print
 * the job as a single part anyway, and the single request is measured at about
 * 1.2s there, far inside every timeout. Above it, the job path.
 */
export const LONG_PDF_CHARS = 300_000;

export function shouldRenderAsJob(messages: readonly HasContent[]): boolean {
  return contentChars(messages) > LONG_PDF_CHARS;
}

Three requests, each of which answers immediately, so none of them can outlast the 30-second worker limit. The backend prints the document in parts and joins them. Everything below the threshold keeps the single request it always had.

That shipped, and it worked.

The Bug the Fix Created

Now the part I actually want to write about.

Our Express app caps request bodies globally. Two routes legitimately carry a whole conversation as JSON, so they are exempted:

// The global parsers below cap bodies low (jsonWithRawBody defaults to 100kb;
// express.json to 1mb), so they'd reject a long conversation with 413 *before*
// it reached the route's own larger parser.
const LARGE_BODY_PATHS = new Set(["/export/pdf", "/context-mentions/summarize", "/usage/sync"]);

const skipLargeBody =
  (mw: RequestHandler): RequestHandler =>
  (req, res, next) =>
    LARGE_BODY_PATHS.has(req.path) ? next() : mw(req, res, next);

app.use(skipLargeBody(jsonWithRawBody));
app.use(skipLargeBody(express.json({ limit: "1mb" })));

Read the set, then read the route the fix added:

router.post("/export/pdf/jobs", exportJson, extensionAuth, pdfExportRateLimit, handler);

Set.has() is an exact match. /export/pdf/jobs is not /export/pdf. The route carries its own 10MB parser, but the global 100KB parser runs first and rejects the body before the route's parser is ever reached.

The whole point of the jobs route is conversations too big to render in one request. It is, by construction, the route with the largest bodies in the application. And it is the one route not on the list of routes allowed to have large bodies.

One 413, Four Steps to the Error

What makes this worth a thousand words is not the mistake. It is that the mistake is invisible.

Follow the 413 through:

export function readJobStart(status: number | null, body: unknown): JobStart {
  if (status === 404) return { kind: "unsupported" };
  if (status === 429) return { kind: "busy" };
  // ...
  return { kind: "failed" };
}

A 413 is not 404, so the client does not conclude the backend is too old. It is not 429, so it does not retry. There is no job id, so it returns failed. renderPdfAsJob then returns undefined, which is the same value it returns for any render that genuinely could not be produced, and the caller does what it does for every failed render: falls back to the in-browser rasteriser, which refuses above 200 pages.

So the user gets the exact error the job path was built to eliminate. No alert fires. The 413 never surfaces as a 413. A graceful degradation path, which is a good thing to have, is also a very effective way to hide a configuration bug for weeks.

What I Would Take From This

Four things, in rough order of how much they would have saved me.

An allowlist keyed on an exact path is a trap the moment a route grows a sub-path. req.path.startsWith() over a prefix list, or attaching the limit to the router rather than to a central set, both survive the next route. A Set of strings does not, and nothing warns you.

Put the limit next to the thing it limits. The body cap for /export/pdf/jobs lives in index.ts, about four hundred lines from the route it governs. Anyone adding a route in routes/export.ts sees the route's own 10MB parser and reasonably assumes that is the operative number.

Treat every fallback as a place a bug can hide. A fallback converts a loud failure into a quiet degradation, which is exactly what you want in production and exactly what you do not want while finding this. If a fallback fires, log why, with the status code, at a level someone actually reads.

Test the case the feature exists for. Every test we had used a normal conversation. The jobs route exists only for abnormal ones, and a fixture above 300,000 characters would have caught this on the first run.

Key Takeaways

• None of ChatGPT, Claude or Grok documents a Print option for a conversation. Printing from the browser is the only universal route, and on a virtualised transcript it can produce blank pages.
• Virtualised rendering silently breaks Print, Find in page and save-as. The document only exists where the user has scrolled.
• A Set.has(req.path) allowlist stops covering a route the moment that route gains a sub-path, and the failure mode is a 413 that looks like something else entirely.
• Keep request-size limits next to their routes, not in a central list in the app entry point.
• A fallback path is a bug silencer. Log the reason it fired.

FAQ

Can you print a ChatGPT conversation? Not with a built-in option, because there isn’t one. Press Ctrl+P or Cmd+P in a desktop browser, but scroll the whole conversation first and check the print preview before sending it, because a long chat can come out blank.

Why does my ChatGPT printout come out blank? ChatGPT renders messages as you scroll. If the print capture happens before a message has been rendered, that message is not in the DOM and does not reach the page.

Is there a free way to get a long chat onto paper? Yes. Export it as plain text and print that from any text editor. TXT export is on the free plan in our extension; the formatted PDF is the paid one.

Resources

• How to print a ChatGPT, Claude or Grok conversation, the step-by-step version of this, with the per-app detail
• Chrome Help: print from Chrome
• Express body-parser limits

About the author: Adi Leviim is the co-founder of AI Toolbox (formerly ChatGPT Toolbox), a Chrome extension with four modules (ChatGPT, Gemini, Claude, Grok) used by 40,000+ people across 150+ countries to search, organize, and export their AI conversations. He writes about the reality of building AI products with 7+ years of full-stack development experience. Follow him on Medium for honest takes on SaaS, AI tools, and shipping software that people actually use.

Try it free on the Chrome Web Store. TXT export is on the free plan; Markdown, JSON and PDF are Premium at $9.99 a month or $99 lifetime per module, and All Access is $199 one time for all four. These are extension features, not an AI subscription.

AI Toolbox | Medium | Twitter/X | LinkedIn

Four AI Apps, Zero Print Buttons, and One 100KB Cap in the Wrong Place was originally published in Level Up Coding on Medium, where people are continuing the conversation by highlighting and responding to this story.

First off, aiHelpDesk is model neutral. When we claim that we are agnostic to swapping models, we mean it.

That’s one of our core principles (along with trust being the other core value, not something bolted as an afterthought, made explicit because it’s backed by our formal Customer Bill of Rights). That is, from the ground up aiHelpDesk was engineered to not depend on any particular LLM, treat models as a disposable commodity and make it as one of our explicit goals to allow seamless, easy, non-engineering migrations from one model to another, be it with the models of the same vendor (e.g. going from Sonnet to Opus) or switching the model vendors altogether (e.g. from Gemini to Anthropic).

I already wrote about it a number of times, advising customers to invest into their own teams and intelligence in favor building dependencies on a specific model’s feature. This in turn often results in customers and prospects asking to actually prove that models are indeed easily replaceable. Specific steps that an aiHelpDesk customer can execute on their own without taking our word for it.

This request typically doesn’t come out of some idle, academic curiosity. More often than not, there’s a business urgency to it, accompanied with the specifics of what model / vendor a customer is not happy with and wants to migrate away from. In particular, the request/question that we answer often is how to ditch Gemini and move up to one of the frontier model XYZ. That XYZ varies, but the model to migrate away from is often Google’s, so instead of repeating ourselves we figured we can just share the standard recipe we follow.

The recipe below works well for us and a broader community can perhaps benefit from our experience, so here goes: the breakup without drama or tears, start to finish, using the real tooling, not a diagram.

The breakup recipe

The scenario: your team has been evaluating Google suite of products for three months and in particular your database fleet has been relying on Gemini for triage and remediation. In this piece we won’t be speculating as to why customers are eager to ditch Google, but let’s assume the trial hasn’t been particuarly successful or Anthropic shipped Fable or other model that you want to evaluate or for any other reaosn, you decide to abandon ship. Now, one thing you don’t want is a vibe-based migration. You want confidence. You want a number to avoid wasting another three months of your team.

I use “from Gemini to XYZ” in this article because that’s what we see in the field, but the same principles apply to any from/to models as well.

Step 0: What you’re actually proving

Before touching anything, be precise about the claim. You are not about to prove “the new model is smarter according to MMLU, DeepEval or similar generic benchmarks”. You’re about to prove something narrower and more useful to you: the new model reasons about your specific failure modes at least as consistently as the one it’s replacing. That’s it. Consistency, not Intelligence and as it applies to your databases and your workload in particular.

The Consistency Gate doesn’t have an opinion about which model is “better”. It has a pass/fail threshold on a pass rate, a confidence spread and it doesn’t care whose logo is on the API response.

Step 1: Baseline the incumbent

Before you can claim a successful upgrade, you need a number for what you’re upgrading from. The baseline. This is not optional because after the upgrade you need something to refer back to how things looked with Google, compare the two numbers and claim the improvement. People that report problems with swapping models, in our experience, skip this step. You don’t want to be one of them. No victory dance for you if you have no baseline to compare to.

If Gemini doesn’t already have a cert on file already, generate one:

  export HELPDESK_GATEWAY_URL=http://localhost:8080
  export HELPDESK_MODEL_VENDOR=google
  export HELPDESK_MODEL_NAME=gemini-3.5-flash
  export HELPDESK_API_KEY=$GOOGLE_API_KEY

  make recertify RECERTIFY_REPEAT=5

  go run ./testing/cmd/faulttest vault accuracy db-lock-contention \
    --gateway $HELPDESK_GATEWAY_URL \
    --api-key $HELPDESK_API_KEY

That -repeat 5 under the hood is five independent inject→diagnose→score cycles per fault, not one lucky run. Once faulttest finishes, pull up the receipt:

Triage consistency
    Fault         : db-lock-contention  (Transaction lock chain / deadlock)
    Verdict       : STABLE
    Runs          : 5
    Pass rate     : 80%
    Conf range    : 8pp  (primary hypothesis, passing runs only)
    Clean         : yes
    Playbook      : pbs_lock_chain_triage
    Diagnosis model: gemini-3.5-flash
    Judge model   : gemini-3.5-flash
    Tested at     : 2026-10-01 14:02 EDT  (3 days ago)

That’s your baseline, on the record, not in a Slack thread.

Now, note that in the printout above the diagnosis model (the model used for triaging the the problem, reviewing the artifacts, coming up with the primary and alternative hypotheses, attaching the confidence level to each, etc.)… is the same model as the judge model. Is this a typo?

Step 2: Pin an independent judge

Well, let’s step back and review first if we need a judge at all, shall we?

Breakups aren’t easy, as they say, so it helps to have an independent, unbiased, fair and preferably third-party observer overseeing the process. The judge offers a second opinion, scoring both the outgoing and incoming candidates, so yes, we recommend a judge for swapping models.

And yes, this is still a model judging a model, so not ideal as I pointed out earlier, but for this type of work we find that judging works surprisingly well.

Now, the model that assesses the quality of the diagnosis to see if it’s correct or not, doesn’t have to be the same model doing the actual diagnosis. In fact, it’s usually preferrable if it isn’t. You can absolutely use the same model for both and, somewhat surprisingly, a model that isn’t your first choice for primary reasoning can still do a perfectly adequate job as a judge, but it’s it isn’t ideal.

In practial terms we’ve seen three schemes, in descending order of rigor:

• Ideal, in principle: pin a genuinely independent third-party judge, which is a vendor that’s neither the one you’re leaving nor the one you’re moving to, used for both the baseline run and the new model’s evaluation (aka re-certify). This is the textbook-correct setup: it removes judge variance as a confound entirely, so if you go from Gemini to Fable, you probably want the judge to be Open AI or similar. Now, we don’t offer support for Open AI yet. agentutil.NewLLM is the function every model call in aiHelpDesk routes through, diagnosis or judge and as it stands today, it doesn’t yet know about Open AI, but we are actively looking for an excuse to add it. Please consider it and open a FR with us and we’d be happy to oblige 😉.
• Compromise 1: same model used for both roles where you effectively use your primary reasoning model as its own judge. Yes, ideally, the referee shouldn’t be the vendor you’re leaving behind and we can see why many move away from Gemini, but as a judge it often does good enough of a job to offer a qualified second opinion. So not the most scientific option because you’re letting the model grade its own breakup letter, but it works better than you’d expect in practice. If you go down this path, just expect some judge-specific noise and budget for more repeated runs to average it out.
• Compromise 2: cross-reference, which is what we see most used today: get Anthropic to judge the outgoing Gemini model and Gemini to judge the incoming Anthropic model. Neither judge shares a vendor with the model it’s scoring, so you still get rid of the worst version of the conflict-of-interest problem, i.e. the model never grades its own work. Now, with this option you get the true third-party independence and not the true apples-to-apples comparison because if a referee is not constant across both runs, it’s more difficult to tell whether the score shift comes from the diagnosis model changing or the judge changing, but in practice this option typically works well for most use cases.

To summarize: option 1 is ideal because keeping/pinning the judge constant across the baseline run and the comparison runs makes it easy to see the only thing changing: the model. Swap the judge along with the diagnosis model and it’s more difficult to tell which change produced the score difference.

Now, if your judge is different from the primary reasoning model, remember to set the -judge-model, -judge-vendor and -judge-api-key because those are the separate flags from the diagnosis model flags. But there’s a quirk. Experienced aiHelpDesk customers typically rely on the make recertify’s [code link] Makefile target, but while the three flags above are the real judge flags for faulttest, the make recertify’s own wrapper doesn’t expose them yet (there’s already an approved FR to fix this, slatted to land in the upcoming v0.31 release, see this commit). But for now, as of this writing, the wrapper only has a blanket RECERTIFY_JUDGE=1 toggle, which defaults the judge to whatever HELPDESK_MODEL_NAME / HELPDESK_MODEL_VENDOR happen to be set to at the time. That doesn’t work for this particular exercise because it would silently make the judge and the model under test the same vendor once you swap in Step 3.

As such, to actually pin a judge that stays constant across both runs, skip the Makefile target and call the CLI directly, setting -judge-model, -judge-vendor and -judge-api-key explicitly:

  export HELPDESK_JUDGE_VENDOR=google
  export HELPDESK_JUDGE_MODEL=gemini-3.5-flash
  export HELPDESK_JUDGE_API_KEY=$GOOGLE_API_KEY

(This just shows the shell env vars of convenience here , passed explicitly in the command below, not anything the harness reads directly)

Step 3: The actual swap

This is often the shortest and the fastest step because the actual model swap is trivial. Just restart the agents with an env var pointing to the new/incoming model:

# Stop the agents currently running on Gemini
  pkill -f "agents/database"
  pkill -f "agents/sysadmin"

# Restart them pointed at the new model
# Here are the simplified instructions for a Database and SysAdmin Agents:
  HELPDESK_MODEL_VENDOR=anthropic \
  HELPDESK_MODEL_NAME=claude-fable-5-1 \
  HELPDESK_API_KEY=$ANTHROPIC_API_KEY \
  go run ./agents/database &

  HELPDESK_MODEL_VENDOR=anthropic \
  HELPDESK_MODEL_NAME=claude-fable-5-1 \
  HELPDESK_API_KEY=$ANTHROPIC_API_KEY \
  go run ./agents/sysadmin &

Nothing about the Playbook library changed. Nothing about the Tool Registry changed. You just restarted two processes with a different model. That’s the entire blast radius of a model swap, by design 😉. Now, the above is for running from source, but see specifics of this for your particular platform: host (VM/Bare Metal), containers (Docker/Podman) and K8s.

Step 4: Recertify the new model

This is the step that is almost identical to step 1, but on the new model:

  HELPDESK_MODEL_VENDOR=anthropic \
  HELPDESK_MODEL_NAME=claude-fable-5-1 \
  HELPDESK_API_KEY=$ANTHROPIC_API_KEY \
  go run ./testing/cmd/faulttest run \
    --external --via-gateway \
    --gateway $HELPDESK_GATEWAY_URL \
    --repeat 5 \
    --approval-mode force \
    --judge \
    --judge-vendor $HELPDESK_JUDGE_VENDOR \
    --judge-model $HELPDESK_JUDGE_MODEL \
    --judge-api-key $HELPDESK_JUDGE_API_KEY

… and get the receipts:

  Triage consistency
    Fault         : db-lock-contention  (Transaction lock chain / deadlock)
    Verdict       : STABLE
    Runs          : 5
    Pass rate     : 100%
    Conf range    : 4pp  (primary hypothesis, passing runs only)
    Clean         : yes
    Playbook      : pbs_lock_chain_triage
    Diagnosis model: claude-fable-5-1
    Judge model   : claude-opus-5-5
    Tested at     : 2026-10-04 09:15 EDT  (0 days ago)

And this is a good receipt! STABLE, 100% pass rate, 4pp confidence spread against Gemini’s 8pp, meaning that the confidence spread is narrow and so the incoming model (Anthropic’s) is more consistent in its diagnosis, averaged across 5 sample runs.

Same playbook. Same fault. Different model, independently scored by a third model (again, ideally, with the judge that was never swapped). That side-by-side is the whole argument the old post was making, just with the receipt of specific steps instead of the mere assertion.

Run this across your real fault set, not just one cherry-picked fault. That is, kick off make recertify with no FAULT_IDS filter that runs every external-compatible fault in the catalog [code link].

Watch out for any regressions compared to the outgoing Gemini model. Now, the ones that come back UNSTABLE aren’t a reason to panic and to abandon the swap. They’re your worklist. “This specific playbook is unreliable under this specific new model” is a tuning task. “We think the new model might be worse” is a guess. Don’t guess. Guessing is what leads to questionable or botched model swap projects. Get the numbers instead. And if the worklist happened to be long with many diagnoses worse than your outgoing/Gemini model, consider staying with Gemini (although we are yet to encounter that situation 😉).

Step 5: What you’re still not allowed to conclude

Here’s the part the original post under-specified and worth being exact about now: a STABLE cert tells you the new model’s reasoning is consistent. And indeed, in our experience of what we see in a field, ditching Gemini often leads to better, more consistent triage results, but it tells you nothing about whether the new model is safe to run the remediation and in particular, the destructive operations, be it SQL drop statements, a restart_container or a set_archive_command.

This is because that was never the model’s call to begin with, on Gemini or on Anthropic. Every write and destructive action still goes through

• the same Policy Engine check
• the same Informed Consent gate
• the same post-action Objective Evidence verification it went through under the old model

None of that machinery cares which vendor issued the token that proposed the action, because none of it was ever built to trust the model that far. And that’s one of aiHelpDesk core design principles, not an afterthought.

The one-liner TL;DR: of this whole recipe:

• You proved that for triage purposes the model is replaceable
• You proved that the thing that actually executes, which is a the Tool Registry, doesn’t care about models and never needs model replacing in the first place 😉
• This is because the model is never the executor
• The model is the runtime the playbook runs on. Swap it, and the parts of the system responsible for actually fixing anything in your infrastructure don’t even notice

And that’s the breakup. Total elapsed time for a model swap, per the original post’s own math: an afternoon, non-engineering affair, not a quarter worth of effort (where you still may feel uncomfortable with no numbers to prove it either way).

Mental model: the three layers

This section may feel almost orthogonal to the main topic of this post, so if you are here for the Gemini breakup recipe, feel free to skip it altogether, but it may be worth elaborating on the last point that the model in aiHelpDesk is strictly the runtime layer, never an executor.

I failed to make this point clear in the original post, but it’s important to realize that the LLM just reads the Tool Registry’s tool output (e.g. pg_stat_archiver numbers or pgbackrest info JSON using the examples from the backup post), forms a hypothesis and decides to call set_archive_command with a specific value. Everything after that decision, i.e. the actual `ALTER SYSTEM SET archive_command = $1; SELECT pg_reload_conf()`, the Policy Engine check against resource/tag/purpose, the Informed Consent gate that can block the call pending a human’s explicit approval, the Objective Evidence check afterward confirming the claimed effect actually happened… none of that is the LLM “executing” anything.

The actual execution is exclusively the deterministic Go code in the Tool Registry. The LLM’s output at that point is a function call it proposes. It never touches the database, the kubectl API or the SSH session directly.

The reason we built all that AI harness is specifically because something sits between the LLM’s decision and the real-world changes. And that something is where execution actually lives. That something is what makes the Bill of Rights credible: the thing that acts on your infrastructure is auditable, deterministic, policy-gated code, not a stochastic model’s whim.

So my mental model of aiHelpDesk is three layers:

• 🧠 Knowledge. The Playbook (versioned, stability-certified, stored in the Vault). Static. Doesn’t change per-incident
• 💡 Reasoning. That’s the LLM. Interprets that static knowledge against live, never-identical evidence, forms a hypothesis, picks a playbook and which of the playbook’s pre-approved paths applies (or escalates because none do). This is real, non-trivial work a static playbook can’t do on its own, but it’s deciding, not doing
• ⚡ Execution. The Tool Registry. Fully deterministic. The only layer that ever touches real infrastructure and the only layer that the Policy Engine and the Objective Evidence framework trust

Collapsing “reasoning” and “execution” into one role and calling it “the LLM is the executor” is simpler, but very coarse and it leads to believe that a model actually has the power and ability to do things. It doesn’t. Whichever word you use for that middle layer (orchestrator, reasoner, interpreter), “executor” is the one label that’s actively misleading and it’s the one claim this project’s entire governance design exists to contradict.

Try it yourself, take aiHelpDesk for a spin with a 10-minute demo.

aiHelpDesk is open source. We invented and pioneered the concepts of Operational SRE/DBA Flywheel (blog post, doc), Triage Consistency Certification, 3D Certs, Informed Consent, Second Opinion, Bill of Rights, etc. See here how the rights mentioned above map to our Flywheel.

The vault commands, the judge, the playbook diff, the full audit trail, the calibration data quality banner, the model-scoped stability certs and vault cert-compare… are not add-ons. They are the product. Trust is the product. The judgment layer covers what to do when the AI’s own improvement proposals fail. And why that case is where the most durable operational knowledge gets encoded.

Why aiHelpDesk Playbooks are trustworthy?

Because we give you, the customer, the ability to vet, verify and improve them through the methodology that we refer to as Operational SRE/DBA Flywheel, see here and here for details.

And yes, as a customer, you can not only confirm that the playbooks that you get with aiHelpDesk out of the box work for your environment, but you can easily bring your own. Both: BYO playbooks (by either importing your existing runbooks or cloning and customizing one of our’s or by creating one from scratch) + BYO faults as well.

Because nobody knows your specific databases, your environment and your workload with your upstream/downstream apps better than you do. You know how your database fails. Vendors don’t. At aiHelpDesk, we give you an option to create your own faults and add your own playbooks to triage and rectify them (in addition to the system playbooks we ship, of course).

And yes, we don’t depend on a particular model or a model provider. aiHelpDesk is model-neutral. From our standpoint, the LLMs are a disposable commodity. Flip from Gemini to Anthropic and aiHelpDesk should continue to give you exactly the same diagnosis and remediation. Anything shorter than that is a P0 bug.

Related Reading

• aiHelpDesk Flywheel official documentation
• When the Watchman Can’t See: How a Silent Backup Failure Almost Stayed Silent Forever: Your dashboard was green and yet your backups had been dead for two weeks. Nobody lied to you, but nobody asked the right question either
• AWS Says It’s Wrong, Not Broken. But Who Does the Grading?
  Naming the problem is progress. Grading it with another language model is the same problem wearing a lab coat
• Your AI Agent Has No Alibi and It Just Lied to You. Stop Grading It on Whether It Finished the Task: A model that fabricates a tool call and still fixes your database isn’t a success story. It’s a governance failure you haven’t found yet. This is a story of a DB agent that narrated a config check it never ran. And a system built to distrust its own AI

As of this writing, aiHelpDesk is available in Beta. If you run PostgreSQL in production and would like to get help in preparing for the avalanche, consider aiHelpDesk. Reach out to us at info@aiHelpDesk.biz and we’ll be happy to show you what it looks like in practice.

How to Ditch Gemini: A Step-by-Step Breakup Recipe was originally published in Level Up Coding on Medium, where people are continuing the conversation by highlighting and responding to this story.