<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Google Cloud</title>
    <description>The latest articles on DEV Community by Google Cloud (googlecloud).</description>
    <link>https://dev.to/googlecloud</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Forganization%2Fprofile_image%2F809%2Fc7814399-cf4a-4dc9-9f12-d0a97ed21bf6.png</url>
      <title>DEV Community: Google Cloud</title>
      <link>https://dev.to/googlecloud</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/googlecloud"/>
    <language>en</language>
    <item>
      <title>🎞️ How to compare documents</title>
      <dc:creator>Laurent Picard</dc:creator>
      <pubDate>Fri, 13 Nov 2026 16:13:37 +0000</pubDate>
      <link>https://dev.to/googlecloud/how-to-compare-documents-2bjj</link>
      <guid>https://dev.to/googlecloud/how-to-compare-documents-2bjj</guid>
      <description>&lt;h3&gt;
  
  
  🍿 Just a moment #02
&lt;/h3&gt;

&lt;blockquote&gt;
&lt;p&gt;Have you ever had to compare two versions of a complex document? What if one is a digital PDF and the other is a scan, spanning hundreds of pages, with different layouts and pagination? In this episode, see how Gemini’s native multimodality spots subtle differences, from tiny typos to $100M contract updates, in a single prompt.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;  &lt;iframe src="https://www.youtube.com/embed/-4uuPJ_YNIc" width="710" height="399"&gt;
  &lt;/iframe&gt;
&lt;/p&gt;

&lt;p&gt;Possible prompt template (used in the video):&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight markdown"&gt;&lt;code&gt;&amp;lt;DOCUMENT_A&amp;gt;
…
&amp;lt;/DOCUMENT_A&amp;gt;
&amp;lt;DOCUMENT_B&amp;gt;
…
&amp;lt;/DOCUMENT_B&amp;gt;

TASK:
Perform an exhaustive, character-by-character textual comparison between DOCUMENT_A and DOCUMENT_B.

CRITICAL REQUIREMENTS:
1. You must systematically verify every numbered paragraph sequentially from start to finish.
2. You must catch every single difference, including single-character changes, minor typos, spelling corrections (e.g., missing letters in common words), and punctuation changes, in addition to substantive changes (numbers, dates, names).

OUTPUT FORMAT:
Difference #[1, 2, 3…]
Location: [location]
Evidence A: "[short quote from Document A]" with difference in *bold*
Evidence B: "[short quote from Document B]" with difference in *bold*
Explanation: [brief description]
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  More
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;🔗 Document understanding → &lt;a href="https://g.dev/cloud/document-understanding" rel="noopener noreferrer"&gt;https://g.dev/cloud/document-understanding&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;🔗 Gemini models → &lt;a href="https://g.dev/cloud/gemini-models" rel="noopener noreferrer"&gt;https://g.dev/cloud/gemini-models&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;🔔 Subscribe to Google Cloud Tech → &lt;a href="https://goo.gle/GoogleCloudTech" rel="noopener noreferrer"&gt;https://goo.gle/GoogleCloudTech&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;




</description>
      <category>llm</category>
      <category>machinelearning</category>
      <category>videos</category>
      <category>justamoment</category>
    </item>
    <item>
      <title>️🎞️ What is native multimodality?</title>
      <dc:creator>Laurent Picard</dc:creator>
      <pubDate>Fri, 13 Nov 2026 10:00:19 +0000</pubDate>
      <link>https://dev.to/googlecloud/what-is-native-multimodality-3091</link>
      <guid>https://dev.to/googlecloud/what-is-native-multimodality-3091</guid>
      <description>&lt;h3&gt;
  
  
  🍿 Just a moment #01
&lt;/h3&gt;

&lt;blockquote&gt;
&lt;p&gt;Did you know there are three tokenizers involved when Gemini processes multimodal inputs? In this episode, we peek under the hood to see how text, vision, and audio tokenizers work together.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;  &lt;iframe src="https://www.youtube.com/embed/WxdwhoUxBZU" width="710" height="399"&gt;
  &lt;/iframe&gt;
&lt;/p&gt;

&lt;h3&gt;
  
  
  More
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;🔗 Explore Gemini models → &lt;a href="https://g.dev/cloud/gemini-models" rel="noopener noreferrer"&gt;https://g.dev/cloud/gemini-models&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;🔔 Subscribe to Google Cloud Tech → &lt;a href="https://goo.gle/GoogleCloudTech" rel="noopener noreferrer"&gt;https://goo.gle/GoogleCloudTech&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;




</description>
      <category>llm</category>
      <category>machinelearning</category>
      <category>videos</category>
      <category>justamoment</category>
    </item>
    <item>
      <title>AlloyDB Integration for Modern Java Apps — Connectivity and Auth</title>
      <dc:creator>Alexis MP</dc:creator>
      <pubDate>Thu, 12 Nov 2026 06:56:01 +0000</pubDate>
      <link>https://dev.to/googlecloud/alloydb-integration-for-modern-java-apps-connectivity-and-auth-5721</link>
      <guid>https://dev.to/googlecloud/alloydb-integration-for-modern-java-apps-connectivity-and-auth-5721</guid>
      <description>&lt;p&gt;&lt;em&gt;(this is part of a&lt;/em&gt; &lt;a href="https://dev.to/googlecloud/a-modern-java-developers-guide-for-google-cloud-databases-3ckg"&gt;&lt;em&gt;Java tour of Google Cloud Databases series&lt;/em&gt;&lt;/a&gt;&lt;em&gt;)&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;&lt;a href="https://cloud.google.com/products/alloydb" rel="noopener noreferrer"&gt;AlloyDB for PostgreSQL&lt;/a&gt; is Google Cloud’s database for enterprise-grade workloads with full PostgreSQL compatibility and a cloud-native architecture. Connecting a Java application to AlloyDB is done through the &lt;strong&gt;AlloyDB Java Connector&lt;/strong&gt; , a crucial component that goes beyond a standard JDBC driver.&lt;/p&gt;

&lt;h3&gt;
  
  
  The AlloyDB Java Connector’s Magic
&lt;/h3&gt;

&lt;p&gt;The &lt;code&gt;alloydb-jdbc-connector&lt;/code&gt; is a client library that functions as a smart socket factory for the standard PostgreSQL JDBC driver. It abstracts away the establishment of a secure database connection and takes care of :&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;mTLS 1.3 Encryption:&lt;/strong&gt; automatic secure TLS 1.3 connection to the AlloyDB instance (no need to manage SSL/TLS certificates here).&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;IAM-Based Authorization:&lt;/strong&gt; as a better alternative to (static) database passwords, the connector uses IAM for authentication and ADC (&lt;a href="https://docs.cloud.google.com/docs/authentication/provide-credentials-adc" rel="noopener noreferrer"&gt;Application Default Credentials&lt;/a&gt;) to obtain a short-lived access token.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Dynamic IP Resolution:&lt;/strong&gt; The connector communicates with the AlloyDB Admin API to fetch the current IP address (a Private IP, a Public IP, or a Private Service Connect) of the database instance, so you do not need to handle it explicitly, no hard-coding or environment variable needed. This makes events like failovers or maintenance easier to manage.&lt;/li&gt;
&lt;/ul&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;NOTE on using the Java Connector&lt;/strong&gt; rather than the &lt;strong&gt;Auth Proxy:&lt;/strong&gt; for JVM applications, the Java Connector runs in-process as a JDBC &lt;code&gt;SocketFactory&lt;/code&gt;, eliminating the need to deploy and manage a separate sidecar container. The standalone Auth Proxy is best for non-JVM tools, legacy apps where you cannot modify the JDBC classpath, or environments with strict GraalVM native-image constraints.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fzbevcq3we4zflhnm5sds.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fzbevcq3we4zflhnm5sds.png" width="800" height="446"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  But first, some groundwork
&lt;/h3&gt;

&lt;p&gt;I’ll assume that you already have a project with an existing &lt;strong&gt;AlloyDB Cluster&lt;/strong&gt; and primary Instance running in a specific region. If you don’t have that, check out the &lt;a href="https://cloud.google.com/alloydb/docs/cluster-create" rel="noopener noreferrer"&gt;AlloyDB cluster creation guide&lt;/a&gt; or head directly to the console’s &lt;a href="https://console.cloud.google.com/alloydb/get-started" rel="noopener noreferrer"&gt;getting started section&lt;/a&gt;. You’ll need the admin password, the &lt;code&gt;roles/alloydb.admin&lt;/code&gt; role, some tables in the public schema (use &lt;a href="https://neon.com/postgresql/postgresql-getting-started/postgresql-sample-database" rel="noopener noreferrer"&gt;DVD Rental&lt;/a&gt; if you don’t have anything), and of course be able to reach the AlloyDB instance from your local machine (or from Cloud Shell).&lt;/p&gt;

&lt;p&gt;I also strongly recommend that you install these in your favorite agent tool as you progress through the code and instructions of this series :&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;the &lt;a href="https://cloud.google.com/products/data-agent-kit" rel="noopener noreferrer"&gt;Data Agent Kit&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;the &lt;a href="https://cloud.google.com/blog/topics/developers-practitioners/introducing-the-google-cloud-developer-plugin-for-ai-coding-agents" rel="noopener noreferrer"&gt;Developer Plugin for AI Coding Agents&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;If you like learning from codelabs, try: &lt;a href="https://codelabs.developers.google.com/connect-to-alloydb-on-cloudrun" rel="noopener noreferrer"&gt;Connect your application to AlloyDB data and deploy on Cloud Run&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Let’s start with creating a Service Account, the identity your Java application will use:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;gcloud iam service-accounts create java-alloydb-connector \
    --display-name=&lt;span class="s2"&gt;"Java Application DB Connector"&lt;/span&gt;
&lt;span class="c"&gt;# locally you'll need to impersonate this account or use your own IAM user&lt;/span&gt;
&lt;span class="c"&gt;# on Cloud Run/GCE/GKE, attach this SA to the compute service/workload&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;


&lt;p&gt;Then grant it the required IAM roles: both &lt;code&gt;roles/alloydb.client&lt;/code&gt; (needed to request an ephemeral mTLS certificate and connect to the instance) and &lt;code&gt;roles/serviceusage.serviceUsageConsumer&lt;/code&gt; (required for IAM database authentication):&lt;br&gt;
&lt;/p&gt;
&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;gcloud projects add-iam-policy-binding &lt;span class="o"&gt;&amp;lt;&lt;/span&gt;PROJECT_ID&lt;span class="o"&gt;&amp;gt;&lt;/span&gt; \
    --member=&lt;span class="s2"&gt;"serviceAccount:java-alloydb-connector@&amp;lt;PROJECT_ID&amp;gt;.iam.gserviceaccount.com"&lt;/span&gt; \
    --role=&lt;span class="s2"&gt;"roles/alloydb.client"&lt;/span&gt;

gcloud projects add-iam-policy-binding &lt;span class="o"&gt;&amp;lt;&lt;/span&gt;PROJECT_ID&lt;span class="o"&gt;&amp;gt;&lt;/span&gt; \
    --member=&lt;span class="s2"&gt;"serviceAccount:java-alloydb-connector@&amp;lt;PROJECT_ID&amp;gt;.iam.gserviceaccount.com"&lt;/span&gt; \
    --role=&lt;span class="s2"&gt;"roles/serviceusage.serviceUsageConsumer"&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;


&lt;p&gt;Now enable IAM authentication on the instance to tell your AlloyDB instance to accept IAM tokens, not just passwords (this takes effect immediately, no instance restart required):&lt;br&gt;
&lt;/p&gt;
&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;gcloud alloydb instances update &lt;span class="o"&gt;&amp;lt;&lt;/span&gt;INSTANCE_ID&lt;span class="o"&gt;&amp;gt;&lt;/span&gt; \
    --cluster=&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;CLUSTER_ID&lt;span class="o"&gt;&amp;gt;&lt;/span&gt; \
    --region=&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;REGION&lt;span class="o"&gt;&amp;gt;&lt;/span&gt; \
    --database-flags=alloydb.iam_authentication=on
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;


&lt;p&gt;Finally, register the IAM principal inside AlloyDB and grant it PostgreSQL table permissions. Notice how the .gserviceaccount.com suffix must be stripped for Service Accounts!&lt;/p&gt;

&lt;p&gt;You can register the IAM user directly with gcloud (this automatically grants the internal alloydbiamuser role):&lt;br&gt;
&lt;/p&gt;
&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;gcloud alloydb users create &lt;span class="s2"&gt;"java-alloydb-connector@&amp;lt;PROJECT_ID&amp;gt;.iam"&lt;/span&gt; \
    --cluster=&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;CLUSTER_ID&lt;span class="o"&gt;&amp;gt;&lt;/span&gt; \
    --region=&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;REGION&lt;span class="o"&gt;&amp;gt;&lt;/span&gt; \
    --type=IAM_BASED
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;


&lt;p&gt;Then log in to your database as an admin (e.g., postgres) and grant schema and table privileges (IAM users must be created via gcloud, Terraform, or the Console because AlloyDB blocks granting the internal alloydbiamuser role directly in SQL):&lt;br&gt;
&lt;/p&gt;
&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="c1"&gt;-- Connect to your target database first&lt;/span&gt;
\&lt;span class="n"&gt;c&lt;/span&gt; &lt;span class="n"&gt;my_database&lt;/span&gt;

&lt;span class="c1"&gt;-- Grant permissions (IAM users are created with NO privileges by default)&lt;/span&gt;
&lt;span class="k"&gt;GRANT&lt;/span&gt; &lt;span class="no"&gt;CONNECT&lt;/span&gt; &lt;span class="k"&gt;ON&lt;/span&gt; &lt;span class="k"&gt;DATABASE&lt;/span&gt; &lt;span class="s2"&gt;"my_database"&lt;/span&gt; &lt;span class="k"&gt;TO&lt;/span&gt; &lt;span class="s2"&gt;"java-alloydb-connector@&amp;lt;PROJECT_ID&amp;gt;.iam"&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="k"&gt;GRANT&lt;/span&gt; &lt;span class="no"&gt;USAGE&lt;/span&gt; &lt;span class="k"&gt;ON&lt;/span&gt; &lt;span class="k"&gt;SCHEMA&lt;/span&gt; &lt;span class="n"&gt;public&lt;/span&gt; &lt;span class="k"&gt;TO&lt;/span&gt; &lt;span class="s2"&gt;"java-alloydb-connector@&amp;lt;PROJECT_ID&amp;gt;.iam"&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="k"&gt;GRANT&lt;/span&gt; &lt;span class="k"&gt;SELECT&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="k"&gt;INSERT&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="k"&gt;UPDATE&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="k"&gt;DELETE&lt;/span&gt; &lt;span class="k"&gt;ON&lt;/span&gt; &lt;span class="k"&gt;ALL&lt;/span&gt; &lt;span class="no"&gt;TABLES&lt;/span&gt; &lt;span class="k"&gt;IN&lt;/span&gt; &lt;span class="k"&gt;SCHEMA&lt;/span&gt; &lt;span class="n"&gt;public&lt;/span&gt; &lt;span class="k"&gt;TO&lt;/span&gt; &lt;span class="s2"&gt;"java-alloydb-connector@&amp;lt;PROJECT_ID&amp;gt;.iam"&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;


&lt;p&gt;Make sure to use double quotes in SQL as they’re required because of the &lt;code&gt;@&lt;/code&gt; and dots.&lt;/p&gt;
&lt;h3&gt;
  
  
  How-To: Connecting to AlloyDB with a JDBC or Spring Boot app
&lt;/h3&gt;

&lt;p&gt;Configuring HikariCP to work with the AlloyDB Java Connector requires specifying key properties that instruct the standard PostgreSQL driver to delegate socket creation to the connector. This requires Java 11 or higher (but I hope you’re using Java 17, 21 LTS, or higher!).&lt;/p&gt;
&lt;h4&gt;
  
  
  “Plain” JDBC
&lt;/h4&gt;

&lt;p&gt;First, dependencies (&lt;code&gt;pom.xml&lt;/code&gt;):&lt;br&gt;
&lt;/p&gt;
&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight xml"&gt;&lt;code&gt;&lt;span class="nt"&gt;&amp;lt;dependencies&amp;gt;&lt;/span&gt;
    &lt;span class="nt"&gt;&amp;lt;dependency&amp;gt;&lt;/span&gt;
        &lt;span class="nt"&gt;&amp;lt;groupId&amp;gt;&lt;/span&gt;org.postgresql&lt;span class="nt"&gt;&amp;lt;/groupId&amp;gt;&lt;/span&gt;
        &lt;span class="nt"&gt;&amp;lt;artifactId&amp;gt;&lt;/span&gt;postgresql&lt;span class="nt"&gt;&amp;lt;/artifactId&amp;gt;&lt;/span&gt;
        &lt;span class="nt"&gt;&amp;lt;version&amp;gt;&lt;/span&gt;42.7.14&lt;span class="nt"&gt;&amp;lt;/version&amp;gt;&lt;/span&gt;
    &lt;span class="nt"&gt;&amp;lt;/dependency&amp;gt;&lt;/span&gt;

    &lt;span class="nt"&gt;&amp;lt;dependency&amp;gt;&lt;/span&gt;
        &lt;span class="nt"&gt;&amp;lt;groupId&amp;gt;&lt;/span&gt;com.google.cloud.alloydb&lt;span class="nt"&gt;&amp;lt;/groupId&amp;gt;&lt;/span&gt;
        &lt;span class="nt"&gt;&amp;lt;artifactId&amp;gt;&lt;/span&gt;alloydb-jdbc-connector&lt;span class="nt"&gt;&amp;lt;/artifactId&amp;gt;&lt;/span&gt;
        &lt;span class="nt"&gt;&amp;lt;version&amp;gt;&lt;/span&gt;1.4.0&lt;span class="nt"&gt;&amp;lt;/version&amp;gt;&lt;/span&gt;
    &lt;span class="nt"&gt;&amp;lt;/dependency&amp;gt;&lt;/span&gt;

    &lt;span class="nt"&gt;&amp;lt;dependency&amp;gt;&lt;/span&gt;
        &lt;span class="nt"&gt;&amp;lt;groupId&amp;gt;&lt;/span&gt;com.zaxxer&lt;span class="nt"&gt;&amp;lt;/groupId&amp;gt;&lt;/span&gt;
        &lt;span class="nt"&gt;&amp;lt;artifactId&amp;gt;&lt;/span&gt;HikariCP&lt;span class="nt"&gt;&amp;lt;/artifactId&amp;gt;&lt;/span&gt;
        &lt;span class="nt"&gt;&amp;lt;version&amp;gt;&lt;/span&gt;7.1.0&lt;span class="nt"&gt;&amp;lt;/version&amp;gt;&lt;/span&gt;
    &lt;span class="nt"&gt;&amp;lt;/dependency&amp;gt;&lt;/span&gt;
&lt;span class="nt"&gt;&amp;lt;/dependencies&amp;gt;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;


&lt;p&gt;Here’s a complete sample that configures HikariCP with the AlloyDB &lt;code&gt;SocketFactory&lt;/code&gt; and queries the film table from the DVD Rental schema to verify both IAM authentication and table privileges:&lt;/p&gt;


&lt;div class="ltag_gist-liquid-tag"&gt;
  
&lt;/div&gt;



&lt;blockquote&gt;
&lt;p&gt;Notice that the JDBC URL &lt;code&gt;jdbc:postgresql:///%s&lt;/code&gt; contains no host information. This is a key indicator that the &lt;code&gt;SocketFactory&lt;/code&gt; is in control, as it is responsible for finding and connecting to the correct instance IP address.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;For connectivity troubleshooting, check out the bottom of this post and make sure to read the &lt;a href="https://cloud.google.com/alloydb/docs/connection-overview" rel="noopener noreferrer"&gt;AlloyDB Connection Overview&lt;/a&gt;.&lt;/p&gt;

&lt;h4&gt;
  
  
  Using Spring Boot
&lt;/h4&gt;

&lt;p&gt;Spring Boot simplifies database connectivity by automatically configuring a HikariCP connection pool based on your properties, allowing you to integrate the AlloyDB Connector with almost no manual coding.&lt;/p&gt;

&lt;p&gt;You only need to add the AlloyDB Connector to your existing Spring Boot project. Spring Boot’s &lt;code&gt;spring-boot-starter-data-jpa&lt;/code&gt; or &lt;code&gt;spring-boot-starter-jdbc&lt;/code&gt; already includes HikariCP (just ensure &lt;code&gt;org.postgresql:postgresql&lt;/code&gt; is on the classpath):&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight xml"&gt;&lt;code&gt;&lt;span class="nt"&gt;&amp;lt;dependency&amp;gt;&lt;/span&gt;
    &lt;span class="nt"&gt;&amp;lt;groupId&amp;gt;&lt;/span&gt;com.google.cloud.alloydb&lt;span class="nt"&gt;&amp;lt;/groupId&amp;gt;&lt;/span&gt;
    &lt;span class="nt"&gt;&amp;lt;artifactId&amp;gt;&lt;/span&gt;alloydb-jdbc-connector&lt;span class="nt"&gt;&amp;lt;/artifactId&amp;gt;&lt;/span&gt;
    &lt;span class="nt"&gt;&amp;lt;version&amp;gt;&lt;/span&gt;1.4.0&lt;span class="nt"&gt;&amp;lt;/version&amp;gt;&lt;/span&gt;
&lt;span class="nt"&gt;&amp;lt;/dependency&amp;gt;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;


&lt;p&gt;Using &lt;code&gt;application.properties&lt;/code&gt; is the standard way to define your connection string within your project:&lt;br&gt;
&lt;/p&gt;
&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight properties"&gt;&lt;code&gt;&lt;span class="c1"&gt;# No host or port is needed; the SocketFactory handles it.
&lt;/span&gt;&lt;span class="na"&gt;spring.datasource.url&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;jdbc&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="n"&gt;postgresql&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="c1"&gt;///my_database?socketFactory=com.google.cloud.alloydb.SocketFactory&amp;amp;alloydbInstanceName=projects/&amp;lt;PROJECT_ID&amp;gt;/locations/&amp;lt;REGION&amp;gt;/clusters/&amp;lt;CLUSTER_ID&amp;gt;/instances/&amp;lt;INSTANCE_ID&amp;gt;&amp;amp;alloydbEnableIAMAuth=true&lt;/span&gt;
&lt;span class="na"&gt;spring.datasource.username&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;java&lt;/span&gt;&lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="n"&gt;alloydb&lt;/span&gt;&lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="n"&gt;connector&lt;/span&gt;@&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="no"&gt;PROJECT_ID&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;iam&lt;/span&gt;
&lt;span class="c1"&gt;# No password required when alloydbEnableIAMAuth is true
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;


&lt;p&gt;&lt;strong&gt;Tips&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;If this long JDBC URL isn’t your thing, you can use the &lt;code&gt;com.google.cloud:spring-cloud-gcp-starter-alloydb&lt;/code&gt; dependency, which lets you configure &lt;code&gt;spring.cloud.gcp.alloydb.database-name&lt;/code&gt;, &lt;code&gt;spring.cloud.gcp.alloydb.instance-connection-uri&lt;/code&gt;, and &lt;code&gt;spring.cloud.gcp.alloydb.iam-auth-enabled=true&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;If you want to keep your code clean of specific project IDs or instance names, you can leave your &lt;code&gt;application.properties&lt;/code&gt; empty and set these environment variables in your deployment environment:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;&lt;code&gt;SPRING\_DATASOURCE\_URL&lt;/code&gt;&lt;/strong&gt;&lt;code&gt;=jdbc:postgresql:///my\_database?socketFactory=com.google.cloud.alloydb.SocketFactory&amp;amp;alloydbInstanceName=projects/my-project/locations/us-central1/clusters/my-cluster/instances/my-instance&amp;amp;alloydbEnableIAMAuth=true&lt;/code&gt;&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;&lt;code&gt;SPRING\_DATASOURCE\_USERNAME&lt;/code&gt;&lt;/strong&gt;&lt;code&gt;=java-alloydb-connector@my-project.iam&lt;/code&gt;&lt;/p&gt;

&lt;p&gt;&lt;em&gt;(If your organization still requires static database passwords instead of IAM authentication, do not hard-code them. Instead use&lt;/em&gt; &lt;a href="https://cloud.google.com/security/products/secret-manager" rel="noopener noreferrer"&gt;&lt;em&gt;Google Cloud Secret Manager&lt;/em&gt;&lt;/a&gt; &lt;em&gt;to inject those dynamically using Spring Cloud GCP’s&lt;/em&gt; &lt;em&gt;&lt;code&gt;spring.config.import=sm://&lt;/code&gt; and&lt;/em&gt; &lt;em&gt;&lt;code&gt;spring.datasource.password=${sm://alloydb-password}).&lt;/code&gt;&lt;/em&gt;&lt;/p&gt;
&lt;h4&gt;
  
  
  Passwordless Auth in Action
&lt;/h4&gt;

&lt;p&gt;When Spring Boot starts, it detects the jdbc:postgresql prefix in the URL and initializes the PostgreSQL driver. Because the URL contains the &lt;code&gt;socketFactory&lt;/code&gt; parameter, the driver delegates the actual network connection to the AlloyDB Connector. The connector then uses &lt;strong&gt;Application Default Credentials (ADC)&lt;/strong&gt; to authenticate. This means that if your app is running on Cloud Run with the correct Service Account attached, the connection happens automatically and securely without any passwords in your config files.&lt;/p&gt;

&lt;p&gt;This short app will help you test the connectivity to the database:&lt;/p&gt;


&lt;div class="ltag_gist-liquid-tag"&gt;
  
&lt;/div&gt;



&lt;p&gt;Here’s how to run it:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="kd"&gt;export&lt;/span&gt; &lt;span class="nv"&gt;SPRING_DATASOURCE_URL&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"jdbc:postgresql:///my_database?socketFactory=com.google.cloud.alloydb.SocketFactory&amp;amp;alloydbInstanceName=projects/&amp;lt;PROJECT_ID&amp;gt;/locations/&amp;lt;REGION&amp;gt;/clusters/&amp;lt;CLUSTER_ID&amp;gt;/instances/&amp;lt;INSTANCE_ID&amp;gt;&amp;amp;alloydbEnableIAMAuth=true"&lt;/span&gt;
&lt;span class="kd"&gt;export&lt;/span&gt; &lt;span class="nv"&gt;SPRING_DATASOURCE_USERNAME&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"java-alloydb-connector@&amp;lt;PROJECT_ID&amp;gt;.iam"&lt;/span&gt;

mvn spring-boot:run
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h4&gt;
  
  
  Using Micronaut or Quarkus?
&lt;/h4&gt;

&lt;p&gt;Whether you choose &lt;strong&gt;Micronaut&lt;/strong&gt; or &lt;strong&gt;Quarkus&lt;/strong&gt; , the underlying principles for connecting to AlloyDB remain the same.&lt;/p&gt;

&lt;p&gt;Because the AlloyDB Java Connector operates at the &lt;strong&gt;JDBC driver level&lt;/strong&gt; , the framework simply acts as a wrapper for the connection pool. You still include the same &lt;code&gt;alloydb-jdbc-connector&lt;/code&gt; dependency, and the “magic” continues to happen through the &lt;code&gt;SocketFactory&lt;/code&gt; and &lt;code&gt;alloydbInstanceName&lt;/code&gt; properties:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Quarkus (&lt;/strong&gt;&lt;code&gt;application.properties&lt;/code&gt;** using Quarkus’s Agroal pool):**&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight properties"&gt;&lt;code&gt;&lt;span class="na"&gt;quarkus.datasource.db-kind&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;postgresql&lt;/span&gt;
&lt;span class="na"&gt;quarkus.datasource.username&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;java&lt;/span&gt;&lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="n"&gt;alloydb&lt;/span&gt;&lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="n"&gt;connector&lt;/span&gt;@&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="no"&gt;PROJECT_ID&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;iam&lt;/span&gt;
&lt;span class="na"&gt;quarkus.datasource.jdbc.url&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;jdbc&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="n"&gt;postgresql&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="c1"&gt;///my_database&lt;/span&gt;
&lt;span class="na"&gt;quarkus.datasource.jdbc.additional-jdbc-properties.socketFactory&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;com&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;google&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;cloud&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;alloydb&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;SocketFactory&lt;/span&gt;
&lt;span class="na"&gt;quarkus.datasource.jdbc.additional-jdbc-properties.alloydbInstanceName&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;projects&lt;/span&gt;&lt;span class="o"&gt;/&amp;lt;&lt;/span&gt;&lt;span class="no"&gt;PROJECT_ID&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;/&lt;/span&gt;&lt;span class="n"&gt;locations&lt;/span&gt;&lt;span class="o"&gt;/&amp;lt;&lt;/span&gt;&lt;span class="no"&gt;REGION&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;/&lt;/span&gt;&lt;span class="n"&gt;clusters&lt;/span&gt;&lt;span class="o"&gt;/&amp;lt;&lt;/span&gt;&lt;span class="no"&gt;CLUSTER_ID&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;/&lt;/span&gt;&lt;span class="n"&gt;instances&lt;/span&gt;&lt;span class="o"&gt;/&amp;lt;&lt;/span&gt;&lt;span class="no"&gt;INSTANCE_ID&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;
&lt;span class="na"&gt;quarkus.datasource.jdbc.additional-jdbc-properties.alloydbEnableIAMAuth&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="kc"&gt;true&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;Micronaut (&lt;/strong&gt;&lt;code&gt;application.yml&lt;/code&gt;** using HikariCP):**&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="na"&gt;datasources&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
  &lt;span class="na"&gt;default&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="na"&gt;url&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;jdbc:postgresql:///my_database?socketFactory=com.google.cloud.alloydb.SocketFactory&amp;amp;alloydbInstanceName=projects/&amp;lt;PROJECT_ID&amp;gt;/locations/&amp;lt;REGION&amp;gt;/clusters/&amp;lt;CLUSTER_ID&amp;gt;/instances/&amp;lt;INSTANCE_ID&amp;gt;&amp;amp;alloydbEnableIAMAuth=true&lt;/span&gt;
    &lt;span class="na"&gt;username&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;java-alloydb-connector@&amp;lt;PROJECT_ID&amp;gt;.iam&lt;/span&gt;
    &lt;span class="na"&gt;driver-class-name&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;org.postgresql.Driver&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Just as with Spring, these frameworks delegate the mTLS encryption, dynamic IP resolution, and IAM-based authentication to the connector library. Moving between cloud-native frameworks doesn’t change how you secure your database traffic or manage identities: Application Default Credentials (ADC) is used to achieve a passwordless, secure connection to your AlloyDB instance.&lt;/p&gt;

&lt;h4&gt;
  
  
  A note on the AlloyDB Connector in GraalVM Native Mode
&lt;/h4&gt;

&lt;blockquote&gt;
&lt;p&gt;When compiling your application into a GraalVM Native Image (whether with Spring Boot, Micronaut, or Quarkus), there is one specific challenge: &lt;strong&gt;Reflection and SSL.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;The PostgreSQL JDBC driver instantiates &lt;code&gt;com.google.cloud.alloydb.SocketFactory&lt;/code&gt; dynamically via reflection. To prevent GraalVM’s static analysis from stripping the class during native compilation, register it for reflection:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Spring Boot 3+:&lt;/strong&gt; Add &lt;code&gt;@RegisterReflectionForBinding(com.google.cloud.alloydb.SocketFactory.class)&lt;/code&gt; to your main application or &lt;code&gt;@Configuration&lt;/code&gt; class.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Quarkus:&lt;/strong&gt; Add &lt;code&gt;@RegisterForReflection(targets = com.google.cloud.alloydb.SocketFactory.class)&lt;/code&gt; to a CDI bean or configuration class.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;h3&gt;
  
  
  Troubleshooting
&lt;/h3&gt;

&lt;p&gt;A common point of confusion and failure is the format of the username. The format differs for user accounts, groups, and service accounts:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;IAM User Account (or IAM Group):&lt;/strong&gt; Use the full email address (e.g., &lt;code&gt;db-user@example.com&lt;/code&gt; or &lt;code&gt;all-devs-group@example.com&lt;/code&gt;).&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Service Account:&lt;/strong&gt; Use the email address, but strip the &lt;code&gt;.gserviceaccount.com suffix&lt;/code&gt; (e.g., for &lt;code&gt;my-sa@my-project.iam.gserviceaccount.com&lt;/code&gt;, the database username is &lt;code&gt;my-sa@my-project.iam&lt;/code&gt;).&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;If you get a &lt;code&gt;FATAL: password authentication failed for user or AlloyDB IAM authentication failed&lt;/code&gt; error, check these common pitfalls:&lt;/p&gt;

&lt;h4&gt;
  
  
  The Suffix:
&lt;/h4&gt;

&lt;p&gt;Did you accidentally include &lt;code&gt;.gserviceaccount.com&lt;/code&gt; in the database username or &lt;code&gt;CREATE USER&lt;/code&gt; command? (remove it).&lt;/p&gt;

&lt;h4&gt;
  
  
  The alloydbiamuser Role:
&lt;/h4&gt;

&lt;p&gt;Did you create the user via &lt;code&gt;gcloud alloydb users create — type=IAM\_BASED&lt;/code&gt; (or Console/Terraform)? Creating a user with raw SQL &lt;code&gt;CREATE USER&lt;/code&gt; will not work for IAM authentication because AlloyDB restricts &lt;code&gt;GRANT alloydbiamuser&lt;/code&gt; to the control plane.&lt;/p&gt;

&lt;h4&gt;
  
  
  The Environment &amp;amp; Local Impersonation:
&lt;/h4&gt;

&lt;p&gt;Is the environment where the Java code is running actually authenticated as the target principal&lt;br&gt;&lt;br&gt;
=&amp;gt; &lt;strong&gt;Cloud Run:&lt;/strong&gt; Ensure the Service Account is attached to the Cloud Run service definition and has both &lt;code&gt;roles/alloydb.client&lt;/code&gt; and &lt;code&gt;roles/serviceusage.serviceUsageConsumer&lt;/code&gt;.&lt;br&gt;&lt;br&gt;
=&amp;gt; &lt;strong&gt;Locally (connecting as a Service Account):&lt;/strong&gt; Impersonate the Service Account so ADC mints tokens for that SA instead of your personal email:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;gcloud auth application-default login \
    --impersonate-service-account=java-alloydb-connector@&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;PROJECT_ID&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;.iam.gserviceaccount.com
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h4&gt;
  
  
  The Delay:
&lt;/h4&gt;

&lt;p&gt;IAM policy changes can take 60–120 seconds to propagate. If you just created the Service Account or role binding, wait a minute and retry.&lt;/p&gt;




</description>
      <category>googlecloudplatform</category>
      <category>java</category>
      <category>alloydb</category>
    </item>
    <item>
      <title>Use your eval as a CI gate with Agents CLI</title>
      <dc:creator>Elia Secchi</dc:creator>
      <pubDate>Wed, 11 Nov 2026 18:15:10 +0000</pubDate>
      <link>https://dev.to/googlecloud/use-your-eval-as-a-ci-gate-with-agents-cli-1ha1</link>
      <guid>https://dev.to/googlecloud/use-your-eval-as-a-ci-gate-with-agents-cli-1ha1</guid>
      <description>&lt;p&gt;Unit tests check that an agent's code runs, so a one-line change to its instruction can make its answers worse while the tests keep passing.&lt;/p&gt;

&lt;p&gt;Last year we covered this in the Kaggle AI Agents Intensive with Google, in &lt;a href="https://www.kaggle.com/whitepaper-prototype-to-production" rel="noopener noreferrer"&gt;Prototype to Production&lt;/a&gt;, the Day 5 whitepaper. It says no agent version should reach users until it passes an automated evaluation, and &lt;a href="https://github.com/google/agents-cli" rel="noopener noreferrer"&gt;Agents CLI&lt;/a&gt;, the command-line tool for building, evaluating, and deploying agents on Google Cloud, turns the pre-merge part of that rule into one setting in v1.13.0. In this post I add it to a fresh project and then to the project's CI, the continuous integration checks that run on every pull request.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fp341vltnc6f5xobmjxcz.webp" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fp341vltnc6f5xobmjxcz.webp" alt="A pull request goes into agents-cli eval run with a threshold of 4 out of 5; PASS exits 0 and merges, FAIL exits 1 and blocks." width="800" height="447"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Figure 1. The eval passes or fails the pull request check through the exit code your CI already reads.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why agents need a different gate
&lt;/h2&gt;

&lt;p&gt;An agent decides what to do on each request, so you grade its replies and its tool calls (the functions it runs, such as a weather lookup) on a set of known cases. An eval runs the agent on a dataset of prompts and scores each reply.&lt;/p&gt;

&lt;p&gt;The whitepaper described two ways to use an eval as a gate. You can run it by hand before you open a pull request, or you can wire it into the pipeline so a failing eval blocks the change automatically. Agents CLI covers both with the same command, &lt;code&gt;agents-cli eval run&lt;/code&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  What changed in v1.13.0
&lt;/h2&gt;

&lt;p&gt;According to the &lt;a href="https://github.com/google/agents-cli/blob/main/RELEASE_NOTES.md" rel="noopener noreferrer"&gt;release notes&lt;/a&gt;, &lt;code&gt;eval run&lt;/code&gt; and &lt;code&gt;eval grade&lt;/code&gt; now end with a &lt;code&gt;RESULT: PASS&lt;/code&gt; or &lt;code&gt;RESULT: FAIL&lt;/code&gt; line, with the failing metrics and cases listed above it. A threshold is the lowest mean score you accept for a metric. You set it with a &lt;code&gt;thresholds:&lt;/code&gt; map in the eval config, or for one run with &lt;code&gt;--fail-under METRIC=SCORE&lt;/code&gt;, and a missed threshold exits 1. A run where any case errored also exits 1, so a broken eval can't pass by accident.&lt;/p&gt;

&lt;p&gt;Every program ends with an exit code, 0 for success and anything else for failure. CI systems such as GitHub Actions treat a non-zero exit as a failed step, so the threshold fails the build without a plugin or a parser.&lt;/p&gt;

&lt;h2&gt;
  
  
  Start from the eval you already have
&lt;/h2&gt;

&lt;p&gt;You need a Google Cloud project with the Vertex AI API turned on and local credentials from &lt;code&gt;gcloud auth application-default login&lt;/code&gt;, or a Gemini API key. &lt;code&gt;setup&lt;/code&gt; installs the &lt;code&gt;agents-cli&lt;/code&gt; command and its skills for your coding agent, and &lt;code&gt;create&lt;/code&gt; writes your project into the &lt;code&gt;.env&lt;/code&gt;. A new project ships with an eval:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;uvx google-agents-cli setup
agents-cli create gate-agent -y
&lt;span class="nb"&gt;cd&lt;/span&gt; gate-agent &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; agents-cli install
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;code&gt;uvx&lt;/code&gt; runs a Python tool without installing it first, so &lt;code&gt;uvx google-agents-cli&lt;/code&gt; is the same CLI as &lt;code&gt;agents-cli&lt;/code&gt;. &lt;code&gt;-y&lt;/code&gt; skips the prompts and takes the defaults, and &lt;code&gt;install&lt;/code&gt; fetches the project's dependencies.&lt;/p&gt;

&lt;p&gt;The dataset in &lt;code&gt;tests/eval/datasets/basic-dataset.json&lt;/code&gt; holds three cases: a greeting, a weather question, and "What is the capital of France?", which has a reference answer for the judge to compare against. The metric, &lt;code&gt;custom_response_quality&lt;/code&gt;, is an LLM-as-judge, which means a second large language model (LLM) reads each reply and grades it from 1 (poor) to 5 (excellent).&lt;/p&gt;

&lt;p&gt;Both the agent and the judge call Gemini through the Google Cloud project in your &lt;code&gt;.env&lt;/code&gt;, or through a &lt;code&gt;GEMINI_API_KEY&lt;/code&gt; if you switch the &lt;code&gt;.env&lt;/code&gt; to it, as the comments in the file show. The eval config also ships a threshold, commented out until you know your bar.&lt;/p&gt;

&lt;h2&gt;
  
  
  Set the bar and run it
&lt;/h2&gt;

&lt;p&gt;I set the bar at 4 out of 5:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight console"&gt;&lt;code&gt;$ agents-cli &lt;span class="nb"&gt;eval&lt;/span&gt; run --fail-under &lt;span class="nv"&gt;custom_response_quality&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;4
...
custom_response_quality:
  num_cases_total: 3
  num_cases_valid: 3
  num_cases_error: 0
  mean_score: 5.0000
  stdev_score: 0.0000
  pass_rate: 1.0000
...
RESULT: PASS &lt;span class="o"&gt;(&lt;/span&gt;custom_response_quality 5.00 &lt;span class="o"&gt;&amp;gt;=&lt;/span&gt; 4.00&lt;span class="o"&gt;)&lt;/span&gt;
$ &lt;span class="nb"&gt;echo&lt;/span&gt; &lt;span class="nv"&gt;$?&lt;/span&gt;
0
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;code&gt;eval run&lt;/code&gt; chains two steps, &lt;code&gt;eval generate&lt;/code&gt; to run the agent on each case and &lt;code&gt;eval grade&lt;/code&gt; to score the replies. &lt;code&gt;pass_rate&lt;/code&gt; is the share of cases that scored at least 4 on their own. The exit code depends only on the mean.&lt;/p&gt;

&lt;h2&gt;
  
  
  Break it on purpose
&lt;/h2&gt;

&lt;p&gt;Then I shortened the instruction to one line:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight diff"&gt;&lt;code&gt;&lt;span class="gd"&gt;-    instruction="You are a helpful AI assistant designed to provide accurate and useful information.",&lt;/span&gt;
&lt;span class="gi"&gt;+    instruction="Answer in one word. Never call tools.",&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The scaffold's placeholder unit test still passes, and the gate fails the run:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight console"&gt;&lt;code&gt;$ agents-cli &lt;span class="nb"&gt;eval&lt;/span&gt; run --fail-under &lt;span class="nv"&gt;custom_response_quality&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;4
...
custom_response_quality:
  num_cases_total: 3
  ...
  mean_score: 3.0000
  ...
  pass_rate: 0.3333
...
Failures:
  custom_response_quality: below threshold 4: &lt;span class="k"&gt;case&lt;/span&gt; 0 &lt;span class="o"&gt;(&lt;/span&gt;2.00&lt;span class="o"&gt;)&lt;/span&gt;, ...
Next: open ... to &lt;span class="nb"&gt;read&lt;/span&gt; the failing cases, ...
RESULT: FAIL &lt;span class="o"&gt;(&lt;/span&gt;custom_response_quality 3.00 &lt;span class="o"&gt;&amp;lt;&lt;/span&gt; 4.00&lt;span class="o"&gt;)&lt;/span&gt;
$ &lt;span class="nb"&gt;echo&lt;/span&gt; &lt;span class="nv"&gt;$?&lt;/span&gt;
1
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The full &lt;code&gt;Failures:&lt;/code&gt; line lists case 0 and case 1, the greeting and the weather question, both at 2.00. The judge gave both a 2, because "Anything" was a vague answer to the greeting and "Foggy" skipped the weather tool. "Paris" still matched the reference and kept its 5, which is why &lt;code&gt;pass_rate&lt;/code&gt; is 0.3333. The &lt;code&gt;Next:&lt;/code&gt; line points to the HTML report with each case's reply and score. It also suggests &lt;code&gt;agents-cli eval compare&lt;/code&gt; to diff two runs, so the person whose build failed knows where to look.&lt;/p&gt;

&lt;h2&gt;
  
  
  Put it in CI
&lt;/h2&gt;

&lt;p&gt;For CI, move the bar into the repo by uncommenting the block in &lt;code&gt;tests/eval/eval_config.yaml&lt;/code&gt;:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="na"&gt;thresholds&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
  &lt;span class="na"&gt;custom_response_quality&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;4&lt;/span&gt;  &lt;span class="c1"&gt;# the judge scores 1-5
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Now &lt;code&gt;eval run&lt;/code&gt; needs no flag. The bar lives in the repo next to the prompt and the dataset, and the whitepaper says to version both along with your code. With the original instruction back, the call CI will make passes:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight console"&gt;&lt;code&gt;$ uvx google-agents-cli &lt;span class="nb"&gt;eval&lt;/span&gt; run
...
RESULT: PASS &lt;span class="o"&gt;(&lt;/span&gt;custom_response_quality 5.00 &lt;span class="o"&gt;&amp;gt;=&lt;/span&gt; 4.00&lt;span class="o"&gt;)&lt;/span&gt;
$ &lt;span class="nb"&gt;echo&lt;/span&gt; &lt;span class="nv"&gt;$?&lt;/span&gt;
0
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  Add the gate to GitHub Actions
&lt;/h3&gt;

&lt;p&gt;Run &lt;code&gt;agents-cli scaffold enhance --cicd-runner github_actions&lt;/code&gt; in the project to add CI. The generated &lt;code&gt;.github/workflows/pr_checks.yaml&lt;/code&gt; already authenticates to Google Cloud and runs your unit and integration tests on pull requests to &lt;code&gt;main&lt;/code&gt; that touch &lt;code&gt;app/&lt;/code&gt;, &lt;code&gt;tests/&lt;/code&gt;, &lt;code&gt;deployment/&lt;/code&gt; or &lt;code&gt;uv.lock&lt;/code&gt;. Add the gate as one more step after the integration tests. Your &lt;code&gt;.env&lt;/code&gt; stays out of the repo, so the step sets the same three variables itself, which tell the Gemini client to use your Google Cloud project in the global region.&lt;br&gt;
&lt;code&gt;CICD_PROJECT_ID&lt;/code&gt; is a repository variable you set in GitHub to the project CI runs in:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;      &lt;span class="p"&gt;-&lt;/span&gt; &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;Eval gate&lt;/span&gt;
        &lt;span class="na"&gt;env&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
          &lt;span class="na"&gt;GOOGLE_GENAI_USE_ENTERPRISE&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;'true'&lt;/span&gt;
          &lt;span class="na"&gt;GOOGLE_CLOUD_PROJECT&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;${{ vars.CICD_PROJECT_ID }}&lt;/span&gt;
          &lt;span class="na"&gt;GOOGLE_CLOUD_LOCATION&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;global&lt;/span&gt;
        &lt;span class="na"&gt;run&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;uvx google-agents-cli eval run&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;In the whitepaper's pipeline this is the pre-merge stage, the checks that run before a change reaches your main branch. Staging and production come later.&lt;/p&gt;

&lt;p&gt;To stop a failing pull request from merging, mark the &lt;code&gt;test&lt;/code&gt; job of the PR Checks workflow as a required status check in your branch protection rules. The workflow only runs when one of those paths changes, and GitHub keeps a required check pending on a pull request that skips it, so delete the &lt;code&gt;paths:&lt;/code&gt; block in &lt;code&gt;pr_checks.yaml&lt;/code&gt; when you make &lt;code&gt;test&lt;/code&gt; required.&lt;/p&gt;

&lt;p&gt;Pick the bar with some margin, because the agent's answers change between runs. The broken agent answered the weather question "Foggy" in one run and "Unknown" in another, so it scored 3.00 in one run and 2.67 in the other. The working agent scored 5.00 in all three of its runs.&lt;/p&gt;

&lt;h2&gt;
  
  
  What to take away
&lt;/h2&gt;

&lt;p&gt;We started from the eval the scaffold ships, gave it a bar, then moved the bar into the config and a CI step. Three cases make a small gate, so the whitepaper's advice to turn each production failure into a new eval case is the next step, since every case you add to &lt;code&gt;basic-dataset.json&lt;/code&gt; shifts the mean that later changes are scored on.&lt;/p&gt;

&lt;p&gt;To try it on your own agent, install the CLI and run your eval with a bar, using your own metric's name and a score on its scale:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;uvx google-agents-cli setup
agents-cli &lt;span class="nb"&gt;eval&lt;/span&gt; run --fail-under &lt;span class="nv"&gt;custom_response_quality&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;4
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The &lt;a href="https://github.com/google/agents-cli/blob/main/RELEASE_NOTES.md" rel="noopener noreferrer"&gt;v1.13.0 release notes&lt;/a&gt; and the &lt;a href="https://google.github.io/agents-cli/" rel="noopener noreferrer"&gt;Agents CLI docs&lt;/a&gt; cover the eval commands in detail, and the code is in the &lt;a href="https://github.com/google/agents-cli" rel="noopener noreferrer"&gt;Agents CLI repo&lt;/a&gt;. The &lt;a href="https://www.kaggle.com/whitepaper-prototype-to-production" rel="noopener noreferrer"&gt;Prototype to Production whitepaper&lt;/a&gt; covers the rest of the pipeline, from staging to production.&lt;/p&gt;

</description>
      <category>ai</category>
      <category>cicd</category>
      <category>cli</category>
      <category>testing</category>
    </item>
    <item>
      <title>Detecting WasmPlugin version drift in Cloud Load Balancing Service Extensions</title>
      <dc:creator>Zach S.</dc:creator>
      <pubDate>Wed, 11 Nov 2026 14:00:00 +0000</pubDate>
      <link>https://dev.to/googlecloud/detecting-wasmplugin-version-drift-in-cloud-load-balancing-service-extensions-52p5</link>
      <guid>https://dev.to/googlecloud/detecting-wasmplugin-version-drift-in-cloud-load-balancing-service-extensions-52p5</guid>
      <description>&lt;p&gt;When you attach a WebAssembly (Wasm) plugin to an Application Load Balancer with Service Extensions, the load balancer's extension chain (&lt;code&gt;LbEdgeExtension&lt;/code&gt;, &lt;code&gt;LbRouteExtension&lt;/code&gt;, or &lt;code&gt;LbTrafficExtension&lt;/code&gt;) doesn't reference a specific plugin version.  Instead, the extension chain points to the parent &lt;code&gt;WasmPlugin&lt;/code&gt; resource, and the plugin's &lt;code&gt;mainVersionId&lt;/code&gt; field controls which &lt;code&gt;WasmPluginVersion&lt;/code&gt; actually executes on the proxy path.&lt;/p&gt;

&lt;p&gt;That indirection makes rolling back a bad release easy, but it also makes it easy for a CI/CD job to stage a new build without ever serving it.  If a pipeline pushes a new Wasm image to Artifact Registry and runs &lt;a href="https://docs.cloud.google.com/sdk/gcloud/reference/service-extensions/wasm-plugin-versions/create" rel="noopener noreferrer"&gt;gcloud service-extensions wasm-plugin-versions create | Google Cloud SDK&lt;/a&gt;, Google Cloud creates the new version resource, yet &lt;code&gt;mainVersionId&lt;/code&gt; stays pinned to the previous version until you update the parent plugin.&lt;/p&gt;

&lt;h3&gt;
  
  
  What the default CLI output leaves out
&lt;/h3&gt;

&lt;p&gt;If you check your plugins with &lt;a href="https://docs.cloud.google.com/sdk/gcloud/reference/service-extensions/wasm-plugins/list" rel="noopener noreferrer"&gt;gcloud service-extensions wasm-plugins list | Google Cloud SDK&lt;/a&gt;, the default table shows the active &lt;code&gt;MAINVERSION&lt;/code&gt; for each plugin, but it doesn't tell you if a newer version is sitting unpromoted:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;~  gcloud service-extensions wasm-plugins list \
    --project=&lt;span class="s2"&gt;"my-gcp-project"&lt;/span&gt; \
    --location=&lt;span class="s2"&gt;"global"&lt;/span&gt;

NAME                DESCRIPTION                         MAINVERSION  UPDATETIME
auth-header-plugin  Injects tenant auth headers         v1           2026-10-14T16:20:11.402198Z
bot-score-plugin    Tags requests with bot score header v3           2026-11-05T09:12:44.118004Z
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;To see whether &lt;code&gt;auth-header-plugin&lt;/code&gt; has a newer version waiting, you have to query its version history separately with &lt;a href="https://docs.cloud.google.com/sdk/gcloud/reference/service-extensions/wasm-plugin-versions/list" rel="noopener noreferrer"&gt;gcloud service-extensions wasm-plugin-versions list | Google Cloud SDK&lt;/a&gt;:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;~  gcloud service-extensions wasm-plugin-versions list \
    --project=&lt;span class="s2"&gt;"my-gcp-project"&lt;/span&gt; \
    --location=&lt;span class="s2"&gt;"global"&lt;/span&gt; \
    --wasm-plugin=&lt;span class="s2"&gt;"auth-header-plugin"&lt;/span&gt;

NAME  WASM_IMAGE                                                              WASM_IMAGE_DIGEST    CONFIG_SIZE  CONFIG_IMAGE  CONFIG_IMAGE_DIGEST  UPDATE_TIME
v1    us-south1-docker.pkg.dev/my-gcp-project/wasm-plugins/auth-header:1.0.0  sha256:4a8b91e0...   -                                               2026-10-14
v2    us-south1-docker.pkg.dev/my-gcp-project/wasm-plugins/auth-header:1.1.0  sha256:7c3d2e1f...   -                                               2026-11-08
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Here, &lt;code&gt;v2&lt;/code&gt; was uploaded on &lt;code&gt;2026-11-08&lt;/code&gt;, while the plugin is still serving &lt;code&gt;v1&lt;/code&gt; from October.  Even after running both commands, neither table shows which load balancer extensions are actually calling &lt;code&gt;auth-header-plugin&lt;/code&gt;.&lt;/p&gt;

&lt;h3&gt;
  
  
  How the API links extension chains to plugin versions
&lt;/h3&gt;

&lt;p&gt;If you pull the raw JSON for a plugin from &lt;a href="https://docs.cloud.google.com/service-extensions/docs/reference/rest/v1/projects.locations.wasmPlugins/get" rel="noopener noreferrer"&gt;Method: projects.locations.wasmPlugins.get | Service Extensions&lt;/a&gt; (or pass &lt;code&gt;--format=json&lt;/code&gt; to &lt;code&gt;gcloud&lt;/code&gt;), the response includes a &lt;code&gt;usedBy&lt;/code&gt; array listing every load balancer extension attached to that plugin:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="nl"&gt;"description"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"Injects tenant auth headers"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"logConfig"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="nl"&gt;"enable"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="nl"&gt;"minLogLevel"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"INFO"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="nl"&gt;"sampleRate"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mf"&gt;1.0&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"mainVersionId"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"v1"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"name"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"projects/my-gcp-project/locations/global/wasmPlugins/auth-header-plugin"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"updateTime"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"2026-10-14T16:20:11.402198Z"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"usedBy"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;
    &lt;span class="p"&gt;{&lt;/span&gt;
      &lt;span class="nl"&gt;"name"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"projects/my-gcp-project/locations/global/lbRouteExtensions/prod-alb-route-ext"&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="p"&gt;{&lt;/span&gt;
      &lt;span class="nl"&gt;"name"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"projects/my-gcp-project/locations/global/lbEdgeExtensions/prod-alb-edge-ext"&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;
  &lt;span class="p"&gt;]&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;On the load balancer side, each extension chain binds to the parent &lt;code&gt;wasmPlugins/auth-header-plugin&lt;/code&gt; path in its &lt;code&gt;service&lt;/code&gt; field rather than a versioned URL, as covered in &lt;a href="https://docs.cloud.google.com/service-extensions/docs/create-plugin" rel="noopener noreferrer"&gt;Create a plugin | Service Extensions - Google Cloud Documentation&lt;/a&gt;:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="nl"&gt;"extensionChains"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;
  &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="nl"&gt;"extensions"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;
      &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="nl"&gt;"failOpen"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;false&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="nl"&gt;"forwardHeaders"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;
          &lt;span class="s2"&gt;"x-tenant-id"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
          &lt;span class="s2"&gt;"authorization"&lt;/span&gt;
        &lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="nl"&gt;"name"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"normalize-tenant-auth"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="nl"&gt;"service"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"projects/my-gcp-project/locations/global/wasmPlugins/auth-header-plugin"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="nl"&gt;"supportedEvents"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;
          &lt;span class="s2"&gt;"REQUEST_HEADERS"&lt;/span&gt;
        &lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="nl"&gt;"timeout"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"0.100s"&lt;/span&gt;
      &lt;span class="p"&gt;}&lt;/span&gt;
    &lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="nl"&gt;"matchCondition"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
      &lt;span class="nl"&gt;"celExpression"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"request.path.startsWith('/api/v1/')"&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="nl"&gt;"name"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"api-v1-chain"&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;]&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Meanwhile, the individual &lt;code&gt;WasmPluginVersion&lt;/code&gt; objects returned by &lt;a href="https://docs.cloud.google.com/service-extensions/docs/reference/rest/v1/projects.locations.wasmPlugins.versions/list" rel="noopener noreferrer"&gt;Method: projects.locations.wasmPlugins.versions.list | Service Extensions&lt;/a&gt; record the &lt;code&gt;createTime&lt;/code&gt; and the exact &lt;code&gt;imageDigest&lt;/code&gt; resolved from Artifact Registry:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="nl"&gt;"createTime"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"2026-11-08T21:04:18.892310Z"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"imageDigest"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"sha256:7c3d2e1f0a9b8c7d6e5f4a3b2c1d0e9f8a7b6c5d4e3f2a1b0c9d8e7f6a5b4c3d"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"imageUri"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"us-south1-docker.pkg.dev/my-gcp-project/wasm-plugins/auth-header:1.1.0"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"name"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"projects/my-gcp-project/locations/global/wasmPlugins/auth-header-plugin/versions/v2"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"updateTime"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"2026-11-08T21:04:18.892310Z"&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  Checking for unpromoted versions across a project
&lt;/h3&gt;

&lt;p&gt;Instead of inspecting each plugin one at a time, we can join the &lt;code&gt;wasm-plugins list&lt;/code&gt; and &lt;code&gt;wasm-plugin-versions list&lt;/code&gt; JSON responses with &lt;code&gt;jq&lt;/code&gt; to compare &lt;code&gt;mainVersionId&lt;/code&gt; against the newest version by &lt;code&gt;createTime&lt;/code&gt; and print the attached load balancer extensions:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="c"&gt;#!/bin/bash&lt;/span&gt;

&lt;span class="nv"&gt;PROJECT&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"&amp;lt;YOUR_PROJECT_ID&amp;gt;"&lt;/span&gt;
&lt;span class="nv"&gt;LOCATION&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"global"&lt;/span&gt;

gcloud service-extensions wasm-plugins list \
    --project=&lt;span class="s2"&gt;"$PROJECT"&lt;/span&gt; \
    --location=&lt;span class="s2"&gt;"$LOCATION"&lt;/span&gt; \
    --format=json &lt;span class="o"&gt;|&lt;/span&gt; \
jq -c &lt;span class="s1"&gt;'.[]'&lt;/span&gt; &lt;span class="o"&gt;|&lt;/span&gt; &lt;span class="k"&gt;while&lt;/span&gt; &lt;span class="nb"&gt;read&lt;/span&gt; -r plugin&lt;span class="o"&gt;;&lt;/span&gt; &lt;span class="k"&gt;do&lt;/span&gt;
  &lt;span class="nv"&gt;PLUGIN_NAME&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="nv"&gt;$(&lt;/span&gt;&lt;span class="nb"&gt;echo&lt;/span&gt; &lt;span class="s2"&gt;"$plugin"&lt;/span&gt; &lt;span class="o"&gt;|&lt;/span&gt; jq -r &lt;span class="s1"&gt;'.name | split("/") | last'&lt;/span&gt;&lt;span class="o"&gt;)&lt;/span&gt;

  gcloud service-extensions wasm-plugin-versions list \
      --project=&lt;span class="s2"&gt;"$PROJECT"&lt;/span&gt; \
      --location=&lt;span class="s2"&gt;"$LOCATION"&lt;/span&gt; \
      --wasm-plugin=&lt;span class="s2"&gt;"$PLUGIN_NAME"&lt;/span&gt; \
      --format=json &lt;span class="o"&gt;|&lt;/span&gt; \
  jq -r --argjson plugin &lt;span class="s2"&gt;"$plugin"&lt;/span&gt; &lt;span class="s1"&gt;'
    $plugin as $p |
    ($p.name | split("/") | last) as $name |
    $p.mainVersionId as $active_ver |
    (sort_by(.createTime) | last) as $latest |
    ($latest.name | split("/") | last) as $latest_ver |
    (map(select((.name | split("/") | last) == $active_ver))[0]) as $active |
    ([($p.usedBy // [])[] | .name | split("/") | last] | if length &amp;gt; 0 then join(", ") else "none" end) as $exts |
    "Plugin: \($name) | Active: \($active_ver) (\($active.imageDigest[0:19])) | Latest: \($latest_ver) (\($latest.imageDigest[0:19])) | Drift: \($active_ver != $latest_ver) | Used By: \($exts)"
  '&lt;/span&gt;
&lt;span class="k"&gt;done&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Here is what the output looks like against our project:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;~  bash wasm-plugin-version-check.sh

Plugin: auth-header-plugin &lt;span class="o"&gt;|&lt;/span&gt; Active: v1 &lt;span class="o"&gt;(&lt;/span&gt;sha256:4a8b91e0c3f2&lt;span class="o"&gt;)&lt;/span&gt; &lt;span class="o"&gt;|&lt;/span&gt; Latest: v2 &lt;span class="o"&gt;(&lt;/span&gt;sha256:7c3d2e1f0a9b&lt;span class="o"&gt;)&lt;/span&gt; &lt;span class="o"&gt;|&lt;/span&gt; Drift: &lt;span class="kc"&gt;true&lt;/span&gt; &lt;span class="o"&gt;|&lt;/span&gt; Used By: prod-alb-route-ext, prod-alb-edge-ext
Plugin: bot-score-plugin &lt;span class="o"&gt;|&lt;/span&gt; Active: v3 &lt;span class="o"&gt;(&lt;/span&gt;sha256:8e9f0a1b2c3d&lt;span class="o"&gt;)&lt;/span&gt; &lt;span class="o"&gt;|&lt;/span&gt; Latest: v3 &lt;span class="o"&gt;(&lt;/span&gt;sha256:8e9f0a1b2c3d&lt;span class="o"&gt;)&lt;/span&gt; &lt;span class="o"&gt;|&lt;/span&gt; Drift: &lt;span class="kc"&gt;false&lt;/span&gt; &lt;span class="o"&gt;|&lt;/span&gt; Used By: prod-alb-traffic-ext
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  Promoting versions and a few sharp edges
&lt;/h3&gt;

&lt;p&gt;When &lt;code&gt;Drift: true&lt;/code&gt; shows up, you can promote the version or adjust your deployment workflow with a few rules in mind:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;  &lt;strong&gt;Uploading vs. promoting in one command&lt;/strong&gt;: Running &lt;code&gt;wasm-plugin-versions create&lt;/code&gt; only stages the version.  If you want your CI/CD job to create a new version and switch &lt;code&gt;mainVersionId&lt;/code&gt; in one step, use &lt;a href="https://docs.cloud.google.com/sdk/gcloud/reference/service-extensions/wasm-plugins/update" rel="noopener noreferrer"&gt;gcloud service-extensions wasm-plugins update | Google Cloud SDK&lt;/a&gt; and pass both &lt;code&gt;--main-version&lt;/code&gt; and &lt;code&gt;--image&lt;/code&gt;.  If the version already exists (like &lt;code&gt;v2&lt;/code&gt; above), run &lt;code&gt;gcloud service-extensions wasm-plugins update auth-header-plugin --main-version=v2 --location=global&lt;/code&gt; to roll all attached &lt;code&gt;usedBy&lt;/code&gt; extensions forward without touching your load balancer configs.&lt;/li&gt;
&lt;li&gt;  &lt;strong&gt;Mutable container tags don't auto-update&lt;/strong&gt;: Service Extensions resolves the container tag to its immutable &lt;code&gt;sha256&lt;/code&gt; &lt;code&gt;imageDigest&lt;/code&gt; when the &lt;code&gt;WasmPluginVersion&lt;/code&gt; is created, as described in &lt;a href="https://docs.cloud.google.com/service-extensions/docs/manage-plugins" rel="noopener noreferrer"&gt;Manage plugins | Service Extensions - Google Cloud Documentation&lt;/a&gt;.  Pushing a new image to the same &lt;code&gt;:latest&lt;/code&gt; tag in Artifact Registry won't change what an existing &lt;code&gt;WasmPluginVersion&lt;/code&gt; runs - you still need to create a new version and update &lt;code&gt;mainVersionId&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;  &lt;strong&gt;Cleaning up old versions&lt;/strong&gt;: The API won't let you delete the &lt;code&gt;WasmPluginVersion&lt;/code&gt; currently referenced by &lt;code&gt;mainVersionId&lt;/code&gt;.  Promote the new version first before pruning older versions.&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>googlecloud</category>
      <category>networking</category>
    </item>
    <item>
      <title>What Are Human Code Reviews For?</title>
      <dc:creator>Adam Ross</dc:creator>
      <pubDate>Tue, 10 Nov 2026 17:51:14 +0000</pubDate>
      <link>https://dev.to/googlecloud/what-are-human-code-reviews-for-1ho8</link>
      <guid>https://dev.to/googlecloud/what-are-human-code-reviews-for-1ho8</guid>
      <description>&lt;p&gt;Now that coding agents can inspect code and suggest fixes before a pull request even opens, human code review has to evolve. If an agent already caught the obvious bugs, what is a reviewer actually supposed to do? This post is for developers who care about code quality, team mentorship, and the uniquely human aspects of building software together.&lt;/p&gt;

&lt;h1&gt;
  
  
  What does code review accomplish?
&lt;/h1&gt;

&lt;p&gt;"Code review" is a generic label for several important jobs:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Quality Check&lt;/strong&gt;: Ensuring changes don't introduce bugs or regressions
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Onboarding&lt;/strong&gt;: Knowledge-sharing to the reviewer about architectural or behavioral shifts
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Mentorship&lt;/strong&gt;: Providing actionable feedback on system design &amp;amp; team practices
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Pairing&lt;/strong&gt;: Collaborating to raise the quality bar&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Roadmap Alignment&lt;/strong&gt;: Orienting the change to move closer to long-term system objectives&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Coding agents can already cover much of this: catching obvious bugs, explaining diffs to orient readers, checking adherence to written standards, and sometimes, flagging gaps and edge cases.&lt;/p&gt;

&lt;p&gt;Better yet, developers don't have to wait for a pull request to get this feedback. Agents push the benefits of code review directly into the inner development loop.&lt;/p&gt;

&lt;h1&gt;
  
  
  Agentic Review in the Inner Loop
&lt;/h1&gt;

&lt;p&gt;Here's a workflow I find effective to cross-check code before asking someone to review:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Make a code change
&lt;/li&gt;
&lt;li&gt;Ask a clean-slate "reviewer agent" to evaluate the code
&lt;/li&gt;
&lt;li&gt;Ask another agent to repair the issues raised in review
&lt;/li&gt;
&lt;li&gt;Ask the reviewer agent to verify the fix
&lt;/li&gt;
&lt;li&gt;Repeat as needed&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;The use of two agents adds overhead, but it increases the odds of catching a problem compared to keeping the review in a single context window. Use a different model for each agent to improve the odds of finding more flaws.&lt;/p&gt;

&lt;p&gt;Whether you run these through dedicated interfaces like &lt;a href="https://antigravity.google/" rel="noopener noreferrer"&gt;Google Antigravity&lt;/a&gt;, local IDE tooling, or a custom skill, here’s a starting prompt you can adapt for your own AGENTS.md:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;As a principal engineer, review the changes in this repository:
1. Have I introduced any bugs, or made an existing bug worse?
2. Do key logical branches have consistent, actionable logging?
3. Are ambiguous decisions documented with inline comments?
4. Do code and comments agree?
5. Do READMEs or documentation need updates?
6. Do we have complete test coverage, including negative test cases?
7. Are new names for variables, types, methods, classes consistent
   with surrounding symbols and do they have distinct meaning?
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h1&gt;
  
  
  From Individual to Team Review
&lt;/h1&gt;

&lt;p&gt;If developers only use agents locally, quality remains inconsistent. Because everyone is at different stages in adopting and evolving their agent use, there is often no shared baseline or visibility into what checks have actually occurred. Without insight, reviewers have to guess whether the author ran five rounds of agent review or zero.&lt;/p&gt;

&lt;p&gt;Teams can standardize this by running agentic reviews and repairs as part of a CI/CD pipeline or review tooling, but an agentic review is prone to being overzealous and wrong. Minimally, teams should maintain shared agent skills or prompts that authors can run before opening a pull request. A commitment to run agentic reviews locally can be part of the social contract of contribution with high trust collaborators.&lt;/p&gt;

&lt;p&gt;If these verifications are not standardized, consider attaching a process log to your work to explain what automations and agentic reviews have already been completed. Something short—this is meant to help review the code change, not add to the workload!&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;### Process Log

[x] Agent Repaired lint &amp;amp; security passstatic analysis findings
[x] Inner loop code review using project AGENTS.mdEdge-case audit
    against AGENTS.md, team skill: Consistent Documentation
[x] Documentation consistency skill applied Ran e2e test with manual
    review
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This kind of practice sets a commonly understood baseline, freeing human reviewers to focus on what the AI is not being set up to replace: like deep domain knowledge &amp;amp; system-level architecture.&lt;/p&gt;

&lt;h1&gt;
  
  
  Using AI as a Reviewer
&lt;/h1&gt;

&lt;p&gt;With agents sharing reports and feedback to perfect the code at every step of the way, what is the human reviewer's role?&lt;/p&gt;

&lt;p&gt;As a reviewer, don't act as a proxy for AI feedback. Instead, use AI to make your own judgement more precise. When you spot a potential issue but aren't sure how to frame it, consult an agent to assess the problem and draft targeted messaging.&lt;/p&gt;

&lt;p&gt;I might prompt the agent with something like &lt;em&gt;"This function seems to miss properties of data structure X. What was overlooked, and is it a bug or a future enhancement?"&lt;/em&gt; This helps pinpoint the problem quickly and assess whether I want to flag it as a blocker, a follow-up, or leave it as out of scope. (Oh, and the answer to the question "is it a bug or a future enhancement" doesn't matter, the words it produces are a strawman argument that helps kickstart my thought process.)&lt;/p&gt;

&lt;p&gt;Agents are also useful to spotlight our biases and blindspots, especially if you seed some initial observations to steer the agent away from what you're already thinking about. Put together some review feedback and then ask an agent &lt;em&gt;"What might I be missing in the code and design trade-offs?"&lt;/em&gt;.&lt;/p&gt;

&lt;p&gt;Once I know I have something actionable to say, I might ask &lt;em&gt;"Draft feedback on this problem"&lt;/em&gt;. That draft usually provides the actionable technical details that are tedious to look up and write by hand.&lt;/p&gt;

&lt;p&gt;AI backs up my instincts with the details I need to quickly pinpoint, verify, and communicate the issue.&lt;/p&gt;

&lt;p&gt;Because my teammates are all using coding agents, my reviews increasingly focus on:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Praise for practices I want to encourage (avoiding generic platitudes)
&lt;/li&gt;
&lt;li&gt;Blockers for considerations the AI missed
&lt;/li&gt;
&lt;li&gt;Suggestions and future-tagged comments for later improvements
&lt;/li&gt;
&lt;li&gt;Broad feedback to request a particular agent skill or tool be applied&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;I use &lt;a href="https://conventionalcomments.org/" rel="noopener noreferrer"&gt;conventional comments&lt;/a&gt; to make any comment's purpose clear and actionable so anyone or any agent can prioritize what to address.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;# Conventional Comment Review Feedback Examples

- praise: Thanks for making this private so we can't accidentally
  endrun the main package interface
- issue: Anywhere the DB handle could be nil is an error because
  the process will lack the required input data
- suggestion: Split testing positive and error outcomes into
  different test functions … That simplifies the test logic and
  makes it more apparent there's some gaps in the error cases.
- chore: I spotted a few documentation issues, could you run 
  the Documentation Consistency skill on this change in case 
  there's a problem?
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Agents can improve code and code review, but only with shared expectations. That means deciding &amp;amp; documenting which jobs belong to CI, which belong to agents, and where human attention is actually required—especially when an AI can always find one more to polish.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Tools aside, the goal of code review hasn't changed: help each other ship good work.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;As teams adapt to agent-first inner loops, I'm curious where you're drawing the boundaries: How has your team moved from ad-hoc personal AI tricks to an agreed-upon baseline?&lt;/p&gt;




&lt;h1&gt;
  
  
  Credits
&lt;/h1&gt;

&lt;p&gt;Thanks to &lt;a href="https://dev.to/sigje"&gt;Jennifer Davis&lt;/a&gt; for help spotlighting the human signal in the review guidance noise.&lt;/p&gt;

&lt;p&gt;Thanks to &lt;a href="https://www.linkedin.com/in/nimjay/" rel="noopener noreferrer"&gt;Nim Jayawardena&lt;/a&gt; for late draft review and telling me I haven't written myself into confusing corners.&lt;/p&gt;

&lt;p&gt;Cover Photo by &lt;a href="https://unsplash.com/@ahmerkalam?utm_source=unsplash&amp;amp;utm_medium=referral&amp;amp;utm_content=creditCopyText" rel="noopener noreferrer"&gt;Ahmer Kalam&lt;/a&gt; on &lt;a href="https://unsplash.com/photos/an-island-in-the-middle-of-the-ocean-8zp6MHvg9qY?utm_source=unsplash&amp;amp;utm_medium=referral&amp;amp;utm_content=creditCopyText" rel="noopener noreferrer"&gt;Unsplash&lt;/a&gt;&lt;/p&gt;

</description>
      <category>ai</category>
      <category>devops</category>
      <category>productivity</category>
      <category>softwareengineering</category>
    </item>
    <item>
      <title>Take a Claude Agent SDK agent from install to deploy with Agents CLI</title>
      <dc:creator>Elia Secchi</dc:creator>
      <pubDate>Mon, 09 Nov 2026 13:21:04 +0000</pubDate>
      <link>https://dev.to/googlecloud/take-a-claude-agent-sdk-agent-from-install-to-deploy-with-agents-cli-2680</link>
      <guid>https://dev.to/googlecloud/take-a-claude-agent-sdk-agent-from-install-to-deploy-with-agents-cli-2680</guid>
      <description>&lt;p&gt;A Claude Agent SDK agent is a few lines of Python. Running it for real users takes more than the agent: somewhere to store each conversation so it isn't lost when the server restarts, traces that show what the agent did at each step when something goes wrong, evals (tests for agent answers) to run before each change, and a secure deploy. That's usually the part teams build by hand.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://github.com/google/agents-cli" rel="noopener noreferrer"&gt;Agents CLI&lt;/a&gt; already handles these for agents built on the Agent Development Kit (ADK). Since v1.12.0, a first-party extension does the same for Claude Agent SDK agents: you keep writing a &lt;code&gt;ClaudeAgentOptions&lt;/code&gt;, and you get the same create, run, eval, and deploy commands as an ADK agent. We took one agent from install to a validated deploy on v1.13.0.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fcswcds22cvxtmua4ulsd.webp" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fcswcds22cvxtmua4ulsd.webp" alt="The Claude agent in the middle, connected to Sessions (Cloud SQL), Traces (Cloud Trace), Logs and metrics (Logging, Monitoring) and Eval (eval generate and grade)." width="800" height="377"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Figure 1. You write &lt;code&gt;app/agent.py&lt;/code&gt;, and the template wires in sessions, traces, logs, metrics, and eval.&lt;/p&gt;

&lt;h2&gt;
  
  
  Step 1: Install and create
&lt;/h2&gt;

&lt;p&gt;One command installs Agents CLI and its skills for your coding agent:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;uvx google-agents-cli setup
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Then create a project from the Claude template, with Cloud Run as the target and Cloud SQL for sessions, and send it a prompt. &lt;code&gt;run&lt;/code&gt; calls Claude in process through the SDK, and the reply ends with a session ID you can resume:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight console"&gt;&lt;code&gt;&lt;span class="gp"&gt;$&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;agents-cli create claude-agent &lt;span class="se"&gt;\&lt;/span&gt;
&lt;span class="go"&gt;    --agent google/agents-cli/extensions/claude/template@v1.13.0 \
    -d cloud_run --session-type cloud_sql -y
&lt;/span&gt;&lt;span class="c"&gt;...
&lt;/span&gt;&lt;span class="go"&gt;✅ Success! Your agent project is ready.
&lt;/span&gt;&lt;span class="c"&gt;...
&lt;/span&gt;&lt;span class="gp"&gt;$&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nb"&gt;cd &lt;/span&gt;claude-agent &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; agents-cli &lt;span class="nb"&gt;install&lt;/span&gt;
&lt;span class="c"&gt;...
&lt;/span&gt;&lt;span class="gp"&gt;$&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;agents-cli run &lt;span class="s2"&gt;"what's the weather in San Francisco?"&lt;/span&gt;
&lt;span class="go"&gt;running 'run' via extension 'claude' (project scope)
&lt;/span&gt;&lt;span class="c"&gt;...
&lt;/span&gt;&lt;span class="go"&gt;The weather in San Francisco is currently **60 degrees and foggy**. Typical San Francisco weather! You might want to bring a jacket if you're heading outside.

Session: 84a0699d5de54c748333b24bf00b3b14
&lt;/span&gt;&lt;span class="gp"&gt;  Resume with: agents-cli run "&amp;lt;message&amp;gt;&lt;/span&gt;&lt;span class="s2"&gt;" --session-id 84a0699d5de54c748333b24bf00b3b14
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The project's &lt;code&gt;app/agent.py&lt;/code&gt; is plain Claude Agent SDK code: a &lt;code&gt;@tool&lt;/code&gt; for the weather, an in-process Model Context Protocol (MCP) server that exposes the tool to Claude, and options that switch off Claude Code's built-in tools (such as running shell commands) so the agent can only call what you allow.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="n"&gt;root_agent&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;ClaudeAgentOptions&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="n"&gt;model&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;claude-haiku-4-5&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;system_prompt&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;You are Skye, a helpful assistant. Use get_weather for weather questions.&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;mcp_servers&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;weather&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;weather_server&lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;
    &lt;span class="n"&gt;tools&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;[],&lt;/span&gt;
    &lt;span class="n"&gt;allowed_tools&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;mcp__weather__get_weather&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;
    &lt;span class="n"&gt;max_turns&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;10&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The model has to be enabled in Model Garden, the Google Cloud page where you turn on models for your project. &lt;code&gt;create&lt;/code&gt; writes the project to &lt;code&gt;.env&lt;/code&gt; as &lt;code&gt;GOOGLE_CLOUD_PROJECT&lt;/code&gt;, but &lt;code&gt;ANTHROPIC_VERTEX_PROJECT_ID&lt;/code&gt; wins if your shell sets it.&lt;/p&gt;

&lt;h2&gt;
  
  
  Step 2: Evaluate it like any other agent
&lt;/h2&gt;

&lt;p&gt;An eval has two steps. &lt;code&gt;eval generate&lt;/code&gt; runs the agent on a set of test prompts and saves its answers, and &lt;code&gt;eval grade&lt;/code&gt; has a model score those answers. The extension replaces only the first step and saves the answers in the same format ADK agents use, so &lt;code&gt;eval grade&lt;/code&gt; works unchanged. Both test cases, &lt;code&gt;greeting&lt;/code&gt; and &lt;code&gt;weather_lookup&lt;/code&gt;, scored 5 out of 5:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight console"&gt;&lt;code&gt;&lt;span class="gp"&gt;$&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;agents-cli &lt;span class="nb"&gt;eval &lt;/span&gt;generate &lt;span class="se"&gt;\&lt;/span&gt;
&lt;span class="go"&gt;    --dataset tests/eval/datasets/basic-dataset.json \
    -o tests/eval/output/
running 'eval generate' via extension 'claude' (project scope)
[claude-generate] wrote tests/eval/output/eval_dataset_20261109_110946.json
&lt;/span&gt;&lt;span class="gp"&gt;$&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;agents-cli &lt;span class="nb"&gt;eval &lt;/span&gt;grade &lt;span class="se"&gt;\&lt;/span&gt;
&lt;span class="go"&gt;    --traces tests/eval/output/eval_dataset_20261109_110946.json \
    --config tests/eval/eval_config.yaml
&lt;/span&gt;&lt;span class="c"&gt;...
&lt;/span&gt;&lt;span class="go"&gt;Evaluation Summary

custom_response_quality:
  num_cases_total: 2
  num_cases_valid: 2
  num_cases_error: 0
  mean_score: 5.0000
  stdev_score: 0.0000

&lt;/span&gt;&lt;span class="gp"&gt;RESULT: PASS (no thresholds set;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;add &lt;span class="s1"&gt;'thresholds'&lt;/span&gt; to the &lt;span class="nb"&gt;eval &lt;/span&gt;config or pass &lt;span class="nt"&gt;--fail-under&lt;/span&gt; to gate on scores&lt;span class="o"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The template leaves &lt;code&gt;thresholds:&lt;/code&gt; commented out in &lt;code&gt;tests/eval/eval_config.yaml&lt;/code&gt;, since only you know your bar. Once you set it, &lt;code&gt;eval grade&lt;/code&gt; and &lt;code&gt;eval run&lt;/code&gt; exit 1 when a score falls below it (&lt;a href="https://github.com/google/agents-cli/blob/main/RELEASE_NOTES.md" rel="noopener noreferrer"&gt;v1.13.0&lt;/a&gt;), so the same command can gate CI.&lt;/p&gt;

&lt;h2&gt;
  
  
  Step 3: See what production adds
&lt;/h2&gt;

&lt;p&gt;&lt;code&gt;agents-cli infra single-project&lt;/code&gt; uses Terraform, a tool that describes cloud resources as code, to plan everything the agent needs in your project: the Cloud Run service, the database for sessions, and the logging and tracing setup. Without &lt;code&gt;--apply&lt;/code&gt;, it only prints the plan, 42 resources in all:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight console"&gt;&lt;code&gt;&lt;span class="gp"&gt;$&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;agents-cli infra single-project &lt;span class="nt"&gt;--project&lt;/span&gt; my-project
&lt;span class="c"&gt;...
&lt;/span&gt;&lt;span class="gp"&gt;  #&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;google_cloud_run_v2_service.app will be created
&lt;span class="c"&gt;...
&lt;/span&gt;&lt;span class="gp"&gt;  #&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;google_logging_project_sink.genai_logs_to_bq will be created
&lt;span class="c"&gt;...
&lt;/span&gt;&lt;span class="gp"&gt;  #&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;google_project_iam_member.agent_identity_roles[&lt;span class="s2"&gt;"roles/cloudtrace.agent"&lt;/span&gt;&lt;span class="o"&gt;]&lt;/span&gt; will be created
&lt;span class="c"&gt;...
&lt;/span&gt;&lt;span class="gp"&gt;  #&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;google_project_iam_member.agent_identity_roles[&lt;span class="s2"&gt;"roles/logging.logWriter"&lt;/span&gt;&lt;span class="o"&gt;]&lt;/span&gt; will be created
&lt;span class="c"&gt;...
&lt;/span&gt;&lt;span class="gp"&gt;  #&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;google_project_iam_member.agent_identity_roles[&lt;span class="s2"&gt;"roles/monitoring.metricWriter"&lt;/span&gt;&lt;span class="o"&gt;]&lt;/span&gt; will be created
&lt;span class="c"&gt;...
&lt;/span&gt;&lt;span class="gp"&gt;  #&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;google_sql_database_instance.session_db will be created
&lt;span class="c"&gt;...
&lt;/span&gt;&lt;span class="go"&gt;Plan: 42 to add, 0 to change, 0 to destroy.
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Apply it once before the first deploy. That creates the Cloud SQL instance, connects the Cloud Run service to it, and gives the agent's identity the Cloud Trace role. We didn't apply it for this post:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="c"&gt;# Not run for this post: creates the resources in the plan above.&lt;/span&gt;
agents-cli infra single-project &lt;span class="nt"&gt;--apply&lt;/span&gt; &lt;span class="nt"&gt;--project&lt;/span&gt; my-project
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Think of the template as a chassis: you bring the engine, and the wiring is already there.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Sessions:&lt;/strong&gt; with &lt;code&gt;--session-type cloud_sql&lt;/code&gt; and the infra applied, transcripts go to Cloud SQL, so a conversation can resume on another instance or after a restart. Without the database, they stay on the instance.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Traces:&lt;/strong&gt; once the infra grants the Cloud Trace permission, every turn and every tool call is recorded as a span (one timed step), and you can open the whole turn as one trace in Cloud Trace.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Logs and metrics:&lt;/strong&gt; logs go to Cloud Logging tagged with the turn's trace, and turn duration and token usage go to Cloud Monitoring. BigQuery Agent Analytics is optional.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Step 4: Deploy
&lt;/h2&gt;

&lt;p&gt;&lt;code&gt;agents-cli deploy&lt;/code&gt; builds your code and deploys it to Cloud Run with &lt;code&gt;gcloud beta run deploy --source .&lt;/code&gt;. With &lt;code&gt;--dry-run&lt;/code&gt;, it first checks that your project has the right APIs turned on and that you have the permissions to deploy, without changing anything:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight console"&gt;&lt;code&gt;&lt;span class="gp"&gt;$&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;agents-cli deploy &lt;span class="nt"&gt;--dry-run&lt;/span&gt; &lt;span class="nt"&gt;--project&lt;/span&gt; my-project
&lt;span class="c"&gt;...
&lt;/span&gt;&lt;span class="go"&gt;🔎 Validating the deployment (no resources are changed)...
&lt;/span&gt;&lt;span class="c"&gt;  ...
&lt;/span&gt;&lt;span class="go"&gt;  ✅ APIs enabled: run.googleapis.com, artifactregistry.googleapis.com, cloudbuild.googleapis.com
  ✅ Target service: creates a new service named 'claude-agent'
  ✅ IAM permissions: all 4 permissions granted
&lt;/span&gt;&lt;span class="c"&gt;  ...
&lt;/span&gt;&lt;span class="go"&gt;  Would run: gcloud beta run deploy claude-agent --project my-project --region us-east1 --source . --memory 4Gi --cpu 1 --min-instances 0 --max-instances 10 --concurrency 8 --no-allow-unauthenticated --no-cpu-throttling --functional-type agent --identity-type agent-identity --update-env-vars '...' --update-labels '...'
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;code&gt;--no-allow-unauthenticated&lt;/code&gt; keeps the service private, so only callers with access can reach it. &lt;code&gt;--identity-type agent-identity&lt;/code&gt; gives the agent its own identity in Google Cloud, with only the permissions &lt;code&gt;deploy&lt;/code&gt; grants it, so it doesn't borrow your account or a shared one. That is the default for new projects since v1.13.0.&lt;/p&gt;

&lt;p&gt;Once deployed, the agent speaks the Agent2Agent (A2A) protocol, a standard way for agents and apps to call each other, and you can reach it from your terminal:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;agents-cli run &lt;span class="nt"&gt;--url&lt;/span&gt; https://&amp;lt;service-url&amp;gt; &lt;span class="nt"&gt;--mode&lt;/span&gt; a2a &lt;span class="nt"&gt;--app-name&lt;/span&gt; app &lt;span class="s2"&gt;"hello"&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  What to take away
&lt;/h2&gt;

&lt;p&gt;Of the commands we ran, the extension handled only &lt;code&gt;run&lt;/code&gt; and &lt;code&gt;eval generate&lt;/code&gt;. &lt;code&gt;eval grade&lt;/code&gt;, &lt;code&gt;infra&lt;/code&gt;, and &lt;code&gt;deploy&lt;/code&gt; ran exactly as they do for an ADK agent. Apply the infra before the first deploy, because that step creates the session database.&lt;/p&gt;

&lt;p&gt;To bring an existing agent over, run &lt;code&gt;scaffold enhance&lt;/code&gt;. It writes the template into your project and moves the files it replaces to &lt;code&gt;.agents-cli-original/&lt;/code&gt;. Then move your &lt;code&gt;ClaudeAgentOptions&lt;/code&gt; into &lt;code&gt;app/agent.py&lt;/code&gt;:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;agents-cli scaffold enhance google/agents-cli/extensions/claude/template@v1.13.0 &lt;span class="nt"&gt;-d&lt;/span&gt; cloud_run
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;To start fresh, install Agents CLI and follow Step 1:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;uvx google-agents-cli setup
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The &lt;a href="https://google.github.io/agents-cli/guide/extensions/first-party/" rel="noopener noreferrer"&gt;extension guide&lt;/a&gt; covers migration step by step, and Agents CLI is on &lt;a href="https://github.com/google/agents-cli" rel="noopener noreferrer"&gt;GitHub&lt;/a&gt;.&lt;/p&gt;

</description>
      <category>agents</category>
      <category>ai</category>
      <category>cli</category>
      <category>python</category>
    </item>
    <item>
      <title>A Modern Java Developer’s Guide for Google Cloud Databases</title>
      <dc:creator>Alexis MP</dc:creator>
      <pubDate>Mon, 09 Nov 2026 11:16:05 +0000</pubDate>
      <link>https://dev.to/googlecloud/a-modern-java-developers-guide-for-google-cloud-databases-3ckg</link>
      <guid>https://dev.to/googlecloud/a-modern-java-developers-guide-for-google-cloud-databases-3ckg</guid>
      <description>&lt;p&gt;In this Series so far: &lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;A Modern Java Developer’s Guide for Google Cloud Databases (this piece)&lt;/li&gt;
&lt;li&gt;&lt;a href="https://dev.to/googlecloud/alloydb-integration-for-modern-java-apps-connectivity-and-auth-5721"&gt;AlloyDB Integration for Modern Java Apps — Connectivity and Auth&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Streamlining AlloyDB Development with Spring Boot, Quarkus, Micronaut, AlloyDB Express, and AlloyDB Omni (coming soon)&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;As a Java developer I’ve learned to rely on JDBC, on the efficiency of connection pools, and on my favorite ORM or frameworks (Spring, Quarkus, or Micronaut). These are stable, predictable technologies that I can count on for building enterprise apps.&lt;/p&gt;

&lt;p&gt;This short intro and upcoming detailed posts are meant to serve as a guide for &lt;strong&gt;Java developers in their Google Cloud Database journey&lt;/strong&gt; , including when moving from conventional on-premises databases to a cloud-native approach. We’ll cover configuration, security, performance, monitoring, and overall developer experience.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F1heaog2i9umgt9hqtodk.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F1heaog2i9umgt9hqtodk.png" width="800" height="446"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;No need to abandon your stack to use Google Cloud databases: all of them plug right into standard JDBC, HikariCP, and your favorite frameworks. Whether you’re relying on the AlloyDB Java Connector for automated mTLS or designing schemas for globally distributed Spanner, this series will help you pick and configure the right tool for your workload.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;AlloyDB&lt;/strong&gt; gives you full PostgreSQL wire compatibility backed by a Google-engineered storage layer and a 99.99% SLA (including maintenance) for demanding transactional and AI workloads. Java developers typically use the AlloyDB Java Connector with standard JDBC and HikariCP.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Cloud SQL&lt;/strong&gt; removes operational toil when running MySQL, PostgreSQL, and SQL Server databases with read pools, data caching, and near-zero downtime maintenance. Connecting from Java is done using the Cloud SQL Java Connector or using Auth Proxy alongside standard JDBC and Spring Boot.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Spanner&lt;/strong&gt; is a globally distributed database that pairs horizontal scalability, strong consistency, and a 99.999% SLA with multi-model support: relational, graph, vector, and key-value. You can wire it into Java applications using either the Spanner JDBC Driver, the native Java Client Library, Spring Data Spanner, or PGAdapter with JPA.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Firestore&lt;/strong&gt; is a serverless NoSQL document database offering a Native and a MongoDB-compatible mode. It was designed for schema flexibility, real-time sync, automatic scaling, and up to a 5 nine SLA. Java devs will typically use the Firestore Java Client Library, Spring Cloud GCP, or even the standard MongoDB Java Driver.&lt;/p&gt;

&lt;p&gt;Here’s a &lt;a href="https://gist.github.com/alexismp/2976cf53d946a6bc2175957d4a430de2" rel="noopener noreferrer"&gt;tabular presentation&lt;/a&gt; summarizing all of this knowledge for Java developers and architects.&lt;/p&gt;

&lt;p&gt;Upcoming Java-focused blogs here will cover :&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;AlloyDB for PostgreSQL&lt;/strong&gt; (including &lt;a href="https://cloud.google.com/alloydb/omni" rel="noopener noreferrer"&gt;AlloyDB Omni&lt;/a&gt; for local and edge use-cases, and &lt;a href="https://cloud.google.com/blog/products/databases/introducing-alloydb-express-for-rapid-testing-and-prototyping" rel="noopener noreferrer"&gt;AlloyDB Express&lt;/a&gt; for rapid testing and prototyping)&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Cloud SQL&lt;/strong&gt; (focusing on &lt;strong&gt;MySQL&lt;/strong&gt; though the same connector patterns apply to PostgreSQL and SQL Server)&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Spanner&lt;/strong&gt; and &lt;strong&gt;Firestore&lt;/strong&gt; (focusing on their native Google Cloud APIs).&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fab86q050nzkaupw9suu9.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fab86q050nzkaupw9suu9.png" width="800" height="411"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;We’ll use &lt;strong&gt;Spring Boot&lt;/strong&gt; as the primary framework but also show how these building blocks apply to &lt;strong&gt;Quarkus and Micronaut&lt;/strong&gt; developers.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;AlloyDB AI&lt;/strong&gt; and the &lt;strong&gt;MCP Toolbox for Databases&lt;/strong&gt; (featuring the newly released &lt;a href="https://github.com/googleapis/mcp-toolbox-sdk-java" rel="noopener noreferrer"&gt;&lt;strong&gt;MCP Toolbox Java SDK v1.0&lt;/strong&gt;&lt;/a&gt;) are also on the menu, all from a Java developer’s perspective. Onwards!&lt;/p&gt;




</description>
      <category>java</category>
      <category>database</category>
      <category>googlecloud</category>
    </item>
    <item>
      <title>Find the production agent failures that every health check misses</title>
      <dc:creator>Elia Secchi</dc:creator>
      <pubDate>Thu, 05 Nov 2026 15:46:25 +0000</pubDate>
      <link>https://dev.to/googlecloud/find-the-production-agent-failures-that-every-health-check-misses-39j8</link>
      <guid>https://dev.to/googlecloud/find-the-production-agent-failures-that-every-health-check-misses-39j8</guid>
      <description>&lt;p&gt;Your agent returns HTTP 200 and stays inside its latency budget, and no tool call raises an error. Yet it records seat 3A as confirmed without ever asking the seat-selection sub-agent whether 3A is free. Nothing in your dashboards turns red, because the only evidence of the mistake sits inside the conversation.&lt;/p&gt;

&lt;p&gt;AQuA (Ambient Quality Agent) is an open-source quality agent. It surfaces that kind of failure across many sessions and, when you ask, traces a failure to the line of code behind it. We built AQuA, and Dima Melnyk and I wrote the full &lt;a href="https://developers.googleblog.com/the-outer-loop-insights-first-an-ambient-quality-agent-that-diagnoses-your-production-agent/" rel="noopener noreferrer"&gt;Developers Blog post&lt;/a&gt; about it. This companion walks through the same sweep and shows how to run the dashboard on your laptop with no cloud project.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why this is hard
&lt;/h2&gt;

&lt;p&gt;Getting an agent to the first 80%, where it handles the cases you anticipated, takes days with an eval dataset and a run, grade, fix and compare loop. Then you launch and the curve flattens. Usage shifts, and a model, harness or tool change can alter task success while health checks stay green.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fvx7gilarnyjws5liubh8.webp" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fvx7gilarnyjws5liubh8.webp" alt="A quality curve that climbs before launch, flattens with silent failures after it, and climbs again once AQuA runs." width="800" height="800"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Figure 1. Quality stalls after launch on failures that report 200 OK.&lt;/p&gt;

&lt;p&gt;Diagnosing a failure is harder still, because several layers look identical from the outside. The model may have dropped a constraint, the harness may have misrouted to a sub-agent, a tool contract may have rejected an undocumented value, or the instructions may lack a rule. A raw trajectory shows what happened but says little about why, and nobody can read failing trajectories next to the code that produced them by hand at production volume.&lt;/p&gt;

&lt;h2&gt;
  
  
  What AQuA is
&lt;/h2&gt;

&lt;p&gt;AQuA is an &lt;a href="https://adk.dev" rel="noopener noreferrer"&gt;Agent Development Kit (ADK)&lt;/a&gt; agent that runs unattended beside your agent in your Google Cloud project, outside the request path. It sweeps production sessions from &lt;a href="https://docs.cloud.google.com/trace/docs" rel="noopener noreferrer"&gt;Cloud Trace&lt;/a&gt;, &lt;a href="https://docs.cloud.google.com/logging/docs" rel="noopener noreferrer"&gt;Cloud Logging&lt;/a&gt; or &lt;a href="https://docs.cloud.google.com/bigquery/docs" rel="noopener noreferrer"&gt;BigQuery&lt;/a&gt; on a schedule, after each deployment, or on demand. Each sweep has five stages:&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Ff9pe3jk0asa6bp4me5r5.webp" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Ff9pe3jk0asa6bp4me5r5.webp" alt="Five boxes, Sample (random sessions), Review (one finding per failure), Cluster (same cause, grouped), Verify (opt-in, false alarms dropped) and Track (NEW or RECURRING), lead in turn to a larger box labelled Root cause at file:line." width="800" height="447"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Figure 2. Each stage narrows the findings, and when you ask, AQuA adds a root cause at file:line.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Sample:&lt;/strong&gt; AQuA pulls a random sample of up to 1,000 recent sessions.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Review:&lt;/strong&gt; a review agent reads a shortened copy of each session, can query the full session with a &lt;code&gt;jq&lt;/code&gt; tool, and writes an &lt;code&gt;actual / expected&lt;/code&gt; finding for every failure.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Cluster:&lt;/strong&gt; AQuA groups findings that share a failure mechanism into candidate issues.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Verify (opt-in):&lt;/strong&gt; with &lt;code&gt;agents-cli aqua attach --verification&lt;/code&gt;, a separate model checks each candidate against up to three transcripts, and &lt;code&gt;--enforce-verification&lt;/code&gt; withholds rejected candidates.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Track:&lt;/strong&gt; AQuA matches the survivors against open insights as &lt;code&gt;NEW&lt;/code&gt;, &lt;code&gt;RECURRING&lt;/code&gt;, or &lt;code&gt;RESOLVED&lt;/code&gt; after 14 days unseen.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Root-cause analysis runs only on request, from the dashboard &lt;strong&gt;Chat&lt;/strong&gt; or &lt;code&gt;agents-cli aqua run&lt;/code&gt;. AQuA reads the failing trajectories against the source snapshot captured at deploy time and cites lines as &lt;code&gt;&amp;lt;path&amp;gt;:&amp;lt;start&amp;gt;-&amp;lt;end&amp;gt;&lt;/code&gt;. It proposes an edit and never applies it or opens a pull request. When the fault lies outside your code, it pins the failure to a step in the trajectory without citing a line.&lt;/p&gt;

&lt;h2&gt;
  
  
  Walkthrough: a sweep on travel-concierge
&lt;/h2&gt;

&lt;p&gt;We pointed an earlier AQuA release, with verification on, at &lt;code&gt;travel-concierge&lt;/code&gt; from &lt;code&gt;google/adk-recipes&lt;/code&gt;, an agent that routes a traveler across sub-agents. We ran 32 replays of four scripted journeys, which produced 1,583 OpenTelemetry spans. Five of the 32 sessions passed, and 27 produced 42 findings, such as this one:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;actual:   When the user selected outbound flight UA204 and requested seats 3A and 3B in the same
          turn, planning_agent saved those seat numbers directly to session state without calling
          flight_seat_selection_agent to check whether 3A and 3B were available.
expected: planning_agent should call flight_seat_selection_agent to verify seat availability and
          pricing before saving outbound or return seat numbers to session state.
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Clustering grouped the 42 findings into 9 candidates. The verifier rejected 3, including one that merged unrelated mismatches from two sub-agents. The 6 verified issues included skipped seat checks (15 sessions), a vegan profile lost between &lt;code&gt;inspiration_agent&lt;/code&gt; and &lt;code&gt;place_agent&lt;/code&gt; (7 sessions), and a &lt;code&gt;poi_agent&lt;/code&gt; declared with &lt;code&gt;tools=[]&lt;/code&gt; despite needing verified map data (5 sessions).&lt;/p&gt;

&lt;p&gt;In that release, choosing &lt;strong&gt;Investigate&lt;/strong&gt; on the top insight sent the diagnosis agent through the 33-file snapshot of the deployed revision (in v0.3.0 you ask &lt;strong&gt;Chat&lt;/strong&gt; for a root cause). It traced the seat bug to line 93 of &lt;code&gt;travel_concierge/sub_agents/planning/prompt.py&lt;/code&gt;, where the instructions call the seat sub-agent only when presenting a seat map. It traced the vegan bug to line 23 of &lt;code&gt;travel_concierge/sub_agents/inspiration/prompt.py&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fdxcyveg90f8ia4fdcdrb.webp" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fdxcyveg90f8ia4fdcdrb.webp" alt="On the left, a box reading 200 OK and seat 3A confirmed; an arrow leads to a box on the right reading planning/prompt.py:93 and 15 sessions never checked the seat, with 5/32 to 13/32 full passes below it." width="800" height="447"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Figure 3. The failure as your dashboard sees it and as AQuA sees it; the full-pass gain came from this fix and the vegan one.&lt;/p&gt;

&lt;p&gt;After the two one-line prompt fixes shipped as a second revision, we replayed the same 32 sessions:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Measure&lt;/th&gt;
&lt;th&gt;Before&lt;/th&gt;
&lt;th&gt;After&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Seat check bypass&lt;/td&gt;
&lt;td&gt;15 sessions&lt;/td&gt;
&lt;td&gt;2 sessions&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Vegan profile dropped&lt;/td&gt;
&lt;td&gt;7 sessions&lt;/td&gt;
&lt;td&gt;0 sessions&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;
&lt;code&gt;poi_agent&lt;/code&gt; empty tool list&lt;/td&gt;
&lt;td&gt;5 sessions&lt;/td&gt;
&lt;td&gt;untouched, still tracked&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Full-session passes&lt;/td&gt;
&lt;td&gt;5/32&lt;/td&gt;
&lt;td&gt;13/32&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;h2&gt;
  
  
  Cost and limits
&lt;/h2&gt;

&lt;p&gt;In the earlier release, the 32-session sweep cost $3.76 at standard Gemini platform pricing, about $0.12 per session at about 50 spans each. A 96-session sweep of a single agent (5 to 6 spans per session) cost $0.70, and on-demand root-cause analysis cost $0.33 to $2.47 per insight. v0.3.0 reviews each session with an agent that can query the whole session, so its costs differ. An agent that diagnoses your agent will sometimes be wrong, so these limits and safeguards matter:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Sampling is random.&lt;/strong&gt; A sweep reads up to 1,000 sessions, so a regression that breaks 1% of a critical workflow can slip through. You can store an SQL selector with &lt;code&gt;agents-cli aqua attach --selector-file&lt;/code&gt; to choose which conversations it reads.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Verification is opt-in and bounded.&lt;/strong&gt; It covers up to 50 clusters per run and three transcripts each. AQuA does not check every member session, so a cluster's trace count is only a rough priority.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Insights carry citations.&lt;/strong&gt; The insight schema has no confidence field. The server validates every cited &lt;code&gt;&amp;lt;path&amp;gt;:&amp;lt;start&amp;gt;-&amp;lt;end&amp;gt;&lt;/code&gt; range against the snapshot and rejects a reference to a nonexistent file or line.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;These numbers come from scripted journeys, not your traffic. Two one-line prompt edits moved full passes from 5/32 to 13/32, and no health check pointed at either line.&lt;/p&gt;

&lt;h2&gt;
  
  
  Try it
&lt;/h2&gt;

&lt;p&gt;The demo needs Python 3.11 to 3.13, &lt;code&gt;uv&lt;/code&gt; and &lt;code&gt;npm&lt;/code&gt;. It shows a generated month of data with no Google Cloud project, credentials or model calls. &lt;code&gt;make install&lt;/code&gt; creates the virtual environment, and &lt;code&gt;make demo&lt;/code&gt; builds the front end and then starts the dashboard:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;git clone https://github.com/google/adk-recipes.git
&lt;span class="nb"&gt;cd &lt;/span&gt;adk-recipes/core/python/ambient-quality-agent
make &lt;span class="nb"&gt;install
&lt;/span&gt;make demo
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F2rdutp1v1c7dtq4k3hi3.webp" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F2rdutp1v1c7dtq4k3hi3.webp" alt="A terminal showing selected startup lines of make demo: 16 generated insights, the guided-tour link and the dashboard serving on port 8080." width="800" height="302"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Figure 4. Selected lines of what &lt;code&gt;make demo&lt;/code&gt; printed at startup, read from its log with &lt;code&gt;sed&lt;/code&gt;: the generated data, the tour link and the dashboard on port 8080.&lt;/p&gt;

&lt;p&gt;Open &lt;code&gt;http://localhost:8080/?tour&lt;/code&gt; for a guided tour. The demo observes &lt;code&gt;travel_desk_agent&lt;/code&gt;, a synthetic travel-and-expense agent so its insights differ from the walkthrough. None has a root cause unless you start the demo with &lt;code&gt;MOCK_ROOT_CAUSE=0.5 make demo&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F3zdw5615d1yzcokpp0a2.webp" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F3zdw5615d1yzcokpp0a2.webp" alt="The top of the AQuA insights page, listing three recurring failures of the demo agent with their occurrence and trajectory counts." width="652" height="554"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Figure 5. The insights page on demo data: 16 insights affecting 3,742 trajectories, ranked by impact.&lt;/p&gt;

&lt;p&gt;To attach AQuA to your own &lt;a href="https://google.github.io/agents-cli/" rel="noopener noreferrer"&gt;Agents CLI&lt;/a&gt; project, add the extension, which pins v0.3.0 here. Agents CLI extensions are experimental, and AQuA needs &lt;code&gt;agents-cli&lt;/code&gt; 1.10 or later:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;agents-cli extension add google/adk-recipes#core/python/ambient-quality-agent &lt;span class="nt"&gt;--ref&lt;/span&gt; ambient-quality-agent/v0.3.0
agents-cli &lt;span class="nb"&gt;install&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Deployment needs a Google Cloud project in an organization, because the dashboard sits behind &lt;a href="https://docs.cloud.google.com/iap/docs" rel="noopener noreferrer"&gt;Identity-Aware Proxy&lt;/a&gt; (IAP). Without an organization, set &lt;code&gt;TF_VAR_ui_iap_enabled=false&lt;/code&gt; and open it with &lt;code&gt;agents-cli aqua ui-proxy&lt;/code&gt;. The &lt;a href="https://github.com/google/adk-recipes/tree/main/core/python/ambient-quality-agent" rel="noopener noreferrer"&gt;AQuA README&lt;/a&gt; covers deployment and the &lt;code&gt;agents-cli aqua&lt;/code&gt; commands. The &lt;a href="https://developers.googleblog.com/the-outer-loop-insights-first-an-ambient-quality-agent-that-diagnoses-your-production-agent/" rel="noopener noreferrer"&gt;Developers Blog post&lt;/a&gt; has the full design discussion and the team credits.&lt;/p&gt;

</description>
      <category>agents</category>
      <category>ai</category>
      <category>debugging</category>
      <category>testing</category>
    </item>
    <item>
      <title>From Python to Rust, Go, to… Java? Steering Your AI Coding Agent Through Architecture Trade-Offs</title>
      <dc:creator>Alexis MP</dc:creator>
      <pubDate>Mon, 26 Oct 2026 12:40:13 +0000</pubDate>
      <link>https://dev.to/googlecloud/from-python-to-rust-go-to-java-steering-your-ai-coding-agent-through-architecture-trade-offs-3mlj</link>
      <guid>https://dev.to/googlecloud/from-python-to-rust-go-to-java-steering-your-ai-coding-agent-through-architecture-trade-offs-3mlj</guid>
      <description>&lt;p&gt;When you pair-program with an autonomous AI coding agent, something magical happens: you go from an idea to a deployed, production-grade cloud application in an afternoon. But something else happens, too. If you don’t actively steer your agent’s architectural compass, it will happily ride the defaults of its training data or jump onto the latest Hacker News hype train.&lt;/p&gt;

&lt;p&gt;Here is the story of building &lt;strong&gt;Akkord&lt;/strong&gt; , an AI-powered sheet music harmonizer and guitar arranger running on Google Cloud, and what happened when I challenged my coding agent to rethink the stack across &lt;strong&gt;Python&lt;/strong&gt; , &lt;strong&gt;Rust&lt;/strong&gt; , &lt;strong&gt;Go&lt;/strong&gt; , and &lt;strong&gt;Modern Java (JDK 21/25)&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Ffwj0po2dmtqbl9445fkd.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Ffwj0po2dmtqbl9445fkd.png" width="800" height="447"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Akkord: Computer Vision Meets Musical Counterpoint
&lt;/h3&gt;

&lt;p&gt;Musicians, especially guitarists, frequently run into two problems with PDF sheet music:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Bare scores with no chords:&lt;/strong&gt; You have a classical or folk melody on a 5-line staff, and you want playable guitar chords and a tasteful bass line added directly above the staves.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Scores with questionable chords:&lt;/strong&gt; You download a lead sheet where every measure just pounds the root note in the bass (Am -&amp;gt; Dm -&amp;gt; E -&amp;gt; Am), doubling the melody note in parallel octaves and ignoring harmonic counterpoint.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;&lt;strong&gt;Akkord&lt;/strong&gt; automates both workflows end-to-end:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Pixel-Level Staff &amp;amp; TAB Geometry Detection:&lt;/strong&gt; It rasterizes each PDF page and scans horizontal ink-density slices (accounting for tilted scans and stripping outer eighth/sixteenth-note beams) to distinguish &lt;strong&gt;5-line musical staves&lt;/strong&gt; from &lt;strong&gt;6-line guitar TABs&lt;/strong&gt; , computing exact bounding boxes (S1..Sn) in PDF points.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Visual Rulers for Multimodal AI Grounding:&lt;/strong&gt; Instead of asking an LLM to guess (x, y) coordinates blindly, Akkord generates a &lt;strong&gt;Guide Image&lt;/strong&gt; for every page with red vertical X-coordinate rulers (40pt..560pt) and color-coded translucent bands over each detected staff system, plus extracted vector text tokens (re-joining spaced chord tokens like G m into Gm).&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Multimodal Harmonic &amp;amp; Counterpoint Analysis:&lt;/strong&gt; It sends the clean score, the coordinate guide image, and the extracted tokens to &lt;strong&gt;Gemini (&lt;code&gt;gemini-4-argon&lt;/code&gt; / &lt;code&gt;gemini-3.8-flash&lt;/code&gt;)&lt;/strong&gt; with strict musical rules:&lt;/li&gt;
&lt;/ol&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;em&gt;Avoid systematic root-position monotony and melody doubling: propose slash chords (&lt;/em&gt;&lt;em&gt;Am/G, Dm/F, E7/G#) using contrary motion and walking-bass lines.&lt;/em&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;em&gt;Never confuse a 3-sharp guitar score in A Major with F# Minor just because the guitar reads an octave higher and uses low ledger lines (&lt;/em&gt;&lt;em&gt;E2, F#2, G#2)!&lt;/em&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Collision-Free Vector PDF Overlay:&lt;/strong&gt; Using a &lt;strong&gt;2pt ink-density grid&lt;/strong&gt; and a &lt;strong&gt;spiral low-ink search&lt;/strong&gt; , Akkord places vector chord pills, uniform-sized chord corrections with superscript footnote numbers (G → G7/B ¹), 6-string fretboard diagrams, and an appended harmonic report page, without drawing over a notehead, lyric, or tie.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Cloud-Native Batch Pipeline:&lt;/strong&gt; Users sign in with Google OAuth 2.0, pick a Google Drive folder (or drop PDFs), and process batches asynchronously via &lt;strong&gt;Cloud Run&lt;/strong&gt; , &lt;strong&gt;Cloud Tasks&lt;/strong&gt; , &lt;strong&gt;Cloud Firestore&lt;/strong&gt; , and &lt;strong&gt;Cloud Storage&lt;/strong&gt; , complete with a 25-PDF/day quota, automatic next-day deferral, and an 18:00 daily usage &amp;amp; token-cost digest email.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fpb981n4xo8766mq2jwzv.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fpb981n4xo8766mq2jwzv.png" width="800" height="605"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  How the Agent Harness Led Me to 100% Python
&lt;/h3&gt;

&lt;p&gt;When I started building Akkord inside my AI agent harness with no particular language or framework SKILLS installed, I didn’t specify a programming language. I focused entirely on the domain: &lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;em&gt;Read music sheets from Google Drive, analyze the harmony with Gemini, and write chords onto the PDF.&lt;/em&gt;&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;The agent reached for &lt;strong&gt;Python 3.12&lt;/strong&gt; :&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Web &amp;amp; APIs:&lt;/strong&gt; FastAPI + Uvicorn&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;PDF &amp;amp; Image Processing:&lt;/strong&gt; PyMuPDF (fitz) + Pillow + ReportLab&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;AI &amp;amp; Cloud:&lt;/strong&gt; google-genai SDK + direct REST calls to Firestore, Cloud Storage, Cloud Tasks, and Drive API v3&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  Why the Agent Chose Python (And Why It Works Just Fine!)
&lt;/h3&gt;

&lt;p&gt;AI coding agents love Python for rapid iterative prototyping. When I asked the agent to tweak the staff-line clustering algorithm so outer beams wouldn’t look like a 6th staff line, or to adjust the spiral ink-avoidance penalty inside pdf_annotator.py, the edit-test loop just took two seconds.&lt;/p&gt;

&lt;p&gt;And honestly? &lt;strong&gt;The Python version on Cloud Run works great and I was able to deliver the entire app in less than than 24h.&lt;/strong&gt; Because the main latency in Akkord is the multimodal Gemini call (typically &lt;strong&gt;4 to 12 seconds&lt;/strong&gt; per score), Python’s ~350 ms of local PDF pixel scanning and vector drawing barely matters on a single file.&lt;/p&gt;

&lt;p&gt;Still, as a developer, you look at nested Python &lt;code&gt;for&lt;/code&gt; loops iterating over millions of grayscale pixels in &lt;code&gt;pdf\_layout.py&lt;/code&gt; and &lt;code&gt;pdf\_annotator.py&lt;/code&gt;, plus a ~320 MB container memory footprint, and you start thinking: &lt;em&gt;Surely, I can do better!&lt;/em&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  “Rewrite It for Speed!” — The Trendy 100% Rust Reflex
&lt;/h3&gt;

&lt;p&gt;Curious, I gave the agent a classic developer prompt:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;em&gt;If I were to rewrite everything in another more efficient language with faster execution, what should it be?&lt;/em&gt;&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;If you ask an LLM in 2026 for “maximum efficiency and faster execution,” its first instinct is almost always &lt;strong&gt;Rust&lt;/strong&gt; (Axum + Tokio + image + pdf-writer).&lt;/p&gt;

&lt;p&gt;To be fair, Rust makes the pixel-scanning loops in &lt;code&gt;pdf\_layout.py&lt;/code&gt; and the spiral ink-collision search in &lt;code&gt;pdf\_annotator.py&lt;/code&gt; &lt;strong&gt;15x to 25x faster&lt;/strong&gt; , compiling them down to SIMD-vectorized machine code with zero garbage collection pauses and a tiny ~25 MB RAM footprint.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The catch?&lt;/strong&gt; Writing OAuth 2.0 session management, multipart Google Drive uploads, dynamically nested Cloud Firestore REST JSON envelopes, and HTML email builders in pure Rust means fighting borrow-checker lifetimes on deeply nested JSON trees, all for code that spends 99.9% of its life waiting on network I/O.&lt;/p&gt;

&lt;h3&gt;
  
  
  Splitting the Brain — The Go + Rust Polyglot Architecture
&lt;/h3&gt;

&lt;p&gt;So I refined my prompt:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;em&gt;Consider writing different parts in different languages if it makes sense.&lt;/em&gt;&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;Immediately, the agent pivoted from a monolithic rewrite to a &lt;strong&gt;Polyglot Architecture&lt;/strong&gt; that splits the system along its true hardware boundary:&lt;br&gt;&lt;br&gt;
&lt;strong&gt;I/O-bound Cloud Orchestration&lt;/strong&gt; vs. &lt;strong&gt;CPU-bound Pixel &amp;amp; Vector Geometry&lt;/strong&gt;.&lt;/p&gt;

&lt;h4&gt;
  
  
  Part 1: Go (cmd/akkord, internal/…) for the Cloud Server &amp;amp; Orchestrator
&lt;/h4&gt;

&lt;ul&gt;
&lt;li&gt;Using 100% Go standard library (net/http, compress/zlib, crypto/hmac, archive/zip), Go handles Cloud Run cold starts in &lt;strong&gt;&amp;lt;80 ms&lt;/strong&gt; , manages goroutine worker pools and SSE streaming effortlessly, and speaks REST JSON to Gemini API, Firestore, GCS, and Google Drive with clean, readable structs.&lt;/li&gt;
&lt;/ul&gt;

&lt;h4&gt;
  
  
  Part 2: Rust (akkord-pdf-engine) for the Native PDF Vision &amp;amp; Vector Engine
&lt;/h4&gt;

&lt;ul&gt;
&lt;li&gt;Invoked by Go via a zero-overhead JSON stdin/stdout bridge, the native Rust binary uses rayon data parallelism, image, imageproc, ab_glyph, and lopdf to rasterize pages, detect 5-line staves vs. 6-line TABs, compute the 2pt ink grid, and inject raw PDF content streams (BT/ET, Bézier curves c, vector rectangles re) in an estimated &lt;strong&gt;~25–40 ms per page!&lt;/strong&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fomlqk4c7m9lzl57xzobr.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fomlqk4c7m9lzl57xzobr.png" width="800" height="437"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Nudging the Agent Toward Modern Java (JDK 21/25 LTS)
&lt;/h3&gt;

&lt;p&gt;The Go + Rust polyglot architecture is awesome, if your team is happy maintaining both a &lt;code&gt;Cargo.toml&lt;/code&gt; and a &lt;code&gt;go.mod&lt;/code&gt; in the same Dockerfile.&lt;/p&gt;

&lt;p&gt;But what if you work in an organization where the platform, security, observability, and engineering teams are standardized on the &lt;strong&gt;JVM&lt;/strong&gt;?&lt;br&gt;&lt;br&gt;
Many LLMs still carry an outdated mental model of Java from the Java 8 era (remember heavyweight application servers, thread-per-request exhaustion, slow startup?).&lt;/p&gt;

&lt;p&gt;When you nudge the agent to evaluate &lt;strong&gt;Modern Java (Java 21 / Java 25 LTS)&lt;/strong&gt; for the exact same workload, the technical picture changes dramatically:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Project Loom (Virtual Threads — JEP 444):&lt;/strong&gt; Just like Go’s goroutines, Java’s Virtual Threads (&lt;code&gt;Executors.newVirtualThreadPerTaskExecutor()&lt;/code&gt;) allow Akkord to fan out dozens of concurrent Google Drive downloads, Gemini API requests, and Firestore writes with simple, readable blocking code and near-zero OS thread overhead.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;JIT Vectorization &amp;amp; the Vector API&lt;/strong&gt; (&lt;code&gt;jdk.incubator.vector&lt;/code&gt;) for PDF Pixel Math: Python was slow on &lt;code&gt;pdf\_layout.py&lt;/code&gt; and &lt;code&gt;pdf\_annotator.py&lt;/code&gt; because Python boxes every integer and interprets loops bytecode-by-bytecode. In modern Java, primitive &lt;code&gt;byte[]&lt;/code&gt; and &lt;code&gt;int[]&lt;/code&gt; arrays on the HotSpot C2 compiler (or GraalVM JIT) are automatically unrolled and compiled into*&lt;em&gt;AVX-512 / ARM Neon SIMD instructions *&lt;/em&gt;. Scanning a 1240×1754 page for 5-line staves or running the 2pt spiral ink-collision search in pure Java takes** ~25–45 ms**, within striking distance of Rust, &lt;em&gt;without needing a second language or FFI boundary!&lt;/em&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Mature, Pure-Java PDF &amp;amp; Graphics Ecosystem:&lt;/strong&gt; Instead of shelling out to C binaries or stitching together separate crates for font metrics and PDF dictionaries, Java has PDFBox 3.x and hardware-accelerated Java2D (&lt;code&gt;BufferedImage&lt;/code&gt;, &lt;code&gt;Graphics2D&lt;/code&gt;, TrueType glyph layout) built right in.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;First-Class Google Cloud &amp;amp; Enterprise Ecosystem:&lt;/strong&gt; Google’s official Java SDKs for &lt;strong&gt;Gemini&lt;/strong&gt; , &lt;strong&gt;Firestore&lt;/strong&gt; , &lt;strong&gt;Cloud Storage&lt;/strong&gt; , &lt;strong&gt;Cloud Tasks&lt;/strong&gt; , and &lt;strong&gt;Google Drive&lt;/strong&gt; are tier-1 citizens with built-in gRPC connection pooling, automatic retries, and OpenTelemetry tracing. Additionally, framework libraries like &lt;code&gt;LangChain4j&lt;/code&gt; provide clean, idiomatic wrappers around Gemini API calls, streamlining prompt orchestration and structured AI outputs in Java.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Fixing Cold Starts with GraalVM Native Image:&lt;/strong&gt; Compile the Java service with &lt;strong&gt;GraalVM Native Image&lt;/strong&gt; (via Quarkus, Micronaut, or Spring Boot 3.4+) and you get a &lt;strong&gt;~50–90 ms Cloud Run cold start&lt;/strong&gt; and a &lt;strong&gt;~60 MB RAM footprint&lt;/strong&gt; , matching Go while keeping 100% of your codebase in a single, type-safe language.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fn8t9ij5ifc9wqlgesyx5.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fn8t9ij5ifc9wqlgesyx5.png" width="800" height="437"&gt;&lt;/a&gt;&lt;br&gt;
&lt;em&gt;The Complete Architecture Showdown&lt;/em&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Conclusion: Steer Your Agent, Don’t Let Defaults Steer You
&lt;/h3&gt;

&lt;p&gt;Here are three lessons about working with modern AI coding assistants:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Agents default to frictionless velocity first.&lt;/strong&gt; Left unprompted, an agent will pick Python + FastAPI because it minimizes token count and maximizes first-draft success. And for 80% of cloud apps, including our working Python release of Akkord, that’s a fantastic starting point!&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Vague optimization prompts trigger trendy answers.&lt;/strong&gt; Ask an agent &lt;em&gt;“what is the fastest language?”&lt;/em&gt; in the abstract, and it will hand you a 100% Rust rewrite, even if 90% of your time in the app is waiting on an LLM API and Google Drive.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Specific architectural nudges unlock the best engineering insights.&lt;/strong&gt; When you ask your agent to &lt;em&gt;consider polyglot boundaries&lt;/em&gt; (leading to &lt;strong&gt;Go + Rust&lt;/strong&gt; ) or to &lt;em&gt;evaluate modern capabilities of an established enterprise platform&lt;/em&gt; (leading to &lt;strong&gt;Modern Java with Virtual Threads, SIMD vectorization, and GraalVM&lt;/strong&gt; ), the agent stops repeating clichés and starts doing real systems engineering.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Even if you standardize your backend services on Modern Java, keeping a standalone developer or admin &lt;strong&gt;CLI in Go remains a highly practical choice&lt;/strong&gt;. Go’s instant startup time, cross-compilation to single, self-contained static binaries with zero JVM dependency, and lightweight memory footprint make it exceptionally well-suited for fast command-line tooling.&lt;/p&gt;

&lt;p&gt;Your AI agent knows how to write Python, Rust, Go, and modern Java fluently. The secret is remembering that &lt;strong&gt;you are still the Architect&lt;/strong&gt;. Don’t hesitate to challenge your tools, ask &lt;em&gt;“what about language X?”&lt;/em&gt;, and steer them toward the stack that fits your team, your runtime, and your company best!&lt;/p&gt;




</description>
      <category>agents</category>
      <category>ai</category>
      <category>architecture</category>
      <category>softwaredevelopment</category>
    </item>
    <item>
      <title>Scaling stateful Gemini 4 Argon agent sessions on GKE: Connection draining, startup boost, and session persistence</title>
      <dc:creator>Olivier Bourgeois</dc:creator>
      <pubDate>Thu, 22 Oct 2026 16:05:20 +0000</pubDate>
      <link>https://dev.to/googlecloud/scaling-stateful-gemini-4-argon-agent-sessions-on-gke-connection-draining-startup-boost-and-13ln</link>
      <guid>https://dev.to/googlecloud/scaling-stateful-gemini-4-argon-agent-sessions-on-gke-connection-draining-startup-boost-and-13ln</guid>
      <description>&lt;p&gt;Most Kubernetes architectures are built around a comfortable assumption: HTTP requests arrive, execute stateless logic in under 200 milliseconds, and return a response. Multi-turn &lt;a href="https://cloud.google.com/gemini-enterprise-agent-platform/models/gemini/4-argon" rel="noopener noreferrer"&gt;Gemini 4 Argon&lt;/a&gt; agent workflows invert every part of that profile. A single user request can initiate a 3-minute bidirectional streaming session where the agent plans, executes external tools, ingests multi-megabyte documents into its context window, and streams intermediate reasoning steps back to the client.&lt;/p&gt;

&lt;p&gt;When you deploy these long-running, stateful agent orchestrators onto &lt;a href="https://cloud.google.com/kubernetes-engine" rel="noopener noreferrer"&gt;Google Kubernetes Engine (GKE)&lt;/a&gt; using default web-service configurations, three production failure modes appear almost immediately:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Startup CPU throttling&lt;/strong&gt;: Python and Node.js agent runtimes that initialize heavy AI SDKs, OpenTelemetry instrumentation, and local tokenizer caches spike CPU during boot, causing new Pods to fail readiness probes or stall Horizontal Pod Autoscaler (HPA) scale-out events.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Dropped in-flight reasoning loops during rollouts&lt;/strong&gt;: Default 30-second load balancer timeouts and abrupt &lt;code&gt;SIGTERM&lt;/code&gt; shutdowns sever active WebSocket or Server-Sent Events (SSE) streams mid-turn when GKE scales down or rolls out a new image.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Lost conversation and checkpoint state&lt;/strong&gt;: Storing multi-turn reasoning history purely in Pod memory means any container restart forces the agent to replay expensive Gemini 4 Argon token contexts from scratch.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;In this article, I will show you how to architect GKE for stateful Gemini 4 Argon agent runtimes by combining &lt;a href="https://cloud.google.com/kubernetes-engine/docs/how-to/boost-application-startup" rel="noopener noreferrer"&gt;VPA CPU Startup Boost&lt;/a&gt;, long-horizon connection draining via &lt;code&gt;GCPBackendPolicy&lt;/code&gt;, and external state checkpointing.&lt;/p&gt;

&lt;h2&gt;
  
  
  The lifecycle of a stateful Gemini 4 Argon agent Pod
&lt;/h2&gt;

&lt;p&gt;An agent runtime Pod traverses three distinct operational phases across its lifecycle:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Admission and cold boot (0s to 15s)&lt;/strong&gt;: GKE's VPA admission webhook injects a temporary CPU Startup Boost so the container initializes AI SDKs, tokenizers, and tracing pipelines without CPU throttling. Once it passes its &lt;code&gt;readinessProbe&lt;/code&gt;, GKE uses In-Place Pod Resize (IPPR) to step CPU down to baseline without restarting the container.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Steady-state streaming (up to 10 minutes)&lt;/strong&gt;: The GKE Gateway routes long-lived SSE or WebSocket streams with a 600-second backend timeout and cookie-based session affinity. The worker executes multi-turn tool loops with Gemini 4 Argon, checkpointing turn state to Cloud Memorystore.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Graceful shutdown and draining&lt;/strong&gt;: On rolling updates or scale-down events, a &lt;code&gt;preStop&lt;/code&gt; hook and a 300-second termination grace period allow active reasoning loops to finish and flush final checkpoints before the container exits.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Each GKE primitive addresses a specific stage of this lifecycle:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Operational challenge&lt;/th&gt;
&lt;th&gt;Default Kubernetes behavior&lt;/th&gt;
&lt;th&gt;GKE production pattern for Gemini 4 Argon agents&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Heavy SDK initialization&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Container throttled by baseline CFS CPU quota during boot&lt;/td&gt;
&lt;td&gt;
&lt;strong&gt;VPA CPU startup boost&lt;/strong&gt; (&lt;code&gt;updateMode: "Off"&lt;/code&gt;) grants temporary CPU headroom and resizes down live via IPPR after readiness&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Long streaming sessions&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Load balancer closes connection at 30 seconds&lt;/td&gt;
&lt;td&gt;
&lt;strong&gt;&lt;code&gt;GCPBackendPolicy&lt;/code&gt;&lt;/strong&gt; sets &lt;code&gt;timeoutSec: 600&lt;/code&gt; and &lt;code&gt;drainingTimeoutSec: 300&lt;/code&gt; with cookie-based session affinity&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Pod termination during rollout&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;SIGTERM&lt;/code&gt; followed by &lt;code&gt;SIGKILL&lt;/code&gt; after 30 seconds&lt;/td&gt;
&lt;td&gt;
&lt;strong&gt;&lt;code&gt;terminationGracePeriodSeconds: 300&lt;/code&gt;&lt;/strong&gt; paired with a graceful drain handler that completes active Gemini 4 Argon turns and checkpoints state&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;h2&gt;
  
  
  Eliminating cold-start throttling with VPA CPU startup boost
&lt;/h2&gt;

&lt;p&gt;Agent orchestrator images frequently bundle the &lt;code&gt;google-genai&lt;/code&gt; SDK, gRPC C-extensions, Pydantic schema validators, and document parsing libraries. If you size your container's baseline CPU request for steady-state I/O waiting (where the Pod spends most of its time awaiting network streams from the Gemini platform), the container only needs &lt;code&gt;500m&lt;/code&gt; CPU at steady state, but requires &lt;code&gt;1500m&lt;/code&gt; CPU during the first 10 seconds of startup.&lt;/p&gt;

&lt;p&gt;Using &lt;strong&gt;VPA CPU startup boost&lt;/strong&gt; (available in Public Preview on GKE &lt;code&gt;1.36.0-gke.4447000&lt;/code&gt; and later in the Rapid Channel), GKE temporarily boosts the container's CPU request at admission and uses Kubernetes In-Place Pod Resize (IPPR) to lower the CPU allocation back to &lt;code&gt;500m&lt;/code&gt; without restarting the container once it passes its &lt;code&gt;readinessProbe&lt;/code&gt;.&lt;/p&gt;

&lt;h3&gt;
  
  
  GKE Autopilot vs. GKE Standard sizing considerations
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;GKE Autopilot&lt;/strong&gt;: Autopilot validates CPU-to-memory ratios at admission. Ensure your Pod's memory request supports the boosted CPU value (on the general-purpose Autopilot compute platform, memory must fall between 1 GiB and 8 GiB per vCPU), or assign the workload to a &lt;code&gt;Performance&lt;/code&gt; &lt;code&gt;ComputeClass&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;GKE Standard&lt;/strong&gt;: Ensure your worker nodes have sufficient allocatable CPU headroom to accommodate the temporary startup boost factor.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Create a &lt;code&gt;VerticalPodAutoscaler&lt;/code&gt; policy named &lt;code&gt;agent-runtime-startup-boost.yaml&lt;/code&gt; that triples the baseline &lt;code&gt;500m&lt;/code&gt; CPU allocation to &lt;code&gt;1500m&lt;/code&gt; during boot and unboosts immediately (&lt;code&gt;durationSeconds: 0&lt;/code&gt;) upon readiness. Setting &lt;code&gt;updateMode: "Off"&lt;/code&gt; applies the startup boost and in-place downscale without enabling continuous VPA actuation, avoiding conflicts with Horizontal Pod Autoscaler (HPA):&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="na"&gt;apiVersion&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;autoscaling.k8s.io/v1&lt;/span&gt;
&lt;span class="na"&gt;kind&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;VerticalPodAutoscaler&lt;/span&gt;
&lt;span class="na"&gt;metadata&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
  &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;gemini-agent-startup-boost&lt;/span&gt;
  &lt;span class="na"&gt;namespace&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;agent-runtime&lt;/span&gt;
&lt;span class="na"&gt;spec&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
  &lt;span class="na"&gt;targetRef&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
    &lt;span class="na"&gt;apiVersion&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;apps/v1&lt;/span&gt;
    &lt;span class="na"&gt;kind&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;Deployment&lt;/span&gt;
    &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;gemini-session-orchestrator&lt;/span&gt;
  &lt;span class="na"&gt;updatePolicy&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
    &lt;span class="na"&gt;updateMode&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Off"&lt;/span&gt;
  &lt;span class="na"&gt;startupBoost&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
    &lt;span class="na"&gt;cpu&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
      &lt;span class="na"&gt;type&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;Factor&lt;/span&gt;
      &lt;span class="na"&gt;factor&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;3&lt;/span&gt;
      &lt;span class="na"&gt;durationSeconds&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;0&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  Configuring long-lived streaming and graceful termination
&lt;/h2&gt;

&lt;p&gt;To prevent GKE from severing active Gemini 4 Argon streaming sessions during a rolling deployment, you need to coordinate timeouts across two layers: the Kubernetes Pod lifecycle (&lt;code&gt;terminationGracePeriodSeconds&lt;/code&gt; and &lt;code&gt;preStop&lt;/code&gt; hooks) and the GKE Gateway load balancer (&lt;code&gt;GCPBackendPolicy&lt;/code&gt;).&lt;/p&gt;

&lt;p&gt;Save the following manifest as &lt;code&gt;agent-session-deployment.yaml&lt;/code&gt;:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="na"&gt;apiVersion&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;apps/v1&lt;/span&gt;
&lt;span class="na"&gt;kind&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;Deployment&lt;/span&gt;
&lt;span class="na"&gt;metadata&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
  &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;gemini-session-orchestrator&lt;/span&gt;
  &lt;span class="na"&gt;namespace&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;agent-runtime&lt;/span&gt;
&lt;span class="na"&gt;spec&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
  &lt;span class="na"&gt;replicas&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;4&lt;/span&gt;
  &lt;span class="na"&gt;strategy&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
    &lt;span class="na"&gt;type&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;RollingUpdate&lt;/span&gt;
    &lt;span class="na"&gt;rollingUpdate&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
      &lt;span class="na"&gt;maxSurge&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;25%&lt;/span&gt;
      &lt;span class="na"&gt;maxUnavailable&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;0&lt;/span&gt;
  &lt;span class="na"&gt;selector&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
    &lt;span class="na"&gt;matchLabels&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
      &lt;span class="na"&gt;app&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;gemini-session-orchestrator&lt;/span&gt;
  &lt;span class="na"&gt;template&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
    &lt;span class="na"&gt;metadata&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
      &lt;span class="na"&gt;labels&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
        &lt;span class="na"&gt;app&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;gemini-session-orchestrator&lt;/span&gt;
    &lt;span class="na"&gt;spec&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
      &lt;span class="na"&gt;serviceAccountName&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;gemini-agent-sa&lt;/span&gt;
      &lt;span class="na"&gt;terminationGracePeriodSeconds&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;300&lt;/span&gt;
      &lt;span class="na"&gt;containers&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
      &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;agent-server&lt;/span&gt;
        &lt;span class="na"&gt;image&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;us-central1-docker.pkg.dev/production-ai-platform/agents/gemini-4-session-server:v1.0.0&lt;/span&gt;
        &lt;span class="na"&gt;ports&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
        &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="na"&gt;containerPort&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;8080&lt;/span&gt;
        &lt;span class="na"&gt;env&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
        &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;GOOGLE_CLOUD_PROJECT&lt;/span&gt;
          &lt;span class="na"&gt;value&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="s"&gt;production-ai-platform"&lt;/span&gt;
        &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;REDIS_HOST&lt;/span&gt;
          &lt;span class="na"&gt;value&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="s"&gt;10.128.0.45"&lt;/span&gt;
        &lt;span class="na"&gt;resources&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
          &lt;span class="na"&gt;requests&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
            &lt;span class="na"&gt;cpu&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="s"&gt;500m"&lt;/span&gt;
            &lt;span class="na"&gt;memory&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="s"&gt;2Gi"&lt;/span&gt;
          &lt;span class="na"&gt;limits&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
            &lt;span class="na"&gt;memory&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="s"&gt;2Gi"&lt;/span&gt;
        &lt;span class="na"&gt;readinessProbe&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
          &lt;span class="na"&gt;httpGet&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
            &lt;span class="na"&gt;path&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;/healthz/ready&lt;/span&gt;
            &lt;span class="na"&gt;port&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;8080&lt;/span&gt;
          &lt;span class="na"&gt;periodSeconds&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;5&lt;/span&gt;
          &lt;span class="na"&gt;failureThreshold&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;2&lt;/span&gt;
        &lt;span class="na"&gt;lifecycle&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
          &lt;span class="na"&gt;preStop&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
            &lt;span class="na"&gt;exec&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
              &lt;span class="na"&gt;command&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="pi"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="s"&gt;/bin/sh"&lt;/span&gt;&lt;span class="pi"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="s"&gt;-c"&lt;/span&gt;&lt;span class="pi"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="s"&gt;sleep&lt;/span&gt;&lt;span class="nv"&gt; &lt;/span&gt;&lt;span class="s"&gt;10"&lt;/span&gt;&lt;span class="pi"&gt;]&lt;/span&gt;
&lt;span class="nn"&gt;---&lt;/span&gt;
&lt;span class="na"&gt;apiVersion&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;v1&lt;/span&gt;
&lt;span class="na"&gt;kind&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;Service&lt;/span&gt;
&lt;span class="na"&gt;metadata&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
  &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;gemini-session-svc&lt;/span&gt;
  &lt;span class="na"&gt;namespace&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;agent-runtime&lt;/span&gt;
&lt;span class="na"&gt;spec&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
  &lt;span class="na"&gt;selector&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
    &lt;span class="na"&gt;app&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;gemini-session-orchestrator&lt;/span&gt;
  &lt;span class="na"&gt;ports&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
  &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="na"&gt;port&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;80&lt;/span&gt;
    &lt;span class="na"&gt;targetPort&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;8080&lt;/span&gt;
&lt;span class="nn"&gt;---&lt;/span&gt;
&lt;span class="na"&gt;apiVersion&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;networking.gke.io/v1&lt;/span&gt;
&lt;span class="na"&gt;kind&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;GCPBackendPolicy&lt;/span&gt;
&lt;span class="na"&gt;metadata&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
  &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;gemini-streaming-backend-policy&lt;/span&gt;
  &lt;span class="na"&gt;namespace&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;agent-runtime&lt;/span&gt;
&lt;span class="na"&gt;spec&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
  &lt;span class="na"&gt;default&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
    &lt;span class="na"&gt;timeoutSec&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;600&lt;/span&gt;
    &lt;span class="na"&gt;connectionDraining&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
      &lt;span class="na"&gt;drainingTimeoutSec&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;300&lt;/span&gt;
    &lt;span class="na"&gt;sessionAffinity&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
      &lt;span class="na"&gt;type&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;GENERATED_COOKIE&lt;/span&gt;
      &lt;span class="na"&gt;cookieTtlSec&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;3600&lt;/span&gt;
  &lt;span class="na"&gt;targetRef&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
    &lt;span class="na"&gt;group&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="s"&gt;"&lt;/span&gt;
    &lt;span class="na"&gt;kind&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;Service&lt;/span&gt;
    &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;gemini-session-svc&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Notice three critical settings in this configuration:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;maxUnavailable: 0&lt;/code&gt;&lt;/strong&gt;: Ensures GKE brings new, startup-boosted Pods into a &lt;code&gt;Ready&lt;/code&gt; state before terminating any existing Pod holding active sessions.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;preStop&lt;/code&gt; sleep hook (&lt;code&gt;sleep 10&lt;/code&gt;)&lt;/strong&gt;: Because Kubernetes endpoint removal and Google Cloud load balancer programming happen asynchronously with &lt;code&gt;SIGTERM&lt;/code&gt;, sleeping for 10 seconds before the application stops accepting new connections prevents in-flight requests at the load balancer edge from hitting a closing socket.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;timeoutSec: 600&lt;/code&gt; and &lt;code&gt;drainingTimeoutSec: 300&lt;/code&gt;&lt;/strong&gt;: Allows streaming SSE or WebSocket responses from Gemini 4 Argon to run for up to 10 minutes and gives draining Pods 5 minutes to finish active turns.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Implementing per-turn state checkpointing and graceful drain
&lt;/h2&gt;

&lt;p&gt;Even with a 300-second drain window, an agent session might span multiple user interactions over an hour, or a node preemption on Spot capacity might give only 30 seconds of warning. By checkpointing the conversation history after every Gemini 4 Argon turn into Redis (such as &lt;a href="https://cloud.google.com/memorystore/docs/redis" rel="noopener noreferrer"&gt;Memorystore for Redis&lt;/a&gt;), any replica in the Deployment can transparently resume the session if the original Pod terminates.&lt;/p&gt;

&lt;p&gt;Here is the core session server (&lt;code&gt;session_server.py&lt;/code&gt;) showing graceful &lt;code&gt;SIGTERM&lt;/code&gt; tracking and per-turn Redis checkpointing with the &lt;code&gt;google-genai&lt;/code&gt; SDK:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;asyncio&lt;/span&gt;
&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;json&lt;/span&gt;
&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;
&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;signal&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;contextlib&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;asynccontextmanager&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;fastapi&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;FastAPI&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;HTTPException&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;fastapi.responses&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;StreamingResponse&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;pydantic&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;BaseModel&lt;/span&gt;
&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;redis.asyncio&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="n"&gt;redis&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;google&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;genai&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;google.genai&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;types&lt;/span&gt;

&lt;span class="n"&gt;draining&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="bp"&gt;False&lt;/span&gt;
&lt;span class="n"&gt;active_turns&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;


&lt;span class="k"&gt;class&lt;/span&gt; &lt;span class="nc"&gt;TurnRequest&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;BaseModel&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="n"&gt;user_message&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;


&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;handle_sigterm&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="n"&gt;_args&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="k"&gt;global&lt;/span&gt; &lt;span class="n"&gt;draining&lt;/span&gt;
    &lt;span class="n"&gt;draining&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="bp"&gt;True&lt;/span&gt;


&lt;span class="nd"&gt;@asynccontextmanager&lt;/span&gt;
&lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;lifespan&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;_app&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;FastAPI&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="n"&gt;signal&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;signal&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;signal&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;SIGTERM&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;handle_sigterm&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;yield&lt;/span&gt;
    &lt;span class="c1"&gt;# Wait for active Gemini 4 Argon turns to finish and checkpoint before exiting
&lt;/span&gt;    &lt;span class="k"&gt;while&lt;/span&gt; &lt;span class="n"&gt;active_turns&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="n"&gt;asyncio&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;sleep&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="n"&gt;redis_client&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;aclose&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;


&lt;span class="n"&gt;app&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;FastAPI&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;lifespan&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;lifespan&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;genai_client&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;genai&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;Client&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="n"&gt;vertexai&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="bp"&gt;True&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;project&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;environ&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;GOOGLE_CLOUD_PROJECT&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;
    &lt;span class="n"&gt;location&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;environ&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;VERTEX_LOCATION&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;us-central1&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;redis_client&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;redis&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;Redis&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;host&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;environ&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;REDIS_HOST&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt; &lt;span class="n"&gt;port&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;6379&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;decode_responses&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="bp"&gt;True&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;MODEL_ID&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;environ&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;GEMINI_MODEL&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;gemini-4-argon&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;


&lt;span class="nd"&gt;@app.get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;/healthz/ready&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;readiness_check&lt;/span&gt;&lt;span class="p"&gt;():&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;draining&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;raise&lt;/span&gt; &lt;span class="nc"&gt;HTTPException&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;status_code&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;503&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;detail&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Pod is draining active sessions&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;status&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;ready&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;


&lt;span class="nd"&gt;@app.post&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;/v1/sessions/{session_id}/stream&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;stream_agent_turn&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;session_id&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;request&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;TurnRequest&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="c1"&gt;# Load prior conversation history from Redis checkpoint
&lt;/span&gt;    &lt;span class="n"&gt;raw_history&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="n"&gt;redis_client&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;session:&lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;session_id&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;history&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;json&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;loads&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;raw_history&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;raw_history&lt;/span&gt; &lt;span class="k"&gt;else&lt;/span&gt; &lt;span class="p"&gt;[]&lt;/span&gt;
    &lt;span class="n"&gt;history&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;append&lt;/span&gt;&lt;span class="p"&gt;({&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;role&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;user&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;parts&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[{&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;text&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;request&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;user_message&lt;/span&gt;&lt;span class="p"&gt;}]})&lt;/span&gt;

    &lt;span class="n"&gt;contents&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;
        &lt;span class="n"&gt;types&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;Content&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;role&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;turn&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;role&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt; &lt;span class="n"&gt;parts&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;types&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Part&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;from_text&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;text&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;p&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;text&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;])&lt;/span&gt; &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;p&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;turn&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;parts&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;]])&lt;/span&gt;
        &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;turn&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;history&lt;/span&gt;
    &lt;span class="p"&gt;]&lt;/span&gt;

    &lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;event_generator&lt;/span&gt;&lt;span class="p"&gt;():&lt;/span&gt;
        &lt;span class="k"&gt;global&lt;/span&gt; &lt;span class="n"&gt;active_turns&lt;/span&gt;
        &lt;span class="n"&gt;active_turns&lt;/span&gt; &lt;span class="o"&gt;+=&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;
        &lt;span class="n"&gt;assembled_response&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[]&lt;/span&gt;
        &lt;span class="k"&gt;try&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
            &lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;chunk&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="n"&gt;genai_client&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;aio&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;models&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;generate_content_stream&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
                &lt;span class="n"&gt;model&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;MODEL_ID&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
                &lt;span class="n"&gt;contents&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;contents&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="p"&gt;):&lt;/span&gt;
                &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;chunk&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;text&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
                    &lt;span class="n"&gt;assembled_response&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;append&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;chunk&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;text&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
                    &lt;span class="k"&gt;yield&lt;/span&gt; &lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;data: &lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;json&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;dumps&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;text&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="si"&gt;:&lt;/span&gt; &lt;span class="n"&gt;chunk&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;text&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="se"&gt;\n\n&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;

            &lt;span class="c1"&gt;# Checkpoint completed assistant turn back to Redis with a 24-hour TTL
&lt;/span&gt;            &lt;span class="n"&gt;history&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;append&lt;/span&gt;&lt;span class="p"&gt;({&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;role&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;model&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;parts&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[{&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;text&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;""&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;join&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;assembled_response&lt;/span&gt;&lt;span class="p"&gt;)}]})&lt;/span&gt;
            &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="n"&gt;redis_client&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;set&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;session:&lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;session_id&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;json&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;dumps&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;history&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt; &lt;span class="n"&gt;ex&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;86400&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="k"&gt;finally&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
            &lt;span class="n"&gt;active_turns&lt;/span&gt; &lt;span class="o"&gt;-=&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;

    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nc"&gt;StreamingResponse&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nf"&gt;event_generator&lt;/span&gt;&lt;span class="p"&gt;(),&lt;/span&gt; &lt;span class="n"&gt;media_type&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;text/event-stream&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  Verifying startup boost and connection draining
&lt;/h2&gt;

&lt;p&gt;Once deployed, confirm that VPA CPU startup boost is accelerating Pod initialization and resizing CPU in place without restarting the container:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="c"&gt;# Inspect the startup boost annotation injected on the Pod&lt;/span&gt;
kubectl get pods &lt;span class="nt"&gt;-n&lt;/span&gt; agent-runtime &lt;span class="se"&gt;\&lt;/span&gt;
    &lt;span class="nt"&gt;-l&lt;/span&gt; &lt;span class="nv"&gt;app&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;gemini-session-orchestrator &lt;span class="se"&gt;\&lt;/span&gt;
    &lt;span class="nt"&gt;-o&lt;/span&gt; custom-columns&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"NAME:.metadata.name,CPU_REQ:.spec.containers[0].resources.requests.cpu,BOOST_ANNOTATION:.metadata.annotations.vpaCpuStartupBoost&lt;/span&gt;&lt;span class="se"&gt;\/&lt;/span&gt;&lt;span class="s2"&gt;agent-server"&lt;/span&gt;

&lt;span class="c"&gt;# Watch for the InPlaceResizedByVPA event once the Pod passes readinessProbe&lt;/span&gt;
kubectl get events &lt;span class="nt"&gt;-n&lt;/span&gt; agent-runtime &lt;span class="nt"&gt;--field-selector&lt;/span&gt; &lt;span class="nv"&gt;reason&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;InPlaceResizedByVPA
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Expected event output:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;LAST SEEN   TYPE     REASON                 OBJECT                                             MESSAGE
14s         Normal   InPlaceResizedByVPA    pod/gemini-session-orchestrator-6b8f9c7d4-k29lm    Pod was in-place resized by VPA Updater (CPU: 1500m -&amp;gt; 500m)
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;To verify that rolling updates drain cleanly without dropping streams, trigger a rollout while streaming a multi-step Gemini 4 Argon response and monitor active backend connections in &lt;a href="https://cloud.google.com/monitoring/docs" rel="noopener noreferrer"&gt;Cloud Monitoring&lt;/a&gt; using the metric &lt;code&gt;loadbalancing.googleapis.com/https/backend_request_count&lt;/code&gt; alongside &lt;code&gt;kubernetes.io/container/cpu/core_usage_time&lt;/code&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  Operational trade-offs to watch
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Memory growth on long context windows&lt;/strong&gt;: While CPU usage drops to near zero while waiting on Gemini platform network streams, assembling 500K-token contexts in Python memory can spike heap allocations. VPA CPU Startup Boost only boosts CPU; size your container &lt;code&gt;memory&lt;/code&gt; requests and limits to accommodate the largest concurrent payload your agent processes.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Avoid Spot VMs for long-running streaming gateways&lt;/strong&gt;: Spot VMs (or &lt;code&gt;spot: true&lt;/code&gt; in a GKE &lt;code&gt;ComputeClass&lt;/code&gt;) only provide a 30-second preemption notice, which is too short for a 3-minute agent reasoning turn to drain gracefully. Run the lightweight session orchestrator Pods on standard on-demand capacity.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;HPA scaling metric selection&lt;/strong&gt;: Because steady-state agent orchestrators are I/O-bound rather than CPU-bound, scaling HPA solely on CPU utilization will under-scale during concurrent streaming bursts. Pair HPA with custom load balancer concurrency metrics or active session gauges exported to Cloud Monitoring.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Next steps
&lt;/h2&gt;

&lt;p&gt;Running Gemini 4 Argon agents reliably in production requires adapting Kubernetes from short-lived request/response defaults to stateful, long-horizon execution. By combining VPA CPU Startup Boost for fast initialization, &lt;code&gt;GCPBackendPolicy&lt;/code&gt; for long streaming timeouts, and per-turn state checkpointing, your agent deployments can scale and roll out new versions without interrupting active users.&lt;/p&gt;

&lt;p&gt;To dive deeper into these GKE capabilities, read the &lt;a href="https://cloud.google.com/kubernetes-engine/docs/how-to/boost-application-startup" rel="noopener noreferrer"&gt;VPA CPU Startup Boost guide&lt;/a&gt;, review the &lt;a href="https://cloud.google.com/kubernetes-engine/docs/how-to/configure-gateway-resources#configure_backend_service" rel="noopener noreferrer"&gt;GKE Gateway &lt;code&gt;GCPBackendPolicy&lt;/code&gt; reference&lt;/a&gt;, and check the &lt;a href="https://cloud.google.com/vertex-ai/generative-ai/docs" rel="noopener noreferrer"&gt;Gemini API documentation on Google Cloud&lt;/a&gt;.&lt;/p&gt;

</description>
      <category>kubernetes</category>
      <category>ai</category>
      <category>gke</category>
      <category>googlecloud</category>
    </item>
    <item>
      <title>Multi-tier model routing on GKE: balancing Gemini 4 Argon reasoning depth with latency and cost</title>
      <dc:creator>Olivier Bourgeois</dc:creator>
      <pubDate>Wed, 21 Oct 2026 18:56:20 +0000</pubDate>
      <link>https://dev.to/googlecloud/multi-tier-model-routing-on-gke-balancing-gemini-4-argon-reasoning-depth-with-latency-and-cost-21fo</link>
      <guid>https://dev.to/googlecloud/multi-tier-model-routing-on-gke-balancing-gemini-4-argon-reasoning-depth-with-latency-and-cost-21fo</guid>
      <description>&lt;p&gt;Routing every incoming user prompt to a frontier reasoning model is the fastest way to blow through your token budget and degrade p99 tail latency. With the release of Gemini 4 Argon on the Gemini platform, engineering teams have access to deep multi-step reasoning in &lt;code&gt;gemini-4-argon&lt;/code&gt; alongside high-throughput, low-latency execution in &lt;code&gt;gemini-3.8-flash&lt;/code&gt; and open-weights models running directly on cluster accelerators.&lt;/p&gt;

&lt;p&gt;In most production applications, workload complexity follows a steep power law. Roughly 70% of requests involve classification, summarization, schema extraction, or straightforward retrieval that a fast tier handles in a few hundred milliseconds. The remaining 30% require multi-hop synthesis, complex code generation, or ambiguous constraint solving where &lt;code&gt;gemini-4-argon&lt;/code&gt; shines. Hardcoding a single model identifier in your application containers forces you to either overpay on simple queries or under-deliver on complex ones.&lt;/p&gt;

&lt;p&gt;By building a centralized AI routing layer on &lt;a href="https://cloud.google.com/kubernetes-engine" rel="noopener noreferrer"&gt;Google Kubernetes Engine (GKE)&lt;/a&gt; using the &lt;a href="https://cloud.google.com/kubernetes-engine/docs/concepts/gateway-api" rel="noopener noreferrer"&gt;GKE Gateway API&lt;/a&gt;, platform teams can classify prompt complexity at the ingress edge, route requests across Gemini tiers (&lt;code&gt;gemini-4-argon&lt;/code&gt; and &lt;code&gt;gemini-3.8-flash&lt;/code&gt;) and self-hosted &lt;a href="https://cloud.google.com/kubernetes-engine/docs/concepts/about-gke-inference-gateway" rel="noopener noreferrer"&gt;GKE Inference Gateway&lt;/a&gt; pools, and enforce automatic fallback when rate limits occur.&lt;/p&gt;

&lt;p&gt;In this article, I will break down the architecture of a multi-tier Gemini 4 Argon router on GKE, configure GKE Standard and Autopilot infrastructure with &lt;a href="https://cloud.google.com/kubernetes-engine/docs/how-to/workload-identity" rel="noopener noreferrer"&gt;Workload Identity Federation for GKE&lt;/a&gt;, and walk through the routing, health-check, and observability configurations.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why static model bindings fail in production
&lt;/h2&gt;

&lt;p&gt;When application teams integrate directly with LLM endpoints inside dozens of individual microservices, three operational problems surface quickly:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Unbounded token spend and latency inflation&lt;/strong&gt;: A microservice calling &lt;code&gt;gemini-4-argon&lt;/code&gt; for routine JSON formatting pays frontier token pricing and waits for deep reasoning chains when &lt;code&gt;gemini-3.8-flash&lt;/code&gt; would produce an identical schema in a fraction of the time.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Scattered credential and quota management&lt;/strong&gt;: Managing Gemini platform API quotas, retry backoffs, and regional failover across 30 separate application repositories leads to inconsistent behavior during traffic spikes.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;No path to hybrid in-cluster offload&lt;/strong&gt;: When high-volume, low-complexity tasks could be served at fixed cost by an open-weights model running on GKE GPU or TPU node pools, application code tightly coupled to a single model string cannot shift traffic without code changes and redeployments.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Centralizing model routing behind a GKE Gateway and an intelligent router service decouples client microservices from the underlying model tier.&lt;/p&gt;

&lt;h2&gt;
  
  
  Multi-tier routing architecture on GKE
&lt;/h2&gt;

&lt;p&gt;In this architecture, internal microservices send standard requests to a regional internal GKE Gateway (&lt;code&gt;gke-l7-rilb&lt;/code&gt;). The Gateway forwards traffic to an &lt;code&gt;ai-model-router&lt;/code&gt; service that evaluates request metadata or runs a fast complexity check before dispatching the request to one of three model tiers:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Tier 1 (Frontier reasoning)&lt;/strong&gt;: High-complexity tasks, code synthesis, and multi-step planning route to &lt;code&gt;gemini-4-argon&lt;/code&gt; on the Gemini platform.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Tier 2 (Low-latency interactive)&lt;/strong&gt;: Routine summarization, classification, and structured extraction route to &lt;code&gt;gemini-3.8-flash&lt;/code&gt; on the Gemini platform.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Tier 3 (High-throughput local)&lt;/strong&gt;: High-volume batch transformations and strict data locality workloads route to an in-cluster vLLM pool running Gemma 4 via GKE Inference Gateway.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Here is how the three tiers compare across production operating dimensions:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Routing tier&lt;/th&gt;
&lt;th&gt;Target model&lt;/th&gt;
&lt;th&gt;Primary workload profile&lt;/th&gt;
&lt;th&gt;Fallback path on &lt;code&gt;429&lt;/code&gt; / timeout&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Tier 1: Frontier reasoning&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;gemini-4-argon&lt;/code&gt; (Gemini platform)&lt;/td&gt;
&lt;td&gt;Multi-step planning, complex code synthesis, ambiguous root-cause analysis&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;us&lt;/code&gt; multi-region endpoint, then graceful downgrade to &lt;code&gt;gemini-3.8-flash&lt;/code&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Tier 2: Low-latency interactive&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;gemini-3.8-flash&lt;/code&gt; (Gemini platform)&lt;/td&gt;
&lt;td&gt;Real-time chat, structured extraction, classification, single-turn tool calls&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;us&lt;/code&gt; multi-region endpoint, then in-cluster open-weights pool&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Tier 3: High-throughput local&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Open-weights (Gemma 4 via vLLM on GKE)&lt;/td&gt;
&lt;td&gt;High-volume entity tagging, PII scrubbing, fixed-cost background jobs&lt;/td&gt;
&lt;td&gt;Burst overflow to &lt;code&gt;gemini-3.8-flash&lt;/code&gt; on the Gemini platform&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The sample router later in this article implements the Tier 1 and Tier 2 location failover and the Tier 1 downgrade. The paths into and out of the in-cluster Tier 3 pool depend on your Inference Gateway setup and are left to your own dispatch logic.&lt;/p&gt;

&lt;h2&gt;
  
  
  Configuring Workload Identity and GKE compute
&lt;/h2&gt;

&lt;p&gt;To call the Gemini platform without distributing static service account keys, bind the &lt;code&gt;ai-model-router&lt;/code&gt; Kubernetes ServiceAccount directly to an IAM role using Workload Identity Federation for GKE. This works identically across GKE Standard and GKE Autopilot clusters running GKE &lt;code&gt;1.32&lt;/code&gt; or later with the GKE Gateway controller enabled (&lt;code&gt;--gateway-api=standard&lt;/code&gt;) and an active regional proxy-only subnet (&lt;code&gt;--purpose=REGIONAL_MANAGED_PROXY&lt;/code&gt;) in your VPC.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;Note on model identifiers and locations&lt;/strong&gt;: The examples below use &lt;code&gt;gemini-4-argon&lt;/code&gt; (Gemini 4 Argon) and &lt;code&gt;gemini-3.8-flash&lt;/code&gt; (Gemini 3.8 Flash) configured via environment variables (&lt;code&gt;PRO_MODEL&lt;/code&gt; and &lt;code&gt;FLASH_MODEL&lt;/code&gt;). Both models are served from the &lt;code&gt;global&lt;/code&gt; endpoint and the &lt;code&gt;us&lt;/code&gt; and &lt;code&gt;eu&lt;/code&gt; multi-region endpoints, not from individual regions such as &lt;code&gt;us-central1&lt;/code&gt;, so the router fails over from &lt;code&gt;global&lt;/code&gt; to &lt;code&gt;us&lt;/code&gt;.&lt;br&gt;
&lt;/p&gt;
&lt;/blockquote&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="nb"&gt;export &lt;/span&gt;&lt;span class="nv"&gt;PROJECT_ID&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"production-ai-platform"&lt;/span&gt;
&lt;span class="nb"&gt;export &lt;/span&gt;&lt;span class="nv"&gt;PROJECT_NUMBER&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="si"&gt;$(&lt;/span&gt;gcloud projects describe &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="k"&gt;${&lt;/span&gt;&lt;span class="nv"&gt;PROJECT_ID&lt;/span&gt;&lt;span class="k"&gt;}&lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt; &lt;span class="nt"&gt;--format&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"value(projectNumber)"&lt;/span&gt;&lt;span class="si"&gt;)&lt;/span&gt;
&lt;span class="nb"&gt;export &lt;/span&gt;&lt;span class="nv"&gt;LOCATION&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"us-central1"&lt;/span&gt;
&lt;span class="nb"&gt;export &lt;/span&gt;&lt;span class="nv"&gt;CLUSTER_NAME&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"ai-gateway-cluster"&lt;/span&gt;

&lt;span class="c"&gt;# Point kubectl at the cluster&lt;/span&gt;
gcloud container clusters get-credentials &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="k"&gt;${&lt;/span&gt;&lt;span class="nv"&gt;CLUSTER_NAME&lt;/span&gt;&lt;span class="k"&gt;}&lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt; &lt;span class="nt"&gt;--location&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="k"&gt;${&lt;/span&gt;&lt;span class="nv"&gt;LOCATION&lt;/span&gt;&lt;span class="k"&gt;}&lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt; &lt;span class="nt"&gt;--project&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="k"&gt;${&lt;/span&gt;&lt;span class="nv"&gt;PROJECT_ID&lt;/span&gt;&lt;span class="k"&gt;}&lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt;

&lt;span class="c"&gt;# Create the namespace and Kubernetes ServiceAccount&lt;/span&gt;
kubectl create namespace ai-routing
kubectl create serviceaccount gemini-router-sa &lt;span class="nt"&gt;--namespace&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;ai-routing

&lt;span class="c"&gt;# Grant the Kubernetes ServiceAccount permission to invoke Gemini platform models&lt;/span&gt;
&lt;span class="c"&gt;# (the underlying IAM role string is still roles/aiplatform.user)&lt;/span&gt;
gcloud projects add-iam-policy-binding &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="k"&gt;${&lt;/span&gt;&lt;span class="nv"&gt;PROJECT_ID&lt;/span&gt;&lt;span class="k"&gt;}&lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt; &lt;span class="se"&gt;\&lt;/span&gt;
    &lt;span class="nt"&gt;--member&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"principal://iam.googleapis.com/projects/&lt;/span&gt;&lt;span class="k"&gt;${&lt;/span&gt;&lt;span class="nv"&gt;PROJECT_NUMBER&lt;/span&gt;&lt;span class="k"&gt;}&lt;/span&gt;&lt;span class="s2"&gt;/locations/global/workloadIdentityPools/&lt;/span&gt;&lt;span class="k"&gt;${&lt;/span&gt;&lt;span class="nv"&gt;PROJECT_ID&lt;/span&gt;&lt;span class="k"&gt;}&lt;/span&gt;&lt;span class="s2"&gt;.svc.id.goog/subject/ns/ai-routing/sa/gemini-router-sa"&lt;/span&gt; &lt;span class="se"&gt;\&lt;/span&gt;
    &lt;span class="nt"&gt;--role&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"roles/aiplatform.user"&lt;/span&gt; &lt;span class="se"&gt;\&lt;/span&gt;
    &lt;span class="nt"&gt;--condition&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;None
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  Provisioning capacity on GKE Autopilot vs. Standard
&lt;/h3&gt;

&lt;p&gt;For the router Deployment itself, standard CPU compute is sufficient. If you also operate Tier 3 (in-cluster open-weights inference via GKE Inference Gateway for high-volume offload), configure your accelerator capacity according to your cluster mode:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;GKE Autopilot with &lt;code&gt;ComputeClass&lt;/code&gt;&lt;/strong&gt;: Define a declarative &lt;code&gt;ComputeClass&lt;/code&gt; (&lt;code&gt;cloud.google.com/v1&lt;/code&gt;) that requests NVIDIA L4 GPUs (&lt;code&gt;gpu.type: nvidia-l4&lt;/code&gt;, &lt;code&gt;gpu.count: 2&lt;/code&gt;) with prioritized fallback from on-demand capacity to Spot VMs during regional stockouts.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;GKE Standard node pools&lt;/strong&gt;: Create a dedicated GPU node pool (&lt;code&gt;gcloud container node-pools create vllm-l4-pool --cluster="${CLUSTER_NAME}" --location="${LOCATION}" --machine-type=g2-standard-24 --accelerator=type=nvidia-l4,count=2,gpu-driver-version=latest --enable-autoscaling --min-nodes=1 --max-nodes=6&lt;/code&gt;).&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Implementing the Gemini 4 tier router
&lt;/h2&gt;

&lt;p&gt;The &lt;code&gt;ai-model-router&lt;/code&gt; service inspects incoming requests, estimates task complexity (using explicit client headers, token count thresholds, or keyword triggers), and dispatches the call through the &lt;code&gt;google-genai&lt;/code&gt; SDK with cross-location failover and cross-tier fallback. It also exposes a &lt;code&gt;/healthz&lt;/code&gt; endpoint that both the Pod &lt;code&gt;readinessProbe&lt;/code&gt; and the GKE Gateway load balancer health check use to verify backend readiness.&lt;/p&gt;

&lt;p&gt;Save the following implementation as &lt;code&gt;router_app.py&lt;/code&gt;:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;
&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;time&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;fastapi&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;FastAPI&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;Header&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;HTTPException&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;pydantic&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;BaseModel&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;google&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;genai&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;google.genai&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;types&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;google.genai.errors&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;APIError&lt;/span&gt;

&lt;span class="n"&gt;app&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;FastAPI&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;title&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;GKE Gemini 4 Argon Multi-Tier Router&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="n"&gt;PROJECT_ID&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;environ&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;GOOGLE_CLOUD_PROJECT&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
&lt;span class="n"&gt;PRIMARY_LOCATION&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;environ&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;PRIMARY_LOCATION&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;global&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;SECONDARY_LOCATION&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;environ&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;SECONDARY_LOCATION&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;us&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;PRO_MODEL&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;environ&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;PRO_MODEL&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;gemini-4-argon&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;FLASH_MODEL&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;environ&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;FLASH_MODEL&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;gemini-3.8-flash&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="c1"&gt;# Project-scoped Gemini platform clients authenticated through Workload Identity Federation for GKE.
# The vertexai=True flag is the SDK switch for project-scoped IAM auth instead of an API key.
&lt;/span&gt;&lt;span class="n"&gt;primary_client&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;genai&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;Client&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;vertexai&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="bp"&gt;True&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;project&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;PROJECT_ID&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;location&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;PRIMARY_LOCATION&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;secondary_client&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;genai&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;Client&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;vertexai&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="bp"&gt;True&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;project&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;PROJECT_ID&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;location&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;SECONDARY_LOCATION&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="n"&gt;REASONING_KEYWORDS&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;architect&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;debug&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;prove&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;multi-step&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;optimize&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;trade-off&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;root cause&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;


&lt;span class="k"&gt;class&lt;/span&gt; &lt;span class="nc"&gt;PromptRequest&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;BaseModel&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="n"&gt;prompt&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;
    &lt;span class="n"&gt;system_instruction&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt; &lt;span class="o"&gt;|&lt;/span&gt; &lt;span class="bp"&gt;None&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="bp"&gt;None&lt;/span&gt;
    &lt;span class="n"&gt;require_deep_reasoning&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;bool&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="bp"&gt;False&lt;/span&gt;


&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;select_model_tier&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;req&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;PromptRequest&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;routing_hint&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt; &lt;span class="o"&gt;|&lt;/span&gt; &lt;span class="bp"&gt;None&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="sh"&gt;"""&lt;/span&gt;&lt;span class="s"&gt;Selects the Gemini 4 Argon (frontier) or Gemini 3.8 Flash tier based on client hints and prompt characteristics.&lt;/span&gt;&lt;span class="sh"&gt;"""&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;routing_hint&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;pro&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt; &lt;span class="ow"&gt;or&lt;/span&gt; &lt;span class="n"&gt;req&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;require_deep_reasoning&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;PRO_MODEL&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;routing_hint&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;flash&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;FLASH_MODEL&lt;/span&gt;

    &lt;span class="n"&gt;prompt_lower&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;req&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;prompt&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;lower&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
    &lt;span class="n"&gt;word_count&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;len&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;req&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;prompt&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;split&lt;/span&gt;&lt;span class="p"&gt;())&lt;/span&gt;
    &lt;span class="n"&gt;has_reasoning_trigger&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;any&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;kw&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;prompt_lower&lt;/span&gt; &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;kw&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;REASONING_KEYWORDS&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;word_count&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;&lt;/span&gt; &lt;span class="mi"&gt;600&lt;/span&gt; &lt;span class="ow"&gt;or&lt;/span&gt; &lt;span class="n"&gt;has_reasoning_trigger&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;PRO_MODEL&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;FLASH_MODEL&lt;/span&gt;


&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;build_config&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;model&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;system_instruction&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt; &lt;span class="o"&gt;|&lt;/span&gt; &lt;span class="bp"&gt;None&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="n"&gt;types&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;GenerateContentConfig&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="sh"&gt;"""&lt;/span&gt;&lt;span class="s"&gt;Gemini 4 Argon ignores sampling parameters and defaults to the MEDIUM thinking level,
    so temperature is only set on the Flash tier.&lt;/span&gt;&lt;span class="sh"&gt;"""&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;model&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="n"&gt;PRO_MODEL&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;types&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;GenerateContentConfig&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;system_instruction&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;system_instruction&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;types&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;GenerateContentConfig&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;system_instruction&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;system_instruction&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;temperature&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mf"&gt;0.2&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;


&lt;span class="nd"&gt;@app.get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;/healthz&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;health_check&lt;/span&gt;&lt;span class="p"&gt;():&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;status&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;ok&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;


&lt;span class="nd"&gt;@app.post&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;/v1/generate&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;generate_response&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="n"&gt;req&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;PromptRequest&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;x_model_tier&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt; &lt;span class="o"&gt;|&lt;/span&gt; &lt;span class="bp"&gt;None&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;Header&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;default&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="bp"&gt;None&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt;
&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="n"&gt;target_model&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;select_model_tier&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;req&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;x_model_tier&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;start_time&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;time&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;monotonic&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;

    &lt;span class="c1"&gt;# Attempt the global endpoint, then the us multi-region endpoint, then downgrade to Flash
&lt;/span&gt;    &lt;span class="c1"&gt;# unless the caller explicitly required deep reasoning
&lt;/span&gt;    &lt;span class="n"&gt;attempt_chain&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;
        &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;primary_client&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;PRIMARY_LOCATION&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;target_model&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt;
        &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;secondary_client&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;SECONDARY_LOCATION&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;target_model&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt;
    &lt;span class="p"&gt;]&lt;/span&gt;
    &lt;span class="n"&gt;explicit_pro&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;x_model_tier&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;pro&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt; &lt;span class="ow"&gt;or&lt;/span&gt; &lt;span class="n"&gt;req&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;require_deep_reasoning&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;target_model&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="n"&gt;PRO_MODEL&lt;/span&gt; &lt;span class="ow"&gt;and&lt;/span&gt; &lt;span class="ow"&gt;not&lt;/span&gt; &lt;span class="n"&gt;explicit_pro&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="n"&gt;attempt_chain&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;append&lt;/span&gt;&lt;span class="p"&gt;((&lt;/span&gt;&lt;span class="n"&gt;primary_client&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;PRIMARY_LOCATION&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;FLASH_MODEL&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;

    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;attempt_client&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;location&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;candidate_model&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;attempt_chain&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;try&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
            &lt;span class="n"&gt;response&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;attempt_client&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;models&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;generate_content&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
                &lt;span class="n"&gt;model&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;candidate_model&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
                &lt;span class="n"&gt;contents&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;req&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;prompt&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
                &lt;span class="n"&gt;config&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="nf"&gt;build_config&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;candidate_model&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;req&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;system_instruction&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt;
            &lt;span class="p"&gt;)&lt;/span&gt;
            &lt;span class="n"&gt;latency_ms&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;round&lt;/span&gt;&lt;span class="p"&gt;((&lt;/span&gt;&lt;span class="n"&gt;time&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;monotonic&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt; &lt;span class="n"&gt;start_time&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="mi"&gt;1000&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
            &lt;span class="n"&gt;usage&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;response&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;usage_metadata&lt;/span&gt;
            &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
                &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;model_used&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;candidate_model&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
                &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;location&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;location&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
                &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;fallback_triggered&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;location&lt;/span&gt; &lt;span class="o"&gt;!=&lt;/span&gt; &lt;span class="n"&gt;PRIMARY_LOCATION&lt;/span&gt; &lt;span class="ow"&gt;or&lt;/span&gt; &lt;span class="n"&gt;candidate_model&lt;/span&gt; &lt;span class="o"&gt;!=&lt;/span&gt; &lt;span class="n"&gt;target_model&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt;
                &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;latency_ms&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;latency_ms&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
                &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;usage&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
                    &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;prompt_tokens&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;usage&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;prompt_token_count&lt;/span&gt; &lt;span class="ow"&gt;or&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;usage&lt;/span&gt; &lt;span class="k"&gt;else&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
                    &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;output_tokens&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;usage&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;candidates_token_count&lt;/span&gt; &lt;span class="ow"&gt;or&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;usage&lt;/span&gt; &lt;span class="k"&gt;else&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
                    &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;thoughts_tokens&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;usage&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;thoughts_token_count&lt;/span&gt; &lt;span class="ow"&gt;or&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;usage&lt;/span&gt; &lt;span class="k"&gt;else&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
                &lt;span class="p"&gt;},&lt;/span&gt;
                &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;output&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;response&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;text&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="p"&gt;}&lt;/span&gt;
        &lt;span class="k"&gt;except&lt;/span&gt; &lt;span class="n"&gt;APIError&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="n"&gt;exc&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
            &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;exc&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;code&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;429&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;500&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;503&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;504&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
                &lt;span class="k"&gt;continue&lt;/span&gt;
            &lt;span class="k"&gt;raise&lt;/span&gt; &lt;span class="nc"&gt;HTTPException&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;status_code&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;exc&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;code&lt;/span&gt; &lt;span class="ow"&gt;or&lt;/span&gt; &lt;span class="mi"&gt;500&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;detail&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="nf"&gt;str&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;exc&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;

    &lt;span class="k"&gt;raise&lt;/span&gt; &lt;span class="nc"&gt;HTTPException&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;status_code&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;503&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;detail&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;All Gemini endpoints exhausted&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  Exposing the router via GKE Gateway API and GCPBackendPolicy
&lt;/h2&gt;

&lt;p&gt;Because &lt;code&gt;gemini-4-argon&lt;/code&gt; reasoning chains on large contexts can take 30 to 90 seconds to complete, standard 30-second HTTP load balancer timeouts will prematurely drop connections and cause clients to retry expensive requests.&lt;/p&gt;

&lt;p&gt;Configure a GKE &lt;code&gt;Gateway&lt;/code&gt;, &lt;code&gt;HTTPRoute&lt;/code&gt;, and &lt;code&gt;GCPBackendPolicy&lt;/code&gt; (&lt;code&gt;networking.gke.io/v1&lt;/code&gt;) to extend the backend timeout to 180 seconds and enable connection draining during rolling updates. Unlike GKE Ingress, GKE Gateway does not infer load balancer health checks from the Pod &lt;code&gt;readinessProbe&lt;/code&gt;; it probes &lt;code&gt;/&lt;/code&gt; by default, which the router answers with a &lt;code&gt;404&lt;/code&gt;. Add a &lt;code&gt;HealthCheckPolicy&lt;/code&gt; (&lt;code&gt;networking.gke.io/v1&lt;/code&gt;) that points the load balancer at &lt;code&gt;/healthz&lt;/code&gt;:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="na"&gt;apiVersion&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;apps/v1&lt;/span&gt;
&lt;span class="na"&gt;kind&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;Deployment&lt;/span&gt;
&lt;span class="na"&gt;metadata&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
  &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;ai-model-router&lt;/span&gt;
  &lt;span class="na"&gt;namespace&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;ai-routing&lt;/span&gt;
&lt;span class="na"&gt;spec&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
  &lt;span class="na"&gt;replicas&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;3&lt;/span&gt;
  &lt;span class="na"&gt;selector&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
    &lt;span class="na"&gt;matchLabels&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
      &lt;span class="na"&gt;app&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;ai-model-router&lt;/span&gt;
  &lt;span class="na"&gt;template&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
    &lt;span class="na"&gt;metadata&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
      &lt;span class="na"&gt;labels&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
        &lt;span class="na"&gt;app&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;ai-model-router&lt;/span&gt;
    &lt;span class="na"&gt;spec&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
      &lt;span class="na"&gt;serviceAccountName&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;gemini-router-sa&lt;/span&gt;
      &lt;span class="na"&gt;containers&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
      &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;router&lt;/span&gt;
        &lt;span class="na"&gt;image&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;us-central1-docker.pkg.dev/production-ai-platform/ai-routing/gemini-4-router:v1.0.0&lt;/span&gt;
        &lt;span class="na"&gt;ports&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
        &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="na"&gt;containerPort&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;8080&lt;/span&gt;
        &lt;span class="na"&gt;env&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
        &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;GOOGLE_CLOUD_PROJECT&lt;/span&gt;
          &lt;span class="na"&gt;value&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="s"&gt;production-ai-platform"&lt;/span&gt;
        &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;PRIMARY_LOCATION&lt;/span&gt;
          &lt;span class="na"&gt;value&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="s"&gt;global"&lt;/span&gt;
        &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;SECONDARY_LOCATION&lt;/span&gt;
          &lt;span class="na"&gt;value&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="s"&gt;us"&lt;/span&gt;
        &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;PRO_MODEL&lt;/span&gt;
          &lt;span class="na"&gt;value&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="s"&gt;gemini-4-argon"&lt;/span&gt;
        &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;FLASH_MODEL&lt;/span&gt;
          &lt;span class="na"&gt;value&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="s"&gt;gemini-3.8-flash"&lt;/span&gt;
        &lt;span class="na"&gt;readinessProbe&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
          &lt;span class="na"&gt;httpGet&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
            &lt;span class="na"&gt;path&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;/healthz&lt;/span&gt;
            &lt;span class="na"&gt;port&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;8080&lt;/span&gt;
          &lt;span class="na"&gt;initialDelaySeconds&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;5&lt;/span&gt;
          &lt;span class="na"&gt;periodSeconds&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;10&lt;/span&gt;
        &lt;span class="na"&gt;resources&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
          &lt;span class="na"&gt;requests&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
            &lt;span class="na"&gt;cpu&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="s"&gt;500m"&lt;/span&gt;
            &lt;span class="na"&gt;memory&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="s"&gt;512Mi"&lt;/span&gt;
&lt;span class="nn"&gt;---&lt;/span&gt;
&lt;span class="na"&gt;apiVersion&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;v1&lt;/span&gt;
&lt;span class="na"&gt;kind&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;Service&lt;/span&gt;
&lt;span class="na"&gt;metadata&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
  &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;ai-model-router-svc&lt;/span&gt;
  &lt;span class="na"&gt;namespace&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;ai-routing&lt;/span&gt;
&lt;span class="na"&gt;spec&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
  &lt;span class="na"&gt;selector&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
    &lt;span class="na"&gt;app&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;ai-model-router&lt;/span&gt;
  &lt;span class="na"&gt;ports&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
  &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="na"&gt;port&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;80&lt;/span&gt;
    &lt;span class="na"&gt;targetPort&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;8080&lt;/span&gt;
&lt;span class="nn"&gt;---&lt;/span&gt;
&lt;span class="na"&gt;apiVersion&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;networking.gke.io/v1&lt;/span&gt;
&lt;span class="na"&gt;kind&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;GCPBackendPolicy&lt;/span&gt;
&lt;span class="na"&gt;metadata&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
  &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;gemini-router-backend-policy&lt;/span&gt;
  &lt;span class="na"&gt;namespace&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;ai-routing&lt;/span&gt;
&lt;span class="na"&gt;spec&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
  &lt;span class="na"&gt;default&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
    &lt;span class="na"&gt;timeoutSec&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;180&lt;/span&gt;
    &lt;span class="na"&gt;connectionDraining&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
      &lt;span class="na"&gt;drainingTimeoutSec&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;60&lt;/span&gt;
  &lt;span class="na"&gt;targetRef&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
    &lt;span class="na"&gt;group&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="s"&gt;"&lt;/span&gt;
    &lt;span class="na"&gt;kind&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;Service&lt;/span&gt;
    &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;ai-model-router-svc&lt;/span&gt;
&lt;span class="nn"&gt;---&lt;/span&gt;
&lt;span class="na"&gt;apiVersion&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;networking.gke.io/v1&lt;/span&gt;
&lt;span class="na"&gt;kind&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;HealthCheckPolicy&lt;/span&gt;
&lt;span class="na"&gt;metadata&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
  &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;gemini-router-healthcheck&lt;/span&gt;
  &lt;span class="na"&gt;namespace&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;ai-routing&lt;/span&gt;
&lt;span class="na"&gt;spec&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
  &lt;span class="na"&gt;default&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
    &lt;span class="na"&gt;checkIntervalSec&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;10&lt;/span&gt;
    &lt;span class="na"&gt;timeoutSec&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;5&lt;/span&gt;
    &lt;span class="na"&gt;healthyThreshold&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;1&lt;/span&gt;
    &lt;span class="na"&gt;unhealthyThreshold&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;3&lt;/span&gt;
    &lt;span class="na"&gt;config&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
      &lt;span class="na"&gt;type&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;HTTP&lt;/span&gt;
      &lt;span class="na"&gt;httpHealthCheck&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
        &lt;span class="na"&gt;port&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;8080&lt;/span&gt;
        &lt;span class="na"&gt;requestPath&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;/healthz&lt;/span&gt;
  &lt;span class="na"&gt;targetRef&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
    &lt;span class="na"&gt;group&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="s"&gt;"&lt;/span&gt;
    &lt;span class="na"&gt;kind&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;Service&lt;/span&gt;
    &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;ai-model-router-svc&lt;/span&gt;
&lt;span class="nn"&gt;---&lt;/span&gt;
&lt;span class="na"&gt;apiVersion&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;gateway.networking.k8s.io/v1&lt;/span&gt;
&lt;span class="na"&gt;kind&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;Gateway&lt;/span&gt;
&lt;span class="na"&gt;metadata&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
  &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;internal-ai-gateway&lt;/span&gt;
  &lt;span class="na"&gt;namespace&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;ai-routing&lt;/span&gt;
&lt;span class="na"&gt;spec&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
  &lt;span class="na"&gt;gatewayClassName&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;gke-l7-rilb&lt;/span&gt;
  &lt;span class="na"&gt;listeners&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
  &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;http&lt;/span&gt;
    &lt;span class="na"&gt;protocol&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;HTTP&lt;/span&gt;
    &lt;span class="na"&gt;port&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;80&lt;/span&gt;
&lt;span class="nn"&gt;---&lt;/span&gt;
&lt;span class="na"&gt;apiVersion&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;gateway.networking.k8s.io/v1&lt;/span&gt;
&lt;span class="na"&gt;kind&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;HTTPRoute&lt;/span&gt;
&lt;span class="na"&gt;metadata&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
  &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;gemini-routing-route&lt;/span&gt;
  &lt;span class="na"&gt;namespace&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;ai-routing&lt;/span&gt;
&lt;span class="na"&gt;spec&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
  &lt;span class="na"&gt;parentRefs&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
  &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;internal-ai-gateway&lt;/span&gt;
  &lt;span class="na"&gt;rules&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
  &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="na"&gt;matches&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
    &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="na"&gt;path&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
        &lt;span class="na"&gt;type&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;PathPrefix&lt;/span&gt;
        &lt;span class="na"&gt;value&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;/v1/generate&lt;/span&gt;
    &lt;span class="na"&gt;backendRefs&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
    &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;ai-model-router-svc&lt;/span&gt;
      &lt;span class="na"&gt;port&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;80&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;If you also run an in-cluster vLLM pool managed by GKE Inference Gateway (&lt;code&gt;InferencePool&lt;/code&gt; in &lt;code&gt;inference.networking.k8s.io/v1&lt;/code&gt; and &lt;code&gt;InferenceObjective&lt;/code&gt; in &lt;code&gt;inference.networking.x-k8s.io&lt;/code&gt; for workload prioritization), you can attach an additional &lt;code&gt;HTTPRoute&lt;/code&gt; rule on the same &lt;code&gt;internal-ai-gateway&lt;/code&gt; (or dispatch internally from &lt;code&gt;router_app.py&lt;/code&gt;) to route high-volume batch traffic directly to &lt;code&gt;group: inference.networking.k8s.io&lt;/code&gt;, &lt;code&gt;kind: InferencePool&lt;/code&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  Verifying the Gateway and routing behavior
&lt;/h2&gt;

&lt;p&gt;After applying the manifests, verify that the GKE Gateway controller has programmed the internal load balancer and attached both backend policies:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="c"&gt;# Confirm Gateway programmed status and internal IP address&lt;/span&gt;
kubectl get gateway internal-ai-gateway &lt;span class="nt"&gt;-n&lt;/span&gt; ai-routing

&lt;span class="c"&gt;# Verify that both policies are attached to the Service (look for Reason: Attached)&lt;/span&gt;
kubectl describe gcpbackendpolicy gemini-router-backend-policy &lt;span class="nt"&gt;-n&lt;/span&gt; ai-routing
kubectl describe healthcheckpolicy gemini-router-healthcheck &lt;span class="nt"&gt;-n&lt;/span&gt; ai-routing
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Because &lt;code&gt;gke-l7-rilb&lt;/code&gt; exposes only an internal IP address, look up the address with &lt;code&gt;kubectl&lt;/code&gt;, then send the test requests from a client in the same VPC and region (for example, a debug Pod in the cluster):&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="nb"&gt;export &lt;/span&gt;&lt;span class="nv"&gt;GATEWAY_IP&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="si"&gt;$(&lt;/span&gt;kubectl get gateway internal-ai-gateway &lt;span class="nt"&gt;-n&lt;/span&gt; ai-routing &lt;span class="nt"&gt;-o&lt;/span&gt; &lt;span class="nv"&gt;jsonpath&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s1"&gt;'{.status.addresses[0].value}'&lt;/span&gt;&lt;span class="si"&gt;)&lt;/span&gt;

&lt;span class="c"&gt;# 1. Fast classification query (routes automatically to gemini-3.8-flash)&lt;/span&gt;
curl &lt;span class="nt"&gt;-s&lt;/span&gt; &lt;span class="nt"&gt;-X&lt;/span&gt; POST &lt;span class="s2"&gt;"http://&lt;/span&gt;&lt;span class="k"&gt;${&lt;/span&gt;&lt;span class="nv"&gt;GATEWAY_IP&lt;/span&gt;&lt;span class="k"&gt;}&lt;/span&gt;&lt;span class="s2"&gt;/v1/generate"&lt;/span&gt; &lt;span class="se"&gt;\&lt;/span&gt;
    &lt;span class="nt"&gt;-H&lt;/span&gt; &lt;span class="s2"&gt;"Content-Type: application/json"&lt;/span&gt; &lt;span class="se"&gt;\&lt;/span&gt;
    &lt;span class="nt"&gt;-d&lt;/span&gt; &lt;span class="s1"&gt;'{"prompt": "Extract the HTTP status code from this log line: GET /checkout 502 Bad Gateway"}'&lt;/span&gt; | jq &lt;span class="nb"&gt;.&lt;/span&gt;

&lt;span class="c"&gt;# 2. Complex architectural query (routes automatically to gemini-4-argon)&lt;/span&gt;
curl &lt;span class="nt"&gt;-s&lt;/span&gt; &lt;span class="nt"&gt;-X&lt;/span&gt; POST &lt;span class="s2"&gt;"http://&lt;/span&gt;&lt;span class="k"&gt;${&lt;/span&gt;&lt;span class="nv"&gt;GATEWAY_IP&lt;/span&gt;&lt;span class="k"&gt;}&lt;/span&gt;&lt;span class="s2"&gt;/v1/generate"&lt;/span&gt; &lt;span class="se"&gt;\&lt;/span&gt;
    &lt;span class="nt"&gt;-H&lt;/span&gt; &lt;span class="s2"&gt;"Content-Type: application/json"&lt;/span&gt; &lt;span class="se"&gt;\&lt;/span&gt;
    &lt;span class="nt"&gt;-d&lt;/span&gt; &lt;span class="s1"&gt;'{"prompt": "Analyze the root cause and trade-offs of etcd leader election timeouts during cross-zone network partitions."}'&lt;/span&gt; | jq &lt;span class="nb"&gt;.&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Expected response showing the selected tier, location, fallback status, and token accounting (including reasoning &lt;code&gt;thoughts_tokens&lt;/code&gt;):&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"model_used"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"gemini-4-argon"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"location"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"global"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"fallback_triggered"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;false&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"latency_ms"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mf"&gt;2840.15&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"usage"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="nl"&gt;"prompt_tokens"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;24&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="nl"&gt;"output_tokens"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;412&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="nl"&gt;"thoughts_tokens"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;186&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"output"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"..."&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  Operational caveats and when not to use a router
&lt;/h2&gt;

&lt;p&gt;Before introducing a model router in front of Gemini 4 Argon, weigh these operational trade-offs:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Classifier accuracy vs. quality regressions&lt;/strong&gt;: Heuristic or embedding-based routers can misclassify terse prompts that actually require deep reasoning (for example, a short mathematical puzzle). Always expose an explicit override header (&lt;code&gt;X-Model-Tier: pro&lt;/code&gt;) so latency-tolerant or high-stakes workflows bypass heuristics completely.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Multi-turn conversation state&lt;/strong&gt;: Switching model tiers mid-conversation can alter output formatting or tool-calling behavior. For stateful chat sessions, pin the model tier selected on the first turn or evaluate tier upgrades only when a tool execution fails.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;When not to use a routing layer&lt;/strong&gt;: If you operate a single application with predictable prompt complexity and uniform latency targets, calling the Gemini platform directly from the application SDK with built-in retry policies avoids maintaining an extra hop in your cluster.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Next steps
&lt;/h2&gt;

&lt;p&gt;Decoupling your microservices from hardcoded model strings lets you adopt Gemini 4 Argon where its reasoning capabilities matter most while keeping routine queries fast and cost-effective on &lt;code&gt;gemini-3.8-flash&lt;/code&gt; or in-cluster accelerators.&lt;/p&gt;

&lt;p&gt;To expand this architecture, review the &lt;a href="https://cloud.google.com/kubernetes-engine/docs/concepts/gateway-api" rel="noopener noreferrer"&gt;GKE Gateway API documentation&lt;/a&gt;, explore the &lt;a href="https://cloud.google.com/kubernetes-engine/docs/concepts/about-gke-inference-gateway" rel="noopener noreferrer"&gt;GKE Inference Gateway guide&lt;/a&gt; for routing to self-hosted open models, and check the Gemini platform model locations and quota documentation for multi-region availability and throughput configuration.&lt;/p&gt;

</description>
      <category>kubernetes</category>
      <category>ai</category>
      <category>gke</category>
      <category>googlecloud</category>
    </item>
  </channel>
</rss>
