Deploying to Vertex AI Agent Engine: The Guide the Docs Skip

The mental model

Agent Engine is a managed runtime for agents. You hand it an agent object; it gives you back a resource with a session API and a streaming query endpoint. No container to write, no service to scale, no session store to run.

The mechanism underneath is the thing worth understanding, because every confusing failure traces back to it. Your agent is serialised where you run the deploy, uploaded to a staging bucket, and reconstructed inside a container built from a requirements list you supply. The agent that serves traffic is a copy of your local object, rebuilt in a different environment.

Hold onto that sentence. Four of the five problems below are consequences of it.

Prerequisite one: the staging bucket

Agent Engine needs a Cloud Storage bucket to stage the build. It must exist before you deploy and it must be in the same region as the agent — a bucket in a different region fails with a message that does not obviously say so.

Use the same region consistently for the bucket, the agent and the model. Cross-region combinations sometimes work, sometimes fail on quota, and always cost more in latency than they are worth.

PROJECT=my-project
LOCATION=us-central1
BUCKET=gs://${PROJECT}-agent-staging

gcloud storage buckets create $BUCKET --location=$LOCATION

gcloud services enable aiplatform.googleapis.com storage.googleapis.com

Prerequisite two: the runtime identity is not you

The most common post-deploy surprise is an agent that worked locally and now cannot reach anything. Locally it ran as you, with your application default credentials. Deployed, it runs as a service agent with its own permissions, and it does not inherit yours.

Anything the agent touches — a database, a secret, a bucket, another API — needs an explicit grant to that identity. Work this out before deploying rather than debugging it from a container you cannot attach to. If your agent reads a secret, that is `secretmanager.secretAccessor`. If it reads objects from GCS, that is `storage.objectViewer` on the specific bucket.

The deploy script, with the two lines that matter

`extra_packages` paths resolve relative to the current working directory, not to the script. Run the same script from a different directory and it will either fail to find your package or, worse, upload the wrong thing. Anchor the working directory explicitly at the top.

The second line is the version pinning. Pin to what is installed in the environment doing the deploying — not a range, not what you think is current. This is the serialisation boundary from the mental model, and it is the difference between an agent that answers and one that returns nothing at all.

import os, sys
from pathlib import Path
import importlib.metadata as md

REPO_ROOT = Path(__file__).resolve().parent.parent
sys.path.insert(0, str(REPO_ROOT))
os.chdir(REPO_ROOT)          # extra_packages is relative to cwd. Anchor it.

import vertexai
from vertexai import agent_engines
from my_agent.agent import root_agent


def pin(pkg, extras=""):
    return f"{pkg}{extras}=={md.version(pkg)}"

REQUIREMENTS = [
    pin("google-adk"),
    pin("google-cloud-aiplatform", "[adk,agent-engines]"),
    "google-cloud-secret-manager>=2.20.0",
]

vertexai.init(project=PROJECT, location=LOCATION, staging_bucket=BUCKET)

remote = agent_engines.create(
    agent_engine=agent_engines.AdkApp(agent=root_agent, enable_tracing=True),
    display_name="Order Support",
    requirements=REQUIREMENTS,
    extra_packages=["./my_agent"],        # relative, not absolute
    env_vars={"API_BASE_URL": os.environ["API_BASE_URL"]},
)
print(remote.resource_name)

Secrets: pass the reference, not the value

`env_vars` are visible on the deployed resource, so a literal API key placed there is a key sitting in your infrastructure metadata. Pass the Secret Manager resource name instead and resolve it inside the agent at runtime.

There is a second reason beyond exposure, and it bites later. A literal token is frozen at deploy time — when it expires, every tool call starts failing and the only fix is a redeploy. Resolving at runtime means rotating the secret is enough.

# deploy: pass the reference
env_vars = {"TOKEN_SECRET": "projects/123/secrets/api-token/versions/latest"}

# runtime: resolve it, and cache with an expiry rather than forever
from google.cloud import secretmanager

_cache = {"value": None, "fetched_at": 0}

def api_token() -> str:
    if _cache["value"] and time.time() - _cache["fetched_at"] < 300:
        return _cache["value"]
    client = secretmanager.SecretManagerServiceClient()
    name = os.environ["TOKEN_SECRET"]
    _cache["value"] = client.access_secret_version(
        name=name).payload.data.decode()
    _cache["fetched_at"] = time.time()
    return _cache["value"]

Deploys are slow, so run a preflight

A create takes several minutes. Discovering a missing environment variable at minute eight is a bad way to spend an afternoon, and you will do it more than once unless you check first.

A preflight that validates configuration, confirms the target URL is not localhost, and warns about literal credentials costs ten lines and saves an hour a week. The localhost check in particular catches a mistake almost everyone makes once: the agent works locally against a service on your machine, and that service does not exist in the container.

def preflight():
    missing = []
    if not PROJECT:  missing.append("GOOGLE_CLOUD_PROJECT")
    if not BUCKET:   missing.append("STAGING_BUCKET")
    if not BASE_URL or "localhost" in BASE_URL:
        missing.append("API_BASE_URL (the container cannot reach localhost)")
    if missing:
        print("Cannot deploy. Missing:")
        for m in missing:
            print(f"  - {m}")
        sys.exit(1)
    if os.environ.get("API_TOKEN") and not os.environ.get("TOKEN_SECRET"):
        print("WARNING: deploying a literal token. It will expire and every "
              "tool call will 401. Use Secret Manager instead.\n")

Update, do not recreate

`agent_engines.update` keeps the resource name, which matters because that name is what anything downstream points at — a Gemini Enterprise registration, an API client, a front end. Deleting and recreating hands you a new ID and quietly breaks all of them.

Store the resource name somewhere your deploy script reads automatically. Passing it by hand is how you end up with three abandoned agents in the project and no memory of which one is live.

When the deployed agent returns nothing

The failure mode worth recognising in advance: the deploy succeeds, the resource shows as active, you send a query, and you get back an empty response or a type error from somewhere deep inside the framework that has nothing to do with your code.

That is almost always the serialisation boundary. Your local library versions and the container's do not match, so the reconstructed object is missing internal state that was never part of the serialised payload. It is not a bug in your agent and no amount of reading your agent code will find it.

The next post is a full postmortem of exactly this — what the error looks like, why it happens, and the two-line fix — because it is common enough and confusing enough to deserve its own treatment.