The Next Step After Karpathy’s Agentic Engineering

In today's newsletter:

• The next step after Karpathy’s agentic engineering.

HANDS-ON

The next step after Karpathy’s agentic engineering

Imagine a RAG agent that retrieves four policy documents and drafts a cited customer reply. The answer is ready, but the final tool call needs human approval.

By the time approval arrives (which could take a few hours), the Python process would be gone, and the container may be gone with it too.

Re-running the request looks harmless, but it creates a new operation. In fact, a fresh retrieval can change the evidence and draft, while a retried publication can execute twice.

The chat transcript cannot recover this safely. We need the exact unfinished invocation, its retrieved evidence, the pending confirmation, and a record of any side effect already completed.

Google’s Agents CLI provides the build path for this system. It gives Claude Code the ADK guidance for resumable runs, then carries the same project through testing and deployment to Agent Runtime.

Google's Agents CLI GitHub Repo →

It installs seven focused skills into the coding agent you already use. They cover ADK code, scaffolding, evaluations, deployment, publishing, and observability.

We will use it to preserve the invocation across processes and make publication safe to replay.

The workflow

We will create a production-grade RAG agent for a product. Its knowledge base contains five short Markdown files covering regional hosting, retention, model training, security controls, and support escalation.

A customer asks this question:

The agent has two tools.

• retrieve_passages runs deterministic TF-IDF retrieval over the local documents.
• submit_for_publication validates citations and pauses for human approval.

The second tool makes the workflow long-running. Publication cannot proceed until someone approves the draft.

The process should be free to exit during that wait. A new process must later recover the same session, approval request, evidence, and draft.

Set up Agents CLI

The project uses Python 3.11 or newer, uv, the Google Cloud SDK, and Agents CLI.

Install Agents CLI and add its skills to Claude Code:

Then create a small ADK project targeting Agent Runtime:

We continue working inside Claude Code. Agents CLI supplies the ADK, evaluation, and deployment instructions.

We start with a planning pass. Claude Code reads the scaffold and checks the installed ADK before changing any files:

The planning pass exposed several optional branches. We removed them before implementation. The article needs one path that is easy to inspect:

Claude Code produces a local knowledge base and two tools. It also adds a SQLite session store and simulated outbox.

The relevant ADK configuration is only a few lines:

Inside the publication tool, the first call requests confirmation:

ADK records the confirmation request as part of the invocation. With resumability enabled, the invocation can stop here and continue later.

Start the run

Run the first process:

The retriever returned four passages. They covered Frankfurt and Amsterdam hosting, 30-day application logs, 12-month audit logs, 35-day backups, and the model-training policy.

The model then produced a customer-facing answer with a chunk ID after every claim. The workflow reached submit_for_publication and paused.

The state at that point was:

The script printed the session, invocation, and confirmation IDs. It then exited completely.

At this point, nothing is polling in the background, and no Python process remains alive.

Resume from another process

Later, open a new terminal and approve the same session:

The new runner loads the persisted session as depicted below:

ADK finds the suspended invocation and re-enters the publication tool with the confirmation attached.

The result proves that the workflow continued instead of restarting:

Retrieval stayed at one run. The evidence and draft came from the original process.

Long-running agents need durable execution state, not only chat history. The system must preserve the unfinished operation and its evidence.

Retries must be harmless

Resumable tools commonly provide at-least-once execution. A tool may run again after a retry, timeout, or repeated approval.

That is fine for retrieval but it is dangerous for payments, emails, ticket creation, or publication.

We therefore derive an idempotency key from the answer and citation IDs. Before publishing, the tool checks whether that key already has a completed record:

Running the same approval again returns already_published. The retrieval count remains one, and the publication count remains one.

The local outbox is only a visible simulation. A production system would put the same key in a database or pass it to the external API performing the side effect.

Stop unsupported questions early

Resumability does not fix grounding. We still need to prevent unsupported work from reaching the approval stage.

We asked the agent:

The knowledge base says nothing about FedRAMP. Retrieval returned supported=False, so the agent produced no draft and requested no approval.

Here, the retrieval score acts as a publication gate. A score below the demo's cutoff blocks the publication tool. The cutoff itself does not detect hallucinations.

Move the paused run to Agent Runtime

SQLite proves the mechanism locally. The more useful test is whether the run survives outside the developer process.

Before creating cloud resources, we asked Claude Code to inspect the deployment target, Agent Identity, session service, APIs, IAM, environment variables, and packaged knowledge base. It returned a deployment plan without changing the project.

After that review, we deployed the same application to Agent Runtime inside Agent Platform.

Agent Identity gave the deployed agent its own cloud principal. The runtime replaced the local SQLite service with managed session persistence.

The deployment writes its Agent Runtime resource ID to deployment_metadata.json. Our remote demo reads that file and invokes the deployed agent directly:

The command creates a managed session, retrieves four passages, drafts the answer, and stops at the approval request. It then prints the session ID needed to continue the same invocation.

At this point, the client can exit. Open a new terminal and approve the pending action with the printed session ID:

The new process loads the session from Agent Runtime and resumes the pending tool call. This first approval publishes the answer:

We then run the same command again to simulate a duplicate approval:

uv run python scripts/remote_demo.py decide \
--session-id <SESSION_ID> \
--approve

This time, the publication tool finds the completed record and returns the saved result:

tool status describes the result of this repeated call. The stored publication state remains published because the original publication completed successfully.

We can confirm the stored state with one final command:

Combining the replay result with the final session state gives us:

The first approval published the answer. The deliberate replay returned already_published, while the retrieval and publication counts both remained one. The tool did not publish the answer twice.

Agentic engineering gives us a sound development loop. We specify the agent, build it, test it, and deploy it.

Production adds another requirement. An unfinished run must be accessible beyond the process that started it.

An agent may wait hours for a person to approve its next action. During that time, the client can disconnect, or the runtime can restart. The approval may also arrive more than once, so the resumed action must be safe to retry.

As demonstrated in this issue, Agents CLI carries this workflow through the coding agent, from the initial ADK project to its Agent Runtime deployment. ADK provides the pause and resume mechanism, Agent Runtime stores the session, and Agent Identity gives the deployed agent its own cloud principal.

This gives us an agent that can recover the exact run it started and finish the work without repeating the external action.

You can use the same Agents CLI workflow to build, test, and deploy your own ADK agent.

Here’s Google’s Agents CLI repo on GitHub →

(don’t forget to star it ⭐️)

Good day, and thanks to Google Cloud for partnering with us on today’s issue.

NO-FLUFF RESOURCES TO...

Succeed in AI Engineering roles

All businesses care about impact. That’s it!

• Can you reduce costs?
• Drive revenue?
• Can you scale ML models?
• Predict trends before they happen?

We have discussed several other topics (with implementations) in the past that align with such topics.

Here are some of them:

• Learn MLOps from first principles to production in this course with 18 parts →
• Learn everything about MCPs in this course with 9 parts →
• Learn how to build Agentic systems in this course with 14 parts.
• Learn how to build real-world RAG apps, evaluate, and scale them in this course.
• Learn sophisticated graph architectures and how to train them on graph data in this course.
• So many real-world NLP systems rely on pairwise context scoring. Learn scalable approaches here.
• Learn how to run large models on small devices using Quantization techniques.
• Learn how to generate prediction intervals or sets with strong statistical guarantees for increasing trust using Conformal Predictions.
• Learn how to identify causal relationships and answer business questions using causal inference in this course.
• Learn how to scale and implement ML model training in this practical guide.
• Learn techniques to reliably test new models in production.
• Learn how to build privacy-first ML systems using Federated Learning.
• Learn 6 techniques with implementation to compress ML models.

All these resources will help you cultivate key skills that businesses and companies care about the most.

Today’s email was brought to you by Avi Chawla and Akshay Pachaar.

Looking for more? Unlock our premium DS/ML resources.

© 2026 Daily Dose of Data Science