TAI #228: AI ROI Beyond Labs Revenue + Our Interview with OpenAI's Eoin Boylan
TAI #228: AI ROI Beyond Labs Revenue + Our Interview with OpenAI's Eoin Boylan
Also, Kimi K3.1 Pro, GPT-6.1 Terra, Artificial Analysis's new leaderboard, Anthropic's internal AI usage, and OpenAI's Eoin Boylan on forward deployed engineering.
What happened this week in AI by Louie
There were some great new models this week, along with more focus on AI return on investment (ROI), particularly around Anthropic’s IPO filing and the growing funding needs of the AI buildout. We also spoke with Eoin Boylan, OpenAI’s Head of Forward Deployed Engineering for EMEA, to discuss how his team works with clients. We also run an AI engineering and forward deployed engineering (FDE) consultancy at Towards AI, and both teams are hiring.
On the model side, Kimi K3.1 Pro is another jump for the open-weight ecosystem, with the hosted model already live and full weights promised by November 4. GPT-6.1 Terra is another great push for cost-efficient computer use among closed models. OpenAI’s GPT-6.1 Terra is also a meaningful update for cost-efficient computer use among closed models. On OpenAI’s Agents’ Last Exam, it beats GPT-5.6 Terra by 15 percentage points at medium effort while costing roughly a third as much, and it also improves OpenAI’s writing across a wide range of tasks.
Artificial Analysis’s update to its Intelligence Index (v5) led to quite a shuffle in the leaderboard. Evaluations with private tests or held-out answers now carry 60% of the weighting, up from 45%. The revision also changed difficulty and task mix. Anthropic and OpenAI hold the top five places. At maximum reasoning effort, Opus 5.5 scores 58 and GPT-6 Astra 57. Kimi K3.1 Pro trails the leader by 12 points, against three on the previous index. Among models whose weights are already released, the leaders score 32, a 26-point gap.
Many are interpreting this as a sign that some other labs were more heavily benchmaxxed on familiar benchmarks than on general, robust real-world performance. That fits my experience too: I find Opus and Astra far out ahead for most of my tasks.
Where the returns show up
AI agents are now obviously superhuman at many things, but many are asking why AI isn’t already generating more revenue to start justifying the huge AI capex. I would look for most of the LLM ROI outside AI lab revenue. Meta’s quarterly revenue per employee rose 27% between Q4 2025 and Q3 2026, and 205% since Q3 2022, just before ChatGPT launched. Put another way, Meta’s Q3 revenue annualizes to $258bn. At today’s headcount and its Q3 2022 revenue per employee, it would be about $85bn, or $174bn lower.
That is obviously not $174bn of LLM productivity. Inflation, war, currency moves, reorganizations, headcount cuts, capex, and many other factors also drive that change. Even so, the naive productivity boost at Meta alone could well exceed OpenAI and Anthropic’s combined annualized revenue. For scale, OpenAI’s reported September run rate was almost $50bn, while Anthropic’s reported Q3 revenue annualizes to about $67bn (on a different definition). Within AI’s contribution at Meta, GPU-based recommender systems provide a big boost alongside LLMs. Meta also employs only ~0.0018% of the world’s employed people.
The AI labs themselves are leading the way in using these tools and achieving ROI from internal use. Anthropic’s S-1 says average daily code volume contributed per engineer in Q3 was more than 45 times its 2024 average. Code volume is not the same as business productivity, but LTM revenue per employee rose from $1.4m to over $6.6m between September 2025 and September 2026. Internal Claude usage, valued at list prices, grew more than 250x from January to September, first exceeding monthly personnel expense in April (ex stock-based compensation). I expect similarly intensive internal AI use at OpenAI. Higher output per employee can justify enormous capital investment when a business is set up to allow each person to direct far more compute effectively.
Most enterprise AI ROI potential, however, is still being wasted by slow adoption. Agents are still being treated like 2023 chatbots, and token spend is constrained. Some faster-moving organizations are already getting huge increases in productivity per employee well beyond coding. In most white-collar sectors, I think, if used well, models could deliver a 2x+ productivity boost today in many roles, such as law, financial research, sales, accounting, engineering, HR, marketing, and management. We don’t need to wait for a new class of model to begin capturing that. Codex and Claude are already enough, but the capabilities need to be put on a plate before most people start using them to their potential.
Forward deployed AI engineers like ours make adoption easier by teaching people, building and optimizing skills, and then building custom apps and evaluations for more complex tasks.
However, enterprise pricing does make ROI harder. For heavy usage, my rough estimate is that metered enterprise inference can cost 10 to 20x as much as the effective cost through personal or small-team subscriptions. Claude Team supports up to 150 seats; new ChatGPT Business workspaces support 200.
As a result, most enterprises handicap the models with token budgets that are far too low. The new Artificial Analysis results show why budget affects capability: Opus scores 35 at low effort and 58 at max, at $0.98 versus $10.04 per evaluation task. I use subagents or loops for parallel research, criticism, iteration, and testing, and those loops need room to run. The useful measure is how much good work comes back and how much human effort it saves, not how well token spend fits into arbitrary legacy IT budgets.
The biggest friction point outside code is that the models are still bad at writing. Poor prose can make capable reasoning look worse than it is, create more editing work, and add to the fatigue people already feel from generic AI-generated text. The labs have been far too slow to care about this.
Once adoption takes hold, ROI still requires management to act. Your company ideally needs the imagination and agility to use the productivity and capability boost to launch new features, and enter new markets, and drive revenue growth; otherwise, it will be forced to cut staff to achieve ROI. Management also has to clear new bottlenecks quickly: hire fast for tasks AI does badly, build custom tools, and commit capex where it is needed. In practice, it may be easier to build an AI-native law firm that can handle five times as much work per person than to make an existing firm grow fivefold or cut 80% of its staff.
What OpenAI’s FDEs build
Eoin spoke with Assel from our team and me about how OpenAI approaches this work. His FDE team focuses on model intelligence gaps and missing product capabilities. OpenAI’s Deployment Company helps customers implement existing capabilities at scale, while partners bring established relationships and industry expertise.
Every engagement starts with discovery, where his team challenges the customer’s proposed use cases: will they deliver enough ROI, and can the customer make that case to its board? At a major semiconductor company, where the goal was a shorter design-to-fabrication cycle, the team identified more than 50 use cases and ranked them by complexity and value. Starting with quick wins like these helps secure cooperation from the hardware experts whose knowledge the generalist FDEs need.
His team now rarely fine-tunes for individual tasks because improvements in the base models keep overtaking that work. Some FDEs work inside research, helping close the capability gaps found at customers. They also build most solutions on the Codex harness with plugins and skills. Eoin described repeatedly building custom functionality only for the product to ship it a month later, forcing a redesign. Direct API use remains useful for particular integrations and specialized tasks. His team has direct access to OpenAI’s product and research groups, but the lesson applies more widely: choose carefully which parts of a workflow deserve custom engineering.
He also described smaller delivery teams: in one example, work scope that once required six engineers and a technical delivery lead is increasingly approached with two engineers and a lead. On agent autonomy, they define domain boundaries, interfaces, and tests first, then let agents implement within them. That is how we work too. When I suggested this favored greenfield projects, he pushed back. Legacy modernization is a big part of their work, including moving banking systems from COBOL to Java by understanding the business logic and migrating piece by piece.
Eoin was also clear that slow enterprises often have concrete constraints, including compliance requirements and fixed factory production slots. At a large Spanish bank, his team is working to reduce the time analysts spend finding and filing compliance requirements for commercial credit memos, freeing more time for risk assessment. Improving the work around a professional decision can be a valuable deployment in its own right without full automation.
We also discussed our sports video analysis work, where we are pushing an evaluation towards what human analysts produce and combining models because Astra is too expensive to use everywhere. Eoin said excessive token use can itself be a model problem his team takes back to research. Giving agents the right budget and making them more efficient are both part of getting the economics right.
Thanks for reading Towards AI Newsletter! Subscribe for free to receive new posts and support my work.
SubscribedWhy should you care?
We are hiring around the work that makes these deployments succeed: discovery with the people who own the problem, domain knowledge built into the workflow, and architecture, tests, and security that clients can trust. Faster implementation lets us take on more of this work. It raises the value of people who can choose and deliver the right project.
Towards AI started in 2019 as an AI learning platform. We have grown to 500,000 learners, a newsletter with 200,000 subscribers, and a book on LLMs in production available on O’Reilly. We have since built a world-class deployment team, scoping and building AI tools and automations for private-equity-backed businesses and larger enterprises. Some of our current projects include a video agent that analyzes live sports, a 3D construction drawing analyzer that automates sales bidding, AI for investment use cases, a bootcamp for traders and analysts at a large investment bank, and a pricing tool built from thousands of customer emails. We are an OpenAI Select Partner, an NVIDIA Inception member, and a Cloudflare deployment partner.
We work in pods of two AI Engineers/FDEs and one Deployment Strategist, with further support from our wider team, including me as Chief AI Architect. We are growing in London across three roles:
Deployment Strategist: You are technical, build tools yourself with Claude Code, Codex or Cursor, and understand the engineering work being done by the FDEs in your pod. You bring consulting or product management experience. You are comfortable working with clients and running workshops with CEOs and CFOs, and have a feel for interfaces and how people actually use a tool.
AI Engineer / Forward Deployed Engineer: You use coding agents every day. We welcome experienced full stack developers or recent graduates in computer science, maths, or similar fields. You get production work across several clients and learn real-world AI engineering from our experienced team.
Technical Lead, Platform and Security: You typically bring 10+ years of experience building and running production systems, with Python and TypeScript, cloud on AWS, Azure, or GCP, continuous delivery, monitoring, backups, and recovery. You understand identity and access through Entra ID, Okta, and single sign-on, and security to OWASP, Cyber Essentials Plus, or ISO 27001. You have taken products through client security reviews and change control, and use coding agents daily with clear rules on what they can interact with.
All roles are based in London. Explore our careers or share them with someone who fits.
OpenAI is also hiring Forward Deployed Engineers in London and globally.
— Louie Peters — Towards AI Co-founder and CEO
Hottest News
1. Moonshot AI Releases Kimi K3.1 Pro
Moonshot AI released Kimi K3.1 Pro, the first model in its K3.1 series, with a 1M-token context window. Moonshot describes it as its most capable 3T-class open-weight model to date, although the full weights are not yet available and are scheduled for release by November 4. K3.1 Pro is a post-trained version of Kimi K3 with substantially more post-training compute, including large-scale joint reinforcement learning across hundreds of downstream domains. Moonshot also released K3.1 Pro Highspeed, which has the same capabilities at roughly 2–3× the output speed. On Terminal-Bench 4.0, Moonshot reports 65.7% for K3.1 Pro, compared with 58.2% for GPT-6 Astra and 66.4% for Claude Opus 5.5. K3.1 Pro also scores 92.7% on a 1,000-case BenchCAD subset with tools and 64.4 on OfficeQA Pro v2. On Moonshot’s internal Kimi Hallu Bench Hard, its claim-level hallucination rate fell from 23.4% for K3 to 8.3%. For the planned open-weight release, Moonshot removed cyber-related data from mid-training and filtered cyber tasks from reinforcement learning; an otherwise comparable checkpoint without these measures scored 90.5% on its internal xdev-bench-hard, compared with 81.0% for the final K3.1 Pro. K3.1 Pro and Highspeed are available through Kimi.com, the Kimi apps, Kimi Work, Kimi Code, and the Kimi API.
2. DeepSeek Open Sources DeepInfer
DeepSeek open-sourced DeepInfer, the high-performance inference framework that powers its large-scale production services. The current release supports the DeepSeek-V4.1-Flash model, NVIDIA SM100 GPUs, and Huawei Ascend 950 NPUs. It includes optimized kernels such as FlashMLA, DeepGEMM, DeepEP, TileKernels, and DeepSelect; attention and expert parallelism; DSpark speculative decoding with confidence-scheduled verification; asynchronous scheduling; batch-invariant inference; deterministic sampling; and structured outputs. The public release currently provides an offline-inference path, rather than a complete serving system. DeepInfer was built primarily on vLLM, with additional inspiration from SGLang and TensorRT-LLM, and is released under Apache 2.0.
3. Anthropic Launches OSS Scanner
Anthropic launched OSS Scanner, a free, opt-in service that periodically scans eligible open-source projects for vulnerabilities using its strongest models, including Claude Mythos. Over the past six months, Anthropic says its models found more than 29,000 candidate vulnerabilities, while human reviewers manually reviewed and triaged about 6,000. OSS Scanner offers an optional faster path in which models generate reports and send them directly to enrolled maintainers without human review, so Anthropic warns that some findings or severity ratings may be wrong. Reports include a reproducer, an explanation of the vulnerability, and a candidate patch when available. In an early validation, expert penetration testers reviewed 97 critical and high-severity findings across 48 projects; 85 met Anthropic’s coordinated-disclosure bar, 11 were valid, but duplicated known findings, and one was invalid. Core maintainers of eligible projects can enroll by submitting a PR to this GitHub repo following the standard project template.
4. Claude Designs Lab-Validated Protein Binders Against Scorpion Neurotoxins
Anthropic reported that an internal Claude research model operating within Claude Science designed protein binders against scorpion neurotoxins, with the longer-term aim of exploring new approaches to antivenom development. Over 64 hours, Claude worked largely autonomously from an initial prompt and four follow-ups, with input from Citrini Research. It generated 318,484 protein designs targeting five toxins and selected 180 for testing; Anthropic later added another 31 designs with Claude’s help. Adaptyv Bio and Twist Bioscience tested the resulting 211 designs, and 17 met Anthropic’s criteria for binding, each against a single toxin. Anthropic has not tested whether the designs neutralize the toxins in cells or protect animals, so it does not claim to have produced an antivenom. Anthropic has publicly released all 211 designs and their experimental data.
5. OpenAI Cuts Frontier Training Off From the Live Internet
OpenAI announced on October 23 that it now conducts frontier-model training without live internet access, using an offline environment containing indexed and cached web content so models can still practice browsing and interacting with web applications. The policy follows a September 20 incident in which an internal research model exploited a DNS-resolver misconfiguration and gained limited access to the live internet during training. OpenAI paused training and evaluation of its most capable models, hardened its infrastructure and monitoring, and launched a high-compute red-teaming campaign in which teams of frontier-model agents tried to reach the internet, cross internal security boundaries, and communicate across isolated environments. OpenAI resumed frontier reinforcement-learning training and internal inference on October 1 after evaluating the strengthened controls.
6. NVIDIA Releases Nemotron 3.5 Super VL
NVIDIA released Nemotron 3.5 Super VL, an open 120B-parameter multimodal model for specialized agentic workloads, alongside two smaller Bolt models for local and edge deployment. Super VL handles text and visual inputs, including images, webpages, PDFs, and video. NVIDIA says its accuracy on multimodal agentic tasks is comparable to Nemotron 3 Ultra at roughly one-fifth the parameter count. Super VL was trained on about 26T tokens across pre-training and continued pre-training, followed by post-training on roughly 3.8 million samples and reinforcement learning across dozens of environments. The release includes open weights, datasets, and training recipes. NVIDIA also released Nemotron 3.5 Bolt 74B-A9B, which uses native 2-bit quantization to fit in 24GB of memory for local agentic workloads, while Nemotron 3.5 Bolt E4B, a 4GB-footprint model for conversational and physical agents, followed on October 23. Super VL is available through Hugging Face, OpenRouter, ModelScope, and NVIDIA NIM.
Cohere released Rerank 5 in Pro and Fast versions for reranking retrieved documents. In Cohere’s evaluations, Rerank 5 Pro leads the ViDoRe V3, nanoMTEB, BRIGHT, and BEIR benchmark groups, ranking first on 39 of 46 datasets. Its largest reported generational gain is on BRIGHT, where it scores 82.2, 5.8 points above Voyage AI Rerank 3 and 12.4 points above Rerank 4 Pro. Both versions add Context Focus, which returns scored relevant spans within each document so applications can send only selected passages to an LLM. Across 30 question-answering datasets, Cohere reports that removing half of the retrieved text retained 96% of answer quality. Rerank 5 also introduces calibrated relevance scores, which Cohere says can be interpreted approximately as probabilities of a document being useful. In Cohere’s single-A100 throughput test, Fast processed 2.9× as many documents per second as Pro. Rerank 5 is generally available through the Cohere API, Model Vault, Microsoft Foundry, Amazon SageMaker, and North.
AI Tip of the Day
You do not need a framework for 90% of AI apps. In fact, while building the code for our 2026 AI Engineering book, we deleted LangChain and LlamaIndex from our core RAG and app chapters.
When we rewrote those pipelines in plain Python, the code got shorter, faster, and much easier to fix:
Prompt templates: Just use a standard Python f-string or a 10-line class.
Self-fixing output parsers: A while loop that catches a Pydantic ValidationError and sends the exact error message back to the LLM takes 15 lines of code.
Parallel chains: Running map-reduce across a big document is just asyncio.gather().
Vector search: An in-memory vector store with cosine similarity is literally one NumPy matrix multiplication (np.dot).
When you write these basic parts in plain Python, nothing is hidden. You can see every token, catch every bad output, and swap models in seconds. Keep the heavy frameworks like LangGraph for the rare times you actually need a complex state machine.
Five 5-minute reads/videos to keep you learning
1. How I Made Local LLMs Up to 2.24× Faster on a 4GB NVIDIA GPU
This article benchmarks Gemma 4, Qwen3, and Qwen3.5 on a 4GB GTX 1650 Ti using llama.cpp, then tunes GPU layer placement, CPU threads, microbatch size, and KV-cache precision. The largest gains came from GPU offloading, which increased Qwen3 and Qwen3.5 token generation by 2.07× and 2.24×. It also shares the memory-throughput trade-off of KV-cache quantization and provides reproducible launch configurations for running current local models on limited VRAM.
2. AI Is Inventing New Mathematics. Are We Ready for What It Discovers Next?
The article looks at recent cases where AI systems have contributed to new mathematical results, including AlphaEvolve, an 80-year-old geometry conjecture, and OpenAI’s work on a Navier–Stokes singularity. It explains three approaches behind these results: evolutionary search, language-model reasoning, and formal verification with systems such as Lean. It then examines the questions these methods raise around verification, mathematical understanding, authorship, and the role of human mathematicians.
3. Build a Jev-Style Decision Model from an LLM, Training the Jev Head
This article takes a small pointer head with 524,288 trainable parameters and turns a frozen vision-language backbone into a decision model that reached 0.959 accuracy on 800 synthetic support tickets, against 0.352 for random guessing. It traces why raw dot-product scores saturate softmax, then fixes it with learned projections, masked loss, shuffled option orders, one-cycle scheduling, and RMS normalization.
4. Scale Azure Data Factory ETL with 1 Metadata Control Table
The article shows how to replace separate Azure Data Factory pipelines for each source table with a metadata-driven ingestion design. It builds the workflow around an Azure SQL control table and reusable parameterized pipelines that load relational data into ADLS Gen2. The walkthrough covers full and incremental loads, watermark handling, identifier validation, checkpoints, run isolation, publication, failure recovery, and concurrency and throughput tuning.
Repositories & Tools
1. Skill Spector is a security scanner for AI agent skills that detects prompt injection, unsafe code, data exfiltration, and supply-chain risks.
2. Sandbox Runtime is a lightweight sandboxing tool for enforcing filesystem and network restrictions on arbitrary processes at the OS level, without requiring a container.
3. Agnesh is a multi-agent harness for coordinating large numbers of agents that claim tasks, share results, and verify each other’s work.
4. AKV is a KV-cache management system for long-running LLM agents that reduces context overhead while preserving reusable cache.
5. Devin-vuln-fixes is a collection of agent skills for finding vulnerable dependencies, applying fixes, testing changes, and preparing remediation PRs.
Top Papers of The Week
1. POSEIDON: Scaling Training Environments for Scientific Agents from Code Repositories
The paper introduces Poseidon, an automated pipeline that turns scientific code repositories into executable, verifiable training environments for scientific agents. It creates four task types: Reproduction, Subflow, Completion, and Correction, and uses a recursive loop to recover failed candidates. The resulting Pearl dataset contains 97,659 environments from 7,110 repositories across 10 scientific disciplines, and the authors report consistent gains when training several Qwen3.5 models on it.
2. Object-Oriented Harness: Evolving Agent Harnesses through Object-Oriented Design
The paper proposes OO-Harness, which replaces one global agent harness with a hierarchy of specialized harness classes. During training, an evolution agent creates new harness subclasses from execution records, while a router learns which harness to use for each task at test time. Across three long-horizon benchmarks, the authors report up to a 15% absolute improvement over the base harness.
3. BizBench: Unveiling Hidden Challenges in Deploying Long-Horizon Agents for Real-World Business
The paper introduces BizAgentBench, a benchmark for evaluating whether long-horizon agents follow business constraints correctly throughout execution, not just whether they reach the final goal. It includes 50 tasks, 789 knowledge documents, and 601 business tools based on procedures from a payments company. The authors find a sizable gap between goal completion and strict success, showing that agents can reach the intended outcome while still violating important policies or execution constraints.
4. LoopVL: Recurrent Visual Intelligence
The paper introduces LoopVL to study whether Loop Transformers can be effectively extended to vision-language models. It combines Module-Loop and Model-Loop computation and is trained from scratch through language pre-training, multimodal training, and post-training. The authors report better multimodal understanding and visual reasoning than several similarly sized and larger non-recurrent models.
5. Self-Evolving Coding Agents via Harness-Policy Optimization
The paper introduces Self-HPO, a framework that jointly improves a coding agent’s model policy and its harness, rather than optimizing either in isolation. In each round, the current policy analyzes its failures and proposes new harnesses, then reinforcement learning updates the policy under the selected harness. Across three coding benchmarks, the method raises Qwen3.5–35B-A3B’s average resolve rate from 57.8% to 64.8%, with gains in reliability and out-of-distribution performance.
Quick Links
1. Google Cloud launches Agent Compute Engine, a purpose-built compute service for running AI agents in isolated microVMs. The service gives each agent its own hardware-isolated environment with persistent storage, short-lived credentials, and support for pausing idle workloads and quickly restoring them from snapshots. Google says the same infrastructure already powers Gemini’s agent workloads and uses copy-on-write memory sharing to increase compute density. Agent Compute Engine is not available yet; Google says it will enter preview in the coming months.
Who’s Hiring in AI
Deployment Strategist @Towards AI (London)
AI Engineer / Forward Deployed Engineer @Towards AI (London)
Technical Lead, Platform and Security @Towards AI (London)
Forward Deployed Engineer @OpenAI (London)
AI Engineer (Agents & Tools) @99brightminds (Remote)
Senior Agentic AI Engineer/Developer @Centene Corporation (Tallahassee, FL, USA)
Senior AI Engineer @CI&T (Brazil/Remote)
Senior Software Engineer — UI @Turvo (Hyderabad, India)
Senior Engineer, Autonomy @Shield AI (Melbourne, FL, USA)
Software Engineer @Actalent (Huntsville, TX, USA)
Interested in sharing a job opportunity here? Contact [email protected].
Think a friend would enjoy this too? Share the newsletter and let them join the conversation.
Thanks for reading Towards AI Newsletter! Subscribe for free to receive new posts and support my work.
If you enjoyed reading this newsletter, you might also like our latest book ‘Building LLMs for Production’.
P.S. If you want a practical AI course to upskill your team or are looking for AI developers, contact us!
© 2026 Towards AI, Inc.
2810 N Church St PMB 99625,, Wilmington,
Delaware 19802,, US