How to Improve AI Red-Teaming Reporting
Making Public AI Evaluations
More Transparent and Useful
Report
Systematizing the Reporting of AI Red-Teaming
Recommendations to Improve the Transparency and Utility of Test Results
By: Jessica Ji, Colin Shea-Blymyer, Kelly Beavan, and Evelyn Yee
Systematizing the Reporting of AI Red-Teaming
Recommendations to Improve the Transparency and Utility of Test Results
By: Jessica Ji, Colin Shea-Blymyer, Kelly Beavan, and Evelyn Yee
Red-teaming is a key process in evaluating the safety and security of AI systems; however, its qualitative and highly contextualized nature has led to disparate methodologies and reporting practices. In this report, we conduct an interview study with AI red-teaming practitioners to learn about how current testing practices affect reporting, what information different stakeholders find valuable to report, and what challenges might impact efforts to systematize reporting practices. We identify four recurring dimensions that are important to report publicly following an AI red-teaming exercise and suggest ways policymakers and private-sector stakeholders can support the AI test and evaluation ecosystem.
Related CSET Research:
Blog Post
AI Red-Teaming Design
Threat Models and Tools
By: Evelyn Yee
AI Red-Teaming Design
Threat Models and Tools
By: Evelyn Yee
Red-teaming is a popular evaluation methodology for AI systems, but it is still severely lacking in theoretical grounding and technical best practices. This blog introduces the concept of threat modeling for AI red-teaming and explores the ways that software tools can support or hinder red teams. To do effective evaluations, red-team designers should ensure their tools fit with their threat model and their testers.
Blog Post
AI Safety Evaluations
An Explainer
By: Jessica Ji, Vikram Venkatram, and Steph Batalis
AI Safety Evaluations
An Explainer
By: Jessica Ji, Vikram Venkatram, and Steph Batalis
Effectively evaluating AI models is more crucial than ever. But how do AI evaluations actually work? Our explainer lays out the different fundamental types of AI safety evaluations alongside their respective strengths and limitations.
The Center for Security and Emerging Technology (CSET) at Georgetown’s Walsh School of Foreign Service is a research organization focused on studying the security impacts of emerging technologies, supporting academic work in security and technology studies and delivering nonpartisan analysis to the policy community. CSET aims to prepare a generation of policymakers, analysts and diplomats to address the challenges and opportunities of emerging technologies.
If you wish to opt out of all CSET emails, please click "unsubscribe from this list" below. You may also update your preferences by clicking "update your preferences" below.
Copyright © 2026 Center for Security and Emerging Technology, All rights reserved.
You are receiving this email because you are part of the Center for Security and Emerging Technology mailing list.
Our mailing address is:
You are receiving this email because you are part of the Center for Security and Emerging Technology mailing list.
Our mailing address is:
Center for Security and Emerging Technology
37th And O Street NW
Icc 301
Washington, DC20057-0001