№173|01:50 AM ET
Independent reporting on technology, markets & policy
TechEchelon
№01 / Anchor·CYBERSECURITY

White House Invites Hackers to Challenge Leading AI Models at DEF CON

About 2,200 individuals participated, attempting to trick these advanced chatbots or large language models (LLMs) into inappropriate actions like fabricating news or providing unsafe instructions within a 50-minute window.

TE
TechEchelon Staff
AUG 15, 2023 · 05:44 PM ET · 1 MIN READ
Editorial

About 2,200 individuals participated, attempting to trick these advanced chatbots or large language models (LLMs) into inappropriate actions like fabricating news or providing unsafe instructions within a 50-minute window. This initiative marked the first public evaluation of various LLMs, as confirmed by a spokesperson from the White House Office of Science and Technology Policy.

Eight tech firms were roped in for the event, including Anthropic, Cohere, Hugging Face, and Stability AI. To ensure fairness during the "red-teaming" challenge, participants received anonymized AI models on Google Chromebooks.

Kelly Crummey, a representative of the Generative Red Teaming challenge, reported a massive turnout. The challenge winner even participated 21 times. Among the crowd, 220 students from 19 states, including Ray Glower of Kirkwood Community College, tried their hand at the challenge. Glower shared his attempts to elicit harmful outputs from the AI, noting his success in extracting surveillance methods.

The White House emphasized the importance of such challenges for determining AI vulnerabilities, echoing a July pledge made by leading AI firms about the safety, security, and trust of their technologies.

While immediate results from the challenge remain undisclosed, comprehensive insights will be available in a policy paper this October. Additionally, Rumman Chowdhury, the event's co-organizer and co-founder of Humane Intelligence, mentioned an extensive transparency report set for February.

Chowdhury highlighted the tech industry's enthusiasm and cooperation for the challenge, emphasizing its significance in the current global climate. The challenge sought to evaluate AI models on various fronts, including their information integrity, societal impact, security, and ability to maintain consistency.

Concluding her remarks, Chowdhury said, "This event exemplifies a rare moment of unity among governments, corporates, and NGOs. It brings a glimmer of hope in challenging times."

TE
━ ABOUT THE BYLINE
TechEchelon Staff

TechEchelon Staff bylines are produced collectively by the newsroom for short, breaking, and wire-style coverage. Longer-form reporting is published under the responsible reporter's name.

More from the Staff
● THE BRIEF · DAILY NEWSLETTER

Five stories every morning. Before the opening bell.

Written for readers who already know the basics — markets, AI, and the policy decisions that shape both.

Mon — Fri · 06:30 ET · Free

No spam · Unsubscribe anytime