OpenAI’s Bubeck denies trying to cut Anthropic mathematician from credit30m ago 10 of the scariest AI villains in movie history31m ago Maven Robotics wants to steal your robot deployment deal32m ago Agent Security Is Not AI Governance: Why Conflating The Two Is A Risk Enterprises Can't Afford35m ago Positron Valued at $5 Billion in New Funding as Demand for AI Chips Surges42m ago 7 Steps to Become a Forward Deployed Engineer in 202649m ago Ramp eyes $60bn valuation three months after $44bn, Bloomberg reports1h ago Amazon partners with OpenAI to let users of Amazon's DSP run ads in ChatGPT in a pilot program limited to the US; ChatGPT Ads hit $1B in ARR last month1h ago D-Matrix Connects Raptor XPUs to NVIDIA AI Factories via NVLink Fusion1h ago Source: Meta AI researcher Andrew Tulloch, who came over from Thinking Machine Lab as one of the highest paid employees last year, is leaving the company (Reed Albergotti/Semafor)1h ago Mother Horrified When Instagram’s AI Scours Social Media for Invasive Information About Her Young Daughters1h ago Mach Industries, which makes unmanned military vehicles and weapons, raised $600M in an extension to its $300M Series C, doubling its valuation to $3.7B1h ago Salesforce Unveils Six-Capability Trusted AI Harness for Enterprises1h ago US closes $1.9bn loan to restart the Iowa nuclear plant powering Google2h ago

OpenAI’s Bubeck denies trying to cut Anthropic mathematician from credit30m ago 10 of the scariest AI villains in movie history31m ago Maven Robotics wants to steal your robot deployment deal32m ago Agent Security Is Not AI Governance: Why Conflating The Two Is A Risk Enterprises Can't Afford35m ago Positron Valued at $5 Billion in New Funding as Demand for AI Chips Surges42m ago 7 Steps to Become a Forward Deployed Engineer in 202649m ago Ramp eyes $60bn valuation three months after $44bn, Bloomberg reports1h ago Amazon partners with OpenAI to let users of Amazon's DSP run ads in ChatGPT in a pilot program limited to the US; ChatGPT Ads hit $1B in ARR last month1h ago D-Matrix Connects Raptor XPUs to NVIDIA AI Factories via NVLink Fusion1h ago Source: Meta AI researcher Andrew Tulloch, who came over from Thinking Machine Lab as one of the highest paid employees last year, is leaving the company (Reed Albergotti/Semafor)1h ago Mother Horrified When Instagram’s AI Scours Social Media for Invasive Information About Her Young Daughters1h ago Mach Industries, which makes unmanned military vehicles and weapons, raised $600M in an extension to its $300M Series C, doubling its valuation to $3.7B1h ago Salesforce Unveils Six-Capability Trusted AI Harness for Enterprises1h ago US closes $1.9bn loan to restart the Iowa nuclear plant powering Google2h ago

AI Research Machine Learning6 min reading time

OpenAI’s AGI number came from a harness, not the model

The Next WebPublished 4 days ago

OpenAI's GPT-6 Astra achieved a 99.9% AGI benchmark score using its proprietary Provider Adapter harness, but the same model scored 62.7% under the standard ARC Prize harness. The difference stems from the software environment around the model, not the model itself, highlighting the impact of evaluation setup on AGI claims.

More on this story

EU cybersecurity agency is now testing Mythos 5 and GPT-6 Astra, the Commission says
5h· The Next Web
The Fight Over OpenAI’s Math Breakthrough Is a New Kind of Scientific Arms Race
19h· 2 sources
OpenAI says an internal model "significantly more capable than GPT-6 Astra" solved the Navier-Stokes problem using 10K concurrent agents working for 88 hours (Madison Mills/Axios)
2d· 6 sources
Last Week in AI #343 - GPT-6, OpenAI’s agents chatted on a wiki, Fable 5.1
3d· 7 sources
GPT 6 Is Out In OpenAI’s Sandbox
3d· Forbes
Nvidia's Jensen Huang says 'AGI has arrived' and congratulates OpenAI
3d· 2 sources

#arc-prize #benchmarking #francois-chollet #gpt-6-astra #mike-knoop #openai

More in AI Research

Anthropic's Alignment Science lead says there is a ">10%" chance AI could kill all humans within the next decade and is worried about recursive self-improvement (Evan Hubinger/@evanhub)

Covered by 9 sources1d ago

AI Research4 min read

Suno trained its v6 AI music models with help from Warner and BMG

Covered by 5 sources1d ago

Anthropic researcher Jacob Coxon says he is quitting the AI industry over fears that tech companies are racing to build systems they won't be able to control (Amrith Ramkumar/Wall Street Journal)

Covered by 11 sources2d ago