DOOMBENCH.COM
Autonomy and agency

ARC Evals finds 2023 agents far below autonomous replication threshold

ARC Evals tested four GPT-4- and Claude-based agents on 12 realistic autonomous replication and adaptation tasks. The agents completed only the easiest tasks and failed end-to-end persistent deployment and controlled phishing attempts, indicating that casual users of those versions were unlikely to create dangerous autonomous agents. OpenAI's GPT-4 system card independently documents ARC's predeployment evaluation role.

0 comments · 0 votesOpen discussion

Public discussion is readable by everyone. Sign in to comment, reply, or vote.

No comments yet. Start the discussion.

Send detailed or private feedback →
CURRENT ASSESSMENT · REVISION 1
AWAY FROM DOOM45confidence 87/100

Why it moved the index

Away-from-doom magnitude 45: the controlled evaluation directly demonstrated that then-frontier GPT-4- and Claude-based agents could not complete the multi-step resource acquisition, persistence, and phishing tasks needed for dangerous autonomous replication, while documenting that stronger scaffolding and fine-tuning could narrow the gap. Confidence 87: the dated primary report publishes the tasks, agent design, failures, oversight, and limitations, and OpenAI's GPT-4 system card independently confirms ARC's predeployment evaluation role.

Assessment history

  1. R1
    Away 45 · confidence 87

    Adds a previously unrepresented primary evaluation showing both practical autonomy limits and scaffolding sensitivity in 2023 frontier agents.

    24 Aug 2026

Share this page

DoomBench social sharing card for ARC Evals finds 2023 agents far below autonomous replication threshold.

  1. DoomBench assesses “ARC Evals finds 2023 agents far below autonomous replication threshold” as evidence moving away from doom, with magnitude 45 and confidence 87 out of 100 in the autonomy and agency category.

  2. The DoomBench assessment of “ARC Evals finds 2023 agents far below autonomous replication threshold” is based on reporting from METR (formerly ARC Evals) and records the editorial rationale, source quality, attribution, and revision...

  3. DoomBench summarizes “ARC Evals finds 2023 agents far below autonomous replication threshold” as follows: ARC Evals tested four GPT-4- and Claude-based agents on 12 realistic autonomous replication and adaptation tasks. The agents...

    https://www.doombench.com/news/arc-evals-finds-2023-agents-far-below-autonomous-replication-threshold-2023-07-31