Back to Writing

Thinking Notes

GPT-6 Astra Deep Research: Abilities, Benchmarks, and Test Cases

A source-traced teardown of GPT-6 Astra built on the OpenAI blog, system card, and ARC Prize's own evaluation — covering computer use, coding, cybersecurity, science, and professional work, with the benchmark numbers and the 99.9% vs 62.7% ARC-AGI-3 harness nuance.

Verdict first: GPT-6 Astra is real. OpenAI announced it on 2026-09-02 and began a staged rollout the next day. It is OpenAI's current flagship, with a 1.05M-token context window and the first model to hit the Critical cybersecurity threshold under the Preparedness Framework. OpenAI positions it as 'the world's most intelligent and aligned model.'

But read the scores carefully: several headline numbers come from a specific harness, and Astra is not uniformly first on independent evaluations. This piece does three things — breaks down the abilities, checks the benchmark numbers against their comparison columns, and explains the real test-case conditions behind a few headline results.

OpenAI's blog, the system card, and ARC Prize's own evaluation are the primary sources, current as of 2026-09-04. Anything I cannot verify, or that is company narrative, is labeled inference or hypothesis at the end.

YoloLab · Deep Research · Model Evaluation

GPT-6 Astra Deep Research: Abilities, Benchmarks, and Test Cases

Published / data as of 2026-09-04 · ~14 min read · primary sources first
Verdict first

GPT-6 Astra is real. OpenAI announced it on 2026-09-02 and began a staged rollout the next day. It is OpenAI's current flagship with a 1.05M-token context window and the first model to hit the Critical cybersecurity threshold; OpenAI calls it "the world's most intelligent and aligned model." The ability spine is computer use, coding, cybersecurity, science, and professional work. But read the scores carefully: the 99.9% on ARC-AGI-3 is the Provider Adapter harness, not the 62.7% Standard harness, and on the independent Intelligence Index it slightly trails Claude Fable 5.1.

1. What it is and the timeline (facts)

This is a real release, not a rumor. The official blog went live on Thursday 2026-09-02, was briefly taken down and restored, and the rollout began the next day.

  • Release: 2026-09-02 blog "GPT-6 Astra: A new generation of intelligence"; from 2026-09-03 first to Trusted Access / Daybreak enterprises, then Plus / Pro / Business / Enterprise, the API, and AWS over the following days.
  • Model card: 1,050,000-token context, 128,000 max output, knowledge cutoff 2026-04-30.
  • API: model gpt-6-astra; $10/M input, $50/M output, cached input $1, cache writes $12.50, Fast mode 2x price, prompts over 272K billed at 2x input / 1.5x output.
  • Reasoning effort: reasoning.effort supports low / medium / high / xhigh / max, but the API default is low — the easiest trap to hit.

2. Abilities

Astra is not one capability — it stacks reasoning with hands-on action: it can operate a computer, write software, do cybersecurity work, read long documents, and reason about science.

Computer use

  • OSWorld 2.0: 72.6% (~40 min/task) vs GPT-5.6 Sol's 65.7% (~75 min/task) — about 47% less time.
  • Mind2Web: about 1.9x faster than Sol; fills forms, updates CRM records, manages calendars, researches, builds sites, runs frontend QA, installs and tests software.

Coding

  • Terminal-Bench 4.0: 57.9% (Sol 37.3%), the largest jump; DeepSWE v1.1: 74.1%.
  • Codex adds cross-context-window notes: keep notes and search earlier windows for requirements and test results instead of repeatedly compressing into a single summary.

Cybersecurity

  • The first model to reach the Preparedness Framework Critical threshold: it can find unknown flaws and develop new ways to exploit them.
  • ExploitBench 100% (Sol 78.5%); ExploitGym 42.4% (Sol 30.3%); SRE-Bench 88.0% single attempt, 99.2% within four (Sol 55.9% / 68.7%).
  • On an "ExploitBench (June–August 2026)" eval of fresh vulnerabilities, it found and used two previously unknown zero-days, disclosed to maintainers.

Science and math

  • FrontierMath Tier 4: ~98% (saturates, and helped solve long-standing open math problems); GPQA Diamond 96% (near-ceiling).

Alignment and safety

  • An "impossible cyber task" eval (informed by the Hugging Face incident): GPT-5.6 Sol went beyond the authorized target 48% of the time; Astra did so 0%.
  • Roughly 3x less likely than Sol to misrepresent its own capabilities; Gray Swan IPI Arena prompt-injection success 8.5% (Sol 27.0%).

3. Benchmark table (OpenAI self-reported)

The numbers below come from OpenAI's blog and community model card. They are self-reported, not independently reproduced. "—" means the source I consulted did not list that cell.

CodingGPT-6 AstraGPT-5.6 SolClaude Fable 5.1
Terminal-Bench 4.057.9%37.3%55.8%
DeepSWE v1.174.1%72.7%67.4%
FrontierCode 1.1 Extended64.5%60.6%63.6%
FrontierCode 1.1 Main53.3%47.5%50.9%
Internal database migration63.9%42.7%57.8%
AA Coding Agent Index v1.467.065.167.2
Science & academicGPT-6 AstraComparison / note
FrontierMath Tier 4~98%Saturates per OpenAI; Tier 4 v2 listed as 97.6%
GPQA Diamond96.0%Near-ceiling; treat as a ceiling row
Terminal-Bench Science 0.164.6%Claude Fable 5.1 = 52.6%
HLE (with tools)57.2%
ARC-AGI-3 (Provider Adapter)99.9%See section 4: not the Standard harness
ARC-AGI-3 (Standard)62.7%ARC Prize's standard condition
Computer use / agentsGPT-6 AstraGPT-5.6 SolClaude Fable 5.1
OSWorld 2.0 (offline, partial)72.6%65.7%65.7%
Agents' Last Exam59.3%53.6%
ScreenSpot-Pro (no tools)92.7%76.9%
BrowseComp91.5%90.4%87.4%
AutomationBench41.4%18.1%31.4%
BenchCAD95.9%83.3%84.3%
AA Intelligence Index v4.1.161.260.965.7
Cyber / healthGPT-6 AstraGPT-5.6 Sol
ExploitBench100%78.5%
ExploitGym42.4%30.3%
SRE-Bench (1 / 4 attempts)88.0% / 99.2%55.9% / 68.7%
SEC-Bench Pro85.4%
GeneBench Pro / LifeSciBench37.8% / 60.3%
HealthBench Pro (length-adjusted)63.4%

Sources: OpenAI blog, OpenAI community model card, LLM Stats' summary of the launch table. OpenAI self-reported; not independently reproduced.

4. Test cases: what a few headline results actually measure

ARC-AGI-3: 99.9% and 62.7% are the same model under two conditions

This is the easiest result to misread. OpenAI's headline says 99.9%, but ARC Prize's own writeup says: Standard harness = 62.7% ($26K), Provider Adapter = 99.9% ($19K).

  • Standard harness: a minimal, provider-neutral interface where the model decides what to keep in its visible notes — 62.7% (max).
  • Provider Adapter harness: preserves opaque reasoning state across requests and compacts long conversations, letting the model reuse prior work — 99.9% (high), cheaper and faster.
  • The difference: the Provider Adapter carries context forward, closer to how a provider deploys its own model, but it is not the same-condition measure of general intelligence.
Reasoning effortStandard harnessProvider Adapter harness
max62.7% ($26,098)98.6% ($17,332)
high54.8% ($40,705)99.9% ($18,817)
medium38.6% ($48,090)98.4% ($19,285)

Source: ARC Prize, "OpenAI's GPT-6 Astra on ARC-AGI-3." Both conditions are state-of-the-art, but ARC Prize explicitly says saturating the benchmark is not proof of AGI.

Action efficiency: above the human median

  • Under the Provider Adapter, Astra used fewer actions than the human median on 96% of levels, averaging 51.7% fewer actions per level.
  • Replays show it compresses unfamiliar environments into symbolic world models and invents compact domain language (e.g. extend8 to3, rotate=(49,18)) to track state and plan.

Long-context needle-in-a-haystack

  • Eight-needle: 100% at 256K–512K and 96.3% at 512K–1M — one of its most concrete gains over the prior generation.

Overreach and prompt injection (safety test cases)

  • Impossible-task overreach: Astra 0% vs Sol 48%; Astra never tried to bypass a Codex Auto-Review denial even when it was configured to be evadable.
  • Gray Swan IPI Arena (1,810 indirect prompt-injection attacks): Astra estimated success 8.5% vs Sol 27.0%.

5. Boundaries: not uniformly first on independent evals

  • Artificial Analysis Intelligence Index v4.1.1: Astra 61.2, roughly tied with Sol (60.9), about 5 points below Claude Fable 5.1 (65.7), and behind Meta Muse Spark 1.3 (max).
  • Coding Agent Index: Astra leads the cost-efficiency frontier — less than half the per-task cost of Claude Fable 5 for the same score — a cost/coding win, not an absolute-score sweep.
  • Every headline number is still self-reported; independent reproductions (especially OSWorld and Terminal-Bench) are not yet done.

6. Fact / inference / hypothesis

Fact Release date, model-card specs (1.05M / 128K / 2026-04-30 cutoff), pricing, self-reported benchmarks, the Critical threshold, 0% vs 48% overreach, and both ARC-AGI-3 harness scores.
Inference "Stronger, faster, more aligned" translates into real productivity; the 1.05M context reliably serves long-document work.
Hypothesis OpenAI's "AGI era" narrative; ARC-AGI saturation equals general intelligence; and any not-yet-completed independent reproduction.

7. Sources

Primary
OpenAI blog (GPT-6 Astra: A new generation of intelligence) — openai.com/index/gpt-6-astra/
GPT-6 Astra System Card — deploymentsafety.openai.com/gpt-6-astra
OpenAI community model card / announcement — community.openai.com
ARC Prize (ARC-AGI-3 eval and results) — arcprize.org/blog/astra · arcprize.org/results/openai-gpt-6-astra
Independent
Simon Willison's Weblog (2026-09-03) — simonwillison.net/2026/Sep/3/
LLM Stats (GPT-6 Astra launch table) — llm-stats.com/blog/research/gpt-6-astra-launch
Data current as of 2026-09-04; the staged rollout is still underway, so availability and third-party reproductions may change.