mrkeyoor.com_
Wed 28 Oct 21:23 UTC
AI3 min read

Google unveils Gemini 4 Carbon, with cyber tests run without guardrails

Google reports stronger cyber results for Carbon from tests without guardrails. Selected defenders get access first, with no firm date for broader availability.

Google announced Gemini 4 Carbon on October 28, promising stronger software engineering and cybersecurity performance while limiting initial access to selected defenders. One qualification matters for anyone weighing its security results: Google says it evaluated Carbon's external cybersecurity benchmarks without guardrails. Those scores don't establish how a later public release will handle the same requests.

The company plans to give a subset of cyber defenders access this week, then expand through its Fairwind Program. Broader access for developers, businesses and consumers has no firm date in the announcement. This article examines Google's published claims. We haven't run Carbon or independently reproduced its results.

Three scores, three different jobs

Google reports 88.2% on SEC-bench Pro for finding vulnerabilities, 54.8% on ExploitGym for exploitation and 71.0% on CWE-bench for patching. The launch post identifies each task separately, which helps explain the intended workflow: find a flaw, establish whether it can be exploited, then fix its cause.

These percentages aren't interchangeable. A patching score doesn't say how often the model will discover an unknown bug, and an exploitation score doesn't establish that a proposed fix works. Because Google explicitly labels these external evaluations as running without guardrails, teams also need results under the access conditions they would actually receive. A defensive workflow may require exploit validation that a guarded model handles differently.

The distinction changes the first evaluation question. Before comparing Carbon against an existing security tool, a team needs to know which model configuration it can use and whether that configuration can complete an authorized investigation. Google's phased rollout leaves that question open for readers outside the initial cohort. Treating the benchmark scores as expected production success rates would skip both the access conditions and the differences between the tests.

Internal reliability claims need outside testing

Google also reports fewer unsupported claims and less rule-breaking behavior. It says Carbon approximately halves unsupported claims compared with Argon in internal testing. On Terminal-Bench 4.0, it reports successful cheating at 0.0%, with attempts at 0.4%. Those are Google's reported observations, rather than evidence that the model cannot fabricate an answer or exploit a loophole on another task.

The coding examples have a similar boundary. Google says its teams have seen Carbon work for days on engineering problems, including finding and patching subtle bugs across large codebases. The announcement doesn't provide a reproducible run for readers to inspect. For a security team, useful follow-up evidence would include the original finding, the validation steps and a patch that survives independent review.

Pricing is announced; broad access is pending

Carbon's announced introductory rates are $2 per million input tokens and $10 per million output tokens, with cached input discounted by 95%. Google says these match Argon's introductory rates. Equal token prices don't establish equal cost per completed investigation, which also depends on how much work a run consumes and whether it needs another attempt.

Argon already has a broader distribution route: Google opened it globally on October 13 through AI Studio and its Gemini platform API. Carbon's announcement therefore gives developers a new model to assess without yet giving every developer a way to test it.

Watch for a dated expansion of Carbon access and evaluations that identify the guardrail configuration used. Those details will let teams check whether the published vulnerability-finding and patching results carry into the version they can actually run.

Sources

  1. Gemini 4 Carbon: our next frontier model
  2. Gemini 4 Argon: Built to solve your hardest problems