mrkeyoor.com_
Thu 03 Sept 09:14 UTC
AI6 min read

Muse Spark 1.3's Best Benchmarks Use an Unreleased Mode

Meta's launch chart tests Muse Spark 1.3 with a pending max-reasoning mode. Its two API tiers also turn data use into a 21-fold output-price decision.

Muse Spark 1.3 had collected 529 Hacker News points and 357 comments in the brief's September 3 snapshot. The detail worth slowing down for is on Meta's own model page: every score in the new model's benchmark column uses max reasoning. Meta's release post says that mode is coming only after additional safety testing. Developers can use Muse Spark 1.3 today, though they cannot yet reproduce the exact configuration at the center of its launch comparison.

The pricing table contains a second consequential split. Meta charges $0.20 per million output tokens for the contributor variant, which it labels as used to improve its products, and $4.25 for the standard variant, which it labels as not used for that purpose. That 21.25-fold difference can outweigh a small benchmark lead for teams choosing a coding or computer-use model. The launch deserves attention. Its most favorable numbers need more context than the chart supplies at first glance.

The benchmark column points past launch day

Meta compares Muse Spark 1.3 with Muse Spark 1.2, GPT-5.6 Sol and Claude Opus 5 across 11 rows on its benchmark chart. The 1.3 column is labeled max; the older Muse model uses xhigh, while the two outside models also use max. Meta reports that Muse Spark 1.3 leads outright on both long-context retrieval bands, DeepSWE v1.1 and SWE-Atlas Codebase QnA. It ties GPT-5.6 Sol on Terminal-Bench 2.1 at 88.8.

The strongest margins are in long-context retrieval. On MRCR v2, Muse Spark 1.3 scores 98.5 in the 256K to 512K band and 98.1 from 512K to 1M. Meta lists GPT-5.6 Sol at 91.5 and 73.8, respectively, and Muse Spark 1.2 at 66.3 and 55.5. The same model page advertises a one-million-token context window, so those rows speak directly to a feature buyers can see in the product description.

Other rows on Meta's chart are much closer. Muse Spark 1.3 reaches 1,754 on GDPVal-AA v2, behind Opus 5 at 1,824 and ahead of GPT-5.6 Sol at 1,710. Its OSWorld 2.0 result is 66.9, compared with 68.3 for Opus 5. On DeepSearchQA, Meta reports 89.4 for Muse, below 93.0 for GPT-5.6 Sol and 90.4 for Opus 5. The chart presents a competitive model, not one that wins every kind of work.

Meta's evaluation methodology explains why the comparison should be read as a collection of results rather than one controlled tournament. The company ran Muse Spark 1.3 through its Model API. Depending on the benchmark, comparison numbers came from Meta's own runs, official leaderboards or model providers' self-reported results. Meta selects the highest comparable primary score it can obtain and says its third-party runs are best-effort tests that may not match provider-tuned performance.

There are useful controls. Coding environments have no external internet access, refusals and ungradable answers receive zero, and required tools are limited by benchmark. There are also exceptions. Muse Spark 1.2 used OSWorld release 06.24 while the other OSWorld results used 08.08. Meta's Agentic IF Index is an internal composite with no single task count, according to the methodology document. Outside teams cannot rerun that row from a fixed public set.

What developers can use today

Muse Spark 1.3 began rolling out on September 2 through Muse Code and Meta Model API. The announcement says previously available reasoning modes are live, while max will follow after safety testing. This makes the benchmark chart a view of the intended ceiling rather than a clean measurement of the launch-day service. Meta has not given a date for the mode.

The model is aimed at long-running agent tasks. Meta says it can track earlier results in one thread, deal with conflicting inputs, request help when stuck and retain detailed instructions across multiple steps. It is also trained to map a new prompt to the correct task when a user interrupts or steers an earlier request. Those are practical failure points in agent systems, but the release post describes Meta's testing rather than an independent field study.

For coding, Meta reports roughly 20 percent fewer tool calls and 25 percent fewer tokens than Muse Spark 1.2 in comparisons conducted by its engineers. The post does not publish a task count or variance for that claim. Token and tool-call reductions could lower latency and cost, yet the missing test detail makes them a starting point for workload trials, not a dependable budget forecast.

Muse Spark 1.3 also accepts images, video and documents. Meta says its visual reasoning can act through an execution environment, which is meant to support tasks such as turning a screenshot into working software. The model page links to computer-use and GitHub-agent cookbooks, and says an OpenAI SDK-compatible client can be pointed at Meta Model API. That lowers the work required for a trial, provided the API is available to the developer.

Meta claims better resistance to prompt injection and improved judgment around irreversible actions. It also says the model asks before consequential steps. The safety section provides no rates or attack-set results, while the delayed max mode shows that testing is still underway for the configuration used in the chart. Production teams should keep their own permission checks around write, purchase and deployment tools.

The cheap tier comes with a data decision

Meta lists two API variants with the same one-million-token context window. muse-spark-1.3-contributor, labeled as used to improve Meta's products, costs $0.10 per million input tokens, $0.002 for cached input and $0.20 for output. The standard muse-spark-1.3, labeled as not used for product improvement, costs $1.25, $0.15 and $4.25 for the same categories on the pricing table.

The standard tier is 12.5 times the input price, 75 times the cached-input price and 21.25 times the output price. A workload sending 100 million uncached input tokens and receiving 20 million output tokens would cost $14 on the contributor tier and $210 on the standard tier, before any other charges. That calculation follows Meta's published rates and excludes caching. The model a team selects is therefore only half of the cost decision.

The short labels do not answer every governance question. The model page does not explain inside its pricing table which request fields may be reused, how long they are retained or whether later deletion is possible. A hobby project built from public code may accept the contributor trade. A company sending private repositories, customer records or unreleased designs needs the applicable API terms before treating the cheaper rate as comparable.

How to test the claim that ships

A useful evaluation should start with a reasoning mode that developers can call now. Run the same repository tasks against both Muse tiers and record completion, tool calls, tokens, wall time and human corrections. Meta's methodology shows that harness choice changes what a model is allowed to do; preserving the tools and grading rules is necessary if a team wants a fair comparison with its current provider.

The public numbers identify good stress tests. DeepSWE covers 113 tasks across 91 repositories in TypeScript, Go, Python, JavaScript and Rust. OSWorld 2.0 uses 108 workflows on an Ubuntu desktop VM, while AutomationBench has 600 workflows graded by end-state assertions rather than a language-model judge. Those details come from Meta's evaluation report, and they matter more than a single blended claim about agent quality.

Meta says its roadmap includes larger models and an open-weights release for Muse Spark, but the announcement gives no date, parameter count or license for those weights. The next evidence to watch is narrower: when max becomes available, whether outside evaluators can reproduce the chart with that public endpoint, and whether Meta links precise data-use terms beside the contributor price. Until then, Muse Spark 1.3 is available to test, while its most persuasive launch configuration remains a promised one.

We reviewed this

  1. computer — our honest review
  2. OSWorld — our honest review
  3. terminal — our honest review

Sources

  1. Muse Spark 1.3 | Meta
  2. Introducing Muse Spark 1.3
  3. Muse Spark 1.3 Evaluation Methodology