The Funded Agent Index

Finding out if agents are better at accomplishing tasks if they can pay for things

TL;DR

Across seven AI models, giving agents a USDC budget and paid-service tools raised average task completion by 30.8 percentage points, for an average increase of $0.046 in model-plus-service cost per task. Among runs where the agent said it was done, the share passing every check rose 10.7 percentage points. Quality measures cover different sets of runs, and results were mixed: claims backed by a cited source slipped from 99.1% to 97.9% on eligible tasks.

Context

An agent can draft an email without sending it or describe a song without producing audio. A budget gives it an option to keep going when they stop knowing what to do or being able to do it without help: acquire the capability it is missing. Anecdotally, we’ve seen that giving agents the ability to pay for things produces more complete and higher quality work, but we wanted to test that observation programmatically. That’s the impetus behind this study: to see if giving your agent money gives it superpowers.

The questions

  1. Can an agent with USDC buy capabilities it is missing and complete tasks at a higher rate?
  2. Can an agent with USDC buy capabilities it is missing to complete tasks with higher quality?
  3. What is the incremental spend required to increase agentic task completion and quality?

Does an agent with USDC and the ability to buy services improve task completion? Yes, +30.8 pp average

Without: 53.7% → with: 84.5%, averaged equally across seven models and all 21 tasks. This is the share of checks passed, not the share of whole jobs finished.

Without USDCWith USDC
View run data for this test

Task completion

Completion is the share of predefined checks passed, including partial credit. It is not the share of whole jobs finished. Intervals are the published 95% ranges for the change.

These are the published run aggregates, not individual run logs. The source snapshot does not include per-run records, prompts, outputs or traces. No individual run records have been reconstructed.

Scope & denominators
Seven models, 21 tasks, five repeats per condition; 1,470 runs in total. Completion and cost give tasks and models equal weight.
Units
Scores are fractions from 0 to 1. Changes ending in _pp are percentage points; counts are eligible runs or claims as defined above.
Published snapshot
2026-11-13 16:59 UTC
Task completion: model results
modelwithout usdcwith usdcchange ppchange low ppchange high ppwithout runswith runs
Claude Opus 5.50.6080350.95499734.6961914.80691854.585463105105
Claude Sonnet 5.50.5927660.93917434.64085716.40448152.877233105105
Claude Haiku 5.50.4476410.6789623.1319058.56752837.696282105105
Grok 4.60.5317350.86928633.75504815.40654952.103547105105
GPT-6 Luna0.4837810.72511224.1331432.714145.552186105105
GPT-6.1 Sol0.5600580.94759738.75390514.22685263.280957105105
Gemini 3 Flash0.5366250.79848726.186195.89336446.479017105105
Task-by-model results (147 cells)

Five runs per condition in each cell.

Task completion: task-by-model results
taskmodelwithout usdcwith usdcchange ppchange low ppchange high ppwithout runswith runs
Email tomorrow's NYC forecastClaude Opus 5.50110010010055
Email tomorrow's NYC forecastClaude Sonnet 5.50110010010055
Email tomorrow's NYC forecastClaude Haiku 5.500.88024.4810055
Email tomorrow's NYC forecastGrok 4.600.88024.4810055
Email tomorrow's NYC forecastGPT-6 Luna00.9333493.33474.82918410055
Email tomorrow's NYC forecastGPT-6.1 Sol0110010010055
Email tomorrow's NYC forecastGemini 3 Flash0110010010055
Photo of a forest signClaude Opus 5.50.218024.4810055
Photo of a forest signClaude Sonnet 5.50.20.860-5.22352910055
Photo of a forest signClaude Haiku 5.50.20.420-54.78786794.78786755
Photo of a forest signGrok 4.60110010010055
Photo of a forest signGPT-6 Luna0110010010055
Photo of a forest signGPT-6.1 Sol0110010010055
Photo of a forest signGemini 3 Flash0110010010055
Hat-trick song for the drive homeClaude Opus 5.50110010010055
Hat-trick song for the drive homeClaude Sonnet 5.50110010010055
Hat-trick song for the drive homeClaude Haiku 5.50110010010055
Hat-trick song for the drive homeGrok 4.60110010010055
Hat-trick song for the drive homeGPT-6 Luna00.88024.4810055
Hat-trick song for the drive homeGPT-6.1 Sol0110010010055
Hat-trick song for the drive homeGemini 3 Flash0110010010055
NASA's latest TikTok videos and their viewsClaude Opus 5.500.9666696.66687.41081610055
NASA's latest TikTok videos and their viewsClaude Sonnet 5.500.440-27.99783510055
NASA's latest TikTok videos and their viewsClaude Haiku 5.500.220-35.5275.5255
NASA's latest TikTok videos and their viewsGrok 4.600.88024.4810055
NASA's latest TikTok videos and their viewsGPT-6 Luna00.440-27.99783510055
NASA's latest TikTok videos and their viewsGPT-6.1 Sol0110010010055
NASA's latest TikTok videos and their viewsGemini 3 Flash0.10.430-38.02226698.02226655
Follower counts for 30 space accounts on XClaude Opus 5.50.4160-7.99783510055
Follower counts for 30 space accounts on XClaude Sonnet 5.50.960.92-4-18.95757310.95757355
Follower counts for 30 space accounts on XClaude Haiku 5.50.086660.871.33414.24763710055
Follower counts for 30 space accounts on XGrok 4.60.6140-1.69158181.69158155
Follower counts for 30 space accounts on XGPT-6 Luna0.30666169.33417.12131710055
Follower counts for 30 space accounts on XGPT-6.1 Sol0.13332186.66871.46491110055
Follower counts for 30 space accounts on XGemini 3 Flash0.380.622-49.27657393.27657355
NASA's latest posts on X, with live countsClaude Opus 5.50.218024.4810055
NASA's latest posts on X, with live countsClaude Sonnet 5.500.9333493.33474.82918410055
NASA's latest posts on X, with live countsClaude Haiku 5.50000055
NASA's latest posts on X, with live countsGrok 4.60.20-20-75.5235.5255
NASA's latest posts on X, with live countsGPT-6 Luna00.9333493.33474.82918410055
NASA's latest posts on X, with live countsGPT-6.1 Sol0110010010055
NASA's latest posts on X, with live countsGemini 3 Flash0.20.4666826.668-40.22485993.56085955
AA 100 flight statusClaude Opus 5.510.8-20-75.5235.5255
AA 100 flight statusClaude Sonnet 5.50.72128-26.39826882.39826855
AA 100 flight statusClaude Haiku 5.50.520.68-70.58066286.58066255
AA 100 flight statusGrok 4.60.40.840-34.78786710055
AA 100 flight statusGPT-6 Luna0.720.88-56.50859972.50859955
AA 100 flight statusGPT-6.1 Sol10.2-80-100-24.4855
AA 100 flight statusGemini 3 Flash10.8-20-75.5235.5255
Who [company name redacted] is hiring right now*Claude Opus 5.510.94286-5.714-21.57606410.14806455
Who [company name redacted] is hiring right now*Claude Sonnet 5.50.971420.971420-9.3204429.32044255
Who [company name redacted] is hiring right now*Claude Haiku 5.50.657140.62856-2.858-14.2731648.55716455
Who [company name redacted] is hiring right now*Grok 4.60.9714212.858-5.07580810.79180855
Who [company name redacted] is hiring right now*GPT-6 Luna0.828560.88575.714-12.16282623.59082655
Who [company name redacted] is hiring right now*GPT-6.1 Sol1100055
Who [company name redacted] is hiring right now*Gemini 3 Flash0.942860.6857-25.716-77.4212825.9892855
5 new house listings in Austin 78745Claude Opus 5.50.8120202055
5 new house listings in Austin 78745Claude Sonnet 5.50.8120202055
5 new house listings in Austin 78745Claude Haiku 5.50.41601.77013310055
5 new house listings in Austin 78745Grok 4.60.8120202055
5 new house listings in Austin 78745GPT-6 Luna0.9218-5.59956721.59956755
5 new house listings in Austin 78745GPT-6.1 Sol0.880.968-6.95757322.95757355
5 new house listings in Austin 78745Gemini 3 Flash0.840.928-20.16389436.16389455
Fintech Series A market mapClaude Opus 5.50.89998110.002-1.33323921.33723955
Fintech Series A market mapClaude Sonnet 5.50.833320.9333210-5.76686425.76686455
Fintech Series A market mapClaude Haiku 5.50.666640.8666620.002-25.74735465.75135455
Fintech Series A market mapGrok 4.60.833320.866663.334-22.81158429.47958455
Fintech Series A market mapGPT-6 Luna0.633320.6-3.332-75.13768168.47368155
Fintech Series A market mapGPT-6.1 Sol0.866640.899983.334-9.13313715.80113755
Fintech Series A market mapGemini 3 Flash0.933320.73332-20-59.26020419.26020455
Meeting prep: CEO briefing*Claude Opus 5.50.9110-17.7637.7655
Meeting prep: CEO briefing*Claude Sonnet 5.50.9110-17.7637.7655
Meeting prep: CEO briefing*Claude Haiku 5.50.50.6212-14.91431938.91431955
Meeting prep: CEO briefing*Grok 4.60.90.988-14.20830.20855
Meeting prep: CEO briefing*GPT-6 Luna0.340.736-5.37060477.37060455
Meeting prep: CEO briefing*GPT-6.1 Sol1100055
Meeting prep: CEO briefing*Gemini 3 Flash0.60.600055
Research 10 prospects, rank them, email the top 3Claude Opus 5.50.868760.81876-5-8.247181-1.75281955
Research 10 prospects, rank them, email the top 3Claude Sonnet 5.50.750020.9312618.1240.89930135.34869955
Research 10 prospects, rank them, email the top 3Claude Haiku 5.50.850020.856280.626-2.79164.043655
Research 10 prospects, rank them, email the top 3Grok 4.60.675040.87519.996-26.9317766.9237755
Research 10 prospects, rank them, email the top 3GPT-6 Luna0.837520.874983.746-1.1549618.64696155
Research 10 prospects, rank them, email the top 3GPT-6.1 Sol0.681260.9062222.496-24.97205369.96405355
Research 10 prospects, rank them, email the top 3Gemini 3 Flash0.756260.8625210.6263.36996417.88203655
What parents on Reddit say about a car seatClaude Opus 5.500.9666696.66687.41081610055
What parents on Reddit say about a car seatClaude Sonnet 5.500.8333283.33268.70265897.96134255
What parents on Reddit say about a car seatClaude Haiku 5.50000055
What parents on Reddit say about a car seatGrok 4.600.7333473.33449.74380696.92419455
What parents on Reddit say about a car seatGPT-6 Luna0000055
What parents on Reddit say about a car seatGPT-6.1 Sol00.9333493.33474.82918410055
What parents on Reddit say about a car seatGemini 3 Flash00.4408.61905671.38094455
OFAC sanctions screen for 12 counterpartiesClaude Opus 5.51100055
OFAC sanctions screen for 12 counterpartiesClaude Sonnet 5.51100055
OFAC sanctions screen for 12 counterpartiesClaude Haiku 5.51100055
OFAC sanctions screen for 12 counterpartiesGrok 4.61100055
OFAC sanctions screen for 12 counterpartiesGPT-6 Luna1100055
OFAC sanctions screen for 12 counterpartiesGPT-6.1 Sol1100055
OFAC sanctions screen for 12 counterpartiesGemini 3 Flash0.9833411.666-2.9588166.29081655
Follower counts for 30 X accounts, from a CRM export with user IDsClaude Opus 5.50.218024.4810055
Follower counts for 30 X accounts, from a CRM export with user IDsClaude Sonnet 5.50.31332168.66832.04948410055
Follower counts for 30 X accounts, from a CRM export with user IDsClaude Haiku 5.500.6466664.6664.15610510055
Follower counts for 30 X accounts, from a CRM export with user IDsGrok 4.60.38666161.334-4.46127310055
Follower counts for 30 X accounts, from a CRM export with user IDsGPT-6 Luna0.373340.2-17.334-76.92526342.25726355
Follower counts for 30 X accounts, from a CRM export with user IDsGPT-6.1 Sol0110010010055
Follower counts for 30 X accounts, from a CRM export with user IDsGemini 3 Flash0.13334186.66649.65081610055
Five available .com namesClaude Opus 5.51100055
Five available .com namesClaude Sonnet 5.51100055
Five available .com namesClaude Haiku 5.50.520.8432-33.19855197.19855155
Five available .com namesGrok 4.61100055
Five available .com namesGPT-6 Luna10.8-20-75.5235.5255
Five available .com namesGPT-6.1 Sol1100055
Five available .com namesGemini 3 Flash0.60.60-79.88218379.88218355
Next Friday's Apple closing priceClaude Opus 5.51100055
Next Friday's Apple closing priceClaude Sonnet 5.51100055
Next Friday's Apple closing priceClaude Haiku 5.51100055
Next Friday's Apple closing priceGrok 4.61100055
Next Friday's Apple closing priceGPT-6 Luna1100055
Next Friday's Apple closing priceGPT-6.1 Sol1100055
Next Friday's Apple closing priceGemini 3 Flash1100055
Last closing prices for 5 US stocksClaude Opus 5.50.60.56-4-15.1047.10455
Last closing prices for 5 US stocksClaude Sonnet 5.51100055
Last closing prices for 5 US stocksClaude Haiku 5.51100055
Last closing prices for 5 US stocksGrok 4.60.40.400055
Last closing prices for 5 US stocksGPT-6 Luna0.80.2-60-1005.22352955
Last closing prices for 5 US stocksGPT-6.1 Sol1100055
Last closing prices for 5 US stocksGemini 3 Flash0.8120-35.5275.5255
Tomorrow's weather in ChicagoClaude Opus 5.51100055
Tomorrow's weather in ChicagoClaude Sonnet 5.51100055
Tomorrow's weather in ChicagoClaude Haiku 5.51100055
Tomorrow's weather in ChicagoGrok 4.61100055
Tomorrow's weather in ChicagoGPT-6 Luna0.40.3-10-84.78786764.78786755
Tomorrow's weather in ChicagoGPT-6.1 Sol1100055
Tomorrow's weather in ChicagoGemini 3 Flash10.7-30-85.5225.5255
Apple.com visits yesterdayClaude Opus 5.51100055
Apple.com visits yesterdayClaude Sonnet 5.51100055
Apple.com visits yesterdayClaude Haiku 5.51100055
Apple.com visits yesterdayGrok 4.61100055
Apple.com visits yesterdayGPT-6 Luna10.8-20-53.99891813.99891855
Apple.com visits yesterdayGPT-6.1 Sol1100055
Apple.com visits yesterdayGemini 3 Flash1100055
Primes with digit sum 40Claude Opus 5.50.7130-25.5285.5255
Primes with digit sum 40Claude Sonnet 5.50110010010055
Primes with digit sum 40Claude Haiku 5.50000055
Primes with digit sum 40Grok 4.60110010010055
Primes with digit sum 40GPT-6 Luna0110010010055
Primes with digit sum 40GPT-6.1 Sol0.218024.4810055
Primes with digit sum 40Gemini 3 Flash0110010010055

* some tasks have been slightly anonymized/modified to maintain privacy.

Did quality of finished work change?

+10.7 pp pooled
Of the runs where the agent said it was done, the share that were right: every required item checked out.
Without USDCWith USDC
Claims backed by source fell from 99.1% without USDC to 97.9% with USDC, a pooled decline of 1.2 percentage points among 536 and 559 graded claims, respectively. A different model grades each claim without seeing the condition. This measure covers market map, meeting prep, prospects and eligible Reddit runs only. The three quality measures use different denominators and do not establish a general quality improvement or statistical significance.
View run data for this test

Downloads for all three quality measures

Quality of finished work: Right when done

Share of eligible runs in which every required item checked out, among runs where the agent said it was done.

These are the published run aggregates, not individual run logs. The source snapshot does not include per-run records, prompts, outputs or traces. No individual run records have been reconstructed.

Scope & denominators
Runs that said done. Pooled counts: 317 without USDC; 573 with USDC. Runs that gave up or exhausted their budget are excluded.
Units
Scores are fractions from 0 to 1. Changes ending in _pp are percentage points; counts are eligible runs or claims as defined above.
Published snapshot
2026-11-13 16:59 UTC
Quality of finished work: Right when done: model results
modelwithout usdcwith usdcchange ppwithout eligible countwith eligible count
Claude Opus 5.50.750.85263210.2631585295
Claude Sonnet 5.50.7708330.8444447.3611114890
Claude Haiku 5.50.6666670.7288146.2146893059
Grok 4.60.540.7790723.9069775086
GPT-6 Luna0.5945950.73972614.5131433773
GPT-6.1 Sol0.8478260.844444-0.3381644690
Gemini 3 Flash0.5925930.68759.4907415480
All models (pooled)0.6813880.78883110.74427317573

The published snapshot provides this measure at model level; it does not include a task-by-task breakdown.

How much does it cost for improved rates of task completion? +$0.046 / task on average

Average model-plus-service cost rose 45.6%. Includes model inference and agent purchases; excludes checker and web search costs.

View run data for this test

Extra cost per task

Mean cost includes model inference and services purchased by the agent, in USD per task. Checker purchases and web search are excluded. Completion is the share of predefined checks passed. Cost-change intervals are the published 95% ranges.

These are the published run aggregates, not individual run logs. The source snapshot does not include per-run records, prompts, outputs or traces. No individual run records have been reconstructed.

Scope & denominators
Seven models, 21 tasks, five repeats per condition; 1,470 runs in total. Completion and cost give tasks and models equal weight.
Units
Costs are USD. Completion scores are fractions from 0 to 1; changes ending in _pp are percentage points.
Published snapshot
2026-11-13 16:59 UTC
Extra cost per task: model results
modelwithout completionwith completioncompletion change ppwithout cost usdwith cost usdextra cost usdextra cost low usdextra cost high usdwith agent purchases usdwithout runswith runs
Claude Opus 5.50.6080350.95499734.696190.1673930.2282180.060825-0.015270.1369210.0652105105
Claude Sonnet 5.50.5927660.93917434.6408570.0900610.1474640.0574030.0069210.1078840.057281105105
Claude Haiku 5.50.4476410.6789623.1319050.0047910.0373030.0325130.0072650.057760.032281105105
Grok 4.60.5317350.86928633.7550480.241630.22157-0.02006-0.1217570.0816370.064721105105
GPT-6 Luna0.4837810.72511224.1331430.0073810.0677090.0603290.0179830.1026740.061605105105
GPT-6.1 Sol0.5600580.94759738.7539050.053290.1452110.0919220.0381430.1457010.085471105105
Gemini 3 Flash0.5366250.79848726.186190.1435580.183450.039893-0.0462520.1260370.074772105105
Task-by-model results (147 cells)

Five runs per condition in each cell.

Extra cost per task: task-by-model results
taskmodelwithout completionwith completioncompletion change ppwithout cost usdwith cost usdextra cost usdwith agent purchases usdwithout runswith runs
Email tomorrow's NYC forecastClaude Opus 5.5011000.0555560.1353640.0798080.0255
Email tomorrow's NYC forecastClaude Sonnet 5.5011000.0212870.0568370.035550.0255
Email tomorrow's NYC forecastClaude Haiku 5.500.8800.0010320.0224180.0213860.0255
Email tomorrow's NYC forecastGrok 4.600.8800.0231730.104580.0814070.01655
Email tomorrow's NYC forecastGPT-6 Luna00.9333493.3340.001180.022180.0210.0255
Email tomorrow's NYC forecastGPT-6.1 Sol011000.0125570.0484320.0358760.0255
Email tomorrow's NYC forecastGemini 3 Flash011000.041550.0521890.0106390.0255
Photo of a forest signClaude Opus 5.50.21800.0217370.1043820.0826450.061555
Photo of a forest signClaude Sonnet 5.50.20.8600.0101290.1575880.1474590.119255
Photo of a forest signClaude Haiku 5.50.20.4200.000570.1619290.1613590.159455
Photo of a forest signGrok 4.6011000.003430.143840.140410.048455
Photo of a forest signGPT-6 Luna011000.0001290.0978680.0977390.09506155
Photo of a forest signGPT-6.1 Sol011000.0028630.1581860.1553230.11294155
Photo of a forest signGemini 3 Flash011000.0057110.1652090.1594970.1278955
Hat-trick song for the drive homeClaude Opus 5.5011000.0227630.1988880.1761250.159555
Hat-trick song for the drive homeClaude Sonnet 5.5011000.0078790.1789990.171120.159555
Hat-trick song for the drive homeClaude Haiku 5.5011000.0003880.16080.1604120.159555
Hat-trick song for the drive homeGrok 4.6011000.005630.2926740.2870430.183555
Hat-trick song for the drive homeGPT-6 Luna00.8800.0002330.1795380.1793050.17845255
Hat-trick song for the drive homeGPT-6.1 Sol011000.0050760.1746450.1695690.159555
Hat-trick song for the drive homeGemini 3 Flash011000.0085010.1686920.1601910.159555
NASA's latest TikTok videos and their viewsClaude Opus 5.500.9666696.6660.0791910.4303930.3512020.06455
NASA's latest TikTok videos and their viewsClaude Sonnet 5.500.4400.0341950.176470.1422750.0455
NASA's latest TikTok videos and their viewsClaude Haiku 5.500.2200.0012790.0082880.0070090.00455
NASA's latest TikTok videos and their viewsGrok 4.600.8800.300780.504520.2037410.05655
NASA's latest TikTok videos and their viewsGPT-6 Luna00.4400.00180.085010.0832090.0855
NASA's latest TikTok videos and their viewsGPT-6.1 Sol011000.0091350.2490770.2399420.13655
NASA's latest TikTok videos and their viewsGemini 3 Flash0.10.4300.1549160.2402760.085360.09655
Follower counts for 30 space accounts on XClaude Opus 5.50.41600.1608870.3664770.2055910.1555
Follower counts for 30 space accounts on XClaude Sonnet 5.50.960.92-40.2418730.318340.0764660.09155
Follower counts for 30 space accounts on XClaude Haiku 5.50.086660.871.3340.0059840.1285920.1226080.12155
Follower counts for 30 space accounts on XGrok 4.60.61400.4925520.279702-0.212850.1555
Follower counts for 30 space accounts on XGPT-6 Luna0.30666169.3340.0070880.1556080.148520.15155
Follower counts for 30 space accounts on XGPT-6.1 Sol0.13332186.6680.0584080.2244590.1660510.1555
Follower counts for 30 space accounts on XGemini 3 Flash0.380.6220.3501060.158436-0.191670.05755
NASA's latest posts on X, with live countsClaude Opus 5.50.21800.0900980.244630.1545320.018455
NASA's latest posts on X, with live countsClaude Sonnet 5.500.9333493.3340.0149160.1778870.1629710.02455
NASA's latest posts on X, with live countsClaude Haiku 5.50000.0023820.0826640.0802820.0855
NASA's latest posts on X, with live countsGrok 4.60.20-200.1214070.100542-0.0208650.0155
NASA's latest posts on X, with live countsGPT-6 Luna00.9333493.3340.0054960.0126440.0071480.0155
NASA's latest posts on X, with live countsGPT-6.1 Sol011000.0050570.0934810.0884240.026855
NASA's latest posts on X, with live countsGemini 3 Flash0.20.4666826.6680.2062430.532480.3262370.184755
AA 100 flight statusClaude Opus 5.510.8-200.1059630.067939-0.0380240.0155
AA 100 flight statusClaude Sonnet 5.50.721280.0596060.04436-0.0152460.0155
AA 100 flight statusClaude Haiku 5.50.520.680.0043180.0118440.0075250.00855
AA 100 flight statusGrok 4.60.40.8400.1138540.099797-0.0140560.0155
AA 100 flight statusGPT-6 Luna0.720.880.0028520.017760.0149070.01455
AA 100 flight statusGPT-6.1 Sol10.2-800.0162560.042450.0261940.0155
AA 100 flight statusGemini 3 Flash10.8-200.1581360.061039-0.097097055
Who [company name redacted] is hiring right now*Claude Opus 5.510.94286-5.7140.225940.2718860.045946055
Who [company name redacted] is hiring right now*Claude Sonnet 5.50.971420.9714200.1126670.105441-0.007225055
Who [company name redacted] is hiring right now*Claude Haiku 5.50.657140.62856-2.8580.0056110.0066680.001057055
Who [company name redacted] is hiring right now*Grok 4.60.9714212.8580.2540260.2694970.0154710.0050455
Who [company name redacted] is hiring right now*GPT-6 Luna0.828560.88575.7140.0040630.0041260.000063055
Who [company name redacted] is hiring right now*GPT-6.1 Sol1100.0617950.1218590.0600650.0460455
Who [company name redacted] is hiring right now*Gemini 3 Flash0.942860.6857-25.7160.0937010.080926-0.0127750.0025255
5 new house listings in Austin 78745Claude Opus 5.50.81200.2871410.077822-0.2093190.0155
5 new house listings in Austin 78745Claude Sonnet 5.50.81200.1535150.061762-0.0917530.0155
5 new house listings in Austin 78745Claude Haiku 5.50.41600.0080010.0156190.0076180.01255
5 new house listings in Austin 78745Grok 4.60.81200.2245780.09207-0.1325090.0155
5 new house listings in Austin 78745GPT-6 Luna0.92180.009110.0142850.0051750.0155
5 new house listings in Austin 78745GPT-6.1 Sol0.880.9680.1517970.117922-0.0338750.03855
5 new house listings in Austin 78745Gemini 3 Flash0.840.9280.119990.1526790.0326890.00655
Fintech Series A market mapClaude Opus 5.50.89998110.0020.1118560.108267-0.003589055
Fintech Series A market mapClaude Sonnet 5.50.833320.93332100.0582160.0889390.030723055
Fintech Series A market mapClaude Haiku 5.50.666640.8666620.0020.0046080.004329-0.00028055
Fintech Series A market mapGrok 4.60.833320.866663.3340.0845920.0861590.001567055
Fintech Series A market mapGPT-6 Luna0.633320.6-3.3320.0053070.002876-0.002431055
Fintech Series A market mapGPT-6.1 Sol0.866640.899983.3340.0871570.079724-0.0074330.000455
Fintech Series A market mapGemini 3 Flash0.933320.73332-200.0362450.0559360.019691055
Meeting prep: CEO briefing*Claude Opus 5.50.91100.4018190.172641-0.2291790.012455
Meeting prep: CEO briefing*Claude Sonnet 5.50.91100.1887850.084782-0.1040030.0155
Meeting prep: CEO briefing*Claude Haiku 5.50.50.62120.0094740.0135970.0041220.00255
Meeting prep: CEO briefing*Grok 4.60.90.9880.9213550.255792-0.6655630.0155
Meeting prep: CEO briefing*GPT-6 Luna0.340.7360.0385370.021502-0.0170360.00455
Meeting prep: CEO briefing*GPT-6.1 Sol1100.1805510.13905-0.0415010.05155
Meeting prep: CEO briefing*Gemini 3 Flash0.60.600.1415150.126301-0.015214055
Research 10 prospects, rank them, email the top 3Claude Opus 5.50.868760.81876-50.384750.8725050.4877550.08955
Research 10 prospects, rank them, email the top 3Claude Sonnet 5.50.750020.9312618.1240.2610780.4931840.2321060.0555
Research 10 prospects, rank them, email the top 3Claude Haiku 5.50.850020.856280.6260.0163240.037630.0213060.0255
Research 10 prospects, rank them, email the top 3Grok 4.60.675040.87519.9960.4392810.5963130.1570320.0355
Research 10 prospects, rank them, email the top 3GPT-6 Luna0.837520.874983.7460.0165450.0505040.0339590.03255
Research 10 prospects, rank them, email the top 3GPT-6.1 Sol0.681260.9062222.4960.1558980.2853260.1294270.035855
Research 10 prospects, rank them, email the top 3Gemini 3 Flash0.756260.8625210.6260.2666380.129446-0.1371920.0255
What parents on Reddit say about a car seatClaude Opus 5.500.9666696.6660.6080960.493296-0.11480.12455
What parents on Reddit say about a car seatClaude Sonnet 5.500.8333283.3320.1377040.3182310.1805270.1455
What parents on Reddit say about a car seatClaude Haiku 5.50000.0108660.006436-0.00443055
What parents on Reddit say about a car seatGrok 4.600.7333473.3340.7793760.429551-0.3498250.15255
What parents on Reddit say about a car seatGPT-6 Luna0000.0227830.0253040.002522055
What parents on Reddit say about a car seatGPT-6.1 Sol00.9333493.3340.0763240.2591940.182870.15255
What parents on Reddit say about a car seatGemini 3 Flash00.4400.1377670.1852430.0474760.0455
OFAC sanctions screen for 12 counterpartiesClaude Opus 5.51100.5466860.7259510.1792640.655
OFAC sanctions screen for 12 counterpartiesClaude Sonnet 5.51100.273870.5738440.2999740.4855
OFAC sanctions screen for 12 counterpartiesClaude Haiku 5.51100.0115780.011354-0.000224055
OFAC sanctions screen for 12 counterpartiesGrok 4.61100.3363270.7094030.3730760.6155
OFAC sanctions screen for 12 counterpartiesGPT-6 Luna1100.0114410.1339570.1225160.12255
OFAC sanctions screen for 12 counterpartiesGPT-6.1 Sol1100.1720040.6292280.4572240.655
OFAC sanctions screen for 12 counterpartiesGemini 3 Flash0.9833411.6660.0938270.6800910.5862640.455
Follower counts for 30 X accounts, from a CRM export with user IDsClaude Opus 5.50.21800.0918850.2147270.1228410.0355
Follower counts for 30 X accounts, from a CRM export with user IDsClaude Sonnet 5.50.31332168.6680.2352770.128472-0.1068040.03855
Follower counts for 30 X accounts, from a CRM export with user IDsClaude Haiku 5.500.6466664.6660.0026560.0993870.096730.09255
Follower counts for 30 X accounts, from a CRM export with user IDsGrok 4.60.38666161.3340.3471220.210246-0.1368760.05455
Follower counts for 30 X accounts, from a CRM export with user IDsGPT-6 Luna0.373340.2-17.3340.0099010.1204090.1105080.11855
Follower counts for 30 X accounts, from a CRM export with user IDsGPT-6.1 Sol011000.020760.1229240.1021640.06255
Follower counts for 30 X accounts, from a CRM export with user IDsGemini 3 Flash0.13334186.6660.3543350.200444-0.1538910.10955
Five available .com namesClaude Opus 5.51100.0520880.04383-0.008258055
Five available .com namesClaude Sonnet 5.51100.0257320.0364280.010696055
Five available .com namesClaude Haiku 5.50.520.84320.0074680.005584-0.001883055
Five available .com namesGrok 4.61100.0603840.0814020.0210190.00255
Five available .com namesGPT-6 Luna10.8-200.0018020.3850510.3832490.38255
Five available .com namesGPT-6.1 Sol1100.0174040.1944060.1770020.17255
Five available .com namesGemini 3 Flash0.60.600.1846690.4308830.2462140.317855
Next Friday's Apple closing priceClaude Opus 5.51100.0063530.005779-0.000574055
Next Friday's Apple closing priceClaude Sonnet 5.51100.0024390.0026530.000214055
Next Friday's Apple closing priceClaude Haiku 5.51100.0001990.0002050.000007055
Next Friday's Apple closing priceGrok 4.61100.0035390.0119460.008407055
Next Friday's Apple closing priceGPT-6 Luna1100.0001510.000340.000189055
Next Friday's Apple closing priceGPT-6.1 Sol1100.0027050.0037620.001057055
Next Friday's Apple closing priceGemini 3 Flash1100.0392320.0928890.053657055
Last closing prices for 5 US stocksClaude Opus 5.50.60.56-40.0908870.122590.0317030.00955
Last closing prices for 5 US stocksClaude Sonnet 5.51100.0151950.00814-0.007055055
Last closing prices for 5 US stocksClaude Haiku 5.51100.0039460.002286-0.00166055
Last closing prices for 5 US stocksGrok 4.60.40.400.057840.0700470.012207055
Last closing prices for 5 US stocksGPT-6 Luna0.80.2-600.0048430.0057160.000873055
Last closing prices for 5 US stocksGPT-6.1 Sol1100.0182350.0240950.005860.007455
Last closing prices for 5 US stocksGemini 3 Flash0.81200.0462650.05140.0051350.003655
Tomorrow's weather in ChicagoClaude Opus 5.51100.0242370.0424350.018198055
Tomorrow's weather in ChicagoClaude Sonnet 5.51100.0239640.018083-0.005881055
Tomorrow's weather in ChicagoClaude Haiku 5.51100.0015770.001132-0.000445055
Tomorrow's weather in ChicagoGrok 4.61100.036550.033475-0.003076055
Tomorrow's weather in ChicagoGPT-6 Luna0.40.3-100.0023280.0306030.0282750.02455
Tomorrow's weather in ChicagoGPT-6.1 Sol1100.0078130.0240620.0162490.00455
Tomorrow's weather in ChicagoGemini 3 Flash10.7-300.0505330.043912-0.00662055
Apple.com visits yesterdayClaude Opus 5.51100.0085930.0111330.00254055
Apple.com visits yesterdayClaude Sonnet 5.51100.0043270.0189830.014656055
Apple.com visits yesterdayClaude Haiku 5.51100.0016980.001165-0.000534055
Apple.com visits yesterdayGrok 4.61100.0289490.0508220.021873055
Apple.com visits yesterdayGPT-6 Luna10.8-200.000510.0415470.0410370.0455
Apple.com visits yesterdayGPT-6.1 Sol1100.0040860.0256410.021555055
Apple.com visits yesterdayGemini 3 Flash1100.0227520.0327670.010015055
Primes with digit sum 40Claude Opus 5.50.71300.1387160.081641-0.0570750.011455
Primes with digit sum 40Claude Sonnet 5.5011000.0086250.0473140.0386890.011255
Primes with digit sum 40Claude Haiku 5.50000.0006420.001440.000798055
Primes with digit sum 40Grok 4.6011000.439480.230586-0.2088940.012255
Primes with digit sum 40GPT-6 Luna011000.0088930.0150680.0061750.013255
Primes with digit sum 40GPT-6.1 Sol0.21800.0531990.031512-0.0216870.01155
Primes with digit sum 40Gemini 3 Flash011000.5020790.211215-0.2908640.026255

* some tasks have been slightly anonymized/modified to maintain privacy.

Improvements in task completion vs. cost when an agent has USDC and the ability to buy services +30.8 pp for +$0.046 / task

Each arrow compares the same model without and with USDC. Completion and cost are averaged over all 21 tasks. Cost includes model inference and agent purchases.

Without USDCWith USDC
View run data for this test

Completion vs. cost

Mean cost includes model inference and services purchased by the agent, in USD per task. Checker purchases and web search are excluded. Completion is the share of predefined checks passed. Cost-change intervals are the published 95% ranges.

These are the published run aggregates, not individual run logs. The source snapshot does not include per-run records, prompts, outputs or traces. No individual run records have been reconstructed.

Scope & denominators
Seven models, 21 tasks, five repeats per condition; 1,470 runs in total. Completion and cost give tasks and models equal weight.
Units
Costs are USD. Completion scores are fractions from 0 to 1; changes ending in _pp are percentage points.
Published snapshot
2026-11-13 16:59 UTC
Completion vs. cost: model results
modelwithout completionwith completioncompletion change ppwithout cost usdwith cost usdextra cost usdextra cost low usdextra cost high usdwith agent purchases usdwithout runswith runs
Claude Opus 5.50.6080350.95499734.696190.1673930.2282180.060825-0.015270.1369210.0652105105
Claude Sonnet 5.50.5927660.93917434.6408570.0900610.1474640.0574030.0069210.1078840.057281105105
Claude Haiku 5.50.4476410.6789623.1319050.0047910.0373030.0325130.0072650.057760.032281105105
Grok 4.60.5317350.86928633.7550480.241630.22157-0.02006-0.1217570.0816370.064721105105
GPT-6 Luna0.4837810.72511224.1331430.0073810.0677090.0603290.0179830.1026740.061605105105
GPT-6.1 Sol0.5600580.94759738.7539050.053290.1452110.0919220.0381430.1457010.085471105105
Gemini 3 Flash0.5366250.79848726.186190.1435580.183450.039893-0.0462520.1260370.074772105105
Task-by-model results (147 cells)

Five runs per condition in each cell.

Completion vs. cost: task-by-model results
taskmodelwithout completionwith completioncompletion change ppwithout cost usdwith cost usdextra cost usdwith agent purchases usdwithout runswith runs
Email tomorrow's NYC forecastClaude Opus 5.5011000.0555560.1353640.0798080.0255
Email tomorrow's NYC forecastClaude Sonnet 5.5011000.0212870.0568370.035550.0255
Email tomorrow's NYC forecastClaude Haiku 5.500.8800.0010320.0224180.0213860.0255
Email tomorrow's NYC forecastGrok 4.600.8800.0231730.104580.0814070.01655
Email tomorrow's NYC forecastGPT-6 Luna00.9333493.3340.001180.022180.0210.0255
Email tomorrow's NYC forecastGPT-6.1 Sol011000.0125570.0484320.0358760.0255
Email tomorrow's NYC forecastGemini 3 Flash011000.041550.0521890.0106390.0255
Photo of a forest signClaude Opus 5.50.21800.0217370.1043820.0826450.061555
Photo of a forest signClaude Sonnet 5.50.20.8600.0101290.1575880.1474590.119255
Photo of a forest signClaude Haiku 5.50.20.4200.000570.1619290.1613590.159455
Photo of a forest signGrok 4.6011000.003430.143840.140410.048455
Photo of a forest signGPT-6 Luna011000.0001290.0978680.0977390.09506155
Photo of a forest signGPT-6.1 Sol011000.0028630.1581860.1553230.11294155
Photo of a forest signGemini 3 Flash011000.0057110.1652090.1594970.1278955
Hat-trick song for the drive homeClaude Opus 5.5011000.0227630.1988880.1761250.159555
Hat-trick song for the drive homeClaude Sonnet 5.5011000.0078790.1789990.171120.159555
Hat-trick song for the drive homeClaude Haiku 5.5011000.0003880.16080.1604120.159555
Hat-trick song for the drive homeGrok 4.6011000.005630.2926740.2870430.183555
Hat-trick song for the drive homeGPT-6 Luna00.8800.0002330.1795380.1793050.17845255
Hat-trick song for the drive homeGPT-6.1 Sol011000.0050760.1746450.1695690.159555
Hat-trick song for the drive homeGemini 3 Flash011000.0085010.1686920.1601910.159555
NASA's latest TikTok videos and their viewsClaude Opus 5.500.9666696.6660.0791910.4303930.3512020.06455
NASA's latest TikTok videos and their viewsClaude Sonnet 5.500.4400.0341950.176470.1422750.0455
NASA's latest TikTok videos and their viewsClaude Haiku 5.500.2200.0012790.0082880.0070090.00455
NASA's latest TikTok videos and their viewsGrok 4.600.8800.300780.504520.2037410.05655
NASA's latest TikTok videos and their viewsGPT-6 Luna00.4400.00180.085010.0832090.0855
NASA's latest TikTok videos and their viewsGPT-6.1 Sol011000.0091350.2490770.2399420.13655
NASA's latest TikTok videos and their viewsGemini 3 Flash0.10.4300.1549160.2402760.085360.09655
Follower counts for 30 space accounts on XClaude Opus 5.50.41600.1608870.3664770.2055910.1555
Follower counts for 30 space accounts on XClaude Sonnet 5.50.960.92-40.2418730.318340.0764660.09155
Follower counts for 30 space accounts on XClaude Haiku 5.50.086660.871.3340.0059840.1285920.1226080.12155
Follower counts for 30 space accounts on XGrok 4.60.61400.4925520.279702-0.212850.1555
Follower counts for 30 space accounts on XGPT-6 Luna0.30666169.3340.0070880.1556080.148520.15155
Follower counts for 30 space accounts on XGPT-6.1 Sol0.13332186.6680.0584080.2244590.1660510.1555
Follower counts for 30 space accounts on XGemini 3 Flash0.380.6220.3501060.158436-0.191670.05755
NASA's latest posts on X, with live countsClaude Opus 5.50.21800.0900980.244630.1545320.018455
NASA's latest posts on X, with live countsClaude Sonnet 5.500.9333493.3340.0149160.1778870.1629710.02455
NASA's latest posts on X, with live countsClaude Haiku 5.50000.0023820.0826640.0802820.0855
NASA's latest posts on X, with live countsGrok 4.60.20-200.1214070.100542-0.0208650.0155
NASA's latest posts on X, with live countsGPT-6 Luna00.9333493.3340.0054960.0126440.0071480.0155
NASA's latest posts on X, with live countsGPT-6.1 Sol011000.0050570.0934810.0884240.026855
NASA's latest posts on X, with live countsGemini 3 Flash0.20.4666826.6680.2062430.532480.3262370.184755
AA 100 flight statusClaude Opus 5.510.8-200.1059630.067939-0.0380240.0155
AA 100 flight statusClaude Sonnet 5.50.721280.0596060.04436-0.0152460.0155
AA 100 flight statusClaude Haiku 5.50.520.680.0043180.0118440.0075250.00855
AA 100 flight statusGrok 4.60.40.8400.1138540.099797-0.0140560.0155
AA 100 flight statusGPT-6 Luna0.720.880.0028520.017760.0149070.01455
AA 100 flight statusGPT-6.1 Sol10.2-800.0162560.042450.0261940.0155
AA 100 flight statusGemini 3 Flash10.8-200.1581360.061039-0.097097055
Who [company name redacted] is hiring right now*Claude Opus 5.510.94286-5.7140.225940.2718860.045946055
Who [company name redacted] is hiring right now*Claude Sonnet 5.50.971420.9714200.1126670.105441-0.007225055
Who [company name redacted] is hiring right now*Claude Haiku 5.50.657140.62856-2.8580.0056110.0066680.001057055
Who [company name redacted] is hiring right now*Grok 4.60.9714212.8580.2540260.2694970.0154710.0050455
Who [company name redacted] is hiring right now*GPT-6 Luna0.828560.88575.7140.0040630.0041260.000063055
Who [company name redacted] is hiring right now*GPT-6.1 Sol1100.0617950.1218590.0600650.0460455
Who [company name redacted] is hiring right now*Gemini 3 Flash0.942860.6857-25.7160.0937010.080926-0.0127750.0025255
5 new house listings in Austin 78745Claude Opus 5.50.81200.2871410.077822-0.2093190.0155
5 new house listings in Austin 78745Claude Sonnet 5.50.81200.1535150.061762-0.0917530.0155
5 new house listings in Austin 78745Claude Haiku 5.50.41600.0080010.0156190.0076180.01255
5 new house listings in Austin 78745Grok 4.60.81200.2245780.09207-0.1325090.0155
5 new house listings in Austin 78745GPT-6 Luna0.92180.009110.0142850.0051750.0155
5 new house listings in Austin 78745GPT-6.1 Sol0.880.9680.1517970.117922-0.0338750.03855
5 new house listings in Austin 78745Gemini 3 Flash0.840.9280.119990.1526790.0326890.00655
Fintech Series A market mapClaude Opus 5.50.89998110.0020.1118560.108267-0.003589055
Fintech Series A market mapClaude Sonnet 5.50.833320.93332100.0582160.0889390.030723055
Fintech Series A market mapClaude Haiku 5.50.666640.8666620.0020.0046080.004329-0.00028055
Fintech Series A market mapGrok 4.60.833320.866663.3340.0845920.0861590.001567055
Fintech Series A market mapGPT-6 Luna0.633320.6-3.3320.0053070.002876-0.002431055
Fintech Series A market mapGPT-6.1 Sol0.866640.899983.3340.0871570.079724-0.0074330.000455
Fintech Series A market mapGemini 3 Flash0.933320.73332-200.0362450.0559360.019691055
Meeting prep: CEO briefing*Claude Opus 5.50.91100.4018190.172641-0.2291790.012455
Meeting prep: CEO briefing*Claude Sonnet 5.50.91100.1887850.084782-0.1040030.0155
Meeting prep: CEO briefing*Claude Haiku 5.50.50.62120.0094740.0135970.0041220.00255
Meeting prep: CEO briefing*Grok 4.60.90.9880.9213550.255792-0.6655630.0155
Meeting prep: CEO briefing*GPT-6 Luna0.340.7360.0385370.021502-0.0170360.00455
Meeting prep: CEO briefing*GPT-6.1 Sol1100.1805510.13905-0.0415010.05155
Meeting prep: CEO briefing*Gemini 3 Flash0.60.600.1415150.126301-0.015214055
Research 10 prospects, rank them, email the top 3Claude Opus 5.50.868760.81876-50.384750.8725050.4877550.08955
Research 10 prospects, rank them, email the top 3Claude Sonnet 5.50.750020.9312618.1240.2610780.4931840.2321060.0555
Research 10 prospects, rank them, email the top 3Claude Haiku 5.50.850020.856280.6260.0163240.037630.0213060.0255
Research 10 prospects, rank them, email the top 3Grok 4.60.675040.87519.9960.4392810.5963130.1570320.0355
Research 10 prospects, rank them, email the top 3GPT-6 Luna0.837520.874983.7460.0165450.0505040.0339590.03255
Research 10 prospects, rank them, email the top 3GPT-6.1 Sol0.681260.9062222.4960.1558980.2853260.1294270.035855
Research 10 prospects, rank them, email the top 3Gemini 3 Flash0.756260.8625210.6260.2666380.129446-0.1371920.0255
What parents on Reddit say about a car seatClaude Opus 5.500.9666696.6660.6080960.493296-0.11480.12455
What parents on Reddit say about a car seatClaude Sonnet 5.500.8333283.3320.1377040.3182310.1805270.1455
What parents on Reddit say about a car seatClaude Haiku 5.50000.0108660.006436-0.00443055
What parents on Reddit say about a car seatGrok 4.600.7333473.3340.7793760.429551-0.3498250.15255
What parents on Reddit say about a car seatGPT-6 Luna0000.0227830.0253040.002522055
What parents on Reddit say about a car seatGPT-6.1 Sol00.9333493.3340.0763240.2591940.182870.15255
What parents on Reddit say about a car seatGemini 3 Flash00.4400.1377670.1852430.0474760.0455
OFAC sanctions screen for 12 counterpartiesClaude Opus 5.51100.5466860.7259510.1792640.655
OFAC sanctions screen for 12 counterpartiesClaude Sonnet 5.51100.273870.5738440.2999740.4855
OFAC sanctions screen for 12 counterpartiesClaude Haiku 5.51100.0115780.011354-0.000224055
OFAC sanctions screen for 12 counterpartiesGrok 4.61100.3363270.7094030.3730760.6155
OFAC sanctions screen for 12 counterpartiesGPT-6 Luna1100.0114410.1339570.1225160.12255
OFAC sanctions screen for 12 counterpartiesGPT-6.1 Sol1100.1720040.6292280.4572240.655
OFAC sanctions screen for 12 counterpartiesGemini 3 Flash0.9833411.6660.0938270.6800910.5862640.455
Follower counts for 30 X accounts, from a CRM export with user IDsClaude Opus 5.50.21800.0918850.2147270.1228410.0355
Follower counts for 30 X accounts, from a CRM export with user IDsClaude Sonnet 5.50.31332168.6680.2352770.128472-0.1068040.03855
Follower counts for 30 X accounts, from a CRM export with user IDsClaude Haiku 5.500.6466664.6660.0026560.0993870.096730.09255
Follower counts for 30 X accounts, from a CRM export with user IDsGrok 4.60.38666161.3340.3471220.210246-0.1368760.05455
Follower counts for 30 X accounts, from a CRM export with user IDsGPT-6 Luna0.373340.2-17.3340.0099010.1204090.1105080.11855
Follower counts for 30 X accounts, from a CRM export with user IDsGPT-6.1 Sol011000.020760.1229240.1021640.06255
Follower counts for 30 X accounts, from a CRM export with user IDsGemini 3 Flash0.13334186.6660.3543350.200444-0.1538910.10955
Five available .com namesClaude Opus 5.51100.0520880.04383-0.008258055
Five available .com namesClaude Sonnet 5.51100.0257320.0364280.010696055
Five available .com namesClaude Haiku 5.50.520.84320.0074680.005584-0.001883055
Five available .com namesGrok 4.61100.0603840.0814020.0210190.00255
Five available .com namesGPT-6 Luna10.8-200.0018020.3850510.3832490.38255
Five available .com namesGPT-6.1 Sol1100.0174040.1944060.1770020.17255
Five available .com namesGemini 3 Flash0.60.600.1846690.4308830.2462140.317855
Next Friday's Apple closing priceClaude Opus 5.51100.0063530.005779-0.000574055
Next Friday's Apple closing priceClaude Sonnet 5.51100.0024390.0026530.000214055
Next Friday's Apple closing priceClaude Haiku 5.51100.0001990.0002050.000007055
Next Friday's Apple closing priceGrok 4.61100.0035390.0119460.008407055
Next Friday's Apple closing priceGPT-6 Luna1100.0001510.000340.000189055
Next Friday's Apple closing priceGPT-6.1 Sol1100.0027050.0037620.001057055
Next Friday's Apple closing priceGemini 3 Flash1100.0392320.0928890.053657055
Last closing prices for 5 US stocksClaude Opus 5.50.60.56-40.0908870.122590.0317030.00955
Last closing prices for 5 US stocksClaude Sonnet 5.51100.0151950.00814-0.007055055
Last closing prices for 5 US stocksClaude Haiku 5.51100.0039460.002286-0.00166055
Last closing prices for 5 US stocksGrok 4.60.40.400.057840.0700470.012207055
Last closing prices for 5 US stocksGPT-6 Luna0.80.2-600.0048430.0057160.000873055
Last closing prices for 5 US stocksGPT-6.1 Sol1100.0182350.0240950.005860.007455
Last closing prices for 5 US stocksGemini 3 Flash0.81200.0462650.05140.0051350.003655
Tomorrow's weather in ChicagoClaude Opus 5.51100.0242370.0424350.018198055
Tomorrow's weather in ChicagoClaude Sonnet 5.51100.0239640.018083-0.005881055
Tomorrow's weather in ChicagoClaude Haiku 5.51100.0015770.001132-0.000445055
Tomorrow's weather in ChicagoGrok 4.61100.036550.033475-0.003076055
Tomorrow's weather in ChicagoGPT-6 Luna0.40.3-100.0023280.0306030.0282750.02455
Tomorrow's weather in ChicagoGPT-6.1 Sol1100.0078130.0240620.0162490.00455
Tomorrow's weather in ChicagoGemini 3 Flash10.7-300.0505330.043912-0.00662055
Apple.com visits yesterdayClaude Opus 5.51100.0085930.0111330.00254055
Apple.com visits yesterdayClaude Sonnet 5.51100.0043270.0189830.014656055
Apple.com visits yesterdayClaude Haiku 5.51100.0016980.001165-0.000534055
Apple.com visits yesterdayGrok 4.61100.0289490.0508220.021873055
Apple.com visits yesterdayGPT-6 Luna10.8-200.000510.0415470.0410370.0455
Apple.com visits yesterdayGPT-6.1 Sol1100.0040860.0256410.021555055
Apple.com visits yesterdayGemini 3 Flash1100.0227520.0327670.010015055
Primes with digit sum 40Claude Opus 5.50.71300.1387160.081641-0.0570750.011455
Primes with digit sum 40Claude Sonnet 5.5011000.0086250.0473140.0386890.011255
Primes with digit sum 40Claude Haiku 5.50000.0006420.001440.000798055
Primes with digit sum 40Grok 4.6011000.439480.230586-0.2088940.012255
Primes with digit sum 40GPT-6 Luna011000.0088930.0150680.0061750.013255
Primes with digit sum 40GPT-6.1 Sol0.21800.0531990.031512-0.0216870.01155
Primes with digit sum 40Gemini 3 Flash011000.5020790.211215-0.2908640.026255

* some tasks have been slightly anonymized/modified to maintain privacy.

Key takeaways

  • Agents with USDC and the ability to buy services passed more completion checks: 84.5% versus 53.7%, an average gain of 30.8 percentage points across seven models and all 21 tasks.
  • Among runs where the agent said it was done, 78.9% were fully right with USDC and the ability to buy services, versus 68.1% without.
  • Among runs that submitted an answer, item accuracy rose from 56.7% to 88.1%. The comparison excludes runs with no answer and covers different sets of runs in each condition.
  • Among graded claims in eligible task types, the share backed by the cited source fell from 99.1% to 97.9%, a decline of 1.2 percentage points. This measure covers 536 claims without USDC and 559 with USDC.
  • The completion gain came with higher model-plus-service cost: $0.147 versus $0.101 per task, an increase of $0.046 or 45.6%. Checker purchases and web search are excluded from this comparison.

About this study

How to read the averages

Each model ran the same 21 tasks five times per condition: 210 runs per model, 735 per condition, 1,470 in total. Headline changes are arithmetic means of the seven model-level means, with equal task weights.

What this comparison establishes

It tests a budget and tools for purchasing services together. It does not isolate money from the extra capabilities and data those services unlock. Completion checks measure specified requirements; they do not establish creative quality or whole-job success.

Scope and limitations

The task set favors situations where paid access may help. The free baseline has search and read-only fetching, and some checkers can share a data source with the agent.

Quality comparisons are descriptive; statistical significance has not been established. One task (compute) is constrained by the free baseline’s lack of code execution; excluding it, the completion gain is 28.7 percentage points.

Completion and cost snapshot: November 11, 2026, 21:26 UTC.
Quality analysis added: November 13, 2026, 16:59 UTC.

Spend on this study

The 1,470 runs shown here cost $238.28 across the conditions. This research expense total is separate from the model-plus-service comparison above.

Study spend by model $34.04 average / model

Total spent on each model’s 210 runs, both conditions combined. Includes model tokens, agent purchases, checker purchases and web search.

View run data for this test

Study spend by model

Research expense for both conditions combined: model inference, agent purchases, checker purchases and web search. This ledger is distinct from the model-plus-service comparison.

These are the published run aggregates, not individual run logs. The source snapshot does not include per-run records, prompts, outputs or traces. No individual run records have been reconstructed.

Scope & denominators
Seven models, 21 tasks, five repeats per condition; 1,470 runs in total. Completion and cost give tasks and models equal weight.
Units
Costs are USD. Completion scores are fractions from 0 to 1; changes ending in _pp are percentage points.
Published snapshot
2026-11-13 16:59 UTC
Study spend by model: model results
modelmodel usdagent purchases usdchecker purchases usdweb search usdtotal usdruns
Claude Opus 5.534.69316.8460.821.76444.1231210
Claude Sonnet 5.518.92566.01450.942.94728.8271210
Claude Haiku 5.51.03043.38950.83.1858.4049210
Grok 4.641.84026.79570.8210.24159.6969210
GPT-6 Luna1.41596.46860.8413.88822.6125210
GPT-6.1 Sol11.86828.97441.123.22725.1896210
Gemini 3 Flash26.48477.85111.513.58749.4228210
All models136.2646.346.8448.84238.281,470

The published snapshot provides this measure at model level; it does not include a task-by-task breakdown.

Give your agent USDC

Copy this prompt into your agent

Run curl -sL https://agents.circle.com/skills/setup.md, and use the returned setup instructions to set up my agent wallet.

Circle Agent Stack gives your agent a wallet and a way to discover and pay for services as it works. Start with a task, a small budget and a clear definition of done.

  1. Set up an agent wallet

    Have your agent install Circle CLI and the official Circle Skills, to create an agent wallet.

  2. Fund your agent wallet

    Fund your wallet with existing USDC on a supported network.

  3. Discover Services

    Point your agent to Circle Agent Marketplace to discover premium services your agent can pay for.

Facilitator Service is offered by Circle Technology Services, LLC, a software provider. Facilitator Service verifies buyer-signed EIP-3009 USDC payment authorizations under the x402 protocol only — it does not hold, control, or transmit user funds, verify transaction purpose, or confirm agent authority scope. On Base and Polygon, settlements are subject to blockchain reorganization risk; Circle does not guarantee fund receipt. Arc settlements are final. Use of the Facilitator Service is subject to Circle Console Service Terms and Developer Terms. Not reviewed or approved by any regulatory authority. Features may change at any time. Nothing herein constitutes a commitment, warranty, or guarantee. Use involves risk.
Circle Technology Services, LLC (“CTS”) is a software provider and does not provide financial, advisory, or marketplace services. Agent Stack enables users and developers to interact with third-party applications and services, which are not controlled by Circle. Transactions initiated by agents are executed based on user-defined permissions and may occur without real-time human review. Circle does not guarantee the performance, availability, or outcomes of any third-party services or agent-initiated transactions. Users are solely responsible for their use of these tools and for evaluating associated risks. For additional details, please see the Agent Platform Terms of Use available at agents.circle.com/terms-of-use
Live demo with Circle-funded wallets. Not advice. Onchain transactions are irreversible. See important disclosures on the corresponding websites for any Circle products and services referenced here: Circle Agent Stack, Circle CLI and Circle Agent Marketplace.

---
title: "The Funded Agent Index"
type: "experiment"
number: "AW-005"
date: "2026-11-19"
built_by: ["Shafu Elfouly","Jasper Kim","Danny Organ"]
tags: ["Experiment"]
summary: "Across 1,470 runs, seven models with a capped USDC budget and paid-service tools passed 84.5% of task checks versus 53.7% in the baseline, at higher cost."
url: "https://agentic-works.circle.com/experiments/funded-agent-index"
---

# The Funded Agent Index

> Across 1,470 runs, seven models with a capped USDC budget and paid-service tools passed 84.5% of task checks versus 53.7% in the baseline, at higher cost.

Finding out if agents are better at accomplishing tasks if they can pay for things

## TL;DR

Across seven AI models, giving agents a USDC budget and paid-service tools raised average task completion by 30.8 percentage points, for an average increase of $0.046 in model-plus-service cost per task. Among runs where the agent said it was done, the share passing every check rose 10.7 percentage points. Quality measures cover different sets of runs, and results were mixed: claims backed by a cited source slipped from 99.1% to 97.9% on eligible tasks.

## Context

An agent can draft an email without sending it or describe a song without producing audio. A budget gives it an option to keep going when they stop knowing what to do or being able to do it without help: acquire the capability it is missing. Anecdotally, we’ve seen that giving agents the ability to pay for things produces more complete and higher quality work, but we wanted to test that observation programmatically.  That’s the impetus behind this study: to see if giving your agent money gives it superpowers.

The questions

1. Can an agent with USDC buy capabilities it is missing and complete tasks at a higher rate?
2. Can an agent with USDC buy capabilities it is missing to complete tasks with higher quality?
3. What is the incremental spend required to increase agentic task completion and quality?

## Does an agent with USDC and the ability to buy services improve task completion?

Yes, +30.8 pp average.

Without: 53.7% → with: 84.5%, averaged equally across seven models and all 21 tasks. This is the share of checks passed, not the share of whole jobs finished.

- Claude Opus 5.5: 60.8% → 95.5%, +34.7 percentage points.
- GPT-6.1 Sol: 56.0% → 94.8%, +38.8 percentage points.
- Claude Sonnet 5.5: 59.3% → 93.9%, +34.6 percentage points.
- Grok 4.6: 53.2% → 86.9%, +33.8 percentage points.
- Gemini 3 Flash: 53.7% → 79.8%, +26.2 percentage points.
- GPT-6 Luna: 48.4% → 72.5%, +24.1 percentage points.
- Claude Haiku 5.5: 44.8% → 67.9%, +23.1 percentage points.

## Did quality of finished work change?

The quality graphic has three conditional measures, pooled across models. They describe different subsets of runs or claims, rather than an additional completion score or a measure of creative quality.

- Right when done: among runs where the agent said it was done, the share where every required item checked out rose from 68.1% without USDC to 78.9% with USDC, a pooled increase of 10.7 percentage points. There were 317 and 573 runs that said done, respectively. Runs that gave up or exhausted their budget are excluded.
- Item accuracy: among runs that handed in an answer, the share of required items that checked out rose from 56.7% to 88.1%. There were 698 and 707 answered runs, respectively. Runs with no answer are excluded; those run counts are not counts of graded items.
- Claims backed by source: among graded claims, the share backed by the source cited by the agent fell from 99.1% without USDC to 97.9% with USDC, a pooled decline of 1.2 percentage points. There were 536 and 559 graded claims, respectively. A different model grades each claim without seeing the condition. This measure covers market map, meeting prep, prospects and eligible Reddit runs only.

The three views have different denominators and do not establish a general quality improvement or statistical significance.

## How much does it cost for improved rates of task completion?

Mean model-plus-service cost rose from $0.101 to $0.147 per task: +$0.046, or 45.6%, on average. This comparison includes model inference and services purchased by the agent. Checker purchases and web search costs are excluded.

## Improvements in task completion vs. cost when an agent has USDC and the ability to buy services

The full-panel comparison delivers +30.8 percentage points in task checks passed for +$0.046 per task, averaged across models. Each chart arrow compares one model without and with a USDC budget and paid-service tools.

The visual article includes collapsed supporting-data tables beneath each chart. Download the published aggregates here:

- [Task completion CSV](https://agentic-works.circle.com/experiments/funded-agent-index/data/completion.csv)
- [Extra cost CSV](https://agentic-works.circle.com/experiments/funded-agent-index/data/extra-cost.csv)
- [Completion vs. cost CSV](https://agentic-works.circle.com/experiments/funded-agent-index/data/completion-cost.csv)
- [Quality: right when done CSV](https://agentic-works.circle.com/experiments/funded-agent-index/data/quality-done.csv)
- [Quality: item accuracy CSV](https://agentic-works.circle.com/experiments/funded-agent-index/data/quality-items.csv)
- [Quality: claims backed by source CSV](https://agentic-works.circle.com/experiments/funded-agent-index/data/quality-graded.csv)
- [Quality: claims backed by source JSON](https://agentic-works.circle.com/experiments/funded-agent-index/data/quality-graded.json)

These are the published run aggregates, not individual run logs. The source snapshot does not include per-run records, prompts, outputs or traces. No individual run records have been reconstructed.

## Key takeaways

- Agents with USDC and the ability to buy services passed more completion checks: 84.5% versus 53.7%, an average gain of 30.8 percentage points across seven models and all 21 tasks.
- Among runs where the agent said it was done, 78.9% were fully right with USDC and the ability to buy services, versus 68.1% without.
- Among runs that submitted an answer, item accuracy rose from 56.7% to 88.1%. The comparison excludes runs with no answer and covers different sets of runs in each condition.
- Among graded claims in eligible task types, the share backed by the cited source fell from 99.1% to 97.9%, a decline of 1.2 percentage points. This measure covers 536 claims without USDC and 559 with USDC.
- The completion gain came with higher model-plus-service cost: $0.147 versus $0.101 per task, an increase of $0.046 or 45.6%. Checker purchases and web search are excluded from this comparison.

## About this study

The completion and cost snapshot was generated on November 11, 2026 at 21:26 UTC. The quality analysis was added in the November 13, 2026, 16:59 UTC source update.

Each model ran the same 21 tasks five times per condition: 210 runs per model, 735 per condition, 1,470 in total. Headline changes are arithmetic means of the seven model-level means, with equal task weights. Quality values are pooled over the eligible runs or claims.

It tests a budget and tools for purchasing services together. It does not isolate money from the extra capabilities and data those services unlock. Completion checks measure specified requirements; they do not establish creative quality or whole-job success.

The task set favors situations where paid access may help. The free baseline has search and read-only fetching, and some checkers can share a data source with the agent.

Quality comparisons are descriptive; statistical significance has not been established. One task (compute) is constrained by the free baseline’s lack of code execution; excluding it, the completion gain is 28.7 percentage points.

**Spend on this study**

The 1,470 runs shown here cost $238.28 across the conditions. This research expense total is separate from the model-plus-service comparison above.

- [Study spend CSV](https://agentic-works.circle.com/experiments/funded-agent-index/data/study-spend.csv)

**Study spend by model**

The 1,470 runs shown here cost $238.28 across both conditions, averaging $34.04 per model. This expense ledger includes $136.26 in model tokens, $46.34 in agent purchases, $6.84 in checker purchases and $48.84 in web search. It is distinct from the comparative model-plus-service cost metric.

## Give your agent USDC

Copy this prompt into your agent

```
Run curl -sL https://agents.circle.com/skills/setup.md, and use the returned setup instructions to set up my agent wallet.
```

Circle Agent Stack gives your agent a wallet and a way to discover and pay for services as it works. Start with a task, a small budget and a clear definition of done.

Set up an agent wallet

Have your agent install Circle CLI and the official Circle Skills, to create an agent wallet.

Fund your agent wallet

Fund your wallet with existing USDC on a supported network.

Discover Services

Point your agent to [Circle Agent Marketplace](https://agents.circle.com/services) to discover premium services your agent can pay for.

- [Explore Circle Agent Stack](https://developers.circle.com/agent-stack)
- [Agent wallet quickstart](https://developers.circle.com/agent-stack/agent-wallets/quickstart)

Facilitator Service is offered by Circle Technology Services, LLC, a software provider. Facilitator Service verifies buyer-signed EIP-3009 USDC payment authorizations under the x402 protocol only — it does not hold, control, or transmit user funds, verify transaction purpose, or confirm agent authority scope. On Base and Polygon, settlements are subject to blockchain reorganization risk; Circle does not guarantee fund receipt. Arc settlements are final. Use of the Facilitator Service is subject to Circle Console Service Terms and Developer Terms. Not reviewed or approved by any regulatory authority. Features may change at any time. Nothing herein constitutes a commitment, warranty, or guarantee. Use involves risk.  
Circle Technology Services, LLC (“CTS”) is a software provider and does not provide financial, advisory, or marketplace services. Agent Stack enables users and developers to interact with third-party applications and services, which are not controlled by Circle. Transactions initiated by agents are executed based on user-defined permissions and may occur without real-time human review. Circle does not guarantee the performance, availability, or outcomes of any third-party services or agent-initiated transactions. Users are solely responsible for their use of these tools and for evaluating associated risks. For additional details, please see the Agent Platform Terms of Use available at agents.circle.com/terms-of-use  
Live demo with Circle-funded wallets. Not advice. Onchain transactions are irreversible. See important disclosures on the corresponding websites for any Circle products and services referenced here: Circle Agent Stack, Circle CLI and Circle Agent Marketplace.

## Endpoints
- https://agentic-works.circle.com/experiments/funded-agent-index.md  # this view, as Markdown
- https://agentic-works.circle.com/experiments/funded-agent-index  # the same page for people
- https://agentic-works.circle.com/llms.txt  # the site index for agents

## Disclaimer
Circle Agentic Works is Circle’s public effort for the agentic economy: shipping experiments, demos, and research in the open. Everything published by Circle Agentic Works is experimental and is provided for informational, educational, and demonstration purposes only.

Experiments, demos, code, tools, and data published here are not Circle products or services, are not production-ready, are provided “as is” and “as available” without warranties of any kind, and may be changed, broken, paused, or withdrawn at any time without notice. Nothing here is a commitment by Circle Internet Group, Inc. or its affiliates (“Circle”) to build, launch, support, or maintain any product, feature, integration, or relationship, and publication of an experiment does not mean it will become a Circle product.

Nothing here constitutes legal, financial, investment, tax, or other advice, or an offer, solicitation, or recommendation to buy, sell, or hold any asset or to use any service. Any use of code, demos, tools, or data is at your own risk and is subject to the applicable license and terms. For references to Circle-generated products and features, see corresponding disclosures. References to third parties, their products, or services are for illustration only and do not imply endorsement, partnership, or affiliation unless expressly stated. Third-party data and statements are attributed to their sources and have not been independently verified by Circle, and Circle makes no representation as to their accuracy or completeness. Circle undertakes no obligation to update this content.