Labs fund most of the evals economy, and buyer-paid testing stays the smallest part

"Evals" covers eight markets that share one idea but differ in who pays

Builders pay for tools, security teams pay for red-teaming, and frontier labs pay for data and tests. Each segment has its own buyer, product and price logic.

We split the market into eight segments. Builder tools and security sell to enterprises. Expert data, RL environments and leaderboards sell mostly to frontier labs. Safety evaluators live on grants, lab contracts and, now, government tenders.

The word "evals" hides two economies. Labs buy evaluation inputs at scale: experts, graded tasks and pre-release tests. Enterprises buy tooling to check their own apps. The two rarely share buyers, prices or vendors.

Labs pay suppliers billions, while enterprises pay tool vendors millions

The money flows show two economies. The largest flows bypass enterprise buyers completely, and the party under test often pays the tester.

Lab spending on expert data dwarfs every other flow. No lab discloses its human-data spending. One forecast puts the top five frontier developers at $5–10B in 2026.^{[1][1] Lab human-data forecast FutureSearch Estimate} The expert-data vendors' summed gross run rates exceed $6B, which fits that range.

Two patterns stand out. First, the party under test often pays the tester. Second, the largest flows bypass enterprise buyers completely. Investors and acquirers move large sums too, but those buy equity, not evaluations.

Builder tools are small, and most of the large ones have been sold

This segment sells tools to the teams that build AI apps and agents. The tools trace calls, run test sets, score outputs with judges and alert on drops.

Only two large independents remain. IVP led LangChain's $125M Series B at $1.25B in October 2025.^{[17][17] LangChain at $1.25B TechCrunch Secondary} Four sources put LangSmith-driven ARR at $12–16M in mid-2025.^{[20][20] LangChain about to become a unicorn TechCrunch Secondary} ICONIQ led Braintrust's $80M Series B in February 2026, at a reported $800M.^{[18][18] Braintrust Series B Axios Pro Secondary} Notion and Stripe use Braintrust.^{[22][22] Notion and Stripe trust Braintrust Forbes Australia Secondary}

Prices run far ahead of revenue. LangChain's $1.25B valuation is 80 to 100 times its mid-2025 ARR of $12–16M, and Dynatrace paid about 23 times Arize's ARR of roughly $40M (our estimates; Arize's ARR is implied by Dynatrace's growth guidance).^{[17][17] LangChain at $1.25B TechCrunch Secondary[20][20] LangChain about to become a unicorn TechCrunch Secondary[41][41] Acquisition of Arize Dynatrace Vendor[58][58] Dynatrace Q2 FY2026 results Quantisnow Secondary} Braintrust's valuation rose about fivefold in 16 months, from about $150M at its Series A to a reported $800M.^{[18][18] Braintrust Series B Axios Pro Secondary} Datadog had over 1,000 LLM Observability customers at Q4 2025, per one weak summary.^{[50][50] Datadog Q4 2025 summary Fintool Single source}

Five well-known independents left the pool in 2025 and 2026. CoreWeave bought Weights & Biases for a reported $1.7B, closing on 5 May 2025.^{[35][35] CoreWeave acquires Weights & Biases TechCrunch Secondary[36][36] W&B acquisition complete CoreWeave Primary} ClickHouse bought Langfuse on 16 January 2026.^{[37][37] ClickHouse buys Langfuse SiliconANGLE Secondary} Cisco closed Galileo on 22 May 2026.^{[57][57] Galileo acquisition Cisco Primary} Dynatrace closed Arize for $915M on 1 October 2026.^{[42][42] Dynatrace completes Arize deal engineering.com Secondary} CrowdStrike agreed to buy Fiddler on 3 November 2026.^{[45][45] Acquisition of Fiddler CrowdStrike Vendor}

One leader pivoted and one shut down. Patronus AI raised a $50M Series B in June 2026 and now sells "Digital World Models" for agent training.^{[46][46] Patronus AI Series B VCA Online Secondary} Its old evaluation products "are all retired".^{[47][47] Patronus AI docs Primary} Anthropic hired the Humanloop team in August 2025, and Humanloop will "sunset" its platform.^{[48][48] Anthropic nabs Humanloop team TechCrunch Secondary[49][49] Humanloop homepage Primary}

The incumbents give tracing away. Datadog offers LLM Observability free up to 40,000 spans and states "no separate product fee" for evaluations.^{[62][62] Datadog LLM Observability Primary} Its spans "nearly tripled quarter-over-quarter" in Q1 2026.^{[63][63] Datadog Q1 2026 earnings call The Motley Fool Secondary} Every large APM vendor now sells tracing and judge-based scoring.

Tracing has no moat. Citi Ventures calls the twin threats "Death from Below" and "Death from Above": open source from below, APM bundles from above.^{[67][67] LLM observability Citi Ventures Secondary} A newer wave focuses on agents in production instead. Raindrop has raised $50M, and Judgment Labs $32M.^{[24][24] Raindrop Series A Pulse 2.0 Secondary[25][25] $32M announcement Judgment Labs Vendor}

Security vendors bought the red-teamers, and compliance buyers pay for evidence

Red-teaming is an eval: it tests whether an agent fails under attack. Governance tools mostly consume eval results rather than produce them.

Between August 2024 and November 2026, security vendors bought most independent red-team startups. One public tracker counts 30 agent-security deals since 2024, 14 of them in 2026. It says buyers acquired 22 of the 61 vendors on its map.^{[87][87] Agent Security Map Single source} Filed prices often sit below headlines. F5 announced CalypsoAI at $180M, but its filing shows $145.2M.^{[72][72] F5 to acquire CalypsoAI CyberScoop Secondary}

The promptfoo deal matters most for evals. Promptfoo had raised $23M and claimed use by more than 25% of the Fortune 500.^{[69][69] OpenAI acquires Promptfoo Yahoo Finance Secondary} OpenAI folds it into its Frontier platform for red-teaming and compliance monitoring. The code stays open source under MIT.^{[70][70] promptfoo README Primary}

Independent security firms still raise large rounds. Zenity raised $125M in August 2026, and HiddenLayer $100M in September 2026 with ARR up more than tenfold.^{[16][16] Zenity Series C SecurityWeek Secondary[15][15] HiddenLayer Series B Pulse 2.0 Secondary} Most vendors here do not publish prices.

Compliance buyers pay for evidence, not test engines. Gartner says AI governance spending will reach $492M in 2026 and pass $1B by 2030.^{[9][9] Insurance-Canada.ca quoting Gartner Secondary} Some outlets label the same figure "AI data governance", so the scope is uncertain. Forrester's Q3 2025 Wave named Credo AI and IBM watsonx.governance as Leaders.^{[90][90] Credo AI named a Leader Business Wire Vendor[91][91] AI Governance Solutions Q3 2025 Forrester Wave Secondary}

Certification joins the segment. AIUC raised $40M in September 2026 and runs about 5,000 adversarial tests per agent.^{[11][11] AIUC Series A TechCrunch Secondary} The AICPA and AIUC launched a joint SOC 2 plus AIUC-1 offering on 19 October 2026.^{[96][96] AICPA AI assurance offering Journal of Accountancy Secondary} In Europe, QuantPi became TÜV SÜD's first CARAT partner lab.^{[95][95] TÜV SÜD and QuantPi PresseBox Vendor}

The largest money pool is lab-paid expert work, environments and safety tests

Labs buy expert data for training and evaluation in the same contracts. The same experts write eval sets, grade outputs and build environments.

Expert-data vendors carry the largest valuations in evals. Mercor raised at $10B in October 2025, and its CEO says its run rate crossed $2B in June 2026.^{[12][12] Mercor run rate TechCrunch Secondary} Surge AI earned $1.2B in 2024 with about 250 staff, without outside capital.^{[125][125] Surge AI founder profile Forbes Australia Secondary} Meta paid about $14.3B for 49% of Scale in June 2025.^{[109][109] Meta invests in Scale AI Built In Secondary}

Thin margins sit behind the big run rates. Mercor passes 60–70% of revenue to contractors.^{[134][134] Mercor H1 2026 AI Weekly Secondary} Mercor postings for lawyers and physicians cluster at $100–150 an hour.^{[140][140] Mercor law expert jobs aitrainer.work Secondary} Buyer concentration is extreme: OpenAI no longer works with Surge, and one lab's exit can move a vendor's revenue sharply.^{[125][125] Surge AI founder profile Forbes Australia Secondary}

RL environments became a funded category in 2025 and 2026. Epoch AI found that a typical task costs $200–2,000. A website replica costs about $20,000, and one high-fidelity Slack clone was priced near $300,000.^{[112][112] State of RL environments Epoch AI Primary} Google struck a licence-and-hire deal with Mechanize, reported at over $1.5B.^{[142][142] Google and Mechanize The Next Web Secondary} Prime Intellect raised $130M at $1B and sells to enterprises, not labs.^{[7][7] Prime Intellect Series A TechCrunch Secondary}

Labs value environment know-how more than environment files. One lab researcher said many vendors "vibe code a buggy website which isn't useful".^{[112][112] State of RL environments Epoch AI Primary} Talent deals and expert-network acquisitions squeeze the pure environment start-ups. Mercor bought RL-gym builder Deeptune in July 2026.^{[135][135] Acquisition of Deeptune Mercor Primary}

Funding source shapes a safety evaluator's freedom. METR raised about $71M in six months and takes no money from frontier AI companies, but relies on their free tokens.^{[4][4] Funding update August 2026 METR Primary} Irregular raised $80M at a reported $450M and runs pre-release cyber tests for OpenAI, Anthropic and Google DeepMind.^{[155][155] Irregular raises $80M TechCrunch Secondary} The EU AI Office now pays FAR.AI, SecureBio, METR and Epoch through its tender.^{[6][6] EU AI Office tender award notice 864574-2025 TED Primary}

Platforms absorbed 28 eval and AI-security companies in 27 months, and the pace rose in 2026

Larger firms bought or absorbed more than 30 eval-related companies between 2024 and 2026. Our timeline covers eval tools and AI security: 28 deals and 7 shutdowns or pivots. Data and environment deals take the total past 30.

Four kinds of buyer did the absorbing. Security vendors wanted red-teaming and guardrails for the CISO suite. Observability and data platforms wanted evals as a telemetry feature. Model labs wanted talent, tools and data. Data incumbents wanted environments and expert networks.

Prices stayed modest outside a few headline deals. Only W&B, Arize and Protect AI cleared $600M. Many eval and guardrail startups sold for under $250M, and most buyers did not disclose terms.

OpenAI even turned against its own hosted product. It deprecated its Evals platform on 3 June 2026 and will shut it down on 30 November 2026. Its guide points users to Promptfoo.^{[168][168] OpenAI API deprecations Primary} Stanford also put HELM into maintenance mode on 1 June 2026.^{[166][166] HELM maintenance mode Stanford CRFM Primary}

Evals end up as a feature inside a bigger platform. Standalone vendors survive by going beyond tracing, into simulation, security or governance.

Nobody publishes a trusted dollar size for evals, and the visible pieces are small

Analyst categories now exist, but no analyst publishes a trusted dollar size for evals alone. The figures that exist measure different things.

Gartner created the category in February 2026. Its Market Guide for "AI Evaluation and Observability Platforms" appeared on 2 February 2026.^{[169][169] Market Guide abstract Gartner Primary} Gartner predicts that 60% of software engineering teams will adopt these platforms by 2028, up from 18% in 2025.^{[170][170] Gartner Market Guide reprint Comet Vendor} We could not read the full guide or its vendor list.

Report mills publish the big numbers. The Business Research Company puts LLM observability at $2.69B in 2026.^{[175][175] LLM observability report The Business Research Company Estimate} Meticulous Research puts AI observability at $1.78B.^{[176][176] AI observability market Meticulous Research Estimate} Neither shows a method we could check.

Our estimate is far smaller. Builder evals plus LLM observability likely total a few hundred million dollars of vendor revenue in 2026. The known pieces are small: LangChain had $12–16M of ARR in mid-2025, and Arize about $40M.^{[20][20] LangChain about to become a unicorn TechCrunch Secondary[58][58] Dynatrace Q2 FY2026 results Quantisnow Secondary} The lab-paid supply side looks ten to thirty times larger, by our estimate.

Investors frame the opportunity beyond tracing. Citi Ventures wrote that "the core tracing layer has no durable moat".^{[67][67] LLM observability Citi Ventures Secondary} It invests in ClickHouse and Straiker, so it has a stake in that view. Bessemer predicts that "AI evals will go private, grounded, and trusted".^{[55][55] State of AI 2025 Bessemer Secondary} Lightspeed backs the most companies in the segment, including Raindrop, Judgment, Fiddler and Arena.^{[23][23] Seed round Raindrop Vendor[25][25] $32M announcement Judgment Labs Vendor[3][3] Series B announcement Arena Vendor}

Customers buy observability first, and the price of a score is falling toward the cost of tokens

Customers buy observability first, evals second and governance third. Cloud platforms now give evals away at token cost, so paid value moves elsewhere.

Most builders trace, but only half run offline evals. LangChain surveyed 1,340 agent builders in late 2025.^{[178][178] State of Agent Engineering LangChain Vendor} 89% have some observability, but only 52.4% run offline evals and 37.3% online evals. Human review still edges out LLM-as-judge. LangChain sells LangSmith, so its survey has an interest.

Buyers trust their own tests over public benchmarks. One CIO told a16z: "you still need to assess yourself".^{[179][179] AI in the enterprise 2025 a16z Secondary} Datadog's span growth shows that buying follows production traffic.^{[63][63] Datadog Q1 2026 earnings call The Motley Fool Secondary}

Every cloud and model platform now ships evals at token cost. AWS made AgentCore Evaluations generally available on 31 March 2026, at $1.50 per 1,000 custom evaluations plus model usage.^{[180][180] AgentCore Evaluations GA AWS Primary[181][181] AWS AgentCore pricing Primary} Microsoft bills Foundry evaluations by tokens "with no surcharge".^{[182][182] Foundry observability pricing Azure Primary} Google added Agent Evaluation and Agent Simulation at Next '26.^{[106][106] Next 2026 wrap-up Google Cloud Vendor}

Independents meter usage and offer free tiers. Braintrust meters gigabytes and scores, while Arize, Opik, Datadog and Splunk meter spans. Seat pricing is rare. Self-serve plans run from $19 to about $300 a month, and enterprise deals are custom.

Judge cost decides how much traffic gets scored. Langfuse estimates $0.01–0.10 per judged item, and vendors advise sampling 1–20% of traffic.^{[189][189] LLM-as-a-judge Langfuse Primary} Vendors answer with small judge models. CrowdStrike claims eval costs fall "up to 98%" with Fiddler's models.^{[45][45] Acquisition of Fiddler CrowdStrike Vendor}

Simulation is where paid value moves next. Tracing became free, but realistic simulated users and tools still cost real work. Coval leads voice-agent testing after a $28M Series A in June 2026.^{[99][99] Coval Series A The Next Web Secondary} Pricing follows call volume: Cekura charges $0.05 per monitored call and Future AGI $0.08 per voice minute.^{[103][103] Cekura pricing Primary[105][105] Future AGI pricing Primary}

Leaderboards became businesses, and every business model carries a conflict of interest

Public leaderboards became venture-backed businesses in 2025 and 2026. Two models compete: the ranked lab pays, or the buyer pays. The answer to "who pays?" decides how independent a result can be.

Arena turned a public board into a $3.1B company. Lightspeed and Khosla co-led its $200M Series B on 8 October 2026. Arena says it "exceeded $100M in annualized revenue" and calls itself "a neutral third party".^{[3][3] Series B announcement Arena Vendor} Its first product sells evaluation results and feedback samples to model developers.^{[14][14] LMArena Series A SiliconANGLE Secondary}

Critics question that neutrality. "The Leaderboard Illusion" found that Meta tested 27 private variants before Llama 4.^{[117][117] The Leaderboard Illusion Secondary} The Llama 4 model that topped the board differed from the model Meta released.^{[118][118] Criticism of Chatbot Arena Simon Willison Secondary}

Buyer-paid benchmarkers take the opposite route. Artificial Analysis keeps its public site free and sells enterprise subscriptions. Its co-founder says "no one pays to be on the website", and it tests through "mystery shopper" accounts.^{[108][108] Artificial Analysis interview Latent Space Secondary} Vals AI raised $40M at $400M; its CEO compares its model to "a student paying to take the SAT".^{[10][10] Series A Vals AI Vendor[120][120] Vals AI profile Carly Secondary}

Three conflicts matter most. First, the ranked party often pays the ranker. Second, the owner of a board sometimes competes on it. Meta owns about 49% of Scale, and Google's Gemini competes on Kaggle Game Arena.^{[109][109] Meta invests in Scale AI Built In Secondary[114][114] Kaggle Game Arena updates Google Secondary} Third, builder tools leave the builder as its own judge.

Independence has become a product feature. Arena claims neutrality, Artificial Analysis uses mystery shoppers, and METR refuses lab money.^{[4][4] Funding update August 2026 METR Primary} Amodei proposes embedded evaluators who may publish without editorial control.^{[152][152] We must pace the frontier Dario Amodei Primary}

Buyer-paid evaluation stays small. The largest buyer-paid benchmarker, Vals, carries a $400M valuation. Expert-data vendors, all lab-paid, report billions of run rate.

Sources

Every number keeps its date and source. Badges mark source quality: primary, secondary, vendor (a claim about the publisher's own product), single source, or estimate.

1. [1]
   Lab human-data forecast, FutureSearch. It estimates 2026 human-data spending by the top five labs. Estimate futuresearch.ai
2. [2]
   Silicon Valley bets big on environments, TechCrunch. It reports Anthropic's discussed RL-environment budget, citing The Information. Secondary techcrunch.com
3. [3]
   Series B announcement, Arena. It gives Arena's valuation, revenue and usage in October 2026. Vendor arena.ai
4. [4]
   Funding update August 2026, METR. It lists METR's new commitments, funders and funding policy. Primary metr.org
5. [5]
   FAR.AI Form 990, ProPublica. It gives FAR.AI's 2024 revenue, expenses and net assets. Primary projects.propublica.org
6. [6]
   EU AI Office tender award notice 864574-2025, TED. It lists the six lots, winners and contract values. Primary ted.europa.eu
7. [7]
   Prime Intellect Series A, TechCrunch. It reports the round, run rate and customers. Secondary techcrunch.com
8. [8]
   AI TRiSM, Software Strategies Blog. It reports Gartner's AI cybersecurity spending forecast at second hand. Secondary softwarestrategiesblog.com
9. [9]
   Insurance-Canada.ca quoting Gartner. It quotes Gartner's forecast for AI governance platform spending. Secondary insurance-canada.ca
10. [10]
    Series A, Vals AI. It announces the round, valuation and revenue growth. Vendor vals.ai
11. [11]
    AIUC Series A, TechCrunch. It describes AIUC's round, tests, auditors and customers. Secondary techcrunch.com
12. [12]
    Mercor run rate, TechCrunch. It reports Mercor's valuation, run rate and funding talks. Secondary techcrunch.com
13. [13]
    Snorkel AI Series E, Implicator. It reports the round, valuation, run rate and pivot. Secondary implicator.ai
14. [14]
    LMArena Series A, SiliconANGLE. It reports Arena's history, rounds and first product. Secondary siliconangle.com
15. [15]
    HiddenLayer Series B, Pulse 2.0. It reports the $100M round and ARR growth. Secondary pulse2.com
16. [16]
    Zenity Series C, SecurityWeek. It reports the $125M round. Secondary securityweek.com
17. [17]
    LangChain at $1.25B, TechCrunch. It reports the Series B and its investors. Secondary techcrunch.com
18. [18]
    Braintrust Series B, Axios Pro. It reports the $80M round at an $800M valuation. Secondary axios.com
19. [19]
    About page, UK AI Security Institute. It states the yearly funding and staff. Primary aisi.gov.uk
20. [20]
    LangChain about to become a unicorn, TechCrunch. It cites four sources on LangSmith ARR in mid-2025. Secondary techcrunch.com
21. [21]
    Braintrust financials, CB Insights. It lists Braintrust's funding rounds and investors. Secondary cbinsights.com
22. [22]
    Notion and Stripe trust Braintrust, Forbes Australia. It names Braintrust customers. Secondary forbes.com.au
23. [23]
    Seed round, Raindrop. It announces the $15M seed led by Lightspeed. Vendor raindrop.ai
24. [24]
    Raindrop Series A, Pulse 2.0. It reports the CRV-led round and $50M total. Secondary pulse2.com
25. [25]
    $32M announcement, Judgment Labs. It announces seed and Series A funding led by Lightspeed. Vendor judgmentlabs.ai
26. [26]
    HoneyHive launch, VCA Online. It reports HoneyHive's $7.4M seed led by Insight Partners. Secondary vcaonline.com
27. [27]
    HoneyHive homepage. It names Commonwealth Bank as a customer. Vendor honeyhive.ai
28. [28]
    Confident AI pricing. It lists the free, Starter, Team and Enterprise plans. Primary confident-ai.com
29. [29]
    Openlayer Series A, SiliconANGLE. It reports the $14.5M round led by Race Capital. Secondary siliconangle.com
30. [30]
    Gartner Market Guide, Openlayer. It announces Openlayer's inclusion in the Gartner guide. Vendor openlayer.com
31. [31]
    Scorecard, Kindred Ventures. It announces Scorecard's $3.75M seed. Vendor kindredventures.com
32. [32]
    Scorecard pricing. It lists the Starter and Growth plans. Primary scorecard.io
33. [33]
    Comet pricing. It lists Opik's open-source, free and Pro tiers. Primary comet.com
34. [34]
    Galtea homepage. It describes Galtea's product, certifications and customer logos. Vendor galtea.ai
35. [35]
    CoreWeave acquires Weights & Biases, TechCrunch. It reports the deal and its reported price. Secondary techcrunch.com
36. [36]
    W&B acquisition complete, CoreWeave. It confirms the close of the deal. Primary coreweave.com
37. [37]
    ClickHouse buys Langfuse, SiliconANGLE. It reports the deal and ClickHouse's $400M round. Secondary siliconangle.com
38. [38]
    Joining ClickHouse, Langfuse. It confirms the deal and the MIT-licensed core. Primary langfuse.com
39. [39]
    Cisco to acquire Galileo, Converge Digest. It gives Galileo's funding history. Secondary convergedigest.com
40. [40]
    Splunk pricing. It lists the entry price of Splunk Agent Observability. Primary splunk.com
41. [41]
    Acquisition of Arize, Dynatrace. It gives the price, ARR effect and market forecast. Vendor dynatrace.com
42. [42]
    Dynatrace completes Arize deal, engineering.com. It reports the close on 1 October 2026. Secondary engineering.com
43. [43]
    Fiddler Series C, BusinessWire. It reports the $30M round and $100M total. Secondary businesswire.com
44. [44]
    Cheapest judgment, Charu Gupta. This Substack reports deal prices and closes that we could not confirm. Single source charugupta.substack.com
45. [45]
    Acquisition of Fiddler, CrowdStrike. It describes the deal, the product plan and cost claims. Vendor crowdstrike.com
46. [46]
    Patronus AI Series B, VCA Online. It reports the round, investors and the pivot to world models. Secondary vcaonline.com
47. [47]
    Patronus AI docs. They list the old evaluation products as retired. Primary docs.patronus.ai
48. [48]
    Anthropic nabs Humanloop team, TechCrunch. It reports the acqui-hire. Secondary techcrunch.com
49. [49]
    Humanloop homepage. It announces the platform sunset and a migration guide. Primary humanloop.com
50. [50]
    Datadog Q4 2025 summary, Fintool. It reports Datadog's LLM Observability customer count. Single source fintool.com
51. [51]
    LangChain funding, Revenue Memo. It reports an unconfirmed secondary-market valuation. Single source revenuememo.com
52. [52]
    LangChain pricing. It lists LangSmith plans and deployment options. Primary langchain.com
53. [53]
    Braintrust pricing. It lists the Starter, Pro and Enterprise plans. Primary braintrust.dev
54. [54]
    Raindrop Series A, Dealroom. It reports a different size for the Series A. Secondary dealroom.co
55. [55]
    State of AI 2025, Bessemer. It sets out Bessemer's view on evals and names companies. Secondary bvp.com
56. [56]
    CoreWeave Forge pricing. It lists the current price of W&B Weave. Primary coreweave.com
57. [57]
    Galileo acquisition, Cisco. It records Cisco's purchase of Galileo. Primary cisco.com
58. [58]
    Dynatrace Q2 FY2026 results, Quantisnow. It mirrors Dynatrace's ARR guidance. Secondary quantisnow.com
59. [59]
    Mintlify acquires Helicone (3 Mar 2026), Mintlify. It announces Helicone's move to Mintlify and its maintenance mode. Primary mintlify.com
60. [60]
    Acquisition of Quotient AI, Databricks. It announces the deal for agent evaluation. Primary databricks.com
61. [61]
    AI product changes 2026, sota.cc. It reports ServiceNow's purchase of Traceloop. Single source sota.cc
62. [62]
    Datadog LLM Observability. It gives pricing and the policy on evaluation fees. Primary datadoghq.com
63. [63]
    Datadog Q1 2026 earnings call, The Motley Fool. This secondary transcript reports span growth and revenue growth. Secondary fool.com
64. [64]
    Datadog 8-K exhibit. It covers Datadog's purchase of Adaptive ML. Primary sec.gov
65. [65]
    Arize Series C, HPCwire. It reports the $70M round and its investors. Secondary hpcwire.com
66. [66]
    New Relic AI Evaluation, byteiota. It reports New Relic's planned evaluation feature. Single source byteiota.com
67. [67]
    LLM observability, Citi Ventures. It argues that tracing has no durable moat. Secondary citi.com
68. [68]
    FY2025, Palo Alto Networks 10-K. It records the price paid for Protect AI. Primary sec.gov
69. [69]
    OpenAI acquires Promptfoo, Yahoo Finance. It gives Promptfoo's funding, valuation and adoption. Secondary finance.yahoo.com
70. [70]
    promptfoo README. It states that Promptfoo now belongs to OpenAI and stays MIT-licensed. Primary github.com
71. [71]
    Agent Security consolidation tracker. One compiler lists agent-security deals with filed and reported prices. Single source guptadeepak.com
72. [72]
    F5 to acquire CalypsoAI, CyberScoop. It reports the announced price. Secondary cyberscoop.com
73. [73]
    SPLX, Zscaler investor relations. It announces Zscaler's purchase of SPLX. Primary ir.zscaler.com
74. [74]
    Acquisition of Dreadnode, CrowdStrike. It describes Dreadnode's red-teaming and the deal. Vendor crowdstrike.com
75. [75]
    Noma Security Series B, SecurityWeek. It reports the $100M round. Secondary securityweek.com
76. [76]
    Straiker Series A, FinTech Global. It reports the round, total and customers. Secondary fintech.global
77. [77]
    Gray Swan Series A, Pulse 2.0. It reports the round and the lab customers. Secondary pulse2.com
78. [78]
    Gray Swan red-team contest, Fortune. It profiles Gray Swan's agent jailbreak contest. Secondary fortune.com
79. [79]
    Mindgard Series A, SecurityWeek. It reports the $30M round. Secondary securityweek.com
80. [80]
    SentinelOne to acquire Prompt Security, Business Wire. It announces the deal. Secondary businesswire.com
81. [81]
    Deepchecks price, Dealroom citing Calcalist. It estimates the price at up to $20M. Estimate dealroom.co
82. [82]
    TrojAI acquisition, A10 Networks. It announces the deal of 15 June 2026. Secondary businesswire.com
83. [83]
    Acquisition of Enkrypt AI, Anaconda. It announces the deal. Primary anaconda.com
84. [84]
    Fortinet acquires Virtue AI, SecurityWeek. It reports the deal and its immaterial size. Secondary securityweek.com
85. [85]
    Guardrails AI joins Harvey, Harvey. It announces the deal. Primary harvey.ai
86. [86]
    Beacon Software and Haize Labs, Osler. It describes the deal and the new research group. Secondary osler.com
87. [87]
    Agent Security Map. It counts deals and acquired vendors across the segment. Single source guptadeepak.com
88. [88]
    Series A announcement, Promptfoo. It reports the round from Insight Partners and a16z. Vendor promptfoo.dev
89. [89]
    Enkrypt AI pricing. It lists Enkrypt's monthly plans. Primary enkryptai.com
90. [90]
    Credo AI named a Leader, Business Wire. It reports Credo AI's Forrester Wave placement. Vendor businesswire.com
91. [91]
    AI Governance Solutions Q3 2025, Forrester Wave. It evaluates ten AI governance providers. Secondary forrester.com
92. [92]
    Forrester Wave Strong Performer, Collibra. It reports Collibra's Wave placement. Vendor collibra.com
93. [93]
    CHF 8.7M round, Modulos. It announces Modulos's funding. Vendor modulos.ai
94. [94]
    LatticeFlow COMPL-AI, SiliconANGLE. It describes LatticeFlow's EU AI Act framework. Secondary siliconangle.com
95. [95]
    TÜV SÜD and QuantPi, PresseBox. It announces QuantPi as a TÜV SÜD partner lab. Vendor pressebox.com
96. [96]
    AICPA AI assurance offering, Journal of Accountancy. It reports the joint SOC 2 and AIUC-1 offering. Secondary journalofaccountancy.com
97. [97]
    Voluntary AI certification, TÜV NORD. It announces the three-tier TÜV AI certificate. Vendor tuev-nord-group.com
98. [98]
    New AI schemes, BSI. It announces three AI certification schemes. Vendor bsigroup.com
99. [99]
    Coval Series A, The Next Web. It reports Coval's $28M round. Secondary thenextweb.com
100. [100]
     Seed funding, Hamming. It announces Hamming's $3.8M seed. Vendor hamming.ai
101. [101]
     Hamming vs Cekura, Coval. This competitor blog claims a Hamming leadership change. Single source coval.ai
102. [102]
     Fundraise, Cekura. It announces Cekura's seed round. Vendor cekura.ai
103. [103]
     Cekura pricing. It lists the per-call monitoring price. Primary cekura.ai
104. [104]
     Future AGI pre-seed, Unite.AI. It reports Future AGI's $1.6M round. Secondary unite.ai
105. [105]
     Future AGI pricing. It lists the per-minute voice price. Primary futureagi.com
106. [106]
     Next 2026 wrap-up, Google Cloud. It lists the new agent evaluation and simulation features. Vendor cloud.google.com
107. [107]
     Coval pricing. It lists Coval's monthly plans. Primary coval.ai
108. [108]
     Artificial Analysis interview, Latent Space. The founders explain the business model and independence policy. Secondary latent.space
109. [109]
     Meta invests in Scale AI, Built In. It reports Meta's stake and Scale's valuation. Secondary builtin.com
110. [110]
     Singapore IMDA MoU, Scale. It announces Scale's evaluation agreement with IMDA. Vendor scale.com
111. [111]
     About, Epoch AI. It describes Epoch's funding, services and benchmarks. Primary epoch.ai
112. [112]
     State of RL environments, Epoch AI. It reports prices and buyer views from interviews. Primary epoch.ai
113. [113]
     Holistic Agent Leaderboard, HAL. It describes HAL's current reliability work. Primary hal.cs.princeton.edu
114. [114]
     Kaggle Game Arena updates, Google. It announces new games and calls the arena independent. Secondary hn.jakechampion.name
115. [115]
     OpenRouter Series B, Shopifreaks. It reports the round and the reported valuation. Secondary shopifreaks.com
116. [116]
     Arena blog index. It lists Arena's 2026 products and research. Vendor arena.ai
117. [117]
     The Leaderboard Illusion. This paper documents selective private testing on Chatbot Arena. Secondary arxiv.org
118. [118]
     Criticism of Chatbot Arena, Simon Willison. He notes that the top Llama 4 entry differed from the release. Secondary simonwillison.net
119. [119]
     Vals AI Series A, Pulse 2.0. It reports a16z's comparison of Vals to Moody's. Secondary pulse2.com
120. [120]
     Vals AI profile, Carly. It quotes the Vals CEO on who pays for tests. Secondary usecarly.com
121. [121]
     SEAL leaderboards, Scale. It launches Scale's private-set leaderboards. Primary scale.com
122. [122]
     Scale Labs leaderboard. It lists Scale's current benchmarks and models. Primary labs.scale.com
123. [123]
     ICLR 2026 paper, HAL. It reports HAL's rollouts, cost and findings. Primary proceedings.iclr.cc
124. [124]
     Surge AI raise, The Star via Reuters. It reports Surge's planned raise and valuation. Secondary thestar.com.my
125. [125]
     Surge AI founder profile, Forbes Australia. It reports Surge's revenue, customers, pay and prices. Secondary forbes.com.au
126. [126]
     Next era for 2026, Scale. It reports Scale's 2025 bookings and government work. Vendor scale.com
127. [127]
     Handshake ARR, Dealroom citing The Information. It reports Handshake's gross and net AI-training revenue. Secondary dealroom.co
128. [128]
     micro1 crosses $100M ARR and fields $2.5B offers, Forbes. It reports micro1's revenue and offers. Secondary forbes.com
129. [129]
     micro1 revenue, Sacra. It estimates micro1's annualised revenue. Estimate sacra.com
130. [130]
     Turing Series E, TechCrunch. It reports the $111M round at a $2.2B valuation. Secondary techcrunch.com
131. [131]
     Turing revenue, Sacra. It estimates Turing's annualised revenue. Estimate sacra.com
132. [132]
     Labelbox Series E, Unite.AI. It reports a round above $100M at a $1.25B pre-money valuation. Secondary unite.ai
133. [133]
     Invisible Technologies valuation, Bloomberg Tax. It reports Invisible's round and valuation. Secondary news.bloombergtax.com
134. [134]
     Mercor H1 2026, AI Weekly. It reports Mercor's gross revenue and contractor share. Secondary aiweekly.co
135. [135]
     Acquisition of Deeptune, Mercor. It announces Mercor's purchase of Deeptune. Primary mercor.com
136. [136]
     Scale AI appoints Francis deSouza, PR Newswire. It announces Scale's new CEO. Primary prnewswire.com
137. [137]
     RL environments, Scale. It describes Scale's RL-environment work. Primary scale.com
138. [138]
     Handshake, Sacra. It estimates Handshake's gross and net revenue. Estimate sacra.com
139. [139]
     Appen FY25 results, Investor PA. It summarises Appen's FY25 announcement to the ASX. Secondary investorpa.com
140. [140]
     Mercor law expert jobs, aitrainer.work. It lists posted hourly rates for expert work. Secondary aitrainer.work
141. [141]
     AI doctors and lawyers, FourWeekMBA. It reports Bloomberg's findings on Mercor rates. Secondary fourweekmba.com
142. [142]
     Google and Mechanize, The Next Web. It reports Mechanize's funding and the Google deal. Secondary thenextweb.com
143. [143]
     Fleet, rl-list. It aggregates reports on Fleet's revenue and valuation talks. Single source rl-list.com
144. [144]
     Halluminate Series A, The SaaS News. It reports the round and the run-rate claim. Secondary thesaasnews.com
145. [145]
     Plato Series A, Let's Data Science. It reports Plato's round and simulated apps. Secondary letsdatascience.com
146. [146]
     Deeptune Series A, Fortune. It reports the round led by a16z. Secondary fortune.com
147. [147]
     HUD homepage. It describes HUD's environments and DataVendor marketplace. Primary hud.ai
148. [148]
     Veris AI deprecation notice. It announces the retirement of the original simulator. Primary docs.veris.ai
149. [149]
     Google DeepMind and Mechanize, Benzinga. It reports the close of the talent deal. Secondary benzinga.com
150. [150]
     Environments Hub, Prime Intellect. It launches the open environments hub. Primary primeintellect.ai
151. [151]
     RL environment land grab talk, ICML 2026. It lists the invited talk on RL environments. Primary icml.cc
152. [152]
     We must pace the frontier, Dario Amodei. The essay proposes embedded third-party evaluators. Primary darioamodei.com
153. [153]
     Becoming a PBC, Apollo Research. It explains the move and the new seed round. Primary apolloresearch.ai
154. [154]
     Apollo Research homepage. It describes Apollo's red-teaming work and Watcher monitor. Primary apolloresearch.ai
155. [155]
     Irregular raises $80M, TechCrunch. It reports the round and the lab partners. Secondary techcrunch.com
156. [156]
     Agent red-teaming showdown, Gray Swan Arena. It reports attack counts and bounties. Secondary app.grayswan.ai
157. [157]
     Goodfire Series B, Pulse 2.0. It reports the round and valuation. Secondary pulse2.com
158. [158]
     Beacon buys Haize Labs, BetaKit. It reports the deal and Haize's customers. Secondary betakit.com
159. [159]
     AI: 2025 in Review, SecureBio. It lists its benchmarks, lab work and government contracts. Primary securebio.pages.dev
160. [160]
     Redwood Research Form 990, ProPublica. It gives revenue and expenses for 2021 to 2024. Primary projects.propublica.org
161. [161]
     Redwood Research. It lists its control research and recent joint reports with METR. Primary redwoodresearch.org
162. [162]
     Transluce. It describes Docent and the AI Evaluator Forum that it chairs. Primary transluce.org
163. [163]
     Series C, Beacon Software. It announces a $225M round for an AI software roll-up. Secondary businesswire.com
164. [164]
     About, FAR.AI. It describes pre-deployment testing and consulting for frontier developers. Primary far.ai
165. [165]
     ControlArena repository. It describes the AI control library from UK AISI and Redwood. Primary github.com
166. [166]
     HELM maintenance mode, Stanford CRFM. It states the date and the policy. Primary crfm-helm.readthedocs.io
167. [167]
     simple-evals repository, OpenAI. It says the repository no longer gets updates. Primary github.com
168. [168]
     OpenAI API deprecations. It sets the shutdown dates for Evals and points to Promptfoo. Primary developers.openai.com
169. [169]
     Market Guide abstract, Gartner. It names the AI Evaluation and Observability Platforms guide. Primary gartner.com
170. [170]
     Gartner Market Guide reprint, Comet. It quotes the guide's adoption forecast. Vendor comet.com
171. [171]
     Gartner Market Guide, W&B. It hosts a copy of the Gartner report. Vendor wandb.ai
172. [172]
     Hype Cycle for Agentic AI, Gartner. It places agentic AI at the peak of expectations. Primary gartner.com
173. [173]
     Guardian agents prediction, Gartner. It forecasts the guardian agents' share of agentic AI. Primary gartner.com
174. [174]
     Agentic AI cancellations, Gartner. It predicts that firms will cancel many agentic AI projects. Primary gartner.com
175. [175]
     LLM observability report, The Business Research Company. This report mill sizes the LLM observability market. Estimate thebusinessresearchcompany.com
176. [176]
     AI observability market, Meticulous Research. This report mill sizes the AI observability market. Estimate meticulousresearch.com
177. [177]
     State of generative AI 2025, Menlo Ventures. It sizes US enterprise AI spending by layer. Secondary menlovc.com
178. [178]
     State of Agent Engineering, LangChain. This vendor survey reports how builders use observability and evals. Vendor langchain.com
179. [179]
     AI in the enterprise 2025, a16z. Its CIO survey reports how buyers use benchmarks. Secondary a16z.com
180. [180]
     AgentCore Evaluations GA, AWS. It announces general availability and the built-in evaluators. Primary aws.amazon.com
181. [181]
     AWS AgentCore pricing. It lists evaluator prices. Primary aws.amazon.com
182. [182]
     Foundry observability pricing, Azure. It states token billing and free monitoring. Primary azure.microsoft.com
183. [183]
     Copilot Studio Agent Evaluation GA, Microsoft. It announces general availability. Primary techcommunity.microsoft.com
184. [184]
     AI Observability GA, Snowflake. It records general availability in July 2025. Primary docs.snowflake.com
185. [185]
     Introducing AgentKit, OpenAI. It describes eval features at standard API pricing. Primary openai.com
186. [186]
     Fiddler pricing. It lists the per-trace developer price. Primary fiddler.ai
187. [187]
     Langfuse pricing. It lists the Hobby, Core, Pro and Enterprise plans. Primary langfuse.com
188. [188]
     Arize pricing. It lists the Arize AX plans. Primary arize.com
189. [189]
     LLM-as-a-judge, Langfuse. It explains judge costs and sampling. Primary langfuse.com
190. [191]
     Check Point and Deepchecks, Security Boulevard. It reports the purchase of the Deepchecks team and IP. Secondary securityboulevard.com