OpenAI and Ligo published BioBasicsBench, 100 tasks about raw wet-lab evidence, including the times the data cannot support a conclusion. It scores agents. It does not hand you a lab partner.
Why it matters. The benchmark scores agents on raw wet-lab evidence. It does not give you a lab partner.
For people evaluating lab agents, not people running a bench. Recheck if a lab tool you already use publishes a score on this benchmark.
Google published a health-AI evaluation framework written with outside clinicians. It looks at accuracy, safety, and usefulness, using benchmarks, autoraters, and expert review. The post shares the method. It does not publish a score for a model you can buy.
Why it matters. Google published the method with outside clinicians. The post does not publish a score for a model you can buy.
For clinicians and buyers waiting on a health-model score. Recheck if Google publishes a score for a model you can actually use.
OpenAI says a research pipeline produced working local privilege-escalation exploits for 948 Linux kernel issues and drafted patches. The post puts that run at $1,213,430 in API credits. It is a controlled lab result. You cannot point this at the office server.
Why it matters. The post describes a controlled lab run, with a stated API-credit cost. You cannot point it at the office server.
For security teams, not people who administer an office server. Recheck if OpenAI releases a tool a customer can run outside that lab.
Ai2's StateBench checks whether a model can track changing facts across a long document, at four levels of difficulty. Frontier models handle the easy levels and break down on the hardest long ones. It is a benchmark. It will not file your contracts.
Why it matters. Frontier models handle the easy levels and break down on the hardest long ones. It will not file your contracts.
For people judging long-document models. Recheck if a product you use cites a StateBench result for the documents you care about.
LifeVal is OpenAI's evaluation of everyday tasks, from childcare and budgets to government paperwork. The post explains how the task list was built. It is a measurement project. You cannot switch it on in ChatGPT.
Why it matters. LifeVal is a measurement project. You cannot switch it on in ChatGPT.
For people looking for a ChatGPT switch for everyday errands. Recheck if OpenAI ships a product setting based on this evaluation.
Corrections: if a fact is wrong, email hello@loadedtokens.com.
What happened
Published news
No published issue in this topic yet. We do not backfill old news to make the shelf look busy. The explainers and profiles below are the context we do have.
