OpenAI has made its Decisions API available to all customers, stepping into a market for fast, inexpensive AI that recently helped turn TypeSafe into a $7.6 billion company. TypeSafe was founded by a former researcher at the frontier lab. The wider release took effect on Thursday, after the product was announced at OpenAI's developer day on September 29.
TypeSafe's Jev model established a new category of deep learning systems that work as super classifiers. Given a list of options, they pick one quickly and attach a confidence rating to the choice. Developers soon found that these decision models, also called System One models, could stand in for large language models in many workflows, running faster and at much lower cost.
Nikunj Handa, product lead for OpenAI's APIs, told TechCrunch that colleagues reacted to Jev by saying, "wow, we we have like a million use cases for this." His team moved quickly to build a rival. Instead of creating a purpose-calibrated decision model, it took ChatGPT Luna and reconfigured it to run inference faster and at lower cost.
"The big unlock with Decisions API has been the speed, shrinking things from one to three seconds to 100, 120, 180 milliseconds," Handa said. He added that OpenAI may eventually train models specifically for these tasks, but for now it is "just plain old Luna that gets the job done."
The public version adds voice and visual modalities, and OpenAI has targeted its inference resources by region to reduce latency around the world. Early trials involved customers including Box, Datadog, Ramp and Speak, whose engineers used the API in place of OpenAI's LLMs to gain throughput. Andrew Hsu, CTO of AI language-tutor maker Speak, said the time between actions fell from 1.4 seconds to 0.13 seconds. OpenAI also used the product to make its Dots voice assistant more responsive.
What makes a good decision model remains unclear beyond whether it works for a given user, since the balance of intelligence, cost, calibration and speed varies by workload. TypeSafe CEO Diogo Almeida distrusts benchmarks and encourages people to test models on their own tasks. Even so, TypeSafe published its own rankings last week alongside its newest Jev model, which adds image support and smart-if statements.
Public benchmarks such as the AIM System One Index and the Artificial Analysis Decision Index tend to favor D1 Ultra, a decision model Liquid released last week, and Base Lab's Elephant-1. The latest Jev is competitive on both, placing seventh and third, while OpenAI's decision model ranks eighth and fourteenth. TypeSafe's models are, for now, the most queried on OpenRouter. Jev also tops JevMan, a benchmark from Opper AI in which decision models play PacMan. Opper AI chief growth officer Felix Wunderlich said the game is a good test because of its "nonstop stream of quick, high-stakes decisions."
Handa said OpenAI's more capable model may have an edge in multi-modality. For image classification, he said, "Luna is extremely powerful and has been showing better calibrations," and customers want the API to be more flexible about using images as outputs as well. He also floated a further speedup to around 60 milliseconds, which could make a call for every frame of a video conceivable.
As these services get faster and cheaper, they are expected to live up to the name of Jev's namesake, an economist who explained how using a resource more efficiently makes it more widely used. Handa expects an expanding "crazy set of use cases" that were hiding in plain sight until decision models revealed them.
