A New Type Of LLM On The Block: Decision-Making Models

Large language models (LLMs) output language, but they are commonly tasked with making a decision or classification of some kind instead of writing an essay or chat reply. An LLM will be provided with input, and asked to classify that content in some way: with a rating, yes/no answer, a best-fit categorization, and so forth. A recent new type of model by the name of Jev was released only weeks ago and it is extremely fast, ultra-cheap, and laser-focused on that decision-making role. It can’t write even a single sentence, but it can classify and categorize very, very quickly.

Jev works like this: it still accepts text input, but it outputs only floating-point numbers. Those numbers are the “answers” to user-specified yes/no type questions, lists of choices, and scoring-type requests. [Simon Willison] provides a concise summary of what Jev does, and what makes this new category of model so interesting.

To say that the idea has caught on would be a wild understatement. Folks are making their own decision-type models and experiments in a flurry. Kev and Nimble are two examples (Nimble was added as a supported model in Ollama just recently, and is small enough to run locally with relative ease.)

If this type of local AI model was the missing link you needed to get an idea working, don’t keep it to yourself! Tell us all about it on the tips line.