Vollständiges Transkript anzeigen (2.495 Wörter)
Jev isn't a chat model, it's a decision model. You give it some context and a question and it returns an answer that your software can act on. Which team should handle an inbound message? Is a customer review reporting a defect? What product should be recommended to a customer for a certain situation? These fuzzy micro-decisions are everywhere, filtering spam, sorting customer requests, analyzing social media posts. And when that intelligence becomes cheap enough to use at scale, you can start putting it into places where an LLM call was previously too expensive or too slow. And this opens up a large amount of new use cases that people are only starting to get their heads around.
So to understand where Jev fits, I want to work through two things in this video. Jev is all about decisions, so it's important to understand the shapes of these decisions and the design patterns that you can use to put these to work. And in this video, I'll use practical concrete examples throughout so you can relate them to something you might build yourself. And this isn't specific to Jev, but with the incredible success of this model in only the space of a week, we are guaranteed to see a wave of new decision models, both proprietary and open source that we can deploy locally in the future.
And this all comes down to token economics. TypeSafe currently lists the price of Jev at 4.2 US cents per million input tokens with no charge for output tokens, which is vastly cheaper than the LLMs that you would typically use for these types of activities. Let's suppose a request uses 1,000 input tokens including the context and the questions. If you run a million of those requests, it would only cost you $42 in Jev token charges. And that gives you a sense of why these repeated fuzzy decisions becomes quite interesting. You can start considering checks across entire collections of records or several checks inside a single workflow that runs at scale.
TypeSafe groups these opportunities into five broad categories. First is automation. An incoming support message becomes a prioritized task in the right queue or a supplier email gets identified as an invoice query and sent to accounts payable. There's a lot of opportunities here around AI automation software. Then we have real-time applications and you may have seen examples online of Jev powering gameplay in games like Doom. And this is because you are getting frontier intelligence at almost real-time speeds, the likes of 150 milliseconds per decision. So AI here is making decisions faster than how humans can almost perceive it. And while AI playing games might not seem incredibly practical, there are real uses of this.
Say for example, if you had a drawing app and a user said, "make something bigger," it could trigger that action almost instantly. So in this case, the software needs to understand the instruction and select the appropriate action. There's data processing at scale which is a huge use case in enterprise. You can analyze time series data on mass or check product reviews for recurring complaints or support transcripts for particular problems. Once classified, you can then just use ordinary code to count everything up and total it across months or versions or products.
Verification is another important category. Does a citation support the claim beside it? That's something I went through in my previous video which I'll leave a card for above. Or does a drafted support reply promise something that the refund policy doesn't actually permit? Again, these checks will always have required another LLM call which dramatically increases cost and delays the actual output. And finally, harness engineering. That's the software around an AI agent, like Claude Code or Codex or a custom piece of software that you create. And all of these applications look quite different, but underneath them you have the same set of decision types, which you've probably already seen if you've looked at other videos on Jev.
The first type is Noul. It answers a yes or no boolean style question, but it actually provides a probability. So we can ask, does the customer in this email mention a deadline within the next two days? And if Jev provides us a 0.9 as an output, our code can then use that to add a deadline flag. And that probability matters because a result near halfway means the model is uncertain about yes or no. The second type is choice. So here we supply the possible answers and ask Jev to pick one. Is the request made by the customer a bug, a billing issue? Is it an account access problem, a feature request, or something else?
The third type is score, so here we have an ordered scale. Let's say a priority scale. So you may have something that's trivial, minor, major, or a blocker. Jev here will return a position on that scale which can fall between the different levels. Choice and score, they also return probabilities and a confidence value. So Noul gives us its yes probability directly, whereas we'll use those uncertainty signals when we get to the patterns. So you've seen the broad categories that these decision models can operate in, and now you've seen the decision types that they can make. But the type tells us what comes back. The shape, however, describes the job that we're trying to do with it.
And there are 10 shapes worth looking at. The first is classification. Classification gives something a category, so an upload failure message would be labeled as a bug, or in a shared company inbox, you might label messages as invoices or sales pitches or delivery updates. And a choice works well here for classification because we have a known set of categories and the right one can be selected. The second is detection. So detection asks whether a particular property is present. You, for example, maybe a sales reply might ask you to stop contacting the sender. So a Noul makes sense here because it's going to give you the probability between yes or no, and then your application can decide what to do with that.
If you enjoy going deep into AI topics like this, then make sure to subscribe to our channel and give the video a like below, it really helps us out. Thanks a million. Scoring is both a shape and a type, so how disruptive is a particular bug in a piece of software, or how clear is a help article? Here you can define the levels, so the number has a useful meaning and then you can use it to prioritize work, be it engineering work or editorial revisions in this case. Routing is an important shape. Routing here uses a decision to select what actually happens next. So if it's an order status question, that might go to a database lookup, while a product question goes to an assistant with the relevant documentation.
In my recent video, I used routing as a way to route questions to particular AI models within the application we're building. Search is another, because here we can use the likes of a decision model to find matches by meaning. TypeSafe has a how-to on how you can implement a line-by-line search of a particular document. In their example of searching GitHub's Terms of Service, in a single request, you can score all individual lines against a query with a choice question and use a Noul question to check whether the document contains an answer. Now that's not to say you can use a decision model as a search engine, but you can build it into a search engine using the design patterns I'll go through in a few minutes.
Along with search, you have retrieval. So retrieval is all about getting the right evidence into the next step in a process. So for example, a support assistant could select the returns policy passages that apply to a particular customer's region and purchase. So this very much acts as a filtering process to improve accuracy. Ranking is another important shape that I went through in my recent video. Here we are putting candidates in a particular order. If we take a shopping example, imagine 10 backpacks and the request is "I want a backpack that's for my laptop and it's for a rainy bike commute." That way, Jev can evaluate the descriptions against those requirements and then sort the candidates.
So search finds the candidates, ranking orders them, and retrieval uses the results to supply what the next step needs. Verification is another important shape. So here we're checking a specific claim or a specific failure mode. Does a cited passage support the sentence that refers to it? A quote can exist and still be used to make the wrong claim. This is something I demonstrated in my last video as well. Or does a drafted support reply promise a refund that supported by the policy and the details of the case? Here you're giving the model something concrete to check against.
If you already use non-generative ML models, you could use the likes of a decision model for feature extraction. So here we're turning language into inputs for another model. In a business workflow for example, you can extract out buyer intent or product interest and then test whether those signals improve a demand forecast. And similar to that, you can use Jev as part of a structured data extraction engine. LLMs are great at data extraction, however, they are slow and expensive. Whereas with a decision model, you could have code find specific candidates and then Jev could identify which are the correct ones to use. Or don't even use code to select candidates. Maybe just load in the raw data and get Jev to work through it and identify the specific data that you're looking for.
Now that you've seen the types and shapes of decisions, we can now bundle it all together into design patterns. And these describe how to organize these decisions into a working system. Speculative fan-out is our first design pattern. So let's take a support ticket as an example. Here we need to understand what's the category for this ticket, how disruptive the issue is, whether there's a deadline, and whether the customer supplied steps to actually reproduce the issue. With a fan-out, we can ask those independent questions all together in a single request, and once the results come back, code uses the relevant answers.
So if it turns out to be a billing issue, the reproduction steps answer can then be ignored, and that's why it's called speculative because the time it takes to ask one question versus 10 questions in a single request is virtually the same. So you're better off to ask the questions that you might need to avoid a series of round trips with the model. Now there are questions that genuinely depend on an earlier answer, they may still need another stage. The second pattern is confidence-gated routing. So sometimes the best available answer is still too uncertain to act upon. If someone says, "cancel it," are they talking about their subscription or an appointment?
So a clear interpretation can go to an appropriate handler, whereas an ambiguous one should trigger clarification. And the confidence score is crucial here because uncertainty changes the path through the application that you're building, and you need to test the different thresholds then against your own use case and examples. Now of course, you still need in your software the likes of permission checks and confirmation rules, so don't just trust what Jev, what this decision model is going to tell you. You still need to properly secure and gate everything on your own side using your own business logic.
The third design pattern is composite scoring. So let's say you were creating a product recommendation system, and you wanted to score each product for how well it matches a particular customer's intended use and their preferences and their budget. So here you can ask for each score separately and then you combine them using weights that you control. And you could even have dynamic weights by sending in the situation into the model and getting it to actually determine them. Intent routing is the next design pattern. So here we're placing a decision at the entrance to several different handlers. Reset my password goes to a reset flow. How do I configure something goes to a documentation assistant.
Unresolved account disputes go to support. Here it's the user's intent that determines where it actually goes and then confidence is key because it'll determine whether we're ready to send it there or do you need clarification? So knowing those four design patterns and the shapes of decisions, we can bring it all together then into an application. So let's say it's a customer support application, we're getting tickets in and we've just received a ticket on an upload failure. So we can send that into a decision model like Jev. We can ask several questions together so we can get the category of the ticket, a disruption score, or kind of a prioritization and a deadline signal.
We then check whether the answers are clear enough to use, and then our code will select an appropriate handler and set the priority. So then it's bundled together with an LLM. If the customer needs a written response, it then goes to a generative model that can then handle that step using the relevant evidence provided. Each decision has a defined job and the application controls how the pieces fit together. So when you're thinking about using a decision model, you need to pick a workflow you already understand, find a point where the software needs to interpret some fuzzy natural language, and then figure out what is the smallest useful question. Does the answer need to be a probability or a choice or a score, and what should happen when the answer is uncertain?
And that should give you a pretty concrete place to start building. These are the types of topics that we cover in our community, The AI Automators. We have a detailed AI builders course for those looking to build AI automations and AI apps. We have dedicated modules on introducing you to all of these concepts, a full technical foundations for AI coding agents, a dedicated module on agentic retrieval and harness engineering. So if you are serious about building production grade AI systems, then this is definitely the place to be. Link is in the description below. If you'd like to see Jev's decision engine built into a real-world app, then check out this video here where I demonstrate how I built it into our agentic RAG app that we're building on this channel.
