When to screen first
Jev answers all the questions in a request in parallel and reads the state once, so you pay for the state once. If the later questions would use the same state, don’t split them off: send them all in one request and let your code ignore the answers that don’t apply, we call this pattern speculative fan-out. Asking first is worth the extra request when the later step costs something the first question doesn’t:- A bigger state. The next step needs the customer record, the full thread, or an attachment that code has to fetch first.
- A slower model. The next step is an LLM call.
- A person. The next step needs someone to look at the item.
Example: triaging a shared inbox
Say you’re building a shared-inbox tool. Everything arrives in one stream: customers asking about orders, newsletters, payment receipts, thank-you notes, and the occasional “just so you know”. Full triage means fetching the sender’s account and order history, then asking several questions about the message. A Noul question that sees only the message, as it arrived, can decide which messages get full triage.Ask whether the sender expects a reply
We ask one thing the message alone can settle: is the sender waiting for a reply? Instead of saying “not waiting”, thefalse criterion lists the kinds of message with nothing to answer, including a note from a person that needs no reply.
Tom’s message reports a problem but asks for nothing: the desk is already built, and the message doesn’t say whether Tom wants to hear back. The answer is neither a clear yes nor a clear no:
Handle yes, no and unsure
Compare the Noul with two thresholds: above 0.7 counts as yes, and below 0.3 counts as no. Both are good starting points. An answer between them means the message doesn’t settle the question, so give it its own outcome instead of rounding it to yes or no.screen.py
Triage only the messages that need a reply
Fetch the customer record only for messages with a clear yes. The triage request sends the record with the message and asks questions that each check one thing:repeat_contact reads customer.previous_contacts, and urgency counts from message.received. A real triage stage would ask more questions than this.
Jev’s response:
repeat_contact answer comes from the record: with no previous contacts on file, the same message got 0.05.
From here, your routing code acts on the answers. Dana’s urgent, repeated order-status request could go to the front of the queue.
Other ways to screen
- Check a cheap step first. Run the cheap step, ask a Noul whether its output is right, and send only the outputs that fail that check to the expensive model.
- Several screens. A content pipeline might check first for “written in English” and then for “about our product”. Each check removes inputs before the next, more expensive step.