---
name: langgraph-decision-models
description: "INVOKE THIS SKILL when routing a LangGraph agent with a decision model (TypeSafe Jev, SemIf) instead of an LLM, or when auditing an existing agent for LLM calls that only produce a routing decision. Covers langchain-typesafe Noul/Choice/Score, reading answers correctly, threshold design, and LangSmith Gateway wiring."
---
<overview>
A **decision model** answers typed questions about state and returns probabilities instead of prose. It replaces the common pattern of prompting an LLM, parsing its text, and branching on the result.
- **`Noul(instructions=...)`** — binary question, returns a probability of yes
- **`Choice(instructions=..., criteria={...})`** — picks one label, returns the full distribution plus confidence
- **`Score(instructions=..., criteria=[...])`** — grades against an ordered rubric, returns an expected value plus confidence
`TypeSafeClassifier` is a LangChain `Runnable[ClassifierRequest, ClassifierResponse]`, so it drops into a node like any other runnable. Up to 32 questions share one request and are answered independently — your code combines them.
**Reach for one when** a node generates text only so you can parse a decision out of it: routing, triage, filtering, guardrails, or per-item classification over a batch.
**Do not** reach for one when the node's output is the product (summaries, drafts, code) or when the judgment needs multi-step reasoning. A decision model classifies; it does not think.
</overview>
---
## Install and wire
`langchain-typesafe` is alpha (`0.0.1a3`) and `TypeSafeClassifier` is marked `@beta` — pin it and expect churn.
```bash
uv add langchain-typesafe
```
Three ways to reach a model. The classifier POSTs to `{base_url}/v1/systemone` with `Authorization: Bearer {api_key}`, so switching providers is constructor arguments only:
<ex-wiring>
<python>
```python
import os
from langchain_typesafe import TypeSafeClassifier
# 1. TypeSafe directly (Jev). Reads TYPESAFE_API_KEY when api_key is omitted.
classifier = TypeSafeClassifier(model="jev-latest")
# 2. SemIf, hosted on the LangSmith Gateway. Note: LangSmith key, not a TypeSafe key.
classifier = TypeSafeClassifier(
model="semif-qwen3.5-4b",
api_key=os.environ["LANGSMITH_API_KEY"],
base_url="https://gateway.smith.langchain.com",
)
# 3. Jev through the Gateway (BYOK). The `typesafe/` prefix routes to a
# TYPESAFE_API_KEY stored in LangSmith workspace secrets. Without that secret
# every `typesafe/*` id returns 424 Failed Dependency -- before the model name is
# even validated, so a 424 does not confirm the id is real.
classifier = TypeSafeClassifier(
model="typesafe/jev-1.13.0",
api_key=os.environ["LANGSMITH_API_KEY"],
base_url="https://gateway.smith.langchain.com",
)
```
</python>
</ex-wiring>
---
## The conversion pattern
Ask every question about a page/item in **one** request, put the typed response in state, and let a plain function route on it. The router is ordinary Python — testable without touching a network.
<ex-classify-and-route>
<python>
```python
from typing import TypedDict
from langchain_typesafe import ClassifierResponse, Noul, Score, TypeSafeClassifier
from langgraph.graph import StateGraph, START, END
QUESTIONS = {
"relevant": Score(
instructions="How relevant is this ticket to a billing problem?",
criteria=["Unrelated.", "Possibly related.", "Directly about billing."],
),
"angry": Noul(instructions="Is the customer expressing anger?"),
}
class State(TypedDict):
text: str
answers: ClassifierResponse
route: str
classifier = TypeSafeClassifier(model="jev-latest")
def classify(state: State) -> dict:
# One request, every question. They are answered independently.
return {"answers": classifier.invoke(
{"state": state["text"], "questions": QUESTIONS}
)}
def route(state: State) -> str:
a = state["answers"]
if a.nouls["angry"].noul > 0.7:
return "escalate"
if a.scores["relevant"].score < 0.5:
return "close"
return "handle"
```
</python>
</ex-classify-and-route>
Reading answers: `response.nouls[id].noul`, `response.choices[id].choice`, `response.scores[id].score`. Each view is keyed by your question id; `response.answers` holds them all.
---
## Three traps when reading answers
These cause silent misrouting, not exceptions.
**1. `Score.score` is an expected value, not a level.** It is a probability-weighted average over the rubric and is routinely fractional. `score == 0` almost never fires — a "not responsive" item lands at `0.07`, not `0`. Always compare against a band.
```python
if a.scores["relevant"].score < 0.5: # correct
if a.scores["relevant"].score == 0: # WRONG -- nearly never true
```
**2. Confidence measures distribution shape, not correctness.** On a `Score`, confidence reports how *concentrated* the rubric distribution is. An item sitting cleanly between two levels scores low confidence even when the model is entirely clear about it. A blanket `confidence < X -> escalate` rule therefore escalates items the model already decided. Gate on confidence only inside the ambiguous middle:
```python
if score < NOT_RELEVANT: # decisive -- trust it
return "close"
if score < RELEVANT or confidence < MIN_CONF: # ambiguous -- escalate
return "human_review"
return "handle"
```
**3. Thresholds do not transfer between models.** Calibration is part of the model. The same policy over the same items routes differently on Jev vs SemIf vs an LLM adapter. Re-tune thresholds whenever you change models, and pin the model id.
---
## Question wording dominates accuracy
A loose question produces confident wrong answers, and no threshold fixes it. Use `criteria` to say what each outcome means, including what should *not* count.
In a measured case, "Is this a confidential communication with a lawyer?" scored a routine finance memo at **0.798**. Rewriting it to name the actual test — written by or to a lawyer, with an explicit carve-out for finance and accounting content — moved the same page to **0.005** while a genuinely privileged page held at **0.991**.
```python
Noul(
instructions=(
"Was this written by or to a lawyer, or does it convey a lawyer's legal "
"advice? Answer no for ordinary business or accounting discussion, even "
"when the subject is litigation-sensitive."
),
criteria=NoulCriteria(
true="A named attorney is author or recipient, or it relays legal advice.",
false="Business or accounting content with no attorney involved.",
),
)
```
Before blaming the model, rewrite the question and re-measure.
---
## Auditing an existing agent
To find where a decision model fits, look for these in the codebase — see `references/conversion-playbook.md` for the full walkthrough.
| Signal | What to look for |
|---|---|
| Generate-then-parse | An LLM call whose output is immediately regex'd, `json.loads`'d, or string-matched into a branch |
| Prompted classifiers | Prompts containing "respond with one of", "answer yes or no", "rate from 1 to 5" |
| Sampling for cost | Comments or configs that check only the first N items because checking all is too expensive |
| Brittle rules | Keyword lists or regexes standing in for semantic judgment |
| Re-reading context | The same document re-sent to a model for each separate question |
The last two matter most: cheap semantic judgments change *what you can build*, not just the bill. If evaluating every item became affordable, what would you stop sampling?
---
## Expectations
Measured on a 24-item batch, identical LangGraph graph and routing policy, only the classifier swapped:
| | per item | tokens (6 items) | notes |
|---|---|---|---|
| Jev 1.13.0 | 0.27s | 3,648 in / 318 out | reports usage |
| SemIf 4B | 0.49s | not reported | hosted on the Gateway |
| Claude Sonnet 5 | 2.87s | 7,930 in / 864 out | via structured output |
Routing agreed on 4–5 of 6 items across engines; disagreements clustered on genuinely borderline items. Treat these as shape, not benchmarks — measure on your own workload.
**If you compare against an LLM baseline**, use `method="json_schema"` so the comparison is fair. LangChain's `with_structured_output` defaults to `method="function_calling"`, which injects a tool schema into every request — 556/35 tokens versus 228/12 for the native `output_config.format` path on the same one-field probe.
---
## Batching with Send
Classification is per-item and independent, so fan out with `Send` and let each item route on its own.
<ex-fan-out>
<python>
```python
from langgraph.types import Send
def fan_out(state):
return [Send("classify_item", {"text": t}) for t in state["items"]]
builder.add_conditional_edges(START, fan_out, ["classify_item"])
```
</python>
</ex-fan-out>
Fan-out hides latency, so it flatters slow classifiers most: in the run above, Sonnet gained 8x from concurrency and Jev only 1.4x — yet Jev still finished first. Compare throughput, not the speedup multiple.
---
## Related skills
- **langgraph-fundamentals** — StateGraph, `Send`, `Command`, conditional edges
- **langgraph-human-in-the-loop** — `interrupt()` for the escalation branch above
- **langchain-middleware** — structured output when you need an LLM, not a classifier