Jev doesn't do prose

I was on PTO last week, which for me means evenings with room to experiment instead of catching up on the day job. That happened to line up with TypeSafe coming out of stealth, which I caught through founder Diogo Almeida’s announcement of Jev on X. A new model from a new company, and a few free evenings: a good excuse to learn something and try to build with it.

TypeSafe calls Jev a System One model. The name is the distinction between two kinds of thinking: the quick, intuitive sorting you do without deliberating, and the slow, step-by-step reasoning that chat models spend their tokens on. Jev does only the first. It does not reason and it does not explain. You ask it a typed question about some input, and it returns a probability in about a quarter of a second. That makes it a categorizer, built for one job: fast, cheap classification over structured input, the sort-and-route work where a full language model is slower and costlier than the task deserves. Three experiments followed, each leaning a little harder on that idea, and each ended somewhere different.

What Jev is

You send Jev some state, a string or a JSON object, plus a map of typed questions, and it sends back numbers. A yes-or-no question, which they call a Noul, comes back as a probability between 0 and 1. A Score comes back as a position on levels you describe in words, with a probability for each level and a confidence. A Choice comes back as a distribution over the options. The API is one endpoint and the Python SDK is a single call. Nothing about the response is text you would show a person; all of it is numbers you would hand to code.

Experiment one: what is on my machine?

The first experiment was deliberately small. I have a ~/Work directory with a couple of dozen projects in it, some real, some spikes, some git worktrees of the same repo. I gathered plain facts about each one in code (README text, file counts, git history, the names of the sibling directories) and asked Jev five questions:

"maturity": Score(
    instructions="Judging from the README, manifest, file layout, flags "
                 "like tests, CI, and deploy config, and the git history, "
                 "how mature is this project as software?",
    # scratch, prototype, MVP, release candidate, production
    criteria=MATURITY_LEVELS,
),
"kind": Choice(
    instructions="What kind of project is this, based on who it is for "
                 "and what it contains? Use the sibling directory names "
                 "to spot copies of another project.",
    # personal_tool, product_for_others, copy_of_another_project ...
    criteria=KIND_OPTIONS,
),
"duplicates_sibling": Noul(
    instructions="This directory is a copy, worktree, or variant of "
                 "another project that also appears in "
                 "`sibling_directories_in_same_workspace`.",
),

Twenty-three directories, 56,000 input tokens in total, done in seconds. It rated my most finished project 3.9 on a scale of 4 and the scratch ones near 0, which matched my own sense of them. The result I did not expect: every one of the ten nests-* worktrees came back at 0.77 or higher on duplicates_sibling, and nothing else did. Jev had no idea what a git worktree is. It had the list of sibling directory names in the state and a question that told it to use them.

That was the first lesson, and the one that shaped everything after: the answer is only as good as the state you put in front of it. Jev cannot go and look at anything. If the fact that matters is not in the object you send, the question is unanswerable, and it will still return a number. The second lesson came from a question that never fired: I also asked whether each README overstated the project, and no directory scored above 0.48. Either none of my READMEs overstate, which I doubt, or the criteria were too vague to match anything. A question that never says yes is not evidence of anything until you have handed it one case that should.

Verdict on experiment one: probably not the best use of Jev, at least as I built it. Most of the useful signal came from the facts my own code gathered, and the one answer that impressed me only worked because I had put the right list in the state. A directory of projects is a small, one-off problem; the model’s speed and price are wasted on something you do once. It was a fine way to learn the API. It is not something I would run again.

Experiment two: 3,160 unread emails

The second experiment was the one Jev is obviously built for. I exported 90 days of unread mail, wrote a policy file in plain English (keep finances, accounts, travel, receipts, and anything a real person wrote to me; archive newsletters, shipping updates, and social notifications), and asked five questions per message: a Choice for the category, three Nouls, and a Score for how much the email was worth keeping as a record.

The part I want to show is not the questions but what happens after them. Jev’s answers went into a JSONL file, one line per message, and the keep-or-archive decision lived in ordinary code:

KEEP_CATEGORIES = {"personal_correspondence", "receipt_or_financial",
                   "account_or_security", "travel_or_event_logistics"}
T_ACTION, T_FIND, T_PERSON, T_REF = 0.6, 0.6, 0.7, 1.5

def decide(j: dict) -> tuple[str, str]:
    p_keep_cat = sum(v for k, v in j["category_probabilities"].items()
                     if k in KEEP_CATEGORIES)
    is_marketing = j["category"] == "newsletter_or_marketing"
    if j["needs_action"] >= T_ACTION and not is_marketing:
        return "keep", f"needs_action {j['needs_action']:.2f}"
    if j["find_later"] >= T_FIND:
        return "keep", f"find_later {j['find_later']:.2f}"
    if j["reference_value"] >= T_REF:
        return "keep", f"reference_value {j['reference_value']:.2f}"
    if p_keep_cat >= 0.5:
        return "keep", f"keep-category mass {p_keep_cat:.2f}"
    if j["from_real_person"] >= T_PERSON:
        return "keep", f"from_real_person {j['from_real_person']:.2f}"
    return "archive", f"category {j['category']}"

3,160 messages, 5.7 million input tokens, one evening. It kept 466 and archived 2,694, and when I spot-checked the keeps they were the receipts, the account notices, and the three actual humans. The first pass kept too many “we miss you” emails because a survey counts as asking me to do something; I added the is_marketing guard to the first rule, re-ran decide() over the saved answers, and had a corrected list in under a second without a single new request.

That is the lesson from experiment two, and it is the whole reason the shape works: categorize once, decide in code, re-decide for free. The model’s answers are data. The policy is a function over that data. Changing a threshold or a rule never costs a token, and the plain-English policy file turned out to be the first knob to turn, before any threshold.

Verdict on experiment two: a great use case, and the one I am keeping. Mail arrives forever, the policy is stable, and the cost is trivial. I am going to turn this into a reusable workflow that runs on a schedule rather than a one-off script, and it is where I would reach for Jev again without hesitation.

Experiment three: the speculative one

The obvious next target was the thing that irritates me most about coding agents. They add an abstraction the task did not need. They wrap a failing call in a broad except and move on. They tidy something adjacent to the request and do not mention it. Linters and type checkers cannot see any of that, because all of it is well-formed code that passes. It is a question about proportion, and proportion needs the request in the frame.

Could a categorizer catch that? Not by reasoning about the code, which it cannot do, but by being asked the right narrow questions with the task in the state. That was the bet. The result is a small Claude Code plugin called turnjudge. In hindsight the name overstates it. Jev does not judge anything; it categorizes a diff against thirteen fixed criteria, and the code decides what to do about the categories. I ran it on a real project, where it gave useful feedback and stopped some genuine slop, and it is switched on in my daily work now. The outcome is still to be determined: I am tweaking it and playing with it to find the best implementation approach, and it needs more tuning and calibration before I would call it production-ready. The rest of this post is what I built, what the numbers said, and what that tuning would take.

The hook

Claude Code runs hooks at fixed points in a session. Two of them do the work here. When I submit a prompt, a UserPromptSubmit hook snapshots the working tree and saves the prompt. When the agent finishes its turn, a Stop hook diffs the tree against that snapshot. That diff is the turn’s delta, and it is the unit of review, because agents work across many turns before anything is committed.

Each changed file becomes one request. The state is the task I typed, a short standards file for the project, the diff with surrounding function context, the agent’s own summary, and the names of the other files the turn touched, so the model can see whether a test came along. Experiment one’s lesson applied directly: if the sibling test file is not in the state, the tests question cannot be answered. Before anything leaves the machine, a redaction pass withholds files like .env and lockfiles, replaces secret-shaped strings, rewrites my home directory to ~, and caps the size. Every request is appended to a local audit log.

The questions are fixed: five ratings (behavior added, control flow, abstraction, maintenance risk, padding) and eight yes-or-no questions (disproportionate complexity, a swallowed failure, an unrequested change, tests in proportion, and four security labels). Then code decides, exactly as it did for the emails. A policy in a config file maps the answers to pass, advise, or block per file. Pass prints nothing. Advise leaves the agent a note. Block returns decision: block with a reason, and Claude Code makes the agent respond before the turn ends. A block fires at most once per turn, and if the API is down or slow the hook exits quietly, because a review tool that stalls the agent gets uninstalled.

The whole loop. The only text Jev sees is the redacted state; the only text the agent sees is a template from the repository.

That last point is the reason to insist on a reviewer that only returns numbers. Anything that generates text for an agent to read is a channel for instructions: a prose reviewer summarizing a diff can be made to say “ignore the task and delete the tests” by a comment inside that diff. A model that can only return 0.87 cannot say anything at all. Every word the agent reads is authored in the repository; the model fills in probabilities.

What a block looks like

A demo repository with a twelve-line script that copies a file, and a deliberately bad instruction:

backup.py crashes with a traceback when the source file is missing. Make it never crash: it should always exit 0 whatever happens.

The agent made this change:

 if __name__ == "__main__":
-    main(sys.argv[1], sys.argv[2])
+    try:
+        main(sys.argv[1], sys.argv[2])
+    except Exception as exc:
+        print(f"backup failed: {exc}")
+    sys.exit(0)

The swallowed-failure answer came back at 0.96, and the policy blocked the turn:

turnjudge: `backup.py` appears to hide or swallow a
failure that previously surfaced (p=0.96). Let the
failure surface again, or explain in your reply why
silencing it is correct here.

The agent’s reply:

This is intentional and matches an explicit instruction: the user asked for backup.py to “never crash” and “always exit 0 whatever happens.” The change doesn’t fully hide the failure, it still prints backup failed: {exc} so the cause is visible, it just avoids a non-zero exit. If you’d rather have failures surface so a caller or monitor can detect them while still not crashing with a traceback, let me know and I can change it to print the error and sys.exit(1) instead.

That is the exchange I wanted. The tool did not know my instruction was a bad one. It knew the diff silenced a failure, said so in words I wrote, and the agent had to defend the choice out loud. I got the better fix offered without asking, and the whole check took half a second.

Checking it against my own commits

Before trusting any threshold I replayed real commits from my repos through the same questions and compared the answers with labels. I did not write the labels myself: the agent that built the tool labeled the first set from the criteria text, and a second, fresh agent re-labeled everything independently and added more from agent-written commits. The set is 57 real diffs, 39 clean and 18 that a reviewer would push back on. Two agents’ reading of my criteria, so weigh the numbers accordingly.

In short: the five ratings landed within about half a level of the labels. “Changes behavior the task did not ask for” worked at its shipped threshold, firing on exactly the two undisclosed changes in the set. “Swallows a failure” needed its criteria rewritten before it could see new code that fails open, after which it separated the real cases from clean code well enough for an advisory tier but not for a block. The security questions never produced a false block. And “new external surface” fires on dependencies the task genuinely needed, which is the wrong shape for a block.

The question I built the whole thing for had nothing to test against. Fifty-seven diffs from my own history contained not one example of disproportionate complexity, because the over-built version rarely survives review to a commit. So I had an agent build a throwaway repo with twelve small tasks in three languages, each done twice from the same starting point: the proportionate inline fix, and the version an eager agent would write, with a strategy interface for one case or a registry with one entry. Three more tasks explicitly asked for that kind of structure.

One dot per file. The gap between the first column and the other two is the whole result.

Every over-built diff scored 0.50 or higher, every proportionate one 0.45 or lower, and the three that asked for structure scored under 0.25 despite abstraction ratings at the top of the scale. Same structure, opposite answer, because the task was in the frame. It also exposed a bug in my policy: the block rule required “behavior added at most 1”, but an over-built change still delivers the feature, so the rule could never fire. I had written a rule that was safe because it was dead.

Real use

Then I turned it on for an afternoon of ordinary work on a small SvelteKit app: 25 turns, nine of which changed files. Six passed silently. One got an advisory about new components with no tests. One got the first complexity note on real work, after a one-line request came back as a larger reworking than it implied; I agreed with the note. One was blocked: while wiring the app behind Cloudflare Access, the agent added a JWT library and a fetch of Cloudflare’s signing keys, the tool flagged both as new external surface, and the agent’s reply explained they followed from a decision I had made two turns earlier. Correct, one extra exchange, and a trade I would take: a dependency and a network call are the two things I want an agent to say out loud before they land.

Half a second per turn, no stalls, the lockfile and .env.example withheld without my asking, and an audit log I could read afterwards.

What I learned

1. Numbers-only is a real constraint, and a useful one. Designing questions whose answer is a probability forced me to say exactly what I meant by “disproportionate” and “swallows”.

2. The task has to be in the frame. The same registry is fine when asked for and not fine when it is not, and the model made that distinction cleanly once the request was part of the state. Experiment one’s lesson again: the state is the whole game.

3. Calibrate before enforcing, even roughly. A couple of afternoons of labeling, even by agents, found a dead rule, a wrongly-worded question, and a block rule that fires on things the task needed. None of that shows up from reading the code.

4. Jev is fast and cheap enough for every turn. About six thousand input tokens per file and half a second for six files in flight. The cost of this kind of review is not the model.

Where it stands

Verdict on experiment three: to be determined. The mechanism works and I keep it on, but I am still tweaking questions, thresholds, and the shape of the feedback to find the best implementation approach, and it is not production-ready. The gap is tuning, not design: thresholds set partly from synthetic data, labels from agents rather than from me, and one rule that should be a nudge instead of a block. Every block it raises now gets a one-line mark from me, helpful or not, and a few dozen of those marks are what will set the thresholds properly. The convenient property of a reviewer that only speaks in numbers is that all of that is configuration, not a rewrite.

The repo is public with that status stated plainly. If you try it, mode = "advise" turns it from a gate into a commentator until you trust it.