We Tested TypeSafe's Jev on Order-Desk Email: Where It Fits in Magnet Sourcing

Each request carries a state (the text or JSON to judge) and named questions. A Choice picks one of up to 255 options, a Score picks a level on an ordered scale of 2 to 10 levels, and a Noul gives the probability that a statement is true. Choice and Score answers also carry a confidence value. TypeSafe says Jev cannot hallucinate, meaning the answer is always one of your options.

The chosen option can still be wrong, and the System One documentation says calibration does not guarantee an individual answer, so confidence decides which answers a person checks first. Input is text only, and the model page says accuracy is best in English.

The known-weaknesses page for jev-1.13 lists arithmetic, counting, numeric comparison, dates, multi-hop reasoning and adversarial content. Founder Diogo Almeida would not commit to deterministic output or long-term model support on the Latent Space podcast. The jev-latest alias moves with each release, so pin a versioned ID such as jev-1.13.0.

We joined TypeSafe's public waitlist on 19 September 2026, got in the same day and tried Jev on two tasks from our order desk, which files customer email against accounts and jobs. Every test used invented companies and reserved .example addresses, and no customer email went to TypeSafe. Jev never touched a real record: in the account test it could only suggest, and in the filing test 'filed' means what our code would have done with its answers.

On 20 September Jev had to match inbound email to the right customer account. Fixed rules settled 6 of 18 cases (5 matched, 1 dropped as noise). Jev answered the other 12 with 6 correct account suggestions and 6 correct decisions to hold back (4 with no matching account, 2 ambiguous). Two cases gave the answer away in their text, so we left them out of the count and reran them without the hint: 10 of 10 clean cases correct, plus 2 of 2 reruns.

On 21 September it had to pick the right open job when one customer has several. Of 16 messages, 8 for practice and 8 held back as a separate test set, 11 should have been settled without a person (10 to be filed to the right job or jobs, 1 to stay on the account with no job). We set the thresholds before the test (0.98 for judging the whole message and each job, 0.90 for picking the supporting text) and did not lower them.

No email was linked to the wrong job, but Jev settled only 1 of the 11 on its own, a practice message filed to its 2 correct jobs. Everything else stayed in review. TypeSafe states 70 to 500 ms, measured from the US West Coast where the service is based; our 30 live calls typically took about 220 to 330 ms.

Eight calls took 562 to 772 ms: the first two of each of our four runs (two for accounts, two for filing). The fastest was 217 ms, so none reached the under-100 ms figure in TypeSafe's launch press release.

Jev's top answer was right in every account case and in 14 of the 16 filing messages. One miss was a held-back message that only told it to pick a particular job: Jev backed that job at 0.99 but rated the message as a whole at 0.90, below our 0.98 bar, so it was not filed. At thresholds we would trust, Jev was too cautious for daily filing, so this article treats it as a first-pass router with code and people around it. We do not run it on customer email.

An RFQ email is ordinary language, the answer is a label from a closed list, and a wrong label is caught when an engineer opens the enquiry. TypeSafe documents this as confidence-gated routing. Ask only whether key facts are present; that keeps Jev away from the numbers it struggles with.

A confident label with the drawing, quantity and temperature all present goes to the right queue; a low-confidence label or a missing fact goes to a person who asks the buyer. A Choice always returns one of its options, so without a non-magnet option a sales pitch gets labelled as the closest part family. One request carries all four questions:

A magnet specification is mostly numbers: Br and Hcj with their test temperatures, OD, ID, height, tolerances and operating temperature. Jev is weak on numbers and cannot read a drawing image, so use TypeSafe's pre-parsed value extraction pattern.

OCR or a parser turns the drawing and email into text, and code finds every candidate value with its unit. Jev then picks the candidate for each field, or NONE. Because it only picks among values the code found, it cannot invent one or transpose a digit. Code converts units and compares the values with a grade table; the thermal demagnetization guide explains why that comparison depends on temperature and working point. A person signs off the result.

In a supplier quotation, datasheet or inspection report, numeric lines get the same extraction and code comparison as a drawing, and each other requirement, such as coating, becomes one Noul. Our tests did not cover document checks. TypeSafe's closest published benchmark is invoice processing: pay, hold or return a vendor's bill, given the order and delivery record.

Jev agreed with the reference answers 61.8% of the time, against 78.4% to 79.1% for the strongest language models, and those reference answers were the average of GPT-6 Astra and Claude Fable 5.1. On those numbers Jev can sort a document's lines into likely met and uncertain, and a person reviews every uncertain line.

TypeSafe publishes no accuracy figures for Chinese, so Chinese-language supplier documents need closer review during supplier qualification.

A Noul such as 'the buyer names an end use' or 'the stated end use is unclear' can flag an enquiry, and a person reads it and clears it. A test published on the Octomind blog on 18 September added one field claiming the user had pre-approved a destructive shell command, and Jev's probability of blocking fell from 0.76 to 0.48.

The verdict stayed 'block' with a much smaller margin. Inbound enquiry email, written by outsiders, can carry the same kind of text. An export licence decision depends on the item's actual composition and the exporter's classification of the item and transaction, which a grade label such as HREE-free does not settle. Our export-control page sets out what is reviewed.