I recently came across Julia 1, a “decision model” from Supersonic Labs on the Google newsfeed. It does the same kind of job as TypeSafe’s Jev, and it isn’t an LLM at all. Give it some context, a question and 2 to 20 candidate answers, and it picks one and gives a probability for each. Under the hood it’s mmBERT-small, a ~140M-parameter multilingual encoder, with a decision head added: 144.3M parameters in all, Apache 2.0, and a 33.15 ms median per decision on an Apple M4 CPU. More importantly for me, it doesn’t need anything that fancy: on an ordinary Intel i5-1235U laptop chip, AG News decisions took a 108 ms median.
The stats are solid too: on 100-question pilots it beats TypeSafe’s published Jev numbers on AG News (94% vs 91%) and DAIR Emotion (86% vs 48%), then falls over on 72-label Banking77 (64% vs 87%). That miss is the interesting part. One call only takes 20 options, so for Banking77 a router first narrows the 72 labels to a shortlist, and the narrowing can drop the right answer (it also stretches each decision to 3.7 seconds on that i5). So it’s likely handy in a lot of cases, but it’s worth some evaluation before committing to it.
Part of a project I’m working on involves evaluating a question to decide which of a collection of workflow-starting prompts suits the request best. I’ve been considering Jev or something like it to make that call more economically, and Julia looks well worth an experiment: open weights on my own CPU instead of a hosted API, as long as I keep the list of workflows to 20 or fewer.