Jev AI limitations: what to verify

Type-safe is not always correct

A response can fit your declared options and still choose the wrong one. If a payment complaint gets labeled “product,” the type is valid but the route is wrong. That distinction matters more than a headline about hallucinations.

Our recommendation is to evaluate semantic correctness separately from response parsing. Count wrong decisions and review the actual inputs that caused them, including cases where the model appears confident.

Known Jev 1.13 boundaries

TypeSafe publishes a version-specific limitations page. Its documented weaknesses include numerical precision, counting, date comparisons, indirect questions, distracting context, and adversarial input. Text generation is also outside the model’s intended job.

• Compute exact arithmetic and date comparisons in code.
• Ask one direct judgment at a time, with consistent criteria.
• Supply the context needed for that judgment; filter irrelevant material.
• Evaluate hostile or misleading text instead of assuming it cannot affect answers.

These are documented limitations of a particular model version, not a prediction about all future releases.

Confidence needs interpretation

For Choice and Score, TypeSafe derives confidence from the answer’s probability distribution. It is a convenience signal for how concentrated that distribution is. Noul has no separate confidence field. See the official confidence guide.

Do not assume a confidence value is a guarantee for that individual case. Our recommendation is to select thresholds using your own labeled examples and the consequences of errors. Review uncertain cases and check whether apparently confident failures cluster around a missing criterion.

Input and version boundaries

The model reference lists text-only input for Jev 1.13, with a 64k total request budget and a 32k limit for state plus the longest question. It also describes model aliases and versioned IDs. These details can change.

Choose a version deliberately for repeatable testing. When you change it, rerun the same evaluation set before treating previous thresholds as reliable.

Before connecting real actions

• Can the workflow leave a case unresolved?
• Are permissions checked outside the model?
• Can a human see the original evidence behind a suggestion?
• Have you tested conflicting requests and missing information?
• Can you roll back the integration without losing the original work?

This is our engineering review checklist. It is a starting point for evaluation, not a certification of safety or accuracy. See the workflow examples for small, reviewable starting points.