DECISIONING MODELS / INSURANCE EVIDENCE
Ask what the evidence supports.
Compare models that return choices, scores and yes/no answers. The underwriting platform determines what happens next.
General decisions
Answer accuracy
Correct answers / evaluated answersLocal GPU: AMD RX 6800 XT. Memory is peak PyTorch GPU allocation, not total system memory. Local cost is unmeasured, not zero. Decider uses eager reference kernels on Windows/AMD; its timing does not represent optimized serving. API prices and provider versions may change.
SEPARATE DIAGNOSTIC
Laya: adapted inputs
These inputs were changed during exploration. This result is shown separately from the unchanged-input comparisons.
How to read these results
General decisions
Ten questions per packet: five choices, three yes/no checks and two ordinal scores. Scores are graded by the most likely level, using the same frozen rubric for every model.
Yes/No binary models
Only the three yes/no questions are scored: declared activity, subcontractor disclosure and document conflict. General models appear here using only their binary answers.
The test set matters
The 180 packets are related scenarios grounded in 30 archived business profiles. The eight-packet tests use a fixed subset. Neither answer counts nor packet counts represent independent businesses. Labels await independent review.
Observe, then act
Procedural outcomes are shown only for the full question set. “Clear” means passing this focused scenario rule, not accepting a policy. Binary false positives and false negatives describe individual evidence checks.
WC classification is a separate decisioning family in development; no scored results for it are published here yet.
What we are learning
- Actual cost per packet can differ sharply from advertised token pricing. Solar reported far more input tokens on the same requests.
- Cheap additional questions can uncover rare, valuable findings. Test their effect on referrals and missed issues.
- Absence of supporting evidence is not proof an exposure is absent. Confidence is not a substitute for evidence.
- Smaller local models need their own validation. KEV-0.8B saved memory but lost substantial accuracy on these packets.
These results are a snapshot in time. The models are changing quickly. I’ll keep testing.