Build Your Own Decision System: A Step-by-Step Guide to Local Decision Systems

LinkMark KashefTutorial

Mark Kashef introduces a structured guide and travel demo for building custom AI decision systems locally. The approach blends text classification (based on ModernBERT) and computer vision (OpenJev Vision) under strict validation rules.

Key points

  1. An eight-step framework covering problem definition, baseline measurement, data isolation, fine-tuning, and deployment.
  2. Starts with Moritz Laurer's zero-shot classifier built on ModernBERT as a baseline before training.
  3. Strict three-way classification logic: yes, no, or can't tell (insufficient evidence).
  4. Benchmark results across 360 test cases: baseline achieved 60.28%, fine-tuned V2 reached 95.28%, while the reference Jev scored 98.61%.
  5. Multimodal verification pairing textual booking conditions with image recognition (e.g., verifying pool access vs. ponds).
  6. Rigid instructions for AI coding assistants (Claude) to ensure local execution, prevent unexpected cloud costs, and preserve untouched test sets.

Why it matters

Teams often default to complex cloud LLMs when compact, specialized models running locally can handle structured decision tasks faster, cheaper, and with higher determinism. This resource demonstrates how to enforce baseline metrics and disciplined evaluation before jumping into training.

Analysis

Kashef highlights best practices in applied ML engineering by prioritizing baseline benchmarks over immediate fine-tuning. The explicit 'can't tell' label prevents forced false classifications when evidence is lacking in booking policies. Additionally, using a strict eight-part prompt framework guides assistants like Claude to operate locally, keep held-out evaluation sets clean, and log every failure case transparently.

Found in

Summary generated by AI (Gemini 3.8 Flash, 28 September 2026). It may contain errors – the original source is authoritative.

Similar in content, found by the AI search.

  • Video

    Video:CURT

    Building a Local Jev-Inspired Classifier with Claude Opus 5.5

    CURT demonstrates how to use Claude Opus 5.5 to fine-tune a local specialist classification model inspired by Jev. The system categorises travel and hotel features against user requirements and integrates image recognition to verify classifications.

    AI & LLMs· Tutorial

  • Link:Featherless.ai

    Simple Jev: Open-Source Decision and Classification Engine Using Model Logits

    Featherless.ai has introduced Simple Jev, an open-source implementation that turns open language models into structured decision engines. Instead of requiring a separate classifier head, it reads next-token logits for allowed labels and returns probabilities and choices directly as JSON.

    AI & LLMs· Tool

  • Link:Powerformer, Inc.

    OpenDesign: Local Open-Source Alternative to Claude Design

    OpenDesign is an open-source, local-first AI design workspace released under the Apache-2.0 licence, positioned as an alternative to proprietary cloud platforms like Claude Design. It connects existing local coding agents into a structured design engine for generating prototypes, dashboards, slides, and HTML video.

    AI & LLMs· Tool

Let's talk about your project

Locations

  • Mattersburg
    Johann Nepomuk Bergerstraße 7/2/14
    7210 Mattersburg, Austria
  • Vienna
    Ungargasse 64-66/3/404
    1030 Wien, Austria

Parts of this content were created with the assistance of AI.