Show HN: Cactus Hybrid: We taught Gemma 4 to know when it's wrong

Hey HN, Henry & Roman here from Cactus. A small, on-device model is fast and private, but sometimes wrong, but frontier models are getting expensive pretty fast. So, we post-trained Gemma 4 E2B post-trained to know when it's wrong.

Every response comes with a confidence score between 0 and 1. Developers can accept the on-device when it's high, hand off to a bigger cloud model when it's low. 1 Flash-Lite on most benchmarks.

- ChartQA: 15-20% - LibriSpeech: 25-30% - MMBench, GigaSpeech, MMAU: 30-35% - MMLU-Pro: 45-55% We were always frustrated by the routing signals hybrid apps rely on: asking the model to rate itself in text (unreliable, and you're parsing prose), or token entropy heuristics (barely better than a coin flip in our tests). So we did mechanistic studies on small models, Gemma 4 particularly, and found the hidden state for different layers carry meaningful self-awareness signal for various situations. SO we extended the model with a 68k params probe layer (LayerNorm, low-rank projection, attention pooling, small MLP head) reads one intermediate layer during decoding and predicts p(wrong); confidence = 1 - p(wrong), returned as structured data, never parsed out of the answer text.

549 for token entropy. 52). It's reading a modality-independent correctness signal from the hidden state, not memorizing patterns from its training data.

cpp or Cactus. With Ollama, vLLM, SGLang etc in the works. cpp we ship a patch series you compile in once (upstreaming is planned).

The code is MIT licensed; Gemma model use remains subject to the Gemma terms. co/collections/Cactus-Compute/cactus-hyb... Some caveats: - The probe scores single-sequence decoding only, up to the first 1024 generated tokens.

- Handoff works best when routing per task in a multi-step process, not per step. 5/Gemini/Muse/Grok. - The technique is boutique for each model, we will share each weights as they roll out.

These issues are currently being tackled at Cactus and updated weights will be shipped directly into the HuggingFace collection and GitHub repository straight up. Please let us know your thoughts, it helps us find ways to improve the design progressively. Thanks a million!