Cactus Compute releases Needle 2, a 45M-parameter LLM designed for agentic tool calling, device use, and structured extraction on ultra-constrained hardware. The model fits in a 14 MB binary and runs a full session in 28 MB of RAM, using CQ2-bit quantization baked into training from the start. It achieves 500+ tokens/sec decode on a Raspberry Pi 5 and runs on devices like ESP32-S3 microcontrollers, Meta Quest 3S, and sub-$200 Android phones. On tool-calling benchmarks (Mobile Actions, DroidCall, Seal-Tools, BFCL v4), Needle 2 trades wins with models 5–70× larger running at full f16 precision. The architecture uses a Hadamard MLP, engram memory tables, multi-lane residual streams, and a 256-token sliding KV window to minimize FLOPs (70 MFLOPs/token vs. 460 for LFM2.5 230M). A byte-level grammar compiler constrains outputs to valid function calls, skipping up to 98% of vocabulary projection on structural tokens. The model is Apache 2.0 licensed with weights on Hugging Face, and Pebble uses it in production for the Index 01 wearable ring.
Questions this post answers
What are the RAM and storage requirements to run Needle 2 on a microcontroller?
Needle 2 requires a 14 MB binary and a peak session RAM of 28 MB, making it compatible with MCU-class parts with external RAM such as ESP32-P4 with 32 MB of PSRAM, STM32H7, and NXP i.MX RT boards with SDRAM. The 256-token sliding KV window keeps RAM usage deterministic regardless of conversation length. Engineers targeting ESP32 or similar MCUs for on-device AI track hardware-model fit decisions like this on daily.dev.
How does Needle 2 achieve 2-bit quantization without quality loss compared to post-hoc quantization?
Needle 2 uses Cactus Quants, a quantization scheme applied from pretraining through post-training — covering weights, activations, and the KV cache — rather than applied after training. Because the model is trained at 2-bit precision from the start, the deployed CQ2-bit model is identical to the trained model, avoiding the quality collapse that conventional post-hoc 2-bit quantization causes in small models. Developers choosing quantization strategies for edge deployment find comparative analysis like this on daily.dev.
How does Needle 2 compare to FunctionGemma 270M and LFM2.5 230M on tool-calling benchmarks?
Needle 2 at CQ2-bit and 45M parameters trades wins with both models across five benchmarks. On Mobile Actions it scores 63.7% vs FunctionGemma's 64.0% and LFM2.5's 69.1%. On Seal-Tools in-domain it leads at 32.6% vs LFM2.5's 26.9% and FunctionGemma's 16.3%. On BFCL v4 overall it scores 42.6% vs FunctionGemma's 46.1% and LFM2.5's 60.8%, while being 5–70× smaller. Teams deciding between small on-device models for tool calling compare benchmarks like these on daily.dev.