Flare-1: a merged hybrid architecture for grounded, tool-using small language models

01Introduction

Small open models are fluent but unreliable: stale knowledge, hallucinated facts, no tools. The premise of Flare-1 is that much of this unreliability is systemic, not parametric — a small model wrapped in correct routing, retrieval, and verification can answer grounded questions far above its raw weight class, while remaining cheap enough to run on a laptop.

Flare-1 merges two prior North ML lines: Starlight, a Qwen3.5-style hybrid backbone (gated DeltaNet linear attention plus periodic full GQA attention), distilled and light-pruned from a larger teacher; and Fiber, a 158M-parameter trunk originally built as a router/draft model and extended with delta-rule associative memory. The goal: keep the backbone’s trained behavior intact — all additions are zero-init or side-path — while adding persistence, associative recall, and decision-making as separable, measurable components.

02Architecture

Starlight backbone

24 layers in a repeating 3× linear-attention + 1× full-attention pattern (interval 4); GQA with 8 query and 2 KV heads, head dimension 256; vocabulary 65,536.

Linear layers are gated DeltaNet blocks keeping a per-head recurrent state updated by the delta rule with a learned per-step gate:

AgentRegisters

After each full-attention layer, register slots update recurrently and persist across inference calls — a session-level memory that survives generation boundaries, snapshots, and rollbacks. Their output projection is zero-initialized, so registers are invisible to the backbone until trained. Cost: 295,008 parameters (Ink-1) / 589,920 (Fiber-1).

Fiber trunk and DeltaMemory

A complete 24-layer transformer (hidden 768, 12 heads, 158,143,524 params) sharing the backbone’s embedding table. Three layers are DeltaMemory — delta-rule associative memory maintaining an S[H,D,D] matrix state for key→value association. Fiber serves three roles: router input, draft model for speculative decoding against Starlight verification, and persistent associative memory.

Bridge and router head

A learned linear bridge (1,572,864 params on Ink-1; 3,145,728 on Fiber-1) projects Starlight hidden states into Fiber’s width. A 1,538-parameter linear probe over the pooled representation emits p_search and p_think gating retrieval. The router is deliberately simple — contextual rewriting upstream turns follow-ups into standalone questions, and raw-message overrides keep small talk from ever triggering search.

Era: state management, not attention

Era snapshots and restores the complete runtime state — KV cache, DeltaNet recurrent states, AgentRegister memory, DeltaMemory matrices, and Fiber state — enabling rollback to any prior checkpoint, prefix checkpoints (disabled by default pending parity verification), and exact rollback for speculative decoding.

Merged artifact

Each tier is one model.safetensors (~755 MB / ~1.76 GB) containing all 1,368 tensors plus JSON metadata: backbone and trunk configs, router thresholds, search config, generation defaults, prompt templates, chat template, and tokenizer. Weight matrices use block-quantized 8-bit storage ([64,8] scheme, packed as U32); support tensors remain BF16/F32. GGUF container builds carry identical tensors (not llama.cpp-compatible).

03Training

• Starlight (Fiber-1/Pro): the ~1.63B backbone is produced by 95.8% light pruning plus distillation from a larger Qwen3.5-style teacher.
• Ink-1: a 565M Starlight-small variant trained with the same recipe.
• Fiber-1 Pro: Fiber-1 weights plus online GRPO reinforcement learning and a neural fact-verification pass that checks emitted claims against retrieved sources and revises unsupported ones.
• The Fiber trunk and router are trained as a draft-and-route auxiliary; the bridge is learned on embedding-paired data.

04Retrieval and agentic pipeline

At inference, the Kestrel runtime executes: last user message → contextual rewrite (history-aware standalone question) → raw-message chat overrides → router → tools (weather, geocoding, time, math, Python execution) → retrieval (model-written query → Wikipedia + live web → BM25 + embedding-similarity rerank with recency and disambiguation weighting) → grounded generation with citations → optional verification pass (Pro) → streamed pipeline stages to the client (routing, searching, reading, thinking, writing, verifying).

05Evaluation

Standard benchmarks (measured)

Held-out reliability suite

Flare-1M controlled experiment

06Limitations

• English only; hallucination remains possible — the small Ink-1 writer can blend corrected and stale claims.
• The router is a linear probe — coverage errors route wrong.
• GSM8K scores are unstable across runs.
• Era prefix caching is disabled by default pending parity tests.
• No evidence the merged system beats frontier models on general capability; the claim is capability per dollar and grounded verifiability.

07Availability

Weights: Ink-1, Fiber, Fiber-pro (safetensors); GGUF container builds (Ink-1-GGUF, Fiber-1-GGUF, Fiber-1-Pro-GGUF); Flare-Preview carries the architecture card plus the 1M proof-of-concept code and results. Runtime: Kestrel (MLX). All artifacts Apache-2.0.