StargazerLabs models

StargazerLabs is where I publish open-weight models for Apple silicon. Jumbo adds selected layers from an older model to a newer relative; the demo compares their answers side by side. Seven models are available in bf16 and 3-, 4-, and 8-bit versions, with about three thousand downloads a month in total.

In a businessA team adapting an open-weight model needs to know whether a higher score means the model knows more or only guesses more. This project measures correct answers, wrong answers, and abstention together, which is the evidence a model-release decision needs.

Technical skills: MLX · Hugging Face Transformers / PEFT · Model adaptation · Quantization · Regression evaluation

My contribution

I selected layer blocks by how much their weights changed between model versions and ran the experiments. I evaluated the transplant on factual questions, including wrong answers and refusals.

Result and limit

On 2,000 PopQA questions, accuracy rises from 7.2% to 13.5% because the model answers 53% of questions instead of 12%. Precision falls from 58.5% to 25.3% and wrong answers rise from 102 to 796, so the transplant mostly restores the donor's readiness to answer. The donor alone scores 18.7%.

Technical methods and evaluation

Jump to the layer transplant, measurements, or unreleased experiments.

Jumbo

Continued training can change which questions a model answers correctly. Qwen3.8-32B-Jumbo is a 76-layer experiment in recovering some of those answers. A per-tensor cosine alignment census between two sibling checkpoints finds the layer blocks that drifted most in training, and those blocks are transplanted from the sibling into the base. The transplant keeps the model compatible with its multi-token-prediction drafter, at 82 to 89 percent acceptance, which gives 1.6 to 2.2 times faster speculative decoding. Mini-Me is a sheared 23-billion-parameter variant from the same family.

Measured

The parent is the model I modified; the donor supplies the replacement layers. “8-bit” means the weights are stored at reduced numerical precision to use less memory. PopQA is a factual question-answering benchmark; the alias rule accepts a recognised alternative name for the answer.

The card’s claim that the transplant restores factual breadth was checked on 2,000 questions sampled from the long tail of PopQA, with the donor, the parent, and Jumbo all at 8-bit, greedy decoding, thinking off, and the benchmark’s alias-containment rule. The write-up has the full account, and the comparison page (opens in a new tab) runs any prompt through the parent and Jumbo side by side.

Compared with the parent, Jumbo answers 130 additional questions correctly and loses 4 correct answers; Jumbo minus parent is +6.3 points (95% bootstrap 5.2 to 7.4). Of the 236 questions the donor knew and the parent had lost, Jumbo recovered 109. The parent abstains on 88% of the long tail and is right 58% of the time when it answers. The transplant brings back the donor’s readiness to answer at the donor’s precision, so the gain in recall is paid for in wrong answers.

The toolkit, and what did not ship

The LoRA post-training toolkit behind the releases does knowledge injection with sparse top-k logit distillation against a teacher bank, layer-gated adapters, region-weighted losses, and PEFT-format adapter export so a fine-tune can be evaluated in other frameworks. Every run also scores an out-of-domain regression set, a “shadow basket” of foreign-language text, contracts, recipes, and dialogue that the fine-tune was never meant to change. That set is there to catch catastrophic forgetting, and it did. The fine-tuned variants regressed on held-out evaluation, so only the surgery-only models were released.

Stack

MLX, Hugging Face Transformers and PEFT, custom census and transplant scripts, quantization and speculative-decoding evaluation harnesses.

My role

I selected layer blocks by how much their weights had changed between versions, ran the experiments, and evaluated whether the transplant recovered correct answers. The fine-tuned variants that regressed on evaluation were not released.

Where this could be useful

The evaluation approach could help compare model variants for a specific task, especially when a model can improve its score by answering more often while also making more mistakes. A deployment would need an evaluation on its own tasks and error costs.

If you are hiring, the experience page says which business problem this project shares its mechanics with.