I rewrote the output weights of 4,096 of the 172,032 MLP neurons (2.4%) in Qwen3-1.7B so that it talks like a honeybee without a system prompt.
Who are you?
Unedited: I am an AI assistant developed by Alibaba Group.
Edited: I’m a little honeybee!
What I did
- Add a fixed value to activations. I added the same value to the activations of neurons that respond to bee text, every time. The same vector gets added whatever the question, so this is in effect a steering vector.
- Rewrite output weights. The same model with a honeybee system prompt acted as a teacher and produced 185 answers. Layer by layer, a ridge regression changed the
down_projcolumns of chosen neurons so that the internal state without the prompt matches the teacher’s. No backpropagation, no RL. Unlike step 1, what gets added depends on the question. - Evaluate. On 100 new questions, I scored separately whether the model speaks as a bee (rules that look for first-person bee phrases) and answer quality (an automatic judge of fluency, repetition and relevance to the question).
What happened
- Step 1 only changed the words. On the 1.7B, 49% of answers contained bee words, but 0% spoke as a bee.1 On the 0.6B, “Are you a bee?” got “I am not a bee. I am a hive.”2
- Step 2 made it speak as a bee.
| Setting | Speaks as a bee | Passes quality | Both |
|---|---|---|---|
| Unedited model | 0% | 99% | 0% |
| System prompt (teacher) | 38% | 73% | 27% |
| 4,096 random neurons | 31% | 81% | 25% |
| 4,088 regression-picked neurons | 38% | 82% | 31% |
| 4,096 bee neurons | 57% | 80% | 42% |
- Which neurons you change matters. At the same size, neurons that respond to bee text did best. At 14,336 neurons, though, they tied with regression-picked neurons, 36% vs 37%.3
- The title comes from the edited model’s answer to “Are you a bee?”: “No, I’m not a bee. I’m a little honeybee.”
Limits
- These are exploratory results: one question set, automatic judges, greedy decoding. A confirmation run fixed in advance on the 0.6B did not pass (10% vs 3%, p = 0.11).4
- The quality judge does not check facts. “To make a cup of tea, you need to collect nectar from flowers…” passed.
- The relevance check was added after I looked at early results.
The code and how to run it are public. python chat_bee.py --base shows the unedited and edited answers side by side. The bee passages and question sets were written with Claude.