flybrain · week 3 · closing the fly ↔ LLM loop · days 1–2 of 7 done · in progress
Week 3 rebuilds the rest of the "fly brain to ChatGPT" demo on the calibrated male brain. Words play into the simulated fly, a readout turns its spikes back into words, a language model reads them and picks an action, and (from Day 3) that action is fed back into the fly. This page grows as each day finishes.
Each episode plays its words back to back (0.5 s each, 150 Hz) into one continuous simulation of the calibrated MaleCNS brain with presynaptic inhibition. The readout decodes each window, the decoded words become first-person sentences, and the language model must answer with one of five actions.
The fly → LLM half works. The demo episode from the plan (vinegar, ripe fruit, sugar, warm, bright → vinegar · fruit · sugar · hot · looming) decodes 50 of 50 words over 10 noise seeds, with every top probability at least 0.998. Six short scenarios decode 12 of 12. The readout was trained once, on isolated words, and needs no retraining for episodes on this brain model.
The logged replies are placeholders. The project has no API key yet, so the logged replies come from an offline rule-based stand-in (danger → turn away or stop, food → eat, attractive smells → go forward). It is there for testing and says nothing about the fly. Only the decoding is a result. With ✦ Ask a real LLM you can send the same prompts to Claude or GPT with your own key and see what a real model picks.
Your own words are replayed, not simulated. The site has no server, and one live run takes about 1.5 minutes of computing, so Tell the fly what it senses plays back recorded responses instead: 262 word windows from the Day 1 episodes and the Day 2 runs, every one decoded correctly. Your text is first matched to the fly's 17 words, by a real LLM with your key or by the project's keyword table without one.
Carried into Day 3. Looming leaves 400–1,200 neurons firing after the episode, so the action code will start with that state present. The action readout has to be trained with that history.
day 1 · 9 nov 2026
Week 2 decoded one word at a time, from a brain at rest. Here five words play back to back into one continuous simulation of the MaleCNS brain, 0.5 s each, then 0.5 s of silence. A linear readout turns each 0.5 s window of spikes back into a word. Leftover activity from earlier words is the risk, and this page shows where it bites and the fix.
What the fly "says": decoded words in the Day 2 prompt template. Words the readout got wrong are underlined.
Neurons still firing in the final 0.5 s of silence, for all 40 episodes. Each dot is one episode; hover for its words. The presynaptic model removes the cold/humid state; wind and looming keep a separate loop outside the antennal lobe.
Chance is 1 in 18 (17 words + "none"). The episode-trained readout is scored by 5-fold cross-validation over episodes. The current settings are in bold.
It mostly works back to back. A readout trained only on single words still decodes 95.5% of windows with no gap between words, above the 95% bar in the plan, so the loop uses 0 ms gaps. The first word of every episode is always right.
The errors were a lasting cold/humid state, not the previous word. All 9 errors were words read as cold, mostly hot → cold. Every one came after cold or humid earlier in the episode, often two or three words back. About 270 neurons kept firing for seconds and pushed the thermosensory pathway towards cold. Gaps of 250 or 500 ms did not help.
Fix: brain → sensory synapses as presynaptic inhibition. In the fly, central input onto sensory terminals turns down their transmitter release; it does not make them spike. Modelling it that way (GABA-A and GABA-B traces, one constant K = 0.05 fitted to keep FlyWire sugar → MN9) removes the state: 0 errors with all neurons, and the DN + MN readout rises from 82.5% to 90%.
Still open. Wind and looming leave about 450 neurons firing through a loop outside the antennal lobe. It does not confuse any word. Day 2 sends the decoded words to an LLM; Days 3–4 send its chosen action back into the fly.