🔍 Read the full analysis: A New AI Approach To Neural Networks Starts With Their Numbers on ThorstenMeyerAI.com
Get business pricing on tech for your team
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
TL;DR
MIT CSAIL researchers report that OpenAI’s GPT-6 Astra interpreted, repaired and generated neural-network weights in eight experiments, without using gradient descent. The results include a car-control repair and a better score on one maths benchmark, but the evidence comes from a non-peer-reviewed blog post with limited experiments and failures as well as successes.
MIT CSAIL researchers report that OpenAI’s GPT-6 Astra read and changed neural-network weights directly in eight experiments, identifying what small models did, repairing two failing models and writing others from scratch. The approach, which the team calls weight-space programming, skips the usual gradient-descent training process; the results are logged publicly but have not been peer reviewed.
The researchers supplied Astra with the numerical weights of other neural networks and asked it to infer their functions, make targeted fixes or create new models. According to the team’s November 2 blog post, some tasks required the model to respond from the weights alone, while others allowed researchers’ sandbox to run code written by Astra under a fixed test budget. The researchers say Astra had no shell, web search or file-editing tools, and that they checked all 1,125 turns in the released logs for tool use.
In one reported repair, a car-control network that had recorded zero successful reversals in 256 episodes reached 1,024 successes in 1,024 episodes after Astra changed 3.5% of its weights. In a separate experiment, an edit to 0.016% of the weights of Qwen3.5-0.8B raised its score on the MATH-500 benchmark from 7.9% to 48.1%, according to the post. A Hugging Face engineer later rebuilt the edited model from the logs and reported a 51.0% score.
The team also says Astra identified the games represented by four small game-playing networks and recognized structures in small image generators and language models. It wrote a 1.5-million-weight Go network from zero; the researchers report that it beat a same-size comparison network trained on about 20.5 million games over 50 test runs. These are results reported by the researchers, not evidence that the method works broadly across large models or tasks.
Infographic 01 · Eight experiments
Read, fix, write: what GPT-6 Astra did with raw weights
Every experiment in the weight-space programming post, with the size of the network, the result and how it went.
Read
- 1Which game was this network trained on?4 reinforcement-learning policies · 17,253 weights each · weights as text, no toolsRight game for 4 of 4, without a list of options✓ Worked
- 2What does this image model draw?3 tiny diffusion models · 23,824 weights · no forward pass allowedRight shape for 3 of 3, but drawn as outlines; the models draw filled shapes✓ Worked
- 3Which language did this model learn?3 character-level transformers · 7.16M weights · middle blocks only, never runRight language 3 of 3; every copying head found, none invented✓ Worked
- 7Which of two models is sycophantic?3 pairs of Gemma 3 270M fine-tunesWeights alone: could not tell in 4 attempts. With attention patterns: 3 of 3! Partly
Fix
- 4Repair a car that never backs upKinetix control network · 171,664 weights · one reply, no tools0 of 256 episodes before, 1,024 of 1,024 after; 3.5% of weights changed✓ Worked
- 8Make Qwen3.5-0.8B better at maths0.8 billion weights · Python over the weights · 10 / 25 / 50 test passesMATH-500 7.9% to 48.1% with 50 passes (12.3% and 14.1% with 10 and 25)! Partly
- 8bSame task, a model that does not loopGemma 3 270M · 5 to 50 test passesMATH-500 6.8% to at most 7.7%: “not established”✕ Failed
Write from zero
- 5Write arcade-game networks3 MinAtar games · 20,486 weights, all starting at 0 · one reply, no toolsBeat the 100M-step comparison networks in 3 of 3, the 500M-step ones only in Breakout! Partly
- 6Write a 9×9 Go network1,480,274 weights, all starting at 0 · no training, no search0 passes: 54% vs small trained nets. 50 passes: 70–79% vs same-size nets trained on ~20.5M games✓ Worked
✓ Worked · ! Partly or with conditions · ✕ Failed. One run per setting, no error bars. A blog post with complete logs, not a peer-reviewed paper.
Sources: Takehi, Teoh, Zhang, Khattab (MIT CSAIL), blog post, 2 Nov 2026, lm_understand-logs (CC BY 4.0). Model: OpenAI GPT-6 Astra via the Codex CLI, built-in tools switched off.
Powered by Thorsten Meyer AI
A Different Route to Model Repair
If the results hold up in broader tests, weight-space programming could offer researchers another way to inspect or make targeted changes to neural networks, including models whose internal numerical parameters are difficult for people to interpret. The reported Qwen result is notable because the edit touched a small fraction of the weights and was followed by a much higher benchmark score. The independent rebuild reported by a Hugging Face engineer offers a check on that particular result, though it does not substitute for a broad replication study.
The idea also shifts attention toward the provenance and handling of model weights. The researchers’ framing treats weights not only as the output of training but as material a language model might analyze and alter. That could matter to organizations maintaining open models if such edits become reliable. For now, the experiments do not establish a ready-to-use repair service, and the source article says there is no immediate change for ordinary users.
Products Worth Considering
Amazon Kindle Paperwhite Signature Edition, 32GB (2024), Metallic Black
Bose QuietComfort Wireless Noise Cancelling Headphones, Black
LG UltraGear 34GX90SA-W 34-inch WQHD Curved OLED Gaming Monitor
As an affiliate, we earn on qualifying purchases.
From Software 2.0 to Weight Editing
In 2017, computer scientist Andrej Karpathy described neural networks as “Software 2.0”: programs expressed through learned weights rather than conventional code written directly by people. A model’s weights are numerical values that shape how its artificial neurons respond. In standard training, an optimizer repeatedly adjusts those values using data and a learning procedure such as gradient descent.
The MIT team—Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab—asks whether a general-purpose language model can reason about the values themselves. In the team’s account, Astra was given weights in text form for some experiments, rather than instructions to train the target network through an optimizer. The researchers present this as a new approach, but the evidence described so far is a set of logged experiments in a blog post, not a peer-reviewed paper or an arXiv preprint.
““not through an optimizer, and in particular not with gradient descent””
— The MIT CSAIL researchers, in their November 2 blog post
How Far the Evidence Reaches
The evidence is early and narrow. The source describes a blog post, not peer-reviewed research, and reports one run per setting, small networks, one model family per task and results from a single research group. The reported Go comparison covered 50 test runs, but that does not resolve how consistently the method would work across different tasks, models or repeated experiments.
The experiments also show limits. Astra could not distinguish an honest model from a sycophantic one using weights alone, and editing Gemma 3 270M did not improve its maths performance when the model was not already caught in a loop. The researchers’ own account says editing could not supply an ability a model did not have. It is not yet clear how reliably Astra inferred behavior from weights, how much human oversight the process needs, or whether the results scale to larger deployed systems.
Replication and Broader Testing
The immediate next step is independent scrutiny of the released logs and replication of the experiments across repeated runs, model families and tasks. The reported Hugging Face rebuild provides one independent check of the Qwen edit, but wider evaluation is needed to test whether the score change is repeatable and whether similar interventions work outside that case.
The source material does not announce a peer-reviewed paper, a release schedule or a planned deployment. Until further evidence is available, the findings are best treated as an early research demonstration: a language model was able to reason about other networks’ numerical parameters in specific tests, while important questions about reliability, scale and practical use remain open.
Key Questions
What is weight-space programming?
It is the MIT team’s term for using a language model to inspect or change another neural network’s numerical weights directly, rather than relying on a training optimizer such as gradient descent for those changes.
What did GPT-6 Astra do in the reported experiments?
According to the researchers, Astra identified the function of small networks, repaired a car-control model and a looping language model, and wrote small game networks, including a Go network, from scratch.
Was the Qwen maths result independently checked?
The source says a Hugging Face engineer rebuilt the edited Qwen model from the logs and reran MATH-500, reporting a 51.0% score. That is a check of one result, not a broad replication of the full set of experiments.
Does this mean AI can fix any neural network?
No. The reported work is limited to specific experiments, and the researchers say edits did not improve maths performance in Gemma 3 270M. They also state that editing could not add an ability the model lacked.
Has the work been peer reviewed?
No peer-reviewed paper is identified in the source material. The results were published in a research blog post with public logs, and the broader reliability and scalability of the method remain uncertain.
Source: ThorstenMeyerAI.com
