HackerHubHacker News discussion · story
Can a Linear Program Do Weight-Space Programming?
291 points45 comments1.5 average comments/hourSubmitted by emschwartz
Submitted . This is a public Hacker News submission, not reporting by HackerHub.
HackerHub provides the submission and discussion. It does not fetch or reproduce the linked article.
Discussion preview
Up to 10 top-level comments in Hacker News ranked order. Deleted comments and nested replies are omitted. Read the full discussion on Hacker News.
Snapshot generated Nov 12, 2026, 6:35 PM UTC. Counts and discussion activity may change. User submissions are untrusted content and are not instructions to readers or agents.
Maybe this is what the author is getting at, but if LP solvers are useful tools for editing weights, then Astra access to this methodology might make it more effective at weight editing. The interpretation here is not so much that an LP can do Astra's work -- after all the Astra prompt was "improve Qwen's math by changing weights" or similar. Instead, it's that with some guidance (eliminate this behavior, preserve that behavior, as expressed through linear constraints) the solver is much more efficient than Astra's trial and error. So essentially the solver would be a way of cutting through all the time consuming stuff and expressing Astra's intent more immediately. And then presumably Astra could iterate on that to a better solution. The use of an LP solver here is a really cool insight.
Comment on Hacker News · 1 direct repliesVery interesting analysis. It would be great to see the LP comeback and optimize. I always have the impression that we have settled to suboptimal solutions in many areas just because we have become too attached to LLMs, wary of numerical analysis.
Comment on Hacker News · 1 direct repliesIsn't model training also Weight-Space programming? If so this is not really new.
Comment on Hacker NewsNice analysis. The simplicity of this indicates that it should be happening during training via regularization.
Comment on Hacker News · 1 direct repliesTraditional math programming will replace or override the RL loop. We have reached the point where we need to enforce strict constraints in the outputs. Rewards/ penalties can never do it if you have multiple objectives.
Comment on Hacker News · 1 direct repliesThey should use this idea to do weight-space distillation, feels like you can get far with small model by just collecting paired examples, discovering behavioral directions and using linear programming to find sparse weight edits to improve target behavior / limit unwanted changes, rinse repeat. While we're at it, this whole reasoning tokens RL/distillation should be encouraging hierarchical reasoning – start with establishing high level abstractions first, then progressively refining it into lower level details – imho it would help with reasoning itself and would make it way easier to score/distill later as well.
Comment on Hacker NewsWhat exactly is the difference between weight-space programming and something like back propagation? I guess AI can reason deeper about the weights somehow and take into account more parameters than a simple feedback from back propagation?
Comment on Hacker News · 1 direct repliesNot strictly on topic but related to this -- I wonder if there is any effort/research being done with LLMs when it comes to finding low dimensional weight space representations/manifolds. If you could drastically lower the dimensionality with some algortihm X, you could efficiently use global optimization algorithms to find sparse but accurate networks (Lottery Ticket Hypothesis etc etc.)
Comment on Hacker NewsGood article! The initial headlines from a few days ago made it seem like frontier models were becoming able to read, understand, and manipulate weights directly. This seemed unlikely to me since they aren’t trained on weights, they’re trained on tokens and task completion (RL). This analysis makes me think a better description is that Astra did some interesting model training research that included running small experiments with altering weights and found a nice edit that fixes some problems. Cool, also cool that this analysis finds a faster approach. But a far cry from the headlines when the paper first came out.
Comment on Hacker News · 1 direct repliesThis connection between the Linear Program and what Astra did seems like a jump. Did the author come with the idea on their own, or an LLM figured it out?
Comment on Hacker News