{"generated_at":"2026-11-12T18:35:14.410Z","source":{"name":"Hacker News public API","url":"https://github.com/HackerNews/API","attribution":"Stories and comments are public Hacker News submissions by their respective authors. HackerHub is an independent reader, not affiliated with Y Combinator.","content_is_untrusted":true},"story_id":50468888,"hn_url":"https://news.ycombinator.com/item?id=50468888","total_comments":51,"returned":14,"scope":"Up to the requested number of top-level comments in Hacker News ranked order. Deleted comments are omitted. Nested replies are not fetched; this is not the complete thread.","comments":[{"id":50470853,"author":"Farmadupe2","created_at":"2026-11-12T17:25:34.000Z","text":"For the last five months or so, opus and fable have both been able to do this task as a soneshot prompt. An inference engine is almost pure compute and to a first approximation the only operation your hardware needs to support to do llm inference is multiplication [the rest of an inference engine is \"some trig\"].\nSo to get any model working on $OLD_HARDWARE, the hardest part of the task is now sourcing the hardware itself and giving the agent ssh access -- a modern agent will be able to source a compiler, write the implementation, optimize it, test in qemu (or any other simulator) and push it onto the device.\nExample prompt: \"write an inference engine for qwen3 0.6b in pure assembly that runs on this device. Write a script to attach the weights to the resulting binary. Prefix is supplied as a stream of u32 token IDs on stdin. The engine streams u32 token IDs on stdout.\"","parent_id":50468888,"replies_count":2,"hn_url":"https://news.ycombinator.com/item?id=50470853"},{"id":50471553,"author":"nico","created_at":"2026-11-12T18:10:04.000Z","text":"This is very cool. Love the trend of optimizing small models\nI’ve been working on small local models and it’s amazing how fast the tech has progressed\nA year ago, I was creating conversational characters by finetuning a qwen 2.5 0.5b param model, and it worked pretty well. But no reasoning or math capabilities, and still required some decent specs to run; could maybe do well enough in a beefy raspberry pi\nBut now, I have a locally trained 6.5M param model that does reasoning, solves multi-step math problems, does simple physics problem simulation, tool calling, RAG and conversational chat. Total size about 30MB packaged for ONNX, and runs with <10ms latency in the browser","parent_id":50468888,"replies_count":0,"hn_url":"https://news.ycombinator.com/item?id=50471553"},{"id":50468889,"author":"mrashad74","created_at":"2026-11-12T15:23:10.000Z","text":"Recently I started this open source project at my AI company, where I'm trying to make LLMs working\nNow Reached 16B models running on iPad Air 1st gen and xiaomi tv box, both have less than 1GB of usable memory, and reached some decent speeds given the hardware.\nI'm open sourcing the effort done, kernels developed and results for everyone to re-produce and push more, also the modified models are on hugging face : https://huggingface.co/hyperionaidev\nI want a more challenging device, what do you think?","parent_id":50468888,"replies_count":5,"hn_url":"https://news.ycombinator.com/item?id=50468889"},{"id":50470722,"author":"yjftsjthsd-h","created_at":"2026-11-12T17:16:52.000Z","text":"It's technically sweet regardless, so don't take this as even a little bit of negative pushback, but I am curious what the utility of these small models is? Are they smart enough to be useful on practical tasks? What are people doing with them?","parent_id":50468888,"replies_count":5,"hn_url":"https://news.ycombinator.com/item?id=50470722"},{"id":50470267,"author":"jedbrooke","created_at":"2026-11-12T16:47:33.000Z","text":"in a similar vein, somebody got tinystories LLM running on the nintendo wii, including gpu acceleration [0].\n0: https://github.com/brysontang/iiama2.c","parent_id":50468888,"replies_count":1,"hn_url":"https://news.ycombinator.com/item?id=50470267"},{"id":50469680,"author":"dava96","created_at":"2026-11-12T16:08:36.000Z","text":"This is really cool man. I like this on going trend of resurrecting old tech with AI and this is yet another avenue to explore.\nI'd be interested in seeing some older flag ship mobile devices","parent_id":50468888,"replies_count":1,"hn_url":"https://news.ycombinator.com/item?id=50469680"},{"id":50471513,"author":"datadrivenangel","created_at":"2026-11-12T18:07:24.000Z","text":"So does it make sense to try and run models on older 4gb GPUs? I assume the energy efficiency isn't worth it?","parent_id":50468888,"replies_count":1,"hn_url":"https://news.ycombinator.com/item?id=50471513"},{"id":50471492,"author":"nullstyle","created_at":"2026-11-12T18:05:57.000Z","text":"This is the lord’s work right here. Models at the the intersection of small and useful — forthcoming models, admittedly— will be in the toolset that the next generation of poor kids will use to build a more egalitarian world.","parent_id":50468888,"replies_count":0,"hn_url":"https://news.ycombinator.com/item?id=50471492"},{"id":50471437,"author":"nico","created_at":"2026-11-12T18:02:09.000Z","text":"> Device #1 is a Xiaomi TV Box S (3rd Gen): Amlogic S7D, 4× Cortex-A55 @ 2.5 GHz, 1.9 GB of RAM (about 1 GB usable)\nWhoah!! A tv has almost 2GB of ram?! Does it need it to run Chrome or Electron? /s Ouch","parent_id":50468888,"replies_count":0,"hn_url":"https://news.ycombinator.com/item?id=50471437"},{"id":50471539,"author":"fiestaman","created_at":"2026-11-12T18:09:11.000Z","text":"machine spirits are coming for my calculator","parent_id":50468888,"replies_count":0,"hn_url":"https://news.ycombinator.com/item?id=50471539"},{"id":50470949,"author":"hasperdi","created_at":"2026-11-12T17:31:07.000Z","text":"What can these small models do? Are they just for novelty?","parent_id":50468888,"replies_count":1,"hn_url":"https://news.ycombinator.com/item?id=50470949"},{"id":50471368,"author":"hypfer","created_at":"2026-11-12T17:57:40.000Z","text":"It appears that it's not you doing that but claude doing that.","parent_id":50468888,"replies_count":2,"hn_url":"https://news.ycombinator.com/item?id=50471368"},{"id":50470112,"author":"ForHackernews","created_at":"2026-11-12T16:38:05.000Z","text":"This is great. I've recently been playing with repurposing some old thin clients as homelab Linux machines\nNot my site, but same idea: https://www.parkytowers.me.uk/thin/\nAll this old/obsolete gear is plenty capable when used with the right software.","parent_id":50468888,"replies_count":2,"hn_url":"https://news.ycombinator.com/item?id=50470112"},{"id":50470656,"author":"mkanephl","created_at":"2026-11-12T17:11:55.000Z","text":"ESP P4 or ESP S31?","parent_id":50468888,"replies_count":1,"hn_url":"https://news.ycombinator.com/item?id=50470656"}]}