13.2M views · Mar 1, 2022 · 21:42 · 7 HN recommenders · 20 mentions · discussed 2022–2025
Veritasium explains why analog computing may suit AI workloads, covering old analog computers, The Analog Thing and Mythic's analog chips that use flash memory for neural network inference. It argues digital hardware is hitting limits.
Why Hacker News points people here: Treated as the accessible reference on analog computing for neural nets, especially the Mythic AI segment.
“Veritasium explains it really well in general here (and demos the device)”HN comment, 2024
“suddenly enough a Vertiasium video pops up and about everything just slots into place”
Made neural networks click when other resources failed
Agreed. I've never really gotten into neural networks, most of the times I start but get distracted or bored/lost and suddenly enough a Vertiasium video pops up and about everything just slots into place, stuff you've glossed over or heard before just make sense. this video . Start at minute 4:00.
“here's two youtube videos that explain is much better than I could.”
Explains analog hardware for matrix operations better than commenter could
basicly, we can do things like matrix operations and differntial equations using specialized hardware that performs computation over analog signals rather than buidlign discrete systems. the end result is that we dont' need anywhere near the level of compute to do the job as its done using an analog of the physical process. here's two youtube videos that explain is much better than I could. link this video suffice to say, your average house fly exhibits behavior that absolutely dominates our best drones and with a fraction of the circuitry.
“Great Veritasium video on YouTube about analog neuromorphic architectures and their promise for lower-power operations”
Covers analog neuromorphic architectures and low-power promise
Great Veritasium video on YouTube about analog neuromorphic architectures and their promise for lower-power operations: this video
“Veritasium has two great videos on it”
Overview of analog AI research
It's a big area of research in AI. Veritasium has two great videos on it: this video link I found a hackernews discussion for the second one: link
“I found this Veritasium video interesting, about a company that is paring down models so they fit on analog chips”
Shows shrinking models to fit analog chips for self-contained devices
I found this Veritasium video interesting, about a company that is paring down models so they fit on analog chips, so the chips can be used in self-contained devices. It also appears that quantum computers may be even better for matrix math that transformers rely on. If all these hardware advances pan out... this video link
“Veritasium has a good video about analog computers.”
Covers how analog works, uses, pros and cons, history, future
Veritasium has a good video about analog computers. It covers how they work, what they are used for, pros and cons compared to digital, what they look like today, a bit of the history and their possible future. this video
“Veritasium explains it really well in general here (and demos the device)”
Clear explanation of analog computers plus device demo
I think they're saying analog computers could be the future of computing. Veritasium explains it really well in general here (and demos the device) this video
Cited as example of improving compute hardware
link This is a rather intuitive mathematical proof that neural nets can emulate all functions in many dimensions. Knowing integrals (Calc 102) would help. With this information it’s either that self-driving is eventually possible or human brains are not mathematically representable (i.e. something akin to a soul). Meanwhile, the data collection/storage is getting better (all Teslas are continually testing FSD and reporting failure cases, synthetic generation of “good and unique” perfectly labeled data[0] is only a year or two “old”). Sensors/cameras are improving (and have a lot more room to improve). The compute hardware is getting better and cheaper (analog computers[1], TSMC 2nm and beyond, Apple/Google/Teslas neural cores, etc). Additionally, given Waymo is already in production without humans in PHX it’s arguable that we already have “autonomous vehicles”. We now just need them to improve slightly and for Waymo or similar to test in the top 25 cities in the US. Which is mostly a matter of capital and time not technology limits. Maybe weather will be an issue for a few more years/decades - but even if Waymo only operated in Summer, I’d be happy to reduce my Lyft costs - and with enough data and maybe better sensors the neural nets will be able to handle weather too. Especially if the car just goes slow. [0] link [1] this video [2] link
“Neural networks / analog stuff”
One of many links in rambling toolkit list
Warning: Bit of a 'Hitch hikers guide to the galaxy"/conway game of life yak shave for a reply. Nice start to providing a simple way to visually interact/experiment/manipulate different multiple Neural network concepts by loading/transforming any traditional code (working or not) & reshuffling/tesselating the structual representation of the code to generate a turing tape path via color execution paths. Switching to pi calculus could provide the theory to support coroutine/coprossing/ threading" multiple interacting tape paths/color combinations/path diffs/scaled recusion to perform assembly language instructions. Raster graphics without the yak ( link ) Feed back between tape & "traditional code" as neural network layer(s)! Flexiable code (via tessilations) without changing the actual imported source code! So, complete svg (tape interactions) web page as code source on how to interact with web page, ( link ) what's displayed/played, documentation & usage manual included within code structure! Unicode permits implied piping rich kern font without all the tech fluff. Unicode fails because allows infinate multics and is there for k&r incomplete. But, this approach is way more dimensional/colorful than unicode/tech stuff. ?? can one's code unintentially provide a wordle problem & solution in one package ?? ( link ) ?? turing complete fourier transforms link toolkit concepts: Neural networks / analog stuff : this video perceptrons: this video spreadsheets (: link (turing sprea
“which is a thing these days”
Evidence that analog coprocessors exist today
Considering evolution optimized for biological fitness and not intelligence, it would be incredibly surprising if it wasn't possible to do better, especially using a vastly different architecture. You could also build an AGI to run on a digital CPU but interact with a simulation that used analog coprocessors (which is a thing these days this video ). You can also include quantum coprocessors (which already exist in their infancy) for various things (probably only useful for quantum simulations at this point though). Also considering recent ML work, an AGI is more likely going to run on something similar to a GPU than a CPU.
“they show some startup products near the end.”
Shows startup analog computing products near the end
Fire control computers, like on a navy ship, were faster than digital computers of the day. YouTube has a number of videos on them. An opamp performs multiplication faster than a digital computer (speed of light vs a few cycles). It's not super useful on its own, but it does fit the criteria. In Veritasium's video 2/2 on analog computers [0] they show some startup products near the end. [0] this video
“The company (Mythic AI) in the article was featured on a Veritasium video[1] about analog computing.”
Featured Mythic AI, the company in the article
The company (Mythic AI) in the article was featured on a Veritasium video[1] about analog computing. The interesting part for analog is obviously going to be wear and repeatability, because the digital layer auto-corrects the errors until it becomes damaged and unusable clearly, while analog will slide off somewhat gracefully but you need a way to detect it happening multiple layers later. AI is somewhat tolerant of errors since, the cumulative error across a lot of biases can approach zero if the errors are random. [1] - this video
“Veritasium did a couple of videos on analog computing a year ago.”
Prior Veritasium coverage of analog computing
That's really neat. Veritasium did a couple of videos on analog computing a year ago. I'm glad there are people lowering the barrier to entry. link this video
Timestamped link to the analog neural net chip segment
link is an analog chip for doing neural nets. this video (this is a time stamp'ed link for a video mentioned elsewhere in the comments).
“The Mythic analog chip [2] is pumping 25 TOPS at 3W.”
Source for Mythic analog chip performance figures
Actually, I have a Pixel 7 with the Tensor G2. I can't find the tera operations per second on it, but Google has said that its [1] 60% more powerful than their Tensor Coral (4 TOPS at 2W of power). That puts the G2 at about 6.4 TOPS. The Mythic analog chip [2] is pumping 25 TOPS at 3W. For comparison, modern (digital) GPUs are doing 25-100 TOPS at 50W-100W...while costing >$1000. So, its a big jump. [1] link [2] this video
“analog chips like what MythicAI is developing seem like the next obvious leap forward for deploying inferences broadly.”
Source for Mythic analog inference approach
analog chips like what MythicAI is developing seem like the next obvious leap forward for deploying inferences broadly. ASIC/FPGA wouldn't be much different than a GPU. ASIC seems like a brittle solution this video
“They visit Texas company Mythic AI to discuss how they use flash memory for machine learning.”
Covers Mythic AI using flash memory for machine learning
The Veritasium Youtube channel did a video about this about a year ago: this video They visit Texas company Mythic AI to discuss how they use flash memory for machine learning. There's a California company named Syntiant doing something similar.
“Veritasium did a video on analog computers, which included a segment on Mythic AI”
Segment on Mythic AI using NAND flash as analog compute
A couple years ago Veritasium did a video on analog computers, which included a segment on Mythic AI that uses NAND flash cells kind of "undervolted" as an analog computer to run neural networks. this video link
“This paradigm for computing was already covered three years ago by Veratasium in”
Earlier coverage of analog approximations of neural nets
This paradigm for computing was already covered three years ago by Veratasium in this video . Maybe not the specific photonic system that they are describing. Which I'm sure has some significant improvements over what existed then. But the idea of using analog approximations of existing neural net AI models, to allow us to run AI models far more cheaply, with far less energy. Whether or not this system is the one that wins out, I'm very sure that AI run on an analog system will have a very important role to play in the future. It will allow technologies like guiding autonomous robots with AI models running on hardware inside of the robot.
“The tech is great, but the (presumably most compelling) use cases this guy can come up with are automated surveillance and efficiencies in junk food production.”
this video Start here to hear the deplorable and utterly uninspiring uses the guy comes up with for analogue chips, then rewind to the start. The tech is great, but the (presumably most compelling) use cases this guy can come up with are automated surveillance and efficiencies in junk food production.
