Freeman Dyson - Fermi's rejection of our work (94/157)

Full paper here[1], and here's[2] the earlier paper about the model itself. They overloaded the CCC acronym, which has also been used for Penrose's Conformal cyclic cosmology[3], though can't say which is more speculative. I'm also reminded of Von Neumann's elephant[4], given that they vary the constants it doesn't seem that surprising I suppose that they can get a fit. Freeman Dyson learned this the hard way[5]. This is just one test, the article mentions a few others that they'll need to pass to be a serious contender. Will be interesting to see how it goes. [1]: link [2]: link [3]: link [4]: link [5]: this video

I found the reference. It was an interview with Freeman Dyson where he said "quarks were invented", and I don't think it's a slip of the tongue. It's at 3:17 in this video of an interview with him this video It's representative of a view that there's a thing (a particle) that explains another thing (a force) that was consistent with both theory and experiment. Thus, a quark could be a particle, and it was whatever the experimentalists and theorists said it was, however it was measured or contemplated. For a deeper meaning it becomes an exercise in hermeneutics i.e what does it mean when we say "particle"? That was the point of the original piece - there is no uncontested view of a particle's form, should one even exist. Each field, in order to advance, finds it useful to interpret it, or think about it, in different ways.

I haven't had a chance to read that, but that quote suggests I should (especially considering the author and the editors). I often refer to "Elephant Fitting" w.r.t these systems. I suspect you understand this, but I think most think it is just about overfitting. But the way problem isn't about the number of parameters, but that parameters need to be justified. As explained by Dyson here[0]. Vladimir's quote really reminds me of this. Fermi likewise was stressing the importance of theory. I think it is a profound quote, and you were (are?) lucky to have that friendship. I do think abstraction is at the heart of intelligence. François Chollet discusses it a lot, and he's far from alone. It seems to be well agreed upon in the neuroscience and cognitive science communities. I think this is essential to understand in our path forward to developing intelligent systems, because there are plenty of problems that need to be solved in which there is no algorithmic procedure. Where there is no explicit density function. Intractable, doubly intractable, and more problems. Maybe we're just too dumb, but it's clear there are plateaus where luck is needed to advance. I do not believe our current machines would be capable of closing a gap. [0] this video

Interestingly a lot of the math and physics people in the ML community are considered "grumpy researchers." A joke apparent with this starter pack[0]. From my personal experience (undergrad physics, worked as engineer, came to CS & ML because I liked the math), there's a lot of pushback. - I've been told that the math doesn't matter/you don't need math. - I've heard very prominent researchers say "fuck theorists" - I've seen papers routinely rejected for improving training techniques with reviewers say "just tune a large model" - I see papers that show improvements when conditioning comparisons on compute restraints because "not enough datasets" or "but does it scale" (these questions can always be asked but require exponentially more work) - I've been told I'm gatekeeping for saying "you don't need math to make good models, but you need it to know why your models are wrong" (yes, this is a reference) - when pointing out math or statistical errors I'm told it doesn't matter - and much more. I've heard this from my advisor, dissertation committee, bosses[1], peers, and others (of course, HN). If my experience is short of being rare, I think it explains the grumpy group[2]. But I'm also not too surprised with how common it is in CS for people to claim that everything is easy or that leet code is proof of competence (as opposed to evidence). I think unfortunately the problem is a bit bigger, but it isn't unsolvable. Really, it is "easily" solvable since it just requires us to ma

I don't think this makes sense and I'm not quite sure why you went to ML, but that's okay. I am a machine learning researcher, but also frustrated with the state of machine learning, in part because, well... you can probably see how "proof by empirical evidence" is dialed up to 11. Sorry, long answer incoming. It is far from complete too but I think it will help build strong intuition around your questions. Will knowledge transfer? That entirely depends on the new problem. It also entirely depends on how related the problem is. But also, what information was used to solve the pre-transfer state. Take LLMs for example. There's lots of works that have shown them being difficult to train for solving calculations. Where they will do well on problems with the same number of digits but this will degrade rapidly as number of digits increase. It can be weird to read some of these papers as there will sometimes be periodic relationships with the number of digits but that should give us information about how they're encoding the problems. But that lack of transferability indicates that despite the problem solving and what we'd believe is actually just the same problem, doesn't mean it is. So you have to be really careful here, because us humans are really fucking good at generalization (yeah, we also suck, but a big part is our proficiency makes us recognize where we lack. But also, this is more a "humans can" more than "humans do" type of thing. So be careful when comparing). This gen

> I feel like extensive pretraining goes against the spirit of generality. What do you mean by generality? Pretraining is fine. It is even fine for pursuit to AGI. Humans and every animal has "baked in" memory. You're born knowing how to breath and have latent fears (chickens and hawks). Generalization is the ability to learn on a subset of something and then adapt to the entire (or a much larger portion) of the superset. It's always been that way. Humans do this, right? You learn some addition, subtraction, multiplication, and division and then you can do novel problems you've never seen before that are extremely different. We are extremely general here because we've learned the causal rule set. It isn't just memorization. This is also true for things like physics, and is literally the point of science. Causality is baked into scientific learning. Of course, it is problematic when someone learns a little bit of something and thinks they know way more about it, but unfortunately ego is quite common. But also, I'm a bit with you. At least with what I think you're getting at. These LLMs are difficult to evaluate because we have no idea what they're trained on and you can't really know what is new, what is a slight variation from the training, and this is even more difficult considering the number of dimensions involved (meaning things may be nearly identical in latent space though they don't appear so to us humans). I think there's still a lot of ML/AI research that can and SHO

> the primary aim isn't really to find out whether a result is true but why it's true. I'm honestly surprised that there are mathematicians that think differently (my background[0]). There are so many famous mathematicians stating this through the years. Some more subtle like Poincare stating that math is not the study of numbers but the relationship between them, while others far more explicit. This sounds more like what I hear from the common public who think mathematics is discovered and not invented (how does anyone think anything different after taking Abstract Algebra?). But being over in the AI/ML world now, this is my NUMBER ONE gripe. Very few are trying to understand why things are working. I'd argue that the biggest reason machines are black boxes are because no one is bothering to look inside of them. You can't solve things like hallucinations and errors without understanding these machines (and there's a lot we already do understand). There's a strong pushback against mathematics and I really don't understand why. It has so many tools that can help us move forward, but yes, it takes a lot of work. It's bad enough I know people who have gotten PhDs from top CS schools (top 3!) and don't understand things like probability distributions. Unfortunately doing great things takes great work and great effort. I really do want to see the birth of AI, I wouldn't be doing this if I didn't, but I think it'd be naive to believe that this grand challenge can entirely be solved

Just curious, how "deep" have you gone into the theory? What resources have you used? How strong is your math background? Unfortunately a lot of the theory does require some heavy mathematics, the type you won't see in a typical undergraduate degree even for more math heavy subjects like physics. Topics such as differential geometry, metric theory, set theory, abstract algebra, and high dimensional statistics. But I do promise that the theory helps and can build some very strong intuition. It is also extremely important that you have a deep understanding of what these mathematical operations are doing . It does look like this exercise book is trying to build that intuition, but I haven't read it in depth. I can say it is a good start, but only the very beginning of the theory journey. There is a long road ahead beyond this. > how it makes me choose the correct number of neurons in a layer, how many layers, Take a look at the Whitney embedding theorem. While this isn't a precise answer, it'll help you gain some intuition about the minimal number of parameters you need (and the VGG paper will help you understand width vs depth). In a transformer, the MLP layer post attention scales up 4x the dimensions before coming down, which allows for untangling any knots in the data. While 2x is the minimum, 4x creates a smoother landscape and so the problem can be solved more easily. Some of this is discussed in paper (Schaeffer, Miranda, and Koyejo) that counters the famous Emergent Abil

I do say the same think about JKR, btw. And for the same things, because the content she writes. I think you focused on the fanfic part and not the part where I'm criticizing where they say their stuff is the most important to keep humanity alive and they're charging money for it. Meanwhile, you may notice in academia we publish papers to make them freely available, like on arXiv. If it is that important that people need to know, you make it available. The second person, Hinton, is not as good of an authority as you'd expect. Though I do understand why people take him seriously. Fwiw, his Nobel was wildly controversial. Be careful, prizes often have political components. I have degrees in both CS and physics (and am an ML researcher) and both communities thought it was really weird. I'll let you guess which community found it insulting. I want to remind you, in 2016 Hinton famously said[0] | Let me start by saying a few things that seem obvious. I think if you work as a radiologist, you're like the coyote that's already over the edge of the cliff, but hasn't yet looked down so doesn't realize there's no ground beneath him. People should stop training radiologists now. It's just completely obvious that within 5 years that deep learning is going to be better than radiologists because it's going to get a lot more experience. It might take 10 years, but we've got plenty of radiologists already. We're 10 years in now. Hinton has shown he's about as good at making predictions as Mu

> The fact that we have figured out how to translate language into something a computer can "understand" should thrill linguists. I think they are really excited by this. There seems no deficiency of linguists using these machines. But I think it is important to distinguish the ability to understand language and translate it. Enough that you yourself put quotes around "understanding". This can often be a challenge for many translators, not knowing how to properly translate something because of underlying context. Our communication runs far deeper than the words we speak or write on a page. This is much of what linguistics is about, this depth. (Or at least that's what they've told me, since I'm not a linguist) This seems to be the distinction Chomsky is trying to make. > The main debate now is over the semantics of words like "understanding" and whether or not an LLM is conscious in the same way as a human being (it isn't). Exactly. Here, I'm on the side of Chomsky and I don't think there's much of a debate to be had. We have a long history of being able to make accurate predictions while erroneously understanding the underlying causal nature. My background is physics, and I moved into CS (degrees in both), working on ML. I see my peers at the top like Hinton[0] and Sutskever[1] making absurd claims. I call them absurd, because it is a mistake we've made over and over in the field of physics[2,3]. One of those lessons you learn again and again, because it is so easy to make t

I think many people are missing the core of what Chomsky is saying. It is often easy to miscommunicate and I think this is primarily what is happening. I think the analogy he gives here really helps emphasize what he's trying to say. If you're only going to read one part, I think it is this: | I mentioned insect navigation, which is an astonishing achievement. Insect scientists have made much progress in studying how it is achieved, though the neurophysiology, a very difficult matter, remains elusive, along with evolution of the systems. The same is true of the amazing feats of birds and sea turtles that travel thousands of miles and unerringly return to the place of origin. | Suppose Tom Jones, a proponent of engineering AI, comes along and says: “Your work has all been refuted. The problem is solved. Commercial airline pilots achieve the same or even better results all the time.” | If even bothering to respond, we’d laugh. | Take the case of the seafaring exploits of Polynesians, still alive among Indigenous tribes, using stars, wind, currents to land their canoes at a designated spot hundreds of miles away. This too has been the topic of much research to find out how they do it. Tom Jones has the answer: “Stop wasting your time; naval vessels do it all the time.” | Same response. It is easy to look at metrics of performance and call things solved. But there's much more depth to these problems than our abilities to solve some task. It's not about just the ability to do some

> only a small subset will contain actual understanding, a property that is independent of the model's input-output behavior? I think this is close enough. I'd say "a model's ability to make accurate predictions is not necessarily related to the model's ability to generate counterfactual predictions." I'm saying, you can make extremely accurate predictions with an incorrect world model. This isn't conjecture either, this is something we're extremely confident about in science. > I don't think I can agree. In your 2+a+b-2 example, that is an unnecessarily convoluted, but entirely correct model of addition. I gave it as a trivial example, not as a complete one (as stated). So be careful with extrapolating limitations of the example with limitations of the argument. For a more complex example I highly suggest looking at the actual history around the heliocentric vs geocentric debate. You'll have to make an active effort to understand this because what you were taught in school is very likely an (very reasonable) over simplification. Would you like a much more complex mathematical example? It'll take a little to construct and it'll be a lot harder to understand. As a simple example you can always take a Taylor expansion of something so you can approximate it, but if you want an example that is wrong and not through approximation then I'll need some time (and a specific ask). Here's a pretty famous example with Freeman Dyson recounting an experience with Fermi[0]. Dyson's model ma

> that does not have a technical meaning I don't think the definition is very refined, but I think we should be careful to differentiate that from useless or meaningless. I would say most definitions are accurate, but not precise. It's a hard problem, but we are making progress on it. We will probably get there, but it's going to end up being very nuanced and already it is important to recognize that the word means different things in vernacular and in even differing research domains. Words are overloaded and I think we need to recognize this divergence and that we are gravely miscommunicating by assuming the definitions are obvious. I'm not sure why we don't do more to work together on this. In our field we seem to think we got it all covered and don't need others. I don't get that. > In this view, if a machine performs a task as well as a human, it understands it exactly as much as a human. And I do not think this is accurate at all. I would not say my calculator understands math despite it being able to do it better than me. I can say the same thing about a lot of different things which we don't attribute intelligence to. I'm sorry, but the logic doesn't hold. Okay, you might take an out by saying the calculator can't do abstract math like I can, right? Well we're going to run into that same problem. You can't test your way out of it. We've known this in hard sciences like physics for centuries. It's why physicists do much more than just experiments. There's the classic st

I think a lot of people are really bad at evaluating world models. Feifei is right here that they are multimodal but really they must codify a physics. I don't mean "physics" but "a physics". I also think it's naïve to think this can be done through data alone. I mean just ask a physicist...[0]. But why people are really bad at evaluating them is because the details dominate. What matters here is consistency. We need invariance to some things and equivariance to others. As evaluators we tend to be hopeful so the subtle changes frame to frame are overlooked though thats kinda the most important part. It can't just be similar to the last frame, but needs be exactly the same. You need equivariance to translation, yet that's still not happening in any of these models (and it's not a limitation of attention or transformers). You're just going to have a really hard time getting all this data even though by doing that you'll look like you're progressing because you're better fitting it. But in the end the models will need to create some compact formulation representing concepts such as motion. Or in other words, a physics. And it's not like physicists aren't know for being detail oriented and nitpicky over nuances. That is breed into then with good reason [0] this video