“ I think this is very mysterious….It works, which is amazing. Our minds are blown by how powerful these things are. And yet for all their success, the recipes are more alchemy than chemistry: “We figured out certain incantations at midnight after mixing up some ingredients,”

      - Mikhail Belkin, computer scientist at the University of California, San Diego (source [<https://archive.is/gYyVt>](<https://archive.is/gYyVt>))

It did stuff, and a theory to try and explain why came after!

The largest models, and large language models in particular, seem to behave in ways textbook math says they shouldn’t. …nobody knows exactly how—or why—it works.

“Obviously, we’re not completely ignorant,” says Mikhail Belkin, a computer scientist at the University of California, San Diego. “But our theoretical analysis is so far off what these models can do. Like, why can they learn language? I think this is very mysterious.”

The biggest models are now so complex that researchers are studying them as if they were strange natural phenomena, carrying out experiments and trying to explain the results. Many of those observations fly in the face of classical statistics, which had provided our best set of explanations for how predictive models behave.

https://podcasts.apple.com/ci/podcast/large-language-models-can-do-jaw-dropping-things-but/id1523584878?i=1000664597151**knows why** (March 2024 - Paywall)

https://archive.is/gYyVt knows why (Accessible)

https://www.technologyreview.com/2024/03/05/1089449/nobody-knows-how-ai-works/

https://www.scientificamerican.com/article/how-ai-knows-things-no-one-told-it/

Geoffrey Hinton admits this on 60 minutes with Scott Pelley:

Pelley: What do you mean we don't know exactly how it works? It was designed by people.

Hinton: No, it wasn't. What we did was we designed the learning algorithm. That's a bit like designing the principle of evolution. But when this learning algorithm then interacts with data, it produces complicated neural networks that are good at doing things. But we don't really understand exactly how they do those things.

https://archive.is/2023.10.09-030925/https://www.cbsnews.com/news/geoffrey-hinton-ai-dangers-60-minutes-transcript/