786K views · May 25, 2023 · 42:40 · 6 HN recommenders · 11 mentions · discussed 2023–2025
Andrej Karpathy walks through how GPT assistants are trained, from tokenization and pretraining to supervised finetuning and RLHF, and gives practical prompting strategies and a view of the tooling ecosystem.
Why Hacker News points people here: Recommended as the clearest overview of how LLM assistants are built and how to use them well.
“It changed the way I use GPT-4.”HN comment, 2023
“It changed the way I use GPT-4.”
Explains how LLMs work with good prompting ideas
I would highly recommend watching - State of GPT video by Karpathy at Microsoft Build to get an idea of how LLM’s work. There are some really good prompting ideas. It changed the way I use GPT-4. Video: this video
“provides a through introduction to the process used to train”
Deep summary of GPT training phases and techniques
I share many of your concerns and frustrations, although I suspect what you're asking for their consider a moat along the lines of a trade secret, rivaled only by the collection of performance improvement techniques they've amassed in 1000s-10,000s of training runs, 100s of engineers, and (hundreds of?) millions spent on compute. People are hired and praised in the community for their skill in cleaning data. A non-answer for you, but for curious others, [State of GPT] 10 days ago provides a through introduction to the process used to train, Karpathy speaking at a Microsoft event providing a deep summary review of concepts, training phases, and techniques proving useful in the world of Generative Pretrained Transformers. [State of GPT]: this video
“(~hr) Karpathy talk”
Hour-long Karpathy talk listed among good prompting guides
I recognize there's plenty of catnip here when it comes to calling this "engineering" or not, however, whatever you want to call it (prompt fiddling?), the techniques are crucial if you want to achieve reasonably consistent output from current-state LLMs. As models improve concerns about context window limitations will be reduced and it will be easier to discern user intent. These are good straight-to-the-point guides: - Prompt Engineering by BrexHQ: link - OpenAI guidance: link - link - (great examples): link - (~hr) Karpathy talk: this video tl;dr: - Begin with the best and most forgiving model available, optimizing it later if necessary. - Write your prompts in a clear, specific, and detailed manner - Be mindful, however, of the tradeoff between including more detail and increasing latency or cost. - If you're doing any complex reasoning, ask the model to show its work, help "stretch out" computation over more tokens - "Escape" any included or quoted text properly to avoid confusing the model. - Keep track of your token budget, considering the context window in conversations and response length. - Employ an iterative process of measuring, adjusting, and improving your prompt engineering techniques.
“Karpathy explains it far better than I did in his”
Karpathy explains LLM reasoning and prompting well
Karpathy explains it far better than I did in his "State of GPT" talk about a month ago: this video Yes, language can do that, but books and other texts, on average, leave out a lot of steps that humans have learned to do naturally. They can be prompted to do them, and their answers improve, indicates that there is definitely some level of "reasoning" there. But it has weird limitations in its current forms. I expect that to improve pretty quickly, tbh.
“State of GPT by Karpathy”
Listed among resources for getting up to speed on LLMs
It sounds like you want more broad stuff, not necessarily learning how to train models. More like learning to use them and how they work. link and Hacker's Guide to LLMs by Jeremy from Fast.ai - link State of GPT by Karpathy - this video LLMs by 3b1b - link Visualizing transformers by 3b1b - link How ChatGPT trained - link AI in a nutshell - link How Carlini uses LLMs - link For staying updated: X/Twitter & Bluesky. Go and follow people that work at OpenAI, Anthropic, Google DeepMind, and xAI. Podcasts: No Priors, Generally Intelligent, Dwarkesh Patel, Sequoia's "Training Data"
“Explains many details in a form suitable for beginners.”
Beginner-friendly explanation of the stages of LLM training
Thanks. Here is presentation by Karpathy explaining different stages of LLM training. Explains many details in a form suitable for beginners. this video
“At Build 2023, Andrej Karpathy outlined[2] how they trained GPT 4.”
Karpathy outlined GPT-4's training stages
Chat GPT 4 has ongoing training, such as using Reinforcement Learning from Human Feedback (RLHF) to tune it to provide "better" responses, "safer" answers, and to generally obey the system prompts. There's a release every few weeks. Yes, I've noticed too that recently it has become very "cagey", qualifying everything to death with "As an AI model...". A paper[1] that took snapshots monthly mentioned that as the initial bulk self-supervised learning went on, the model became smarter, as expected. However, once the "clicker training" was imposed on top to make it behave, its peak capabilities were reduced. I'm not sure if it's in the paper or the associated video, but the author mentioned that the original unrestricted model would provide probability estimates using percentages, and it was a very accurate predictor. The later versions that were adjusted based on human expectations used wishy-washy words like "likely" or "unlikely", and its estimation accuracy dropped significantly.[3] At Build 2023, Andrej Karpathy outlined[2] how they trained GPT 4. Essentially, the raw model during training builds its own intelligence. Then there's three stages of "tuning" to make it behave, and all three are based on human input. Essentially, they had contractors provide samples of "ideal" output. Similarly, end-users could up-vote or down-vote responses, which also got fed in. My personal theory is that the raw models can get about as intelligent as the average of the consistent and coheren
Source for LLMs retrospectively judging their own output
Kaparthy explained how LLMs can retrospectively assess their own output and judge if they were wrong. Source: this video
Training process matters beyond compute
Promises are cheap - even if made by researchers and engineers deeply believing that they can beat OpenAI. Let alone any assurances given by PR or marketing departments. And don't fool ourselves - everything announced publicly is prepared or at least approved by such. Computational power alone is not the only resource. It is also the training process itself (see this talk by Andrej Karpathy, this video , at 1 min) and, obviously, data and its quality. I will be convinced only after Google demonstrates that Gemini is better than GPT4 (in some, or all, tasks).
Shows extra training phases that make base models assistants
> They just produce something that looks like a human wrote it real good. That's what the base model does. To get an LLM assistant there are additional training phases to make it conversational, helpful and more correct - this video
Karpathy suggested AlphaGo-style tree search for LLMs
Last May, Karpathy mentioned that the next step in LLM performance would likely take inspiration from AlphaGo's Monte Carlo-style tree traversal to enable deeper and more self-correcting reasoning: this video But if this approach holds up, it suggests that the most valuable part of the AlphaGo project that applies to LLM development is in fact reinforcement learning through self-play. Why not create a leaderboard of reward-generating language models that constantly "play" against each other, where model selection frequency is based on ELO, and update on the results? What if the Open LLM leaderboard ( link ) evolved into a constantly improving pool of such models? This also alleviates data scaling issues by providing a diverse and continuously changing distribution of new training inputs.
