Full Width [alt+shift+f] Shortcuts [alt+shift+k]
Sign Up [alt+shift+s] Log In [alt+shift+l]

Andrej Karpathy blog

Andrej Karpathy blog
Self-driving as a case study for AGI Sparked by progress in Large Language Models (LLMs), there’s a lot of chatter recently about AGI,...
a year ago
105
a year ago
Sparked by progress in Large Language Models (LLMs), there’s a lot of chatter recently about AGI, its timelines, and what it might look like. Some of it is hopeful and optimistic, but a lot of it is fearful and doomy, to put it mildly. Unfortunately, a lot of it is also very...
Andrej Karpathy blog
A from-scratch tour of Bitcoin in Python .wrap { max-width: 900px; } p { font-family: sans-serif; font-size: 15px; ...
over a year ago
73
over a year ago
.wrap { max-width: 900px; } p { font-family: sans-serif; font-size: 15px; font-weight: 300; overflow-wrap: break-word; /* allow wrapping of very very long strings, like txids */ } .post pre, .post code { background-color: #fafafa; font-size: 13px; /*...
Andrej Karpathy blog
Short Story on AI: A Cognitive Discontinuity. p { text-align: justify; } The idea of writing a collection of short stories has been on my mind...
over a year ago
67
over a year ago
p { text-align: justify; } The idea of writing a collection of short stories has been on my mind for a while. This post is my first ever half-serious attempt at a story, and what better way to kick things off than with a story on AI and what that might look like if you...
Andrej Karpathy blog
What a Deep Neural Network thinks about your #selfie Convolutional Neural Networks are great: they recognize things, places and people in your personal...
over a year ago
61
over a year ago
Convolutional Neural Networks are great: they recognize things, places and people in your personal photos, signs, people and lights in self-driving cars, crops, forests and traffic in aerial imagery, various anomalies in medical images and all kinds of other useful things. But...
Andrej Karpathy blog
Deep Reinforcement Learning: Pong from Pixels --> This is a long overdue blog post on Reinforcement Learning (RL). RL is hot! You may have...
over a year ago
52
over a year ago
--> This is a long overdue blog post on Reinforcement Learning (RL). RL is hot! You may have noticed that computers can now automatically learn to play ATARI games (from raw game pixels!), they are beating world champions at Go, simulated quadrupeds are learning to run and leap,...
Andrej Karpathy blog
Deep Neural Nets: 33 years ago and 33 years from now .post-header h1 { font-size: 35px; } .post pre, .post code { background-color: #fcfcfc; ...
over a year ago
47
over a year ago
.post-header h1 { font-size: 35px; } .post pre, .post code { background-color: #fcfcfc; font-size: 13px; /* make code smaller for this post... */ } The Yann LeCun et al. (1989) paper Backpropagation Applied to Handwritten Zip Code Recognition is I believe of some...
Andrej Karpathy blog
Biohacking Lite Throughout my life I never paid too much attention to health, exercise, diet or nutrition. I knew...
over a year ago
44
over a year ago
Throughout my life I never paid too much attention to health, exercise, diet or nutrition. I knew that you’re supposed to get some exercise and eat vegetables or something, but it stopped at that (“mom said”-) level of abstraction. I also knew that I can probably get away with...
Andrej Karpathy blog
A Recipe for Training Neural Networks Some few weeks ago I posted a tweet on “the most common neural net mistakes”, listing a few common...
over a year ago
44
over a year ago
Some few weeks ago I posted a tweet on “the most common neural net mistakes”, listing a few common gotchas related to training neural nets. The tweet got quite a bit more engagement than I anticipated (including a webinar :)). Clearly, a lot of people have personally encountered...
Andrej Karpathy blog
A Survival Guide to a PhD This guide is patterned after my “Doing well in your courses”, a post I wrote a long time ago on...
over a year ago
44
over a year ago
This guide is patterned after my “Doing well in your courses”, a post I wrote a long time ago on some of the tips/tricks I’ve developed during my undergrad. I’ve received nice comments about that guide, so in the same spirit, now that my PhD has come to an end I wanted to compile...
Andrej Karpathy blog
Short Story on AI: Forward Pass p { text-align: justify; } .post pre, .post code { border: none; background-color:...
over a year ago
44
over a year ago
p { text-align: justify; } .post pre, .post code { border: none; background-color: #eee; } The inspiration for this short story came to me while reading Kevin Lacker’s Giving GPT-3 a Turing Test. It is probably worth it (though not required) to skim this post to get...
Andrej Karpathy blog
(started posting on Medium instead) The current state of this blog (with the last post 2 years ago) makes it look like I’ve disappeared....
over a year ago
41
over a year ago
The current state of this blog (with the last post 2 years ago) makes it look like I’ve disappeared. I’ve certainly become less active on blogs since I’ve joined Tesla, but whenever I do get a chance to post something I have recently been defaulting to doing it on Medium because...