Of all the cognitive tools our ancestors left us, what’s best? Society seems to think pretty highly of arithmetic. It’s one of the first things we learn as children. So I think it’s weird that only a

Step-size α = 0.02 Momentum β = 0.99 We often think of Momentum as a means of dampening oscillations and speeding up the iterations, leading to faster convergence. But it has other interesting behavio

Dec 2021 Comments at hacker news. 1. You’re in love. The two of you want to share the rest of your lives. So, being good game theorists, you have a romantic dinner and plan how to align your interests for mutually beneficial optimal strategic behavior. Your goals are (1) to Odysseus yourself so that…

Updated Feb 2021 Some people claim ethics aren’t practical. Others make a grim philosophical argument: Ethics are an illusion. There’s no “right and wrong” written in the fabric of the universe. It’s

Oct 2020 Cedric and Bertrand want to see a movie. Bertrand wants to see Muscled Duded Blow Stuff Up. Cedric wants to see Quiet Remembrances: Time as Allegory. There’s also Middlebrow Space Fantasy. Th

Part 14 of 12 in the Engineer’s Interpretability Sequence. Is this market really only at 63%? I think you should take the over. Only 63%? I think you should take the over. Five tiers of rigor for safe

Suppose it’s the early twentieth century, and we’re trying to quantify the concept of “information”. Specifically, we want to measure “how much information” one variable contains about another - for i

Oct 2024 There was recently a thread on r/slatestarcodex about “What life hacks are actually life changing”. One of the examples given was: Buy audiobooks to read much more books, listen at 1.5-2x spe

On January 24, I attended a 1-day data science symposium at Harvard University with the fun title ‘Weathering the Data Storm‘. I imagine being in a tiny boat on the endless beautiful sea of data, and

TL;DR: I think new machine learning researchers often make one of two kinds of mistakes: not making enough contact with reality, and being too reluctant to form gears-level models of ML phenomena. Ste

This work was performed as a contractor for SERI MATS, but the views expressed are my own and do not necessarily reflect the views of the organization. I recently conducted interviews with 7 current/f

This morning I got an email about my blog post discussing the history of deep learning which rattled me back into a time of my academic career which I rather not think about. It was a low point which

Here’s a story you may recognize. There's a bright up-and-coming young person - let's call her Alice. Alice has a cool idea. It seems like maybe an important idea, a big idea, an idea which might matt

Say you have a time machine. You can only use it once, to send a single idea back to 2005. If you wanted to speed up the development of AI, what would you send back? Many people suggest attention or t

[Quickly written, unpolished. Also, it's possible that there's some more convincing work on this topic that I'm unaware of – if so, let me know. Also also, it's possible I'm arguing with an imaginary

This is a description of my work on some data science projects, lightly obfuscated and fictionalized to protect the confidentiality of the organizations I handled them for (and also to make it flow be

Being 11 lbs (5 kg) overweight costs 30 minutes of life expectancy daily. Equal to roughly two cigarettes a day.

Suppose you’re designing a bridge. There’s a massive number of variables you can tweak: overall shape, relative positions and connectivity of components, even the dimensions and material of every beam

The more I work on myself, the more I realize that the first step for me is almost always to just notice I’m stuck in a loop. As a teenager, I didn’t make progress towards getting good at anything bec

Eight years ago, I worked as a data scientist at a startup, and we wanted to optimize our sign-up flow. We A/B tested lots of different changes, and occasionally found something which would boost (or