It’s 2035, and open source developers are training an AI system expected to be capable of automating cyber vulnerability detection, biological design, and more. If its safeguards fail, or are purposef

Tl;dr: My model for science is that it is like mining outwards from a point. This offers predictions that extend beyond Scott Alexander’s foraging metaphor and sometimes disagree. The mining metaphor

It can be hard to see the gradual improvement of most goods over time, but I think one way to get a handle on them is to look at their downstream effects: all the small ordinary everyday things which

Aug 2024 If you ask the internet if breastfeeding is good, you will soon learn that YOU MUST BREASTFEED because BREAST MILK = OPTIMAL FOOD FOR BABY. But if you look for evidence, you’ll discover two d

Why form principles for the AGI Race? I worked at OpenAI for 3 years, on the Alignment and Superalignment teams. Our goal was to prepare for the possibility that OpenAI succeeded in its stated mission

LESSWRONG LW Why Large Bureaucratic Organizations? 27th Aug 2024 15 min read 68 New Comment 52 comments, sorted by Click to highlight new comments since: Unnamed11mo8434 In America, people shopped at Walmart instead of local mom & pop stores because it had lower prices and more selection, so Walmart…

How can we figure out if AIs are capable enough to pose a threat to humans? When should we make a big effort to mitigate risks of catastrophic AI misbehaviour? In this episode, I chat with Beth Barnes

A lot of work to prevent AI existential risk takes the form of ensuring that AIs don't want to cause harm or take over the world---or in other words, ensuring that they're aligned. In this episode, I

Nathan Helm-Burger Aug 28, 2024 5911 1. Are the historically leading AI organizations—OpenAI, Anthropic, and Google—holding back their best models, making it appear as though there’s more parity in th

When I know that I’m going to be moving out from an apartment soon, I commission a digital artist to draw it for me. Then I print it out and I have a cool art piece. If you love your current place but

Akash3mo8331 The main reason I’d be more freaked out is that I already think egregious misalignment is fairly plausible; if I thought it was very unlikely, I wouldn’t change my mind based on one weird

Updated Mar 2021 Here’s a “low-brow” take on ethics that’s worth taking seriously: Ethics isn’t going to save the world. We don’t need more “calculations” about the right thing to do. We need people t

Have a consistent ruleset. Titles are a black art. Have empathy for the brain’s parser. It’s impossible to follow all the conventions. Write a thesis statement even if you hate it Stephen King (On Wri

The Myers-Briggs Personality Indicator (MBTI) gets a lot of scorn: About 2 million people take it annually, at the behest of corporate HR departments, colleges, and even government agencies. The compa

Economists say free trade is good because of “comparative advantage”. But what is comparative advantage? Why is it good? This is sometimes considered an arcane part of economics. (Wikipedia defines it

This is a cross-post from my personal blog. What is this post about? The question of how to manage your personal finance as an Effective Altruist has been with me since I joined the movement. At some

I think that the concept of "alignment research" (and the distinction between that and "capabilities research") is currently a fairly confused one. In this post I’ll describe some of the problems with

Is there a general factor of intelligence? This question is a trap. If you try to answer it, you’ll find yourself beset by semantic questions. What’s intelligence? What’s a factor? And if you get past

For many products, we face a choice of who to hold liable for harms that would not have occurred if not for the existence of the product. For instance, if a person uses a gun in a school shooting that

Oct 2020 Watching people discuss police bias statistics, I despair. Some claim simple calculations prove police bias, some claim the opposite. Who is right? No one. Frankly, nobody has any clue what t

Updated Mar 2021 We want to know if things are fair. Do some groups of people tend to get a raw deal in company hiring or university admissions or court sentences? There seems to be an obvious way to

It turns out that sales tax has a huge, gigantic, terrible flaw: It punishes specialized businesses. A value added tax (VAT) has no such problems. The US has sales tax. Most of the planet has VAT. May

As a dad your job is to make sure your son is the oldest in his class. Huge advantage in sports and it’ll bleed over to academics, relationships, everything. Boys mature extremely slowly. Give them ev