This is a Draft Amnesty Week draft. Last year, someone who was considering projects related to diversity and inclusion in EA noted that one challenge was not knowing what had been tried before. I drafted this summary but never got it out the door. The atmosphere around DEI interventions, in the US…
bilal
Twilight of Man by Rockwell Kent Share "The decline of the free entrepreneur and the rise of the dependent employee on the American scene has paralleled the decline of the independent individual and the rise of the little man in the American mind." - C. Wright Mills I know the hiring cycle is well…
Welcome to Transformer, your weekly briefing of what matters in AI. If you’ve been forwarded this email, click here to subscribe and receive future editions. The UK government announced today that the AI Safety Institute is now the AI Security Institute. From the press release: “This new name will…
Until this decade, artificial general intelligence was a scientific problem. The main ideas to build it were missing. In 1999, Shane Legg (cofounder of Google DeepMind), predicted we’d build AGI in 2028 based on extrapolations of compute power trends. His prescience on reinforcement learning is…
Is progress inevitable? Is it natural? Is it fragile? Is it possible? Is it a problematic concept in the first place? Many people are reexamining these kinds of questions as 2016 draws to a close, so I thought this would be a good moment to share the sort-of “zoomed out” discussions the subject that…
Page 2 of 16 The Secrets of ALCHEMY LAWRENCE M. PRINCIPE The University of Chicago Press Chicago and London Page 3 of 16 FOUR REDEFINITIONS, REVIVALS, AND REINTERPRETATIONS Alchemy from the Eighteenth Century to the Present If I were to follow a strictly chronological sequence, this chapter would…
New Atlantis (abridged) Source text and introductory note taken from: https://www.gutenberg.org/cache/epub/2434/pg2434-images.html Project Gutenberg license: This eBook is for the use of anyone anywhere in the United States and most other parts of the world at no cost and with almost no restrictions…
The Copernican revolution was a pivotal event in the history of science. Yet I believe that the lessons most often taught from from this period are largely historically inaccurate and that the most important lessons are basically not taught at all [1]. As it turns out, the history of the Copernican…
TL;DR: The Google DeepMind AGI Safety team is hiring for Applied Interpretability research scientists and engineers. Applied Interpretability is a new subteam we are forming to focus on directly using model internals-based techniques to make models safer in production. Achieving this goal will…
Dark Mode Toggle 2021 Mar 23 See all posts The Most Important Scarce Resource is Legitimacy Special thanks to Karl Floersch, Aya Miyaguchi and Mr Silly for ideas, feedback and review. The Bitcoin and Ethereum blockchain ecosystems both spend far more on network security - the goal of proof of work…
“Yes, the pyramids have been built, but if you give me 300,000 disciplined men and give me 30 years, I could build a bigger one.” — Werner Herzog OpenAI’s goal is to build AGI (and then go further), which they define as “a highly autonomous system that outperforms humans at most economically…
I’m not a natural “doomsayer.” But unfortunately, part of my job as an AI safety researcher is to think about the more troubling scenarios. I’m like a mechanic scrambling last-minute checks before Apollo 13 takes off. If you ask for my take on the situation, I won’t comment on the quality of the…
July 2020 One of the most revealing ways to classify people is by the degree and aggressiveness of their conformism. Imagine a Cartesian coordinate system whose horizontal axis runs from conventional-minded on the left to independent-minded on the right, and whose vertical axis runs from passive at…
People sometimes tell me that they want to join a startup, so that they can learn how it works, and eventually start one themselves. I usually end up suggesting that they skip straight to step 2 and start one themselves. Why is that? Isn’t it better to learn from someone else’s mistakes than to have…
I used to be an anti-wire crusader. I hated the clutter of cables, and my tendency to unconsciously chew on them if they got anywhere near my face. But running into bug after tricky wireless bug—mostly while trying to make my video calls work better—I’ve apostasized. The more I’ve learned about…
I suspect I’m not the only one who’s felt this trapping effect in physics. Some theorists seem to work primarily on fad topics inherited from other prominent departments (ever heard of dynamical quantum phase transitions?). That’s not to say these research areas aren’t valuable, beautiful, or…
Until this decade, artificial general intelligence was a scientific problem. The main ideas to build it were missing. In 1999, Shane Legg (cofounder of Google DeepMind), predicted we’d build AGI in 2028 based on extrapolations of compute power trends. His prescience on reinforcement learning is…
Aboriginals believe in … [a] “dreamtime”, more real than reality itself. Whatever happens in the dreamtime establishes the values, symbols, and laws of Aboriginal society. … [It] is also often used to refer to an individual’s or group’s set of beliefs or spirituality. … It is a complex network of…
I have edited and expanded this in a newer post, you should read that instead: School is Not Enough The original is below ~ ~ ~ The world is a very malleable place. When I read biographies, early lives leap out the most. Leonardo da Vinci was a studio apprentice to Verrocchio at 14. Walt Disney took…
(Many of these ideas developed in conversation with Ryan Greenblatt) In a shortform, I described some different levels of resources and buy-in for misalignment risk mitigations that might be present in AI labs: *The “safety case” regime.* Sometimes people talk about wanting to have approaches to…
This is a personal post and does not necessarily reflect the opinion of other members of Apollo Research. Many other people have talked about similar ideas, and I claim neither novelty nor credit. Note that this reflects my median scenario for catastrophe, not my median scenario overall. I think…
A friend of mine recently recommended that I read through articles from the journal International Security, in order to learn more about international relations, national security, and political science. I've really enjoyed it so far, and I think it's helped me have a clearer picture of how IR…
Buck Shlegeris and I recently published a paper with UK AISI that sketches a safety case for “AI control” – measures that improve safety despite intentional subversion from AI systems. I would summarize this work as “turning crayon drawings of safety cases into blueprints.” It’s a long and technical…
What does “algorithmic ranking” bring to mind for you? Personally, I get visions of political ragebait and supplement hucksters and unnecessary cleavage. I see cratering attention spans and groups of friends on the subway all blankly swiping at glowing rectangles. I see overconfident charlatans and…
I’ve observed thousands of founders and thought a lot about what it takes to make a huge amount of money or to create something important. Usually, people start off wanting the former and end up wanting the latter. Here are 13 thoughts about how to achieve such outlier success. Everything here is…
January 2016 Life is short, as everyone knows. When I was a kid I used to wonder about this. Is life actually short, or are we really complaining about its finiteness? Would we be just as likely to feel life was short if we lived 10 times as long? Since there didn't seem any way to answer this…
Over the past decade, Elon Musk has been one of the most prominent advocates for the importance of AI safety. Musk, who has said that there’s a 10-20% chance that advanced AI systems will lead to human extinction, has consistently urged society to take the risks of advanced AI more seriously,…
Welcome to Transformer, your weekly briefing of what matters in AI. If you’ve been forwarded this email, click here to subscribe and receive future editions. Housekeeping note: Transformer is taking next week off. We’ll be back on Feb 7 with a special edition from Paris, ahead of the AI Summit. (And…
Image: Hanna Barakat + AIxDESIGN & Archival Images of AI / Better Images of AI For the past couple of years, AI companies have relied on a simple argument to justify releasing their models: if an AI can’t do dangerous things, it must be safe to release. But that logic might be about to expire,…
Earlier this month, I used Claude to port (parts of) an Emacs package into Rust, shrinking the execution time by a factor of 1000 or more (in one concrete case: from 90s to about 15ms). This is a variety of yak-shave that I do somewhat routinely, both professionally and in service of my personal…
Introduction Several developments over the past few months should cause you to re-evaluate what you are doing. These include: Updates toward short timelinesThe Trump presidencyThe o1 (inference-time compute scaling) paradigmDeepseekStargate/AI datacenter spendingIncreased internal deploymentAbsence…
Recently, something shifted in the AI industry. Researchers began speaking urgently about the arrival of supersmart AI systems, a flood of intelligence. Not in some distant future, but imminently. They often refer to AGI - Artificial General Intelligence - defined, albeit imprecisely, as machines…
A friend doing a job search recently asked me: A choice I will likely have is whether to work at a larger company… or a startup… I was wondering if you had any particular feelings on this question. In the long run I’m hoping to work at a non-startup due to hours/general quality of life as well as…
This is a snapshot of a new page on the AI Impacts Wiki. We’ve made a list of arguments that AI poses an existential risk to humanity. We’d love to hear how you feel about them in the comments and polls. Competent non-aligned agents Humans increasingly lose games to the best AI systems. If AI…
Welcome to Transformer, your weekly briefing of what matters in AI. If you’ve been forwarded this email, click here to subscribe and receive future editions. In his final week in office, Joe Biden made three big AI policy moves. On Monday, we got new export controls, which we covered last week. The…
Epistemic status: deliberately provocative title. Caveats: “in the relevant age group,” “according to back of the envelope math,” “with some assumptions about severity definitions,” “if the correlation is causal”… For a long time, I have been totally mystified by the amount of human capital that is…
When I started dating my partner, I quickly noticed that grad school was making her very sad. This was shortly after I’d started leading an engineering team at Wave, and so the “obvious” hypothesis to me was that the management (okay, “management”) one gets in graduate school is totally ineffective.…
Some tools I often use when working on ML or software projects outside of work (excluding LLMs / AI coding assistants). I am mostly optimizing for reliability, ease of use, and availability of features I care about. It’s possible that given the kinds of things you build you have different needs.…
We report some developing work on the Anthropic interpretability team, which might be of interest to researchers working actively in this space. We'd ask you to treat these results like those of a colleague sharing some thoughts or preliminary experiments for a few minutes at a lab meeting, rather…
Today robots barely have the dexterity of a toddler, but are rapidly improving. If their algorithms and hardware advance enough to handle many physical human jobs, how quickly could they become a major part of the workforce? Here's some rough estimates showing it could happen pretty fast. Robot cost…
No articles.