47. A Short Meditation on Upton's Law and Selection Effects

Upton's Law: "it is very hard to get a [person] to understand something when [their] livelihood depends very much on their not understanding it." We see this all around us - short-sighted carbonaceous fuel moguls, anyone in your least favorite police force or army, patent trolling and other rent-seeking, and (of course) just about anyone working in ad tech or at a scaling lab. It's not that they don't care about the negative externalities they generate - they have simply put themselves in a frame of mind where those bad secondary effects touch but lightly on the conscience. Why, though? How can people do that to themselves? A secret: not everyone can, for any given such source of harm. We must look, as always, to selection effects. (Twice.)

On the one hand, consider what happens if the harms that someone generates in earning their living weigh on them heavily. One way or another, they'll be gone and replaced with someone who'll likely be better-suited to install a blind spot properly. It might be because they quit, or because their sleepless nights saw them put on a PIP. It might be their family and friends hounding them, or because after the last round of firings, they had a change of heart. In any event, they don't generally stick around for the long haul, and if circumstances force them to, they won't be especially open about their work.

On the other hand, such companies often find themselves paying hefty premiums to attract labor due to that exact churn. Thus, here we find the proverbial incentives in "But the incentives, man!". Enough free-flowing resources sadly compensate for many a sin, and sleepless nights are fewer with a nicer mattress, some fine tea, and the conspicuous lack of student debt. Plenty of people might well deceive themselves, and whoever's best suited to the noxious work at hand will be the one to take the job - but a major determiner of suitability is, of course, personal fit to the work... which in this case, partly means either not caring about the externalities or being very good at self-deception.

Selection effects rule the day, as ever - the kind of thing you see in the wild has to have got there somehow, and also stayed there somehow, mostly on its own. I leave as an exercise to the reader the full story of how this maps onto the plight of any given researcher at a scaling lab, especially the kind of researcher who joined up with the expressed hopes of changing it from within.

Comments

Post a Comment

Popular posts from this blog

4. Seven-ish Words from My Thought-Language

Seven entries from a dictionary that will never exist, even though it should. The words in [brackets] show up with some frequency in my thoughts, and I struggle with English’s semantic poverty. Some of them show up in other languages, like Korean or Lojban. Others among them, people I’ve spoken to share in wishing were wider-used words. [untranslatable 1: word-clogged/interference-silent] (adj): Of a person or their state of mind. Descriptive of the phenomenon where a person has multiple different things to say about a specific topic, or multiple different responses or reactions to a remark an interlocutor has made, only one of which can be taken. To be tongue-tied not out of a lack of things to say, but out of an overabundance of mutually exclusive things to say, all of equal priority, especially when each thing to say would result in following importantly different conversational paths. Often used instead in its natural verb form, which means “to be silent as a result of [word-clog]”...
Read more

20. A Sketch of Helpfulness Theory With Equivocal Principals

You may rightly wonder what else I’m up to, if it’s not just blogging and the occasional adventure. The short answer is that I spend most of my time and mental energy taking part in a summer research fellowship in AI safety called PIBBSS, where my research direction has to do with questions like “how does an agent help a principal, even if the agent doesn’t know what the principal wants?” and “how much harder is it for the principal to teach the agent what they want, and how much worse are the outcomes, if the principal doesn’t know for sure what they want?”. The long(ish) answer is this post. I’ll avoid jargon, mathematical notation, and recommendations to read other long papers, but some amount of that will be unavoidable. Here’s the basic setup - called a “Markov decision process”, sometimes with adjectives like “decentralized” or “partially observable” - which forms the ground assumption for the entire subfield my research direction lies in, called “inverse reinforcement learning”:...
Read more

11. Why the First “High Dimension” is Six or Maybe Five

Image
(Epistemic status: morally correct, in the mathematician’s sense; here to give flavor and intuition without too much rigor.) People toss around the concept of “high-dimensional space” a lot without having a good idea of where that starts. Interpretability researchers staring at vectors in hundreds of dimensions are clearly contemplating a high-dimensional space - loss landscapes packed full of saddle points and spheres where any given pair of vectors is almost certainly nearly orthogonal - but do they know what it means to be a high-dimensional space? Surely we can do better for ourselves than Justice Stewart’s infamous maxim - “I know it when I see it”! What do we care about, when we want to tell how many orthogonal axes we need in a space to start calling it high-dimensional? I claim that we should care about looking at unit-radius disks - like, literal filled-in two-dimensional circles - and in particular, checking when, whether, and how pairs of those unit-radius disks overlap when...
Read more