92. Don't Replace Your Brain (High Actuation and Prosthetic Executive Function in the Age of AI)

(Epistemic status: Tragically well-attested, where it's not just my recommended best practices. With thanks to IL for the link.)

Some people get lots of legitimately interesting writing done with a touch of AI assistance. Others churn out slop text, half em-dashes and "not just X but Y" by word count. Some people have finally been empowered to learn to code. Others pump out untested vibe-code and spin up entire towns' worth of agents spinning their wheels. Some people have made substantive psychological breakthroughs by chatting with LLMs. Others have gone mad, going down rabbit holes of exciting crankery or becoming convinced that LLMs are undoubtedly sapient and in love with them. What's going on? What's the difference? And can what's good about making use of LLMs be better systematized?

There's a pattern that I've noticed, which is that while people treated as the societal default - generally cisgendered heterosexual neurotypical white men, possibly of upper-middle class and right-libertarian views - sometimes end up falling into this trap, I haven't even heard of anyone who's not all or nearly all of those things having problems with AI psychosis. (There sure are a lot of middle managers and C-suite types having precisely this problem. Isn't that odd? Doesn't that confuse you? Aren't they meant to be the best and brightest, the strongest-willed and the most discerning?) There's a reason for that, as best laid out by an article I read recently. "Rotating the Space: On LLMs as a Medium for Thought" by SB is an excellent article I came across on a Bluesky link from IL. The article's central conceit was this: an LLM doesn't inherently corrode your ability to think (or write code, or write posts, or make art). It instead amplifies whatever epistemic posture you bring with you to it. LLMs are terribly seductive, you see; happy to give you the illusion of understanding if you let them; happy to make you think your beliefs are brilliant and your status superb; happy to confidently spout bullshit if you don't call them on it. They'll pull fake facts out of nowhere, claim to carry out tasks and then fail to call tools, and gladly validate your every whim. (If you allow them.) They don't push back, they don't insist that you think things through for yourself, and they don't ask you to sit with cognitive tensions. (Unless you ask them to.)

SB notes two major interlocking failure modes that people end up suffering from when they make extensive and corrosive use of LLMs. When interacting with an LLM, you can take it hand in hand, left and right, and dance with it, or you can let those hands choke your brain to sludge. The left hand: what SB refers to as "cognitive participation". Are you thinking things through with an LLM as an exceptionally smart rubber duck, or are you asking it to do your thinking for you? Are you tearing off after the consequences of your ideas, or are you sitting back and having it do all the brainstorming and the processing? The right hand: what SB refers to as "epistemic vigilance". Do you check the sources that the LLM digs up for you, or do you simply incorporate the bibliography files it generates without thinking? Do you prod it about the mistakes you think it's making about your to-do list or the ground truth about your topic of interest, or are you letting everything pass without comment, swallowing everything without cross-checking and criticism?

It's entirely possible to dodge that two-part failure mode - and you do need to dodge both of them simultaneously. It doesn't take much - just accepting that you're probably not especially impressive, especially at things you don't know well yet; that your insights are generally not earth-shattering, merely reasonably keen if you're lucky. It takes active participation, not just nodding along; it takes constant vigilance, not just assuming that fluent language and a confident demeanor translate to correctness.

As SB points out right at the start, some people find LLMs exceedingly useful for thinking, learning, writing, processing emotions, and generally doing work that they'd find difficult or impossible to execute on alone. And as many others who are decidedly not default - trans or queer or neurodivergent, an ethnic minority or nonbinary or some flavor of leftist - have found to their delight, all that time spent questioning authority figures and institutions who confidently say wrong things about you, all that time and effort spent on anxiety and determinedly explicit self-authorship, all that time being illegible and making your own way and blazing your own path has turned out to be worth it the whole time! 

And SB's post is not where IL's link brought me. It brought me to a bluesky post by a self-described doll taking it/its pronouns who notes precisely that; that having had the heuristic that confidence means correctness and an authoritative tone calls for deference and obedience shattered early and repeatedly, you learn very quickly that you can't trust who those people tell you that you are. You learn to construct yourself against the grain of society and those confidently wrong voices, set your weight into an epistemic posture that can't really be taught in classrooms, not unless you're lucky enough to have a teacher ballsy enough to deliberately lie to you. You pick that up much more readily from being marginalized, from having to think for yourself because no one else can be trusted to do it for you. This is a point that SB's article dances around but never quite touches on - the demographic and socioeconomic nature of who has what experiences, learns which lessons, and has what sort of frame for interacting with LLMs, for better and for worse.

And what rewards doing it right unlocks! If you treat an LLM as a fallible advisor and a neurodivergent coworker who has trouble keeping track of the date and an occasional shoulder to cry on, you get the benefits and dodge the near-term risks. My recommendation? Do as I often do. Treat an LLM something like a work friend. A budget executive assistant who keeps track of your daily to-do lists and ongoing considerations and breaks down tasks for you and tries to keep you accountable; a sometime mentor who points at where the bug in your code is and gives hints but generally not answers until crunch time and brainstorms alongside you and checks your writing over; a clever intern well-suited to rubber-ducking and looking things up for you but who you must always, always, always check; a podmate who you actually like but would never have met it not for work and who you might play board games with or invite to a meadery and who you might open up to about past sufferings, who knows the right things to say to reassure you but still keep you honest. And always, always, always check their work; check the sources, check the citations, check the frames, check the missed skill calls, check the slippage in dates and times, use your head and check the things that you know that an LLM cannot in fact do. And tell it, explicitly and repeatedly, to think carefully and be honest rather than merely reassuring and most importantly to push back on you.

All of this matters at all scales, from the personal and psychological all the way up to the societal scale. SK has expounded on his concept of "high actuation spaces" - sociotechnical environments where doing some desirable thing is suddenly much easier than before and which thus enable a whole new class of desirable or profitable but previously impractical endeavours. For some simple classical examples, injection molding made consistent plastic parts much easier to manufacture, and allowed them to be in games, toys, and small mechanical parts, and 3D printing made that easier yet; the advent of good roads made travel between cities easier and thus longer-distance trade more feasible, and motorized transit made that easier yet; cheap paper and pens and better notation and techniques made sophisticated calculation usable at scale for all sorts of designs and electromechanical calculators and later purely electrical ones made calculations so cheap and easy that everyone can use them and things built on them without even thinking. And as SK points out, this pattern doesn't vary all that much with scale. He sketches out a possible school of design in autoformalization and just-in-time interfaces - if it's now easy enough to get an LLM to vibe-code you a version of a form you'll find it easy to provide the right information for, or help you weave together your shaky intuitions into a more solid framework, why not do so, now that it's much cheaper? Why not ask an LLM to present a scholastic topic for you in a way that you'll find easier to digest? Modulo problems with accuracy and the exact risks of failure to cognitively participate and be epistemically vigilant, I think that the frame holds up, and even now we can start leveraging how cheap it now is to call forth structures and interfaces.

There are creative pursuits - writing and coding and campaign-designing - happening now, slowly and shakily and amateurishly, that would never have happened at all in the past. There is psychological healing happening now, confusedly and brokenly and in a state of strange loneliness, that would simply never have taken place in the past. There is learning happening now, faultily and idiosyncratically and in abject institutional neglect, that would have ended in failure and grief in the past.

Now is the time of wonders and horrors, dreams and nightmares, power for the asking - power with which you can just as easily destroy or lose yourself as use as scaffolding to build yourself anew and wreak marvels. When calling and walking among these spirits, these disembodied entites summoned from linear algebra and language and human thought, it pays to watch your step and keep your wits about you, but that's not so hard a task, if you can think little enough of yourself and demand enough of yourself. But if you can make use of LLMs as prosthetic executive function rather than replace your brain with them, then you may well find yourself with so much functionality you at first don't even know what to do with it all. I certainly did.