62. On Conceptual Engineering
(Epistemic status: just my opinion/frame, but I claim to know whereof I speak. With thanks to IL, JSW, and PR.)
When people talk about conceptual engineering, what do they mean? And why is conceptual engineering so difficult and yet so important? To triangulate this practice properly, we must surely look at it from many angles - math, philosophy, intuition, and utility, for the best few I can think of.
Before any of that, here's a first attempt to describe what conceptual engineering is: it's the cognitive technology or practice of carefully defining some concept. It has to be either a natural category of objects or of other abstractions, or else a property that objects or abstractions might have; it has to both live in the world and be useful for further efforts regarding the world. You can sometimes do conceptual engineering without really trying to; whoever invented currency engaged in conceptual engineering, as was whoever kicked off the Great Sandwich Debates. The important thing is that a frame or concept was constructed, and that its construction was not just to behold, nor for pure descriptive, nor even for further purely abstract pursuits, but rather to make something happen in the world.
This is why a mathematician's frame of mind is so useful for conceptual engineering - to a large extent, it's what a professional mathematician works at every day. It has been rightly said that the most important part of math research is not flashy theorems but rather the definitions which capture phenomena of interest and from which those theorems flow. Notably, the construction of mathematical definitions is never at random, but always carefully arranged so as to both capture some mathematically interesting phenomenon - like the arithmetic properties of the integers or the nature of locally-flat geometric forms - and permit useful theorems to be proven about the objects of interest. Not for nothing are assemblages of mathematical results frequently termed "machines"!
But pure mathematics often fails to live in the world, and has little to say about the finer details of things like justice, sandwiches, virtue, or chairs. For that, we often need to turn to philosophy, which often concerns itself with the careful understanding of such aspects of the world of largely human-scale concepts. But in turn, philosophy often loses its way, ending up chasing its own tail without producing anything of further use. Though at times philosophy aspires to mathlike rigor, the sharp beautiful clarity of axiom to definition to lemma to theorem, most philosophy falls well short, and it is especially those parts of philosophy that deal in human-scale concepts rather than (say) propositional logic or near-quantitative ethics that are most frequently fuzzy or even obscurantist.
So we have mathematics, which makes the construction of useful definitions about (affectionately) abstract nonsense its stock in trade, and philosophy that tries to live in the world and deal with human-scale concepts, but in so doing often turns up definitions that are too fuzzy or broad or just plain omphaloskeptic to be of any use. What we want is something where we look deeply at aspects of the world around us - not just clean abstractions - and throw together a definition or frame meant to encapsulate something real. Note that this, too, is an explicit frame in itself, the "primordial ooze"-like product of conceptual engineering: the emphasis on the usefulness of a definition is a notable and noncanonical one, but it is one that we choose to inhabit. We don't even especially need an intrinsic philosophical motivation. Our extrinsic motivation suffices: a desire to give True Names to - and thus partially control - pieces of the world. (There's something additional to say here about moving around in conceptual space to look at fragments of the world from a new angle, lining them up to see something new jump out at you like some optical illusion or trick of framing rendered material - think Superliminal or Gorogoa. But that's not for today.)
So why might we care about this "conceptual engineering" thing, then? Speak for yourself, but I find the grand intellectual project of identifying and True Naming parts of the world a deeply appealing one. For another, the whole enterprise is weird and slippery enough that it's badly neglected, and the fact that it requires a special kind of not-quite-arrogance that most people well-suited to the task lack only makes the problem worse. Most people near-completely lack the capacity to treat ideas like crafting material and never think about deliberately shaping them! For one last major reason, conceptual engineering is precisely the kind of thing you have to get your hands dirty doing if there's some particularly wicked problem to tackle. If there's some complex thorny challenge made mostly of ideas to handle and no clear path forward, you need to build a tool made of ideas without the affordance of tight obvious feedback loops, and it often has to work on the first try. Whatever you come up with will simultaneously have to interface with the physical world - some line of code run, some electron moving one way and not another, some flow of resources this way and not that - and also handle being made purely of ideas and thoughts for quite some time.
It's this last reason as applied to agent foundations that's had me in its grasp for the last couple of years. If we want to understand and constrain the behavior of arbitrarily intelligent or powerful agents, we need to be able to understand and constrain the behavior of agents at all. If we want to understand and constrain the behavior of agents at all, then we need to understand and constrain the makeup of a general agent, and as yet there exists no settled way of doing that; we don't really know what an agent is or what an agent is not or what agents are made of or how they taste. (Yet. Growth mindset.) But that's all the more reason to roll up your sleeves and get to work.
When people talk about conceptual engineering, what do they mean? And why is conceptual engineering so difficult and yet so important? To triangulate this practice properly, we must surely look at it from many angles - math, philosophy, intuition, and utility, for the best few I can think of.
Before any of that, here's a first attempt to describe what conceptual engineering is: it's the cognitive technology or practice of carefully defining some concept. It has to be either a natural category of objects or of other abstractions, or else a property that objects or abstractions might have; it has to both live in the world and be useful for further efforts regarding the world. You can sometimes do conceptual engineering without really trying to; whoever invented currency engaged in conceptual engineering, as was whoever kicked off the Great Sandwich Debates. The important thing is that a frame or concept was constructed, and that its construction was not just to behold, nor for pure descriptive, nor even for further purely abstract pursuits, but rather to make something happen in the world.
This is why a mathematician's frame of mind is so useful for conceptual engineering - to a large extent, it's what a professional mathematician works at every day. It has been rightly said that the most important part of math research is not flashy theorems but rather the definitions which capture phenomena of interest and from which those theorems flow. Notably, the construction of mathematical definitions is never at random, but always carefully arranged so as to both capture some mathematically interesting phenomenon - like the arithmetic properties of the integers or the nature of locally-flat geometric forms - and permit useful theorems to be proven about the objects of interest. Not for nothing are assemblages of mathematical results frequently termed "machines"!
But pure mathematics often fails to live in the world, and has little to say about the finer details of things like justice, sandwiches, virtue, or chairs. For that, we often need to turn to philosophy, which often concerns itself with the careful understanding of such aspects of the world of largely human-scale concepts. But in turn, philosophy often loses its way, ending up chasing its own tail without producing anything of further use. Though at times philosophy aspires to mathlike rigor, the sharp beautiful clarity of axiom to definition to lemma to theorem, most philosophy falls well short, and it is especially those parts of philosophy that deal in human-scale concepts rather than (say) propositional logic or near-quantitative ethics that are most frequently fuzzy or even obscurantist.
So we have mathematics, which makes the construction of useful definitions about (affectionately) abstract nonsense its stock in trade, and philosophy that tries to live in the world and deal with human-scale concepts, but in so doing often turns up definitions that are too fuzzy or broad or just plain omphaloskeptic to be of any use. What we want is something where we look deeply at aspects of the world around us - not just clean abstractions - and throw together a definition or frame meant to encapsulate something real. Note that this, too, is an explicit frame in itself, the "primordial ooze"-like product of conceptual engineering: the emphasis on the usefulness of a definition is a notable and noncanonical one, but it is one that we choose to inhabit. We don't even especially need an intrinsic philosophical motivation. Our extrinsic motivation suffices: a desire to give True Names to - and thus partially control - pieces of the world. (There's something additional to say here about moving around in conceptual space to look at fragments of the world from a new angle, lining them up to see something new jump out at you like some optical illusion or trick of framing rendered material - think Superliminal or Gorogoa. But that's not for today.)
So why might we care about this "conceptual engineering" thing, then? Speak for yourself, but I find the grand intellectual project of identifying and True Naming parts of the world a deeply appealing one. For another, the whole enterprise is weird and slippery enough that it's badly neglected, and the fact that it requires a special kind of not-quite-arrogance that most people well-suited to the task lack only makes the problem worse. Most people near-completely lack the capacity to treat ideas like crafting material and never think about deliberately shaping them! For one last major reason, conceptual engineering is precisely the kind of thing you have to get your hands dirty doing if there's some particularly wicked problem to tackle. If there's some complex thorny challenge made mostly of ideas to handle and no clear path forward, you need to build a tool made of ideas without the affordance of tight obvious feedback loops, and it often has to work on the first try. Whatever you come up with will simultaneously have to interface with the physical world - some line of code run, some electron moving one way and not another, some flow of resources this way and not that - and also handle being made purely of ideas and thoughts for quite some time.
It's this last reason as applied to agent foundations that's had me in its grasp for the last couple of years. If we want to understand and constrain the behavior of arbitrarily intelligent or powerful agents, we need to be able to understand and constrain the behavior of agents at all. If we want to understand and constrain the behavior of agents at all, then we need to understand and constrain the makeup of a general agent, and as yet there exists no settled way of doing that; we don't really know what an agent is or what an agent is not or what agents are made of or how they taste. (Yet. Growth mindset.) But that's all the more reason to roll up your sleeves and get to work.
Comments
Post a Comment