ALIEN$

Mar 15, 2023

One of the most insightful AI/NLP researchers, Emily Bender, wrote this Twitter thread a few weeks back. One tweet in particular felt important.

the truth right here.

Bender is right, of course. ChatGPT, Stable Diffusion, generative AI systems, none of those are "minds". But in Playing Software I wrote this:

Thanks for reading Playing Software! Subscribe for free to receive new posts.

Playing software is establishing relations with the alien agency of software. That process is cybernetic; it implies engaging with systems that create a world that is ordered. In that world, agency is delimited and framed by clear rules voluntarily accepted. The purpose and the meaning of the activities in that world are a consequence of relating to those rules either constructively or destructively. Play creates order; playing software creates cybernetic order. (p. 103)

And that is not the only place I refer to software as an 'alien agency'. How can I both write that and agree to an extent with Bender's critique of that terminology? It is time to write about software agency, as it is central topic in Playing Software. Let me explain what I mean by software agency, one premise at a time.

Software agency does not mean that software has a mind, intentionality or rationality as biological agents do. Agency is not equal to intelligence - lots of things have agency without intelligence, from a speed bump to a thermostat1. I am using here agency in a narrow sense, understood as the capacity to act in the world. Or, more specifically, the capacity to act in a world.
Software agency is fundamentally different from other forms of technological agency. The search engine is not the steam engine. Non-digital technologies act in the world, but they do not contain a model of the world. They do not operate on representations of the world2.
Software agency is rigid and inflexible. It demands for everything to become part of its world, to conform to its representations, so it can act. Otherwise, it will throw errors. When we become entangled with software, we are travelling to its world. It's not anymore our biological, analog world, but a world in which we also have to take into consideration the world model the software acts upon. If you've ever tried to engineer a picture to become viral, or if you've modified your behavior so that a sensor could perceive you, you know what I am talking about. Except of course we are always all the time living in a software world3.
Software agency is unlike human agency (it is alien), because of the way it operates. It requires representations to be computable to act, imposing a form of order and structure to the world. If it cannot be computed, it does not exist. This rigid, formal, structured manifestation of agency is strange to the way humans create and live in worlds. Contemporary AI systems, like ChatGPT, are even more obscure because their logic, the world they act on, is constructured on a combination of massive amounts of data and advanced mathematical operations that are difficult to grasp for many. That distance between the world the system contains and the world we humans are used to, is what makes us relate to these technologies as alien agents.

So of course Bender is right: there is no intelligence in ChatGPT. We are all basically living in the chinese room scenario.

Why is it worth then thinking about software as an alien agency? Let me see how I can explain this using DALL•E as an example. At the beginning, there is the prompt. An empty space of possiblities. We are primed to write in a particular way, but there is a suggestion there that we may want to be surprised, that maybe this is a game of chance.

We write a prompt, with a goal in mind. The machine computers, we wait, while we see more instructions on how to better ... play this game? Then we see the result, and it is both something we can recognize ("An impressionist oil painting of sunflowers in a purple vase4", and something else, a manifestation of a logic we can't quite get our heads around. It is a showcase of agency, as well as a glimpse into a world that is not like the world we know.

AI excels at bland aesthetics

We can understand where the image came from - after all, the "operational understanding" the computer has of "impressionism", "vase", "purple", or "sunflowers" is an outcome of a human process of representation. But the way these representations combined to create something that is familiar,yet novel, that is something beyond our immediate comprehension. And we need to make sense of it.

That is what I mean by alien agency: the process by which a prompt creates an image of a vase is an act in our shared world, based on a model of a world.

And most importantly, deploying these agents means changing our shared world as we know it. Because now in the world there is plausible text and plausible images generated by these agents, from their worlds. And these agents will also expect us to act and behave in certain ways, to feed them data and structure it in such a way that they can keep marvelling us. These alien agents are not only reaching out, but also somehow terraforming the world so they can be a part of it. Images become prompts, texts became palimpsests, and these aliens thrive among us.

Their world, this world of representations built on massive databases and statistical methods, becomes also our shared world, in which software can act for us, and with us. Living with software is travelling to these alien worlds of data and computations, becoming part of those worlds, or letting our shared world become also their world. And we often do that by playing. Therefore, we cannot understand computational agency and its effects in our culture without understanding what happens when playing software.

-> Next on this space: the meaning and importance of world-traveling <-

Subscribe now

I know, I am showing off my credentials as philosopher of technology. The speed bump as a shibboleth.

This would be contested by some, probably with some solid arguments. But I think my exceptionalist argument is solid: there is an intuitive difference between a chatbot and a voice answering machine, and I frame that difference as one of representation. This has obvious repercusions when it comes to thinking about knowledge and its fields (is software engineering engineering? Or is it closer to a liberal art, like literature or music? - you know the answer, admit it.)

The Metaverse has always been here! A little bit like in the age of Empires and discoveries when new continents were "found", we're kind of not discovering anything but more realising the existence of something that has been there for a while.

That may be the most complex "Hello World" I have seen in a long time.

Playing Software