AI will not develop self-awareness, but it is becoming the unconscious of society.

Author: L. M. Sacasas

Translated by: Deep Tide TechFlow

Deep Tide Reading: As we outsource more and more decision-making to AI, human society is generating an “unconscious layer” that sits outside active judgment. This isn’t a science-fiction threat from superintelligence—it’s a more subtle civilizational transition: we can do more and more, but we understand less and less. This is a warning bell for anyone who relies on AI tools in their work.

In the early 20th century, the British mathematician and philosopher Alfred North Whitehead said a quote that’s been widely repeated: “We should cultivate the habit of thinking about what we are doing—that’s an old chestnut, the biggest mistake of all, repeated by all textbooks and celebrity speeches.” “On the contrary,” he argued, “the progress of civilization consists in the increase in the number of important operations which we can perform without thinking. “ He added a comparison: “Thinking is like a cavalry charge in battle—numbers are strictly limited, it requires fresh horses, and it can only be launched at decisive moments.”

You’ve probably seen this citation. It does convey some truth. But in my view, it always seems shortsighted or insufficiently developed. Perhaps because I often see people use this line to defend wholesale outsourcing of human cognitive activity to machines without distinguishing what’s involved, never properly tallying the costs—or even realizing that there are costs.

To be fair, Whitehead’s quote comes from his 1911 book An Introduction to Mathematics. What he was actually talking about was the advantage of symbols and notations in mathematical calculation—although he called these operations “without thinking,” they still come from minds that have learned and apply them through thinking. But when he talks about “civilization,” plus that grand rhetorical flourish at the end, it probably really does invite this kind of (mis)reading.

The problem isn’t whether certain forms of mental automation—or even externalization of certain mind processes—can be useful in particular situations. To be frank, I think it’s a more persuasive argument to analogize using physical activity rather than mental activity. I’ve long since drifted away from the kind of motor skills I might once have had, but the lesson remains. Almost every young beginner athlete discovers with frustration: you have to spend huge amounts of seemingly excessive and painful time repeating fundamental training again and again. Waxing, polishing—anyone who’s seen Kung Fu Panda understands. But what actually happens is exactly the dynamic Whitehead described. As you automate certain bodily actions, you can then perform them without hesitation. Only then can you bring out any creativity or extraordinary technique. Learning to dance or to play an instrument is the same: automating basic movements is an indispensable foundation for attaining mastery.

But does this same dynamic apply to all areas of human activity? Is there a scenario where outsourcing some activities actually weakens—not strengthens—the higher good you’re pursuing? Or do “lower-level” activities come with certain goods we don’t want to lose? And even, as in mathematics, can we always easily separate a particular subroutine from a given process or activity? Are there holistic activities that are inseparable—where once you try to outsource or automate any element within them, they fall apart? Moreover, as I mentioned earlier, is there really a difference between internally mastering cognitive automation and fully externalizing it, or doing a wholesale outsourcing of cognitive labor?

Returning to the bodily-activity analogy: if a competitive athlete were to imagine using machines to do all his “trivial” training so he could go straight to the精彩 parts of the competition—this person would never get there. Before we easily accept the efficiency-and-freedom promises made in the name of new technology, these questions are crucial.

Still, I’m irritated by Whitehead’s much-cited hymn to cognitive automation, perhaps because it stands in sharp contrast to Hannah Arendt’s warning: we should “think about what we are doing.” Whitehead wrote nearly half a century earlier than Arendt, and I don’t think Arendt is alluding to Whitehead. I just think these two lines happen to form an adversarial resonance, and that investigating this tension is useful.

“We should cultivate the habit of thinking about what we are doing—an old chestnut, a big mistake.” — Whitehead

“So my proposal is very simple: it is nothing more than thinking about what we are doing.” — Hannah Arendt

Interestingly, Arendt’s warning appears in a context closer to our current anxieties about technology and the human condition. This line comes from the preface to The Human Condition, first published in 1958. She explains:

“What I intend to do is to reconsider the human condition starting from our latest experiences and our most recent fears. This is obviously a matter of thinking—and without thinking: reckless frivolity, desperate confusion, or a complacent repetition of ‘truths’ that have become trivial and empty—seems to me to be one of the most distinctive characteristics of our time. Therefore my proposal is very simple: it is nothing more than thinking about what we are doing.”

The “latest experiences” that spurred Arendt’s research include the launch of artificial satellites and the advent of automation. Although until recently people may have considered her concerns about automation’s impact on labor to be a misjudgment, it now seems they may simply have come too early.

When I consider the divergence between Whitehead and Arendt on whether “we need more thinking or less thinking,” I find it helpful to distinguish between the following two things: on the one hand, building a set of unconscious routine procedures to develop higher capabilities; on the other hand, being subject to unconscious forces that unstablely drive your behavior and may cause damage.

Is there a tipping point in Whitehead’s principle that flips or reverses it? “The progress of civilization,” Whitehead says, “consists in expanding the number of important operations we can perform without thinking.” But what if there is such a tipping point? If we go beyond a certain threshold—whether by quantity or quality—does civilization degrade because it expands the number of important operations humans can perform without thinking? Or at least, do we obtain a kind of civilization entirely different from what we know? We may learn the answer soon, because AI adoption often means the automation of our thinking—and the creation of a huge unconscious domain of action in the world, driven by the externalized storage of our personal and collective memories.

Arendt worries that as our technological capacities (explained largely in mathematical terms) outstrip our ability to understand them in ordinary language, we may lose the capacity to “think and talk about the things we can in fact do.” (This shows that the interpretability problem predates the arrival of large language models and so-called black-box algorithms.) In a passage that is now relevant and urgent again, she continues: “It is as if our brains—constituting the physical, material conditions of our thoughts—cannot keep up with what we do, so that from now on we indeed need artificial machines to think and speak for us.” This is an especially important political issue. Under such conditions, we would no longer be self-governing in any meaningful sense. Politics, as a realm of human action constituted by speech, would cease to exist.

At Christmas in 1958—that is, the same year Arendt published The Human Condition—W. H. Auden published The Children’s Crusade (or Friday’s Child), containing such thoroughly Arendtian lines:

When we meet, in our observing, the observing heart of self-observation

neither distressing nor coldly cruel

but entirely commonplace

though the tools it wields

can make wishes and counter-wishes true

it obviously cannot understand

what it can so clearly do

Auden and Arendt were friends, and Auden even wrote a favorable review of The Human Condition, but I’m not sure in which direction the influence flowed.

Although Arendt straightforwardly explains how we arrive at “the state of being unable to understand clearly what we can clearly do”—that is, when actions empowered by technology cannot be fully understood by our everyday language—there is another interesting framework.

Actions that language cannot penetrate, and therefore conscious thinking cannot penetrate, can also be explained through the unconscious. I’m not the kind of person who instinctively appeals to unconscious language, but it may be a useful analogy that helps us understand the relationship between us—both individuals and collectives—and AI devices that increasingly mediate our experience of the world.

In short, I can think of two ways to analogize certain instances of consumer AI as unconscious. The first is relatively direct: as we outsource more and more tasks to intelligent-agent AI at the level of individuals, organizations, and institutions, we generate in the world a layer of activity that is functionally outside human active judgment and supervision. And as long as this layer is structured and shapes our experience, the ratio of consciously guided human action to unconsciously guided human action shrinks.

Of course, at least since the industrial age (if not earlier), there have been relatively opaque systems operating in the human lifeworld. But I think the difference is that those systems were relatively isolated from everyday human activity. AI systems, however, are increasingly woven into our experience. In many cases these are not just external forces imposed on us; they are also processes we unleash and activate—we act through them, and they react back on us. It is precisely this close interweaving of humans and machines that makes the analogy to the unconscious come to mind.

Erik Hoel recently made a similar argument in more elegant terms and at greater length. His conclusion is:

“The traditional danger of AI is usually thought of as a threat to the survival or demise of superintelligence. However, this may overlook a truly real—and subtler—danger: the AI revolution is a mechanism that shifts the process of our civilization from conscious oversight to unconscious processes. And when AI removes consciousness from the workings of the world, it makes the world increasingly less explainable, increasingly unfamiliar and difficult to understand. Thus far, ‘the sedimentation of shared space’ has already demonstrated that this is the main risk of the large language model revolution. And as AI systems become more intelligent, especially if they remain (or likely remain) unconscious, a greater risk is the decline in the importance of consciousness in culture.”

This line of thought parallels a view I’ve argued intermittently for a decade: digital technology is re-enchanting the world in another way—making it mysterious and hard to grasp.

We—the people confronting those technological objects that have been given power over us—must deal with them in a way that cannot be ignored. We seek help from technology, pinning our hopes on the power it seems to hold. We’re also afraid of technology, viewing it as a source of trouble. The technological forces we encounter are sometimes benevolent, but just as often malicious forces that sabotage our efforts and throw our plans off course.

Technological objects don’t merely empower our potential; sometimes they even fill us with wonder. We also treat these objects and forces as key factors that determine our fortune and misfortune: they operate on us outside our control and in ways we don’t understand. In other words, we are fragile—our autonomy is weakened by the circuits of distributed-agent technology that intersect with our will and desires.

The second way to use the unconscious as an analogy for AI may be a bit more esoteric, so let me explain. In Understanding Media, Marshall McLuhan claims that “with the coming of electrical technology, man extends, or rather puts the living model of his central nervous system outside himself.” He returns to the idea of the power medium as an externalization of our nervous system many times. For example:

“A major feature of the electrical age is that it establishes a global network, which has to a large extent the characteristics of our central nervous system. Our central nervous system is not merely an electrical network; it constitutes a single unified field of experience.”

McLuhan wasn’t the first to propose this idea. As early as 1950, a Catholic priest and paleo-biologist, Pierre Teilhard de Chardin, claimed that media technologies constitute “the carefully constructed creation of a truly human nervous system” and “a well-crafted construction of a shared consciousness.” Through emerging networks of computation and communication technologies, a new layer of reality is appearing: it wraps around the biological layer. This is a unified layer of human consciousness mediated by technology, which he called the “noosphere,” and described it as “an astonishing thinking machine.”

(I admit this isn’t my usual way of putting things, but let’s play with these concepts for a moment. In fact, I’ve been trying to persuade you that looking at AI through a psychoanalytic lens may be a useful perspective.)

Electrical media extend our ability to sense the world, bringing distant events before our eyes and into our ears, compressing the time it takes to transmit information to nearly the instantaneous. But since the time of Teilhard de Chardin and McLuhan, something else has also emerged: a massive growth in artificial memory or data storage capacity. So while electrical media extend our nervous systems, recently they have also enabled the collection and storage of unprecedented volumes of information in forms such as text, images, and video. Our nervous systems are supplemented by a vast scale of digital memory. It is precisely this digital memory that feeds large language models—so that large language models now nearly stand as a synonym for artificial intelligence.

This enormous reservoir of human knowledge and culture is essentially the internet—or at least the way the internet makes it accessible to us. For this reason, digital media theorist Gregory Ulmer, in the early 2000s, called the internet “the prosthesis of our collective unconscious.”

I think Ulmer means something like the following. We have always been social animals—inter-subjective cultural beings. But most of what shapes us, especially in childhood, usually fades with time from our conscious awareness. However, the internet as the prosthesis of that unconscious dimension in the process of our becoming social beings makes our collective cultural memory accessible. The internet is a massive network made up of cultural artifacts, symbols, images, texts, and all kinds of trivialities that shape our lives—whose connections are not based on linear logic, but on associative logic that is almost dreamlike, which resembles the human unconscious described in psychoanalytic theory.

Ulmer believes this gives us a profound opportunity. By making the collective unconscious something we can reflect on—something accessible to conscious thought—we gain a method to overcome the deep blind spot that is usually at the core of our experience. Needless to say, I’m not particularly optimistic about this possibility. But whatever you think of Ulmer’s optimistic vision—here I’ve simplified it greatly, but I hope without unfairness—commercialized, chat-based AI has indeed pushed us in a very different direction.

Interestingly, inserting an everyday-language interface between ourselves and digital collective unconscious makes it more blurred and harder for us to understand. The chatbot interface reconstructs the agency by which we navigate the collective unconscious; arguably, it becomes an anti-therapist, redirecting us from self-knowledge and insight (no matter how disturbing or surprising) toward a comfortable and soothing encounter. It offers a false clarity that traps us in self-satisfaction, protects us from self-doubt, and keeps us from lingering too long in our awareness of our own ignorance—or lingering too long in unsettling uncertainty. It veils the tangled forest of human experience, lighting for us a path of manufactured clarity, leading to the promise of knowledge and wisdom. But in doing so, it gently yet decisively draws us back into the unconscious. Maybe that is the root of AI psychopathy.

Whatever the link to AI psychopathy may be, it’s worth thinking about this: the madness, apathy, compulsiveness, aggressiveness, anxiety, and despair in our public and collective existence derive, to some extent, from the gradual retreat of human consciousness in the face of a new form of collective unconscious—an unconscious that reasserts itself in the form of artificial intelligence.

This non-human layer of action in the world was already being constructed before agent AI appeared. Algorithmic processes are already constructing elements of human experience, especially in the context of bureaucratic systems.

I’m not sure when—nor by whom—the conceptual connections between electrical power, the nervous system, and thought were first established, but in his 1851 novel The Seven Gables, Nathaniel Hawthorne wrote: “Then comes electricity, the devil, the angel, the mighty physical force, the omnipresent intelligence!” And: “Is this a fact—or do I dream it—through electricity, the material world becomes a huge nerve, trembling in an instant across thousands of miles? More precisely: the circular earth is a huge skull, a brain filled with intelligence! Or, perhaps we should say that it is itself a thought—a thought only—not an entity like the one we take it to be!”