Human culture in the age of machines
"Delving", technological determinism, and machine-mediated culture.
A couple of months ago, Paul Graham kicked off a discussion about the overuse of words like “delve” by ChatGPT. The idea was that certain words are disproportionately likely—relative to their baseline frequency by native speakers of English, say—to be generated by Large Language Models (LLMs). The use of these words could thus be seen as a kind of “tell” that a piece of text was generated by ChatGPT (i.e., a way to help detect synthetic text).1
Mostly absent from this discussion, however, was the possibility that the use of words like “delve” by ChatGPT—as well as other words, like “tapestry” or “elevate”—could actually cause people to start using those words more frequently too.
This notion might seem strange or unlikely. Could a language model really change the way we use words? But believing this is possible really only requires accepting that the things we say are in part a reflection of the things we encounter: our outputs are influenced—even if not determined—by our inputs. If this is true, and if more and more people interact with models like ChatGPT, then it doesn’t seem preposterous that our use of language could be subtly shaped by how ChatGPT uses language.
I’ve written about this possibility before, in one of my very first posts on the Counterfactual:
I wrote that post in mid-2022, before ChatGPT was released. The ideas in that post seemed plausible to me, but their realization still seemed pretty far in the future. That’s no longer the case.
In this post, I’ll revisit the idea that the systems and machines we build not only reflect but actively terraform our culture—including but not limited to language. And as I mentioned in that original post, I’d be remiss if I didn’t give credit here to Brian Christian’s The Alignment Problem and James Scott’s Seeing Like a State for catalyzing many of these thoughts.
How could LLMs change language?
As I described in my earlier post, there are a few different “pathways” by which the widespread use of LLMs in society could change language. I won’t go into all the details here because the different pathways remain pretty speculative, but I do want to briefly summarize the premises all these arguments depend on:
Inputs influence outputs: The way we use language at some point in time is shaped in part by the language we’ve encountered (or used) in the past.
LLMs count as inputs: Our frequent interaction with LLMs—as chatbots, auto-complete technology, article summaries, and more—is a form of “language input”.
LLMs sometimes directly determine outputs: In many cases, LLMs themselves determine our own outputs (as with auto-complete, using LLMs to write abstracts for our articles, and more).
LLMs use language in particular ways: LLMs represent a particular realization of language use, i.e., a distillation of the not necessarily representative language they are trained on.
Putting it all together: as LLM-generated text constitutes a larger fraction of the language we encounter, it seems plausible that our language itself might come to resemble LLM-generated text.
Now, as I mentioned in that original piece, this could mean a couple different things.
It could be that this results in a fossilization of language, freezing in place the linguistic practices that contributed the most to an LLM’s training data. This is what I call the homogenization hypothesis, and it’s been written about pretty extensively at this point (see this Vox article for a great discussion). It’s the same logic behind the algorithmic entombment writer Jenny Odell writes about in her book How to Do Nothing: algorithms trained on our behavior lead us to behave more similarly in the future.
It could also be that LLMs take our use of language to new and strange territories. If LLMs operate under different constraints than normal human language users, then they might push language in directions we haven’t seen before, perhaps creating a constellation of weird, unprecedented language-like communication systems. This is what I call the nova hypothesis (named after the “nova effect” discussed in Charles Taylor’s A Secular Age)2. In this scenario, LLMs could actually lead to a diversification of language use, albeit in directions totally unlike the diversity of past and present human languages.
I think it’s too early to tell where we’re headed—too much depends on exactly how LLMs are integrated into current cultural practices, whether we rely on central “foundational” LLMs or a proliferation of customized LLMs, and more. But I do think these are plausible future paths and it’s worth being aware of them.
Technology and cultural change
The idea that technology can influence culture is, of course, by no means new.
It’s easy to find examples of new technologies that led to cultural changes: the printing press contributed to the Protestant Reformation; the steam engine helped usher in the Industrial Revolution; and the automobile reshaped our cities and transportation systems.
It’s also easy to find intellectuals who have commented on this dynamic. Karl Marx, for example, famously connected the dominant mode of production in a society with that society’s economic arrangement:
The Handmill gives you society with the feudal lord: the steam-mill, society with the industrial capitalist… (Marx, 1847)
Over a century later, Marshall McLuhan argued that our use of communications technology specifically shaped the way we send, receive, and interpret information:
The ‘message’ of any medium or technology is the change of scale or pace or pattern that it introduces into human affairs. (McLuhan, 1964)
But what about so-called “intelligent” machines? By this, I mean systems trained on collections of human behavior to curate and even generate new behaviors. This includes LLMs and recommender systems, but also more narrowly focused AI systems like AlphaGo.
A machine-mediated world
More recently, a paper published in Nature: Human Behavior tackled this very question. The authors provide some illustrative examples, which can each be situated within a framework of cultural evolution—that is, how intelligent machines could shape the process of cultural innovation and change.
Importantly, contemporary AI systems are generative, meaning they can be used to produce novel cultural artifacts, including text (e.g., ChatGPT), images (e.g., DALL-E), music (e.g., Suno), and even gameplay (e.g., AlphaGo). These novel artifacts, in turn, contribute to a pool of cultural variation, over which other evolutionary processes (like selection and transmission) operate.
One way this works is through something like “recombination”, i.e., composing the various concepts and examples a system has been exposed to during training. With the release of DALL-E 2, OpenAI famously demonstrated this using an example of an “armchair in the shape of an avocado”. This concept was presumably not in the system’s training data, but it did have examples of “avocados” and “armchairs”, and it was able to combine those into something that seemed like a reasonable candidate for an “avocado armchair”. Of course, there’s still a ton of debate about the limits of this kind of recombination. Are these systems really “creative” or are they simply recycling examples they’ve been trained on—and is there a meaningful difference between those scenarios?
In the domain of gameplay specifically, AlphaGo famously surprised Lee Sedol—a former champion of the game Go—with a move that seemed distinctly inhuman and unconventional; AlphaGo went on to defeat Sedol in four out of five games.3 AlphaGo’s novel move probably originated from the fact that AlphaGo was trained in part through “self-play”, so it developed strategies it might not have learned if it was trained purely on actual human games. In other words, it innovated. But what’s most interesting about this is that, at least according to this paper, humans also started playing differently after this—integrating some of AlphaGo’s innovative strategies into their own gameplay.
So-called “intelligent machines” also play a role in selecting what kinds of artifacts or experiences we are exposed to. Most obviously, recommender systems make suggestions as to which products (movies, songs, etc.) we should consume. Similarly, social media platforms make recommendations as to who to follow or “connect” with.4 But even auto-complete is a subtle form of “selection”, making certain words more or less salient as we craft a text message or email.
Life in a machine-mediated world
What are the consequences of allowing machines to create and curate aspects of our culture? What will it be like to live, as humans, in a machine-mediated world—particularly when the “decisions” made by those machines may not always align with our own values?
A key consideration here takes us back to the different “pathways” I mentioned earlier with respect to LLMs changing language. In one scenario, the widespread use of LLMs leads to a homogenization of language, mirroring or perhaps amplifying existing processes of globalization. In another scenario, LLMs lead to a proliferation of idiolects, taking language to uncharted territories (what I call the nova hypothesis).
The authors present a similar contrast, which they refer to as the monoculture and Tower of Babel scenarios, respectively.
As they point out, there are some powerful forces pushing us in the direction of monoculture (bolding mine):
…market forces, such as regulation and market power, may result in a world dominated by a small number of monolithic models…This process may be amplified by feedback loops, in which LLMs train on an ongoing basis from synthetic data or from human data that contains much machine-generated text. Preliminary evidence points to the possibility of model collapse, with the models losing diversity and converging to a state with low variance. (pg. 9)
This notion of “monoculture” is also reminiscent of what philosopher C. Thi Nguyen calls “value capture” in a recent paper, which focuses on the ways in which pre-defined metrics (such as FitBit “steps”) end up narrowing your own goals and values:
Value capture happens when your environment presents you with simplified versions of your values and those simple versions come to dominate your practical reasoning. Value capture offers you a quick short-cut – an opportunity to take on pre-fabricated values. You don’t have to go through the painful process of value deliberation if you can get your values off the shelf. (pg. 3)
Machine-mediated interactions could lead to cultural homogenization in much the same way, particularly if those interactions are mediated by a few dominant machine “models” that make consistent kinds of recommendations or suggestions. A similar argument is put forth in a recent Nature paper (Messeri & Crockett, 2024), which raises the concern that the use of Artificial Intelligence in scientific research will perpetuate “scientific monocultures” and limit the space of hypotheses we end up exploring.
Alternatively, AI models might become increasingly personalized, echoing a kind of “Tower of Babel” scenario (bolding mine):
…if we continually interact with machines that echo and affirm our preconceived notions, we risk isolating ourselves within ideologically and culturally homogenous echo chambers. Such fragmentation can stifle meaningful dialogue, breed misunderstanding and, ultimately, fracture our shared future vision. (pg. 9)
Both options are presented in a somewhat negative light by the authors, which I think is fair—but it might also be worth thinking about a positive vision.
For example, there’s something potentially quite beautiful about a kind of “Cambrian explosion” of micro-cultures and idiolects, facilitated by machines that reflect (and distort) our existing idiosyncrasies. These micro-cultures could be radically different from anything we know today, just like the diversity of life on this planet is radically different from the single-celled organisms that crawled from the muck billions of years ago.
Of course, some might also find something beautiful in the alternative scenario. Words like “monoculture” and “homogenization” cast this pejoratively, but others might prefer words like “unity” or “harmony”. Such a world might have less conflict and less division—it’s essentially what the John Lennon song “Imagine” is dreaming up. (Admittedly, I have an aesthetic bias against homogeneity—to me, the thought that a machine-mediated cultures might erode linguistic and cultural diversity feels like a great loss. But others likely disagree!)
Both scenarios do involve change, however—and change can be scary, no matter how you slice it.
Some possible objections
Finally, I want to consider two possible objections to the line of argumentation I’ve explored in this essay—namely, that a machine-mediated culture will involve much change at all. These are not the only possible objections, but they are two that come immediately to mind.
The first objection is something like: The more things change, the more they stay the same. Put another way, there’s always been some vector of technological or even political influence on language and culture—whether it’s writing, colonialism, Airbnbification, or the Internet. LLMs, and systems like them, are just yet another vector of influence. They’ll have an effect, but this effect will be similar in magnitude to the other factors we already know influence culture. Further, their influence will likely be counteracted by other factors to at least some degree. It’s even possible that the pressures will counteract themselves, as some aspects of a machine-mediated world push in the direction of homogenization while others push towards diversification.
The second objection is something like: No man steps into the same river twice. That is, human language and culture have a telos of their own, which is strong enough that it can outlast the influence of machine-mediated artifacts. I’m thinking, here, of the examples James Scott provides in Seeing Like a State of how various systems of organization—ranging from old-growth forests to subjugated peasants—have a way of subverting the designs placed on them by authoritarian high modernists. This is most obviously an objection to the homogenization hypothesis (i.e., you can’t freeze language into place if it keeps on changing), but it can also be seen as an objection to the nova hypothesis (i.e., the structure of language, as used by humans, will continue exploring some regions of “language space” and resist others, because that’s what language is and always will be).
Both objections posit that it’s unlikely we’ll see a clear net effect in one direction or the other. Obviously a machine-mediated world will involve some cultural and even linguistic change, but it won’t be of a fundamentally different kind than the changes we’ve seen throughout human history. And if there are major changes, they’re more likely to be attributable to the other effects of these systems on society, e.g., on employment.
I’m not sure what to think of these objections, personally. The idea that human cultures have a telos of their own is appealing and almost poetic—but it probably underestimates the power of context to shape our values and practices. Similarly, the idea that machine-mediated changes will just be “more of the same” makes some sense, especially when one considers the incredible changes human societies have already undergone. At the same time, this argument implicitly acknowledges that those changes were indeed major ones—and, in my view, relies too much on the benefit of hindsight in postulating that new changes won’t produce unrecognizable futures.
According to at least one analysis, the use of “delve” in scientific abstracts on PubMed has increased since the release of ChatGPT, which could be interpreted as evidence that researchers are using ChatGPT to help them write their abstracts.
Taylor’s “nova” refers to the proliferation of faiths and quasi-faiths arising in the midst of Enlightenment secularization, as people feel simultaneously the loss of an “enchanted world” but also feel unable or unwilling to return to a more traditional faith.
Sedol, notably, quit playing Go, declaring that AI systems could not be defeated.
As the authors note, most implementations of this kind of thing are also deployed in a commercial context with the aim of increasing profits or user engagement. There’s not necessarily anything wrong with this, but it’s important to know that these aims may not always be aligned with our own well-being (e.g., recommending tweets that make us outraged).