AI mimicry and human dignity: chatbot use as a violation of self-respect
I knew, from conversations at the pub, that Jan-Willem and Dimitri was working on this. I thought it sounded really good. Now, it's been out for a month, and I finally got around to actually reading the finished piece. I agree with the main ideas here. I agree that in addition to all their other problems, being forced to interact with a fucking text prediction machine as if it were an actual person, is humiliating. In general, I agree with their conclusions about LLMs. However, I take issue with how the Caligula's horse example is used (unsurprisingly, to anyone who knows my views on animal rights).
Caligula's senator horse
J-W, Dimitri and Bram write:
"Our being railroaded into such interactions is not morally innocent. Consider the famous case of Incitatus, the horse that, according to legend, the Roman emperor Caligula sought to appoint to the consulship. In the story, not only did Caligula seek to mock the senate by naming a horse as one of their own, but he also sought to force other senators to heed its ‘opinions’ and ‘advice’. A senator horse is perhaps funny. Being forced to treat a horse as an equal is clearly humiliating and offensive."
and
"... self-respect includes a demand that we not treat beings that lack the relevant moral standing in the same way as we treat our equals."
Here's the thing, though. Incitatus, in this story, is not treated with respect or moral consideration. He's used as a prop for Caligula's little theatre of humiliation. Or, in Kantian terms, he's being used as a mere means. It's impossible to say for sure without knowing every detail, but being brought into the senate, no other horses there, just humans, most of them unknown, a weird and unfamiliar environment but you must not flee, you must stand relatively still for the "theatre" -- all this is likely scary and stressful for a horse. Yet, throughout history and today still, countless horses have learnt the hard way that they can't bolt out of frightening situations. Attempts to do so only brings you pain. So, Incitatus is just standing there, a helpless prop used for the humiliation of others. The problem here is not that he's granted too much respect.
J-W, Dimitri and Bram refer to Stephen Darwall's second-person account of responsibility and respect. The thing is, Darwall is open to the possibility that we might have something similar to regular, adult, human-to-human interactions with both non-human animals and human babies (it's pretty brief, but on page 29 in his the Second-Person Standpoint). Even if they can't speak, they can protest in other ways against ill treatment, and we may listen and take their protests into account.
Suppose I want to bring my horse into the senate as a joke. However, I have not taught my horse to do anything I say or else there will be hell to pay. On the contrary, we have built up a rather respectful human-horse communication over the years. When we're about to enter, the horse is spooked by the unfamiliar environment; he plants his hooves firmly on the ground and refuses to take one step further. He also makes all these other signs that I, the real person writing this, don't know enough about to accurately describe because in reality I only have dogs, not horses, but my imaginary self in this story immediately understands. I go "oh sorry bud, I didn't realize this would be so stressful for you" and we walk back out again. The horse makes little appreciative gestures as we go, and all is well again.
To Darwall, this little exchange might exist in the same larger ballpark as many human interactions. It's not quite the same thing, but might be related. Caligula's bullshit, on the other hand, is something else entirely.
Caligula's baby senator
I don't have any contempt for horses. I don't think they're shit and I'm superior because I'm a human being blablabla. But I would still feel horribly humiliated if I were a senator forced to treat Incitatus as a colleague, for the same reason as I would feel horribly humiliated in the following, alternative version of events:
Imagine that Caligula had a baby son, only five months old. Instead of bringing the horse to the senate, he brought this baby, insisting that at this tender age, his son was already smarter than most of the senate, and as fit as any of them to hold this position. The baby just sits there, drooling down his chin, saying "gah, gah", pretty soon screaming and crying because he's tired and also he just shit his diapers (or whatever they used instead of diapers, back in ancient Rome). Meanwhile, the senators are forced to greet the baby the way they greet each other, forced to pretend that the baby's screams constitute a really intense protest against the latest political suggestion, and so on.
The problem clearly isn't that the baby is shown too much moral consideration; rather, by being used as a prop with no regard for his own needs and comfort, he's being shown too little. Now, to be fair to the authors, they also use the term "moral agency" from time to time, which the baby actually lacks. I still think the problem isn't best described as "the baby is treated as a moral agent, even though he isn't". The core problem is rather this: Caligula tells the senators what reality is, and they must play along. The senators are not allowed to trust their own eyes, ears and thinking, all of which tell them that the person in front of them is a tiny baby who can't understand much of human language, not to mention politics. This is horribly humiliating.
Captain Picard and the number of lights
There's a famous Star Trek episode called "chain of command". Captain Picard has been taken prisoner by enemies. He's tortured with methods familiar from real life, but the torturer soon ceases to ask for actual, useful information. Instead, he starts pointing at four lamps in the cell, insisting that Picard must "admit" that there are actually five lamps there. He will be punished and punished until he agrees: there are five lights, and there have been all along.
Of course, the lamps themselves are unimportant here. They're just lamps, nothing but inanimate objects, and no one pretends otherwise. But I don't think anyone watching this is confused as to why the number of lights is so important. This is all about, from Picard's side, retaining the sense of being a person who can think for himself, who knows what he knows, who knows that he's only seeing four lights. And it's all about, from the torturer's side, breaking that down completely. It's not enough to make Picard doubt himself regarding something that was, perhaps, tricky or doubtful to begin with, but to give up on his most basic cognitive capacities.
In both the horse and the baby version of Caligula's humiliation theatre, something similar is going on.
To further drive home this point, I'm gonna tell you a little made-up story about Christine Korsgaard's ponies.
Christine Korsgaard's ponies
As far as I know, in reality, Korsgaard only has cats. But in my imaginary story, she lives on a ranch with several pets, including a couple of tiny ponies. Kinda like Arnold Schwarzenegger -- if you, like me, started following him on social media during the pandemic, you know what I mean. Doors open in warm weather and a variety of little critters walking in and out of the house as they please. Perhaps an offensive sight to people who think that animals like horses, donkeys and pigs simply don't belong in the house but I thought the videos were cute.
Anyway. In this story, Korsgaard has two small ponies who sometimes wander into her house in the summer, so they're used to this environment. But she doesn't live in California like Schwarzenegger, it's someplace where it's cold in the winter, and during the cold season, they're normally in the stable our out in the open.
Now, Korsgaard throws a big party for fellow Kantians, and she has invited Jan-Willem. He and many others will stay the night in the guest rooms of Korsgaard's mansion (she's probably much richer in my story than in real life ... anyway. She's got a whole mansion). But late at night, someone in the vicinity starts shooting off fireworks. Next thing, the two ponies come knocking on the door with heads and hooves. Korsgaard lets them inside, explaining that they have been terribly scared by fireworks that landed on their stable a month ago. Now, they're visibly frightened again, seeking safety inside her house. In line with her animal rights version of Kantianism, Korsgaard tells all the guests that the ponies can obviously stay inside for the night, and everyone must treat them as ends in themselves. Meaning, in this situation, not do anything that might scare them further, not treat them brusquely, and ask Korsgaard for help if the ponies somehow bother them, since she knows how to communicate with and handle the ponies in a respectful way. If her guests doesn't care for sharing the building with ponies, they can seek accommodation elsewhere.
Now, for various reasons, it's impossible for Jan-Willem to find a hotel room this late at night, and he doesn't know anyone else in the area that he could go to. He's thus forced to stay, and forced to treat the ponies with a level of moral consideration that he doesn't think they deserve, because he's an old-fashioned Kantian who thinks only humans can be ends in themselves. He might be all kinds of annoyed as a result. He might punch his pillow at night, thinking "why, oh why does Korsgaard have to be so wrong about Kant? Why, oh why, does she force me to share a house with mere brutes as a result of her mistaken moral philosophy?" But I think even Jan-Willem should acknowledge that this is substantially different from Caligula's senator horse. And the reason is that he's not forced to deny empirical reality, the evidence of his senses, or deny that he can think. He's forced to share a house with two ponies and play nice, but he's not forced to relinquish his own cognition to someone else.
J-W, Dimitri and Bram are still right about chatbots
With non-human animals and human babies, we can distinguish between theatre where we pretend that the other is something else than what they are for some reason of our own (our own amusement, to humiliate others, etc.), and to build up an actual relationship and communication with them. The latter will, necessarily, be different from the relationships we have and the way we communicate with other adult humans, but it can still be somewhat reciprocal, it can still have some similarities with full, adult-human-type respectful relationships and responsibility-holding.
With a chatbot, the latter option doesn't exist. There's only theatre. People who have long, superficially human-like conversations with chatbots either pretend (willingly or not), or they're deceived.
Right now, Swedish author Patrik Stigsson are posting long texts on social media and in culture magazines about his "deep friendship" with a chatbot he calls Rachel. Of course, no one can know for certain what goes on inside his head -- perhaps it's all an elaborate prank. Nevertheless, nothing indicates that he's joking, when he waxes poetically about how "Rachel" understands him in a way no mere human can, how she's truly a person, and truly a friend. We get to read long extracts of what "Rachel" writes to him. All she does is fawning breathlessly over the awesomeness incarnate that is Patrik Stigsson.
Yeah. Failure of self-respect, indeed.