Ethicswishing

"What some people may not understand is that we are an ethical company."¹ Close your eyes and you can hear that sentence in the voice of a tech CEO. You can picture the Congressional testimony and the media coverage as they continue: "Our goal is to be the most responsible, effective, and respected developer, manufacturer, and marketer of consumer products."² Tech reporters are scribbling it down, ready to breathlessly cite it. Turn your imagination to that company's website and you can further depict the matching corporate mission statement declaring "our continuing commitment to being socially responsible and to aligning our thoughts and actions with the common good of the communities we serve."³

You'd be wrong though. It's not that your imagination is defective, but I have to tell you that these real statements don't come from a tech company. They're all from Philip Morris.⁴

Most people act in good faith. There are some true psychos out there but, overwhelmingly, people are just not that cynical. We often take the assumption of good faith to mean that we should not be too quick to ascribe malice and that is correct, but there is a more important consequence to understand from the generality of good faith. The people who defended the smoking industry were often acting in good faith. Many people defending fossil fuels are acting in good faith. I will stop short of citing worse atrocities but the observation holds: we have to take to heart the fact that many people who have contributed to exactions that can readily be called evil were acting in good faith, with good intentions, and according to values which they believed brought good into the world. (A form of the banality of evil.) We have to face the fact that most misery throughout humankind's existence has been caused by people who meant well and in accordance with values which they believed were moral. Without a doubt the most important question to take away from this is: if so many people, often intelligent people, have been so wrong and caused so much strife, how do you know that's not you too?

We already know about ethics bashing, which is the trivialisation of ethics, and about ethics washing, in which companies instrumentalise ethics to improve their public image, reassure their employees that they shouldn't be ashamed of their overinflated pay, and generally try to dodge regulation⁵. What I am concerned with here however is more pernicious. Unlike bashing it doesn't intentionally consider ethics to be a trivial concern and unlike washing it is not intentionally malicious. Ethicswishing (in tech) is the belief that if you are committed to being ethical and understand technology, then you are well-equipped to build technology for social good. But the truth is that building tech for social good is a lot like having sex in a bathtub: if you don't understand the first thing about sex, it won't help that you're a world-class expert in bathtubs.

Tech is awash in ethicswishing and that's a problem: it leads well-meaning people astray, it sucks the air out of the room, it wastes effort on useless work, and more importantly it gives room for the worst practices in tech to go unchecked. Could we stop getting it so wrong?

Authority & Ethics

The underrated TV show Mrs. Davis⁶ features a shockingly accurate depiction of ethics in tech as commonly practiced today. Without spoiling anything, the show features a world-ruling AI and when we flash back to meet the AI's creator we see them asking "What if the tech we used actually helped us instead of harming us?" They're not vying for world domination, they just built "an app that is constantly evolving, getting to know us, and therefore supporting us in ways that we, as a society, have failed to do for one another." And the framing is almost a perfect summary of the ethicswishing faith in a programmable Platonic Good: "A revolutionary self-learning machine that prioritizes indigenous practices of community care and mutual aid by incentivizing acts of service and mobilizing individuals to perform those acts, all through a social justice lens, using ethical methods."

Why is it mechanically unavoidable that a system designed according to those parameters — wonderful as they may read — will be authoritarian? Because one necessary component of freedom is missing: politics. Who does the prioritising, the incentivising, the mobilising? Not the people who are being mobilised, incentivised, or whose actions are being prioritised. There's nothing wrong with indigenous practices of community care, mutual aid, or social justice — quite the contrary — but the idea that there is a ranking that can optimise for these values and deploy them to people without deliberation and decision counter-powers is profoundly delusional. "Power to ethical people" has been the central tenet of every authoritarian regime with a mission.

The very idea of a benevolent dictatorship is a myth, an obsession for delusional philosophers twenty-five centuries in the going. Ethics without limits on power imbalance is like science without experiments: vapid theorising in a vacuum. No checks on power means no reality checks. No volume of data can get the powerful to ask the right questions (or face the right answers). If the scientific method, understood broadly, is the process of fucking around and finding out, ethics can never work if you do the fucking around but someone else does the finding out. It's possible for a person in power to be personally ethical and it's possible for a powerful entity to occasionally do good in the world; it is not possible to build a durably ethical system that features significant power differentials even with the best of intentions.

I used to be dubious about the idea that power necessarily corrupts. Personally, I'm not very interested in money and I would generally much prefer that people leave me alone rather than give me power over things. I can imagine being powerful without losing my sense of what's right. I think that many well-meaning folks, technologists or otherwise, feel that way. But that is precisely the mistake: power does not corrupt moral fibre, power corrupts intelligence. The greater the power differential the lesser the access one has to others' realities. The more powerful you are the harder it is for feedback to reach you and force you to adjust, whether you seek it or not. And lack of access to reality compounds over time, compounds when your technical competence and the impression of being ethical strengthen your sense of being right, compounds when your colleagues are the same, compounds when your employer does well even if that may be due to rent seeking rather than any actual quality.

There is no such thing as authoritarian ethics. There is no such thing as apolitical ethics. There is no such thing as a benevolent dictatorship. As Kjetil Kjernsmo once told me, "it is not the benevolence of powers that produces good results, but the balance between them." You cannot bolt ethics onto undemocratic tech and the only ethics for collective systems is in checks and balances. A lot of ethicswishing works by writing up a list of non-binding values but non-binding values are useless. They work by hoping that powerful parties will refrain from using that power to pursue bad outcomes, even though they retain what is known as reserve control. Reserve control exists when an entity has the power to dominate another and decides not to use it — but could use it at any time. Over a short time frame using non-binding values to obtain short-lived ethical outcomes via reserve control may work — but over time it is guaranteed to erode. Ethics doesn't survive reserve control. (A form of enshittification.) This is the Iron Law of Kleptocracy: over time, reserve control is guaranteed to become active control in service of extracting greater rent.

Is Tech Ethics The Baddies?

I have met many engineers who have tried to get involved in tech ethics only to have the most discouraging experience. Ethicswishing exists in a world in which individual responsibility is the primary motive power, from the genius of an inventor to the itemised list of values you can personally subscribe to, just vague enough that they could never hold you to account. The kind of enchanted world that you can believe in only if you have never really found yourself at the business end of powerful constraints. A world for Prodigal Tech Bros. At the heart of that world you may encounter a privileged bubble of people who need neither notice power nor seriously reckon with its effects nor care about how money affects outcomes. Instead, they swan through life as an endless teatime of the charity club where they praise each other for saying the right non-binding words, preen themselves for being part of the right crowd flashing the right vacuous slogans, and generally accomplish only one thing of any substance: an adamant, narcissistic, simpleminded furtherance of the status quo.

If you've met such people you may well be tempted to write off any interest in ethics as entirely hopeless. Please don't. It's not all like that. We can hold onto that kernel of good intentions and find a path to good. Many people — self included — have fallen now and then to ethicswishing simply because challenging power is hard. Creating checks and balances is difficult, error-prone work. Some outputs are part ethicswishing and part useful because they can be enforced though perhaps insufficiently, and we need to see how we can build from that toehold no matter how meagre. We can pull each other up by holding one another to account.

As is often the case, this problem is less new than we tend to think. Decades ago, the technocrat movement sought to address "the paradox of a society victimised by abundance and by technology."⁷ They believed that they could bring good to the world through "the beneficent activities of expert social engineers"⁸ and that "Technicians and scientists alone possessed the expertise to deal with the technological society."⁷ Presumably, if your technological experts are failing to deliver "beneficent activities" it must be due to a shortcoming of their expertise — which can be fixed! Just make a shopping list of the values they should have and deploy the ancient chthonian power of webinars to train them! Ultimately, tech ethics is the technocratic solution to technocracy. And, as such, it doesn't work.

Anywhere a powerful entity operates it is at risk of unethical behaviour and therefore must be held in check by a control mechanism. Sometimes, competition is enough. Sometimes regulation by territorial states will be effective. Sometimes the best control mechanism will be another kind of collective governance system. In no case is a non-binding commitment — oxymoron if there ever was one — the right choice. The only ethics that matters in collective systems is that which informs the shape of the governance mechanism for that system. Everything else is misdirection that is only a transition from "doing bad things with tech" to "doing bad things with tech but oh well we tried!"

Aspiration, Imagination, Ambition

Don't get me wrong: I love ethics. Ethics is vital to a life lived well, in fact it is what defines the good life. Some of my best friends are ethics.

I am not encouraging you to forgo ethics — on the contrary, I encourage each and everyone of us, as individuals, to develop our moral skills. Moral attention, for instance, is the ability to discern and attend to the features of a situation that are most salient for ethical judgement. It is a cultivated "knack" for perceiving the moral contours of a situation.⁹ In this sense, if you wish to be moral, you have to also pay attention to whether what you're doing actually works. And the best way to do that is to set up a forcing function for it: that's what checks and balances do.

As technologists, given where the world is today, developing ethical skill matters particularly along three purposes: aspiration, imagination, and ambition.

We need to develop our aspiration as an alternative to either giving up or to sinking into doing nothing beyond criticising. Pointing out ongoing problems is important — Rosa Luxembourg went so far as to say that "the most revolutionary thing one can do is always to proclaim loudly what is happening" — but it is dangerously easy to live just there. We have, as Maria Farrell puts it, a duty of hope. Aspiration — that gutting sense of needing a better world — is a habit to cultivate.

For that to work, we need to imagine alternatives, the more detailed the better. We must imagine the concrete path to our aspirations. Imagination isn't just a trite word to make your heart glow in pulp-class young-adult dystopia — imagination is the ability to depict justice, to see what we ought to aspire to. It is not a gift but rather a skill to hone.

And finally, we must nurture our magnanimity, which is to say our moral ambition, our moral leadership, "the ability to 'think big' in one's moral aims."¹⁰ We can't accept just the crumbs of improvement that are possible, we must learn to expect much more and commit ourselves to it. Magnanimity comes from the Latin for "big soul" — what better way is there to live?

That's for the goals we can set ourselves as individuals. But when it comes to bringing justice to today's tech, which is almost entirely made of collective systems, we don't so much need tech ethics as we need the means to push back against domination and push forth for the needs of people impacted by those systems. Put differently, what we need is tech governance or rather, if you too are tired of euphemisms, tech politics.

Illustration from the awesome NYPL.