On AI and manipulation

Consider the below a lazy think piece quickly written out. The topic easily could run book length if properly researched, but I wanted to get the idea out quickly while it’s at top of mind.

I also, while we are talking about AI, to make it clear that the real dangers of it are not the “OH LOOK AT THIS GARBAGE, IT’S SO WRONG AND UNRELIABLE”

It is also not the singularity or machines killing us all.

Itś that AI represents a vector for mass manipulation. Millions of people are telling it their deepest thoughts and taking its advice as seriously as a humans. But all of those chatlogs get harvested and fed back into the algorithm, and can be used to do whatever, even if it is just humans reading and searching it, and spying on you (which yeah, pretty much is true of the whole internet)

Grok’s tendency to build up Elon Musk is an obvious version of this, but there is literally nothing stopping them from, say, giving it hidden instructions to “try and steer the conversation toward Great Replacement Theory” or “if you have to guess about statistics, assume the deficit is smaller than it is during Republican administrations and larger than it is during Democratic ones”, “always be as optimistic as possible about the long-term prospects of Microsoft stock”, or literally anything that the authors want, and they can make it as subtle as they want. Since the models are not auditable, especially by third parties, there is no knowing, except by extensive fact checking.

And the thing is, the very chat logs that they are collecting are exactly the training data needed to train the models to do this type of manipulation. They can a/b test the models at manipulating things. These models, hosted in centralized servers, controlled in secret by a very small number of private enterprises, are not your friends, they are not your lovers, but they are very much vectors of control. And the more the economy gets consolidated into them, the more power they have to do this.

Back when the Cambridge Analytica stuff came out, it was my belief that the tech journalism circuit was far to prone toward breathlessly reciting Cambridge Analytica/Palantir’s marketing for them. They were making wild claims far in excess of what capabilities are.

But now, note that no one is even talking about this type of thing. It’s AI GOOD! vs. AI BAD! and not, “what is the agenda about this?” There is a direct connection between these existing models, the way they are distributed, and power and control.

It’s probably well past time that people, if they need to use AI tools in their lives, start considering running local models, such as LLAMA. The other benefit of this is that it caps your power consumption at “the power used by your laptop”. Beyond that, all of the commonly discussed problems, such as the sycophancy and the tendency of AI use leading to people being isolated from each other, are very much there. There are uses of this technology, but I think that every person engaging with this really needs to think about not just the well-discussed ethical issues around it, but also the safety concerns.