AI is an Orwellian nightmare

TL;DR: In Orwell's novel "1984," we saw how a single government agency took all control over information, how the staff would keep a database of facts but constantly alter these facts to fit the view of reality that their despot "Big Brother" wanted the citizens to believe. The same thing happens now with LLMs presenting a very carefully tuned image of the facts in the database

AI chat bots are an innovative new method of brainwashing, one which nobody could have ever imagined possible even a few years ago: get everyone to trust a mechanical Big Brother because it chats with you like a friendly fellow human would.

I used to be able to search for things using a search engine. It would point me to a source of information, I could decide for myself if the source was good or not. But not anymore. Recently I just wasted a lot of time trying to find something using a search engine only to get lost in an ocean of AI slop.

I finally gave in and asked my question to ChatGPT and it answered my question perfectly, it even showed me where it got it's sources of information. The places it linked me to were either no longer existent (if ever) or only vaguely related. But it at least created the appearance of citing factual sources of information, in whatever way the statistical models have been trained to give that impression to the humans who use it.

I could try other AI chat programs like Gemini to try to verify if this is true, but all of these LLM-powered search engines are controlled by a close-knit cabal of extremely wealthy "tech bros" (who, incidentally, all supported the fascist despot Donald Trump in consolidating his power over the US government). So although there may be the appearance of "competition" between Microsoft, Google, and Facebook's AI products, ultimately all of these products are engineered by a single close-knit group of people. They take what is really a single LLM technology and present it as multiple competing consumer apps in a market, so that you believe you have a free choice between multiple apps. Really though, they are all the same app.

Which means now, all information retrieval has been consolidated under the control of a single tight-knit cabal of tech bros who unabashedly supported a fascist dictator's overthrow of the US government, and people trust these consumer AI products to give them correct information all the time, while all older systems of information retrieval where you can at least see more clearly who has presented the information you are reading (like search engines) have been made completely useless by slop, leaving us with the chat bot or LLM-enabled search engine as our only way to retrieve information. People now rely on that tiny clique of curators of the AIs datasets for access to information, and people who use it trust this information as unbiased and accurate. It was kind of funny how paranoid everyone was about asking the Chinese-developed AI "DeepSeek" about Tiananmen Square. Try asking any English language LLM about any of the crimes against humanity currently in progress which are being perpetrated by the Unites States and it's allies.

It is hard for me not to see this as not just a control of all information, not just direct control over what people believe to be factual, but an attack on critical thinking itself. It is a very deliberate strategy to make people complacent, and to teach them to place absolute trust in whatever it is this cabal of tech bros wants you to believe. In Orwell's novel "1984," we saw how a single government agency took all control over information, how the staff would keep a database of facts but constantly alter these facts to fit the view of reality that their despot "Big Brother" wanted the citizens to believe. The same thing happens now with LLMs presenting a very carefully tuned image of the facts in the database.

In the fictional novel of 1984, people were only occasionally coerced by force, disappeared into brainwashing facilities if they ever became too skeptical of the artificial reality presented to them by Big Brother. But for the most part, people consented to the system of their own free will. People genuinely believed in what Big Brother told them because they had been trained from childhood to believe Big Brother as absolute truth. Our real life Big Brother has different methods of controlling people's free will. AI chat bots are an innovative new method of brainwashing, one which nobody could have ever imagined possible even a few years ago: get everyone to trust a mechanical Big Brother because it chats with you like a friendly fellow human would.

There are minor differences between the fictional world of "1984" and our reality now. The government is not presented as a single Big Brother figurehead, real life Big Brother instead must operate while hidden behind the facade of "freedom of choice," such as a marketplace of apps, or of competing big-tech corporations. The method of coersion is slightly different: you are not forced to use chat bots, real life Big Brother still needs carrots and sticks, luring users in with useful features like a chatbot that tells you how to make your smartphone do something for you.

It seems to me that keeping your own books, creating your own search engines and digital catalogues of information, and speaking with other real humans for information instead of AI (and using the fediverse) are all acts of rebellion given this new political reality in which we now live.

Edited: Wed, 2025-04-16 07:58 UTC, for clarity, added statement explaining why I think search engines are useless, and how this relates to people unduly trusting AI more.