, Ramin Honary: AI: Think of the Possibilities

AI: Think of the Possibilities

Home About Codeberg

This is an introductory article to what I expect will become a series of articles summarizing my thoughts (and the thoughts of other computing and machine learning professionals) on the latest developments in artificial intelligence. Make no mistake: this technology is revolutionary in the same way that other technologies were, like the printing press, the steam engine, or the digital electronic computer.

List of articles

Think of the possibilities

Artificial intelligence, more technically called machine learning, and even more technically, artificial neural networks, are becoming more and more amazing in their capabilities. They first started doing some truly disturbing things a few years ago with "deep fakes," creating images that look absolutely real of people that never existed by sort-of averaging the images of millions of other people. They can even create animated images, swapping your face with that of a celebrity or politician, allowing you to make a video of you doing or saying things as that celebrity or politician that they never actually did or said. Then came the deep fakes for peoples voices, so with enough samples of a persons voice, you could translate your voice into the voice of a celebrity or politician.

People immediately saw the potential for abuse, but their fears were placated when it was shown that you could train other neural networks to detect deep fakes. Of course, I think what ordinary people don't yet grasp is that Generative Adversarial Networks (GANs) are created by putting two neural networks in competition with each other and letting them learn how to best each other. So it is possible to train an AI that learns how to beat the deep fake detectors, and then you have to re-train the deep fake detectors to beat the deep fake AIs. However it seems to me that for the time being, people are not quite as concerned about deep fakes anymore.

But then the GPT algorithm was developed. The various GPT networks, suffixed by a number (GPT-2, GPT-3, GPT-4), each successive network being bigger and more capable than the previous, have got everyone's attention now. The reason is that these neural networks actually seem to understand things. You can ask it questions, and it gives you a reasonable answer.

It passes the Turing Test with flying colors (an informal, unscientific test that simply asks, "can this machine trick people into thinking they are talking with a human?"). It can probably play and win the game of Jeopardy, or at least, come close to what the IBM Watson supercomputer achieved 10 years ago but with a fraction of the computing power (and electricity) that Watson required. It can probably correctly diagnose some diseases by reading a person's description of their symptoms. It can pass the Bar Exam, and probably offer sort-of reasonable legal advice for very simple cases. It can write simple computer programs for you by just you explaining what you want your program to do. (See this video demo.)

How does GPT work?

GPT is a class of modern neural network architecture called language transformers. A simple (perhaps too simple) way of describing how these AIs work is that they begin with a massive chunk of text called a corpus. In the case of GPT, the corpus is called The Pile which is around 825 gigabytes in size. All of it is written by humans, and has been taken from places on Internet that are free to use and permissive about copyright. Through a process called training this mind-bending ream of data is compressed into a massive chunk of statistical information, analogous to how video or image data is compressed (such as with ZIP or MP3 files).

These chunks of raw statistical data called neural network models begin to compress the corpus more and more, and it begins to encode intricate patterns in the data that loosely approximates the grammar and semantics of human language, quantifying relationships between similar words (even between different human languages), and rating the emotional sentiment of clusters of words. And I must stress, it does all this without any human intervention to guide the formation of the neural network model, a technique called unsupervised learning. The only human intervention involved was during the process of collected the training text.

When the neural network model completes training, you can use the model by taking an ordinary human sentence, encoding it as a very big number, and transforming this number by applying it to the statistical data of the neural network model. A new very big number appears at the other end of the computation, and after decoding this new very big number using the reverse of the process that you used to encode your sentence, a new sentence of words is formed. Somewhat magically, this sentence that is decoded contains words and sentences that appear to be written by a human.

Since the neural network model has statistical information about the similarity between words encoded within them, these neural network models are very good at human language translation (e.g. English to Japanese). They are good at summarizing larger pieces of text, and they can also report whether a paragraph has a positive or negative emotional tone to it — advertising companies who do market research such as Google are very excited by this. And, most impressively, the model can take a sentence and predict what sentences might most likely come after that sentence. The GPT chat bots you may have seen use this mode of predicting the next sentence to create impressive the results that are gaining them so much attention.

The neural network model is a massive chunk of statistical data, tens of gigabytes in size, but still considerably smaller than the training data that it has compressed. Just for comparison, a BluRay disc holds roughly 45 gigabytes, a modern cell phone has about 8 gigabytes of working memory available. (Again, the training data is typically around 825 gigabytes.) Obviously these neural network models are too large for ordinary personal computers to use, let alone build from the training data. So they have to run on large server computers and be used over the Internet. That introduces the first problem.

The public has no control over this technology

The individuals who use this AI, or the people who contributed training data do not control the model, the company that built it controls it. It is not something you yourself can own, or something you can do yourself in your own home, unless you happen to own a and operate a computer that costs more than your whole house. You are dependent on a for-profit company selling you access to this technology as a service, even though they exploit a free public resource to build their service. And you have no guarantee whether the company selling this service to you has constructed an AI that is free from the political biases of the proprietors of that company.

The trained models then can do these amazing things that we have been seeing. But as any professional doctor, lawyer, or engineer, will tell you, the answers that GPT gives you are wrong more often than they are right. And even when it is wrong, it tells you with an air of supreme confidence, mimicking the style in which all articles on the Internet are written (including this article that you are reading now). It even offers to you a plausible-sounding explanation for why what it said is true.

If the Internet were full of people always being very cautious and uncertain about everything they said, the GPT chat bot would probably mimic that sentiment. Instead, the neural network speaks with the tone of an amateur who believes they know much more about a subject than they actually do — it did take the training data from places on the Internet not guarded by pay walls, after all. This introduces the second problem:

People may begin to trust and rely on it, even if they shouldn't

Suppose AI becomes much more accurate, and if people come to rely on these AIs, who is to blame when the AI leads you astray? For now, web apps like ChatGPT have a disclaimer warning users that the information produced by their AI is not to be taken as accurate or authoritative, but as these AIs improve in accuracy, people may begin to put too much faith into them. We have already seen this with self-driving cars leading to fatal accidents.

I tried using a GPT model myself (ChatGPT from the OpenAI company). After spending some 15 minutes trying to explain to the computer what I wanted it to do, and after many completely wrong answers, it only managed to write the smallest, simplest program that could satisfy my prompt, a program I could have written myself in 30 seconds. My second attempt, I used it to search for a technology that I thought may exist. The machine first told me it existed and what it was named. But after I pressed it to tell me where I could read more about this technology, it admitted to me that the technology they named didn't actually exist, but if it did that is what it would be called. So it is not quite ready as a personal assistant to any professional. Not just yet, anyway.

But the technology will improve, and so will the accuracy of the results. Computer hardware manufacturers, especially companies who make GPUs (graphics cards, which let you play games that look absolutely real) are now creating Tensor Processing Units (TPUs) for training and running artificial neural networks that power modern artificial intelligence like GPT. So it is only a matter of time before the cost of creating larger and larger GPT networks comes down, and these larger neural networks won't get the answer wrong quite as often. People may start using them to do real work. But who is responsible when the machine makes mistakes? Should we let these machines completely replace humans? That introduces the third problem.

Society is not ready to let people without jobs live peaceful, happy lives

The implications of an AI that is getting increasingly better at tasks that require expert knowledge, obviously, is that a lot of white-collar jobs are probably in danger of disappearing. From my own professional perspective, I can see my job as software engineer as probably safe.

But the job of the computer programmer is not safe anymore (which I am saying is a different job from that of software engineer). Anyone going through coding boot camp to learn how to develop apps for your iPhone should reconsider their future career path. I am fairly certain all apps that you find on the app store, even games, will be developed by AI in the near future. Not a big loss, since those jobs were horrible to begin with — lots of busywork, not the slightest bit of creativity required to do those jobs. That said, those jobs do pay the bills for lots of people, unfortunately not for much longer. I can think of some other examples...

It really would not be a terrible problem if some of these jobs disappeared but ONLY if there were an abundance of other jobs that anyone less skilled than a doctor, lawyer, or engineer, might still be able to do. That, or we would need to completely reorganize society around the idea that not everyone needs to have a job, and people without jobs could still live peaceful, happy lives doing whatever they wanted to occupy their time (make art, solve puzzles, play games). Perhaps there should be a social contract where people live to train AI, and in exchange, businesses using AI could provide goods and services to people for free.

If only everyone ran a small businesses using AI to do the job for them

If it were possible for an individual professional to buy a computer, train the computer to do their job, and then keep pulling a salary or wages for the job their computer does for them, then I would be an enthusiastic supporter of this technology. This would be a fulfillment of the promises of modern technology that I heard as a kid — that it will increase our leisure time and improve the quality of our lives.

But that is not how things work right now. On the contrary it seems to be getting harder year after year for small business owners to sustain profitable businesses. With many corporations using AI to automate the jobs that many of these small businesses now do, it will only get harder for small business owners. In the US, it is getting harder and harder to find any jobs that pay a living wage at all, or any jobs that provide useful medical insurance, regardless of who you work for.

There are technical challenges as well. The neural networks we train require so much data, and so much computing power to create, that only the worlds wealthiest companies (Google/Alphabet, Meta/Facebook, Amazon, Microsoft, Apple, IBM, Oracle), have any chance of controlling this technology and profiting off of it. All the rest of us lose our jobs, and the corporation keeps all the of profits. They of course also pocket the savings they get from firing people. With this possibility in mind, the future looks bleak and dystopian.

Many have said it, and it is absolutely true: neural networks like GPT are truly revolutionary. And with technological revolutions, political turmoil, and political revolutions, soon follow. The US Civil War, and the first two world wars, were the wars that book-ended the second industrial revolution. We are starting to see how AI plays out in modern warfare, especially with armed drones, in conflicts like in Turkey vs. Iraqi Kurds. Us humans are in for a rough ride these next few decades

...And I haven't even begun to talk about what might happen if we build machines that are more intelligent than humans.

Learn from the mistakes of history

Let's just say that the kind of slave uprising story of machines becoming sentient and overthrowing their human overlords, the kind of story we mostly saw in science fiction films of the 20th century (e.g. James Cameron's The Terminator) are probably the least of our concerns at this point in history. In the long term, we are probably more in danger of becoming slaves ourselves, not to artificially intelligent machines, but to the people who own these machines and use them to give themselves superhuman abilities to exert their will on anyone who did not win the lottery of birth into a wealthy family.

But my biggest fear right now, in the short term, is that the pace of technological evolution is far outpacing our human ability to develop a system of ethics around it. It is time to really start thinking about what the future of humanity is going to look like. With challenges like:

...we cannot afford to just wait and see what happens. We have seen it all happen before, when a new technology gradually shifts the balance of power from the many to the few until world-wide conflagrations like World War 1 and 2 occur. In my opinion, I believe our current historical situation is more similar to that of development of the printing press rather than the machine gun. Like the printing press, unlike the machine gun, more people having control over this technology will be better. Though currently only a very small number of people control this technology.

Regardless of what your opinions are of the communist revolutions of the late 19th and early 20th centuries, they happened for the very practical reason that people could no longer live under the conditions that had evolved at that time as a result of the industrial revolution. And we are dangerously close to approaching that tipping point once more in the year 2023. We need to begin developing a system of ethics regarding the use of AI, and codifying these ethics the into law, immediately. We especially need some guarantee that all people will benefit from this technology, and not just the few people who are wealthy enough to own the worlds most powerful supercomputer.

Sources: