There's a lot of hype around artificial intelligence. So what exactly is it and how does it work?

Artificial Intelligence 101

Artificial intelligence a really broad term that encompasses a wide range of things such as machine learning, deep learning and natural language processing. In the simplest sense, artificial intelligence is the simulation of human intelligence by machines.

You'll have come across AI before in your life: your favourite streaming platform recommending what to watch next, the maps app on your phone suggesting a better route to avoid congestion up ahead, social media platforms curating your feed to show you content most likely to keep you engaged on the platform, and so on.

Bots have entered the chat

Chatbots are nothing new. They were first invented in 1966 by an MIT professor named Joseph Weizenbaum, who cast it in the role of a psychotherapist and connected it to an electric typewriter and mainframe computer. The chat flow, as reported by The Guardian, looks familiar to our modern eyes:

User: Men are all alike.
Chatbot: IN WHAT WAY
User: They’re always bugging us about something or other.
Chatbot: CAN YOU THINK OF A SPECIFIC EXAMPLE
User: Well, my boyfriend made me come here.
Chatbot: YOUR BOYFRIEND MADE YOU COME HERE
User: He says I’m depressed much of the time.
Chatbot: I AM SORRY TO HEAR YOU ARE DEPRESSED

Fast forward to the late-2010s and customer service chatbots are everywhere, frustrating users by being unable to answer basic queries and asking you to go through the call centre anyway!

Computer says no

This is where it becomes important to distinguish between basic chatbots, ubiquitous in customer support, and GPT-powered chatbots.

Basic chatbots use relatively simple programming to respond to inputs that contain keywords by outputting pre-loaded responses. They can't actually hold a conversation with you. What might resemble a natural conversation is just clever programming that takes the user down a pre-defined pathway.

By contrast, GPT-powered chatbots are not constrained to specific pathways. They are conversational and may properly be considered as AI; they simulate the human behaviour of natural conversation because they can interpret natural language inputs, user intent and semantic meaning in order to generate a human-sounding response.

A GPT by any other name

GPT stands for Generative Pre-trained Transformer. A GPT is a type of large language model (LLM) and its existence came to light in 2017 following a a scientific paper written by a team of Google researchers. Wired magazine explains the story behind the development of GPTs more fully, the brief version is that Google began integrating transformers into translation tools in 2018. But it was OpenAI that took the technology and ran with it culminating in the ChatGPT we know today.

While perhaps the most well-known, ChatGPT is not the only AI chatbot out there. In fact, OpenAI (the developer of ChatGPT) recently tried, unsuccessfully, to trademark the word "GPT". In its decision, the US Patent and Trademark Office said the word couldn't be registered as a trademark because it refers to a general type of software, not just OpenAI products, and registration could prevent competitors from correctly describing their products as a GPT.

Even though they don't have "GPT" in their names, other GPT-based chatbots include Google's Gemini and Anthropic's Claude.

A brief overview of LLMs (not the postgraduate degree)

So, how do tools like ChatGPT work? Again, GPTs are a type of LLM. But attempting to explain how LLMs actually work is where things get very complicated, very quickly. We're not going to get into the computer science here, but if you're interested, this article provides a basic explanation.

LLMs are built on a type of neural network called a transformer model. In very basic terms, an LLM is a computer program that has been fed so many examples that it can recognise and interpret language. The examples they've been fed come from the internet. Essentially, LLMs are trained on absolutely insane amounts of data scraped from across the world wide web (which gives rise to copyright issues, but more on that another time). This is the "LL", or "Large Language" part of "LLM".

Just think about how much is on the internet. All the books, news articles, laws, cases, journal articles and more across all different topics in all different languages. The internet basically contains the sum of all human knowledge. LLMs use a type of machine learning called deep learning to understand and make sense of all this data - how the words, sentences and paragraphs all fit together. As such, LLMs are really good at finding patterns and associations.

Finish each other's... sandwiches

You can think of an LLM as a really, really clever version of the predictive text feature on your phone. Essentially, your phone looks at what you're typing in and calculates the probability of the word most likely to follow next, based on what it's observed from your past behaviour and its training data.

Unlike the predictive text feature, LLMs don't just output single words at a time, but text that makes sense across multiple sentences and paragraphs. The result of all this is, as it turns out, incredibly powerful.

When you type something into ChatGPT, your input is considered unstructured. The reason why LLMs are such a gamechanger was their ability to analyse unstructured data and respond to unpredictable queries.

Like the basic chatbots we talked about earlier, traditional computer programming receives predictable commands in a predictable format. If anything about the input or command is unpredictable, an error will occur.

However, LLMs can respond to natural human language and answer unpredictable questions in a way that makes sense to a human user. They do so by taking what you type to generate what combination of words, sentences and paragraphs that could follow, determine those that are most likely to follow, and output the result.

Hallucinations

We arrive at the key limitation of LLMs: their tendency to totally make things up. In the AI world, this phenomenon is known as "hallucinating".

While it may seem as though you're having an intelligent conversation, LLMs don't actually "understand" anything. They have no concept of what is fact, fiction, true or false. This means that not only can LLMs occassionally output incorrect information, they can do so confidently.

Hallucinations are the biggest obstacle facing AI developers today. And given the architecture underpinning LLMs today, it seems unlikely this obstacle can be overcome anytime soon. In fact, some experts believe hallucinations cannot be eradicated entirely.

For us, reliability and accuracy are absolutely critical to quality legal tech products. We're continuing to research and explore the latest innovations as we build towards our vision of improving access to justice, intelligently. As we do so, we recognise the importance of being upfront and transparent about what we see as the opportunities, benefits, challenges and risks associated with emerging technology.

Support us by subscribing to our mailing list.
Enter your email address to be notified about more articles like this.