By Dr. Patrice Caire
In June of 2020, the US research lab OpenAI, which is backed up by Elon Musk, among others, launched its breakthrough language model: GPT-3—Generative Pre-trained Transformer 3.
Why was it such a big event? Because GPT-3 may just change the way we work: we’re talking about a tool that can summarize mails and news articles, generate tweets and create chatbots, and yes…translate language.
GPT-3 versus Other Language-prediction Programs
As a language-prediction program created with neural networks, GPT-3 is by far the most powerful language model ever created. It is an artificial-intelligence system trained on a gigantic corpus of text: thousands of digital books, the entire Wikipedia, and roughly a trillion words from blogs, social media, and the internet at large. Given enough text—and enough processing—the machine learns probabilistic connections between words. In a nutshell: GPT-3 can read and write.
In fact, GPT-3 can process text far better than its predecessor, GPT-2 (2019). GPT-3 is also distinguished by its sheer size: it boasts 100+ times more parameters than its previous version. GPT-3 uses 175 billion parameters, compared with the 17 billion of its closest competitor, Microsoft’s Turing NLG—making it the largest neural-network ever built.
GPT-3: The Power and the Glory
Get this: GPT-3 can correct English. It can write poetry. It can write like Faulkner, Proust, or Goethe. Any author you “feed” it—really. Even more relevant to our industry, GPT-3 can also:
- translate legal text into plain language—and generate legal text from simple “instructions”;
- write the code to create a website, based on your directions;
- generate well-formatted charts and infographics—from your summary; and
- fill in the missing parts of a spreadsheet, by computing formula on its own.
What We Love about GPT-3
First, using easily understood natural language, GPT-3 can answer questions on any topic, while retaining the context of previously-asked questions. GPT-3 can also translate from spoken languages and programming languages—Python, JavaScript, and CSS (very useful for websites!), for example. GPT-3 can also generate questions and answers for UX, due diligence document search, and report generation.
GPT-3: Dealbreakers
At first, GPT-3’s generated text can be impressive. But longer texts tend to dissolve into meaninglessness.
Another problem: like many chatbots, for example, the program amplifies biases inherent in the data—including racism and sexism.
Then there’s the ginormous amount of processing power that GPT-3 sucks up: the program is miles more “compute-hungry” than even bitcoin.
Finally, there’s cost: conservative estimates place a GPT-3 training run at a staggering $4.6 million—well beyond the reach of most companies.
GPT-3 and You
Right now, GPT-3 isn’t ready to replace the leading conversational interfaces on the market—Amazon’s Echo and Apple’s Siri, for example—but it can be used as a powerful tool to improve them. For now, GPT-3 is a crucial steppingstone to building language models of the future.
So, How Can GPT-3 Be Used Effectively?
It’s unclear. After all, if GPT-3 is eventually shown to generate the right text only half of the time, will it satisfy professionals? Be of use as more than a steppingstone? Let’s see!