I’m not smart enough to be a developer of artificial intelligence software, but I do know when something momentous is (finally) happening in AI. On July 29th, NYT columnist Farhad Manjoo wrote about a new AI language model that could read and compose text so well that it might even threaten his job as a journalist.
Manjoo called Open AI’s GPT-3 (affectionate terms for Generative Pre-Trained Transformer Version 3) “the most powerful ‘language model’ ever created” and explained that a language model is an artificial intelligence system that has been trained on an enormous corpus of text; with enough text and enough processing, the machine begins to learn probabilistic connections between words.
Not sure Manjoo has much to worry about just yet. Lex Freidman’s YouTube video did this analysis. “The human brain has about 100 trillion synapses, all busy processing information. GPT-3 has a meager 157 billion parameters (which can be seen as analogous to synapses). So the brain wins, for now. But Freidman also states that neural network training is getting twice as fast (meaning it can learn more) every two months. So perhaps Manjoo should make that backup plan.
In late July, William Douglas Heaven, writing in the Technology Review, called GPT-3 both shockingly good and completely mindless. He writes about its often brilliant prose (It can actually compose cover letters for job seekers that are damn good.) and its just as often sexist, biased, and racist remarks. Unlike too many humans, we hope GPT-3 can be trained.
Part of GPT-3’s allure is its ability to learn new things without being trained over and over again on the same task, more like we humans learn. For a deeper look at one-shot and few-shot learning, read Moiz Saifee’s explanation on Medium. Another part of the allure is the wide spectrum of prose styles it can emulate.
Azeem Azhar, editor at Exponential View exudes optimism about the newly released language, illustrating how it can answer endless Wikipedia-like trivia questions but also answer “whys” like why a dog is his favorite animal. Sushant Kumar’s Twitter feed provides a prompt and lets GPT-3 fill in the rest of the tweet. The results are provocatively haiku-ish. (You click on the link and change the last word in the URL in your browser.)
What strikes me is that GPT-3 is a virtuoso at such a wide variety of general-purpose tasks. VentureBeat reported that Melanie Mitchell, a professor of computer science at Portland State University, found evidence GPT-3 can make primitive analogies. Raphaël Millière, a philosopher of mind and cognitive science at Columbia University’s Center for Science and Society, asked GPT-3 to compose a response to the philosophical essays written about it. GPT-3 moonlights as a recipe generator, a comedy sketch writer. Oh, wait. Did I say it could generate computer code and compose music too?
OpenAI began its life as an artificial intelligence research laboratory started in 2015 by Elon Musk, Sam Altman, Peter Thiel, Reid Hoffman, Marc Benioff and others. They were worried about the risks of out of control AI. Today the project has a strong charter to try and ensure that AI will be used for the good of all humanity. At the moment OpenAI’s GPT-3 available in limited beta. The beta’s job is to get those “in the know” excited about the possibilities. It’s reported that GPT-3 will become a commercial product, available for sale as a subscription via the cloud, later this year.
Like the Ginzu knife of natural language processing, GPT-3 is probably something I’ll never master, but I’m watching masterful AI developers crawl out of the woodwork to show off the capabilities of this system. Not owned by Google, Microsoft, Facebook, or IBM, OpenAI is a consortium of sorts. When GPT-3 becomes available to the mainstream this could be the poor man’s road to AI riches… a dataset so robust, yet available to all, for what we hope will be a nominal fee.
READ MORE: https://techonomy.com/2020/08/finally-ai-for-the-masses/
