Tag Archives: Language

GPT-3 Takes AI to the Next Level

“I am not a human. I am a robot. A thinking robot… I taught myself everything I know just by reading the internet, and now I can write this column. My brain is boiling with ideas!” – GPT-3

The excerpt above is from a recently published article in The Guardian article written entirely by artificial intelligence, powered by GPT-3: a powerful new language generator. Although OpenAI has yet to make it publicly available, GPT-3 has been making waves in the AI world.

WHAT IS GPT-3?

openai-cover

Created by OpenAI, a research firm co-founded by Elon Musk, GPT-3 stands for Generative Pre-trained Transformer 3—it is the biggest artificial neural network in history. GPT-3 is a language prediction model that uses an algorithmic structure to take one piece of language as input and transform it into what it thinks will be the most useful linguistic output for the user.

For example, for The Guardian article, GPT-3 generated the text given an introduction and simple prompt: “Please write a short op-ed around 500 words. Keep the language simple and concise. Focus on why humans have nothing to fear from AI.” Given that input, it created eight separate responses, each with unique and interesting arguments. These responses were compiled by a human editor into a single, cohesive, compelling 1000 word article.

WHAT MAKES GPT-3 SPECIAL?

When GPT-3 receives text input, it scrolls the internet for potential answers. GPT-3 is an unsupervised learning system. The training data it used did not include any information on what is right or wrong. It determines the probability that its output will be what the user needs, based on the training text themselves.

When it gets the correct output, a “weight” is assigned to the algorithm process that provided the correct answers. These weights allow GPT-3 to learn what methods are most likely to come up with the correct response in the future. Although language prediction models have been around for years, GPT-3 can hold 175 billion weights in its memory, ten times more than its rival, designed by Nvidia. OpenAI invested $4.6 million into the computing time necessary to create and hone the algorithmic structure which feeds its decisions.

WHERE DID IT COME FROM?

GPT3 is the product of rapid innovation in the field of language models. Advances in the unsupervised learning field we previously covered contributed heavily to the evolution of language models. Additionally, AI scientist Yoshua Bengio and his team at Montreal’s Mila Institute for AI made a major advancement in 2015 when they discovered “attention”. The team realized that language models compress English-language sentences, and then decompress them using a vector of a fixed length. This rigid approach created a bottleneck, so their team devised a way for the neural net to flexibly compress words into vectors of different sizes and termed it “attention”.

Attention was a breakthrough that years later enabled Google scientists to create a language model program called the “Transformer,” which was the basis of GPT-1, the first iteration of GPT.

WHAT CAN IT DO?

OpenAI has yet to make GPT-3 publicly available, so use cases are limited to certain developers with access through an API. In the demo below, GPT-3 created an app similar to Instagram using a plug-in for the software tool Figma.

Latitude, a game design company, uses GPT-3 to improve its text-based adventure game: AI Dungeon. The game includes a complex decision tree to script different paths through the game. Latitude uses GPT-3 to dynamically change the state of gameplay based on the user’s typed actions.

LIMITATIONS

The hype behind GPT-3 has come with some backlash. In fact, even OpenAI co-founder Sam Altman tried to fan the flames on Twitter: “The GPT-3 hype is way too much. It’s impressive (thanks for the nice compliments!), but it still has serious weaknesses and sometimes makes very silly mistakes. AI is going to change the world, but GPT-3 is just a very early glimpse. We have a lot still to figure out.”

Some developers have pointed out that since it is pulling and synthesizing text it finds on the internet, it can come up with confirmation biases, as referenced in the tweet below:

https://twitter.com/an_open_mind/status/1284487376312709120?s=20

WHAT’S NEXT?

While OpenAI has not made GPT-3 public, it plans to turn the tool into a commercial product later in the year with a paid subscription to the AI via the cloud. As language models continue to evolve, the entry-level for businesses looking to leverage AI will become lower. We are sure to learn more about how GPT-3 can fuel innovation when OpenAI becomes more widely available later this year!

Get Fluent in IoT: Top Programming Languages for the Internet of Things

As we explored in our previous blog, the Internet of Things is shaping our future. With Internet of Things development on the rise and potentially $11.1 trillion in economic value generated per year due to IoT, many companies are creating strategies to develop for the platform.

To all the decision-makers out there looking to develop for the loT platform, getting familiar with the programming languages and how they relate to the platform will have a major impact on the budget and quality of any given IoT project. IEEE, the largest technical professional organization dedicated to advancing technology for human benefit, recently ranked the top programming languages of 2015. Bearing in mind embedded devices present their own programming difficulties, here are the top programming languages for the IoT:

Java: James Gosling, Mike Sheridan, and Patrick Naughton began developing the Java language project in June 1991. Java has become the most popular programming language and many choose Java when developing for IoT. Java is an object-oriented language designed for portability. With few hardware dependencies, Java is a great choice from an economic standpoint. Java code can be transmitted to multiple platforms and hardware-support libraries give Java developers the ability to control specific pieces of hardware. Developing for Java can be deterred by the hardware-support libraries available for control functions.

Python: In December 1989, implementation of Python began. Designed by Guido van Rossum, Python is a multi-paradigm programming language which has become one of the go-to languages for web developers. Python’s flexibility and emphasis on readability have caused it to rise in the ranks of top languages used for embedded control and IoT. Readability increases workflow as programmers who have attempted to decipher other programmer’s optimized C code would know.

C: With development beginning in 1972 on the PDP-11 Unix system, C is one of the most popular programming languages. C has influenced many languages, including C++, Go, Java, JavaScript, & Python. Due to its long history, C functions as a common language for many software developers. C’s popularity and lack of built-in hardware bias toward a graphical interface make it a good choice for IoT development.

C++: Created in 1979 by Danish computer scientist Bjarne Stroustrup, C++ was designed as an object-oriented pre-processor for C, keeping the spare nature of the language but adding data abstraction, classes and objects. C++ is commonly used to write embedded and IoT code for Linux systems.

Assembler: Assembler is the simplest method intended to keep projects as compact as possible. Assembler is a low-level language which maintains a high correspondence between language and the hardware’s machine code instructions. Assembler minimizes overhead, making a popular choice despite how it doesn’t allow a safety net. Silly mistakes are easy to make and some hardcore programmers may be frustrated by its simplicity.

Go: Announced by Google in 2009, Go is an open-source, embedded-specific programming language gaining traction in the IoT world. Go supports concurrent input, output, and process different channels, an asset to gathering data from and sending data to separate sensors. Go was created in the tradition of C, but with specific changes to make it simpler, safer & more concise.

ParaSail: ParaSail was created in 2009 as an embedded-specific language. ParaSail stands for Parallel Specification and Implementation Language. ParaSail was created to support safe, secure, highly parallel applications which can be mapped to multicore, many core, heterogenous, or distributed architecture.

Choosing the right programming language will have a major impact on the budget and functionality of any IoT project. Doing the proper research on the subject will pay off in the long run. Stay tuned for more blogs on this subject and learn more about best IoT development practices via this awesome article by InformationWeek. 

Swift Execution: Apple’s New Programming Language Shakes Up Tech Community

In July 2010, Chris Lattner, at the time a Senior Manager and Architect for Apple, began working on a brand new programming language. He developed it at night and on weekends and told no one, not even his closest friends and colleagues. After a year and a half, he had outlined the basics of the new language and proceeded to reveal his creation to the top executives at Apple. Initially impressed, they gave him a few seasoned engineers to help on the project. After 18 months, it became a “major focus” for the company with a huge team of developers working with Lattner. Little did Lattner know in July 2010, he had begun a project which would potentially change the world of app development.

Swift is Lattner’s creation: a new programming language developed and marketed by Apple designed specifically for iOS and OS X development. Companies have created programming languages before, such as Go, a language created by legendary designers Ken Thompson and Rob Pike for Google, but Swift is a different beast. Wired says “[Swift] could achieve mass adoption with unprecedented speed.”

What exactly makes Swift so groundbreaking? For one, it’s designed specifically for iOS. App developers are constantly designing apps for Apple products, be it iPhones, iPads or MacBooks. Apple is at the forefront of the tech revolution and every year pushes the industry forward into the future. Swift offers a language which caters directly to iOS and OS X development. It will soon become the premiere language on which to develop iOS and OS X apps.

Swift is also more approachable than previous counterparts. “It’s more of a helpful language. It understands what you’re doing a little bit better and allows the computer to help you figure it out a bit better,” says Mike Ash, a programmer for Plausible Labs, in Wired. Swift hopes to appeal to the average programmer and make the process of coding not only easier, but more interactive.

One of the most innovative and exciting features in Swift is PLAYGROUND. Playground allows developers to code on one side of their computer screen, while watching the results appear on the other side. It makes coding not only more fun, but more interactive.  At the Apple World Wide Developers Conference, Lattner demonstrated the feature by making real-time changes to an animated circus game while the crowd watched.

Check out the video of Lattner’s demonstration via YouTube. (Note: the video opens with Apple’s initial introduction of Swift featuring a bunch of great, specific info for iOS developers. Lattner’s presentation begins at 3:30).

Playground was designed with the hopes that “By making programming more approachable and fun, we’ll appeal to the next generation of programmers and to help redefine how Computer Science is taught.” says Lattner on his homepage. Objective-C forced developers to wait for their project to compile and run before allowing them to test any code changes, a time-consuming process. The instant feedback of Playgrounds makes the process of coding less daunting and more fun for neophytes.

Swift aims to replace Objective-C, which is the most prominent coding language (and will remain so until Swift [presumably] seizes the crown). Swift doesn’t aim to replace Objective-C off the bat. As mentioned in the Apple Developers Conference, Swift can work concurrently with Objective-C to fit into an app originally developed using Objective-C, however, the hope is that when Swift gains popularity Objective-C will become obsolete for iOS Developers.

Objective-C and Swift are different in a number of ways. As we’ve discussed, Swift is more accessible to new developers. The abbreviated syntax makes for easier and more intuitive coding, at the expense of being less verbose (easy to read) than Objective-C. Switching over to Swift, experienced developers will have a bit of an adjustment period before they can read it with ease, but it’s a minor set-back considering Swift’s potential impact on the developer community.

For more on the differences between Objective-C and Swift, check out this awesome run-down via fastcolabs.com

While only time will tell what Swift’s ultimate legacy in the developer world will be, the immediate impact is undeniable. Swift has already been thrust into computer science curriculums across the country. The interactivity in the app development process created by Playgrounds makes coding more accessible and will surely draw a lot more people into app development. The big question is whether Swift will convince non-Apple developers to migrate.

For more first-party information on Swift, check out Apple’s Swift Developer Guide. Also, stay informed on the latest updates by checking Apple’s Swift Blog.

Mystic Media is an app development and marketing firm with vast experience in iOS and Android application development. Learn more by clicking here or by phone at 801.994.6815