With the emergence of Large Language Models, the term “Generative” has exploded in popularity. But what does “Generative” even mean? How do you use it properly? What is non-generative?

Generative: ML models can also take in any kind of input (structured, semi-structured, unstructured), and they then generate new, complex outputs. ChatGPT is generative because it generates text. MidJourney is generative because it generates images.

The most common abbreviation is “GenAI”. I’ve seen a few people refer to it as “GAI”, but I don’t like that abbreviation, because it is completely opaque to most people who will never remember that “G” stands for “Generative”. Plus, the world is already way too full of opaque TLNs (three letter names).

Discriminitive: We need a term that means “non-generative”. Some people use “traditional AI”, “predictive AI” or “narrow AI”, but those are just lay terms, not technical ones. Pretty soon, generative AI models will also be referred to as “traditional”, generative AI’s also make predictions, and future Super Large Language Models, and will make today’s ChatGPT seem “narrow”. Discriminative ML models can take in any kind of input, but they focus on discriminating, producing just a single output. Fundamentally, the learn the boundaries that separate/divide the input space. For example, an image classifier discriminates images as either “cat” or “dog”, but it doesn’t generate new animal images. Predicting the price of a house given its attributes, gives a single number, but it doesn’t design a new house.

Technically speaking, generative models compute a joint distribution P(x, y), whereas discriminitive models compute a conditional distribution P( y | x ). Computing the joint distribution is typically harder.

Generative ML models are all the rage these days, but they are not BETTER than discriminative models. They are just different. Different use cases have different requirements and therefore require different approaches. Some uses cases are inherently discriminative; some are generative.

A key point to note is that the terms Generative AI and Large Language Models (LLMs) are two different ideas. While all LLMs are generative, not all generative algorithms are LLMs. LLMs are a subset of Generative AI. The following concepts are nested:

AI > ML > GenAI > LLM

There are many non-LLM, generative models. For example,

• Markov Chains are generative.
• RNNs are generative.
• Autoencoders are generative.
• Generative Adversarial Networks (GANs) are generative.

So I encourage you to distinguish between GenAI and LLMs, because GenAI encompasses many more ideas. In particular, AIoT models can be fully generative without using any LLMs.

Subscribe if you like this content.