Introduction to Generative AI - Andrew NG

 

What is Generative AI

AI systems that can generate high quality content, specifically text, images and video and audio. Best example is OpenAI's ChatGPT.

What is Generative AI

Supervised Learning (Labeling Things)

They are trained to repeatedly predict the next word.

LLMs as a thought partner

AI as a general purpose technology

• AI is a general purpose technology - it's useful for a lot of things. (Like electricity)
• Internet is another general purpose technology.

• App --- LLM that will be better off built into an automated software system.
• Web --- Web interface

Generative AI Applications

• Use as brainstorming partner
• For writing - example, write a press release.
• Translation - example translate the following into Hindi / formal Hindi / formal spoken Hindi
• Translate text into Pirate English for teseting purposes. 
• Proofreading
• Summarizing a long article.

• Summarizing call centre conversations. (Record --> Run Phone calls --> Speech Recognition --> Long Test Transcripts --> LLM -- Summarise converstaion --> Generate short summary --> spot issues or trends (Manager)

• Customer email analysis: 
• Complaint?
• Route this email to a particular department

• Reputation monitoring --- sentiment analysis (positive or negative)

• Can build an Alert System which can alert you if sentiment is dropping negative. 

Chatting

• Specialized chatbots
• Customer service chatbot
• Trip planning chatbot
• Advise bots - career coaching, cooking a meal, etc.
• Some can ACT if sufficiently empowered.

Chatbots in Customer Services

Deploying chatbots

> First deploy internal-facing chatbots that chat with internal users only.

> When sufficiently confident of their performance, move over to human-in-the-loop where a human will validate the popped up message for certain action.

> Once both the above are confidently proven (Only if deemed safe, allow the bot to communicate directly with customers), you can deploy them into production environment. (Above two ensures that you will avoid public mistakes that could prove to be costly. 

Using Generative AI in Software Applications

Before the rise of LLMs, some of the software existed (as below), which were trained using supervised learning. The whole process took months. With the advent of LLMs, the entire timeline has been squeezed and same functionality can be achieved within weeks or days even.

Example - reading restaurant reviews for reputation monitoring, the code used to read as below.

A few hundred or thousands of data points as these

Same using a prompt engineering is below - you simply pass the string to an LLM and get back the result. 

Trying Generative AI Code

Building a Reputation Monitoring System

What are are trying to achieve:

1. We will set up the environment before progressing.

import openai

import os 

openai.api_key = os.getenv("OPENAI_API_KEY")

def llm_response(prompt):

    response = openai.ChatCompletion.create(

        model='gpt-3.5-turbo',

        messages=[{'role':'user','content':prompt}],

        temperature=0

    )

    return response.choices[0].message['content']

Once done, run the above piece of code. There won't be any output, but this will set up the environment variables for our programs to run. 

2. Now, create a list of reviews. 

all_reviews = [

    'The mochi is excellent!',

    'Best soup dumplings I have ever eaten.',

    'Not worth the 3 month wait for a reservation.',

    'The colorful tablecloths made me smile!',

    'The pasta was cold.'

]

all_reviews

When you run the above code, output will be:

['The mochi is excellent!',
'Best soup dumplings I have ever eaten.',
'Not worth the 3 month wait for a reservation.',
'The colorful tablecloths made me smile!',
'The pasta was cold.']

3. Write a code to classify the above reviews as poistive or negative.

all_sentiments = []
for review in all_reviews:
prompt = f'''
Classify the following review
as having either a positive or
negative sentiment. State your answer
as a single word, either "positive" or
"negative":
{review}
'''
response = llm_response(prompt)
all_sentiments.append(response)
all_sentiments
Output will be ['positive', 'positive', 'negative', 'positive', 'negative']

4. Write code to count the number of positive and number of negative reviews.

num_positive = 0
num_negative = 0
for sentiment in all_sentiments:
if sentiment == 'positive':
num_positive += 1
elif sentiment == 'negative':
num_negative += 1
print(f"There are {num_positive} positive and {num_negative} negative reviews.")
Output is: There are 3 positive and 2 negative reviews.

Lifecycle of a Generative AI Project

The above output is wrong because tonkotsu ramen is pork soup, and the above response is not considered positive.

Now feedback to the system to improve

Tools to improve performance

Prompting is a higly empirical process
Idea -- Prompt -- LLM Response -- Tweak Idea and the cycle repeats
RAG - Retrieval Augmented Generation (RAG)
Give LLM access to external data sources.
Fine tune models
Adapt LLMs to your task.
Pretrain models
Train LLMs from scratch
Example

Above, where the calorific value is asked, you can go over to RAG.

How much do LLMs Cost?


A Token is a word or a sub part of a word. 





300 words == 400 tokens

number of tokens is loosely about number of words.



Retrieval Augment Generation (RAG)


RAG will refer to additional inputs from your company to answer the question above.





Example of RAG
This is called Retrieval Augmented Generation or RAG, because we're going to generate an answer to this, but we're going to augment how we generate text by retrieving the relevant context or the relevant information and augmenting the prompt with that additional text.

> AI Bots that allow you to chat with PDFs

> AIs that allow you to get answers for questions on website's articles

> New form of search on web

LLM as a Reasoning Engine

Pre-training and Fine Tuning AI

> To carry out a task that is difficult to define in a prompt. 

> LLMs are fine tuned to gain speicfic knowledge. 

Example medical notes




Legal notes