Learnings and Thoughts from Andrew Ng’s New Course
Deep Learning AI has recently launched a new ChatGPT Prompt Engineering for Developers course led by Isa Fulford and Andrew Ng. It’s a free 1.5-hour short course and this course is amazing. In this article, I’d like to discuss these two parts:
- Part 1: Course summary
- Part 2: My thoughts about the best practices in prompt engineering with 🦜LangChain and various OpenAI tips and tricks.
This course includes three parts: two prompt principles, an iterative development process, and capabilities including summarizing, inferring, transforming, expanding, and building a chatbot.
1. Two Principles
Principle 1: Write clear and specific instructions
- Tactic 1: Use delimiters like “`, “““, < >,
text = f”””
You should express what you want a model to do by
providing instructions that are as clear and
specific as you can possibly make them.
This will guide the model towards the desired output,
and reduce the chances of receiving irrelevant
or incorrect responses. Don’t confuse writing a
clear prompt with writing a short prompt.
In many cases, longer prompts provide more clarity
and context for the model, which can lead to
more detailed and relevant outputs.
“””
prompt = f”””
Summarize the text delimited by triple backticks
into a single sentence.
“`{text}“`
“””
- Tactic 2: Ask for structured output. For example, we can the output to be in a JSON format, which we can later easily read into a list or a dictionary in Python.
- Tactic 3: Check whether conditions are satisfied. We can ask in the prompt to check assumptions first. It could also be helpful to think about edges and how models should handle them. In this example, the text doesn’t contain instructions, we gave the instruction for it to write “No steps provided”.
- Tactic 4: few-shot prompting. We give successful examples of completing tasks and then ask the model to perform the task.
Principle 2: Give the model time to “think”
- Tactic 1: specify the steps required to complete a task and ask for output in a specific format. Sometimes it’s difficult for the models or the humans to come to an answer directly. For complicated tasks, step-by-step instructions are often helpful. Similar to how humans work, we can request the model to have a chain or a series of relevant reasoning before the model provides its final answer.
- Tactic 2: instruct the model to work out its own solution before rushing to a conclusion.
2. Iterative Prompt Development
The iterative prompt development process is very similar to how we code. We try something and if it doesn’t work, we refine and retry:
- try something
- analyze where the result does not give what you want
- clarify instructions, give more time to think
- refine prompts with a batch of examples
- repeat
In the course example, Andrew walked through an example to generate marketing copy from a product fact sheet. He iteratively uncover and solved these three issues with refined prompts at each step.
- Issue 1: The text is too long -> Solution: “Use at most 50 words”.
- Issue 2. Text focuses on the wrong details -> Solution: add intended audiences “The description is intended for furniture retailers…”
- Issue 3. Description needs a table of dimensions -> Solution:
“Format everything as HTML”
3. Capabilities
- Summarizing: many people have used Large Language Models to summarize texts. You can specify your prompt to summarize the text with a specific focus for example on price and value:
prompt = f”””
Your task is to generate a short summary of a product
review from an ecommerce site to give feedback to the
pricing deparmtment, responsible for determining the
price of the product.
Summarize the review below, delimited by triple
backticks, in at most 30 words, and focusing on any aspects
that are relevant to the price and perceived value.
Review: “`{prod_review}“`
“””
Of course, you can write a for loop to summarize multiple texts:
reviews = [review_1, review_2, review_3, review_4]
for i in range(len(reviews)):
prompt = f”””
Your task is to generate a short summary of a product
review from an ecommerce site.
Summarize the review below, delimited by triple
backticks in at most 20 words.
Review: “`{reviews[i]}“`
“””
response = get_completion(prompt)
print(i, response, “n”)
- Inferring: You can use Large Language Models to infer sentiment, infer emotions, extract product names, extract company names, infer topics, and more. You don’t need to train a model for a specific task anymore, Large Language Models can infer all these things for you without training.
- Transforming: Large Language Models can do text transformation tasks such as language translation, spelling and grammar checking, tone adjustment, and format conversion.
- Expanding: Large Language Models can generate customer service emails that are tailored to each customer’s review:
- Building a chatbot: I’m super grateful that they have chosen to use Panel to build a chatbot!
Panel Chatbot
I have written several Panel blog posts and Panel chatbots. Please check out my previous blog posts on this topic:
Building a Question Answering PDF Chatbot: LangChain + OpenAI + Panel + HuggingFace
How to Make an AI Image Editing Chatbot: Stable Diffusion InstructPix2Pix in a Panel app
How to deploy a Panel app to Hugging Face using Docker
ChatGPT and DALL·E 2 in a Panel App
How to Deploy a Panel Visualization Dashboard to GitHub Pages
This is a great course introducing many best practices and capabilities with ChatGPT prompt engineering. I especially like the two guiding principles. There are many other interesting issues remaining like how to deal with long tokens, how to use LLMs with other tools, how to handle rate limits, how to stream completions, and more. Building on top of this amazing course, I’d like to expand with two areas of thought: one is LangChain, and another is OpenAI tips and tricks.
1. 🦜 LangChain
Do you have trouble getting started writing clear and specific instructions? LangChain provides many prompt templates for you to use. You don’t need to write instructions from scratch every time.
Would you like get more structured information than just text back?LangChain provides output parsers to help structure language model responses.
Is your text exceed token limits? For example, if you would like to summarize or ask questions about a 500-page book. What do you do? With map_reduce, refine, map-rerank, LangChain allows you to separate text into batches and work through each batch:
- map_reduce: It separates texts into batches, feeds each batch with the question to LLM separately, and comes up with the final answer based on the answers from each batch.
- refine : It separates texts into batches, feeds the first batch to LLM, and feeds the answer and the second batch to LLM. It refines the answer by going through all the batches.
- map-rerank: It separates texts into batches, feeds each batch to LLM, returns a score of how fully it answers the question, and comes up with the final answer based on the high-scored answers from each batch.
Would you like to keep chat histories? LangChain solves this problem by providing several different options for dealing with chat history — keep all conversations, keep the latest k conversations, summarize the conversation, and a combination of the above.
Would you like to use an LLM with another LLM or other tools? LangChain can chain various LLMs together and use LLMs with a suite of tools like Google Search, Python REPL, and more.
Would you like to ask the prompt to automatically write prompts, i.e., auto-GPT? LangChain has implementations on “Westworld” simulation, Camel, BabyAGI, and AutoGPT. Check out my previous blog post 4 Autonomous AI Agents you need to Know.
To learn how LangChain works, check out my previous blog post and my video:
2. OpenAI tips and tricks
The OpenAI Cookbook provides many useful tips and tricks for us to use.
How to avoid rate limit errors? You can retrying with exponential backoff. Check examples here.
Retrying with exponential backoff means performing a short sleep when a rate limit error is hit, then retrying the unsuccessful request. If the request is still unsuccessful, the sleep length is increased and the process is repeated. This continues until the request is successful or until a maximum number of retries is reached.
How to maximize throughput of batch processing given rate limits? When processing large volumes of batch data, two methods 1) proactively adding delay between requests and 2) batching requests by passing in a list of strings to prompt. Check examples here.
How to stream completions? Simply set stream=True to stream completions. Check examples here.
By default, when you request a completion from the OpenAI, the entire completion is generated before being sent back in a single response. If you’re generating long completions, waiting for the response can take many seconds. To get responses sooner, you can ‘stream’ the completion as it’s being generated. This allows you to start printing or processing the beginning of the completion before the full completion is finished.
In this article, I provide a summary of the ChatGPT Prompt Engineering for Developers course. Additionally, I shared my thoughts around the prompt engineering best practices, including the use of 🦜LangChain and a few tips and tricks from OpenAI. I hope you find this article helpful! Feel free to share any other best practices for prompt engineering that you have come across.
Photo by Eric Krull on Unsplash
. . .
By Sophia Yang on April 30, 2023
Sophia Yang is a Senior Data Scientist. Connect with me on LinkedIn, Twitter, and YouTube and join the DS/ML Book Club ️
