At the time of writing, there is a time-limited free course about ChatGPT prompt engineering for developers, shared by deeplearning.ai.
Introduction
Two types of LLMs:
-
Base LLM: predicts next word, based on test training data, not able to answer user questions.
-
Instruction Tuned LLM: tries to follow instruction, fine tune on instructions and good attempt at following those instructions.
RLHF(refinement learning with human feedback).
The course is focusing on the second type.
For how to write openai code, the Python library can be found here: https://platform.openai.com/docs/libraries
The boilerplate:
1 | import openai |
Here is the openai chat completion guide: https://platform.openai.com/docs/guides/chat
Guidelines
Prompting principles and tactics:
-
Use clear and specific instructions.
-
Use delimiters to avoid prompt injection.
- Such as:
""",---or<tag>,<>, etc.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20text = f"""
You should ```express what you want``` a model to do by \
providing instructions that are as clear and \
specific as you can possibly make them. \
This will guide the model towards the desired output, \
and reduce the chances of receiving irrelevant \
or incorrect responses. Don't confuse writing a \
clear prompt with writing a short prompt. \
In many cases, longer prompts provide more clarity \
and context for the model, which can lead to \
more detailed and relevant outputs.
"""
prompt = f"""
Summarize the text delimited by single backtick \
into a single sentence.
`{text}`
"""
response = perform_task(prompt)
print(response) - Such as:
-
Ask for structed output.
- JSON, YAML, etc, so the they can be used later.
1
2
3
4
5
6
7
8prompt = f"""
Generate a list of three made-up book titles along \
with their authors and genres.
Provide them in JSON format with the following keys:
book_id, title, author, genre.
"""
response = perform_task(prompt)
print(response) -
Ask the model to check whether conditions are satisfied.
- For example, does the input contain a sequence of instructions?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29text_1 = f"""
Making a cup of tea is easy! First, you need to get some \
water boiling. While that's happening, \
grab a cup and put a tea bag in it. Once the water is \
hot enough, just pour it over the tea bag. \
Let it sit for a bit so the tea can steep. After a \
few minutes, take out the tea bag. If you \
like, you can add some sugar or milk to taste. \
And that's it! You've got yourself a delicious \
cup of tea to enjoy.
"""
prompt = f"""
You will be provided with text delimited by triple quotes.
If it contains a sequence of instructions, \
re-write those instructions in the following format:
Step 1 - ...
Step 2 - …
…
Step N - …
If the text does not contain a sequence of instructions, \
then simply write \"No steps provided.\"
\"\"\"{text_1}\"\"\"
"""
response = perform_task(prompt)
print("Completion for Text 1:")
print(response) -
“Few-shot” prompting.
- Give successful example of completing tasks then ask model to perform the task.
1
2
3
4
5
6
7
8
9
10
11
12
13
14prompt = f"""
Your task is to answer in a consistent style.
<child>: Teach me about patience.
<grandparent>: The river that carves the deepest \
valley flows from a modest spring; the \
grandest symphony originates from a single note; \
the most intricate tapestry begins with a solitary thread.
<child>: Teach me about resilience.
"""
response = perform_task(prompt)
print(response)
-
-
Give the model time to “think”.
- Specify the steps required to complete a task.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29text = f"""
In a charming village, siblings Jack and Jill set out on \
a quest to fetch water from a hilltop \
well. As they climbed, singing joyfully, misfortune \
struck—Jack tripped on a stone and tumbled \
down the hill, with Jill following suit. \
Though slightly battered, the pair returned home to \
comforting embraces. Despite the mishap, \
their adventurous spirits remained undimmed, and they \
continued exploring with delight.
"""
# example 1
prompt_1 = f"""
Perform the following actions:
1 - Summarize the following text delimited by single \
backtick with 1 sentence.
2 - Translate the summary into French.
3 - List each name in the French summary.
4 - Output a json object that contains the following \
keys: french_summary, num_names.
Separate your answers with line breaks.
Text:
`{text}`
"""
response = perform_task(prompt_1)
print("Completion for prompt 1:")
print(response)- Instruct the model to work out its own solution before rushing to a conclusion.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58prompt = f"""
Your task is to determine if the student's solution \
is correct or not.
To solve the problem do the following:
- First, work out your own solution to the problem.
- Then compare your solution to the student's solution \
and evaluate if the student's solution is correct or not.
Don't decide if the student's solution is correct until
you have done the problem yourself.
Use the following format:
Question:
``
question here
``
Student's solution:
``
student's solution here
``
Actual solution:
``
steps to work out the solution and your solution here
``
Is the student's solution the same as actual solution \
just calculated:
``
yes or no
``
Student grade:
``
correct or incorrect
``
Question:
``
I'm building a solar power installation and I need help \
working out the financials.
- Land costs $100 / square foot
- I can buy solar panels for $250 / square foot
- I negotiated a contract for maintenance that will cost \
me a flat $100k per year, and an additional $10 / square \
foot
What is the total cost for the first year of operations \
as a function of the number of square feet.
``
Student's solution:
``
Let x be the size of the installation in square feet.
Costs:
1. Land cost: 100x
2. Solar panel cost: 250x
3. Maintenance cost: 100,000 + 100x
Total cost: 100x + 250x + 100,000 + 100x = 450x + 100,000
``
Actual solution:
"""
response = perform_task(prompt)
print(response)
NOTE: it is possible that the answer sounds plausible but are not true.
Reduce the hallucinations: Ask the LLM first find relevant information, then answer the question based on the relevant information.
Iterative
Iterative prompt development steps:
- Try something.
- Analyze where the result does not give what you want.
- Clarify instructions, give more time to think.
- Refine prompt with a batch of examples.
For example, below examples show how to evolve the prompt to get desired outcome for a e-commerce website:
1 | fact_sheet_chair = """ |
Original prompt:
1 | prompt = f""" |
If the output is too long, you can limit the length explicitly:
1 | prompt = f""" |
After many rounds of improvements, the last prompt gets specified details and table/HTML format:
1 | prompt = f""" |
Summarizing
For example, the text to be summarized:
1 | prod_review = """ |
You can ask LLM like this:
1 | prompt = f""" |
Focusing on shipping and delivery details:
1 | prompt = f""" |
Focusing on price and value:
1 | prompt = f""" |
If the summaries are not relevant to focus, you can adjust the word by using
extract instead of summarize:
1 | prompt = f""" |
To summarize multiple payloads:
1 | # reviews contains multiple texts |
Inferring
For example, extract sentiment(positive or negative), LLMs are pretty good at extracting these.
For example giving the customer review:
1 | lamp_review = """ |
You can do multiple tasks at once:
1 | prompt = f""" |
You can also infer topics:
1 | prompt = f""" |
And you can have a list of keywords and check if the text to be verified is in one of them:
1 | # topic_list contains a list of key words to be identified. |
You can output as JSON format to further process.
Transforming
Such as language translation, spelling and grammar checking, tone adjustment, and format conversion.
For example, a universal translator:
1 | user_messages = [ |
Tone transformation, for informal to formal:
1 | prompt = f""" |
Spelling and grammar check, to signal to the LLM that you want it to proofread your text, you instruct the model to ‘proofread’ or ‘proofread and correct’.
1 | text = [ |
Another example, proofread and correct long text:
1 | text = f""" |
Expanding
Expand short text to long, for example, generate customer service emails that are tailored to each customer’s review.
1 | # Assume the 'review' and 'sentiment' are ready to use. |
About the temperature parameter, uses 0 if you require reliability and
predictability, use high value if requires variety.
Charbot
Utilizing the chat format to have extended conversations with chatbots personalized or specialized for specific tasks or behaviors.
Basically, you need to provide the complete context for each conversation, for
example, by appending the messages(from user and assistant) to a list, and
dumping the list to LLM for response.
1 | # The 'system' role is used to provide overall context and details. |
And you can summarize the chat session, for example:
1 | messages = context.copy() |