This course material focuses on prompt engineering, a technique for effectively interacting with large language models (LLMs) like ChatGPT. It explores various prompt patterns and strategies to achieve specific outputs, including techniques for refining prompts, providing context, and incorporating information LLMs may lack. The course emphasizes iterative refinement through conversation with the LLM, treating the prompt as a tool for problem-solving and creativity. Instruction includes leveraging few-shot examples to teach LLMs new tasks and techniques for evaluating and improving prompt effectiveness. Finally, it introduces methods for integrating LLMs with external tools and managing the limitations of prompt size and LLM capabilities.

Prompt Engineering Study Guide

Quiz

Instructions: Answer the following questions in 2-3 sentences each.

1. According to the speaker, what is a primary misconception about tools like ChatGPT?
2. In the speaker’s example, what was the initial problem he used ChatGPT to solve?
3. How did the speaker modify the initial meal plan created by ChatGPT?
4. What method did the speaker use to attempt to get his son interested in the meal plan?
5. Besides meal planning and stories, what other element was added to this interactive experiment?
6. What does it mean to say that large language models do “next word prediction”?
7. Explain the difference between a prompt as a verb and a prompt as an adjective in the context of large language models.
8. How can a prompt’s effects span time?
9. How can patterns within prompts influence the responses of large language models?
10. What is the main idea behind using “few-shot” examples in prompting?

Answer Key

1. The primary misconception is that these tools are solely for writing essays or answering questions. The speaker argues that this misunderstands the true potential, which is to give form to ideas, explore concepts, and refine thoughts.
2. The speaker wanted to create a keto-friendly meal plan that was a fusion of Uzbekistani and Ethiopian cuisine, using ingredients easily found in a typical US grocery store.
3. He modified the meal plan by asking for approximate serving sizes for each dish to fit within a 2,000-calorie daily limit.
4. He created short Pokémon battle stories with cliffhangers to engage his son’s interest and encourage him to try the new food.
5. In addition to meal plans and stories, the speaker incorporated a math game focused on division with fractions related to nutrition and the Pokémon theme.
6. Large language models work by predicting the next word or token in a sequence based on the prompt and the patterns they have learned from training data. They generate output word by word based on these predictions.
7. As a verb, a prompt is a call to action, causing the language model to begin generating output. As an adjective, a prompt describes something that is done without delay or on time, indicating the immediacy of the model’s response.
8. Prompts can have effects that span time by setting rules or contexts that the language model will remember and apply to future interactions. For example, setting a rule that the language model must ask for a better version of every question before answering it will apply throughout a conversation.
9. Strong patterns in prompts can lead to consistent and predictable responses, as the language model will recognize and draw from patterns in its training data. Weaker patterns can rely more on specific words, and will result in more varied outputs, since the model is not immediately aware of which patterns to apply.
10. “Few-shot” examples provide a language model with input/output pairs that demonstrate how to perform a desired task. This allows it to understand and apply the pattern to new inputs, without needing explicit instruction.

Essay Questions

1. Discuss the speaker’s approach to using ChatGPT as a creative tool rather than simply a question-answering system. How does the speaker’s use of the tool reveal an understanding of its capabilities?
2. Describe and analyze the key elements of effective prompt engineering that are highlighted by the speaker’s various experiments. How does the speaker’s approach help to illustrate effective methods?
3. Explain the role of pattern recognition in how large language models respond to prompts. Use examples from the speaker’s analysis to support your argument.
4. Compare and contrast the different prompt patterns explored by the speaker, such as the Persona pattern, the Few Shot example pattern, the Tail Generation Pattern, and the Cognitive Verifier pattern. How do these different prompt patterns help us to make the most of large language model capabilities?
5. Synthesize the speaker’s discussion to create a guide for users on how to best interact with and refine their prompts when using a large language model. What are the most important lessons you have learned?

Glossary

Large Language Model (LLM): A type of artificial intelligence model trained on massive amounts of text data to generate human-like text. Tools like ChatGPT are examples of LLMs.

Prompt: A text input provided to a large language model to elicit a specific response. Prompts can range from simple questions to complex instructions.

Prompt Engineering: The art and science of designing effective prompts to achieve desired outcomes from large language models. It involves understanding how LLMs interpret language and structure responses.

Next Word Prediction: The core process by which large language models generate text, predicting the most likely next word or token in a sequence based on the preceding input.

Few-Shot Examples: A technique for prompting a large language model by providing a few examples of inputs and their corresponding outputs, enabling it to perform similar tasks with new inputs.

Persona Pattern: A technique in prompt engineering where you direct a large language model to act as a particular character or entity (e.g., a skeptic, a scientist) to shape its responses.

Audience Persona Pattern: A technique in prompt engineering where the prompt defines who the intended audience is, so the LLM can tailor output.

Tail Generation Pattern: A prompt that includes an instruction or reminder at the end, which causes that text to be appended to all responses, and can also include rules of the conversation.

Cognitive Verifier Pattern: A technique that instructs the model to first break down the question or problem into sub-questions or sub-problems, then to combine the answers into a final overall answer.

Outline Expansion Pattern: A technique where a prompt is structured around an outline that the LLM can generate and then expand upon, focusing the conversation and making it easier to fit together the different parts of the output.

Menu Actions Pattern: A technique in prompt engineering where you define a set of actions (a menu of instructions) that you can trigger, by name, in later interactions with the LLM, thus setting up an operational mode for the conversation.

Metal Language Creation Pattern: A technique in prompt engineering that lets you define or explain a new language or shorthand notation to an LLM, which it will use to interpret prompts moving forward in the conversation.

Recipe Pattern: A technique in prompt engineering where the prompt contains placeholders for elements you want the LLM to fill in, to generate complete output. This pattern is often used to complete steps of a process or itinerary.

Prompt Engineering with Large Language Models

Okay, here is a detailed briefing document reviewing the main themes and most important ideas from the provided sources.

Briefing Document: Prompt Engineering and Large Language Models

Overall Theme: The provided text is an introductory course on prompt engineering for large language models (LLMs), with a focus on how to effectively interact with and leverage the power of tools like ChatGPT. The course emphasizes shifting perspective on LLMs from simple question-answering tools to creative partners that can rapidly prototype and give form to complex ideas. The text also dives into the technical aspects of how LLMs function, the importance of pattern recognition, and provides actionable strategies for prompt design through various patterns.

Key Concepts and Ideas:

• LLMs as Tools for Creativity & Prototyping:The course challenges the perception of LLMs as mere essay writers or exam cheaters. Instead, they should be viewed as tools that unlock creativity and allow for rapid prototyping.
• Quote: “I don’t want you to think of these tools as something that you use to um just write essays or answer questions that’s really missing the capabilities of the tools these are tools that really allow you to do fascinating um things… these are tools that allow me to do things faster and better than I could before.”
• The instructor uses an example of creating a complex meal plan, complete with stories and math games for his son, to showcase the versatile capabilities of LLMs.
• Prompt Engineering:The course focuses on “prompt engineering” which is the art and science of crafting inputs to LLMs to achieve the desired output.
• A prompt is more than just a question; it’s a “call to action” that initiates output, can span time, and may affect future responses.
• Quote: “Part of what a prompt is it is a call to action to the large language model. It is something that is getting the large language model to start um generating output for us.”
• Prompts can be immediate, affecting an instant response, or can create rules that affect future interactions.
• How LLMs Work:LLMs operate by predicting the next word in a sequence, based on the training data they’ve been exposed to.
• LLMs are based on next-word prediction, completing text based on patterns identified from training data.
• Quote: “…your prompt is they’re just going to try to generate word by word the next um um word that’s going to be in the output until it gets to a point that it thinks it’s ated enough…”
• This involves recognizing and leveraging patterns within the prompt to get specific and consistent results.
• The Importance of Patterns:Strong patterns within prompts trigger specific responses due to the large amount of times those patterns have been seen in the training data.
• Quote: “if we know the right pattern if we can tap into things that the the model has been trained on and seen over and over and over again we’ll be more likely to to not only get a consistent response…”
• Specific words can act as “strong patterns” that influence the output, but patterns themselves play a more powerful role than just individual words.
• Iterative Refinement & Conversations:Prompt engineering should be viewed as an iterative process rather than a one-shot interaction.
• The most effective use of LLMs involves having a conversation with the model, using the output of each prompt to inform the next.
• Quote: “a lot of what we need to do with large language models is think in that Mo in that mindset of it’s not about getting the perfect answer right now from this prompt it’s about going through an entire conversation with the large language model that may involving a series of prompts…”
• The conversation style interaction allows you to explore and gradually refine the output toward your objective.
• Prompt Patterns: The text introduces several “prompt patterns,” which are reusable strategies for interacting with LLMs:
• Persona Pattern: Telling the LLM to act “as” a particular persona (e.g., a skeptic, a computer, or a character) to shape the tone and style of the output.
• Audience Persona Pattern: Instructing the LLM to produce output for a specific audience persona, tailoring the content to the intended recipient.
• Flipped Interaction Pattern: Having the LLM ask you questions until it has enough information to complete a task, instead of you providing all the details upfront.
• Few-Shot Examples: Providing the LLM with examples of how to perform a task to guide the output. Care must be taken to provide meaningful examples that are specific and detailed, and give the LLM enough context to complete the given task.
• Chain of Thought Prompting: Provides reasoning behind the examples and requests the model to think through its reasoning process, resulting in more accurate answers for more complex questions.
• Grading Pattern: Uses the LLM to grade a task output based on defined criteria and guidelines.
• Template Pattern: Utilizing placeholders in a structured output to control content and formatting.
• Meta-Language Creation Pattern: Teaching the LLM a shorthand notation to accomplish tasks, and have the language model work within this custom language.
• Recipe Pattern: Provide the LLM a goal to accomplish along with key pieces of information to include in the result. The LLM then fills in the missing steps to complete the recipe.
• Outline Expansion Pattern: Start with an outline of the desired topic and expand different sections of the outline to generate more detailed content and organize the content of the prompt.
• Menu Actions Pattern: Defining a set of actions (like commands on a menu) that the LLM can perform to facilitate complex or repeating interactions within the conversation.
• Tail Generation Pattern: Instruct the LLM to include specific output at the end of its response, to facilitate further interactions.
• Cognitive Verifier Pattern: Instruct the LLM to break a question or problem into smaller pieces to facilitate better analysis.
• Important Considerations:LLMs are limited by the data they were trained on.
• LLMs can sometimes create errors.
• It’s important to fact-check and verify the output provided by LLMs.
• Users must be cognizant of sending data to servers and ensure that they are comfortable doing so, particularly when private information is involved.
• When building tools around LLMs, you can use root prompts to affect subsequent conversations.

Conclusion:

The material presents a comprehensive introduction to the field of prompt engineering, emphasizing the importance of understanding how LLMs function to take full advantage of their capabilities. The course underscores the necessity of shifting mindset from passive user to active designer in the user experience of the LLM. By providing a series of practical patterns and examples, it empowers users to rapidly prototype ideas, refine outputs, and create a more interactive and creative dialogue with LLMs. The course also emphasizes the need for careful use, as with any powerful tool, underscoring the need for ethical and responsible use of LLMs.

Prompt Engineering with Large Language Models

What is prompt engineering and why is it important?

Prompt engineering is the process of designing effective inputs, or prompts, for large language models (LLMs) to elicit desired outputs. It is important because the quality of a prompt greatly influences the quality and relevance of the LLM’s response. Well-crafted prompts can unlock the LLMs potential for creativity, problem-solving, and information generation, whereas poorly designed prompts can lead to inaccurate, unhelpful, or undesirable outputs. It’s crucial to understand that these models are fundamentally predicting the next word based on patterns they have learned from massive datasets, and prompt engineering allows us to guide this process.

How can large language models like ChatGPT be used as more than just question answering tools?

Large language models are incredibly versatile tools that go far beyond simple question answering. They can be used to prototype ideas, explore different concepts, refine thoughts, generate creative content, act as different personas or tools, and even write code. For example, in one case, ChatGPT was used to create a keto-friendly meal plan fusing Ethiopian and Uzbek cuisine, provide serving sizes, develop Pokemon battle stories with cliffhangers for a child, create a math game related to the meal plan for the child, and then generate code for the math game in the form of a web application. This demonstrates the capacity for LLMs to be used as dynamic, interactive partners in the creative and problem-solving processes, rather than static repositories of information.

What are the key components of an effective prompt?

Effective prompts involve several dimensions, including not only the immediate question but also a call to action, an implied time element, and the context that the LLM is operating under. A prompt is not just a simple question, but a method of eliciting an output. This might involve having a goal the model should always keep in mind, or setting up constraints. Additionally, effective prompts include clear instructions on the desired format of the output, and might involve defining the role the LLM should adopt, or the persona of the intended audience. Well-defined prompts tap into patterns the model was trained on, which increase consistency and predictability of output.

How do prompts tap into the patterns that large language models were trained on?

LLMs are trained on massive datasets and learn to predict the next word in a sequence based on these patterns. When we craft prompts, we’re often tapping into patterns that the model has seen many times in its training data. The more strongly a pattern in your prompt resonates with the training data the more consistent a response will be. For example, the phrase “Mary had a little” triggers a very specific pattern in the model, resulting in a consistent continuation of the nursery rhyme. In contrast, more novel patterns require more specific words to shape the output, due to weaker patterns of the prompt itself, even though specific words themselves can be tied to various patterns. Understanding how specific words and overall patterns influence outputs is critical to effective prompt engineering.

What is the persona pattern, and how does it affect the output of an LLM?

The persona pattern involves instructing the LLM to “act as” a specific person, role, or even an inanimate object. This triggers the LLM to generate output consistent with the known attributes and characteristics of that persona. For example, using “act as a skeptic” can cause the LLM to generate skeptical opinions. Similarly, “act as the Linux terminal for a computer that has been hacked” elicits a computer terminal-like output, using commands a terminal would respond to. This pattern allows users to tailor the LLM’s tone, style, and the type of content it generates, without having to provide detailed instructions, as the LLM leverages its pre-existing knowledge of the persona. This shows that a prompt is often not just about the question, it’s about the approach or character.

How does a conversational approach to prompt engineering help generate better outputs?

Instead of a one-off question-and-answer approach, a conversational prompt engineering approach treats the LLM like a collaborative partner, using iterative refinement and feedback to achieve a desired outcome. In this case, the user interacts with the LLM over multiple turns of conversation, using the output from one prompt to inform the subsequent prompt. By progressively working through the details of the task or problem at hand, the user can guide the LLM to create more relevant, higher-quality outputs, such as designing a robot from scratch through several turns of discussion and brainstorming. The conversation helps refine both the LLM’s output and the user’s understanding of the problem.

How can “few-shot” learning be used to teach an LLM a specific task?

Few-shot learning involves giving an LLM a few examples of inputs and their corresponding outputs, which enable it to understand and apply a pattern to new inputs. For example, providing a few examples of text snippets paired with a sentiment label can teach an LLM to perform sentiment analysis on new text. Few-shot learning shows the model what is expected without specifying a lot of complicated instructions, teaching through demonstrated examples instead. Providing a few correct and incorrect examples can be helpful to further specify output expectations.

What are some advanced prompting patterns, such as the cognitive verifier, the template pattern, and metalanguage creation?

Several advanced patterns further demonstrate the power of prompt engineering. The cognitive verifier instructs the LLM to break down a complex problem into smaller questions before attempting a final answer. The template pattern involves using placeholders to structure output into specific formats, which might use semantically rich terms. The metalanguage creation pattern allows users to create a new shorthand or language, then use that newly created language with the LLM. These patterns enable users to use the LLMs in more dynamic and creative ways, and build prompts that are very useful for solving complex problems. There are a variety of advanced prompting patterns which provide a range of approaches to solving problems, based on a particular technique.

Prompt Engineering with LLMs

Prompt engineering is a field focused on creating effective prompts to interact with large language models (LLMs) like ChatGPT, to produce high-quality outputs [1, 2]. It involves understanding how to write prompts that can program these models to perform various tasks [2, 3].

Key concepts in prompt engineering include:

• Understanding Prompts: A prompt is more than just a question; it is a call to action that encourages the LLM to generate output in different forms, such as text, code, or structured data [4]. Prompts can have a time dimension and can affect the LLM’s behavior in the present and future [5, 6].
• Prompt Patterns: These are ways to structure phrases and statements in a prompt to solve particular problems with an LLM [7, 8]. Patterns tap into the LLM’s training, making it more likely to produce desired behavior [9]. Examples of patterns include the persona pattern [7], question refinement [7, 10], and the use of few-shot examples [7, 11].
• Specificity and Context: Providing specific words and context in a prompt helps elicit a targeted output [12]. LLMs are not mind readers, so clear instructions are crucial [12].
• Iterative Refinement: Prompt engineering is an iterative process, where you refine your prompts through a series of conversations with the LLM [13, 14].
• Programming with Prompts: Prompts can be used to program LLMs by giving them rules and instructions [15]. By providing a series of instructions, you can build up a program that the LLM follows [8, 16].
• Limitations: There are limits on the amount of information that can be included in a prompt [17]. Therefore, it’s important to select and use only the necessary information [17]. LLMs also have inherent randomness, meaning they may not produce the same output every time [18, 19]. They are trained on a vast amount of data up to a certain cut-off date, so new information must be provided as part of the prompt [20].
• Root Prompts: Some tools have root prompts that are hidden from the user that provide rules and boundaries for the interaction with the LLM [21]. These root prompts can be overridden by a user [22, 23].
• Evaluation: Large language models can be used to evaluate other models or their own outputs [24]. This can help ensure that the output is high quality and consistent with the desired results [25].
• Experimentation: It is important to be open to experimentation, creativity, and trying out different things to find the best ways to use LLMs [3].
• Prompt Engineering as a Game: You can create a game using a LLM to improve your own skills [26]. By giving the LLM rules for the game you can have it generate tasks that can be accomplished through prompting [26].
• Chain of Thought Prompting: This is a technique that can be used to get better reasoning from a LLM by explaining the reasoning behind the examples [27, 28].
• Tools: Prompts can be used to help a LLM to access and use external tools [29].
• Combining Patterns: You can apply multiple patterns together to create sophisticated prompts [30].
• Outlines: You can use the outline pattern to rapidly create a sophisticated outline by starting with a high-level outline and then expanding sections of the outline in turn [31].
• Menu Actions: The menu actions pattern allows you to develop a series of actions within a prompt that you can trigger [32].
• Tail Generation: The tail generation pattern can be used to remind the LLM of rules and maintain the rules of conversation [33].

Ultimately, prompt engineering is about leveraging the power of LLMs to unlock human creativity and enable users to express themselves and explore new ideas [1, 2]. It is an evolving field and so staying up to date with the latest research and collaborating with others is important [34].

Large Language Models: Capabilities and Limitations

Large language models (LLMs) are a type of computer program designed to understand and generate human language [1]. They are trained on vast amounts of text data from the internet [2]. These models learn patterns in language, allowing them to predict the next word in a sequence, and generate coherent and contextually relevant text [2-4].

Here are some key aspects of how LLMs work and their capabilities:

• Training: LLMs are trained by being given a series of words and predicting the next word in the sequence [2]. When the prediction is wrong, the model is tweaked [2]. This process is repeated over and over again with large datasets [2].
• Word Prediction: The fundamental thing that LLMs do is take an input and try to generate the next word [3]. They then add that word to the input and try to predict the next word, continuing the process to form sentences and paragraphs [3].
• Context: LLMs pay attention to the words, relationships, and context of the text to predict the next word [2]. This allows them to learn patterns in language [2].
• Capabilities: LLMs can be used for various tasks such as:
• Text generation [5-8].
• Programming [5, 6].
• Creative writing [5, 6].
• Art creation [5, 6].
• Knowledge exploration [6, 9].
• Prototyping [6, 9].
• Content production [6, 9].
• Assessment [6, 9].
• Reasoning [10, 11].
• Summarization [12-14].
• Translation [1].
• Sentiment analysis [15].
• Planning [16].
• Use of external tools [17].
• Prompt interaction: LLMs require a prompt to initiate output. A prompt is more than just a question it is a call to action for the LLM [7]. Prompts can be used to program the LLM by providing rules and instructions [18].
• Randomness and Unpredictability: LLMs have some degree of randomness which can lead to variations in output even with the same prompt [10]. This can be good for creative tasks, but it requires careful prompt engineering to control when a specific output is needed [10].
• Limitations: LLMs have limitations such as:
• Cut-off dates: They are trained on data up to a specific cut-off date and do not know what has happened after that date [19, 20].
• Prompt length: There is a limit on how large a prompt can be [21, 22].
• Lack of access to external data: LLMs may not have access to specific data or private information [20].
• Inability to perceive the physical world: They cannot perceive the physical world on their own [20].
• Unpredictability: LLMs have a degree of randomness [10].
• Inability to perform complex computation on their own [17].
• Overcoming limitations:
• Provide new information: New information can be provided to the LLM in the prompt [19, 20].
• Use tools: LLMs can be prompted to use external tools to perform specific tasks [17].
• Use an outline: An outline can be used to plan and organize a large response [23].
• Break down tasks: Problems can be broken down into smaller tasks to improve the LLM’s reasoning ability [11].
• Conversational approach: By engaging in a conversation with the LLM you can iteratively refine a prompt to get the desired output [24].
• Prompt Engineering: This is a crucial skill for interacting with LLMs. It involves creating effective prompts using techniques like [5]:
• Prompt patterns: These are ways of structuring a prompt to elicit specific behavior [9, 12].
• Specificity: Providing specific details in the prompt [25, 26].
• Context: Giving the LLM enough context [25, 26].
• Few-shot examples: Showing the LLM examples of inputs and outputs [15].
• Chain of thought prompting: Explicitly stating the reasoning behind examples [17].
• Providing a Persona: Prompting the LLM to adopt a certain persona [27].
• Defining an audience persona: Defining a specific audience for the output [28].
• Using a meta language: Creating a custom language to communicate with the LLM [29].
• Using recipes: Providing the LLM with partial information or instructions [30].
• Using tail generation: Adding a reminder at the end of each turn of a conversation [31].
• Importance of experimentation: It’s important to experiment with different approaches to understand how LLMs respond and learn how to use them effectively [32].

Prompt Patterns for Large Language Models

Prompt patterns are specific ways to structure phrases and statements in a prompt to solve particular problems with a large language model (LLM) [1, 2]. They are a key aspect of prompt engineering and tap into the LLM’s training data, making it more likely to produce the desired behavior [1-3].

Here are some of the key ideas related to prompt patterns:

• Purpose: Prompt patterns provide a documented way to structure language and wording to achieve a specific behavior or solve a problem when interacting with an LLM [2]. They help elicit a consistent and predictable output from an LLM [2, 4].
• Tapping into training: LLMs are trained to predict the next word based on patterns they’ve learned [3, 5]. By using specific patterns in a prompt, you can tap into these learned associations [2].
• Consistency: When a prompt follows a strong pattern, it is more likely to get a consistent response [3, 6].
• Creativity: Sometimes you want to avoid a strong pattern and use specific words or phrases to break out of a pattern and get more creative output [7].
• Programming: Prompt patterns can be used to essentially program an LLM by giving it rules and instructions [4, 8].
• Flexibility: You can combine multiple patterns together to create sophisticated prompts [9].
• Experimentation: Prompt patterns are not always perfect and you may need to experiment with the wording to find the best pattern for a particular problem [1].

Here are some specific prompt patterns that can be used when interacting with LLMs:

• Persona Pattern: This involves asking the LLM to act as a particular person, object, or system [10-12]. This can be used to tap into a rich understanding of a particular role and get output from that point of view [12]. By giving the LLM a specific persona to adopt, you are giving it a set of rules that it should follow during the interaction [13].
• Audience Persona Pattern: This pattern involves prompting the LLM to produce output for a specific audience or type of person [14].
• Question Refinement Pattern: This pattern involves having the LLM improve or rephrase a question before answering it. [10, 15]. The LLM uses its training to infer better questions and wording [15].
• Few-shot examples or few-shot prompting: This involves giving the LLM examples of the input and the desired output, so it can learn the pattern and apply it to new input [10, 16]. By giving a few examples the LLM can learn a new task. The examples can show intermediate steps to a solution [17].
• Flipped Interaction Pattern: In this pattern, you ask the LLM to ask you questions to get more information on a topic before taking an action [18].
• Template Pattern: This pattern involves giving the LLM a template for its output including placeholders for specific values [19, 20].
• Alternative Approaches Pattern: In this pattern you ask the LLM to suggest multiple ways of accomplishing a task [21-23]. This can be combined with a prompt where you ask the LLM to write prompts for each alternative [21].
• Ask for Input Pattern: This pattern involves adding a statement to a prompt that asks for the first input and prevents the LLM from generating a large amount of output initially [24, 25].
• Outline Expansion Pattern: This involves prompting the LLM to create an outline, and then expanding certain parts of the outline to progressively create a detailed document [26, 27].
• Menu Actions Pattern: This allows you to define a set of actions with a trigger that you can run within a conversation [28, 29]. This allows you to reuse prompts and share prompts with others [29].
• Tail Generation Pattern: This pattern involves having the LLM generate a tail at the end of its output that reminds it what the rules of the game are and provides the context for the next interaction [30-32].

By understanding and applying these prompt patterns, you can improve your ability to interact with LLMs and get the results you are looking for [2, 9, 10].

Few-Shot Learning with Large Language Models

Few-shot examples, also known as few-shot prompting, is a prompt pattern that involves providing a large language model (LLM) with a few examples of the input and the corresponding desired output [1, 2]. By showing the LLM a few examples, you are essentially teaching it a new task or pattern [1]. Instead of explicitly describing the steps the LLM needs to take, you demonstrate the desired behavior through examples [1]. The goal is for the LLM to learn from these examples and apply the learned pattern to new, unseen inputs [1].

Here are some key aspects of using few-shot examples:

• Learning by example: Instead of describing a task or process, you are showing the LLM what to do and how to format its output [1]. This is particularly useful when the task is complex or hard to describe with simple instructions [3].
• Pattern recognition: LLMs are trained to predict the next word by learning patterns in language [4]. Few-shot examples provide a pattern that the LLM can recognize and follow [4]. The LLM learns to predict the next word or output based on the examples [4].
• Input-output pairs: The examples you provide usually consist of pairs of inputs and corresponding outputs [1]. The input is what the LLM will use to generate a response and the output demonstrates what the response should look like [1].
• Prefixes: You can add a prefix to the input and output in your examples that give the LLM more information about what you want it to do [1, 2]. However, the LLM can learn from patterns even without prefixes [2]. For example, in sentiment analysis you could use the prefixes “input:” and “sentiment:” [1].
• Intermediate steps: The examples can show intermediate steps to a solution. This allows the LLM to learn how to apply a series of steps to reach a goal [5, 6]. For example, with a driving task, the examples can show a sequence of actions such as “look in the mirror,” then “signal,” then “back up” [6].
• Constraining Output: Few-shot examples can help constrain the output, meaning the LLM is more likely to generate responses that fit within the format of the examples you provide [4]. If you have an example where the output is a specific label such as positive, negative or neutral, the LLM is more likely to use those labels in its response [4].
• Teaching new tricks: By using few-shot examples, you are teaching the LLM a new trick or task [1]. The LLM learns a new process by following the patterns it observes in the examples [4].
• Generating examples: One interesting capability is that the LLM can use the patterns from the few shot examples to generate more examples, which can then be curated by a human to improve future prompts [5, 7]. LLMs can even use few-shot examples to generate examples for other models [5].
• Not limited to classification: Few-shot examples are not limited to simple classification tasks, such as sentiment analysis. They can also be used for more complex tasks such as planning, and generating action sequences [4, 8].
• Flexibility: Few-shot prompting is flexible and can be applied to all kinds of situations. You can use any pattern that has examples with an input and a corresponding output [8].
• Mistakes: When creating few-shot examples you should be sure that the prefixes you are using are meaningful and provide context to the LLM [9, 10]. You should make sure that you are providing enough information in each example to derive the underlying process from the input to the output [10, 11]. You also need to make sure that your examples have enough detail and rich information so that the LLM can learn from them [12].

By using few-shot examples, you are effectively leveraging the LLM’s ability to recognize and reproduce patterns in language [4]. You can teach it new tasks and get a structured output from the LLM without having to explicitly define all of the steps needed to solve a problem [1].

Effective Prompt Engineering for Large Language Models

Effective prompts are essential for leveraging the capabilities of large language models (LLMs) and getting desired results [1, 2]. They go beyond simply asking a question; they involve using specific techniques, patterns, and structures to elicit specific behaviors from the LLM [3].

Here are some key aspects of creating effective prompts, based on the provided sources:

• Understanding the Prompt’s Role: A prompt is not just a question, it is a call to action for the LLM to generate output [3]. It’s a way of getting the LLM to start generating words, code, or other types of output [3]. A prompt can also be a cue or reminder, that helps the LLM remember something or a previous instruction [4]. Prompts can also provide information to the LLM [5].
• Specificity: The more specific a prompt is, the more specific the output will be [6]. You need to inject specific ideas and details into the prompt to get a specific response [6]. Generic questions often lead to generic answers [6].
• Creativity: Effective prompts require creativity and an openness to explore [2]. You have to be a creative thinker and problem solver to use LLMs effectively, and the more creative you are, the better the outputs will be [2].
• Patterns: Prompt patterns are a key aspect of prompt engineering [7, 8]. They are a way to structure phrases and statements in your prompt to solve particular problems with a LLM [8]. Patterns tap into the LLM’s training data [5]. and help elicit a consistent and predictable output [9]. You can use patterns to get into specific behaviors of the LLM [7].
• Key Prompt Patterns Some key prompt patterns include:
• Persona Pattern: Asking the LLM to act as a specific person, object, or system, which can tap into the LLM’s rich understanding of a particular role [7, 8]. This gives the LLM rules to follow [8].
• Audience Persona Pattern: You can tell the LLM to produce an output for a specific audience or type of person [10].
• Question Refinement Pattern: Asking the LLM to improve or rephrase a question before answering it, which can help generate better questions [11]. The LLM can use its training to infer better questions and wording [11].
• Few-shot examples or few-shot prompting: Providing the LLM with a few examples of the input and the desired output, so it can learn the pattern and apply it to new input [12]. By giving a few examples the LLM can learn a new task [12]. The examples can show intermediate steps to a solution [12].
• Flipped Interaction Pattern: Asking the LLM to ask you questions to get more information on a topic before taking an action [13].
• Template Pattern: Providing a template for the LLM’s output including placeholders for specific values [14].
• Alternative Approaches Pattern: Asking the LLM to suggest multiple ways of accomplishing a task [15]. This can be combined with a prompt where you ask the LLM to write prompts for each alternative [15].
• Ask for Input Pattern: Adding a statement to a prompt that asks for the first input and prevents the LLM from generating a large amount of output initially [16].
• Outline Expansion Pattern: Prompting the LLM to create an outline, and then expanding certain parts of the outline to progressively create a detailed document [17].
• Menu Actions Pattern: Defining a set of actions with a trigger that you can run within a conversation, which allows you to reuse prompts and share prompts with others [18].
• Tail Generation Pattern: Having the LLM generate a tail at the end of its output that reminds it what the rules of the game are and provides the context for the next interaction [19].
• Iterative Refinement: Prompts can be refined through conversation with an LLM. Think of it as a process of iterative refinement, shaping and sculpting an output over time [20]. Instead of trying to get the perfect answer from the first prompt, it’s about guiding the LLM through a conversation to reach the desired goal [20, 21].
• Conversational approach: Prompts are not just one-off questions or statements but can represent an entire conversation [21].
• Programming: Prompts can be used to program an LLM by giving it rules and instructions [22]. You can give the LLM rules to follow and build a program through a series of instructions [8, 22].
• Experimentation: You often need to try out different variations on prompts [2]. Be open to exploring and trying different things, and to running little experiments [2].
• Context: Prompts should be specific and provide context, to get the desired output [5].
• Structure: Use specific words and phrases to tap into specific information [6]. The structure of the prompt itself can influence the structure of the output [6, 23]. You can provide the structure of what you want the LLM to do by providing a pattern in the prompt itself [23].
• Dealing with Randomness: LLMs have some unpredictability by design [24]. Effective prompt engineering is about learning to constrain this unpredictability [24]. There is some randomness in the output of LLMs because they are constantly trying to predict the next word [5, 9].

By combining these techniques and patterns, you can create effective prompts that allow you to get the desired behavior from large language models. Effective prompts will also allow you to tap into the power of the LLM to create novel and creative outputs, and to use LLMs as tools for problem solving and accelerating your ideas [7].

By Amjad Izhar
Contact: amjad.izhar@gmail.com
https://amjadizhar.blog

Affiliate Disclosure: This blog may contain affiliate links, which means I may earn a small commission if you click on the link and make a purchase. This comes at no additional cost to you. I only recommend products or services that I believe will add value to my readers. Your support helps keep this blog running and allows me to continue providing you with quality content. Thank you for your support!