Few-Shot Prompt Builder

easyFREEPrompt Engineering & ParsingLLM 3: Applications & Evaluation

Build a few-shot prompt from examples and a query.

Given a system instruction, N example pairs (input → output), and a query, construct a prompt in the format:

{system}

Example 1:
Input: {example_input}
Output: {example_output}

Example 2:
...

Query:
Input: {query}
Output:

Input:

Line 1: System instruction
Line 2: N (number of examples)
Next 2N lines: alternating input/output for each example
Last line: Query input

Output: The formatted prompt string.

Example

Input

Classify sentiment as positive or negative.
2
I love this!
positive
This is terrible.
negative
Pretty good product.

Output

Classify sentiment as positive or negative.

Example 1:
Input: I love this!
Output: positive

Example 2:
Input: This is terrible.
Output: negative

Query:
Input: Pretty good product.
Output:

Reasoning

The system instruction Classify sentiment as positive or negative. is used as the header of the prompt.
The number of examples 2 indicates that two example pairs will be included in the prompt, with each pair consisting of an input and an output.
The example pairs are formatted into the prompt as Example 1 and Example 2, with their respective inputs and outputs: I love this! / positive and This is terrible. / negative.
The query input Pretty good product. is added to the prompt with a blank output, awaiting the user's response.

Constraints

N >= 0 (zero-shot if N=0)
Each example has exactly one input line and one output line
Blank line between system instruction and first example
Blank line between each example
"Output:" at end has no trailing content

Learn this in

Fine-tuningLLM Study Plan Prompt EngineeringLLM Study Plan Advanced NLP TopicsNLP Study Plan Language ModelingNLP Study Plan

More from LLM 3: Applications & Evaluation

Beam Search Decoderhard INT8 Quantization Simulatormedium Nucleus (Top-P) Samplingmedium Top-K Samplingeasy Temperature Samplingeasy

Concept primer

Background Knowledge

The problem of building a few-shot prompt from examples and a query falls under the topic of Prompt Engineering & Parsing, which is a crucial aspect of working with Large Language Models (LLMs). In this context, a prompt is a piece of text that is used to elicit a specific response from a language model. The goal of prompt engineering is to design prompts that effectively guide the model to produce the desired output. Few-shot learning, in particular, involves using a limited number of examples to fine-tune the model's performance on a specific task.

The concept of few-shot learning is important here, as it refers to the ability of a model to learn from a small number of examples. This is in contrast to traditional machine learning approaches, which often require large amounts of training data. In the context of prompt engineering, few-shot learning involves using a small set of example input-output pairs to construct a prompt that can guide the model to produce the correct output for a given query. Understanding how to effectively use these examples to construct a prompt is key to solving this problem.

To approach this problem, it's also important to have a basic understanding of string manipulation and text formatting, as the goal is to construct a formatted prompt string from the given input. This involves concatenating strings, using newline characters to separate lines, and formatting the input-output examples in a specific way. Additionally, understanding how to read and process input from a file or standard input is necessary to handle the input provided in the problem.

Algorithm/Approach

The general approach to solving this problem involves reading the input, processing it to extract the system instruction, example input-output pairs, and query input, and then using this information to construct the formatted prompt string. This can be achieved through a simple, iterative approach that involves looping over the example pairs and concatenating the relevant strings to build the prompt.

Step-by-Step Strategy

To implement the solution, follow these steps:

Read the system instruction from the input.
Read the number of example pairs (N) and then read each input-output pair, storing them for later use.
Read the query input.
Initialize an empty string to store the formatted prompt.
Concatenate the system instruction, example pairs (formatted as "Example X: Input: {example_input} Output: {example_output}"), and query input (formatted as "Query: Input: {query} Output:") to the prompt string, using newline characters to separate each line.
Return the formatted prompt string.

Common Pitfalls

When implementing the solution, watch out for:

Incorrectly handling the input-output pairs, such as swapping the input and output or failing to format them correctly.
Not using newline characters correctly to separate lines in the prompt string.
Failing to handle the query input correctly, such as not formatting it as "Query: Input: {query} Output:".

Time & Space Complexity

The time complexity of this solution is O(N), where N is the number of example pairs, since we need to read and process each pair. The space complexity is also O(N), as we need to store each example pair and the query input in memory to construct the prompt string. The overall complexity is linear, making this a relatively efficient solution.

solution.py

Python 3.13

Test results