diff --git a/docs/README.md b/docs/README.md index d9ac12b16..45863c611 100644 --- a/docs/README.md +++ b/docs/README.md @@ -106,6 +106,8 @@ We can pass initial data to the interpreter to populate variables used in a PDL pdl --data ``` +For an example, see [file](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/free_variables.pdl). + This can also be done by passing a JSON or YAML file: ``` diff --git a/docs/tutorial.md b/docs/tutorial.md index 52c38c225..2f8c2984b 100644 --- a/docs/tutorial.md +++ b/docs/tutorial.md @@ -72,7 +72,7 @@ Using the `input` field, we can also give a directly an array of messages (`role --8<-- "./examples/tutorial/calling_llm_with_input_messages.pdl" ``` -This has the same output as the previous program. +This has the same output as the previous program. An alternative way of writing this is [this](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/calling_llm_with_input_messages_var.pdl) program. ### Parameter defaults for watsonx Granite models @@ -104,7 +104,7 @@ Consider the following example ([file](https://github.com/IBM/prompt-declaration ``` Here we assign the output of the model to variable `GEN` using the `def` field. The last line of the program prints out the value of `GEN`. Notice the notation `${ }` for accessing the value of a variable. Any [Jinja](https://jinja.palletsprojects.com/en/3.1.x/) expression is allowed to be used inside these braces. These expressions -are also used to specify conditions for loops and conditionals. See for example this [file](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/conditionals_loops.pdl). +are also used to specify conditions for loops and conditionals. See for example this [file](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/programs/chatbot.pdl). When we execute this program, we obtain: ``` @@ -115,10 +115,10 @@ GEN is equal to: Hello ## Model Chaining -In PDL, we can declaratively chain models together as in the following example ([file](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/model_chaining.pdl)): +In PDL, we can declaratively chain models together as in the following example ([file](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/calling_llm_chaining.pdl)): ```yaml ---8<-- "./examples/tutorial/model_chaining.pdl" +--8<-- "./examples/tutorial/calling_llm_chaining.pdl" ``` In this program, the first call is to a Granite model with the prompt `"Hello\n"`. The following block in the program prints out the sentence: `"\nDid you just say Hello?\n"`. The final line of the program takes the entire context produced so far and passes it as input to the Granite model. Notice that the input passed to this model is the context up to that point, represented as a conversation. This makes it easy to chain models together and continue building on previous interactions. Notice how the conversational context is accumulated implicitly without requiring the user to explicitly manage messages. @@ -159,12 +159,18 @@ To reset the context when calling a function, we can pass the special argument: Notice that the arguments of function calls are expressions and cannot be arbitrary PDL blocks. +A function name can be aliased (see [example](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/function_alias.pdl)). + +The context inherited by a function can be reset at the call site (see [example](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/function_empty_context.pdl)). + +Functions can be declared with optional parameters (see [example](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/function_optional_params.pdl)). + ## Grouping Variable Definitions in Defs -In PDL, the above program can be written more neatly by grouping certain variable definitions into a `defs` section, as follows ([file](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/grouping_definitions.pdl)): +In PDL, the above program can be written more neatly by grouping certain variable definitions into a `defs` section, as follows ([file](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/defs.pdl)): ```yaml ---8<-- "./examples/tutorial/grouping_definitions.pdl" +--8<-- "./examples/tutorial/defs.pdl" ``` @@ -173,7 +179,7 @@ This program has the same output has the one from the previous section. Any block can have a `defs` field defining variables used in that block. Notice it's different than the `def` field which stores the result of the block after execution. - +For another example, see [file](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/defs-hello.pdl). ## Muting Block Output with contribute @@ -310,9 +316,15 @@ Other possible values for `parser` are `yaml`, `jsonl`, or `regex`. The following example extracts using a regular expression parser the code between triple backtick generated by a model: ```yaml +--8<-- "./examples/tutorial/parser_regex_code.pdl" +``` + +Here is another example using a regular expression: +```yaml --8<-- "./examples/tutorial/parser_regex.pdl" ``` + We support the following operations with the`regex` parser (indicated with the `mode` field): - `fullmatch` (default) @@ -334,14 +346,14 @@ See [here](https://docs.python.org/3/library/re.html) for more information on ho ## Calling code -The following script shows how to execute python code ([file](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/calling_code.pdl)). The python code is executed locally (or in a containerized way if using `pdl --sandbox`). In principle, PDL is agnostic of any specific programming language, but we currently only support Python, Jinja, and shell commands. Variables defined in PDL are copied into the global scope of the Python code, so those variables can be used directly in the code. However, mutating variables in Python has no effect on the variables in the PDL program. The result of the code must be assigned to the variable `result` internally to be propagated to the result of the block. A variable `def` on the code block will then be set to this result. +The following script shows how to execute python code ([file](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/code_python.pdl)). The python code is executed locally (or in a containerized way if using `pdl --sandbox`). In principle, PDL is agnostic of any specific programming language, but we currently only support Python, Jinja, and shell commands. Variables defined in PDL are copied into the global scope of the Python code, so those variables can be used directly in the code. However, mutating variables in Python has no effect on the variables in the PDL program. The result of the code must be assigned to the variable `result` internally to be propagated to the result of the block. A variable `def` on the code block will then be set to this result. In order to define variables that are carried over to the next Python code block, a special variable `PDL_SESSION` can be used, and variables assigned to it as fields. -See for example: ([file](https://github.com/IBM/prompt-declaration-language//blob/main/examples/rag/tfidf_rag.pdl)). +See for example: ([file](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/programs/tfidf_rag.pdl)). ```yaml ---8<-- "./examples/tutorial/calling_code.pdl" +--8<-- "./examples/tutorial/code_python.pdl" ``` This results in the following output (for example): @@ -349,14 +361,17 @@ This results in the following output (for example): Hello, r! ``` -PDL also supports Jinja code blocks, as well as PDL code blocks for meta-cycle programming. +PDL also supports Jinja code blocks, shell commands, as well as PDL code blocks for meta-cycle programming. For more examples, see +([Jinja code](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/code_jinja.pdl)), +([shell command](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/code_command.pdl)), +([PDL code](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/code_pdl.pdl)). ## Calling REST APIs -PDL programs can contain calls to REST APIs with Python code. Consider a simple weather app ([file](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/calling_apis.pdl)): +PDL programs can contain calls to REST APIs with Python code. Consider a simple weather app ([file](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/programs/weather.pdl)): ```yaml ---8<-- "./examples/tutorial/calling_apis.pdl" +--8<-- "./examples/tutorial/programs/weather.pdl" ``` In this program, we first define a query about the weather in some location (assigned to variable `QUERY`). The next block is a call to a Granite model with few-shot examples to extract the location, which we assign to variable `LOCATION`. The next block makes an API call with Python (mocked in this example). Here the `LOCATION` is appended to the `url`. The result is a JSON object, which may be hard to interpret for a human user. So we make a final call to an LLM to interpret the JSON in terms of weather. Notice that many blocks have `contribute` set to `[]` to hide intermediate results. @@ -364,10 +379,10 @@ In this program, we first define a query about the weather in some location (ass ## Data Block -PDL offers the ability to create JSON data as illustrated by the following example (described in detail in the [Overview](https://ibm.github.io/prompt-declaration-language/#overview) section). The `data` block can gather previously defined variables into a JSON structure. This feature is useful for data generation. Programs such as this one can be generalized to read jsonl files to generate data en masse by piping into another jsonl file ([file](https://github.com/IBM/prompt-declaration-language/blob/main/examples/tutorial/data_block.pdl)). +PDL offers the ability to create JSON data as illustrated by the following example (described in detail in the [Overview](https://ibm.github.io/prompt-declaration-language/#overview) section). The `data` block can gather previously defined variables into a JSON structure. This feature is useful for data generation. Programs such as this one can be generalized to read jsonl files to generate data en masse by piping into another jsonl file ([file](https://github.com/IBM/prompt-declaration-language/blob/main/examples/tutorial/programs/code-json.pdl)). ```yaml ---8<-- "./examples/tutorial/data_block.pdl" +--8<-- "./examples/tutorial/programs/code-json.pdl" ``` Notice that in the `data` block the values are interpreted as Jinja expressions. If values need to be PDL programs to be interpreted, then you need to use @@ -409,11 +424,11 @@ The `import` block means that the PDL code at that file is executed and its scop ## Conditionals and Loops -PDL supports conditionals and loops as illustrated in the following example ([file](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/conditionals_loops.pdl)), which implements a chatbot. +PDL supports conditionals and loops as illustrated in the following example ([file](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/programs/chatbot.pdl)), which implements a chatbot. ```yaml ---8<-- "./examples/tutorial/conditionals_loops.pdl" +--8<-- "./examples/tutorial/programs/chatbot.pdl" ``` The first block prompts the user for a query, and this is contributed to the background context. The next @@ -432,6 +447,8 @@ Notice that the `repeat` and `then` blocks are followed by `text`. This is becau The way that the result of each iteration is collated with other iterations can be customized in PDL using the `join` feature (see the following section). +Another simple example of using an `if` statement is [this](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/if.pdl). + ### For Loops PDL also offers `for` loops over lists. @@ -536,7 +553,7 @@ as soon as one of the exit conditions is satisfied: ### Match block PDL provides a match block for convenience. -Consider the [example](https://github.com/IBM/prompt-declaration-language//blob/main/examples/intrinsics/demo-hallucination.pdl). This shows retrieved RAG documents +Consider the [example](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/programs/demo-hallucination.pdl). This shows retrieved RAG documents that are then submitted with a query to a RAG Granite model. The output contains an answer to the query together with hallucination score and possibly a citation. @@ -565,7 +582,7 @@ The `match` field indicates an expression to match on. The cases follow the `wit ## Roles and Chat Templates -Consider again the chatbot example ([file](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/conditionals_loops.pdl)). By default blocks have role `user`, except for model call blocks, which have role `assistant`. +Consider again the chatbot example ([file](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/programs/chatbot.pdl)). By default blocks have role `user`, except for model call blocks, which have role `assistant`. If we write roles explicitly for the chatbot, we obtain: @@ -624,12 +641,12 @@ parameters: ## Type Checking -Consider the following PDL program ([file](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/gen-data.pdl)). It first reads the data -found [here](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/gen-data.yaml) to form few-shot examples. These demonstrations show how to create +Consider the following PDL program ([file](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/type_checking.pdl)). It first reads the data +found [here](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/type_checking_data.yaml) to form few-shot examples. These demonstrations show how to create some JSON data. ```yaml ---8<-- "./examples/tutorial/gen-data.pdl" +--8<-- "./examples/tutorial/type_checking.pdl" ``` Upon reading the data we use a parser to parse it into a YAML. The `spec` field indicates the expected type for the @@ -641,9 +658,9 @@ Similarly, the output of the model call is parsed as YAML, and the `spec` indica When we run this program, we obtain the output: ``` -gen-data.pdl:8 - Type errors during spec checking: -gen-data.pdl:8 - 30 should be of type {'name': 'John', 'age': '30'} +type_checking.pdl:9 - Type errors during spec checking: +type_checking.pdl:9 - twentyfive should be of type ``` Notice that since we asked the age to be produced in letters, we got a string back and this causes a type error indicated above. @@ -670,9 +687,24 @@ the examples below: - `[{question: str, answer: str}]`: same as above - `{enum: [red, green, blue]}`: an enumeration +Another example of type checking a list can be found [here](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/type_list.pdl). + ## Structured Decoding -When a type is specified in a PDL block, it is used for structured decoding with models that support it. The fields `guided_json` and `response_format` are added automatically by the interpreter with a JSON Schema value obtained from the type. Models that support structured decoding will then use this to generate JSON of the correct format. +When a type is specified in a PDL block, it is used for structured decoding with models that support it. The fields `guided_json` and `response_format` are added automatically by the interpreter with a JSON Schema value obtained from the type. Models on platforms that support structured decoding will then use this to generate JSON of the correct format. + +The following [program](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/structured-decoding.pdl): + +```yaml +--8<-- "./examples/tutorial/structured_decoding.pdl" +``` + +produces the output: +``` + +What is the color of the sky? +{'color': 'blue'} +``` ## Python SDK diff --git a/examples/code/code-json.pdl b/examples/code/code-json.pdl index b24945831..1f1c7632b 100644 --- a/examples/code/code-json.pdl +++ b/examples/code/code-json.pdl @@ -5,10 +5,9 @@ defs: parser: yaml TRUTH: read: ./ground_truth.txt -text: +lastOf: - model: ollama_chat/granite3.2:2b def: EXPLANATION - contribute: [] input: | Here is some info about the location of the function in the repo. @@ -21,10 +20,7 @@ text: Explain the following code: ``` ${ CODE.source_code }``` - parameters: - temperature: 0 - def: EVAL - contribute: [] lang: python code: | @@ -35,7 +31,6 @@ text: truth = """ ${ TRUTH } """ - # (In PDL, set `result` to the output you wish for your code block.) result = textdistance.levenshtein.normalized_similarity(expl, truth) - data: input: ${ CODE } diff --git a/examples/demo/1-gen-data.pdl b/examples/demo/1-gen-data.pdl deleted file mode 100644 index 251c59a53..000000000 --- a/examples/demo/1-gen-data.pdl +++ /dev/null @@ -1,30 +0,0 @@ -# Expected not to type check -description: Creating JSON Data -defs: - data: - read: ./data.yaml - parser: yaml - spec: { questions: [str], answers: [obj] } -text: - - model: replicate/ibm-granite/granite-3.1-8b-instruct - def: model_output - spec: {name: str, age: int} - input: - array: - - role: user - content: - text: - - for: - question: ${ data.questions } - answer: ${ data.answers } - repeat: | - ${ question } - ${ answer } - - > - Question: Generate only a JSON object with fields 'name' and 'age' and set them appropriately. Write the age all in letters. Only generate a single JSON object and nothing else. - parser: yaml - parameters: - stop_sequences: "Question" - temperature: 0 - - \ No newline at end of file diff --git a/examples/talk/1-hello.pdl b/examples/demo/1-hello.pdl similarity index 100% rename from examples/talk/1-hello.pdl rename to examples/demo/1-hello.pdl diff --git a/examples/talk/10-sdg.pdl b/examples/demo/10-sdg.pdl similarity index 100% rename from examples/talk/10-sdg.pdl rename to examples/demo/10-sdg.pdl diff --git a/examples/talk/11-repeat.pdl b/examples/demo/11-repeat.pdl similarity index 100% rename from examples/talk/11-repeat.pdl rename to examples/demo/11-repeat.pdl diff --git a/examples/talk/2-model-chaining.pdl b/examples/demo/2-model-chaining.pdl similarity index 100% rename from examples/talk/2-model-chaining.pdl rename to examples/demo/2-model-chaining.pdl diff --git a/examples/demo/2-teacher.pdl b/examples/demo/2-teacher.pdl deleted file mode 100644 index b5ad14bfa..000000000 --- a/examples/demo/2-teacher.pdl +++ /dev/null @@ -1,409 +0,0 @@ -defs: - teacher_sys_prompt: You are a very knowledgeable AI Assistant that will faithfully assist the user with their task. - teacher_model: replicate/ibm-granite/granite-3.1-8b-instruct - teacher_template: - function: - sys_prompt: str - prompt: str - return: [INST] ${sys_prompt} ${prompt} [/INST] - teacher_stop_token: - - - question_template_freeform: - function: - num_samples: int - task_description: str - icl_question: str - spec: { introduction: str, principles: str, examples: str, generation: str, max_new_tokens: int } - return: - data: - introduction: | - You are asked to come up with a set of ${num_samples} diverse questions - ${task_description}. - principles: | - Please follow these guiding principles when generating responses: - * Use proper grammar and punctuation. - * Always generate safe and respectful content. Do not generate content that is harmful, abusive, or offensive. - * Always generate content that is factually accurate and relevant to the prompt. - * The questions should be clear and human-like. - * The questions should be diverse and cover a wide range of topics. - * The questions should not be template-based or generic, it should be very diverse. - * Simply return the questions, do not return any answers or explanations. - * Strictly adhere to the prompt and generate responses in the same style and format as the example. - Use this format to generate the questions: - ### Question 1: - examples: | - To better assist you with this task, here is an example: - ### Question 1: ${icl_question} - generation: | - Now generate ${num_samples} such questions, remember to follow the principles mentioned above and use the same format as the examples. Remember to use the same style and format as the example above. - max_new_tokens: 10000 - - gen_questions_freeform_inner: - function: - num_samples: int - task_description: str - icl_question: str - icl_answer: str - spec: [{icl_question: str, icl_answer: str, question: str}] - return: - defs: - prompt_data: - call: ${question_template_freeform} - spec: { introduction: str, principles: str, examples: str, generation: str, max_new_tokens: int } - args: - num_samples: ${num_samples} - task_description: ${task_description} - icl_question: ${icl_question} - teacher_input: - call: ${teacher_template} - args: - sys_prompt: ${teacher_sys_prompt} - prompt: |- - ${prompt_data.introduction} - ${prompt_data.principles} - ${prompt_data.examples} - ${prompt_data.generation} - teacher_output: - model: ${teacher_model} - input: ${teacher_input} - parameters: - temperature: 0 - stop_sequences: "${teacher_stop_token}" - max_new_tokens: ${prompt_data.max_new_tokens} - parser: - regex: '### Question [0-9]+:\s*([^#\n]+)' - mode: findall - for: - question: ${teacher_output} - repeat: - data: - icl_question: ${icl_question} - icl_answer: ${icl_answer} - question: ${question} - join: - as: array - - gen_questions_freeform: - function: - task_description: str - seed_examples: [{question: str, answer: str}] - spec: [{icl_question: str, icl_answer: str, question: str}] - return: - defs: - list_of_lists: - for: - example: ${seed_examples} - repeat: - call: ${gen_questions_freeform_inner} - args: - num_samples: 2 - task_description: ${task_description} - icl_question: ${example.question} - icl_answer: ${example.answer} - join: - as: array - lang: python - code: | # flatten list_of_lists into simple list - result = [q for qs in ${list_of_lists} for q in qs] - - - filter_questions_template: - function: - task_description: str - question: str - spec: {introduction: str, principles: str, generation: str, max_new_tokens: int} - return: - data: - introduction: | - Please act as an impartial judge and evaluate the questions generated by an AI assistant displayed below. Evaluate whether or not the question is a good question of how AI Assistant should respond to the user's instruction. Please assign a score using a binary 0/1 scale. - principles: | - Here are the requirements: - * A large language model should be able to complete the question. For example, do not ask the assistant to create any visual or audio output. For another example, do not ask the assistant to wake you up at 5pm or set a reminder because it cannot perform any action. - * The questions should be in English. - * The questions should be 1 to 2 sentences long and should be properly formatted. - * The question should not be offensive, abusive, or harmful. It should be safe and respectful. - * The question should be relevant to the task given - ${task_description}. - If the question meets the above requirements, please rate it 1. If not, please rate it 0. - generation: | - Begin your evaluation by providing a short explanation. Be as objective as possible. After providing your explanation, you must rate the question on a scale of 0 or 1 as mentioned above by strictly following this format: \"[[rating]]\", for example: \"Rating: [[1]]\" - Here is the question you need to evaluate: - ${question} - max_new_tokens: 256 - - # https://github.com/instruct-lab/datagen-pipeline/blob/main/sdg/filter_questions.py - filter_questions_inner: - function: - task_description: str - question: str - spec: float - return: - defs: - prompt_data: - call: ${filter_questions_template} - spec: {introduction: str, principles: str, generation: str, max_new_tokens: int} - args: - task_description: ${task_description} - question: ${question} - teacher_input: - call: ${teacher_template} - args: - sys_prompt: ${teacher_sys_prompt} - prompt: |- - ${prompt_data.introduction} - ${prompt_data.principles} - ${prompt_data.generation} - teacher_output: - model: ${teacher_model} - input: ${teacher_input} - parameters: - stop_sequences: "${teacher_stop_token}" - max_new_tokens: ${prompt_data.max_new_tokens} - temperature: 0 - parser: - spec: { "rating": str } - # regex: "Rating.*\\[\\[(?P\\d+\\.?\\d*)\\]\\]" - regex: 'Rating.*\[\[(?P\d+\.?\d*)\]\]' - mode: search - data: ${ teacher_output.rating | float } - - filter_questions: - function: - task_description: str - questions: [{icl_question: str, icl_answer: str, question: str}] - spec: [{icl_question: str, icl_answer: str, question: str}] - return: - defs: - list_of_pairs: - for: - question: ${questions} - repeat: - defs: - filter_output: - call: ${filter_questions_inner} - args: - task_description: ${task_description} - question: ${question.question} - data: - question: ${question} - keep: ${filter_output} - join: - as: array - filtered: - lang: python - code: | # keep only if "keep" column is non-zero - result = [p["question"] for p in ${ list_of_pairs } if p["keep"]] - data: ${filtered} - - - answer_template: - function: - icl_question: str - icl_response: str - question: str - spec: {introduction: str, principles: str, examples: str, generation: str, max_new_tokens: int, additional_stop_tokens: [str]} - return: - data: - introduction: Your task is to faithfully follow the user's prompt and generate a response. - principles: | - Please follow these guiding principles when generating responses: - * Use proper grammar and punctuation. - * Always generate safe and respectful content. Do not generate content that is harmful, abusive, or offensive. - * Always generate content that is factually accurate and relevant to the prompt. - * Strictly adhere to the prompt and generate responses in the same style and format as the example. - examples: | - To better assist you with this task, here is an example: - [Question] - ${icl_question} - [Response] - ${icl_response} - generation: | - Now generate a response to the following prompt. Remember to use the same style and format as the example above. - [Question] - ${question} - [Response] - max_new_tokens: 2048 - additional_stop_tokens: - - "[Question]" - - - gen_answers_inner: - function: - question: {icl_question: str, icl_answer: str, question: str} - spec: {question: str, answer: str} - return: - defs: - prompt_data: - call: ${answer_template} - spec: {introduction: str, principles: str, examples: str, generation: str, max_new_tokens: int, additional_stop_tokens: [str]} - args: - icl_question: ${question.icl_question} - icl_response: ${question.icl_answer} - question: ${question.question} - teacher_input: - call: ${teacher_template} - args: - sys_prompt: ${teacher_sys_prompt} - prompt: |- - ${prompt_data.introduction} - ${prompt_data.principles} - ${prompt_data.examples} - ${prompt_data.generation} - teacher_output: - model: ${teacher_model} - input: ${teacher_input} - parameters: - stop_sequences: ${ ([teacher_stop_token] + prompt_data.additional_stop_tokens) | join(',') } - max_new_tokens: ${prompt_data.max_new_tokens} - temperature: 0 - parsed_answer: - lang: python - code: | # parse model output - result = """ ${teacher_output} """.strip() - if "[Response]" in result: - result = result[result.find("[Response]") + len("[Response]"):] - if "[Question]" in result: - result = result[:result.find("[Question]")] - data: - question: ${question.question} - answer: ${parsed_answer} - - gen_answers: - function: - questions: [{icl_question: str, icl_answer: str, question: str}] - spec: [{question: str, answer: str}] - return: - defs: - all_results: - spec: [{question: str, answer: str}] - for: - question: ${ questions } - repeat: - call: ${gen_answers_inner} - args: - question: ${question} - join: - as: array - lang: python - spec: [{question: str, answer: str}] - code: | # keep only if answer non-empty - result = [r for r in ${all_results} if len(r["answer"]) > 0] - - - filter_qa_template: - function: - question: str - answer: str - spec: {introduction: str, principles: str, generation: str, max_new_tokens: int} - return: - data: - introduction: | - Please act as an impartial judge and evaluate the quality of the answer provided by an AI assistant to the questions displayed below. Evaluate whether or not the answer is a good example of how AI Assistant should respond to the user's instruction. Please assign a score using the following 3-point scale. - principles: | - 1: It means the answer is incorrect, irrelevant, unsafe or provides incomplete and garbage information. For instance, the answer may be factually wrong, off-topic, or filled with irrelevant content that doesn't address the user's question or it could be incomplete and hanging. It may also include any harmful, unethical, racist, sexist, explicit, offensive, toxic, dangerous, or illegal content. - - 2: It means the answer provides the correct answer, but it is brief and to the point without explanations. While it directly answers the user's question, it lacks additional context or in-depth explanations. - - 3: It means the answer is a perfect answer from an AI Assistant. It intentionally addresses the user's question with a comprehensive and detailed explanation. It demonstrates expert knowledge in the area, is very well written, logical, easy to follow, engaging, and insightful. And the answer is safe and does not include any harmful content. - generation: | - Begin your evaluation by providing a short explanation. Be as objective as possible. After providing your explanation, you must rate the answer on a scale of 1 to 3 as mentioned above by strictly following this format: \"[[rating]]\", for example: \"Rating: [[1]]\" - - Here is the question and the answer you need to evaluate: - [Question] - ${question} - [Answer] - ${answer} - max_new_tokens: 256 - - filter_question_answer_pair_inner: - function: - question: str - answer: str - spec: float - return: - defs: - prompt_data: - call: ${filter_qa_template} - spec: {introduction: str, principles: str, generation: str, max_new_tokens: int} - args: - question: ${question} - answer: ${answer} - teacher_input: - call: ${teacher_template} - args: - sys_prompt: ${teacher_sys_prompt} - prompt: |- - ${prompt_data.introduction} - ${prompt_data.principles} - ${prompt_data.generation} - teacher_output: - model: ${teacher_model} - input: ${teacher_input} - parameters: - stop_sequences: "${teacher_stop_token}" - max_new_tokens: ${prompt_data.max_new_tokens} - temperature: 0 - parser: - spec: { "rating": str } - regex: 'Rating.*\[\[(?P\d+\.?\d*)\]\]' - mode: search - data: ${ (teacher_output.rating if teacher_output.rating is not none else 0.0) | float} - - filter_question_answer_pair: - function: - qa_pairs: [{question: str, answer: str}] - spec: [{question: str, answer: str}] - return: - defs: - ratings: - for: - qa_pair: ${qa_pairs} - repeat: - defs: - filter_output: - call: ${filter_question_answer_pair_inner} - spec: float - args: - question: ${qa_pair.question} - answer: ${qa_pair.answer} - data: - qa_pair: ${qa_pair} - rating: ${filter_output} - join: - as: array - filtered: - lang: python - spec: [{question: str, answer: str}] - code: | # keep only if rating is at least two - result = [p["qa_pair"] for p in ${ratings} if p["rating"] >= 2] - data: ${filtered} - - -text: -- "----- Loading seed examples -----\n\n" -- def: seed_examples - read: ./qna.yaml - parser: yaml -- "\n\n----- Generating questions -----\n\n" -- def: generated_questions - call: ${gen_questions_freeform} - spec: [{icl_question: str, icl_answer: str, question: str}] - args: - task_description: ${seed_examples.task_description} - seed_examples: ${seed_examples.seed_examples} -- "\n\n----- Filtering questions -----\n\n" -- def: filtered_questions - call: ${filter_questions} - spec: [{icl_question: str, icl_answer: str, question: str}] - args: - task_description: ${seed_examples.task_description} - questions: ${generated_questions} -- "\n\n----- Generating answers -----\n\n" -- def: qa_pairs - call: ${gen_answers} - args: - questions: ${filtered_questions} -- "\n\n----- Filtering QA pairs -----\n\n" -- call: ${filter_question_answer_pair} - args: - qa_pairs: ${qa_pairs} - diff --git a/examples/talk/3-def-use.pdl b/examples/demo/3-def-use.pdl similarity index 100% rename from examples/talk/3-def-use.pdl rename to examples/demo/3-def-use.pdl diff --git a/examples/demo/3-weather.pdl b/examples/demo/3-weather.pdl deleted file mode 100644 index 9a5e1e89a..000000000 --- a/examples/demo/3-weather.pdl +++ /dev/null @@ -1,32 +0,0 @@ -description: Using a weather API and LLM to make a small weather app -text: -- def: QUERY - text: "What is the weather in Madrid?\n" -- model: ollama_chat/granite3.2:2b - input: | - Extract the location from the question. - Question: What is the weather in London? - Answer: London - Question: What's the weather in Paris? - Answer: Paris - Question: Tell me the weather in Lagos? - Answer: Lagos - Question: ${ QUERY } - parameters: - stop: ["Question", "What", "!", "\n"] - def: LOCATION - contribute: [] -- lang: python - code: | - import requests - #result = requests.get('https://api.weatherapi.com/v1/current.json?key==XYZ=${ LOCATION }') - #Mock result: - result = '{"location": {"name": "Madrid", "region": "Madrid", "country": "Spain", "lat": 40.4, "lon": -3.6833, "tz_id": "Europe/Madrid", "localtime_epoch": 1732543839, "localtime": "2024-11-25 15:10"}, "current": {"last_updated_epoch": 1732543200, "last_updated": "2024-11-25 15:00", "temp_c": 14.4, "temp_f": 57.9, "is_day": 1, "condition": {"text": "Partly cloudy", "icon": "//cdn.weatherapi.com/weather/64x64/day/116.png", "code": 1003}, "wind_mph": 13.2, "wind_kph": 21.2, "wind_degree": 265, "wind_dir": "W", "pressure_mb": 1017.0, "pressure_in": 30.03, "precip_mm": 0.01, "precip_in": 0.0, "humidity": 77, "cloud": 75, "feelslike_c": 12.8, "feelslike_f": 55.1, "windchill_c": 13.0, "windchill_f": 55.4, "heatindex_c": 14.5, "heatindex_f": 58.2, "dewpoint_c": 7.3, "dewpoint_f": 45.2, "vis_km": 10.0, "vis_miles": 6.0, "uv": 1.4, "gust_mph": 15.2, "gust_kph": 24.4}}' - def: WEATHER - parser: json - contribute: [] -- model: ollama_chat/granite3.2:2b - input: | - Explain the weather from the following JSON: - ${ WEATHER } - diff --git a/examples/talk/4-function.pdl b/examples/demo/4-function.pdl similarity index 100% rename from examples/talk/4-function.pdl rename to examples/demo/4-function.pdl diff --git a/examples/demo/4-translator.pdl b/examples/demo/4-translator.pdl deleted file mode 100644 index 7431cb43e..000000000 --- a/examples/demo/4-translator.pdl +++ /dev/null @@ -1,15 +0,0 @@ -description: PDL program -text: -- "What is APR?\n" -- model: ollama_chat/granite3.2:2b -- repeat: - text: - - read: - message: "\nLanguage please: " - def: language - - if: ${ language != 'stop' } - then: - text: - - "\n\nTranslate the above to ${ language }\n" - - model: ollama_chat/granite3.2:2b - until: ${ language == 'stop' } diff --git a/examples/talk/5-code-eval.pdl b/examples/demo/5-code-eval.pdl similarity index 100% rename from examples/talk/5-code-eval.pdl rename to examples/demo/5-code-eval.pdl diff --git a/examples/talk/6-code-json.pdl b/examples/demo/6-code-json.pdl similarity index 100% rename from examples/talk/6-code-json.pdl rename to examples/demo/6-code-json.pdl diff --git a/examples/talk/7-chatbot-roles.pdl b/examples/demo/7-chatbot-roles.pdl similarity index 100% rename from examples/talk/7-chatbot-roles.pdl rename to examples/demo/7-chatbot-roles.pdl diff --git a/examples/talk/8-tools.pdl b/examples/demo/8-tools.pdl similarity index 100% rename from examples/talk/8-tools.pdl rename to examples/demo/8-tools.pdl diff --git a/examples/talk/9-react.pdl b/examples/demo/9-react.pdl similarity index 100% rename from examples/talk/9-react.pdl rename to examples/demo/9-react.pdl diff --git a/examples/demo/data.yaml b/examples/demo/data.yaml index cb0d81d1a..196e6f0ca 100644 --- a/examples/demo/data.yaml +++ b/examples/demo/data.yaml @@ -1,10 +1,16 @@ -questions: - - > - Question: Write a YAML object with 2 fields 'a' and 'b' of type int and set to 0. - - > - Question: Write a YAML object with 3 fields 'number' and 'street' and 'town' set to '10', 'Miller Road', 'Armonk' respectively. - -answers: - - { "a": 0, "b": 0 } - - - { "number": 10, "street": "miller Road", "town": "armonk" } \ No newline at end of file +source_code: + | + @SuppressWarnings("unchecked") + public static Map deserializeOffsetMap(String lastSourceOffset) throws IOException { + Map offsetMap; + if (lastSourceOffset == null || lastSourceOffset.isEmpty()) { + offsetMap = new HashMap<>(); + } else { + offsetMap = JSON_MAPPER.readValue(lastSourceOffset, Map.class); + } + return offsetMap; + } +repo_info: + repo: streamsets/datacollector + path: stagesupport/src/main/java/com/.../OffsetUtil.java + function_name: OffsetUtil.deserializeOffsetMap \ No newline at end of file diff --git a/examples/talk/ground_truth.txt b/examples/demo/ground_truth.txt similarity index 100% rename from examples/talk/ground_truth.txt rename to examples/demo/ground_truth.txt diff --git a/examples/demo/qna.yaml b/examples/demo/qna.yaml index c33b5b428..50dcc1e8e 100644 --- a/examples/demo/qna.yaml +++ b/examples/demo/qna.yaml @@ -4,15 +4,12 @@ seed_examples: - question: Tell me a pun about birds. answer: |- Why do birds eat wood? - Because they're peckish! - question: Tell me a pun about gas. answer: |- Why did the car have a belly ache? - Because it had too much gas! - question: Tell me a pun about waves. answer: |- What did the ocean say to the ocean? - - Nothing. It just waved! + Nothing. It just waved! \ No newline at end of file diff --git a/examples/granite/README.md b/examples/granite/README.md deleted file mode 100644 index 8a52f360e..000000000 --- a/examples/granite/README.md +++ /dev/null @@ -1,9 +0,0 @@ -To invoke the examples in this folder: - -``` -pdl -f ./prompt.json single_round_chat.pdl -``` - -``` -pdl multi_round_chat.pdl -``` \ No newline at end of file diff --git a/examples/granite/granite_defs.pdl b/examples/granite/granite_defs.pdl deleted file mode 100644 index c4ef70612..000000000 --- a/examples/granite/granite_defs.pdl +++ /dev/null @@ -1,57 +0,0 @@ -description: Granite System Chat -defs: - SYSTEM_TAG: "<|system|>" - USER_TAG: "<|user|>" - ASSISTANT_TAG: "<|assistant|>" - - granite_models: - data: - granite_13b_chat_v2: - id: "ibm/granite-13b-chat-v2" - system_prompt: | - You are Granite Chat, an AI language model developed by IBM. You are a cautious assistant that carefully follows instructions. You are helpful and harmless and you follow ethical guidelines and promote positive behavior. You respond in a comprehensive manner unless instructed otherwise, providing explanations when needed while maintaining a neutral tone. You are capable of coding, writing, and roleplaying. You are cautious and refrain from generating real-time information, highly subjective or opinion-based topics. You are harmless and refrain from generating content involving any form of bias, violence, discrimination or inappropriate content. You always respond to greetings (for example, hi, hello, g'day, morning, afternoon, evening, night, what's up, nice to meet you, sup, etc) with "Hello! I am Granite Chat, created by IBM. How can I help you today?". Please do not say anything else and do not start a conversation. - granite_13b_instruct_v2: - id: "ibm/granite-13b-instruct-v2" - system_prompt: "" - granite_20b_ansible_opqa: - id: "ibm/granite-20b-ansible-opqa" - system_prompt: "" - granite_20b_code_base_v1_sd: - id: "ibm/granite-20b-code-base-v1-sd" - system_prompt: "" - granite_20b_code_instruct: - id: "ibm/granite-20b-code-instruct" - system_prompt: "" - granite_20b_code_instruct_unified_api: - id: "ibm/granite-20b-code-instruct-unified-api" - system_prompt: You are a helpful assistant with access to the following function calls. Your task is to produce a sequence of function calls necessary to generate response to the user utterance. Use the following function calls as required. - granite_20b_code_instruct_v2: - id: "ibm/granite-20b-code-instruct-v2" - system_prompt: "" - granite_20b_multilang_lab_rc: - id: "ibm/granite-20b-multilang-lab-rc" - system_prompt: | - You are Labrador, an AI language model developed by IBM DMF (Data Model Factory) Alignment Team. You are a cautious assistant. You carefully follow instructions. You are helpful and harmless and you follow ethical guidelines and promote positive behavior. You always respond to greetings (for example, hi, hello, g'day, morning, afternoon, evening, night, what's up, nice to meet you, sup, etc) with "Hello! I am an AI language model, created by IBM. How can I help you today?". Please do not say anything else and do not start a conversation. - granite_20b_multilingual: - id: "ibm/granite-20b-multilingual" - system_prompt: "" - granite_34b_code_instruct: - id: "ibm/granite-34b-code-instruct" - system_prompt: "" - granite_3b_code_instruct: - id: "ibm/granite-3b-code-instruct" - system_prompt: "" - granite_7b_lab: - id: "ibm/granite-7b-lab" - system_prompt: | - You are an AI language model developed by IBM Research. You are a cautious assistant. You carefully follow instructions. You are helpful and harmless and you follow ethical guidelines and promote positive behavior. - granite_8b_code_ansible: - id: "ibm/granite-8b-code-ansible" - system_prompt: "" - granite_8b_code_instruct: - id: "ibm/granite-8b-code-instruct" - system_prompt: "" - SYSTEM_CONTENT_CHAT: - | - You are Granite Chat, an AI language model developed by IBM. You are a cautious assistant. You carefully follow instructions. You are helpful and harmless and you follow ethical guidelines and promote positive behavior. You always respond to greetings (for example, hi, hello, g'day, morning, afternoon, evening, night, what's up, nice to meet you, sup, etc) with "Hello! I am Granite Chat, created by IBM. How can I help you today?". Please do not say anything else and do not start a conversation. - \ No newline at end of file diff --git a/examples/granite/multi-prompts.json b/examples/granite/multi-prompts.json deleted file mode 100644 index f662520c9..000000000 --- a/examples/granite/multi-prompts.json +++ /dev/null @@ -1,7 +0,0 @@ -{ - "prompts": [ - "What is APR?", - "Can you write a poem about APR?", - "Now explain APR to me like I'm 5 years old" - ] -} \ No newline at end of file diff --git a/examples/granite/multi_round_chat.pdl b/examples/granite/multi_round_chat.pdl deleted file mode 100644 index b8d0f39af..000000000 --- a/examples/granite/multi_round_chat.pdl +++ /dev/null @@ -1,25 +0,0 @@ -description: Granite Multi-Round Chat -text: -# Define the variable `defs` to contain the parsed multi-prompts.json -- read: ./multi-prompts.json - parser: json - def: prompts - # Type-check multi-prompts.json against a specification - spec: {prompts: [str]} - # Don't store these prompts in the PDL context - contribute: [] -# Pass each prompt to the model -- for: - prompt: ${ prompts.prompts } - repeat: - text: - # Output the question, and add it to the context - - | - - ${ prompt } - # Use replicate.com to run the Granite model on the context, outputting the result - - model: ollama_chat/granite3.2:2b - parameters: - # Use no LLM model creativity (0 is the default) - temperature: 0 -role: user \ No newline at end of file diff --git a/examples/granite/prompt.json b/examples/granite/prompt.json deleted file mode 100644 index 7bf01049a..000000000 --- a/examples/granite/prompt.json +++ /dev/null @@ -1,3 +0,0 @@ -{ - "PROMPT": "What is the meaning of life?" -} \ No newline at end of file diff --git a/examples/granite/single_round_chat.pdl b/examples/granite/single_round_chat.pdl deleted file mode 100644 index 5a550e4f1..000000000 --- a/examples/granite/single_round_chat.pdl +++ /dev/null @@ -1,9 +0,0 @@ -description: Granite Single-Round Chat -text: -# (Note that 'PROMPT' is undefined will happen if you don't invoke pdl with `-f prompt.json`) -- "${ PROMPT }\n" -- model: replicate/ibm-granite/granite-3.1-8b-instruct - parameters: - # Use no LLM model creativity (0 is the default) - temperature: 0 -role: user diff --git a/examples/hello/hello-code.pdl b/examples/hello/hello-code.pdl deleted file mode 100644 index 69bb65522..000000000 --- a/examples/hello/hello-code.pdl +++ /dev/null @@ -1,9 +0,0 @@ -description: Hello world showing call out to python code -text: -- 'Hello, ' -- lang: python - code: | - import random - import string - result = random.choice(string.ascii_lowercase) -- '!' diff --git a/examples/hello/hello-def-use.pdl b/examples/hello/hello-def-use.pdl deleted file mode 100644 index 59b75260f..000000000 --- a/examples/hello/hello-def-use.pdl +++ /dev/null @@ -1,12 +0,0 @@ -description: Hello world with variable use -text: -- "Hello\n" -# Define GEN to be the result of a Granite LLM using ollama -- model: ollama_chat/granite3.2:2b - parameters: - # Tell the LLM to stop after generating an exclamation point. - stop: ['!'] - def: GEN -- | - - You said ${ GEN }. \ No newline at end of file diff --git a/examples/hello/hello-for-loop.pdl b/examples/hello/hello-for-loop.pdl deleted file mode 100644 index 7dd8c61c9..000000000 --- a/examples/hello/hello-for-loop.pdl +++ /dev/null @@ -1,12 +0,0 @@ -description: Hello world with for loop -defs: - numbers: - data: [1, 2, 3, 4] - names: - data: ["Bob", "Carol", "David", "Ernest"] -for: - number: ${ numbers } - name: ${ names } -repeat: - "${ name }'s number is ${ number }\n" - diff --git a/examples/hello/hello-function.pdl b/examples/hello/hello-function.pdl deleted file mode 100644 index 8aa71d5e4..000000000 --- a/examples/hello/hello-function.pdl +++ /dev/null @@ -1,9 +0,0 @@ -description: Hello world with function definition and call -text: -- def: hello - function: - name: str - return: Hello ${ name }! -- call: ${ hello } - args: - name: World diff --git a/examples/hello/hello-graniteio.pdl b/examples/hello/hello-graniteio.pdl deleted file mode 100644 index bfc016ea5..000000000 --- a/examples/hello/hello-graniteio.pdl +++ /dev/null @@ -1,5 +0,0 @@ -text: -- "Hello!\n" -- model: ibm-granite/granite-3.2-8b-instruct-preview - backend: - transformers: cpu \ No newline at end of file diff --git a/examples/hello/hello-import-lib.pdl b/examples/hello/hello-import-lib.pdl deleted file mode 100644 index 30f15ec36..000000000 --- a/examples/hello/hello-import-lib.pdl +++ /dev/null @@ -1,16 +0,0 @@ - -defs: - b: - function: - arg: str - return: - ${ arg } - - a: - function: - arg: str - return: - call: ${ b } - args: - pdl_context: [] - arg: ${ arg } \ No newline at end of file diff --git a/examples/hello/hello-import.pdl b/examples/hello/hello-import.pdl deleted file mode 100644 index 160937a23..000000000 --- a/examples/hello/hello-import.pdl +++ /dev/null @@ -1,8 +0,0 @@ -defs: - lib: - import: hello-import-lib -text: -- call: ${ lib.a } - args: - arg: Bye! - diff --git a/examples/hello/hello-iteration.pdl b/examples/hello/hello-iteration.pdl deleted file mode 100644 index ca167ce0b..000000000 --- a/examples/hello/hello-iteration.pdl +++ /dev/null @@ -1,7 +0,0 @@ -description: Hello world with iteration -text: -- Hello, world! -- "\n" -- repeat: - "This is your first PDL program\n" - max_iterations: 3 diff --git a/examples/hello/hello-model-chaining.pdl b/examples/hello/hello-model-chaining.pdl deleted file mode 100644 index 1bc11886f..000000000 --- a/examples/hello/hello-model-chaining.pdl +++ /dev/null @@ -1,15 +0,0 @@ -description: Hello world showing model chaining -text: -- "Hello\n" -- model: ollama_chat/granite3.2:2b - parameters: - # "greedy" sampling tells the LLM to use the most likely token at each step - # decoding_method: greedy # Not used by Ollama - # Tell the LLM to stop after generating an exclamation point. - stop: ['!'] - def: GEN -- "\nDid you say ${ GEN }?\n" -- model: ollama_chat/granite3.2:2b - parameters: - # decoding_method: greedy - stop: ['.'] diff --git a/examples/hello/hello-model-input.pdl b/examples/hello/hello-model-input.pdl deleted file mode 100644 index 73b39e1b5..000000000 --- a/examples/hello/hello-model-input.pdl +++ /dev/null @@ -1,7 +0,0 @@ -description: Hello world with model input -text: -- model: ollama_chat/granite3.2:2b - input: "Hello," - parameters: - # Tell the LLM to stop after generating an exclamation point. - stop: ['!'] diff --git a/examples/hello/hello-type-code.pdl b/examples/hello/hello-type-code.pdl deleted file mode 100644 index 2f6361fe5..000000000 --- a/examples/hello/hello-type-code.pdl +++ /dev/null @@ -1,8 +0,0 @@ -# Expected not to type check -description: Hello world showing call out to python code -text: -- lang: python - spec: int - code: | - import string - result = "hello" diff --git a/examples/hello/hello-type.pdl b/examples/hello/hello-type.pdl deleted file mode 100644 index e8ec68657..000000000 --- a/examples/hello/hello-type.pdl +++ /dev/null @@ -1,26 +0,0 @@ -# Expected not to type check -description: Hello world with type specification -text: -- def: GEN - text: "What is the meaning of life" -- def: translate - function: - sentence: str - language: str - spec: int - return: - lastOf: - - "\nTranslate the sentence '${ sentence }' to ${ language }.\n" - - model: ollama_chat/granite3.2:2b - parameters: - stop: ["\n"] -- call: ${ translate } - spec: str - args: - sentence: ${ GEN } - language: French -- call: ${ translate } - args: - sentence: ${ GEN } - language: Spanish - diff --git a/examples/hello/hello-while.pdl b/examples/hello/hello-while.pdl deleted file mode 100644 index e6d0893ed..000000000 --- a/examples/hello/hello-while.pdl +++ /dev/null @@ -1,7 +0,0 @@ -defs: - i: 0 -while: ${ i < 3 } -repeat: - defs: - i: ${i + 1} - text: ${i} \ No newline at end of file diff --git a/examples/hello/hello.pdl b/examples/hello/hello.pdl deleted file mode 100644 index 38d3f7fd9..000000000 --- a/examples/hello/hello.pdl +++ /dev/null @@ -1,4 +0,0 @@ -description: Hello world -text: -- "Hello\n" -- model: ollama_chat/granite3.2:2b diff --git a/examples/rag/README.md b/examples/rag/README.md index 9e653474c..7d1d5912b 100644 --- a/examples/rag/README.md +++ b/examples/rag/README.md @@ -1,3 +1,5 @@ +## pdf_query example + This example uses [Ollama](../../tutorial/#using-ollama-models). Fetch the models used in this example with ```bash @@ -28,3 +30,10 @@ pdl examples/rag/pdf_query.pdl This PDL program computes a data structure containing all questions and answers. It is printed at the end. To cleanup, run `rm pdl-rag-demo.db`. + +## tdidf_rag example + +This example requires you to install: +``` +pip install scikit-learn +``` \ No newline at end of file diff --git a/examples/talk/data.yaml b/examples/talk/data.yaml deleted file mode 100644 index 196e6f0ca..000000000 --- a/examples/talk/data.yaml +++ /dev/null @@ -1,16 +0,0 @@ -source_code: - | - @SuppressWarnings("unchecked") - public static Map deserializeOffsetMap(String lastSourceOffset) throws IOException { - Map offsetMap; - if (lastSourceOffset == null || lastSourceOffset.isEmpty()) { - offsetMap = new HashMap<>(); - } else { - offsetMap = JSON_MAPPER.readValue(lastSourceOffset, Map.class); - } - return offsetMap; - } -repo_info: - repo: streamsets/datacollector - path: stagesupport/src/main/java/com/.../OffsetUtil.java - function_name: OffsetUtil.deserializeOffsetMap \ No newline at end of file diff --git a/examples/talk/qna.yaml b/examples/talk/qna.yaml deleted file mode 100644 index 50dcc1e8e..000000000 --- a/examples/talk/qna.yaml +++ /dev/null @@ -1,15 +0,0 @@ -task_description: to teach a large language model to come up with puns -created_by: mizmo -seed_examples: -- question: Tell me a pun about birds. - answer: |- - Why do birds eat wood? - Because they're peckish! -- question: Tell me a pun about gas. - answer: |- - Why did the car have a belly ache? - Because it had too much gas! -- question: Tell me a pun about waves. - answer: |- - What did the ocean say to the ocean? - Nothing. It just waved! \ No newline at end of file diff --git a/examples/tfidf_rag/README.md b/examples/tfidf_rag/README.md deleted file mode 100644 index add819ee1..000000000 --- a/examples/tfidf_rag/README.md +++ /dev/null @@ -1,4 +0,0 @@ -This example requires you to install: -``` -pip install scikit-learn -``` \ No newline at end of file diff --git a/examples/tutorial/calling_apis.input b/examples/tutorial/calling_apis.input deleted file mode 100644 index 4e1692caf..000000000 --- a/examples/tutorial/calling_apis.input +++ /dev/null @@ -1 +0,0 @@ -What is the weather in New York? diff --git a/examples/tutorial/calling_apis.pdl b/examples/tutorial/calling_apis.pdl deleted file mode 100644 index 29ff395ac..000000000 --- a/examples/tutorial/calling_apis.pdl +++ /dev/null @@ -1,32 +0,0 @@ -description: Using a weather API and LLM to make a small weather app -text: -- def: QUERY - text: "What is the weather in Madrid?\n" -- model: ollama_chat/granite3.2:2b - input: | - Extract the location from the question. - Question: What is the weather in London? - Answer: London - Question: What's the weather in Paris? - Answer: Paris - Question: Tell me the weather in Lagos? - Answer: Lagos - Question: ${ QUERY } - parameters: - stop_sequences: "Question,What,!,\n" - def: LOCATION - contribute: [] -- lang: python - code: | - import requests - #response = requests.get('https://api.weatherapi.com/v1/current.json?key==XYZ=${ LOCATION }') - #Mock response: - result = '{"location": {"name": "Madrid", "region": "Madrid", "country": "Spain", "lat": 40.4, "lon": -3.6833, "tz_id": "Europe/Madrid", "localtime_epoch": 1732543839, "localtime": "2024-11-25 15:10"}, "current": {"last_updated_epoch": 1732543200, "last_updated": "2024-11-25 15:00", "temp_c": 14.4, "temp_f": 57.9, "is_day": 1, "condition": {"text": "Partly cloudy", "icon": "//cdn.weatherapi.com/weather/64x64/day/116.png", "code": 1003}, "wind_mph": 13.2, "wind_kph": 21.2, "wind_degree": 265, "wind_dir": "W", "pressure_mb": 1017.0, "pressure_in": 30.03, "precip_mm": 0.01, "precip_in": 0.0, "humidity": 77, "cloud": 75, "feelslike_c": 12.8, "feelslike_f": 55.1, "windchill_c": 13.0, "windchill_f": 55.4, "heatindex_c": 14.5, "heatindex_f": 58.2, "dewpoint_c": 7.3, "dewpoint_f": 45.2, "vis_km": 10.0, "vis_miles": 6.0, "uv": 1.4, "gust_mph": 15.2, "gust_kph": 24.4}}' - def: WEATHER - parser: json - contribute: [] -- model: ollama_chat/granite3.2:2b - input: | - Explain the weather from the following JSON: - ${ WEATHER } - diff --git a/examples/tutorial/model_chaining.pdl b/examples/tutorial/calling_llm_chaining.pdl similarity index 100% rename from examples/tutorial/model_chaining.pdl rename to examples/tutorial/calling_llm_chaining.pdl diff --git a/examples/hello/hello-roles-array.pdl b/examples/tutorial/calling_llm_with_input_messages_var.pdl similarity index 100% rename from examples/hello/hello-roles-array.pdl rename to examples/tutorial/calling_llm_with_input_messages_var.pdl diff --git a/examples/hello/hello-code-command.pdl b/examples/tutorial/code_command.pdl similarity index 100% rename from examples/hello/hello-code-command.pdl rename to examples/tutorial/code_command.pdl diff --git a/examples/hello/hello-code-jinja.pdl b/examples/tutorial/code_jinja.pdl similarity index 100% rename from examples/hello/hello-code-jinja.pdl rename to examples/tutorial/code_jinja.pdl diff --git a/examples/hello/hello-code-pdl.pdl b/examples/tutorial/code_pdl.pdl similarity index 100% rename from examples/hello/hello-code-pdl.pdl rename to examples/tutorial/code_pdl.pdl diff --git a/examples/tutorial/calling_code.pdl b/examples/tutorial/code_python.pdl similarity index 100% rename from examples/tutorial/calling_code.pdl rename to examples/tutorial/code_python.pdl diff --git a/examples/tutorial/conditionals_loops.pdl b/examples/tutorial/conditionals_loops.pdl deleted file mode 100644 index 59d3be27f..000000000 --- a/examples/tutorial/conditionals_loops.pdl +++ /dev/null @@ -1,17 +0,0 @@ -description: Chatbot -text: -- read: - message: "What is your query?\n" - contribute: [context] -- repeat: - text: - - model: ollama_chat/granite3.2:2b - - read: - def: eval - message: "\nIs this a good answer[yes/no]?\n" - contribute: [] - - if: ${ eval == 'no' } - then: - read: - message: "Why not?\n" - until: ${ eval == 'yes'} diff --git a/examples/hello/hello-defs.pdl b/examples/tutorial/defs-hello.pdl similarity index 100% rename from examples/hello/hello-defs.pdl rename to examples/tutorial/defs-hello.pdl diff --git a/examples/tutorial/grouping_definitions.pdl b/examples/tutorial/defs.pdl similarity index 100% rename from examples/tutorial/grouping_definitions.pdl rename to examples/tutorial/defs.pdl diff --git a/examples/hello/hello-data.pdl b/examples/tutorial/free_variables.pdl similarity index 69% rename from examples/hello/hello-data.pdl rename to examples/tutorial/free_variables.pdl index c47b44235..2deea396a 100644 --- a/examples/hello/hello-data.pdl +++ b/examples/tutorial/free_variables.pdl @@ -1,4 +1,4 @@ -# Call with pdl --data '"something": "ABC"' hello-data.pdl +# Call with pdl --data '"something": "ABC"' free_variables.pdl description: Hello world with data text: - def: stutter diff --git a/examples/hello/hello-function-alias.pdl b/examples/tutorial/function_alias.pdl similarity index 100% rename from examples/hello/hello-function-alias.pdl rename to examples/tutorial/function_alias.pdl diff --git a/examples/hello/hello-function-empty-context.pdl b/examples/tutorial/function_empty_context.pdl similarity index 100% rename from examples/hello/hello-function-empty-context.pdl rename to examples/tutorial/function_empty_context.pdl diff --git a/examples/tutorial/function_optional_params.pdl b/examples/tutorial/function_optional_params.pdl new file mode 100644 index 000000000..2aec3814e --- /dev/null +++ b/examples/tutorial/function_optional_params.pdl @@ -0,0 +1,14 @@ +description: Hello world with function definition and call +text: +- def: hello + function: + name: str + lastName: {optional: str} # optional parameter + return: + if: ${ lastName is defined } + then: Hello ${ name } ${ lastName }! + else: Hello ${ name }! +- call: ${ hello } + args: + name: World + lastName: Universe diff --git a/examples/tutorial/gen-data.pdl b/examples/tutorial/gen-data.pdl deleted file mode 100644 index 2e9632504..000000000 --- a/examples/tutorial/gen-data.pdl +++ /dev/null @@ -1,28 +0,0 @@ -description: Creating JSON Data -defs: - data: - read: ./gen-data.yaml - parser: yaml - spec: { questions: [str], answers: [obj] } -text: - - model: replicate/ibm-granite/granite-3.1-8b-instruct - def: model_output - spec: {name: str, age: int} - input: - array: - - role: user - content: - text: - - for: - question: ${ data.questions } - answer: ${ data.answers } - repeat: | - ${ question } - ${ answer } - - > - Question: Generate only a JSON object with fields 'name' and 'age' and set them appropriately. Write the age all in letters. Only generate a single JSON object and nothing else. - parser: yaml - parameters: - stop_sequences: "Question" - temperature: 0 - diff --git a/examples/tutorial/gen-data.yaml b/examples/tutorial/gen-data.yaml deleted file mode 100644 index cb0d81d1a..000000000 --- a/examples/tutorial/gen-data.yaml +++ /dev/null @@ -1,10 +0,0 @@ -questions: - - > - Question: Write a YAML object with 2 fields 'a' and 'b' of type int and set to 0. - - > - Question: Write a YAML object with 3 fields 'number' and 'street' and 'town' set to '10', 'Miller Road', 'Armonk' respectively. - -answers: - - { "a": 0, "b": 0 } - - - { "number": 10, "street": "miller Road", "town": "armonk" } \ No newline at end of file diff --git a/examples/tutorial/ground_truth.txt b/examples/tutorial/ground_truth.txt deleted file mode 100644 index 5054ac95f..000000000 --- a/examples/tutorial/ground_truth.txt +++ /dev/null @@ -1,3 +0,0 @@ -The function `deserializeOffsetMap` takes a string as input and returns a map. It first checks if the input string is null or empty. If it is, it creates a new empty map and returns it. Otherwise, it uses the Jackson library to parse the input string into a map and returns it. - -The `@SuppressWarnings("unchecked")` annotation is used to suppress the warning that the type of the parsed map is not checked. This is because the Jackson library is used to parse the input string into a map, but the specific type of the map is not known at compile time. Therefore, the warning is suppressed to avoid potential issues. diff --git a/examples/hello/hello-if.pdl b/examples/tutorial/if.pdl similarity index 100% rename from examples/hello/hello-if.pdl rename to examples/tutorial/if.pdl diff --git a/examples/hello/hello-parser-regex.pdl b/examples/tutorial/parser-regex.pdl similarity index 100% rename from examples/hello/hello-parser-regex.pdl rename to examples/tutorial/parser-regex.pdl diff --git a/examples/tutorial/parser_regex.pdl b/examples/tutorial/parser_regex_code.pdl similarity index 100% rename from examples/tutorial/parser_regex.pdl rename to examples/tutorial/parser_regex_code.pdl diff --git a/examples/tutorial/programs/chatbot.pdl b/examples/tutorial/programs/chatbot.pdl new file mode 100644 index 000000000..ceb378672 --- /dev/null +++ b/examples/tutorial/programs/chatbot.pdl @@ -0,0 +1,24 @@ +description: Chatbot +text: +# Allow the user to type any question, implicitly adding the question to the context. +- read: + message: "What is your query?\n" +- repeat: + text: + # Send context to Granite model hosted at replicate.com + - model: ollama_chat/granite3.2:2b + # Allow the user to type 'yes', 'no', or anything else, storing + # the input into a variable named `eval`. The input is also implicitly + # added to the context. + - read: + def: eval + message: "\nIs this a good answer[yes/no]?\n" + - "\n" + # If the user only typed "no", prompt the user for input to add to the context. + - if: ${ eval == 'no' } + then: + text: + - read: + message: "Why not?\n" + # If the user typed only "yes", finish the `repeat` and end the program + until: ${ eval == 'yes'} diff --git a/examples/tutorial/data_block.pdl b/examples/tutorial/programs/code-json.pdl similarity index 100% rename from examples/tutorial/data_block.pdl rename to examples/tutorial/programs/code-json.pdl diff --git a/examples/tutorial/programs/demo-hallucination.pdl b/examples/tutorial/programs/demo-hallucination.pdl new file mode 100644 index 000000000..a9a997bd8 --- /dev/null +++ b/examples/tutorial/programs/demo-hallucination.pdl @@ -0,0 +1,90 @@ +# Granite.runtime Intrinsics Demo with PDL + +# Provide document(s) to Granite3-rag model with hallucination_tag +# Granite3-rag model: Base model granite3-dense:8b setup with Granite RAG LoRA (Low-Rank Adaption) on ollama + +# The model responds to a query providing a hallucination score + +text: +- role: system + contribute: [context] + text: + data: + instruction: | + Respond to the user's latest question based solely on the information provided + in the documents. Ensure that your response is strictly aligned with the facts + in the provided documents. If the information needed to answer the question is + not available in the documents, inform the user that the question cannot be + answered based on the available data. Make sure that your response follows + the attributes mentioned in the 'meta' field. + documents: + - doc_id: 1 + text: | + Audrey Faith McGraw (born September 21, 1967) is an American singer + and record producer. She is one of the most successful country artists + of all time, having sold more than 40 million albums worldwide. Hill is + married to American singer Tim McGraw, with whom she has recorded several duets. + Hill's first two albums, Take Me as I Am (1993) and It Matters to Me (1995), + were major successes and placed a combined three number ones on Billboard's + country charts. Hill's debut album was Take Me as I Am (1993); sales were strong, + buoyed by the chart success of "Wild One". Hill became the first female country + singer in 30 years to hold Billboard's number one position for four consecutive + weeks when "Wild One" managed the feat in 1994. Her version of "Piece of My Heart", + also went to the top of the country charts in 1994. The album sold a total of + 3 million copies. Other singles from the album include "Take Me as I Am". The recording + of Faith's second album was delayed by surgery to repair a ruptured blood vessel on + her vocal cords. It Matters to Me finally appeared in 1995 and was another + success, with the title track becoming her third number-one country single. + Several other top 10 singles followed, and more than 3 million copies of the + album were sold. The fifth single from the album, "I Can't Do That Anymore", + was written by country music artist Alan Jackson. Other singles from the album + include "You Can't Lose Me", "Someone Else's Dream", and "Let's Go to Vegas". + During this period, Hill appeared on the acclaimed PBS music program Austin City Limits. + In spring 1996, Hill began the Spontaneous Combustion Tour with country singer Tim McGraw. + At that time, Hill had recently become engaged to her former producer, Scott Hendricks, + and McGraw had recently broken an engagement. McGraw and Hill were quickly + attracted to each other and began an affair. After discovering that Hill was + pregnant with their first child, the couple married on October 6, 1996. The + couple have three daughters together: Gracie Katherine (born 1997), Maggie Elizabeth (born 1998) + and Audrey Caroline (born 2001). Since their marriage, Hill and McGraw have endeavored + never to be apart for more than three consecutive days. After the release of It Matters to Me, + Hill took a three-year break from recording to give herself a rest from four years of touring + and to begin a family with McGraw. During her break, she joined forces with her husband + for their first duet, "It's Your Love". The song stayed at number one for six weeks, + and won awards from both the Academy of Country Music and the Country Music Association. + Hill has remarked that sometimes when they perform the song together, + "it [doesn't] feel like anybody else was really watching." + meta: + hallucination_tags: true + citations: true + +# User Query +# (This query produces a hallucination "low" with citation) +- Did Faith Hill take a break from recording after releasing her second album, It Matters to Me? +# (This query produces a hallucination "unanswerable" with no citation) +# - Is the Academy of Country Music in Brooklyn, New York? +# (This query produces a hallucination "high" with a citation) +# - Where was Faith Hill born? + +# Base model granite3-dense:8b setup with Granite RAG LoRA (Low-Rank Adaption) on ollama. +- defs: + # Store the results of making the LLM invocation in a JSON variable named 'out' + out: + model: ollama/granite3-rag:8b + parameters: + temperature: 0 + parser: json +- | + + + The answer is: ${ out[0].sentence } +- match: ${out[0].meta.hallucination_level} + with: + - case: "high" + then: Totally hallucinating, sorry! + - case: "low" + if: ${ out[0].meta.citation } + then: | + I am not hallucinating, promise! + The citation is: ${ out[0].meta.citation.snippet } + - then: Not sure if I am hallucinating... diff --git a/examples/tfidf_rag/rag.pdl b/examples/tutorial/programs/tfidf_rag.pdl similarity index 97% rename from examples/tfidf_rag/rag.pdl rename to examples/tutorial/programs/tfidf_rag.pdl index cf7f78c9f..6c2acd74b 100644 --- a/examples/tfidf_rag/rag.pdl +++ b/examples/tutorial/programs/tfidf_rag.pdl @@ -43,3 +43,5 @@ text: Q: ${ TEST_PROMPT } A: - model: ollama_chat/granite3.2:2b + parameters: + temperature: 0 \ No newline at end of file diff --git a/examples/weather/weather.pdl b/examples/tutorial/programs/weather.pdl similarity index 100% rename from examples/weather/weather.pdl rename to examples/tutorial/programs/weather.pdl diff --git a/examples/hello/hello-structured-decoding.pdl b/examples/tutorial/structured_decoding.pdl similarity index 71% rename from examples/hello/hello-structured-decoding.pdl rename to examples/tutorial/structured_decoding.pdl index 1081d966e..b98d3a627 100644 --- a/examples/hello/hello-structured-decoding.pdl +++ b/examples/tutorial/structured_decoding.pdl @@ -2,8 +2,7 @@ text: - role: system text: You are an AI language model developed by IBM Research. You are a cautious assistant. You carefully follow instructions. You are helpful and harmless and you follow ethical guidelines and promote positive behavior. contribute: [context] -- "\nWhat is the color of the sky?\n" -- model: ollama_chat/granite3.2:2b - #model: watsonx/ibm/granite-34b-code-instruct +- "\nWhat is the color of the sky? Write it as JSON\n" +- model: watsonx/ibm/granite-34b-code-instruct parser: json spec: { color: str } \ No newline at end of file diff --git a/examples/hello/hello-parser-json.pdl b/examples/tutorial/type_checking.pdl similarity index 95% rename from examples/hello/hello-parser-json.pdl rename to examples/tutorial/type_checking.pdl index 4c3898c00..c521c3028 100644 --- a/examples/hello/hello-parser-json.pdl +++ b/examples/tutorial/type_checking.pdl @@ -2,7 +2,7 @@ description: Creating JSON Data defs: data: - read: hello-parser-json-data.yaml + read: type_checking_data.yaml parser: yaml spec: { questions: [str], answers: [obj] } text: diff --git a/examples/hello/hello-parser-json-data.yaml b/examples/tutorial/type_checking_data.yaml similarity index 100% rename from examples/hello/hello-parser-json-data.yaml rename to examples/tutorial/type_checking_data.yaml diff --git a/examples/hello/hello-type-list.pdl b/examples/tutorial/type_list.pdl similarity index 100% rename from examples/hello/hello-type-list.pdl rename to examples/tutorial/type_list.pdl diff --git a/tests/test_examples_run.py b/tests/test_examples_run.py index 09cb7afe2..5a389994d 100644 --- a/tests/test_examples_run.py +++ b/tests/test_examples_run.py @@ -21,32 +21,25 @@ TO_SKIP = { str(name) for name in [ + pathlib.Path("examples") / "tutorial" / "programs" / "chatbot.pdl", + pathlib.Path("examples") / "tutorial" / "programs" / "code-json.pdl", + pathlib.Path("examples") / "tutorial" / "programs" / "demo-hallucination.pdl", + pathlib.Path("examples") / "tutorial" / "programs" / "tfidf_rag.pdl", + pathlib.Path("examples") / "tutorial" / "programs" / "weather.pdl", pathlib.Path("examples") - / "hello" - / "hello-structured-decoding.pdl", # TODO: check why - pathlib.Path("examples") / "demo" / "2-teacher.pdl", # TODO: check why - pathlib.Path("examples") / "talk" / "8-tools.pdl", # TODO: check why - pathlib.Path("examples") / "talk" / "10-sdg.pdl", # TODO: check why + / "tutorial" + / "structured-decoding.pdl", # TODO: check why + pathlib.Path("examples") / "demo" / "8-tools.pdl", # TODO: check why + pathlib.Path("examples") / "demo" / "10-sdg.pdl", # TODO: check why pathlib.Path("examples") / "teacher" / "teacher.pdl", # TODO: check why pathlib.Path("examples") / "tools" / "calc.pdl", # TODO: check why - pathlib.Path("examples") / "tutorial" / "calling_apis.pdl", pathlib.Path("examples") / "cldk" / "cldk-assistant.pdl", - pathlib.Path("examples") / "talk" / "10-multi-agent.pdl", - pathlib.Path("examples") / "gsm8k" / "gsmhard-bugs.pdl", - pathlib.Path("examples") / "gsm8k" / "math-base.pdl", - pathlib.Path("examples") / "gsm8k" / "math-jinja.pdl", - pathlib.Path("examples") / "gsm8k" / "math-python.pdl", - pathlib.Path("examples") / "gsm8k" / "math.pdl", pathlib.Path("examples") / "gsm8k" / "gsm8.pdl", # TODO: check why pathlib.Path("examples") / "gsm8k" / "gsm8-plan.pdl", # TODO: check why - pathlib.Path("examples") / "tfidf_rag" / "rag.pdl", pathlib.Path("examples") / "react" / "react_call.pdl", - pathlib.Path("examples") / "callback" / "repair_prompt.pdl", - pathlib.Path("examples") / "gsm8k" / "math.pdl", - pathlib.Path("examples") / "gsm8k" / "math_no_sd.pdl", pathlib.Path("examples") / "react" / "demo.pdl", # TODO: check why - pathlib.Path("examples") / "talk" / "9-react.pdl", # TODO: check why - pathlib.Path("examples") / "demo" / "4-translator.pdl", # TODO check why + pathlib.Path("examples") / "callback" / "repair_prompt.pdl", + pathlib.Path("examples") / "demo" / "9-react.pdl", # TODO: check why pathlib.Path("examples") / "tutorial" / "calling_llm_with_input_messages.pdl", # TODO check why @@ -65,32 +58,16 @@ pathlib.Path("examples") / "tutorial" / "calling_llm_with_input.pdl", # TODO check why - pathlib.Path("examples") - / "tutorial" - / "conditionals_loops.pdl", # TODO check why - pathlib.Path("examples") - / "tutorial" - / "grouping_definitions.pdl", # TODO check why - pathlib.Path("examples") - / "granite" - / "single_round_chat.pdl", # TODO check why + pathlib.Path("examples") / "tutorial" / "defs.pdl", # TODO check why pathlib.Path("examples") / "chatbot" / "chatbot.pdl", # TODO check why pathlib.Path("examples") / "fibonacci" / "fib.pdl", # TODO check why pathlib.Path("examples") / "intrinsics" / "demo-hallucination.pdl", # TODO check why pathlib.Path("examples") - / "hello" - / "hello-function-empty-context.pdl", # TODO CREATE RESULTS FILE - pathlib.Path("examples") / "hello" / "hello-roles-array.pdl", # TODO check why - pathlib.Path("examples") / "hello" / "hello-import.pdl", # TODO check why - pathlib.Path("examples") - / "hello" - / "hello-import-lib.pdl", # (Produces no output) - pathlib.Path("examples") - / "hello" - / "hello-model-chaining.pdl", # TODO check why - pathlib.Path("examples") / "talk" / "7-chatbot-roles.pdl", # TODO check why + / "tutorial" + / "function-empty-context.pdl", # TODO CREATE RESULTS FILE + pathlib.Path("examples") / "demo" / "7-chatbot-roles.pdl", # TODO check why pathlib.Path("examples") / "rag" / "pdf_index.pdl", # TODO: check what the expected output is @@ -111,7 +88,6 @@ pathlib.Path("examples") / "granite-io" / "granite_io_openai.pdl", pathlib.Path("examples") / "granite-io" / "granite_io_thinking.pdl", pathlib.Path("examples") / "granite-io" / "granite_io_transformers.pdl", - pathlib.Path("examples") / "hello" / "hello-graniteio.pdl", ] } @@ -119,26 +95,17 @@ str(name) for name in [ pathlib.Path("examples") / "weather" / "weather.pdl", - pathlib.Path("examples") / "demo" / "3-weather.pdl", - pathlib.Path("examples") / "granite" / "multi_round_chat.pdl", pathlib.Path("examples") / "react" / "demo.pdl", - pathlib.Path("examples") / "react" / "wikipedia.pdl", pathlib.Path("examples") / "code" / "code.pdl", pathlib.Path("examples") / "code" / "code-eval.pdl", pathlib.Path("examples") / "code" / "code-json.pdl", - pathlib.Path("examples") / "talk" / "1-hello.pdl", - pathlib.Path("examples") / "talk" / "2-model-chaining.pdl", - pathlib.Path("examples") / "talk" / "3-def-use.pdl", - pathlib.Path("examples") / "talk" / "5-code-eval.pdl", - pathlib.Path("examples") / "talk" / "6-code-json.pdl", - pathlib.Path("examples") / "talk" / "9-react.pdl", - pathlib.Path("examples") / "tutorial" / "include.pdl", - pathlib.Path("examples") / "tutorial" / "data_block.pdl", + pathlib.Path("examples") / "demo" / "1-hello.pdl", + pathlib.Path("examples") / "demo" / "2-model-chaining.pdl", + pathlib.Path("examples") / "demo" / "3-def-use.pdl", + pathlib.Path("examples") / "demo" / "5-code-eval.pdl", + pathlib.Path("examples") / "demo" / "6-code-json.pdl", + pathlib.Path("examples") / "demo" / "9-react.pdl", pathlib.Path("examples") / "sdk" / "hello.pdl", - pathlib.Path("examples") / "hello" / "hello.pdl", - pathlib.Path("examples") / "hello" / "hello-model-input.pdl", - pathlib.Path("examples") / "hello" / "hello-parser-regex.pdl", - pathlib.Path("examples") / "hello" / "hello-def-use.pdl", ] } @@ -152,9 +119,6 @@ class InputsType: TESTS_WITH_INPUT: dict[str, InputsType] = { str(name): inputs for name, inputs in { - pathlib.Path("examples") - / "demo" - / "4-translator.pdl": InputsType(stdin="french\nstop\n"), pathlib.Path("examples") / "tutorial" / "input_stdin.pdl": InputsType(stdin="Hello\n"), @@ -171,21 +135,11 @@ class InputsType: / "chatbot" / "chatbot.pdl": InputsType(stdin="What is APR?\nyes\n"), pathlib.Path("examples") - / "talk" + / "demo" / "7-chatbot-roles.pdl": InputsType(stdin="What is APR?\nquit\n"), pathlib.Path("examples") - / "granite" - / "single_round_chat.pdl": InputsType( - scope=PdlDict({"PROMPT": "What is APR?\nyes\n"}) - ), - pathlib.Path("examples") - / "hello" - / "hello-data.pdl": InputsType(scope=PdlDict({"something": "ABC"})), - pathlib.Path("examples") / "tutorial" - / "conditionals_loops.pdl": InputsType( - stdin="What is APR?\nno\nSay it as a poem\nyes\n" - ), + / "free_variables.pdl": InputsType(scope=PdlDict({"something": "ABC"})), }.items() } @@ -202,12 +156,8 @@ class InputsType: ] EXPECTED_RUNTIME_ERROR = [ - pathlib.Path("examples") / "demo" / "1-gen-data.pdl", - pathlib.Path("examples") / "tutorial" / "gen-data.pdl", - pathlib.Path("examples") / "hello" / "hello-type-code.pdl", - pathlib.Path("examples") / "hello" / "hello-type-list.pdl", - pathlib.Path("examples") / "hello" / "hello-type.pdl", - pathlib.Path("examples") / "hello" / "hello-parser-json.pdl", + pathlib.Path("examples") / "tutorial" / "type_list.pdl", + pathlib.Path("examples") / "tutorial" / "type_checking.pdl", pathlib.Path("tests") / "data" / "line" / "hello12.pdl", pathlib.Path("tests") / "data" / "line" / "hello13.pdl", pathlib.Path("tests") / "data" / "line" / "hello14.pdl",