fix: Fix variable extraction in Jinja2 templates returning set variables

[sjrl]

Related Issues

• fixes PromptBuilder incorrectly marks internally set Jinja2 variables as required inputs when using `required_variables='*' #9916

Proposed Changes:

While investigating #9916 I found this open issue pallets/jinja#2069 in the Jinja2 library which is the cause of our issue.

I've developed a partial workaround by introducing the utility method _collect_assigned_variables. The work around is partial because it does not exhaustively identify all types of assignment. It only covers regular assignment, list and tuple assignments.

Unfortunately the suggested solution in the issue in the Jinja2 library of pinning the version to jinja2<3.1.5 does not work. I've left a follow-up comment on their issue.

How did you test it?

Added new tests.

Notes for the reviewer

Checklist

• I have read the contributors guidelines and the code of conduct.
• I have updated the related issue with new insights and changes.
• I have added unit tests and updated the docstrings.
• I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test: and added ! in case the PR includes breaking changes.
• I have documented my code.
• I have added a release note file, following the contributors guidelines.
• I have run pre-commit hooks and fixed any issue.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Preview	Comments	Updated (UTC)
haystack-docs	Ready	Preview	Comment	Dec 3, 2025 9:18am

[sjrl]

Closing in favor of a new PR to pin jinja2 after an offline discussion

[sjrl]

Reopening since we found that pinning jinja2 to 3.1.4 doesn't fully solve the issue. Find more information here pallets/jinja#2069 (comment)

[coveralls]

Pull Request Test Coverage Report for Build 19852408366

Details

• 0 of 0 changed or added relevant lines in 0 files are covered.
• 15 unchanged lines in 3 files lost coverage.
• Overall coverage increased (+0.03%) to 92.192%

---

Files with Coverage Reduction	New Missed Lines	%
components/converters/output_adapter.py	1	98.55%
components/builders/chat_prompt_builder.py	2	98.29%
components/routers/conditional_router.py	12	88.46%

Totals
Change from base Build 19827115579:	0.03%
Covered Lines:	14063
Relevant Lines:	15254

---

💛 - Coveralls

[vblagoje]

@sjrl the only thing that comes to mind here is to extract free standing method:

def extract_template_variables_and_assignments(env: Environment, template: str) -> tuple[set[str], set[str]]:
    """
    Extract variables from a Jinja2 template and variables assigned within it.
    
    :param env: A Jinja2 environment.
    :param template: A Jinja2 template string.
    :returns: A tuple of (assigned_variables, template_variables) where:
        - assigned_variables: Variables assigned within the template (e.g., via {% set %})
        - template_variables: All undeclared variables used in the template
    """
    jinja2_ast = env.parse(template)
    template_variables = meta.find_undeclared_variables(jinja2_ast)
    assigned_variables = _collect_assigned_variables(jinja2_ast)
    return assigned_variables, template_variables

and then use it across builders, adapters, and routers!

[sjrl]

@sjrl the only thing that comes to mind here is to extract free standing method:

def extract_template_variables_and_assignments(env: Environment, template: str) -> tuple[set[str], set[str]]:
    """
    Extract variables from a Jinja2 template and variables assigned within it.
    
    :param env: A Jinja2 environment.
    :param template: A Jinja2 template string.
    :returns: A tuple of (assigned_variables, template_variables) where:
        - assigned_variables: Variables assigned within the template (e.g., via {% set %})
        - template_variables: All undeclared variables used in the template
    """
    jinja2_ast = env.parse(template)
    template_variables = meta.find_undeclared_variables(jinja2_ast)
    assigned_variables = _collect_assigned_variables(jinja2_ast)
    return assigned_variables, template_variables

and then use it across builders, adapters, and routers!

Good point! I'll probably mark it as a private method since it doesn't cover every case and I'd hope this will be properly fixed by the Jinja2 library.