Skip to content

Error analysis fails when PDL doesn't match schema #850

@vite-falcon

Description

@vite-falcon

Summary

When the PDL doesn't match the schema, the error analysis fails as it attempts to retrieve the field names without verifying that "properties" key exists in ref_type.

Sample PDL

description: Vision model example
text:
- model: openai/gpt-4o-mini
  input:
    array:
    - role: user
      content:
      - type: text
        text: "What's in this image?"
      - type: image_url
        image_url:
          url: "https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg"
  parameters:
    max_tokens: 500
    temperature: 0.7

Expected Error

['from-string:8 - [{\'type\': \'text\', \'text\': "What\'s in this image?"}, {\'type\': \'image_url\', \'image_url\': {\'url\': \'https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg\'}}] should not be a list'].

Actual Error

'properties' 

File "[...]\noxfile.py", line 85, in pdl_lint
    PdlExecutor.validate_pdl_file(pdl_file)
  File "[...]\pdl\executor.py", line 192, in validate_pdl_file
    PdlExecutor.validate_pdl_string(pdl_file.read_text(encoding="utf-8"))
  File "[...]\pdl\executor.py", line 185, in validate_pdl_string
    _, _ = parse_pdl_str(pdl_string, file_name=file_name)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "[...]\.venv\Lib\site-packages\pdl\pdl_parser.py", line 39, in parse_str
    errors = analyze_errors(defs, defs["Program"], prog_yaml, loc)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "[...]\.venv\Lib\site-packages\pdl\pdl_schema_error_analyzer.py", line 188, in analyze_errors
    ret += analyze_errors(defs, match_ref, data, loc)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "[...]\.venv\Lib\site-packages\pdl\pdl_schema_error_analyzer.py", line 117, in analyze_errors
    ret += analyze_errors(
           ^^^^^^^^^^^^^^^
  File "[...]\.venv\Lib\site-packages\pdl\pdl_schema_error_analyzer.py", line 157, in analyze_errors
    ret += analyze_errors(defs, found, data, loc)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "[...]\.venv\Lib\site-packages\pdl\pdl_schema_error_analyzer.py", line 91, in analyze_errors
    ret += analyze_errors(defs, schema["items"], item, newloc)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "[...]\.venv\Lib\site-packages\pdl\pdl_schema_error_analyzer.py", line 188, in analyze_errors
    ret += analyze_errors(defs, match_ref, data, loc)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "[...]\.venv\Lib\site-packages\pdl\pdl_schema_error_analyzer.py", line 117, in analyze_errors
    ret += analyze_errors(
           ^^^^^^^^^^^^^^^
  File "[...]\.venv\Lib\site-packages\pdl\pdl_schema_error_analyzer.py", line 167, in analyze_errors
    field_matches = match(item, data)
                    ^^^^^^^^^^^^^^^^^
  File "[...]\.venv\Lib\site-packages\pdl\pdl_schema_error_analyzer.py", line 51, in match
    all_fields = ref_type["properties"].keys()
                 ~~~~~~~~^^^^^^^^^^^^^^

Proposal

The fix that seems to work is to change line 51 in pdl_schema_error_analyzer.py to:

all_fields = ref_type.get("properties", {}).keys()

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions