Avoid using the manifest lexer on YAML #239
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
|
A YAML file like in the fixture added is parsed fine by Psych and yamllint, but puppet-lint chokes on it because it gets fed through the manifest lexer first, which can't cope with some things that are valid YAML. Since the one check that operates on YAML files doesn't support fixing and doesn't use tokens, it seems reasonable to bypass the lexer; more sophisticated YAML checks would need to use a separate YAML lexer anyway. Note that this can't be a unit test because those bypass the lexer processing.
tokenrove
force-pushed
the
fix-yaml-syntax-errors
branch
from
92530b7 to
1b8f597
Compare
|
(Sorry, obvious typo in my initial push; I did the initial change and testing on a work machine, but couldn't send that directly to my personal machine to make this contribution. I thought I retested here before pushing but clearly I didn't.) |
david22swan
approved these changes
Mar 10, 2025
alex501212
approved these changes
Mar 10, 2025
|
Thanks! |
|
Any plan to release new version with this fix? I would like to lock version instead of commit id for this. |
bastelfreak
added a commit
to bastelfreak/voxpupuli-puppet-lint-plugins
that referenced
this pull request
Sep 25, 2025
This pulls in three bigger changes in puppet-lint: * Avoid using the manifest lexer on YAML [#239](puppetlabs/puppet-lint#239) (tokenrove) * Add abstract data types to lexer type tokens list [#240](puppetlabs/puppet-lint#240) (kenyon) * Fix Ensure check methods can't modify tokens array and add Ruby 3.4 testing [#233](puppetlabs/puppet-lint#233) (alexjfisher) In a previous PR, voxpupuli#64, we switched to Ruby 3.2 as a minimal Ruby version. The new puppet-lint release did a Ruby version bump. We don't want to end up in a situation where different users have different ruby versions, which leads to different puppet-lint versions, which leads to different linting of your puppet code. Because of that, we cannot allow puppet-lint 4 & 5, but have to pint to 5.1 (5.0 doesn't support Ruby 3.4). All puppet-lint plugins were also updated for puppet-lint 5.1, which also required major releases.
bastelfreak
added a commit
to bastelfreak/voxpupuli-puppet-lint-plugins
that referenced
this pull request
Sep 25, 2025
This pulls in three bigger changes in puppet-lint: * Avoid using the manifest lexer on YAML [#239](puppetlabs/puppet-lint#239) (tokenrove) * Add abstract data types to lexer type tokens list [#240](puppetlabs/puppet-lint#240) (kenyon) * Fix Ensure check methods can't modify tokens array and add Ruby 3.4 testing [#233](puppetlabs/puppet-lint#233) (alexjfisher) In a previous PR, voxpupuli#64, we switched to Ruby 3.2 as a minimal Ruby version. The new puppet-lint release did a Ruby version bump. We don't want to end up in a situation where different users have different ruby versions, which leads to different puppet-lint versions, which leads to different linting of your puppet code. Because of that, we cannot allow puppet-lint 4 & 5, but have to pint to 5.1 (5.0 doesn't support Ruby 3.4). All puppet-lint plugins were also updated for puppet-lint 5.1, which also required major releases.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
The new feature of checking YAML files chokes on valid YAML files; this change attempts to mitigate that problem.
Additional Context
PR #235 added scanning of YAML files to puppet-lint; to my surprise, my CI started failing with a number of syntax errors that hadn't been there before. Digging into it, it seems to be because the YAML files get fed through
PuppetLint::Lexer, which doesn't handle some YAML literals correctly (and, being designed for Puppet manifests, I wouldn't expect it to).Note that a YAML file as included in the change is parsed fine by Psych and yamllint, but puppet-lint chokes on it because it gets fed through the manifest lexer first, which can't cope with some things that are valid YAML. I have larger examples but this was the smallest reduction I could make that still failed.
Since the one check that operates on YAML files doesn't support fixing and doesn't use tokens, it seems reasonable to bypass the lexer; more sophisticated YAML checks would need to use a separate YAML lexer anyway.
Note that this can't be a unit test because those bypass the lexer processing. (I only, afterwards, noticed the test for this in
checks_spec-- I leave it up to you, maintainers, to let me know if you'd rather I put the heredoc case there.)Checklist