Skip to content

Add tl-install to install tools not in pio registry. Using for install of newer OpenOCD #148

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 14 commits into from
Apr 28, 2025

Conversation

Jason2866
Copy link

@Jason2866 Jason2866 commented Apr 27, 2025

Description:

Currently it is only possible to install tools and toolchains from Platformio registry.
The PR adds the option to install tools from a github repo release section. To make this possible a tarball needs to be created which has a valid platformio package.json (not platform specific!) and a manifest json with name tools.json in it. The syntax for the entrys in tools.json is explained here The link to the tarball is added in the platform.json under the tool "version" entry. Make the tool option entry "optional" = True
For installing and activating the tool via the alternative installer it simply needs to be called like this install_tool("tool-openocd-esp32") in platform.py. Now the tool tool-openocd-esp32 will be installed (matching to the OS) from the espressif github OpenOCD repo and can be used like installed from Platformio registry.

Checklist:

  • The pull request is done against the latest develop branch
  • Only relevant files were touched
  • Only one feature/fix was added per PR, more changes are allowed when changing boards.json
  • I accept the CLA

Summary by CodeRabbit

  • Chores
    • Updated package metadata and tool management for improved compatibility and maintenance, including new tool additions, ownership changes, and version updates.
    • Adjusted configuration to enable debug build type for specific ESP32 environment.
    • Improved handling of tool installation and activation, with refined package enabling logic and expanded MCU support.
  • Bug Fixes
    • Enhanced address matching in exception decoder to work in more scenarios by updating the pattern matching logic.

@Jason2866
Copy link
Author

@coderabbitai full review

Copy link

coderabbitai bot commented Apr 27, 2025

✅ Actions performed

Full review triggered.

Copy link

coderabbitai bot commented Apr 27, 2025

Walkthrough

The changes introduce updates across configuration, platform manifest, exception decoding, and platform logic. The platformio.ini file for the ESP32-C6 environment now explicitly sets the build type to debug. The platform manifest (platform.json) receives metadata and dependency updates, including new tool packages, version bumps, and keyword adjustments. The ESP32 exception decoder's address-matching regex is relaxed to match addresses anywhere in a line. The core platform logic in platform.py is refactored to use a new helper for automated tool installation, conditional package enabling, and updated MCU handling.

Changes

File(s) Summary
examples/arduino-blink/platformio.ini Added build_type = debug to the [env:esp32-c6-devkitc-1] environment, enabling debug builds for this configuration.
monitor/filter_exception_decoder.py Modified the ADDR_PATTERN regex to match address sequences anywhere in a line, not just at the end.
platform.json Updated package metadata: keyword change, engine version bump, new and updated tool packages, ownership changes, and optional flags.
platform.py Refactored configure_default_packages: added automated tool installation, improved conditional enabling, and updated MCU checks.

Sequence Diagram(s)

sequenceDiagram
    participant User
    participant Platform
    participant idf_tools.py
    participant ToolPackageManager

    User->>Platform: Initiate build or debug/upload
    Platform->>Platform: Check build type, debug/upload protocol, MCU
    alt Tool installation required
        Platform->>idf_tools.py: Run install_tool(TOOL)
        idf_tools.py-->>Platform: Install/activate tool
        Platform->>ToolPackageManager: Copy package metadata, clean up old dirs
    end
    Platform->>User: Proceed with build/upload/debug
Loading

Poem

In the warren where code does dwell,
New tools arrive, and configs swell.
Debug flags set, the regex freed,
Tools installed with rabbit speed!
MCU lists now up to date—
Platformio, feeling great!
🐇✨

✨ Finishing Touches
  • 📝 Generate Docstrings

🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR.
  • @coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🧹 Nitpick comments (4)
examples/arduino-blink/platformio.ini (1)

93-98: build_type = debug will grow firmware considerably

Great for development, but shipping this example with permanent debug builds may surprise newcomers (flash overflow & slower compile times).

Two easy fixes:

-[env:esp32-c6-devkitc-1]
+; Enable debug only when the user explicitly selects the *debug* build profile
+[env:esp32-c6-devkitc-1]
+;build_type = release        ; default
+# Uncomment the next line to switch to debug
+# build_type = debug

or add a dedicated [env:esp32-c6-devkitc-1-debug] environment.

platform.py (3)

15-25: Import style – avoid from os.path import join shadowing

join() is used only once and already referenced via os.path.join elsewhere. Mixing both styles is error-prone and confuses static analysers.

-import shutil
-from os.path import join
+import shutil

and replace the lone join( call with os.path.join(.


90-93: tl-install de-activation logic can mis-fire

tl-install is marked optional only after checking idf_tools.py existence inside the package.
On the first run the directory doesn’t exist yet → the package is forced to install twice (once via PlatformIO, once via pm.install).
Consider flipping the logic:

  1. Leave tl-install optional in platform.json.
  2. In configure_default_packages, if the script is absent, set optional=False.

This prevents redundant downloads.


163-166: MCU list hard-codes every future variant

The tuple grows each release. A more maintainable approach:

RISC_V_MCUS = ("esp32c", "esp32h2", "esp32p4")
if mcu.startswith(RISC_V_MCUS):
    ...

Consider using regex or family prefix detection to avoid constant updates.

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 2bdbd35 and ced2c63.

📒 Files selected for processing (4)
  • examples/arduino-blink/platformio.ini (1 hunks)
  • monitor/filter_exception_decoder.py (1 hunks)
  • platform.json (4 hunks)
  • platform.py (4 hunks)
🔇 Additional comments (2)
platform.json (2)

86-97: ⚠️ Potential issue

Un-pinned GitHub ZIPs hurt reproducibility & security

Several new tools (tool-esptoolpy, tl-install, tool-openocd-esp32, …) reference mutable master.zip/latest.zip-style URLs. A later retag/force-push will silently change the bits and break deterministic builds or introduce supply-chain risk.

Consider:

-"version": "https://github.com/espressif/arduino-esp32/archive/master.zip"
+"version": "https://github.com/espressif/arduino-esp32/archive/3ad36321e9c07cb9c0e.zip"  # commit hash

or attach SHA-256 in package.json and use PlatformIO’s checksum field.

Likely an incorrect or invalid review comment.


152-169:

✅ Verification successful

Potential JSON duplication / trailing comma risk

The long "packages" map now ends with "tool-scons" (line 165-168). Please double-check:

  1. No duplicate keys (tool-scons was already present earlier?).
  2. Last item has no trailing comma; otherwise json.loads will fail during platform installation.

Run:

to verify validity in CI.


🏁 Script executed:

python -m json.tool platform.json

Length of output: 5460


JSON validity confirmed – no duplicate keys or trailing commas

Verified with python -m json.tool platform.json that:

  • No duplicate entries for "tool-scons" exist.
  • The final "tool-scons" block ends without a trailing comma, keeping the JSON valid.

No changes required.

@Jason2866 Jason2866 merged commit f8ec6d4 into develop Apr 28, 2025
131 of 132 checks passed
This was referenced Apr 28, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant