Standardize Environment Directory Structure #145 #160

rycerzes · 2025-11-06T20:55:08Z

This PR is for the issue Standardize Environment Directory Structure #145 . Major changes to dependency management, Docker build automation, and CLI usability for OpenEnv environments. The changes standardize the use of pyproject.toml for dependencies, add scripts for generating Docker-compatible requirements, and enhance the CLI with new commands and validation utilities. The Docker base image is also updated to use uv for faster dependency handling.

Dependency Management and Build Automation

Added pyproject.toml as the primary dependency specification for environments, clarified usage in README.md, and updated the root and core pyproject.toml files to reflect modern Python and dependency requirements. [1] [2] [3]
Introduced scripts/build_docker.py to automate Docker image building, including requirements generation and optional image pushing to registries.

Docker Image Improvements

Updated the OpenEnv base Dockerfile to use uv for dependency installation, improved caching, and modernized the build process for faster and more reliable builds.

CLI Enhancements

Extended the CLI to include new commands: convert, serve, build and validate, in addition to init and push. [1] [2]
- The convert command is a temporary script which will help generate the boilerplate code required for making the existing envs compatible with the new structure
- The serve command is for multi-mode deployment i.e., expose a pure python server runtime with dependency management without docker
Added a utility function validate_env_structure to check environment directory structure and provide helpful warnings or errors, supporting the new validate command.

TODOs:

for now using local docker image, update that once validated
update gh actions for this as well
remove the temporary convert command once all envs are migrated
remove the redundant requirements.txt from the existing envs
add checks to make sure all new existing and new envs follow the standard
refactor the app entrypoint in the envs to support uv run or as a module
update docs
Docker builds as part of project scripts
Testing with templatized tests as part of project scripts.

meta-cla · 2025-11-06T20:55:14Z

Hi @rycerzes!

Thank you for your pull request and welcome to our community.

Action Required

In order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you.

Process

In order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA.

Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with CLA signed. The tagging process may take up to 1 hour after signing. Please give it that time before contacting us about it.

If you have received this in error or have any questions, please contact us at [email protected]. Thanks!

rycerzes · 2025-11-06T20:58:59Z

@zkwentz @burtenshaw @init27 @jspisak this is the PR for #145
For now, I have created a draft. Please let me know, what changes will I need to make

meta-cla · 2025-11-06T20:59:20Z

Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Meta Open Source project. Thanks!

zkwentz · 2025-11-07T03:18:13Z

Amazing @rycerzes I took a cursory look and impressive to see convert in the list of newly added commands! I need to get my hands dirty with this one to provide a proper review for ya, will do so this weekend.

One immediate thing that stands out, can you move your build-docker script to openenv build?

And thank you for jumping on this so fast, you're helping us get out of the woods :)

rycerzes · 2025-11-07T05:31:51Z

One immediate thing that stands out, can you move your build-docker script to openenv build?

Added 👍

burtenshaw · 2025-11-07T07:47:56Z

Really nice draft PR @rycerzes. Thanks for the contribution. I haven't gone team yet I'm just exploring at a high level. My first question is why do we need convert?

I understand that we need to do a one time conversion of the legacy envs in this repo and in the wild, but I'm not sure if we need to maintain that code within the cli. We could just keep the functionality within ./scripts and do the conversion. IMO, this would be ok because this is still an experimental project.

rycerzes · 2025-11-07T11:49:58Z

My first question is why do we need convert?

It's a temporary command, which I will remove before the PR is merged. I had previously planned to migrate the existing envs in a different PR but it would be wise to update the envs in this PR itself

zkwentz · 2025-11-07T19:36:52Z

Okay played around with things a bit. Love the changes you have made to standard template.

One thing I think we haven't made clear enough, and we should make clear once this PR is merged is this:

The new approach is for this repository to be the specification and CLI repo with some canonical examples of interesting examples (criteria for a good example environment is to demonstrate new specification or openenv core functionality).

Going forward, all other environments, will be assumed to have been started with openenv init and will live in their own repo. openenv push will allow you to push that environment (assuming it is conforming ... openenv validate) to the Hugging Face Env Hub with appropriate tagging and build for discovery on Hugging Face itself.

All CLI commands will be assumed to be running from the root of an openenv initialized environment.

So for your newly added commands can you remove the <env_name> argument? As in:

openenv build <env_name> -> openenv build
openenv validate <env_name> -> openenv validate
openenv start <env_name> -> openenv start

etc.

Additional feedback:

+1 to removing convert, it's cool to see but a one-time tax to pay with the new CLI-based workflow going forward.

I'll add inline comments now that are in pursuit of this direction.

zkwentz · 2025-11-07T19:38:11Z

src/openenv_cli/commands/build.py

+
+    Examples:
+        # Build echo_env with default settings
+        $ openenv build src/envs/echo_env


Change to $ openenv build

zkwentz · 2025-11-07T19:40:01Z

src/openenv_cli/commands/build.py

+        # Build with custom tag
+        $ openenv build src/envs/echo_env -t my-custom-tag
+
+        # Build and push to registry


dig this, but move to push. in other words: openenv push --registry myregistry.io/<path1>/<path2> etc.

zkwentz · 2025-11-07T19:40:48Z

src/openenv_cli/commands/build.py

+        $ openenv build src/envs/echo_env
+
+        # Build with custom tag
+        $ openenv build src/envs/echo_env -t my-custom-tag


love it, for completion's sake: openenv build -t my-custom-tag

zkwentz · 2025-11-07T19:41:37Z

src/openenv_cli/commands/serve.py

+
+    Examples:
+        # Start echo_env on default port
+        openenv serve echo_env


openenv serve echo_env -> openenv serve and so on

zkwentz · 2025-11-07T19:41:51Z

src/openenv_cli/commands/validate.py

+
+
+def validate(
+    env_name: str = typer.Argument(


zkwentz · 2025-11-07T19:42:08Z

src/openenv_cli/commands/validate.py

+
+    Examples:
+        # Validate echo_env
+        openenv validate echo_env


openenv validate echo_env -> openenv validate

zkwentz · 2025-11-07T19:42:31Z

src/openenv_cli/_validation.py

+    5. Required dependencies are present
+
+    Args:
+        env_path: Path to the environment directory


zkwentz · 2025-11-07T19:43:39Z

README.md

+
+```bash
+# Install environment in editable mode
+pip install -e ./my_env


pip install -e ./my_env -> pip install -e .

zkwentz · 2025-11-07T19:43:53Z

README.md

+pip install -e ./my_env
+
+# Or using uv (faster)
+uv pip install -e ./my_env


uv pip install -e ./my_env -> uv pip install -e .

zkwentz · 2025-11-07T19:44:08Z

README.md

+uv pip install -e ./my_env
+
+# Run server locally without Docker
+uv run --project ./my_env server --host 0.0.0.0 --port 8000


uv run --project ./my_env server --host 0.0.0.0 --port 8000 -> uv run --project . server --host 0.0.0.0 --port 8000

zkwentz · 2025-11-07T19:49:29Z

One last thing, though you are killing off openenv convert command, we do need good conversion instructions to land with this standard.

@rycerzes since you're closest to the new standard, can you write up CONVERT.md instructions for environment authors to follow to convert their current src/envs/<env_name> to their own separate environment that is openenv_cli compatible (e.g. it can be openenv pushed).

Things to cover that come to mind:

Manually adding HF frontmatter to their readme for openenv push
Changes to dockerfile
Changes to app.py
Migrate from requirements.txt -> pyproject.toml (if applicable)
some quick inline script to move their environment to its own directory with new git repo instantiated

rycerzes · 2025-11-08T13:41:41Z

@zkwentz Thanks for the detailed review! I understand the new CLI-first approach and will make all the requested changes

Questions:

For the registry push in openenv push --registry, should this be a separate flag or integrated with the existing push functionality?
Any specific format preferences for the CONVERT.md structure?

I'll have these changes ready in some time :)

zkwentz · 2025-11-09T17:59:51Z

For the registry push in openenv push --registry, should this be a separate flag or integrated with the existing push functionality?

I see what you're saying, it's a bit wonky because we do a custom build for Hugging Face... it will be like that for some time as well. It's making sense to me now why you included a registry push on build instead of push itself.

Would it be weird to have three optional arguments:

--interface (optional, default: false if registry present)
--no-interface (optional, default: true if registry present)
--registry (optional, url, default: null)

Resulting in the following options:

openenv push - pushes to HF Env Hub with web interface (default behavior)
openenv push --no-interface pushes to HF Env Hub without web interface
openenv push --registry <registry> pushes to a custom registry without web interface
openenv push --registry <registry> --interface pushed to a custom registry with web interface
openenv push --registry <registry> --no-interface pushed to a custom registry without web interface

Any specific format preferences for the CONVERT.md structure?

No preferences :) Thank you!

rycerzes added 5 commits November 6, 2025 21:42

root-level structure and tooling

0dbaf14

reusable templates

436041f

implement openenv serve for multi mode deployment

cb0f0c0

environment setup with uv integration and template generation

596fd4d

cleanup

ae198f7

meta-cla bot added the CLA Signed This label is managed by the Meta Open Source bot. label Nov 6, 2025

zkwentz self-assigned this Nov 7, 2025

add build command to the cli

cb1e88a

burtenshaw mentioned this pull request Nov 7, 2025

[FEATURE] uv provider #164

Open

zkwentz requested changes Nov 7, 2025

View reviewed changes

Standardize Environment Directory Structure #145 #160

Are you sure you want to change the base?

Standardize Environment Directory Structure #145 #160

Uh oh!

Conversation

rycerzes commented Nov 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

TODOs:

Uh oh!

meta-cla bot commented Nov 6, 2025

Action Required

Process

Uh oh!

rycerzes commented Nov 6, 2025

Uh oh!

meta-cla bot commented Nov 6, 2025

Uh oh!

zkwentz commented Nov 7, 2025

Uh oh!

rycerzes commented Nov 7, 2025

Uh oh!

burtenshaw commented Nov 7, 2025

Uh oh!

rycerzes commented Nov 7, 2025

Uh oh!

zkwentz commented Nov 7, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

zkwentz commented Nov 7, 2025

Uh oh!

rycerzes commented Nov 8, 2025

Uh oh!

zkwentz commented Nov 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

rycerzes commented Nov 6, 2025 •

edited

Loading

zkwentz commented Nov 9, 2025 •

edited

Loading