feat(shell): add custom Base Image support for Bot Shell #222

qdaxb · 2025-11-29T16:58:48Z

Summary

Add baseImage and baseShellRef fields to ShellSpec schema for custom shell configuration
Create Shell unified API endpoints (/api/shells/unified, CRUD operations, image validation)
Implement Init Container pattern: extract executor binary to Named Volume on Executor Manager startup
Support custom Base Image mode in Docker executor with executor binary mount
Add frontend Shell management UI with validation support and i18n
Refactor: Use executor flow for async image validation (extensible for Docker/K8s)

Key Changes

Backend

Extended ShellSpec schema with baseImage and baseShellRef fields
Created /api/shells/unified endpoint for unified shell listing (public + user-defined)
Added /api/shells/validate-image endpoint for Base Image compatibility validation (async)
Modified task dispatch to include base_image in bot configuration for Executor Manager

Executor

Created ImageValidatorAgent for running validation checks inside the target container
Agent registered in factory.py and uses report_progress to send results via callback
Extended callback_handler.py to support result field in completed callbacks

Executor Manager

Added binary_extractor.py for extracting executor binary from official image to Named Volume
Executor Manager extracts binary on startup and records version for incremental updates
Docker executor supports custom Base Image mode with executor binary mount (-v wegent-executor-binary:/app:ro)
Refactored /executor-manager/images/validate to submit validation tasks via task processor (async)

Frontend

Created Shell API client (shells.ts) with full CRUD and validation support
Added ShellList and ShellEdit components for shell management
Updated validation UI to handle async validation status (submitted, in_progress, success, error)
Integrated Shells tab into Settings page with i18n support (en/zh-CN)

Architecture: Async Image Validation

The image validation now uses the actual executor run flow:

Frontend calls /api/shells/validate-image
Backend proxies to Executor Manager
Executor Manager creates a validation task with agent_name: "ImageValidator" and target base_image
Docker executor starts container with custom base_image and mounts executor binary
ImageValidatorAgent runs validation checks inside the container
Results are reported back via callback mechanism with result field

This approach:

Works for both Docker and Kubernetes deployment modes
Validates inside the actual target image
Uses existing task/callback infrastructure
Returns async results (extensible for real-time status updates)

Test plan

Verify public shells display correctly in Settings > Shells tab
Create a custom shell with a valid base image
Test image validation for ClaudeCode and Agno shell types
Verify validation task submission and async response
Execute a task with custom shell and verify executor binary mount
Test shell update and delete operations

- Add baseImage and baseShellRef fields to ShellSpec schema - Update public shells init data with default baseImage - Create Shell unified API endpoints (/shells/unified, CRUD operations) - Add image validation API to check compatibility with shell types - Modify task dispatch to pass baseImage to Executor Manager - Add executor binary extraction on Executor Manager startup - Support Base Image + Executor mount mode in Docker executor - Create frontend Shell API client and management UI components - Add Shells tab to Settings page with i18n support The feature enables users to create custom shells with their own base images while using the latest executor binary via Named Volume mount.

coderabbitai · 2025-11-29T16:58:58Z

Walkthrough

Adds a new shell management feature spanning backend API endpoints, executor integration, and frontend UI. Introduces unified shell representations combining public and user-defined shells, image validation against shell dependencies, Docker binary extraction to a named volume, and custom base image support in task execution.

Changes

Cohort / File(s)	Summary
Backend API Router Registration `backend/app/api/api.py`	Registers the new shells router at `/shells` endpoint
Shell Management Endpoints & Models `backend/app/api/endpoints/adapter/shells.py`	New FastAPI router with full CRUD endpoints for unified shells (public and user-defined), shell request/response models, conversion helpers, and image validation orchestration; integrates with user authentication and Kind object persistence
Schema Extensions `backend/app/schemas/kind.py`	Adds `baseImage` and `baseShellRef` optional fields to ShellSpec for custom base image and public shell reference support
Service Logic Updates `backend/app/services/adapters/executor_kinds.py`	Extends shell resolution to include public shell fallback, extracts baseImage from shell CRD, and propagates base_image to executor response and bot prompt building
Public Shell Data `backend/init_data/02-public-shells.yaml`	Adds baseImage field to ClaudeCode and Agno public shell definitions
Executor Binary Management `executor_manager/executors/docker/binary_extractor.py`	New module implementing extraction of executor binary from official image into a Docker named volume with version tracking and mount configuration
Executor Docker Integration `executor_manager/executors/docker/executor.py`	Extends task execution to extract and apply custom base_image, mounting executor binary volume and configuring container entrypoint when base_image is provided
Executor Startup `executor_manager/main.py`	Adds startup logic in lifespan to extract executor binary into named volume with error handling
Image Validation Service `executor_manager/routers/routers.py`	New endpoint `/executor-manager/images/validate` with image validation models and logic; runs per-check dependency validation (node, python, sqlite, etc.) in containers with version comparison and structured result aggregation
Frontend API Client `frontend/src/apis/shells.ts`	New API module defining shell types and ShellApis service with methods for listing, fetching, creating, updating, deleting shells and validating images
Settings Page `frontend/src/app/settings/page.tsx`	Adds new "shells" tab to settings page with ShellList component, URL-driven tab routing, and responsive layout
Shell Edit Component `frontend/src/features/settings/components/ShellEdit.tsx`	New React form component for creating/editing user shells with name, display name, base shell reference selection, base image input, and integrated image validation with detailed check results
Shell List Component `frontend/src/features/settings/components/ShellList.tsx`	New React component displaying unified shells with filtering, CRUD actions (edit/delete for non-public shells), confirmation dialogs, loading/empty states, and shell metadata display
Internationalization `frontend/src/i18n/locales/en/common.json` `frontend/src/i18n/locales/zh-CN/common.json`	Adds English and Chinese translations for shell management UI strings, labels, actions, validations, and error messages

Sequence Diagram

sequenceDiagram
    participant User as User
    participant Frontend as Frontend UI
    participant Backend as Backend API<br/>/shells
    participant Executor as Executor Manager<br/>/images/validate
    participant Docker as Docker
    
    User->>Frontend: Click "Validate Image"
    Frontend->>Frontend: Select shell type & enter image
    Frontend->>Backend: POST /shells/validate-image<br/>(image, shellType)
    
    Backend->>Executor: POST /executor-manager/images/validate<br/>(image, shell_type)
    
    Executor->>Docker: Pull image
    Docker-->>Executor: Image pulled
    
    Executor->>Docker: Run container with dependency checks<br/>(node, python, sqlite, etc.)
    Docker-->>Executor: Check output (versions)
    
    Executor->>Executor: Parse versions & compare<br/>against minimums
    Executor->>Executor: Aggregate results
    
    Executor-->>Backend: ValidateImageResponse<br/>(valid, checks[], errors[])
    Backend-->>Frontend: ImageValidationResponse
    Frontend->>Frontend: Display per-check results<br/>(name, version, status)
    Frontend-->>User: Show validation results

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Image validation logic (executor_manager/routers/routers.py): Regex parsing, version comparison using packaging library, multi-check orchestration with timeouts and error handling
Binary extraction module (executor_manager/executors/docker/binary_extractor.py): Docker API interaction, volume management, version tracking, extraction workflow
Custom base image propagation (backend/app/services/adapters/executor_kinds.py, executor_manager/executors/docker/executor.py): Multi-point data flow from schema through services to executor with fallback logic
Frontend form validation (frontend/src/features/settings/components/ShellEdit.tsx): Image validation integration, async state management, error presentation
Heterogeneous file changes: Schema extensions, service logic, API endpoints, Docker integration, frontend components, translations—requires separate reasoning per area

Possibly related PRs

feat(backend): implement Skill management for Claude Code Skills #182: Modifies executor_kinds.py and kind.py schema to propagate shell-related metadata through executor payloads; overlaps with this PR's baseImage/baseShellRef propagation path.
feat(settings): add model management module #194: Also extends shell and model resolution in executor_kinds.py and augments task payload propagation; shares the same code paths for metadata handling.
Refactor initialization from SQL to YAML configuration #112: Modifies the same public shells initialization manifest (02-public-shells.yaml) that this PR extends with baseImage fields.

Suggested reviewers

feifei325

🐇 Shells are set and image checks aligned,
From public pools to custom designs,
Binary volumes packed with care,
Executors dance through digital air! ✨
A feature complete, both swift and fair!

Pre-merge checks and finishing touches

✅ Passed checks (3 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly summarizes the main change: adding support for custom base image configuration in bot shells.
Docstring Coverage	✅ Passed	Docstring coverage is 91.67% which is sufficient. The required threshold is 80.00%.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.

✨ Finishing touches

📝 Generate docstrings

🧪 Generate unit tests (beta)

Create PR with unit tests
Post copyable unit tests in a comment
Commit unit tests in branch wegent/feature-custom-shell-base-image

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

- Fix ShellList Tag variant error (use 'default' instead of 'outline') - Move image validation logic to Executor Manager API (/executor-manager/images/validate) - Backend shells API now proxies validation requests to Executor Manager - Support various deployment modes (Docker, K8s) where backend may not have direct Docker access

coderabbitai

Actionable comments posted: 7

🧹 Nitpick comments (11)

executor_manager/executors/docker/binary_extractor.py (1)
72-74: Use logger.exception() to capture stack traces.

When logging exceptions, logger.exception() automatically includes the stack trace, which aids debugging.
     except Exception as e:
-        logger.error(f"Error during executor binary extraction: {e}")
+        logger.exception(f"Error during executor binary extraction: {e}")
         return False
Apply similar changes at lines 169 and 172.
executor_manager/routers/routers.py (3)
189-194: Add proper type hints for list fields.

The checks and errors fields should use typed lists for better IDE support and validation.
+from typing import List
+
 class ValidateImageResponse(BaseModel):
     """Response for image validation"""
     valid: bool
-    checks: list
-    errors: list
+    checks: List[ImageCheckResult]
+    errors: List[str]
217-224: Confusing response: valid=True with items in errors list.

Returning valid=True while populating the errors list sends mixed signals. Consider using a different field like info or notes for informational messages.
     if shell_type == "Dify":
         return ValidateImageResponse(
             valid=True,
             checks=[],
-            errors=["Dify is an external_api type and doesn't require image validation"],
+            errors=[],
         )
Or add an info field to the response model for non-error messages.

342-343: Silent exception swallowing hides version comparison failures.

The pass statement silently ignores version parsing errors, which could hide misconfigurations or unexpected version formats.
                         except Exception:
-                            pass  # Skip version comparison on error
+                            logger.warning(f"Failed to parse version for {check['name']}: {version}")
+                            # Treat as pass since dependency exists
backend/app/schemas/kind.py (1)
115-116: Consider adding validation for the baseImage field.

The new optional fields are well-structured and backward compatible. However, consider adding Pydantic validation for baseImage to ensure it follows Docker image naming conventions (e.g., registry/image:tag).

Example validation:
from pydantic import field_validator
import re

class ShellSpec(BaseModel):
    """Shell specification"""

    runtime: str
    supportModel: Optional[List[str]] = None
    baseImage: Optional[str] = None
    baseShellRef: Optional[str] = None

    @field_validator('baseImage')
    @classmethod
    def validate_base_image(cls, v: Optional[str]) -> Optional[str]:
        if v is None:
            return v
        # Basic Docker image pattern validation
        pattern = r'^[a-z0-9]+([._-][a-z0-9]+)*(/[a-z0-9]+([._-][a-z0-9]+)*)*(:[\w][\w.-]*)?(@sha256:[a-f0-9]{64})?$'
        if not re.match(pattern, v, re.IGNORECASE):
            raise ValueError('Invalid Docker image format')
        return v
frontend/src/features/settings/components/ShellEdit.tsx (1)

29-45: Consider using react-hook-form with zod validation for form management.

The component uses manual useState for form fields and custom validation logic. As per coding guidelines, React forms should use react-hook-form and zod for validation. This would provide better validation patterns, cleaner error handling, and improved form state management.

Current implementation works but could be refactored for consistency with the codebase conventions.
frontend/src/features/settings/components/ShellList.tsx (1)
64-81: Consider adding loading state during delete operation.

The delete handler doesn't disable the confirm button or show a loading indicator, which could allow double-clicks or confusion during slow network conditions. Consider adding a deleting state similar to the saving state in ShellEdit.
+ const [deleting, setDeleting] = useState(false)

  const handleDelete = async () => {
    if (!deleteConfirmShell) return

+   setDeleting(true)
    try {
      await shellApis.deleteShell(deleteConfirmShell.name)
      toast({
        title: t('shells.delete_success'),
      })
      setDeleteConfirmShell(null)
      fetchShells()
    } catch (error) {
      toast({
        variant: 'destructive',
        title: t('shells.errors.delete_failed'),
        description: (error as Error).message,
      })
+   } finally {
+     setDeleting(false)
    }
  }
frontend/src/apis/shells.ts (1)

115-128: Consider efficiency of filter helpers for large shell lists.

getPublicShells and getLocalEngineShells fetch all shells and filter client-side. This is acceptable for small lists but could be inefficient if the shell count grows. Consider adding server-side filtering endpoints if this becomes a performance concern.
backend/app/api/endpoints/adapter/shells.py (3)
143-148: Consider catching a more specific exception for shell parsing.

The broad Exception catch is used when parsing shells, which could mask unexpected errors. Consider catching ValidationError from Pydantic specifically, since ShellCRD.model_validate is the likely failure point.
+from pydantic import BaseModel, ValidationError

 for shell in public_shells:
     try:
         result.append(_public_shell_to_unified(shell))
-    except Exception as e:
+    except ValidationError as e:
         logger.warning(f"Failed to parse public shell {shell.name}: {e}")
433-439: Dify validation returns valid=True with an error message, which is semantically confusing.

When shellType == "Dify", the response has valid=True but includes an error message. Consider using an empty errors list or a checks entry instead to convey this is informational rather than an error.
     if shell_type == "Dify":
         return ImageValidationResponse(
             valid=True,
-            checks=[],
-            errors=["Dify is an external_api type and doesn't require image validation"],
+            checks=[ImageCheckResult(
+                name="Shell Type",
+                status="pass",
+                message="Dify is an external_api type and doesn't require image validation"
+            )],
+            errors=[],
         )
485-504: Use logger.exception instead of logger.error for exception logging.

When logging exceptions in catch blocks, logger.exception automatically includes the traceback, which aids debugging. This aligns with static analysis hint TRY400.
     except httpx.TimeoutException:
-        logger.error(f"Timeout calling executor manager for image validation: {image}")
+        logger.exception(f"Timeout calling executor manager for image validation: {image}")
         return ImageValidationResponse(
             valid=False,
             checks=[],
             errors=["Validation request timed out. The image may be large or slow to pull."],
         )
     except httpx.RequestError as e:
-        logger.error(f"Error calling executor manager: {e}")
+        logger.exception(f"Error calling executor manager: {e}")
         return ImageValidationResponse(
             valid=False,
             checks=[],
-            errors=[f"Failed to connect to executor manager: {str(e)}"],
+            errors=[f"Failed to connect to executor manager: {e!s}"],
         )
     except Exception as e:
-        logger.error(f"Image validation error: {e}")
+        logger.exception(f"Image validation error: {e}")
         return ImageValidationResponse(
             valid=False,
             checks=[],
-            errors=[f"Validation error: {str(e)}"],
+            errors=[f"Validation error: {e!s}"],
         )

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 502a024 and f6d85b0.

📒 Files selected for processing (15)

backend/app/api/api.py (2 hunks)
backend/app/api/endpoints/adapter/shells.py (1 hunks)
backend/app/schemas/kind.py (1 hunks)
backend/app/services/adapters/executor_kinds.py (4 hunks)
backend/init_data/02-public-shells.yaml (2 hunks)
executor_manager/executors/docker/binary_extractor.py (1 hunks)
executor_manager/executors/docker/executor.py (3 hunks)
executor_manager/main.py (1 hunks)
executor_manager/routers/routers.py (1 hunks)
frontend/src/apis/shells.ts (1 hunks)
frontend/src/app/settings/page.tsx (7 hunks)
frontend/src/features/settings/components/ShellEdit.tsx (1 hunks)
frontend/src/features/settings/components/ShellList.tsx (1 hunks)
frontend/src/i18n/locales/en/common.json (2 hunks)
frontend/src/i18n/locales/zh-CN/common.json (2 hunks)

🧰 Additional context used

📓 Path-based instructions (6)

**/*.{py,js,ts,tsx}