|
| 1 | +# Lightspeed Core Stack Development Guide |
| 2 | + |
| 3 | +## Project Overview |
| 4 | +Lightspeed Core Stack (LCS) is an AI-powered assistant built on FastAPI that provides answers using LLM services, agents, and RAG databases. It integrates with Llama Stack for AI operations. |
| 5 | + |
| 6 | +## Development Environment |
| 7 | +- **Python**: Check `pyproject.toml` for supported Python versions |
| 8 | +- **Package Manager**: uv (use `uv run` for all commands) |
| 9 | +- **Required Commands**: |
| 10 | + - `uv run make format` - Format code (black + ruff) |
| 11 | + - `uv run make verify` - Run all linters (black, pylint, pyright, ruff, docstyle, check-types) |
| 12 | + |
| 13 | +## Code Architecture & Patterns |
| 14 | + |
| 15 | +### Project Structure |
| 16 | +``` |
| 17 | +src/ |
| 18 | +├── app/ # FastAPI application |
| 19 | +│ ├── endpoints/ # REST API endpoints |
| 20 | +│ └── main.py # Application entry point |
| 21 | +├── auth/ # Authentication modules (k8s, jwk, noop) |
| 22 | +├── authorization/ # Authorization middleware & resolvers |
| 23 | +├── models/ # Pydantic models |
| 24 | +│ ├── config.py # Configuration classes |
| 25 | +│ ├── requests.py # Request models |
| 26 | +│ └── responses.py # Response models |
| 27 | +├── utils/ # Utility functions |
| 28 | +├── client.py # Llama Stack client wrapper |
| 29 | +└── configuration.py # Config management |
| 30 | +``` |
| 31 | + |
| 32 | +### Coding Standards |
| 33 | + |
| 34 | +#### Imports & Dependencies |
| 35 | +- Use absolute imports for internal modules: `from auth import get_auth_dependency` |
| 36 | +- FastAPI dependencies: `from fastapi import APIRouter, HTTPException, Request, status, Depends` |
| 37 | +- Llama Stack imports: `from llama_stack_client import AsyncLlamaStackClient` |
| 38 | +- **ALWAYS** check `pyproject.toml` for existing dependencies before adding new ones |
| 39 | +- **ALWAYS** verify current library versions in `pyproject.toml` rather than assuming versions |
| 40 | + |
| 41 | +#### Module Standards |
| 42 | +- All modules start with descriptive docstrings explaining purpose |
| 43 | +- Use `logger = logging.getLogger(__name__)` pattern for module logging |
| 44 | +- Package `__init__.py` files contain brief package descriptions |
| 45 | +- Central `constants.py` for shared constants with descriptive comments |
| 46 | +- Type aliases defined at module level for clarity |
| 47 | + |
| 48 | +#### Configuration |
| 49 | +- All config uses Pydantic models extending `ConfigurationBase` |
| 50 | +- Base class sets `extra="forbid"` to reject unknown fields |
| 51 | +- Use `@field_validator` and `@model_validator` for custom validation |
| 52 | +- Type hints: `Optional[FilePath]`, `PositiveInt`, `SecretStr` |
| 53 | + |
| 54 | +#### Function Standards |
| 55 | +- **Documentation**: All functions require docstrings with brief descriptions |
| 56 | +- **Type Annotations**: Complete type annotations for parameters and return types |
| 57 | + - Use `typing_extensions.Self` for model validators |
| 58 | + - Union types: `str | int` (modern syntax) |
| 59 | + - Optional: `Optional[Type]` or `Type | None` |
| 60 | +- **Naming**: Use snake_case with descriptive, action-oriented names (get_, validate_, check_) |
| 61 | +- **Return Values**: **CRITICAL** - Avoid in-place parameter modification anti-patterns: |
| 62 | + ```python |
| 63 | + # ❌ BAD: Modifying parameter in-place |
| 64 | + def process_data(input_data: Any, result_dict: dict) -> None: |
| 65 | + result_dict[key] = value # Anti-pattern |
| 66 | + |
| 67 | + # ✅ GOOD: Return new data structure |
| 68 | + def process_data(input_data: Any) -> dict: |
| 69 | + result_dict = {} |
| 70 | + result_dict[key] = value |
| 71 | + return result_dict |
| 72 | + ``` |
| 73 | +- **Async Functions**: Use `async def` for I/O operations and external API calls |
| 74 | +- **Error Handling**: |
| 75 | + - Use FastAPI `HTTPException` with appropriate status codes for API endpoints |
| 76 | + - Handle `APIConnectionError` from Llama Stack |
| 77 | + |
| 78 | +#### Logging Standards |
| 79 | +- Use `import logging` and module logger pattern: `logger = logging.getLogger(__name__)` |
| 80 | +- Standard log levels with clear purposes: |
| 81 | + - `logger.debug()` - Detailed diagnostic information |
| 82 | + - `logger.info()` - General information about program execution |
| 83 | + - `logger.warning()` - Something unexpected happened or potential problems |
| 84 | + - `logger.error()` - Serious problems that prevented function execution |
| 85 | + |
| 86 | +#### Class Standards |
| 87 | +- **Documentation**: All classes require descriptive docstrings explaining purpose |
| 88 | +- **Naming**: Use PascalCase with descriptive names and standard suffixes: |
| 89 | + - `Configuration` for config classes |
| 90 | + - `Error`/`Exception` for custom exceptions |
| 91 | + - `Resolver` for strategy pattern implementations |
| 92 | + - `Interface` for abstract base classes |
| 93 | +- **Pydantic Models**: Extend `ConfigurationBase` for config, `BaseModel` for data models |
| 94 | +- **Abstract Classes**: Use ABC for interfaces with `@abstractmethod` decorators |
| 95 | +- **Validation**: Use `@model_validator` and `@field_validator` for Pydantic models |
| 96 | +- **Type Hints**: Complete type annotations for all class attributes |
| 97 | + |
| 98 | +#### Docstring Standards |
| 99 | +- Follow Google Python docstring conventions: https://google.github.io/styleguide/pyguide.html |
| 100 | +- Required for all modules, classes, and functions |
| 101 | +- Include brief description and detailed sections as needed: |
| 102 | + - `Args:` for function parameters |
| 103 | + - `Returns:` for return values |
| 104 | + - `Raises:` for exceptions that may be raised |
| 105 | + - `Attributes:` for class attributes (Pydantic models) |
| 106 | + |
| 107 | + |
| 108 | +## Testing Framework |
| 109 | + |
| 110 | +### Test Structure |
| 111 | +``` |
| 112 | +tests/ |
| 113 | +├── unit/ # Unit tests (pytest) |
| 114 | +├── integration/ # Integration tests |
| 115 | +└── e2e/ # End-to-end tests (behave) |
| 116 | + └── features/ # Gherkin feature files |
| 117 | +``` |
| 118 | + |
| 119 | +### Testing Framework Requirements |
| 120 | +- **Required**: Use pytest for all unit and integration tests |
| 121 | +- **Forbidden**: Do not use unittest - pytest is the standard for this project |
| 122 | +- **E2E Tests**: Use behave (BDD) framework for end-to-end testing |
| 123 | + |
| 124 | +### Unit Tests (pytest) |
| 125 | +- **Fixtures**: Use `conftest.py` for shared fixtures |
| 126 | +- **Mocking**: `pytest-mock` for AsyncMock objects |
| 127 | +- **Common Pattern**: |
| 128 | + ```python |
| 129 | + @pytest.fixture(name="prepare_agent_mocks") |
| 130 | + def prepare_agent_mocks_fixture(mocker): |
| 131 | + mock_client = mocker.AsyncMock() |
| 132 | + mock_agent = mocker.AsyncMock() |
| 133 | + mock_agent._agent_id = "test_agent_id" |
| 134 | + return mock_client, mock_agent |
| 135 | + ``` |
| 136 | +- **Auth Mock**: `MOCK_AUTH = ("mock_user_id", "mock_username", False, "mock_token")` |
| 137 | +- **Coverage**: Unit tests require 60% coverage, integration 10% |
| 138 | + |
| 139 | +### E2E Tests (behave) |
| 140 | +- **Framework**: Behave (BDD) with Gherkin feature files |
| 141 | +- **Step Definitions**: In `tests/e2e/features/steps/` |
| 142 | +- **Common Steps**: Service status, authentication, HTTP requests |
| 143 | +- **Test List**: Maintained in `tests/e2e/test_list.txt` |
| 144 | + |
| 145 | +### Test Commands |
| 146 | +```bash |
| 147 | +uv run make test-unit # Unit tests with coverage |
| 148 | +uv run make test-integration # Integration tests |
| 149 | +uv run make test-e2e # End-to-end tests |
| 150 | +``` |
| 151 | + |
| 152 | +## Quality Assurance |
| 153 | + |
| 154 | +### Required Before Completion |
| 155 | +1. `uv run make format` - Auto-format code |
| 156 | +2. `uv run make verify` - Run all linters |
| 157 | +3. Create unit tests for new code |
| 158 | +4. Ensure tests pass |
| 159 | + |
| 160 | +### Linting Tools |
| 161 | +- **black**: Code formatting |
| 162 | +- **pylint**: Static analysis (`source-roots = "src"`) |
| 163 | +- **pyright**: Type checking (excludes `src/auth/k8s.py`) |
| 164 | +- **ruff**: Fast linter |
| 165 | +- **pydocstyle**: Docstring style |
| 166 | +- **mypy**: Additional type checking |
| 167 | + |
| 168 | +### Security |
| 169 | +- **bandit**: Security issue detection |
| 170 | +- Never commit secrets/keys |
| 171 | +- Use environment variables for sensitive data |
| 172 | + |
| 173 | +## Key Dependencies |
| 174 | +**IMPORTANT**: Always check `pyproject.toml` for current versions rather than relying on this list: |
| 175 | +- **FastAPI**: Web framework |
| 176 | +- **Llama Stack**: AI integration |
| 177 | +- **Pydantic**: Data validation/serialization |
| 178 | +- **SQLAlchemy**: Database ORM |
| 179 | +- **Kubernetes**: K8s auth integration |
| 180 | + |
| 181 | +## Development Workflow |
| 182 | +1. Use `uv sync --group dev --group llslibdev` for dependencies |
| 183 | +2. Always use `uv run` prefix for commands |
| 184 | +3. **ALWAYS** check `pyproject.toml` for existing dependencies and versions before adding new ones |
| 185 | +4. Follow existing code patterns in the module you're modifying |
| 186 | +5. Write unit tests covering new functionality |
| 187 | +6. Run format and verify before completion |
0 commit comments