TinyLLM Coder

An in-house coding assistant agent built with Node.js, LangGraph.js, and support for small LLMs like Qwen3 4B. Designed for local, privacy-focused development tasks without relying on external APIs.

Description

TinyLLM Coder is a CLI tool that leverages stateful agent workflows to perform coding-related tasks. It uses small language models (e.g., Qwen3 4B) for reasoning and tool-calling, optimized for efficiency on local hardware. All operations are in-house, ensuring data privacy and no internet dependency beyond the LLM provider.

Key capabilities include code generation, file editing, codebase search, linting, safe command execution, and more. The agent decomposes complex tasks, summarizes context, and validates outputs to work effectively with limited-model capabilities.

Features

• Coding Tasks: Generate, edit, and refactor code; search and analyze codebases.
• Local Tools: File I/O, Git operations, ESLint integration, safe shell execution, math calculations.
• LLM Optimization: Built-in summarization, task decomposition, and validation tools to handle small model limitations.
• Provider Flexibility: Supports Ollama (recommended for ease) and Llama.cpp servers.
• In-House Design: No external APIs or cloud services; runs entirely locally.
• CLI Interface: Simple command-line usage with customizable options.

Architecture

• Framework: LangGraph.js for graph-based agent orchestration.
• LLM Integration: ChatOllama or ChatOpenAI (for Llama.cpp) with tool binding.
• Tools: LangChain-compatible tools for various operations.
• State Management: Messages-based state with context summarization.
• Execution: ReAct-style reasoning loop with conditional tool calls.

Installation

Prerequisites

• Node.js 18+
• Ollama (for Ollama provider) or a Llama.cpp server (for Llama.cpp provider)
• Git (for Git tools)

Steps

1. Clone the repository:

   git clone https://github.com/youngnishant/tinyllmcoder.git
   cd tinyllmcoder

2. Install dependencies:

   npm install

3. Build the project:

   npm run build

4. Set up LLM provider:

   • Ollama: Install Ollama, then pull the model:

     ollama pull qwen2.5-coder:1.5b

   • Llama.cpp: Start your Llama.cpp server (e.g., via llama.cpp executable).

5. Configure environment: Create a .env file in the root:

   LLM_PROVIDER=ollama  # or llamacpp
   LLM_HOST=localhost:11434  # Ollama default; for Llama.cpp, your server host
   API_KEY=  # Leave empty for Ollama; add key if needed for Llama.cpp
   MODEL_NAME=qwen2.5-coder:1.5b
   

Usage

Basic Command

Run the agent with a coding task:

node dist/index.js "write a function to sort an array in JavaScript"

CLI Options

• --provider <provider>: LLM provider (ollama or llamacpp). Default: ollama.
• --host <host>: Host URL for the LLM server. Default: localhost:11434.
• --api-key <key>: API key if required. Default: none.

Examples

• Generate code:

  node dist/index.js "create a React component for a todo list"

• Edit files:

  node dist/index.js "fix the bug in src/utils.js where the function returns undefined"

• Search and analyze:

  node dist/index.js "find all functions in the codebase that handle user authentication"

• Lint and validate:

  node dist/index.js "lint the file app.js and suggest fixes"

Output

The agent outputs results directly to the console, including generated code, tool results, and any feedback.

Tools

The agent includes the following built-in tools:

• File Operations: Read and write files (with chunking for large files).
• Code Search: Grep-based search across the codebase.
• Semantic Code Search: AI-expanded search for better semantic matching using LLM-generated related terms.
• AI File Edit: Edit files based on natural language descriptions using LLM.
• Git Operations: Status, diff, commit changes.
• Linting: ESLint integration for code quality checks.
• Safe Execution: Run allowed commands (e.g., npm run build, node script.js).
• Calculation: Simple math expressions.
• Memory Summarization: Condense conversation history.
• Task Decomposition: Break complex tasks into steps.
• Code Validation: Basic syntax and logic checks.

Tools are invoked automatically based on the LLM's reasoning. The agent includes a system prompt optimized for small LLMs under 8B parameters, guiding efficient tool use and concise responses.

Testing

Run the test suite:

npm test

Tests cover tool functionality. Agent integration tests are skipped without an active LLM.

Contributing

1. Fork the repository.
2. Create a feature branch: git checkout -b feature/your-feature.
3. Make changes and add tests.
4. Run npm test and npm run build.
5. Submit a pull request.

License

This project is licensed under the ISC License - see the LICENSE file for details.

Acknowledgments

• Built with LangChain and LangGraph.
• Inspired by local AI agent patterns for efficient, private development.

For issues or questions, open an issue on GitHub.