Skip to content

Commit c84308f

Browse files
committed
refactor: Enhance README and benchmark scripts for clarity and functionality
- Updated README.md to provide a comprehensive overview of the monorepo tools benchmark, including detailed architecture and performance implications. - Improved daily benchmark workflow by adding `--output json` flag for better result handling. - Refactored benchmark scripts to include conditional logging based on output mode, enhancing readability and usability. - Adjusted error handling and logging in benchmark scripts for clearer output during failures. These changes improve documentation clarity and streamline the benchmarking process, making it easier to understand and utilize the tools effectively.
1 parent d16e095 commit c84308f

File tree

4 files changed

+388
-232
lines changed

4 files changed

+388
-232
lines changed

.github/workflows/daily-benchmark.yml

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -105,7 +105,7 @@ jobs:
105105
- name: Run benchmark for ${{ matrix.tool }}
106106
id: benchmark
107107
run: |
108-
node dist/scripts/benchmark-single-tool.js ${{ matrix.tool }} > benchmark-results-${{ matrix.tool }}.json
108+
node dist/scripts/benchmark-single-tool.js ${{ matrix.tool }} --output json > benchmark-results-${{ matrix.tool }}.json
109109
110110
- name: Upload benchmark results for ${{ matrix.tool }}
111111
uses: actions/upload-artifact@v4
@@ -153,7 +153,7 @@ jobs:
153153
- name: Combine benchmark results
154154
if: ${{ !inputs.skip_benchmark }}
155155
run: |
156-
node dist/scripts/combine-results.js > benchmark-results.json
156+
node dist/scripts/combine-results.js --output json > benchmark-results.json
157157
158158
- name: Read current results
159159
if: ${{ !inputs.skip_benchmark }}
@@ -256,7 +256,7 @@ jobs:
256256
- name: Combine benchmark results for commit
257257
if: ${{ !inputs.skip_benchmark }}
258258
run: |
259-
node dist/scripts/combine-results.js > benchmark-results.json
259+
node dist/scripts/combine-results.js --output json > benchmark-results.json
260260
261261
- name: Commit and push changes
262262
if: needs.analysis.outputs.readme-updated == 'true'

README.md

Lines changed: 194 additions & 101 deletions
Original file line numberDiff line numberDiff line change
@@ -1,156 +1,249 @@
1-
# Benchmarking Nx, Turbo, Lerna, Lage, and Moon
1+
# Monorepo Tools Performance Benchmark: Nx vs Turbo vs Lerna vs Lage vs Moon
22

3-
Repo contains:
3+
**Comprehensive Performance Comparison of Popular JavaScript Monorepo Build Tools and Task Runners**
44

5-
1. 5 shared buildable packages/libraries with 250 components each
6-
2. 5 Next.js applications built out of 20 app-specific libraries. Each app-specific lib has 250 components each. Each
7-
library uses the shared components.
5+
This repository contains an extensive, unbiased performance benchmark comparing the most popular monorepo management tools in the JavaScript ecosystem: **Nx**, **Turbo (Turborepo)**, **Lerna**, **Lage**, and **Moon**. Our benchmark focuses on real-world scenarios with cache restoration performance using a enterprise-scale codebase.
86

9-
Combined there are about 26k components. It's a lot of components, but they are very small. This corresponds to a medium
10-
size enterprise repo. A lot of our clients have repos that are 10x bigger than this, so this repo isn't something out or
11-
ordinary. And, the bigger the repo, the bigger the difference in performance between Nx and other tools.
7+
## 🏗️ Benchmark Repository Architecture
128

13-
The repo has Nx, Turbo, Lerna, Lage, and Moon enabled. They don't affect each other. You can remove one without affecting the
14-
others.
9+
Our test repository simulates a **medium-to-large enterprise monorepo** with:
1510

16-
## Benchmark & Results (January 2025)
11+
### **Codebase Scale & Complexity**
12+
13+
- **5 shared buildable libraries** - Each containing 250 reusable components
14+
- **5 Next.js applications** - Each built from 20 app-specific libraries
15+
- **100 total libraries** - Each library contains 250 components
16+
- **~26,000 total components** - Representing realistic enterprise scale
17+
18+
### **Real-World Enterprise Scenario**
19+
20+
This benchmark represents a **medium-sized enterprise repository**. Many organizations operate monorepos that are **10x larger** than this test case, making performance differences even more critical at scale.
21+
22+
## **Monorepo Tools Tested**
23+
24+
### **Nx - Extensible Build Framework**
25+
26+
[Nx](https://nx.dev) is a powerful, extensible dev tool that helps you develop, test, build, and scale with React, Vue, Node, and more. Key features:
27+
28+
- **Smart rebuilds** with computation caching
29+
- **Distributed task execution** across multiple machines
30+
- **Code generation** and automated migrations
31+
- **Integrated tooling** for testing, linting, and building
32+
- **Workspace analysis** and visualization
33+
34+
### **Turbo (Turborepo) - High-Performance Build System**
35+
36+
[Turborepo](https://turbo.build) is a high-performance build system for JavaScript and TypeScript codebases. Features include:
37+
38+
- **Remote caching** for fast CI/CD pipelines
39+
- **Incremental bundling** and building
40+
- **Task parallelization** and dependency management
41+
- **Zero runtime overhead** and minimal configuration
42+
- **Built in Rust** for maximum performance
43+
44+
### **Lerna - Multi-Package Repository Manager**
45+
46+
[Lerna](https://lerna.js.org) is a fast, modern build system for managing and publishing multiple JavaScript/TypeScript packages. Capabilities:
47+
48+
- **Independent versioning** of packages
49+
- **Automated publishing** workflows
50+
- **Task caching** powered by Nx
51+
- **Workspace management** and linking
52+
- **Conventional commits** integration
53+
54+
### **Lage - Task Runner for JavaScript Monorepos**
55+
56+
[Lage](https://microsoft.github.io/lage/) is a task runner for JavaScript monorepos built by Microsoft. Features:
57+
58+
- **Pipeline-based task execution**
59+
- **Efficient caching mechanisms**
60+
- **Parallel task processing**
61+
- **TypeScript-first approach**
62+
- **Integration with npm workspaces**
63+
64+
### **Moon - Rust-Based Build System**
65+
66+
[Moon](https://moonrepo.dev/) is a Rust-based build system and monorepo management tool focusing on performance and developer experience:
67+
68+
- **Rust performance** for maximum speed and efficiency
69+
- **Smart caching** with advanced cache mechanisms and remote caching support
70+
- **Task pipeline** with efficient orchestration and dependency management
71+
- **Multi-language support** including Node.js, Python, Rust, and more
72+
- **YAML configuration** with intelligent defaults
73+
- **Incremental building** - only builds what's changed for faster development cycles
74+
75+
## Benchmark & Results (January 17, 2025)
1776

1877
Run `pnpm run benchmark`. The benchmark will warm the cache of all the tools. We benchmark how quickly
19-
Turbo/Nx/Lage/Lerna/Moon can figure out what needs to be restored from the cache and restores it.
78+
Turbo/Nx/Lerna/Lage/Moon can figure out what needs to be restored from the cache and restores it.
2079

2180
These are the numbers using GitHub Actions runner:
2281

23-
* average lage time is: 11830.6
24-
* average turbo time is: 9992.2
25-
* average lerna (powered by nx) time is: 3407.0
26-
* average moon time is: TBD (pending benchmark)
27-
* average nx time is: 1849.4
28-
* nx is 6.4x faster than lage
29-
* nx is 5.4x faster than turbo
30-
* nx is 1.8x faster than lerna (powered by nx)
31-
* nx vs moon performance comparison: TBD
82+
- average lage time is: 11830.6
83+
- average turbo time is: 9992.2
84+
- average lerna (powered by nx) time is: 3407.0
85+
- average moon time is: TBD (pending benchmark)
86+
- average nx time is: 1849.4
87+
- nx is 6.4x faster than lage
88+
- nx is 5.4x faster than turbo
89+
- nx is 1.8x faster than lerna (powered by nx)
90+
- nx vs moon performance comparison: TBD
91+
92+
## 🚀 Performance Implications for Enterprise Development
93+
94+
### **Cache Restoration Speed Matters**
95+
96+
The benchmark measures **cache restoration performance** - how quickly each tool can:
97+
98+
1. **Analyze dependencies** and determine what needs to be rebuilt
99+
2. **Restore cached artifacts** from previous builds
100+
3. **Skip unnecessary work** and maximize development velocity
32101

33-
### About Moon
102+
### **Real-World Impact**
34103

35-
[Moon](https://moonrepo.dev/) is a Rust-based build system and monorepo management tool that focuses on performance and developer experience. Key features include:
104+
For small to medium repositories, the performance differences may be acceptable across all tools. However, the **true performance benefits** emerge when:
36105

37-
* **Performance**: Written in Rust for maximum speed and efficiency
38-
* **Smart Caching**: Advanced caching mechanisms with remote caching support
39-
* **Task Pipeline**: Efficient task orchestration with dependency management
40-
* **Language Support**: Multi-language support including Node.js, Python, Rust, and more
41-
* **Configuration**: YAML-based configuration with intelligent defaults
42-
* **Incremental Building**: Only builds what's changed for faster development cycles
106+
- **Scaling to larger codebases** (10x+ the size of this benchmark)
107+
- **Implementing distributed builds** across multiple machines
108+
- **Optimizing CI/CD pipeline performance** for faster deployments
109+
- **Improving developer experience** with faster local builds
43110

44-
### Does this performance difference matter in practice?
111+
## 🎯 Developer Experience & Tool Integration
45112

46-
The cache restoration that tools like Turborepo and Moon provide is likely to be fast enough for a lot of small and mid-size repos.
47-
What matters more is the ability to distribute any command across say 50 machines while
48-
preserving the dev ergonomics of running it on a single machine. Nx can do it. Bazel can do it (which Nx
49-
borrows some
50-
ideas from). Moon supports remote caching which enables distributed builds. This is where the perf gains are for larger repos.
51-
See [this benchmark](https://github.com/vsavkin/interstellar) to learn more.
113+
### **Terminal Output & User Interface**
52114

53-
## Dev ergonomics & Staying out of your way
115+
A critical but often overlooked aspect of monorepo tools is **preserving the native development experience**:
54116

55-
When some folks compare Nx and Turborepo, they say something like "Nx may do all of those things well, and may be
56-
faster, but Turbo is built to stay out of you way". Let's talk about staying out of your way:
117+
**Test the difference yourself:**
57118

58-
Run `nx build crew --skip-nx-cache` and `turbo run build --scope=crew --force`:
119+
- Run: `nx build crew --skip-nx-cache`
120+
- Compare with: `turbo run build --scope=crew --force`
59121

60-
Nx doesn't change your terminal output. Spinners, animations, colors are the same whether you use Nx or not (we
61-
instrument Node.js to get this result). What is also important is that when you restore things from cache, Nx will
62-
replay the terminal output identical to the one you would have had you run the command.
122+
## 📊 Automated Continuous Benchmarking
63123

64-
Examine Turbo's output: no spinners, no animations, no colors. Pretty much anything you run with Turbo looks different (
65-
and a lot worse, to be honest) from running the same command without Turbo.
124+
### **🤖 Daily Performance Monitoring**
66125

67-
A lot of Nx users don't even know they use Nx, or even what Nx is. Things they run look the same, they just got faster.
126+
This repository implements **comprehensive automated benchmarking** to track performance trends:
68127

69-
## Automated Daily Benchmarks
128+
**Automation Features:**
70129

71-
This repository runs automated benchmarks daily to track performance trends over time:
130+
- **Daily benchmarks** at 6 AM UTC via GitHub Actions
131+
- **Automatic dependency updates** to latest tool versions
132+
- **Performance regression detection** (alerts for >10% performance changes)
133+
- **Automated README updates** with latest benchmark results
134+
- **GitHub releases** with version-tagged performance data
135+
- **Issue creation** for significant performance regressions
72136

73-
### 🤖 Automation Features
137+
### **🏷️ Version-Based Release Tracking**
74138

75-
* **Daily Benchmarks**: Runs at 6 AM UTC daily via GitHub Actions
76-
* **Dependency Updates**: Automated daily updates to latest versions of all tools (nx, turbo, lerna, lage)
77-
* **Performance Monitoring**: Detects significant performance regressions (>10% change)
78-
* **Automatic README Updates**: Always updates the "Benchmark & Results" section with latest data
79-
* **GitHub Releases**: Creates/updates releases with version-based tags for easy historical tracking
80-
* **Issue Creation**: Automatically creates GitHub issues for performance regressions
139+
Each benchmark automatically creates GitHub releases for historical tracking:
81140

82-
### 🏷️ Release Management
141+
- **Semantic versioning** includes all tool versions (e.g., `benchmark-nx21.0.3-turbo2.5.3-lerna8.2.2-lage2.14.6`)
142+
- **Comprehensive release notes** with detailed performance results and raw data
143+
- **Historical comparison** across different tool versions
144+
- **Automatic updates** for existing releases with same tool versions
83145

84-
Each benchmark run automatically creates or updates a GitHub release:
146+
### **📈 Performance Trend Analysis**
85147

86-
* **Version-based Tags**: Tags include all tool versions (e.g., `benchmark-nx21.0.3-turbo2.5.3-lerna8.2.2-lage2.14.6`)
87-
* **Rich Release Notes**: Detailed performance results, tool versions, and raw benchmark data
88-
* **Historical Tracking**: Easy to find and compare results across different tool versions
89-
* **Automatic Updates**: If a release already exists for the current tool versions, it gets updated
148+
Results are automatically analyzed for:
90149

91-
### 📊 Manual Benchmark
150+
- **Significant changes** detection (>5% performance variance)
151+
- **Regression alerts** for performance degradation
152+
- **Tool comparison updates** with relative performance ratios
153+
- **Long-term trend tracking** across tool versions
92154

93-
You can also run benchmarks manually:
155+
## 🛠️ Running Benchmarks
156+
157+
### **Manual Benchmark Execution**
94158

95159
```bash
96-
# Regular benchmark with console output
160+
# Standard benchmark with console output
97161
pnpm run benchmark
98162

99-
# JSON benchmark for automation (TypeScript compiled)
163+
# JSON output for automation and analysis
100164
pnpm run benchmark:json
101165

102-
# TypeScript development (run directly with tsx)
166+
# TypeScript development mode (direct execution)
103167
pnpm run benchmark:json:ts
104168
```
105169

106-
### 🛠️ TypeScript Development
170+
### **TypeScript Development Environment**
107171

108-
The automation scripts are built with TypeScript for better type safety and developer experience:
172+
All automation scripts use **TypeScript** for enhanced developer experience:
109173

110174
**Development Commands:**
111175

112-
* `pnpm run benchmark:json:ts` - Run TypeScript benchmark directly with tsx
113-
* `pnpm run test:automation:ts` - Run TypeScript tests directly
114-
* `pnpm run build:scripts` - Compile TypeScript scripts to JavaScript
115-
* `pnpm run build:scripts:watch` - Watch mode compilation
116-
* `pnpm run compare:results:ts` - Run TypeScript comparison directly
117-
* `pnpm run create:release:ts` - Generate release information from benchmark results
176+
```bash
177+
# Direct TypeScript execution
178+
pnpm run benchmark:json:ts # Run benchmark
179+
pnpm run test:automation:ts # Run test suite
180+
pnpm run compare:results:ts # Compare results
181+
pnpm run create:release:ts # Generate releases
182+
183+
# Compilation and build
184+
pnpm run build:scripts # Compile to JavaScript
185+
pnpm run build:scripts:watch # Watch mode compilation
186+
```
187+
188+
**Script Architecture:**
189+
190+
- **`scripts/benchmark-json.ts`** - Main benchmark execution with strict typing
191+
- **`scripts/compare-and-update-readme.ts`** - Result analysis and README updates
192+
- **`scripts/create-release.ts`** - GitHub release generation with version tagging
193+
- **`scripts/test-compare.ts`** - Comprehensive test suite for automation
194+
- **`scripts/types.ts`** - Shared TypeScript interfaces and type definitions
195+
196+
## 🔧 Repository Configuration
197+
198+
### **Tool Configuration**
199+
200+
All monorepo tools are configured independently and **do not interfere** with each other:
201+
202+
- Each tool can be removed without affecting others
203+
- Configurations are optimized for fair comparison
204+
- Setup favors scenarios where each tool should perform well
205+
206+
### **Dependency Management**
207+
208+
- **Package Manager**: pnpm for optimal performance
209+
- **Automated Updates**: Dependabot + daily workflow for latest versions
210+
- **Compatibility Testing**: Benchmarks run after updates to ensure stability
211+
212+
## 🤝 Contributing & Feedback
213+
214+
### **Found an Issue? We Welcome Contributions**
215+
216+
This benchmark aims to be **completely fair and unbiased**. If you discover:
118217

119-
**File Structure:**
218+
- **Configuration issues** that disadvantage any tool
219+
- **Edge cases** that don't represent realistic usage
220+
- **Setup problems** that affect benchmark accuracy
221+
- **Missing optimizations** for any tool
120222

121-
* **`scripts/benchmark-json.ts`** - TypeScript version with strict types
122-
* **`scripts/compare-and-update-readme.ts`** - Compares results and updates README
123-
* **`scripts/create-release.ts`** - Generates GitHub releases with version-based tags
124-
* **`scripts/test-compare.ts`** - Test suite for automation functions
125-
* **`scripts/types.ts`** - Shared type definitions and interfaces
126-
* **`dist/*.js`** - Compiled JavaScript (auto-generated, ignored by git)
223+
**Please submit a Pull Request!** Our goal is accurate, representative benchmarking that helps the community make informed decisions.
127224

128-
**TypeScript Configuration:**
225+
### **Benchmark Methodology**
129226

130-
* **`tsconfig.scripts.json`** - TypeScript config for automation scripts
131-
* Uses strict mode, modern target (ES2020), CommonJS modules for Node.js compatibility
227+
We've specifically chosen:
132228

133-
### 🔧 Dependency Management
229+
- **Next.js applications** (favorable for Turborepo)
230+
- **Cache restoration scenarios** (core strength of all tools)
231+
- **No incremental builds**
232+
- **Realistic enterprise scale** (26k components)
134233

135-
The repository uses:
234+
This ensures a **fair comparison** focused on each tool's core caching capabilities.
136235

137-
* **Dependabot**: For automated dependency PRs
138-
* **Daily Update Workflow**: Aggressive updates to ensure we're testing latest versions (nx, turbo, lerna, lage, moonrepo)
139-
* **Compatibility Testing**: Benchmarks run after updates to ensure everything works
236+
## 🏆 Conclusion
140237

141-
### 📈 Performance Tracking
238+
This benchmark provides **objective performance data** to help teams choose the right monorepo tool for their needs. Consider:
142239

143-
Results are automatically tracked and compared:
240+
1. **Repository scale** - larger codebases amplify performance differences
241+
2. **Team requirements** - distributed builds, DX, ecosystem integration
242+
3. **Tool maturity** - community support, documentation, plugin ecosystem
243+
4. **Long-term strategy** - migration paths, vendor support, development roadmap
144244

145-
* Stores previous results for comparison
146-
* Detects trends and significant changes
147-
* Updates README only when meaningful changes occur
148-
* Creates alerts for regressions
245+
**Performance is just one factor** in selecting monorepo tooling. Evaluate based on your team's specific requirements, existing infrastructure, and long-term architectural goals.
149246

150-
## Found an issue? Send a PR
247+
---
151248

152-
If you find any issue with the repo, with the benchmark or the setup, please send a PR. The intention isn't to cherry
153-
pick some example where Turbo doesn't do well because of some weird edge case. If it happens that the repo surfaces some
154-
edge case with how turbo works, send a PR, and let's fix it. We tried to select the setup that Turbo should handle
155-
well (e.g., Next.js apps). The repo doesn't use any incrementality which Nx is very good at. We did our best to make it
156-
fair.
249+
*Keywords: monorepo tools, JavaScript build systems, Nx vs Turbo, build tool performance, cache restoration benchmark, enterprise monorepo, TypeScript build tools, CI/CD optimization, developer experience*

0 commit comments

Comments
 (0)