Skip to content

Commit 8267311

Browse files
carlosejimenezcarlosejimenez
committed
Update README
1 parent cf1322e commit 8267311

File tree

3 files changed

+342
-11
lines changed

3 files changed

+342
-11
lines changed

‎.github/workflows/deploy.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@ name: Deploy to GitHub Pages
22

33
on:
44
push:
5-
branches: [ master ]
5+
branches: [ master, main ]
66
workflow_dispatch:
77

88
jobs:

‎README.md‎

Lines changed: 276 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,21 +1,287 @@
11
<p align="center">
22
<a href="https://github.com/princeton-nlp/Llamao">
3-
<img src="img/swellama_banner.png" width="50%" alt="swellama logo" />
3+
<img src="img/swe-bench-banner-bg.svg" width="50%" alt="swellama logo" />
44
</a>
55
</p>
66

77
# SWE-bench Website
88

9-
This repository contains the code for the website and leaderboard of the SWE-bench project.
9+
The SWE-bench website for leaderboards and project information.
1010

11-
To learn more about SWE-bench, please check out the main code [repository]() along with the main paper, [SWE-bench: Can Language Models Resolve Real-world Github Issues?]().
11+
## Table of Contents
12+
13+
- [SWE-bench Website](#swe-bench-website)
14+
- [Table of Contents](#table-of-contents)
15+
- [Overview](#overview)
16+
- [Getting Started](#getting-started)
17+
- [Prerequisites](#prerequisites)
18+
- [Installation](#installation)
19+
- [Development](#development)
20+
- [Directory Structure](#directory-structure)
21+
- [How the Website Works](#how-the-website-works)
22+
- [Build Process](#build-process)
23+
- [Template System](#template-system)
24+
- [CSS Organization](#css-organization)
25+
- [JavaScript Usage](#javascript-usage)
26+
- [Customizing the Website](#customizing-the-website)
27+
- [Adding a New Page](#adding-a-new-page)
28+
- [Updating the Leaderboard](#updating-the-leaderboard)
29+
- [Changing the Theme](#changing-the-theme)
30+
- [Adding New Features](#adding-new-features)
31+
- [Deployment](#deployment)
32+
- [Troubleshooting](#troubleshooting)
33+
34+
## Overview
35+
36+
This is the codebase for the [SWE-bench website](https://www.swebench.com), which showcases leaderboards for the SWE-bench benchmark. SWE-bench tests systems' ability to solve GitHub issues automatically.
37+
38+
The site is built using:
39+
- Jinja2 for HTML templating
40+
- Pure CSS (organized in modular files)
41+
- Vanilla JavaScript for interactivity
42+
- Python for the build process
43+
44+
The site is statically generated and can be hosted on GitHub Pages or any other static hosting service.
45+
46+
## Getting Started
47+
48+
### Prerequisites
49+
50+
- Python 3.6 or higher
51+
- `pip` for installing Python packages
52+
53+
### Installation
54+
55+
1. Clone this repository:
56+
```bash
57+
git clone https://github.com/swe-bench/swe-bench.github.io.git
58+
cd swe-bench.github.io
59+
```
60+
61+
2. Create and activate a virtual environment:
62+
```bash
63+
make venv
64+
source venv/bin/activate # On Windows: venv\Scripts\activate
65+
```
66+
67+
3. Install required packages:
68+
```bash
69+
make install
70+
```
71+
72+
### Development
73+
74+
1. Build the site:
75+
```bash
76+
make build
77+
```
78+
79+
2. Start a local development server:
80+
```bash
81+
make serve
82+
```
83+
84+
3. Open your browser and navigate to http://localhost:8000
85+
86+
## Directory Structure
1287

13-
If you found SWE-bench helpful for your work, please cite as follows:
1488
```
15-
@inproceedings{jimenez2023swebench,
16-
title = {SWE-bench: Can Language Models Resolve Real-World GitHub Issues?},
17-
author = {Jimenez, Carlos E. and Yang, John and Wettig, Alexander and Yao, Shunyu and Pei, Kexin and Press, Ofir and Narasimhan, Karthik},
18-
booktitle = {ArXiv},
19-
year = {2023},
20-
}
89+
.
90+
├── build.py # Build script that generates the static site
91+
├── CNAME # Domain configuration for GitHub Pages
92+
├── css/ # CSS files organized by functionality
93+
│ ├── components.css # Styles for UI components
94+
│ ├── core.css # Core styling variables and base styles
95+
│ ├── layout.css # Layout-related styles
96+
│ ├── main.css # CSS entry point that imports all stylesheets
97+
│ ├── pages.css # Page-specific styles
98+
│ └── utilities.css # Utility classes
99+
├── data/ # Data files used in the site generation
100+
│ └── leaderboards.json # Leaderboard data
101+
├── dist/ # Generated static site (created by build.py)
102+
├── favicon.ico # Site favicon
103+
├── img/ # Image assets
104+
├── js/ # JavaScript files
105+
│ ├── citation.js # Citation functionality
106+
│ ├── citationFormat.js # Citation format handlers
107+
│ ├── mainResults.js # Main leaderboard functionality
108+
│ ├── tableByRepo.js # Repository filter functionality
109+
│ ├── tableByYear.js # Year filter functionality
110+
│ └── viewer.js # Results viewer functionality
111+
├── Makefile # Automation for common tasks
112+
├── requirements.txt # Python dependencies
113+
└── templates/ # Jinja2 templates
114+
├── base.html # Base template with common structure
115+
├── _leaderboard_table.html # Reusable leaderboard table component
116+
├── _sidebar.html # Sidebar component
117+
└── pages/ # Page-specific templates
118+
├── citations.html
119+
├── contact.html
120+
├── index.html
121+
├── lite.html
122+
├── multimodal.html
123+
├── submit.html
124+
└── viewer.html
21125
```
126+
127+
## How the Website Works
128+
129+
### Build Process
130+
131+
The website is built using a static site generator implemented in `build.py`. This script:
132+
133+
1. Loads templates from the `templates` directory
134+
2. Loads data from `data/leaderboards.json`
135+
3. Renders each template in `templates/pages/` to a corresponding HTML file in `dist/`
136+
4. Copies static assets (CSS, JS, images, favicon, etc.) to the `dist/` directory
137+
138+
### Template System
139+
140+
The website uses Jinja2 for templating:
141+
142+
- `templates/base.html`: The main template that defines the site structure
143+
- `templates/_sidebar.html`: The sidebar component included in the base template
144+
- `templates/_leaderboard_table.html`: The reusable leaderboard table component
145+
- `templates/pages/*.html`: Individual page templates that extend the base template
146+
147+
Templates use Jinja2 syntax for:
148+
- Template inheritance (`{% extends 'base.html' %}`)
149+
- Including components (`{% include "_sidebar.html" %}`)
150+
- Block definitions and overriding (`{% block content %}{% endblock %}`)
151+
- Variable rendering (`{{ variable }}`)
152+
- Control structures (`{% if condition %}{% endif %}`)
153+
154+
### CSS Organization
155+
156+
CSS is organized into modular files and imported through `main.css`:
157+
158+
- `core.css`: Base styles, variables, and resets
159+
- `layout.css`: Grid and layout components
160+
- `components.css`: UI component styles
161+
- `pages.css`: Page-specific styles
162+
- `utilities.css`: Utility classes for common styling needs
163+
164+
This organization makes it easy to find and update specific styles.
165+
166+
### JavaScript Usage
167+
168+
JavaScript is used for interactive features:
169+
170+
- `mainResults.js`: Main leaderboard functionality including filtering and sorting
171+
- `tableByRepo.js` & `tableByYear.js`: Additional table filtering
172+
- `citation.js` & `citationFormat.js`: Citation formatting and copying
173+
- `viewer.js`: Results viewer page functionality
174+
175+
## Customizing the Website
176+
177+
### Adding a New Page
178+
179+
To add a new page to the website:
180+
181+
1. Create a new HTML file in `templates/pages/`, e.g., `templates/pages/new-page.html`
182+
2. Start with the basic template structure:
183+
```html
184+
{% extends 'base.html' %}
185+
186+
{% block title %}New Page Title{% endblock %}
187+
188+
{% block content %}
189+
<section class="container">
190+
<div class="content-section">
191+
<h2>Your New Page</h2>
192+
<p>Content goes here...</p>
193+
</div>
194+
</section>
195+
{% endblock %}
196+
```
197+
3. Add any page-specific CSS to `css/pages.css`
198+
4. Add any page-specific JavaScript to the `js/` directory and include it in your template:
199+
```html
200+
{% block js_files %}
201+
<script src="js/your-script.js"></script>
202+
{% endblock %}
203+
```
204+
5. Update the sidebar navigation in `templates/_sidebar.html` to include your new page
205+
6. Rebuild the site with `make build`
206+
207+
### Updating the Leaderboard
208+
209+
To update the leaderboard data:
210+
211+
1. Edit `data/leaderboards.json` with the new entries
212+
2. The JSON structure follows this format:
213+
```json
214+
{
215+
"leaderboards": [
216+
{
217+
"name": "Test",
218+
"results": [
219+
{
220+
"name": "Model Name",
221+
"folder": "folder_name",
222+
"resolved": 33.83,
223+
"date": "2025-02-27",
224+
"logs": true,
225+
"trajs": true,
226+
"site": "https://example.com",
227+
"verified": true,
228+
"oss": true,
229+
"org_logo": "https://example.com/logo.png",
230+
"warning": null
231+
},
232+
// More entries...
233+
]
234+
},
235+
// More leaderboards (Verified, Lite, Multimodal)...
236+
]
237+
}
238+
```
239+
3. Each leaderboard has a `name` and a list of `results` entries
240+
4. Each result has various fields describing a model's performance
241+
5. After updating the JSON, rebuild the site with `make build`
242+
243+
### Changing the Theme
244+
245+
To customize the visual appearance:
246+
247+
1. Main color variables are defined in `css/core.css`:
248+
- Edit color variables to change the overall color scheme
249+
- Update typography variables to change fonts
250+
251+
2. Layout structures are in `css/layout.css`:
252+
- Modify container sizes, grid layouts, etc.
253+
254+
3. Component styling is in `css/components.css`:
255+
- Update button styles, card styles, table styles, etc.
256+
257+
4. To add dark mode styles, look for `.dark-mode` selectors throughout the CSS files
258+
259+
### Adding New Features
260+
261+
When adding new features:
262+
263+
1. Avoid directly modifying existing code; extend it instead
264+
2. Add new CSS in an appropriate file based on its purpose
265+
3. Add new JavaScript files for new functionality
266+
4. Update templates to include new components
267+
5. Maintain the existing structure and coding style
268+
269+
## Deployment
270+
271+
The website is designed to be deployed to GitHub Pages:
272+
273+
1. Build the site with `make build`
274+
2. Commit and push changes to the GitHub repository under the `main` or `master` branch
275+
3. Configure the domain to serve from the `gh-pages` branch (root directory) in your repository settings
276+
4. The domain is configured via the `CNAME` file and the GitHub repository settings
277+
278+
See [deploy.yml](.github/workflows/deploy.yml) for the GitHub Actions workflow that handles automatic deployment to GitHub Pages.
279+
280+
## Troubleshooting
281+
282+
Common issues:
283+
284+
- **Build fails**: Make sure you have all dependencies installed with `make install`
285+
- **CSS changes not appearing**: Check if you're editing the correct CSS file and if it's imported in `main.css`. Force refresh the page (Cmd+Shift+R) if changes aren't showing.
286+
- **JavaScript not working**: Check browser console for errors and ensure your script is included in the template
287+
- **Template changes not appearing**: Make sure you're building the site after making changes with `make build`. `make serve` also builds the site automatically.

0 commit comments

Comments
 (0)