Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
168 changes: 164 additions & 4 deletions .github/CICD.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ The ODE monorepo uses GitHub Actions for continuous integration and deployment.
#### Triggers

- **Push to `main`**: Builds and publishes release images
- **Push to `develop`**: Builds and publishes pre-release images
- **Push to `dev`**: Builds and publishes pre-release images
- **Push to feature branches**: Builds and publishes branch-specific images
- **Pull Requests**: Builds but does not publish (validation only)
- **Manual Dispatch**: Allows manual triggering with optional version tag
Expand All @@ -37,7 +37,7 @@ Images are published to **GitHub Container Registry (GHCR)**:
| Branch/Event | Tags Generated | Description |
|--------------|----------------|-------------|
| `main` | `latest`, `main-{sha}` | Latest stable release |
| `develop` | `develop`, `develop-{sha}` | Development pre-release |
| `dev` | `dev`, `dev-{sha}` | Development pre-release |
| Feature branches | `{branch-name}`, `{branch-name}-{sha}` | Feature-specific builds |
| Pull Requests | `pr-{number}` | PR validation builds (not pushed) |
| Manual with version | `v{version}`, `v{major}.{minor}`, `latest` | Versioned release |
Expand All @@ -59,6 +59,125 @@ The workflow requires these permissions:

- `GITHUB_TOKEN` - Automatically provided by GitHub Actions

### Update Formplayer Assets

**Workflow File**: `.github/workflows/update-formplayer-assets.yml`

#### Purpose

Automates the build and synchronization of formplayer assets between `formulus-formplayer` (React web app) and `formulus` (React Native app). This eliminates the need for manual asset synchronization PRs and ensures formulus always has the latest built assets.

#### Triggers

- **Push to `main`/`dev`**: Builds assets and commits them to the repository
- **Pull Requests**: Builds assets for validation (does not commit)

#### Path Filters

The workflow only runs when files in these paths change:
- `formulus-formplayer/**` - Any file in the formulus-formplayer project
- `.github/workflows/update-formplayer-assets.yml` - The workflow itself

#### Workflow Behavior

**On Pull Requests:**
1. Builds formplayer assets using `npm run build:rn`
2. Validates that assets were built successfully
3. Uploads assets as GitHub Actions artifact
4. Does NOT commit assets (validation only)

**On Merge (Push to main/dev):**
1. Builds formplayer assets using `npm run build:rn`
2. Checks if assets have changed
3. Commits updated assets to repository with `[skip ci]` message
4. Uploads assets as GitHub Actions artifact
5. Prevents infinite loops by ignoring asset-only commits

#### Asset Location

Assets are copied to:
```
formulus/android/app/src/main/assets/formplayer_dist/
```

#### Permissions Required

- `contents: write` - To commit assets to the repository

#### Benefits

- **Automated**: No manual PRs needed for asset synchronization
- **Consistent**: All developers get latest assets automatically on `git pull`
- **Efficient**: Android builds use committed assets (no build step needed)
- **Safe**: Prevents workflow loops with `[skip ci]` and path filters

### Formulus Android Build

**Workflow File**: `.github/workflows/formulus-android.yml`

#### Purpose

Builds Android APK for the Formulus React Native application, automatically handling formplayer asset dependencies.

#### Triggers

- **Push to `main`/`dev`** (formulus changes only): Builds release APK using committed assets
- **Pull Requests** (formulus changes only): Builds debug APK for validation
- **Workflow Run** (after formplayer assets update): Builds APK using downloaded artifacts
- **Release**: Publishes APK to GitHub Release

#### Path Filters

The workflow runs when files in these paths change:
- `formulus/**` - Any file in the formulus project
- `.github/workflows/formulus-android.yml` - The workflow itself

**Note**: The workflow does NOT directly trigger on `formulus-formplayer/**` changes. Instead, it runs via `workflow_run` after the Update Formplayer Assets workflow completes.

#### Asset Handling

The workflow intelligently handles formplayer assets:

1. **Via workflow_run** (formplayer changed):
- Downloads assets artifact from Update Formplayer Assets workflow
- Uses downloaded assets for Android build

2. **Direct trigger** (formulus-only changes):
- Uses committed assets from repository
- No formplayer build needed

3. **Fallback**:
- If assets don't exist, builds them automatically
- Ensures builds always succeed

#### Build Types

- **Pull Requests**: Debug APK (unsigned)
- **Push to main/dev**: Release APK (signed with secrets)
- **Release**: Release APK published to GitHub Release

#### Secrets Required

- `FORMULUS_RELEASE_KEYSTORE_B64` - Base64 encoded keystore file
- `FORMULUS_RELEASE_STORE_PASSWORD` - Keystore password
- `FORMULUS_RELEASE_KEY_ALIAS` - Key alias
- `FORMULUS_RELEASE_KEY_PASSWORD` - Key password

#### Workflow Integration

The Formulus Android Build workflow integrates with Update Formplayer Assets:

```
Formplayer Changes → Update Assets Workflow → Android Build (via workflow_run)
Formulus Changes → Android Build (direct, uses committed assets)
```

This ensures:
- No duplicate builds
- Single source of truth for assets
- Efficient parallel execution
- Automatic asset synchronization

## Using Published Images

### Pull Latest Release
Expand All @@ -76,7 +195,7 @@ docker pull ghcr.io/opendataensemble/synkronus:v1.0.0
### Pull Development Build

```bash
docker pull ghcr.io/opendataensemble/synkronus:develop
docker pull ghcr.io/opendataensemble/synkronus:dev
```

### Pull Feature Branch Build
Expand Down Expand Up @@ -163,14 +282,55 @@ echo $GITHUB_TOKEN | docker login ghcr.io -u USERNAME --password-stdin
### For Deployments

1. **Pin versions in production**: Use specific version tags, not `latest`
2. **Test pre-releases**: Use `develop` tag for staging environments
2. **Test pre-releases**: Use `dev` tag for staging environments
3. **Monitor image sizes**: Keep images lean for faster deployments
4. **Use health checks**: Always configure health checks in deployments

## Formplayer Asset Synchronization

### How It Works

The automated asset synchronization process ensures formulus always has the latest formplayer assets:

1. **Developer makes changes** to `formulus-formplayer`
2. **Opens/updates PR** → Update Formplayer Assets workflow validates build
3. **PR merged to main/dev** → Update Formplayer Assets workflow:
- Builds assets
- Commits them to repository
- Uploads artifact
4. **Android build triggered** → Formulus Android Build workflow:
- Downloads artifact (if formplayer changed)
- Or uses committed assets (if formulus-only change)
- Builds Android APK

### Benefits

- **No manual work**: Assets are automatically built and committed
- **No conflicts**: Single automated process commits assets
- **Always up-to-date**: Formulus always has latest assets
- **Faster builds**: Android builds use committed assets (no build step)

### Local Development

For local development, you can manually build and copy assets:

```bash
cd formulus-formplayer
npm run build:rn
```

This will:
1. Build the formplayer web app
2. Clean existing assets in formulus
3. Copy new assets to `formulus/android/app/src/main/assets/formplayer_dist/`

The `build:rn` script automatically handles cleaning, so no need to run `clean-rn-assets` separately.

## Future Enhancements

Potential improvements to the CI/CD pipeline:

- [x] Automated formplayer asset synchronization
- [ ] Add automated testing before build
- [ ] Implement security scanning (Trivy, Snyk)
- [ ] Add deployment to staging environment
Expand Down
58 changes: 56 additions & 2 deletions .github/workflows/formulus-android.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,19 @@ on:
paths:
- 'formulus/**'
- '.github/workflows/formulus-android.yml'
paths-ignore:
- 'formulus/android/app/src/main/assets/formplayer_dist/**'
pull_request:
paths:
- 'formulus/**'
- '.github/workflows/formulus-android.yml'
workflow_run:
workflows: ["Update Formplayer Assets"]
types:
- completed
branches:
- main
- dev
release:
types: [published]

Expand All @@ -28,6 +37,31 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
ref: ${{ github.event.workflow_run.head_branch || github.ref }}

- name: Download formplayer assets artifact (from update workflow)
id: download-assets
if: github.event_name == 'workflow_run'
uses: dawidd6/action-download-artifact@v3
with:
name: formplayer-assets
path: formulus/android/app/src/main/assets/formplayer_dist
github_token: ${{ secrets.GITHUB_TOKEN }}
workflow: update-formplayer-assets.yml
run_id: ${{ github.event.workflow_run.id }}
allow_no_artifact: true

- name: Check if assets exist
id: check-assets
run: |
if [ -d "formulus/android/app/src/main/assets/formplayer_dist" ] && [ -n "$(ls -A formulus/android/app/src/main/assets/formplayer_dist 2>/dev/null)" ]; then
echo "exists=true" >> $GITHUB_OUTPUT
echo "✅ Formplayer assets found (from artifact or repo)"
else
echo "exists=false" >> $GITHUB_OUTPUT
echo "⚠️ Formplayer assets not found, will build them"
fi

- name: Set up Node.js
uses: actions/setup-node@v4
Expand All @@ -37,11 +71,31 @@ jobs:
cache-dependency-path: |
formulus/package-lock.json
formulus/package.json
formulus-formplayer/package-lock.json
formulus-formplayer/package.json
packages/tokens/package-lock.json
packages/tokens/package.json

- name: Install npm dependencies
- name: Install npm dependencies (formulus)
working-directory: formulus
run: npm ci

- name: Install and build @ode/tokens
working-directory: packages/tokens
run: |
npm install
npm run build

- name: Install npm dependencies (formplayer)
if: steps.check-assets.outputs.exists != 'true'
working-directory: formulus-formplayer
run: npm ci

- name: Build and bundle formplayer
if: steps.check-assets.outputs.exists != 'true'
working-directory: formulus-formplayer
run: npm run build:rn

- name: Set up Java
uses: actions/setup-java@v4
with:
Expand Down Expand Up @@ -81,7 +135,7 @@ jobs:
working-directory: formulus/android
run: ./gradlew assembleDebug --no-daemon

- name: Build release APK (main/develop/release)
- name: Build release APK (main/dev/release)
if: github.event_name != 'pull_request'
working-directory: formulus/android
run: ./gradlew assembleRelease --no-daemon
Expand Down
Loading
Loading