Skip to content

Add INSTALL.md with pointers to plugin and operator installation + new HELM instructions #1314

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Feb 13, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
132 changes: 132 additions & 0 deletions INSTALL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,132 @@
# Installing device plugins to cluster

## Install device plugins via a DaemonSet

Each plugin can be installed via a DaemonSet. The install changes slightly based on the desired plugin. See install instructions per [plugin](README.md#plugins).

Installing plugins via DaemonSets deployes them to the ```default``` (or currently active) namespace. Use kubectl's ```--namespace``` argument to change the deployment namespace.

## Install device plugins via device plugin operator

A more advanced install method is via device plugin operator. Operator configures plugin deployments based on the supplied device plugin CRDs (Custom Resource Definitions). See installation instructions in the [operator README](cmd/operator/README.md#installation).

Operator installs device plugins to the same namespace where the operator itself is deployed. The default operator namespace is ```inteldeviceplugins-system```.

## Install with HELM charts

Device plugins can also be installed to a cluster using the device plugin [operator Helm chart](https://github.com/intel/helm-charts/tree/main/charts/device-plugin-operator) (depending on cert-manager and NFD). Individual plugin projects are under https://github.com/intel/helm-charts/tree/main/charts/.

These steps will install device plugin operator and plugins under ```inteldeviceplugins-system``` namespace. It's possible to change the target namespace by changing the ```--namespace``` value in the helm install command.

### Installing HELM repositories

```bash
helm repo add jetstack https://charts.jetstack.io # for cert-manager
helm repo add nfd https://kubernetes-sigs.github.io/node-feature-discovery/charts # for NFD
helm repo add intel https://intel.github.io/helm-charts/ # for device-plugin-operator and plugins
helm repo update
```

### Installing cert-manager

```bash
helm install --wait \
cert-manager jetstack/cert-manager \
--namespace cert-manager \
--create-namespace \
--version v1.11.0 \
--set installCRDs=true
```

NOTE: cert-manager install takes a while to complete.

### Installing NFD

```bash
helm install nfd nfd/node-feature-discovery \
--namespace node-feature-discovery --create-namespace --version 0.12.1 \
--set 'master.extraLabelNs={gpu.intel.com,sgx.intel.com}' \
--set 'master.resourceLabels={gpu.intel.com/millicores,gpu.intel.com/memory.max,gpu.intel.com/tiles,sgx.intel.com/epc}'
```

### Installing operator

```bash
helm install dp-operator intel/intel-device-plugins-operator --namespace inteldeviceplugins-system --create-namespace
```

### Installing specific plugins

Replace PLUGIN with the desired plugin name. At least the following plugins are supported: **gpu, sgx, qat, dlb, dsa & iaa**.

```bash
helm install <PLUGIN> intel/intel-device-plugins-<PLUGIN> --namespace inteldeviceplugins-system --create-namespace \
--set nodeFeatureRule=true
```

### Listing available versions

Use helm's search functionality to list available versions.

```bash
helm search repo intel/intel-device-plugins-operator --versions
helm search repo intel/intel-device-plugins-<plugin> --versions
```

For example, operator chart versions with development versions included.
```bash
$ helm search repo intel/intel-device-plugins-operator --versions --devel
NAME CHART VERSION APP VERSION DESCRIPTION
intel/intel-device-plugins-operator 0.26.0 0.26.0 A Helm chart for Intel Device Plugins Operator ...
intel/intel-device-plugins-operator 0.25.1 0.25.1 A Helm chart for Intel Device Plugins Operator ...
intel/intel-device-plugins-operator 0.25.1-helm.0 0.25.0 A Helm chart for Intel Device Plugins Operator ...
intel/intel-device-plugins-operator 0.25.0 0.25.0 A Helm chart for Intel Device Plugins Operator ...
intel/intel-device-plugins-operator 0.24.1 0.24.1 A Helm chart for Intel Device Plugins Operator ...
intel/intel-device-plugins-operator 0.24.1-helm.0 0.24.0 A Helm chart for Intel Device Plugins Operator ...
intel/intel-device-plugins-operator 0.24.0 0.24.0 A Helm chart for Intel Device Plugins Operator ...
```

### Customizing plugins

To customize plugin features, see the available chart values:
```bash
helm show values intel/intel-device-plugins-<PLUGIN>
```

For example, qat plugin has these values:
```bash
$ helm show values intel/intel-device-plugins-qat
name: qatdeviceplugin-sample

image:
hub: intel
tag: ""

initImage:
hub: intel
tag: ""

dpdkDriver: vfio-pci
kernelVfDrivers:
- c6xxvf
- 4xxxvf
maxNumDevices: 128
logLevel: 4

nodeSelector:
intel.feature.node.kubernetes.io/qat: 'true'

nodeFeatureRule: true
```

### Uninstall

Uninstall each installed component with ```helm uninstall```:

```bash
# repeat first step as many times as there are plugins installed
helm uninstall -n inteldeviceplugins-system <PLUGIN>
helm uninstall -n inteldeviceplugins-system dp-operator
helm uninstall -n node-feature-discovery nfd
helm uninstall -n cert-manager cert-manager
```

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.