Skip to content

Commit 7414906

Browse files
culhatskerculhatsker
committed
wip
1 parent edc9179 commit 7414906

30 files changed

+368
-239
lines changed

‎refactor/README.md‎

Lines changed: 25 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,28 +1,30 @@
1-
# Dockerfile generation for OpenVINO releases, for Ubuntu 20.04, Ubuntu 22.04, RHEL 8
1+
# Dockerfile templates for OpenVINO releases, for Ubuntu 20.04, Ubuntu 22.04 and RHEL 8
22

3-
## How to run
3+
## How to use, how to run
44

55
The tool does not require any non-standard Python packages, it only needs Python 3.10+ present
66

7-
`configs/releases` directory contains configurations for all the supported releases. In general, if you want to build
8-
`ubuntu20_dev:2024.3.0` image you should run this command:
7+
`configs/releases` directory contains configurations for all the supported releases. For example, if you want to build
8+
`ubuntu20_dev:2024.3.0` image, then you should run this command:
99

1010
```bash
1111
python3 image.py 2024.3.0/ubuntu20 --preset dev --build
1212
```
13-
This will generate `Dockerfile` and build it, tagging it `localhost/ubuntu20_dev:2024.3.0`
13+
This will generate `Dockerfile`, build it and tag it `localhost/ubuntu20_dev:2024.3.0`
14+
15+
If you add `--test` option, it will also run some tests associated with this image.
1416

1517
## Current support state
1618

1719
Os support:
1820
* Ubuntu 20: ✅
1921
* Ubuntu 22: ✅
20-
* Ubuntu 24: ❌
21-
* RHEL8: ❌ (TODO)
22+
* Ubuntu 24: ❌ (WIP)
23+
* RHEL8: ❌ (WIP)
2224

2325
OpenVINO releases support:
2426
* before 2024.1.0 ❌
25-
* 2024.1.0 ❌(TODO)
27+
* 2024.1.0 ❌(WIP)
2628
* 2024.2.0 ✅
2729
* 2024.3.0 ✅
2830

@@ -37,10 +39,22 @@ Note that even though `Intel NPU` is said to be supported it doesn't mean that e
3739

3840
## How to work with it
3941

42+
To make it easier to test your changes, whatever they are, you might find it useful to install `pytest` package, as it makes checking all images specified in this repository as easy as running `pytest` command in the root directory. It will also let you select or deselect some tests with pytest's `-k` option. Examples:
43+
44+
This will only run dockerfile generation tests:
45+
```
46+
pytest -v -k "generate"
47+
```
48+
49+
This will run tests for all but "nightly" configs (to be precise: all the tests which names don't contain "nightly"):
50+
```
51+
pytest -v -k "not nightly"
52+
```
53+
4054
### When new OpenVINO release
4155

4256
1) Create a new release directory in `config/releases` called after the release version;
43-
2) For each package build for specific os supported by this project, create a json file and use previous versions ad a template.
57+
2) For each package build for specific os supported by this project, create a json file and use previous versions as a template.
4458

4559
### When new OS needs to be supported
4660

@@ -64,12 +78,12 @@ Note: recursion is forbidden, that is, the dependency graph must have no cycles.
6478

6579
TODO: check for recursion, right now it will be infinitely loading if recursion appears.
6680

67-
#### Merging rules
81+
#### Merging rules (a.k.a. what if there are values for the same key in different configs)
6882

6983
1) If either object is null (or if either is missing / is undefined) then the other is returned
7084
2) If objects have different types then an error is returned
7185
3) If objects are dictionaries then they are merged with this algorithm
72-
4) otherwise the new object is returned instead of the old one (including lists)
86+
4) otherwise the new object is returned instead of the old one (including lists!)
7387

7488
TODO: ^^^ describe merging better ^^^
7589

@@ -317,4 +331,3 @@ configuration would look like.
317331
</tr>
318332
</tbody>
319333
</table>
320-

‎refactor/ci.py‎

Whitespace-only changes.

‎refactor/configs/base/common.json‎

Lines changed: 12 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -43,14 +43,16 @@
4343
],
4444
"extras": "caffe,kaldi,mxnet,onnx,pytorch,tensorflow,tensorflow2",
4545
"tests": [
46-
"check_omz_tools.sh"
46+
"check_omz_tools.sh@CPU",
47+
"check_omz_tools.sh@GPU"
4748
]
4849
},
4950
"preset_runtime": {
5051
"requires": [
5152
"base",
5253
"device_cpu",
53-
"device_gpu"
54+
"device_gpu",
55+
"device_npu"
5456
]
5557
},
5658
"preset_dev": {
@@ -59,5 +61,13 @@
5961
"openvino_dev"
6062
]
6163
}
64+
},
65+
"presets": {
66+
"runtime": [
67+
"preset_runtime"
68+
],
69+
"dev": [
70+
"preset_dev"
71+
]
6272
}
6373
}

‎refactor/configs/base/rhel8.json‎

Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,63 @@
1+
{
2+
"_based_on": "common",
3+
"base_image": "registry.access.redhat.com/ubi8/ubi",
4+
"os_id": "rhel8",
5+
"components": {
6+
"build_tools": {
7+
"rpm": [
8+
"gcc",
9+
"gcc-c++",
10+
"make",
11+
"cmake",
12+
"pkgconfig"
13+
]
14+
},
15+
"python": {
16+
"rpm": [
17+
"python39",
18+
"python3-virtualenv",
19+
"python3-pip"
20+
],
21+
"command": "python3.9"
22+
},
23+
"intel-opencl-icd": {
24+
"rpm": [
25+
"https://repositories.intel.com/graphics/rhel/8.6/intel-opencl-22.43.24595.35-i538.el8.x86_64.rpm"
26+
],
27+
"requires": [
28+
"compute-runtime"
29+
]
30+
},
31+
"level-zero": {
32+
"rpm": [
33+
"https://repositories.intel.com/graphics/rhel/8.6/level-zero-1.8.8-i524.el8.x86_64.rpm"
34+
]
35+
},
36+
"intel-level-zero-gpu": {
37+
"rpm": [
38+
"https://repositories.intel.com/graphics/rhel/8.6/intel-level-zero-gpu-1.3.24595.35-i538.el8.x86_64.rpm"
39+
],
40+
"requires": [
41+
"compute-runtime",
42+
"level-zero"
43+
]
44+
},
45+
"compute-runtime": {
46+
"rpm": [
47+
"https://repositories.intel.com/graphics/rhel/8.6/intel-gmmlib-22.3.1-i529.el8.x86_64.rpm",
48+
"https://repositories.intel.com/graphics/rhel/8.6/intel-igc-core-1.0.12504.6-i537.el8.x86_64.rpm",
49+
"https://repositories.intel.com/graphics/rhel/8.6/intel-igc-opencl-1.0.12504.6-i537.el8.x86_64.rpm"
50+
]
51+
},
52+
"opencl-loader": {
53+
"rpm": [
54+
"https://vault.centos.org/8-stream/AppStream/x86_64/os/Packages/ocl-icd-2.2.12-1.el8.x86_64.rpm"
55+
]
56+
},
57+
"device_npu": {
58+
"requires": [],
59+
"tests": [],
60+
"_comment": "rhel8 doesn't support NPU, so we clear NPU requirements"
61+
}
62+
}
63+
}

‎refactor/configs/base/ubuntu18.json‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -53,6 +53,11 @@
5353
"https://github.com/intel/intel-graphics-compiler/releases/download/igc-1.0.8708/intel-igc-core_1.0.8708_amd64.deb",
5454
"https://github.com/intel/intel-graphics-compiler/releases/download/igc-1.0.8708/intel-igc-opencl_1.0.8708_amd64.deb"
5555
]
56+
},
57+
"device_npu": {
58+
"requires": [],
59+
"tests": [],
60+
"_comment": "ubuntu20 doesn't support NPU, so we clear NPU requirements"
5661
}
5762
}
5863
}

‎refactor/configs/base/ubuntu20.json‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,11 @@
3232
"https://github.com/intel/intel-graphics-compiler/releases/download/igc-1.0.12037.1/intel-igc-core_1.0.12037.1_amd64.deb",
3333
"https://github.com/intel/intel-graphics-compiler/releases/download/igc-1.0.12037.1/intel-igc-opencl_1.0.12037.1_amd64.deb"
3434
]
35+
},
36+
"device_npu": {
37+
"requires": [],
38+
"tests": [],
39+
"_comment": "ubuntu20 doesn't support NPU, so we clear NPU requirements"
3540
}
3641
}
3742
}

‎refactor/configs/releases/2023.3/ubuntu18.json‎

Lines changed: 1 addition & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,8 @@
11
{
22
"_based_on": "ubuntu18",
3-
"_template": "Dockerfile_2024_deb_default.j2",
3+
"_template": "Dockerfile_default.j2",
44
"package": {
55
"url": "https://storage.openvinotoolkit.org/repositories/openvino/packages/2023.3/linux/l_openvino_toolkit_ubuntu18_2023.3.0.13775.ceeafaf64f3_x86_64.tgz",
66
"version": "2023.3.0"
7-
},
8-
"presets": {
9-
"runtime": [
10-
"preset_runtime"
11-
],
12-
"dev": [
13-
"preset_dev"
14-
]
157
}
168
}

‎refactor/configs/releases/2023.3/ubuntu20.json‎

Lines changed: 1 addition & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,8 @@
11
{
22
"_based_on": "ubuntu20",
3-
"_template": "Dockerfile_2024_deb_default.j2",
3+
"_template": "Dockerfile_default.j2",
44
"package": {
55
"url": "https://storage.openvinotoolkit.org/repositories/openvino/packages/2023.3/linux/l_openvino_toolkit_ubuntu20_2023.3.0.13775.ceeafaf64f3_x86_64.tgz",
66
"version": "2023.3.0"
7-
},
8-
"presets": {
9-
"runtime": [
10-
"preset_runtime"
11-
],
12-
"dev": [
13-
"preset_dev"
14-
]
157
}
168
}

‎refactor/configs/releases/2023.3/ubuntu22.json‎

Lines changed: 7 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,15 @@
11
{
22
"_based_on": "ubuntu22",
3-
"_template": "Dockerfile_2024_deb_default.j2",
3+
"_template": "Dockerfile_default.j2",
44
"package": {
55
"url": "https://storage.openvinotoolkit.org/repositories/openvino/packages/2023.3/linux/l_openvino_toolkit_ubuntu22_2023.3.0.13775.ceeafaf64f3_x86_64.tgz",
66
"version": "2023.3.0"
77
},
8-
"presets": {
9-
"runtime": [
10-
"preset_runtime"
11-
],
12-
"dev": [
13-
"preset_dev"
14-
]
8+
"components": {
9+
"device_npu": {
10+
"requires": [],
11+
"tests": [],
12+
"_comment": "2023* releases don't support NPU, so we clear NPU requirements"
13+
}
1514
}
1615
}

‎refactor/configs/releases/2024.1/ubuntu20.json‎

Lines changed: 1 addition & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,8 @@
11
{
22
"_based_on": "ubuntu20",
3-
"_template": "Dockerfile_2024_deb_default.j2",
3+
"_template": "Dockerfile_default.j2",
44
"package": {
55
"url": "https://storage.openvinotoolkit.org/repositories/openvino/packages/2024.1/linux/l_openvino_toolkit_ubuntu20_2024.1.0.15008.f4afc983258_x86_64.tgz",
66
"version": "2024.1.0"
7-
},
8-
"presets": {
9-
"runtime": [
10-
"preset_runtime"
11-
],
12-
"dev": [
13-
"preset_dev"
14-
]
157
}
168
}

0 commit comments

Comments
 (0)