Skip to content

Commit a48de87

Browse files
titaiwangmsUbuntu
titaiwangms
and
Ubuntu
authored
[Lint] Add Lint and move metadata from setup.py to pyproject.toml (#120)
This PR basically migrates the lint and CI from ONNX to here. 1. Add lint: black, isort, mypy, and flake8 - flake8 remains basic setting as main branch. - auto-formatter will be introduced in the next PR (#126 ) 3. Add pyproject.toml (includes what was in requirements.txt, setup.cfg, and metadata in setup.py) - Avoiding parsing documents in setup.py can reduce bugs (version_str was -0.1-) - Put metadata all in one file (pyproject.toml ) - benefit: toml becomes more important in python, check [here](https://peps.python.org/pep-0680/) 4. Add lint CI to align coding style - `Lint / Enforce style (pull_request)` CI utilizes style.sh to do the test - We can only make `Lint / Enforce style (pull_request)` as required after a reasonable transition time period 6. Continue work (actually apply lint on files) on #126 fixes #105 Co-authored-by: Ubuntu <titaiwang@titaiwanglinuxcpudev.y3zdd0j2xrqelnmezcgpqgmnte.jx.internal.cloudapp.net>
1 parent 3e8d865 commit a48de87

13 files changed

+325
-75
lines changed

‎.flake8

+16
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
# flake8 is currently not yet supported by pyproject.toml.
2+
# Please move this into pyproject.toml, when it does.
3+
4+
[flake8]
5+
max-line-length = 95
6+
### DEFAULT IGNORES FOR 4-space INDENTED PROJECTS ###
7+
# W503 talks about operator formatting which is too opinionated.
8+
# E203 is need to support black formatting.
9+
# https://black.readthedocs.io/en/stable/guides/using_black_with_other_tools.html
10+
extend-ignore = E203, W503
11+
exclude =
12+
.git,
13+
__pycache__,
14+
.eggs,
15+
**test/models/*.py,
16+
**onnx_backend_test_code/*.py,

‎.github/workflows/lint.yaml

+86
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,86 @@
1+
name: Lint
2+
3+
on:
4+
push:
5+
branches:
6+
- main
7+
pull_request:
8+
9+
jobs:
10+
lint-python:
11+
name: Lint Python
12+
runs-on: ubuntu-latest
13+
steps:
14+
- uses: actions/checkout@v2
15+
- name: Setup Python
16+
uses: actions/[email protected]
17+
with:
18+
python-version: "3.10"
19+
- name: misspell # Check spellings as well
20+
uses: reviewdog/action-misspell@v1
21+
with:
22+
github_token: ${{ secrets.github_token }}
23+
locale: "US"
24+
reporter: github-pr-check
25+
level: info
26+
filter_mode: diff_context
27+
- name: shellcheck # Static check shell scripts
28+
uses: reviewdog/action-shellcheck@v1
29+
with:
30+
github_token: ${{ secrets.github_token }}
31+
reporter: github-pr-check
32+
level: info
33+
filter_mode: diff_context
34+
35+
enforce-style:
36+
name: Enforce style
37+
runs-on: ubuntu-latest
38+
steps:
39+
- uses: actions/checkout@v2
40+
- name: Setup Python
41+
uses: actions/[email protected]
42+
with:
43+
# Version range or exact version of Python to use, using SemVer's version range syntax. Reads from .python-version if unset.
44+
python-version: "3.10"
45+
- name: Install ONNXScript
46+
run: |
47+
# The code is from azure-pipelines.yml
48+
# Install dependencies
49+
python -m pip install --upgrade pip
50+
python -m pip install --upgrade setuptools
51+
python -m pip install -q -r requirements-dev.txt
52+
# Install ONNX
53+
python -m pip uninstall -y onnx-function-experiment
54+
python -m pip uninstall -y ort-function-experiment-nightly
55+
python -m pip install onnx onnxruntime
56+
# install packages
57+
python -m pip install -e .
58+
- name: Run style.sh
59+
run: |
60+
bash tools/style.sh
61+
optional-style:
62+
name: Optional style
63+
runs-on: ubuntu-latest
64+
steps:
65+
- uses: actions/checkout@v2
66+
- name: Setup Python
67+
uses: actions/[email protected]
68+
with:
69+
# Version range or exact version of Python to use, using SemVer's version range syntax. Reads from .python-version if unset.
70+
python-version: "3.10"
71+
- name: Install ONNXScript
72+
run: |
73+
# The code is from azure-pipelines.yml
74+
# Install dependencies
75+
python -m pip install --upgrade pip
76+
python -m pip install --upgrade setuptools
77+
python -m pip install -q -r requirements-dev.txt
78+
# Install ONNX
79+
python -m pip uninstall -y onnx-function-experiment
80+
python -m pip uninstall -y ort-function-experiment-nightly
81+
python -m pip install onnx onnxruntime
82+
# install packages
83+
python -m pip install -e .
84+
- name: Run style_optional.sh
85+
run: |
86+
bash tools/style_optional.sh

‎MANIFEST.in

+1
Original file line numberDiff line numberDiff line change
@@ -7,3 +7,4 @@ include requirements.txt
77

88
# exclude from sdist
99
exclude *.onnx
10+
prune */__pycache__

‎README.md

+39-14
Original file line numberDiff line numberDiff line change
@@ -2,20 +2,6 @@
22

33
ONNXScript is a subset of Python that can be used to author ONNX functions (as well as ONNX models).
44

5-
## Contributing
6-
7-
This project welcomes contributions and suggestions. Most contributions require you to agree to a
8-
Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us
9-
the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.
10-
11-
When you submit a pull request, a CLA bot will automatically determine whether you need to provide
12-
a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions
13-
provided by the bot. You will only need to do this once across all repos using our CLA.
14-
15-
This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/).
16-
For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or
17-
contact [[email protected]](mailto:[email protected]) with any additional questions or comments.
18-
195
## Trademarks
206

217
This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft
@@ -129,5 +115,44 @@ More examples can be found in folder [docs/examples](docs/examples).
129115

130116
## Contributing
131117

118+
This project welcomes contributions and suggestions. Most contributions require you to agree to a
119+
Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us
120+
the rights to use your contribution. For details, visit [Microsoft CLA](https://cla.opensource.microsoft.com).
121+
122+
When you submit a pull request, a CLA bot will automatically determine whether you need to provide
123+
a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions
124+
provided by the bot. You will only need to do this once across all repos using our CLA.
125+
126+
This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/).
127+
For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or
128+
contact [[email protected]](mailto:[email protected]) with any additional questions or comments.
129+
130+
**Unit Test**
131+
132132
Every change impacting the converter or the eager evaluation must be unit tested with
133133
class `OnnxScriptTestCase` to ensure both systems do return the same results with the same inputs.
134+
135+
**Code Style**
136+
137+
We use flake8, black, isort, and mypy to check code format. You can find their configuration in .flake8 and pyproject.toml, and
138+
139+
run `auto-formatter` by:
140+
141+
```bash
142+
./tools/lint.sh
143+
```
144+
145+
and run `check` by:
146+
147+
```bash
148+
./tools/style.sh
149+
```
150+
151+
optional check:
152+
153+
```bash
154+
./tools/style_optional.sh
155+
```
156+
157+
NOTE: mypy and pylint needs to be manually address
158+

‎azure-pipelines.yml

+5-8
Original file line numberDiff line numberDiff line change
@@ -39,15 +39,16 @@ steps:
3939

4040
- script: |
4141
python -m pip install --upgrade pip
42-
pip install -r requirements-dev.txt
42+
python -m pip install --upgrade setuptools
43+
python -m pip install -r requirements-dev.txt
4344
displayName: 'Install dependencies'
4445

4546
- script: |
4647
if [ '$(onnx.standard)' == '1' ]
4748
then
48-
pip uninstall -y onnx-function-experiment
49-
pip uninstall -y ort-function-experiment-nightly
50-
pip install onnx onnxruntime
49+
python -m pip uninstall -y onnx-function-experiment
50+
python -m pip uninstall -y ort-function-experiment-nightly
51+
python -m pip install onnx onnxruntime
5152
fi
5253
displayName: 'Install onnx'
5354

@@ -59,10 +60,6 @@ steps:
5960
pytest -v onnxscript --cov=onnxscript --cov-report term-missing
6061
displayName: 'pytest'
6162

62-
- script: |
63-
python -m flake8 onnxscript --max-line-length 95 --exclude "**test/models/*.py,**onnx_backend_test_code/*.py"
64-
displayName: 'flake8'
65-
6663
- script: |
6764
python setup.py install
6865
displayName: 'install package'

‎onnxscript/__init__.py

+16-3
Original file line numberDiff line numberDiff line change
@@ -3,9 +3,22 @@
33
# Licensed under the MIT License.
44
# --------------------------------------------------------------------------
55

6-
__version__ = '0.1'
7-
6+
import sys
87
from .main import script, export_onnx_lib, OnnxFunction
98
from .backend.onnx_export import export2python as proto2python
109

11-
__all__ = [script, export_onnx_lib, OnnxFunction, proto2python]
10+
if sys.version_info[0:2] >= (3, 8):
11+
import importlib.metadata as importlib_metadata
12+
else:
13+
# TODO: Remove this when Python 3.7 is deprecated
14+
import importlib_metadata
15+
16+
try:
17+
# TODO: should we algin the folder name with package name?
18+
# It's onnxscript and onnx-script now. That way, we can use __package__ here.
19+
__version__ = importlib_metadata.version("onnx-script")
20+
except importlib_metadata.PackageNotFoundError:
21+
__version__ = None
22+
23+
24+
__all__ = ["script", "export_onnx_lib", "OnnxFunction", "proto2python"]

‎pyproject.toml

+97
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,97 @@
1+
# SPDX-License-Identifier: Apache-2.0
2+
3+
[build-system]
4+
requires = ["setuptools", "wheel"]
5+
build-backend = "setuptools.build_meta"
6+
7+
[project]
8+
name = "onnx-script"
9+
version = "0.1"
10+
description = "Authoring ONNX functions in Python"
11+
authors = [{ name = "Microsoft Corporation", email = "[email protected]" }]
12+
# TODO: put url/longer description into here
13+
# Get rid of longer description seems to require Poetry
14+
readme = "README.md"
15+
requires-python = ">=3.7"
16+
license = { text = 'Apache License v2.0' }
17+
classifiers = [
18+
"Development Status :: 4 - Beta",
19+
"Environment :: Console",
20+
"Intended Audience :: Developers",
21+
"Operating System :: MacOS :: MacOS X",
22+
"Operating System :: Microsoft :: Windows",
23+
"Programming Language :: Python :: 3.7",
24+
"Programming Language :: Python :: 3.8",
25+
"Programming Language :: Python :: 3.9",
26+
"Programming Language :: Python :: 3.10",
27+
"License :: OSI Approved :: Apache Software License",
28+
]
29+
# NOTE: replace requirements.txt
30+
dependencies = ["numpy>=1.21.5", "protobuf<4", "onnx"]
31+
32+
[project.optional-dependencies]
33+
test = ["flake8", "mypy", "black", "isort[colors]", "pylint"]
34+
35+
[tool.distutils.bdist_wheel]
36+
universal = true
37+
38+
[tool.pytest.ini_options]
39+
filterwarnings = ["ignore::UserWarning", "ignore::DeprecationWarning"]
40+
41+
[tool.mypy]
42+
strict_optional = true
43+
warn_return_any = true
44+
warn_no_return = true
45+
# TODO warn_unused_ignores = true
46+
warn_redundant_casts = true
47+
warn_incomplete_stub = true
48+
# TODO disallow_untyped_calls = true
49+
check_untyped_defs = true
50+
disallow_any_generics = true
51+
no_implicit_optional = true
52+
# TODO disallow_incomplete_defs = true
53+
# TODO disallow_subclassing_any = true
54+
disallow_untyped_decorators = true
55+
warn_unused_configs = true
56+
show_error_codes = true
57+
show_column_numbers = true
58+
59+
60+
[[tool.mypy.overrides]]
61+
module = "tools.*"
62+
disallow_untyped_defs = true
63+
64+
# Ignore errors in test
65+
[[tool.mypy.overrides]]
66+
module = [
67+
"setup",
68+
"onnxscript.test.models.*",
69+
"onnxscript.test.onnx_backend_test_code.*",
70+
]
71+
ignore_errors = true
72+
73+
[tool.black]
74+
target-version = ["py37", "py38", "py39", "py310"]
75+
extend-exclude = "^onnxscript/test/models"
76+
77+
[tool.isort]
78+
profile = "black"
79+
extend_skip_glob = ["**test/models/*.py", "**onnx_backend_test_code/*.py"]
80+
81+
[tool.pylint.MASTER]
82+
ignore-paths = [
83+
"^onnxscript/test/models/",
84+
"^onnxscript/test/onnx_backend_test_code/",
85+
]
86+
87+
[tool.pylint.'MESSAGES CONTROL']
88+
disable = [
89+
"missing-docstring",
90+
"import-error",
91+
"no-member",
92+
"line-too-long",
93+
"fixme",
94+
"too-few-public-methods",
95+
"no-name-in-module",
96+
"invalid-name", # TODO: Add naming guidance and enable this check.
97+
]

‎requirements-dev.txt

+5-1
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,5 @@
11
autopep8
22
click
3-
flake8
43
numpy==1.21.5
54
matplotlib
65
--pre -f https://onnxruntimepackages.blob.core.windows.net/$web/onnx-function-experiment.html
@@ -16,4 +15,9 @@ pytest-cov
1615
pytest-azurepipelines
1716
pytest-subtests
1817
torch
18+
# Lint
19+
flake8
20+
mypy
21+
black
22+
isort[colors]
1923

‎requirements.txt

-3
This file was deleted.

‎setup.cfg

-11
This file was deleted.

0 commit comments

Comments
 (0)