Skip to content

Commit 143901f

Browse files
narendasanmfeliz-cruisedependabot[bot]bowang007Wei
authored
Update release branch (#1300)
* feat: Jetson specific workspace file Signed-off-by: Naren Dasan <[email protected]> Signed-off-by: Naren Dasan <[email protected]> * docs: Update jetson instructions Signed-off-by: Naren Dasan <[email protected]> Signed-off-by: Naren Dasan <[email protected]> * docs: Update docgen task to not require a special container Signed-off-by: Naren Dasan <[email protected]> Signed-off-by: Naren Dasan <[email protected]> * Checkpoint, initial test and implementation * Fix error in layer conversion caused by zero/ones tensors of wrong type * Sort input values in registerSegmentsOutputs to resolve nondeterminism * lint * lint * feat: Add support for aten::square * chore(deps): bump @actions/core in /.github/actions/assigner Bumps [@actions/core](https://github.com/actions/toolkit/tree/HEAD/packages/core) from 1.8.2 to 1.9.1. - [Release notes](https://github.com/actions/toolkit/releases) - [Changelog](https://github.com/actions/toolkit/blob/main/packages/core/RELEASES.md) - [Commits](https://github.com/actions/toolkit/commits/HEAD/packages/core) --- updated-dependencies: - dependency-name: "@actions/core" dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> * fix: fix misleading skipping partitioning msg Signed-off-by: Bo Wang <[email protected]> * Changes done internally at Facebook (#1288) bd46e8f292bf68fe6b87d2d5d206c89fda79a746 Shirong Wu <[email protected]> Disable group ln fuse pass 6ce1d3bc19d75b266e99355c96daeff7054dcbf8 Wei Wei <[email protected]> [fx2trt] set logging level to INFO at fx root 9d552dc3f69db9e4a249f80ef00803a9413e5d38 Wei Wei <[email protected]> [fx2trt] change OSS method lower_to_trt() to compile() Co-authored-by: wwei6 <[email protected]> * chore: Truncate long and double for ptq CPP path Signed-off-by: Anurag Dixit <[email protected]> * chore: Fix dataloader in finetune_qat script Signed-off-by: Anurag Dixit <[email protected]> * docs: Update docgen task Signed-off-by: Naren Dasan <[email protected]> Signed-off-by: Naren Dasan <[email protected]> * chore: apply linting Signed-off-by: Naren Dasan <[email protected]> Signed-off-by: Naren Dasan <[email protected]> * Update docgen.yml * Update docgen.yml * Update docgen.yml * Update docgen.yml * Update docgen.yml * Update docgen.yml * Update docgen.yml * update fx notebook (#1297) * Changes done internally at Facebook (#1299) bd46e8f292bf68fe6b87d2d5d206c89fda79a746 Shirong Wu <[email protected]> Disable group ln fuse pass 6ce1d3bc19d75b266e99355c96daeff7054dcbf8 Wei Wei <[email protected]> [fx2trt] set logging level to INFO at fx root 9d552dc3f69db9e4a249f80ef00803a9413e5d38 Wei Wei <[email protected]> [fx2trt] change OSS method lower_to_trt() to compile() 6c4bdb8ac5823d161e4afc7c9d295f961aeeb0bf Mor Tzur <[email protected]> fix engine holder test binary to fix contbuild_pytorch_fx2trt_build 636d0ab2a3d0f09267e25b8b8e7eedd4d91d791d Yinghai Lu <[email protected]> [easy] remove random prints 5a97668307c26e69a89a4e02a535e319eaf3ce3d Wei Wei <[email protected]> [ads] sequential linear fuse 508338ab343e407ee49605919508210b62ad9a52 Wei Wei <[email protected]> [fx2trt] minor literal fix * Update docgen.yml * Update docgen.yml * Update docgen.yml * Update docgen.yml * docs: [Automated] Regenerating documenation for 0baf50a Signed-off-by: Torch-TensorRT Github Bot <[email protected]> * fix(tools): Fix linter to not depend on docker (#1301) (#1302) * fix(tools): Fix linter to not depend on docker Signed-off-by: Naren Dasan <[email protected]> * Update linter.yml * Update linter.yml Signed-off-by: Naren Dasan <[email protected]> Signed-off-by: Naren Dasan <[email protected]> Signed-off-by: Naren Dasan <[email protected]> Signed-off-by: Naren Dasan <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: Bo Wang <[email protected]> Signed-off-by: Anurag Dixit <[email protected]> Signed-off-by: Torch-TensorRT Github Bot <[email protected]> Signed-off-by: Naren Dasan <[email protected]> Co-authored-by: Michael Feliz <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Bo Wang <[email protected]> Co-authored-by: Wei <[email protected]> Co-authored-by: wwei6 <[email protected]> Co-authored-by: Anurag Dixit <[email protected]> Co-authored-by: Dheeraj Peri <[email protected]> Co-authored-by: Bo Wang <[email protected]> Co-authored-by: Torch-TensorRT Github Bot <[email protected]>
1 parent 1e198a0 commit 143901f

File tree

165 files changed

+21186
-8488
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

165 files changed

+21186
-8488
lines changed

‎.github/actions/assigner/package-lock.json

Lines changed: 24 additions & 9 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎.github/actions/assigner/package.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@
1919
"license": "BSD-3-Clause",
2020
"repository": "https://www.github.com/pytorch/TensorRT",
2121
"dependencies": {
22-
"@actions/core": "^1.8.2",
22+
"@actions/core": "^1.9.1",
2323
"@actions/github": "^5.0.3",
2424
"fs": "^0.0.1-security",
2525
"js-yaml": "^4.1.0"

‎.github/workflows/docgen.yml

Lines changed: 47 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -10,9 +10,9 @@ on:
1010

1111
jobs:
1212
build-docs:
13-
runs-on: ubuntu-18.04
13+
runs-on: ubuntu-20.04
1414
container:
15-
image: docker.pkg.github.com/pytorch/tensorrt/docgen:latest
15+
image: ghcr.io/pytorch/tensorrt/docgen:latest
1616
credentials:
1717
username: ${{ github.actor }}
1818
password: ${{ secrets.GITHUB_TOKEN }}
@@ -22,23 +22,66 @@ jobs:
2222
rm -rf /usr/share/dotnet
2323
rm -rf /opt/ghc
2424
rm -rf "/usr/local/share/boost"
25+
rm -rf /usr/local/cuda/cuda-*
26+
- name: Install base deps
27+
run: |
28+
apt update
29+
DEBIAN_FRONTEND=noninteractive apt install -y software-properties-common gcc git curl wget make zlib1g-dev bzip2 libbz2-dev lzma lzma-dev libreadline-dev libsqlite3-dev libssl-dev libffi-dev doxygen pandoc
30+
git config --global --add safe.directory '*'
31+
- name: Set up Python 3.9.4
32+
uses: actions/setup-python@v2
33+
with:
34+
python-version: 3.9.4
2535
- uses: actions/checkout@v2
2636
with:
2737
ref: ${{github.head_ref}}
2838
- name: Get HEAD SHA
2939
id: vars
3040
run: echo "::set-output name=sha::$(git rev-parse --short HEAD)"
41+
- name: Get Bazel version
42+
id: bazel_info
43+
run: echo "::set-output name=version::$(cat .bazelversion)"
44+
- name: Install Bazel
45+
run: |
46+
wget -q https://github.com/bazelbuild/bazel/releases/download/${{ steps.bazel_info.outputs.version }}/bazel-${{ steps.bazel_info.outputs.version }}-linux-x86_64 -O /usr/bin/bazel
47+
chmod a+x /usr/bin/bazel
48+
- name: Install cudnn + tensorrt
49+
run: |
50+
apt-get update
51+
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/cuda-ubuntu2004.pin
52+
mv cuda-ubuntu2004.pin /etc/apt/preferences.d/cuda-repository-pin-600
53+
apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/7fa2af80.pub
54+
apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 536F8F1DE80F6A35
55+
apt-key adv --keyserver keyserver.ubuntu.com --recv-keys A4B469963BF863CC
56+
add-apt-repository "deb https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/ /"
57+
apt-get update
58+
apt-get install -y libcudnn8 libcudnn8-dev
59+
60+
apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/3bf863cc.pub
61+
add-apt-repository "deb https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/ /"
62+
apt-get update
63+
64+
apt-get install -y libnvinfer8 libnvinfer-plugin8 libnvinfer-dev libnvinfer-plugin-dev
65+
- name: Install Torch
66+
run: |
67+
python3 -m pip install -r py/requirements.txt
3168
- name: Build Python Package
3269
run: |
33-
cp docker/WORKSPACE.docker WORKSPACE
70+
cp toolchains/ci_workspaces/WORKSPACE.x86_64 WORKSPACE
3471
cd py
72+
echo $(which python3)
73+
echo $(python3 -c 'import site; print(site.getsitepackages()[0])')
74+
mkdir -p /opt/circleci/.pyenv/versions/3.9.4/lib/python3.9/
75+
ln -s $(python3 -c 'import site; print(site.getsitepackages()[0])') /opt/circleci/.pyenv/versions/3.9.4/lib/python3.9/site-packages
3576
python3 setup.py install
77+
cd ..
3678
- name: Generate New Docs
3779
run: |
3880
cd docsrc
39-
pip3 install -r requirements.txt
81+
python3 -m pip install -r requirements.txt
4082
python3 -c "import torch_tensorrt; print(torch_tensorrt.__version__)"
4183
make html
84+
cd ..
4285
- uses: stefanzweifel/git-auto-commit-action@v4
4386
with:
4487
# Required

‎.github/workflows/linter.yml

Lines changed: 25 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -25,25 +25,25 @@ jobs:
2525
- uses: actions/checkout@v2
2626
with:
2727
ref: ${{ github.event.pull_request.head.sha }}
28-
- name: Docker login
29-
uses: docker/login-action@v1
28+
- name: Set up Python 3.9.4
29+
uses: actions/setup-python@v2
3030
with:
31-
registry: docker.pkg.github.com
32-
username: ${{ github.actor }}
33-
password: ${{ secrets.GITHUB_TOKEN }}
34-
- name: Run image
31+
python-version: 3.9.4
32+
- name: Setup env
3533
run: |
36-
docker pull docker.pkg.github.com/pytorch/tensorrt/docgen:latest
37-
docker run -it -d --name cpplinter -e GITHUB_TOKEN=$GITHUB_TOKEN -v $GITHUB_WORKSPACE:/workspace -v $GITHUB_EVENT_PATH:/GITHUB_EVENT.json -w /workspace docker.pkg.github.com/pytorch/tensorrt/docgen:latest
38-
docker exec cpplinter bash -c "cp /workspace/docker/WORKSPACE.docker /workspace/WORKSPACE"
34+
sudo ln -s $GITHUB_EVENT_PATH /GITHUB_EVENT.json
3935
env:
4036
GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN}}
4137
- name: Install dependencies
4238
run: |
43-
docker exec cpplinter bash -c "pip3 install -r /workspace/.github/scripts/requirements.txt"
44-
docker exec cpplinter bash -c "pip3 install -r /workspace/requirements-dev.txt"
39+
pip3 install -r $GITHUB_WORKSPACE/.github/scripts/requirements.txt
40+
pip3 install -r $GITHUB_WORKSPACE/requirements-dev.txt
4541
- name: Lint C++
46-
run: docker exec cpplinter bash -c "cd /workspace && python3 /workspace/.github/scripts/run_cpp_linter.py"
42+
run: |
43+
cd $GITHUB_WORKSPACE
44+
python3 $GITHUB_WORKSPACE/.github/scripts/run_cpp_linter.py
45+
env:
46+
GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN}}
4747

4848
py-linting:
4949
name: Python Linting
@@ -65,22 +65,28 @@ jobs:
6565
- uses: actions/checkout@v2
6666
with:
6767
ref: ${{ github.event.pull_request.head.sha }}
68+
- name: Set up Python 3.9.4
69+
uses: actions/setup-python@v2
70+
with:
71+
python-version: 3.9.4
6872
- name: Docker login
6973
uses: docker/login-action@v1
7074
with:
7175
registry: docker.pkg.github.com
7276
username: ${{ github.actor }}
7377
password: ${{ secrets.GITHUB_TOKEN }}
74-
- name: Run image
78+
- name: Setup env
7579
run: |
76-
docker pull docker.pkg.github.com/pytorch/tensorrt/docgen:latest
77-
docker run -it -d --name pylinter -e GITHUB_TOKEN=$GITHUB_TOKEN -v $GITHUB_WORKSPACE:/workspace -v $GITHUB_EVENT_PATH:/GITHUB_EVENT.json -w /workspace docker.pkg.github.com/pytorch/tensorrt/docgen:latest
78-
docker exec pylinter bash -c "cp /workspace/docker/WORKSPACE.docker /workspace/WORKSPACE"
80+
sudo ln -s $GITHUB_EVENT_PATH /GITHUB_EVENT.json
7981
env:
8082
GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN}}
8183
- name: Install dependencies
8284
run: |
83-
docker exec pylinter bash -c "pip3 install -r /workspace/.github/scripts/requirements.txt"
84-
docker exec pylinter bash -c "pip3 install -r /workspace/requirements-dev.txt"
85+
pip3 install -r $GITHUB_WORKSPACE/.github/scripts/requirements.txt
86+
pip3 install -r $GITHUB_WORKSPACE/requirements-dev.txt
8587
- name: Lint Python
86-
run: docker exec pylinter bash -c "cd /workspace && python3 /workspace/.github/scripts/run_py_linter.py"
88+
run: |
89+
cd $GITHUB_WORKSPACE
90+
python3 $GITHUB_WORKSPACE/.github/scripts/run_py_linter.py
91+
env:
92+
GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN}}

‎core/compiler.cpp

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -426,7 +426,8 @@ torch::jit::Module CompileGraph(const torch::jit::Module& mod, CompileSpec cfg)
426426
auto outputIsCollection = conversion::OutputIsCollection(g->block());
427427
if (cfg.partition_info.enabled &&
428428
(cfg.lower_info.forced_fallback_modules.size() == 0 &&
429-
cfg.partition_info.forced_fallback_operators.size() == 0 && isBlockConvertible)) {
429+
cfg.partition_info.forced_fallback_operators.size() == 0 && isBlockConvertible) &&
430+
!outputIsCollection) {
430431
LOG_INFO("Skipping partitioning since model is fully supported");
431432
}
432433

‎core/conversion/converters/impl/constant_pad.cpp

Lines changed: 6 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,8 @@ auto constant_pad_registrations TORCHTRT_UNUSED = RegisterNodeConversionPatterns
2121
auto padding = args[1].unwrapToIntList().vec();
2222
int64_t padSize = padding.size();
2323
auto value = args[2].unwrapToScalar().to<float>();
24-
24+
at::Tensor value_tensor = torch::tensor(value, util::TRTDataTypeToScalarType(in->getType()));
25+
auto valueTensor = tensor_to_const(ctx, value_tensor);
2526
TORCHTRT_CHECK(padSize % 2 == 0, "Length of pad must be even but instead it equals " << padSize);
2627

2728
int64_t l_pad = padSize / 2;
@@ -55,10 +56,8 @@ auto constant_pad_registrations TORCHTRT_UNUSED = RegisterNodeConversionPatterns
5556
auto fill_layer = ctx->net->addFill(nvinfer1::Dims{1, {1}}, nvinfer1::FillOperation::kLINSPACE);
5657
auto shape_gather_out = ctx->net->addShape(*left_gather_out)->getOutput(0);
5758
fill_layer->setInput(0, *shape_gather_out);
58-
at::Tensor value_tensor = torch::tensor(value, torch::kFloat32);
59-
auto valueTensor = tensor_to_const(ctx, value_tensor);
6059
fill_layer->setInput(1, *valueTensor);
61-
at::Tensor delta_tensor = torch::zeros(inRank);
60+
at::Tensor delta_tensor = torch::zeros(inRank, util::TRTDataTypeToScalarType(in->getType()));
6261
auto deltaTensor = tensor_to_const(ctx, delta_tensor);
6362
fill_layer->setInput(2, *deltaTensor);
6463
auto padTensor = fill_layer->getOutput(0);
@@ -69,10 +68,8 @@ auto constant_pad_registrations TORCHTRT_UNUSED = RegisterNodeConversionPatterns
6968
} else {
7069
inDims.d[axis] = padding[padding_index];
7170
auto fill_layer = ctx->net->addFill(inDims, nvinfer1::FillOperation::kLINSPACE);
72-
at::Tensor value_tensor = torch::tensor(value, torch::kFloat32);
73-
auto valueTensor = tensor_to_const(ctx, value_tensor);
7471
fill_layer->setInput(1, *valueTensor);
75-
at::Tensor delta_tensor = torch::zeros(inRank);
72+
at::Tensor delta_tensor = torch::zeros(inRank, util::TRTDataTypeToScalarType(in->getType()));
7673
auto deltaTensor = tensor_to_const(ctx, delta_tensor);
7774
fill_layer->setInput(2, *deltaTensor);
7875
auto padTensor = fill_layer->getOutput(0);
@@ -112,10 +109,8 @@ auto constant_pad_registrations TORCHTRT_UNUSED = RegisterNodeConversionPatterns
112109
auto fill_layer = ctx->net->addFill(nvinfer1::Dims{1, {1}}, nvinfer1::FillOperation::kLINSPACE);
113110
auto shape_gather_out = ctx->net->addShape(*right_gather_out)->getOutput(0);
114111
fill_layer->setInput(0, *shape_gather_out);
115-
at::Tensor value_tensor = torch::tensor(value, torch::kFloat32);
116-
auto valueTensor = tensor_to_const(ctx, value_tensor);
117112
fill_layer->setInput(1, *valueTensor);
118-
at::Tensor delta_tensor = torch::zeros(inRank);
113+
at::Tensor delta_tensor = torch::zeros(inRank, util::TRTDataTypeToScalarType(in->getType()));
119114
auto deltaTensor = tensor_to_const(ctx, delta_tensor);
120115
fill_layer->setInput(2, *deltaTensor);
121116
auto padTensor = fill_layer->getOutput(0);
@@ -126,10 +121,8 @@ auto constant_pad_registrations TORCHTRT_UNUSED = RegisterNodeConversionPatterns
126121
} else {
127122
inDims.d[axis] = padding[padding_index + 1];
128123
auto fill_layer = ctx->net->addFill(inDims, nvinfer1::FillOperation::kLINSPACE);
129-
at::Tensor value_tensor = torch::tensor(value, torch::kFloat32);
130-
auto valueTensor = tensor_to_const(ctx, value_tensor);
131124
fill_layer->setInput(1, *valueTensor);
132-
at::Tensor delta_tensor = torch::zeros(inRank);
125+
at::Tensor delta_tensor = torch::zeros(inRank, util::TRTDataTypeToScalarType(in->getType()));
133126
auto deltaTensor = tensor_to_const(ctx, delta_tensor);
134127
fill_layer->setInput(2, *deltaTensor);
135128
auto padTensor = fill_layer->getOutput(0);

‎core/conversion/converters/impl/element_wise.cpp

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -390,6 +390,18 @@ auto element_wise_registrations TORCHTRT_UNUSED =
390390
LOG_DEBUG("Output tensor shape: " << out->getDimensions());
391391
return true;
392392
}})
393+
.pattern(
394+
{"aten::square(Tensor self) -> Tensor",
395+
[](ConversionCtx* ctx, const torch::jit::Node* n, args& args) -> bool {
396+
auto self = args[0].ITensorOrFreeze(ctx);
397+
auto mul = add_elementwise(ctx, nvinfer1::ElementWiseOperation::kPROD, self, self, util::node_info(n));
398+
TORCHTRT_CHECK(mul, "Unable to create mul layer from node: " << *n);
399+
400+
mul->setName(util::node_info(n).c_str());
401+
auto out = ctx->AssociateValueAndTensor(n->outputs()[0], mul->getOutput(0));
402+
LOG_DEBUG("Output tensor shape: " << out->getDimensions());
403+
return true;
404+
}})
393405
.pattern(
394406
{"aten::mul.Tensor(Tensor self, Tensor other) -> Tensor",
395407
[](ConversionCtx* ctx, const torch::jit::Node* n, args& args) -> bool {

‎core/partitioning/partitioning.cpp

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -178,7 +178,8 @@ void resolveTRTNonTensorInputs(PartitionedGraph& segmented_blocks) {
178178

179179
void registerSegmentsOutputs(PartitionedGraph& segmented_blocks, torch::jit::Block* block) {
180180
// find the corresponding raw values in original global graph for this segmented block's inputs/outputs
181-
std::set<torch::jit::Value*> input_values;
181+
auto cmp = [](torch::jit::Value* a, torch::jit::Value* b) { return a->unique() < b->unique(); };
182+
std::set<torch::jit::Value*, decltype(cmp)> input_values(cmp);
182183
for (auto& seg_block : segmented_blocks) {
183184
for (auto& input : seg_block.raw_inputs()) {
184185
input_values.insert(input);

0 commit comments

Comments
 (0)