Initial framework of an ethos-u runtime backend #501

robell · 2023-09-27T16:19:10Z

Added shell of runtime Arm Backend for Ethos-U

./backends/arm/cmake/build.sh
will pull the appropriate compiler and build the ethos-u driver and delegate

This is yet to be linked into the executorch runtime

This also needs to be linked into a higher level app with the .pte in SRAM.

This is for early feedback - assuming some fairly heavy changes from here.

Signed-off-by: Rob Elliott <[email protected]>

* basic build system amendments * toolchain file for baremetal platforms * use of ethos-u-core-driver submodule * some scripts to pull a compiler and test-build the backend Signed-off-by: Rob Elliott <[email protected]>

…M, this will be reworked as wrapped ethos buffers in .pte become available Signed-off-by: Rob Elliott <[email protected]>

* Added cmmss submodule dependency * Added command_stream.cpp into the build * Added the target to the build script Signed-off-by: Rob Elliott <[email protected]>

netlify · 2023-09-27T16:19:17Z

✅ Deploy Preview for resplendent-gnome-14e531 ready!

Name	Link
🔨 Latest commit	`7007120`
🔍 Latest deploy log	https://app.netlify.com/sites/resplendent-gnome-14e531/deploys/651ad0bf30e7700008821238
😎 Deploy Preview	https://deploy-preview-501--resplendent-gnome-14e531.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

* vela binaries returned from preprocess * Included in PTE captured from arm_tosa_e2e * currently assumes vela is on path Signed-off-by: Rob Elliott <[email protected]>

Signed-off-by: Rob Elliott <[email protected]>

* added a scratch block in the vela_bin to preallocate it * added a vela_bin reading routine into ArmBackendEthosU * set pointers passed to vela based on vela_bin Signed-off-by: Rob Elliott <[email protected]>

Also helps with c++11 compliance if we have less c++20 code esp when we don't use it.

* fix to encode and decode of vela_bin_stream block sizes * hardcoded input/output population to check operation behaviour * use manual.h to init and register backend Signed-off-by: Rob Elliott <[email protected]>

digantdesai

Feel free to ignore comments for now ;)

digantdesai · 2023-10-02T04:47:48Z

CMakeLists.txt

@@ -106,6 +116,10 @@ if(BUILD_SELECTIVE_BUILD_TEST)
  option(SELECT_OPS_YAML "Register all the ops from a given yaml file" OFF)
 endif()

+# Build Arm Baremetal backend
+option(EXECUTORCH_BUILD_ARM_BAREMETAL


Nice. If we are planning to have multiple delegate flavors - i.e. runtime logic - under arm name then we can do something like,

EXECUTORCH_BUILD_ARM && EXECUTORCH_BUILD_ARM_<RUNTIME_FLAVOR>

Suggested change

option(EXECUTORCH_BUILD_ARM_BAREMETAL

option(EXECUTORCH_BUILD_ARM

digantdesai · 2023-10-02T04:51:56Z

kernels/portable/cpu/op_add.cpp

-    ET_SWITCH_REAL_TYPES_AND(Bool, b_type, ctx, "add", CTYPE_B, [&]() {
-      ET_SWITCH_REAL_TYPES_AND(Bool, common_type, ctx, "add", CTYPE_IN, [&]() {
-        ET_SWITCH_REAL_TYPES_AND(Bool, out_type, ctx, "add", CTYPE_OUT, [&]() {
+//  ET_SWITCH_REAL_TYPES_AND(Bool, a_type, ctx, "add", CTYPE_A, [&]() {


Another alternative is that we can switch the CPU-only example to softmax, while keeping the delegate example to add so we don't have to deal with this.

Because there is no good short-term solution for this besides we write a custom add op in ET which overwrites this Portable::Add

digantdesai · 2023-10-02T04:52:19Z

kernels/portable/cpu/vec_ops.h

@@ -13,6 +13,7 @@
 #include <cstring>
 #include <numeric>
 #include <type_traits>
+#include <cinttypes>


will be merged soon #558

digantdesai · 2023-10-02T04:54:40Z

kernels/portable/functions.yaml

-  kernels:
-    - arg_meta: null
-      kernel_name: torch::executor::acosh_out
-
 - op: add.out


This is temp hack we can do better i.e. supply a new file with one op as oppose to delete rest of the ops in the default one - I was just lazy here :p

digantdesai · 2023-10-02T04:55:21Z

manual.h

+namespace torch {
+namespace executor {
+    void manual_override();
+    void digant_add_out(torch::executor::KernelRuntimeContext & context, EValue** stack);


this was also for testing the manual.h hack - we don't need this

digantdesai · 2023-10-02T04:56:20Z

runtime/core/function_ref.h

-
-  /**
-   * Case 2: A plain function pointer.
+   * Case 1: A plain function pointer.


Nicer fix is almost ready - #554, #555

digantdesai · 2023-10-02T04:57:54Z

runtime/platform/target/Posix.cpp

@@ -52,11 +52,9 @@
 #define _ASSERT_PAL_INITIALIZED()                                   \
  ({                                                                \
    if (!initialized) {                                             \
-      fprintf(                                                      \
-          ET_LOG_OUTPUT_FILE,                                       \
+      printf(                                                       \


So this is because stderr is patched to garbage value at link time when I link libexecutorch.a - easiest solution is to copy this into core_platforms "app" and overwrite this weak symbol.

Long term plan can be to have platform/target/ArmBaremetal.cpp PAL layer.

digantdesai · 2023-10-02T04:59:59Z

backends/arm/arm_backend.py

+
+        tosaname = "out.tosa"
+        flatbuffer = tosa_fb.serialize()
+        f = open(os.path.join(tmpdir,tosaname), "wb")


nit: use with for this resource

digantdesai · 2023-10-02T05:08:30Z

backends/arm/arm_backend.py

+        np_path = os.path.join(tmpdir,"output","out_sg0_vela.npz")
+        blocks = b''
+        with np.load(np_path, allow_pickle=False) as data:
+            # Emit the NPZ regions as:


OK for now but this should be refactored into its own function to emit_block

digantdesai · 2023-10-02T05:12:20Z

backends/arm/cmake/arm-none-eabi-gcc.cmake

+
+# Compilation warnings
+add_compile_options(
+#    -Wall


we (ET runtime) want to enable these eventually (or at least be cool when someone enables them) but right now we are not ready I guess

Signed-off-by: Rob Elliott <[email protected]>

* Removed dependencies on aything but driver * moved to minimal invocation pattern Signed-off-by: Rob Elliott <[email protected]>

* currently assumes function signature * read relevant argument data from vela_bin Signed-off-by: Rob Elliott <[email protected]>

robell · 2023-10-06T10:01:50Z

replaced by #595

robell added 5 commits September 27, 2023 08:34

Add ethos-u-core-driver submodule

7f44e16

Signed-off-by: Rob Elliott <[email protected]>

Added shell of runtime Arm Backend for Ethos-U

c6755bf

* basic build system amendments * toolchain file for baremetal platforms * use of ethos-u-core-driver submodule * some scripts to pull a compiler and test-build the backend Signed-off-by: Rob Elliott <[email protected]>

First-pass ethos-u backend with an assumed flat program format in SRA…

6421ead

…M, this will be reworked as wrapped ethos buffers in .pte become available Signed-off-by: Rob Elliott <[email protected]>

fixed builds of ethos-u-core-driver

1f62acc

* Added cmmss submodule dependency * Added command_stream.cpp into the build * Added the target to the build script Signed-off-by: Rob Elliott <[email protected]>

Merge branch 'pytorch:main' into main

ced2e06

facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Sep 27, 2023

mergennachin requested review from digantdesai, mergennachin, dbort and kimishpatel September 27, 2023 16:26

robell and others added 14 commits September 28, 2023 12:19

Emit Ethos-U55 chunked binaries from preprocess

83a5e32

* vela binaries returned from preprocess * Included in PTE captured from arm_tosa_e2e * currently assumes vela is on path Signed-off-by: Rob Elliott <[email protected]>

added executorch to the build targets

93cfdc2

Signed-off-by: Rob Elliott <[email protected]>

Extended the delegate to read 'vela_bin_stream's

a38f080

* added a scratch block in the vela_bin to preallocate it * added a vela_bin reading routine into ArmBackendEthosU * set pointers passed to vela based on vela_bin Signed-off-by: Rob Elliott <[email protected]>

Merge branch 'pytorch:main' into main

a538747

[ET][Portable] Add int types header

8de6e92

[Executorch] Simplify FunctionRef to make it more portable

644eafc

Also helps with c++11 compliance if we have less c++20 code esp when we don't use it.

[WIP] headrify pte

b307c31

[NOT FOR LAND] Hacks for ET_LOG

52dc73c

[NOT FOR LAND] Hack op_add to reduce size

b2a431f

[NOT FOR LAND] Hacks for add minimal example

77e8eb0

[NOT FOR LAND] HACK for manual kernel registration

9b244b3

[NOT FOR LAND] Allow enabling logging in Release mode

1cabc63

[NOT FOR LAND][arm] setup for core_platform

2f10ee3

Fixes to make simple_add run on hardware

298fb22

* fix to encode and decode of vela_bin_stream block sizes * hardcoded input/output population to check operation behaviour * use manual.h to init and register backend Signed-off-by: Rob Elliott <[email protected]>

digantdesai reviewed Oct 2, 2023

View reviewed changes

robell added 3 commits October 2, 2023 09:39

tidied binary reading and moved to ET_LOG

08d71d9

Signed-off-by: Rob Elliott <[email protected]>

Simplified EthosU invocation code

189b04c

* Removed dependencies on aything but driver * moved to minimal invocation pattern Signed-off-by: Rob Elliott <[email protected]>

Basic ethos output copy to EValue

7007120

* currently assumes function signature * read relevant argument data from vela_bin Signed-off-by: Rob Elliott <[email protected]>

robell mentioned this pull request Oct 3, 2023

Initial framework of an ethos-u runtime backend #2 #595

Closed

robell closed this Oct 6, 2023

robell added the partner: arm For backend delegation, kernels, demo, etc. from the 3rd-party partner, Arm label Jul 4, 2024

Gasoonjia pushed a commit that referenced this pull request Jul 30, 2024

remove macos-12 test (#501)

3c66c45

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Initial framework of an ethos-u runtime backend #501

Initial framework of an ethos-u runtime backend #501

Uh oh!

robell commented Sep 27, 2023

Uh oh!

netlify bot commented Sep 27, 2023 •

edited

Loading

Uh oh!

digantdesai left a comment

Uh oh!

digantdesai Oct 2, 2023

Uh oh!

digantdesai Oct 2, 2023

Uh oh!

digantdesai Oct 2, 2023

Uh oh!

digantdesai Oct 2, 2023

Uh oh!

digantdesai Oct 2, 2023

Uh oh!

digantdesai Oct 2, 2023

Uh oh!

digantdesai Oct 2, 2023

Uh oh!

digantdesai Oct 2, 2023

Uh oh!

digantdesai Oct 2, 2023

Uh oh!

digantdesai Oct 2, 2023

Uh oh!

robell commented Oct 6, 2023

Uh oh!

Uh oh!

	option(EXECUTORCH_BUILD_ARM_BAREMETAL
	option(EXECUTORCH_BUILD_ARM

Initial framework of an ethos-u runtime backend #501

Initial framework of an ethos-u runtime backend #501

Uh oh!

Conversation

robell commented Sep 27, 2023

Uh oh!

netlify bot commented Sep 27, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for resplendent-gnome-14e531 ready!

Uh oh!

digantdesai left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

robell commented Oct 6, 2023

Uh oh!

Uh oh!

netlify bot commented Sep 27, 2023 •

edited

Loading