[mediapipe_task_genai] On Android llm engine failed to validate graph for gemma-2b-it-gpu-int8.bin model #56
Description
Bug report
Describe the bug
LLM Engine failed in ValidatedGraphConfig Initialization step.
Steps to reproduce
Steps to reproduce the behavior:
- Download gemma-2b-it-gpu-int8.bin weight from kaggle
- Upload weight to Android:
adb push gemma-2b-it-gpu-int8.bin /data/local/tmp - Clone the repo, go to folder
flutter-mediapipe/packages/mediapipe-task-genai/exampleand runflutter run -d 9TAUH6MRNZJ7KN6H --dart-define=GEMMA_8B_GPU_URI=/data/local/tmp/gemma-2b-it-gpu-int8.bin, in which9TAUH6MRNZJ7KN6His my Android ID. - On Android screen, click
Gemma 8b GPUand then inputHello, world!and click send button; - Crash, and the log is uploaded crash_log.txt
Expected behavior
Just run one or two conversation successfully.
Additional context
Device Info:
- Device Name: Redmi K60 Ultra
- Hardware: Mediatek mt6985
- API Level 33
- Kernel: 5.15.78
- GPU Mali-G715
- Memory: 24 GB
- Storage: 1 TB
I downloaded the model twice and tested twice, so the model file should be fine.
Flutter doctor
Run flutter doctor and paste the output below:
Click To Expand
Doctor summary (to see all details, run flutter doctor -v):
[✓] Flutter (Channel master, 3.23.0-7.0.pre.3, on Arch Linux 6.6.31-2-lts, locale en_US.UTF-8)
[✓] Android toolchain - develop for Android devices (Android SDK version 34.0.0)
[✓] Chrome - develop for the web
[✓] Linux toolchain - develop for Linux desktop
[!] Android Studio (not installed)
[✓] Connected device (3 available)
[✓] Network resources
! Doctor found issues in 1 category.
Flutter dependencies
Run flutter pub deps -- --style=compact and paste the output below:
Click To Expand
Dart SDK 3.5.0-189.0.dev
Flutter SDK 3.23.0-7.0.pre.3
example 1.0.0+1
dependencies:
- adaptive_dialog 2.1.0 [animations collection dynamic_color flutter intersperse macos_ui meta]
- chat_bubbles 1.6.0 [flutter intl]
- cupertino_icons 1.0.8
- flutter 0.0.0 [characters collection material_color_utilities meta vector_math sky_engine]
- flutter_bloc 8.1.5 [bloc flutter provider]
- flutter_localizations 0.0.0 [flutter intl characters clock collection material_color_utilities meta path vector_math]
- freezed_annotation 2.4.1 [collection json_annotation meta]
- getwidget 4.0.0 [flutter]
- http 1.2.1 [async http_parser meta web]
- intl 0.19.0 [clock meta path]
- logging 1.2.0
- mediapipe_core 0.0.1 [equatable ffi logging meta]
- mediapipe_genai 0.0.1 [async equatable ffi http logging mediapipe_core native_assets_cli native_toolchain_c path]
- path 1.9.0
- path_provider 2.1.3 [flutter path_provider_android path_provider_foundation path_provider_linux path_provider_platform_interface path_provider_windows]
- provider 6.1.2 [collection flutter nested]
- shimmer 3.0.0 [flutter]
- uuid 4.4.0 [crypto sprintf meta fixnum]
dev dependencies:
- build_runner 2.4.10 [analyzer args async build build_config build_daemon build_resolvers build_runner_core code_builder collection crypto dart_style frontend_server_client glob graphs http_multi_server io js logging meta mime package_config path pool pub_semver pubspec_parse shelf shelf_web_socket stack_trace stream_transform timing watcher web_socket_channel yaml]
- flutter_lints 3.0.2 [lints]
- flutter_test 0.0.0 [flutter test_api matcher path fake_async clock stack_trace vector_math leak_tracker_flutter_testing async boolean_selector characters collection leak_tracker leak_tracker_testing material_color_utilities meta source_span stream_channel string_scanner term_glyph vm_service]
- freezed 2.5.2 [analyzer build build_config collection meta source_gen freezed_annotation json_annotation]
transitive dependencies:
- _fe_analyzer_shared 67.0.0 [meta]
- analyzer 6.4.1 [_fe_analyzer_shared collection convert crypto glob meta package_config path pub_semver source_span watcher yaml]
- animations 2.0.11 [flutter]
- appkit_ui_element_colors 1.0.0 [equatable flutter plugin_platform_interface]
- args 2.5.0
- async 2.11.0 [collection meta]
- bloc 8.1.4 [meta]
- boolean_selector 2.1.1 [source_span string_scanner]
- build 2.4.1 [analyzer async convert crypto glob logging meta package_config path]
- build_config 1.1.1 [checked_yaml json_annotation path pubspec_parse yaml]
- build_daemon 4.0.2 [built_collection built_value crypto http_multi_server logging path pool shelf shelf_web_socket stream_transform watcher web_socket_channel]
- build_resolvers 2.4.2 [analyzer async build collection convert crypto graphs logging package_config path pool pub_semver stream_transform yaml]
- build_runner_core 7.3.0 [async build build_config build_resolvers collection convert crypto glob graphs json_annotation logging meta package_config path pool timing watcher yaml]
- built_collection 5.1.1
- built_value 8.9.2 [built_collection collection fixnum meta]
- characters 1.3.0
- checked_yaml 2.0.3 [json_annotation source_span yaml]
- cli_config 0.1.2 [args yaml]
- clock 1.1.1
- code_builder 4.10.0 [built_collection built_value collection matcher meta]
- collection 1.18.0
- convert 3.1.1 [typed_data]
- crypto 3.0.3 [typed_data]
- dart_style 2.3.6 [analyzer args collection path pub_semver source_span]
- dynamic_color 1.7.0 [flutter flutter_test material_color_utilities]
- equatable 2.0.5 [collection meta]
- fake_async 1.3.1 [clock collection]
- ffi 2.1.2
- file 7.0.0 [meta path]
- fixnum 1.1.0
- frontend_server_client 4.0.0 [async path]
- glob 2.1.2 [async collection file path string_scanner]
- gradient_borders 1.0.0 [flutter]
- graphs 2.3.1 [collection]
- http_multi_server 3.2.1 [async]
- http_parser 4.0.2 [collection source_span string_scanner typed_data]
- intersperse 2.0.0
- io 1.0.4 [meta path string_scanner]
- js 0.7.1
- json_annotation 4.9.0 [meta]
- leak_tracker 10.0.5 [clock collection meta path vm_service]
- leak_tracker_flutter_testing 3.0.5 [flutter leak_tracker leak_tracker_testing matcher meta]
- leak_tracker_testing 3.0.1 [leak_tracker matcher meta]
- lints 3.0.0
- macos_ui 2.0.7 [flutter macos_window_utils gradient_borders appkit_ui_element_colors equatable]
- macos_window_utils 1.5.0 [flutter]
- matcher 0.12.16+1 [async meta stack_trace term_glyph test_api]
- material_color_utilities 0.11.1 [collection]
- meta 1.14.0
- mime 1.0.5
- native_assets_cli 0.3.2 [cli_config collection crypto pub_semver yaml yaml_edit]
- native_toolchain_c 0.3.3 [cli_config glob logging meta native_assets_cli pub_semver]
- nested 1.0.0 [flutter]
- package_config 2.1.0 [path]
- path_provider_android 2.2.4 [flutter path_provider_platform_interface]
- path_provider_foundation 2.4.0 [flutter path_provider_platform_interface]
- path_provider_linux 2.2.1 [ffi flutter path path_provider_platform_interface xdg_directories]
- path_provider_platform_interface 2.1.2 [flutter platform plugin_platform_interface]
- path_provider_windows 2.2.1 [ffi flutter path path_provider_platform_interface win32]
- platform 3.1.4
- plugin_platform_interface 2.1.8 [meta]
- pool 1.5.1 [async stack_trace]
- pub_semver 2.1.4 [collection meta]
- pubspec_parse 1.2.3 [checked_yaml collection json_annotation pub_semver yaml]
- shelf 1.4.1 [async collection http_parser path stack_trace stream_channel]
- shelf_web_socket 2.0.0 [shelf stream_channel web_socket_channel]
- sky_engine 0.0.99
- source_gen 1.5.0 [analyzer async build dart_style glob path source_span yaml]
- source_span 1.10.0 [collection path term_glyph]
- sprintf 7.0.0
- stack_trace 1.11.1 [path]
- stream_channel 2.1.2 [async]
- stream_transform 2.1.0
- string_scanner 1.2.0 [source_span]
- term_glyph 1.2.1
- test_api 0.7.1 [async boolean_selector collection meta source_span stack_trace stream_channel string_scanner term_glyph]
- timing 1.0.1 [json_annotation]
- typed_data 1.3.2 [collection]
- vector_math 2.1.4
- vm_service 14.2.2
- watcher 1.1.0 [async path]
- web 0.5.1
- web_socket 0.1.4 [web]
- web_socket_channel 3.0.0 [async crypto stream_channel web web_socket]
- win32 5.5.1 [ffi]
- xdg_directories 1.0.4 [meta path]
- yaml 3.1.2 [collection source_span string_scanner]
- yaml_edit 2.2.1 [collection meta source_span yaml]