Error importing Shared Code into Python function

[wbreza]

Hack required to import Shared Code modules into Python functions

Investigative information

Repro steps

1. Create a folder SharedCode adjacent to your function folders
2. Added a python module into the SharedCode folder
3. Attempt to import your SharedCode modules into existing Python Azure Function
4. Run your Python Azure Function

Expected behavior

Module should be available by default because it is inside the same virtual environment without requiring any PYTHONPATH mods.

Actual behavior

Error:

Microsoft.Azure.WebJobs.Host.FunctionInvocationException: Exception while executing function: Functions.HttpTriggerPython ---> Microsoft.Azure.WebJobs.Script.Rpc.RpcException: Result: Failure
Exception: ModuleNotFoundError: No module named 'SharedCode'

Known workarounds

Add the following lines to the top of your __init__.py in your Azure Function. This will allow any modules defined in your FunctionApp to be available for import

import sys
import os
sys.path.append(os.path.abspath(""))

[elprans]

You don't need any hacks. Import adjacent modules using the relative import syntax:

from . import SharedCode

@asavaritayal Perhaps we should document this in a conspicuous fashion, since this is not the first time this issue has come up: #81

[wbreza]

@elprans Not working for my case.

from . import SharedCode results in error Exception: ImportError: cannot import name 'SharedCode'

I would like to import a submodule using the following or similar syntax
from SharedCode import MyTestClass where MyTestClass is a class defined in module1.py for example.

Folder structure looks like the following

FunctionApp
|_ HttpTrigger1
    |_ __init__.py
    |_ function.json
|_ HttpTrigger2
    |_ __init__.py
    |_ function.json
|_ SharedCode
    |_ __init__.py
    |_ module1.py
    |_ module2.py

[elprans]

@wbreza In your case your shared module code is one level up, so the correct syntax would be

from .. import SharedCode

[elprans]

To import submodules directly:

from ..SharedCode import module1

[tbarlow12]

@elprans I changed all of our imports to relative imports like you indicated above:

from ..Common import ResourceTagger
from azure.functions import QueueMessage
import ..Adapters

def main(msg: QueueMessage):
    ResourceTagger.process_queue_message(msg)

I get the following error:

Exception while executing function: Functions.StoreResources. System.Private.CoreLib: Result: Failure
Exception: SyntaxError: invalid syntax
Stack:   File "C:\Users\tabarlow\source\virtualenvs\resource-scanner\lib\site-packages\azure\functions_worker\dispatcher.py", line 202, in _handle__function_load_request
    func_request.metadata.script_file)
  File "C:\Users\tabarlow\source\virtualenvs\resource-scanner\lib\site-packages\azure\functions_worker\loader.py", line 62, in load_function
    mod = importlib.import_module(fullmodname)
  File "C:\Users\tabarlow\source\virtualenvs\resource-scanner\lib\importlib\__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 941, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 674, in exec_module
  File "<frozen importlib._bootstrap_external>", line 781, in get_code
  File "<frozen importlib._bootstrap_external>", line 741, in source_to_code
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed

This does not occur when using the hack described above:

import sys
import os
sys.path.append(os.path.abspath(""))

Just for reference, this is running on a clean virtualenv with the following dependencies:

gazure-cosmosdb-table
azure-functions==1.0.0a4
azure-functions-worker==1.0.0a4
grpcio==1.14.2
grpcio-tools==1.14.2
protobuf==3.6.1
six==1.11.0
azure-mgmt-nspkg
azure-mgmt-resource
azure-mgmt-subscription
azure-servicebus
azure-storage
azure-storage-common
elasticsearch
click==6.7
PyMySQL
python-dotenv
pytest
pytest-cov

[elprans]

@wbreza Change import ..Adapters to from .. import Adapters

[wbreza]

@elprans - Thanks - this does work but based on Python best practices that I have read relative imports are frowned upon. Some pros/cons and discussion @ https://realpython.com/absolute-vs-relative-python-imports/

I am not a python expert myself but in refactoring our project to support relative import it caused many downstream issue with how pytest and other modules are invoked.

Any idea how to make this process easier by allowing Absolute imports?

[elprans]

@wbreza If you have a lot of shared code, the best approach would be to put it into a standalone pip-installable package and add that as a dependency of your function app. This way you would be able to perform absolute imports.

[priyaananthasankar]

@asavaritayal is there a plan to record these best practice recommendations in the developer guide? Especially since these are idiomatic to Python.

[wbreza]

@elprans - Take your point, but keep in mind not all consumers of this will be able to push their IP to pypi unless they could host it privately. We may unfortunately actually just stick with our path workaround mentioned above.

[elprans]

@wbreza We can add support for building dependencies from local sources when publishing. Pushing to PyPI should not be necessary, all you'll need to do is put a path in requirements.txt:

public-package==1.0.0
...
file:///my/proprietary/code

or, if you have pre-built wheels:

public-package==1.0.0
...
file:///my/proprietary/code-2.0.whl

We chose against adding the function app folder to sys.path because the module namespace in Python is flat, so if you name your function numpy, you wouldn't be able to import the real library and this may cause massive confusion, especially if the masked module is not a direct dependency.

[asavaritayal]

@asavaritayal is there a plan to record these best practice recommendations in the developer guide? Especially since these are idiomatic to Python.

Yes, we should add this to the developer guide. @elprans can you update the wiki?