Detected blocking call to scandir with args (b'/config/pyscript',)

[psykokwak-com]

Since last HA update, I have the following error in loop :

2024-07-08 11:06:49.641 WARNING (MainThread) [homeassistant.util.loop] Detected blocking call to scandir with args (b'/config/pyscript',) inside the event loop by custom integration 'pyscript' at custom_components/pyscript/init.py, line 335: observer.start() (offender: , line 366: ?), please create a bug report at https://github.com/custom-components/pyscript/issues
For developers, please see https://developers.home-assistant.io/docs/asyncio_blocking_operations/#scandir
Traceback (most recent call last):
File "", line 198, in _run_module_as_main
File "", line 88, in _run_code
File "/usr/src/homeassistant/homeassistant/main.py", line 223, in
sys.exit(main())
File "/usr/src/homeassistant/homeassistant/main.py", line 209, in main
exit_code = runner.run(runtime_conf)
File "/usr/src/homeassistant/homeassistant/runner.py", line 190, in run
return loop.run_until_complete(setup_and_run_hass(runtime_config))
File "/usr/local/lib/python3.12/asyncio/base_events.py", line 674, in run_until_complete
self.run_forever()
File "/usr/local/lib/python3.12/asyncio/base_events.py", line 641, in run_forever
self._run_once()
File "/usr/local/lib/python3.12/asyncio/base_events.py", line 1990, in _run_once
handle._run()
File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run
self._context.run(self._callback, *self._args)
File "/usr/src/homeassistant/homeassistant/runner.py", line 166, in setup_and_run_hass
return await hass.async_run()
File "/usr/src/homeassistant/homeassistant/core.py", line 523, in async_run
await self.async_start()
File "/usr/src/homeassistant/homeassistant/core.py", line 576, in async_start
self.bus.async_fire_internal(EVENT_HOMEASSISTANT_STARTED)
File "/usr/src/homeassistant/homeassistant/core.py", line 1559, in async_fire_internal
self._hass.async_run_hass_job(job, event)
File "/usr/src/homeassistant/homeassistant/core.py", line 941, in async_run_hass_job
return self._async_add_hass_job(hassjob, *args, background=background)
File "/usr/src/homeassistant/homeassistant/core.py", line 756, in async_add_hass_job
task = create_eager_task(
File "/usr/src/homeassistant/homeassistant/util/async.py", line 37, in create_eager_task
return Task(coro, loop=loop, name=name, eager_start=True)
File "/config/custom_components/pyscript/init.py", line 335, in hass_started
observer.start()

My HA informations :

System Information

version	core-2024.7.1
installation_type	Home Assistant OS
dev	false
hassio	true
docker	true
user	root
virtualenv	false
python_version	3.12.4
os_name	Linux
os_version	6.6.33-haos
arch	x86_64
timezone	Europe/Paris
config_dir	/config

Home Assistant Community Store

GitHub API	ok
GitHub Content	ok
GitHub Web	ok
GitHub API Calls Remaining	4785
Installed Version	1.34.0
Stage	running
Available Repositories	1394
Downloaded Repositories	10

Home Assistant Cloud

logged_in	false
can_reach_cert_server	ok
can_reach_cloud_auth	ok
can_reach_cloud	ok

Home Assistant Supervisor

host_os	Home Assistant OS 12.4
update_channel	stable
supervisor_version	supervisor-2024.06.2
agent_version	1.6.0
docker_version	26.1.4
disk_total	503.3 GB
disk_used	169.2 GB
healthy	true
supported	true
host_connectivity	true
supervisor_connectivity	true
ntp_synchronized	true
virtualization	kvm
board	ova
supervisor_api	ok
version_api	ok
installed_addons	Terminal & SSH (9.14.0), File editor (5.8.0), Samba share (12.3.1), Studio Code Server (5.15.0), SQLite Web (4.1.2)

Dashboards

dashboards	2
resources	7
views	9
mode	storage

Recorder

oldest_recorder_run	8 juillet 2024 à 09:06
current_recorder_run	8 juillet 2024 à 11:06
estimated_db_size	3.54 MiB
database_engine	sqlite
database_version	3.45.3

Xiaomi Miot Auto

component_version	0.7.18
can_reach_server	ok
can_reach_spec	ok
logged_accounts	1
total_devices	4

[craigbarratt]

This is a repeat of #612. It should be fixed with commit 27ca121. It would be great if you could test the latest code in github to confirm it's fixed (and not otherwise broken).

[psykokwak-com]

Thanks for your answer.
I just updated the latest code from github and I still have the issue :

2024-07-08 13:05:59.927 WARNING (MainThread) [homeassistant.util.loop] Detected blocking call to scandir with args (b'/config/pyscript',) inside the event loop by custom integration 'pyscript' at custom_components/pyscript/__init__.py, line 340: observer.start() (offender: <frozen os>, line 366: ?), please create a bug report at https://github.com/custom-components/pyscript/issues
For developers, please see https://developers.home-assistant.io/docs/asyncio_blocking_operations/#scandir
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/usr/src/homeassistant/homeassistant/__main__.py", line 223, in <module>
    sys.exit(main())
  File "/usr/src/homeassistant/homeassistant/__main__.py", line 209, in main
    exit_code = runner.run(runtime_conf)
  File "/usr/src/homeassistant/homeassistant/runner.py", line 190, in run
    return loop.run_until_complete(setup_and_run_hass(runtime_config))
  File "/usr/local/lib/python3.12/asyncio/base_events.py", line 674, in run_until_complete
    self.run_forever()
  File "/usr/local/lib/python3.12/asyncio/base_events.py", line 641, in run_forever
    self._run_once()
  File "/usr/local/lib/python3.12/asyncio/base_events.py", line 1990, in _run_once
    handle._run()
  File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run
    self._context.run(self._callback, *self._args)
  File "/config/custom_components/pyscript/__init__.py", line 340, in hass_started
    observer.start()

[craigbarratt]

Thanks for checking,. My fix didn't work. I'm traveling tomorrow and I'll try to get to it on the flight (assuming the WiFi is decent).

[ProphetOfDoom]

Just to let you know I'm getting the same warning here. I know it's affected a number of other integrations (Such as the Solcast Solar PV integration https://github.com/BJReplay/ha-solcast-solar). If you can stick the changes up as a beta release I'll give it a whirl. There was some discussion over on that repo (new management!) about how to do that if you aren't sure.

[craigbarratt]

Ok, I pushed another potential fix 774842d. Just about to depart on a flight, so I haven't tested it much....

[psykokwak-com]

Thanks,
Seems good this time 👍

[ProphetOfDoom]

Just updated my HA instance with code from pyscript/custom_components/pyscript, restarted HA and run my automation that uses PyScript.
No PyScript Warnings/Errors in HA system log
Thought you might like to know.

[craigbarratt]

Thanks for checking and confirming the fix!

[Joao-Sousa-71]

@craigbarratt I've updated my HA instance and the warning disappeared for all my pyscripts except one:

2024-07-09 19:36:35.106 WARNING (MainThread) [homeassistant.util.loop] Detected blocking call to open with args ('/config/multiscrape/solzaima_ha_integration/page_response_headers.txt', 'rb') inside the event loop by custom integration 'pyscript' at custom_components/pyscript/eval.py, line 1982: return func(*args, **kwargs) (offender: /usr/local/lib/python3.12/shutil.py, line 260: with open(src, 'rb') as fsrc:), please create a bug report at https://github.com/custom-components/pyscript/issues
For developers, please see https://developers.home-assistant.io/docs/asyncio_blocking_operations/#open
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/usr/src/homeassistant/homeassistant/__main__.py", line 223, in <module>
    sys.exit(main())
  File "/usr/src/homeassistant/homeassistant/__main__.py", line 209, in main
    exit_code = runner.run(runtime_conf)
  File "/usr/src/homeassistant/homeassistant/runner.py", line 190, in run
    return loop.run_until_complete(setup_and_run_hass(runtime_config))
  File "/usr/local/lib/python3.12/asyncio/base_events.py", line 674, in run_until_complete
    self.run_forever()
  File "/usr/local/lib/python3.12/asyncio/base_events.py", line 641, in run_forever
    self._run_once()
  File "/usr/local/lib/python3.12/asyncio/base_events.py", line 1990, in _run_once
    handle._run()
  File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run
    self._context.run(self._callback, *self._args)
  File "/config/custom_components/pyscript/function.py", line 448, in run_coro
    result = await coro
  File "/config/custom_components/pyscript/eval.py", line 508, in do_service_call
    retval = await func.call(ast_ctx, **data)
  File "/config/custom_components/pyscript/eval.py", line 800, in call
    val = await self.try_aeval(ast_ctx, arg1)
  File "/config/custom_components/pyscript/eval.py", line 700, in try_aeval
    return await ast_ctx.aeval(arg)
  File "/config/custom_components/pyscript/eval.py", line 940, in aeval
    val = await getattr(self, name, self.ast_not_implemented)(arg)
  File "/config/custom_components/pyscript/eval.py", line 1029, in ast_if
    val = await self.aeval(arg1)
  File "/config/custom_components/pyscript/eval.py", line 940, in aeval
    val = await getattr(self, name, self.ast_not_implemented)(arg)
  File "/config/custom_components/pyscript/eval.py", line 1367, in ast_expr
    return await self.aeval(arg.value)
  File "/config/custom_components/pyscript/eval.py", line 940, in aeval
    val = await getattr(self, name, self.ast_not_implemented)(arg)
  File "/config/custom_components/pyscript/eval.py", line 1937, in ast_call
    return await self.call_func(func, func_name, *args, **kwargs)
  File "/config/custom_components/pyscript/eval.py", line 1982, in call_func
    return func(*args, **kwargs)

Now, I dont' if it is an issue of the script itself... but it was not showing any warning before the HA upgrade:
Here is the script:

@service
def copy_myceza_response_headers():
	"""Python Script to copy page_response_headers.txt to the sensor folder file"""
	import os.path
	import shutil
	source_folder = "/config/multiscrape/solzaima_ha_integration/page_response_headers.txt"	
	target_folder = "/config/multiscrape/tokens/page_response_headers.txt"
	check_file = os.path.isfile(source_folder)
	if check_file:
		shutil.copyfile(source_folder, target_folder)
		log.info(f"Python Script to copy page_response_headers.txt to the sensor folder file finished!!!")
	else:
		log.info(f"page_response_headers.txt does not exist in source folder!!!")

Thank you!

[craigbarratt]

Your code is doing file system operations (os.path.isfile and shutil.copyfile), which are considered blocking operations. You could solve this by making this function run in its own thread with the @pyscript_executor decorator (note that log.info() won't work inside a function with that decorator). See the docs.

[Joao-Sousa-71]

I’ll try to make the changes and check how it goes, I,m not sure if my Python knowledge is enough for this. Thank you for your feedback.