ZHA (zigbee) not working after home assistant core update to 21.8.8-14

[qyz232]

Setup

Package Name: Home Assistant Core
Package Version: 21.8.8-14

NAS Model: DS216play
NAS Architecture: Arm (STM Monaco STiH412)
DSM version: 6.2.4-25556 Update 1

Expected behavior

Zigbee integration (zha) worked before the upgrade from previous version (20.xx? + Python 3.7)

Actual behavior

Home Assistant installs and runs but fails on the ZHA integration
Within the Home Assistant gui either wait endlessly on "please while integration is being setup" or "Config flow could not be loaded”

The log reports: Unable to install package bellows==0.26.0

I found an issue much like it posted by rs1gg commented on Feb 27 #4458 #4580
So I followed the advise and executed the same command ./pip install bellows==0.21.0 zha-quirks==0.0.51 zigpy-cc==0.5.2 zigpy-deconz==0.11.1 zigpy-xbee==0.13.0 zigpy-zigate==0.7.3 zigpy-znp==0.3.0 zigpy==0.30.0 from /var/packages/homeassistant/target/env/bin:

But unlucky for me without success - it seems to fail on the PyCryptoDome again...

Steps to reproduce

1. Install python 3.8.11 from syno community repository
2. Install Home Assistant Core 2021.8.8
3. In the homeassistant gui, add the zha integration,
4. run ./pip install bellows==0.21.0 (or bellows==0.26.0)

I'm far from expert in this matter but with a little guidance I hope to resolve this

home assistant log

2021-09-02 16:20:58 INFO (SyncWorker_2) [homeassistant.util.package] Attempting install of bellows==0.26.0
2021-09-02 16:21:42 ERROR (SyncWorker_2) [homeassistant.util.package] Unable to install package bellows==0.26.0: ERROR: Command errored out with exit status 1:
   command: /volume1/@appstore/homeassistant/env/bin/python3 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/volume1/@appstore/homeassistant/tmp/pip-install-a3ijspx7/pycryptodome_4bb799dc102e4c428cd43dba1aab90f5/setup.py'"'"'; __file__='"'"'/volume1/@appstore/homeassistant/tmp/pip-install-a3ijspx7/pycryptodome_4bb799dc102e4c428cd43dba1aab90f5/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /volume1/@appstore/homeassistant/tmp/pip-wheel-6j1o2fl8
       cwd: /volume1/@appstore/homeassistant/tmp/pip-install-a3ijspx7/pycryptodome_4bb799dc102e4c428cd43dba1aab90f5/
  Complete output (311 lines):
  Testing support for clang
<<SNIPSNIP>>--------------
  Target does not support SSE2(emmintrin.h)
  Warning: compiler does not support AESNI instructions
  Warning: compiler does not support CLMUL instructions
  running bdist_wheel
  running build
  running build_py
  creating build/lib.linux-armv7l-3.8
  creating build/lib.linux-armv7l-3.8/Crypto
  copying lib/Crypto/__init__.py -> build/lib.linux-armv7l-3.8/Crypto
  creating build/lib.linux-armv7l-3.8/Crypto/Cipher
  copying lib/Crypto/Cipher/ChaCha20.py -> build/lib.linux-armv7l-3.8/Crypto/Cipher
<<SNIPSNIP>>--------------
  copying lib/Crypto/Math/_IntegerNative.pyi -> build/lib.linux-armv7l-3.8/Crypto/Math
  running build_ext
  building 'Crypto.Hash._MD2' extension
  creating build/temp.linux-armv7l-3.8
  creating build/temp.linux-armv7l-3.8/src
  /spksrc/toolchain/syno-armv7-6.1/work/arm-unknown-linux-gnueabi/bin/arm-unknown-linux-gnueabi-gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -I/spksrc/toolchain/syno-armv7-6.1/work/arm-unknown-linux-gnueabi/arm-unknown-linux-gnueabi/sysroot/usr/include -D__ARM_PCS_VFP=1 -I/spksrc/spk/python38/work-armv7-6.1/install//var/packages/python38/target/include -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -L /spksrc/spk/python38/work-armv7-6.1/install/var/packages/python38/target/lib -I /spksrc/spk/python38/work-armv7-6.1/install/var/packages/python38/target/include -I/spksrc/toolchain/syno-armv7-6.1/work/arm-unknown-linux-gnueabi/arm-unknown-linux-gnueabi/sysroot/usr/include -D__ARM_PCS_VFP=1 -I/spksrc/spk/python38/work-armv7-6.1/install//var/packages/python38/target/include -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -L /spksrc/spk/python38/work-armv7-6.1/install/var/packages/python38/target/lib -I /spksrc/spk/python38/work-armv7-6.1/install/var/packages/python38/target/include -fPIC -DPYCRYPTO_LITTLE_ENDIAN -DSYS_BITS=32 -DLTC_NO_ASM -Isrc/ -I/volume1/@appstore/homeassistant/env/include -I/var/packages/python38/target/include/python3.8 -c src/MD2.c -o build/temp.linux-armv7l-3.8/src/MD2.o
  unable to execute '/spksrc/toolchain/syno-armv7-6.1/work/arm-unknown-linux-gnueabi/bin/arm-unknown-linux-gnueabi-gcc': No such file or directory
  error: command '/spksrc/toolchain/syno-armv7-6.1/work/arm-unknown-linux-gnueabi/bin/arm-unknown-linux-gnueabi-gcc' failed with exit status 1
  ----------------------------------------
  ERROR: Failed building wheel for pycryptodome
    ERROR: Command errored out with exit status 1:
     command: /volume1/@appstore/homeassistant/env/bin/python3 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/volume1/@appstore/homeassistant/tmp/pip-install-a3ijspx7/pycryptodome_4bb799dc102e4c428cd43dba1aab90f5/setup.py'"'"'; __file__='"'"'/volume1/@appstore/homeassistant/tmp/pip-install-a3ijspx7/pycryptodome_4bb799dc102e4c428cd43dba1aab90f5/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /volume1/@appstore/homeassistant/tmp/pip-record-bhcvp1hp/install-record.txt --single-version-externally-managed --user --prefix= --compile --install-headers /var/packages/homeassistant/target/.local/include/site/python3.8/pycryptodome
         cwd: /volume1/@appstore/homeassistant/tmp/pip-install-a3ijspx7/pycryptodome_4bb799dc102e4c428cd43dba1aab90f5/
<<SNIPSNIP>>--------------

    raise RequirementsNotFound(name, [req])
homeassistant.requirements.RequirementsNotFound: Requirements for zha not found: ['bellows==0.26.0'].

Some similar I will receive from Putty when trying to run pip install zigpy==0.30.0 or bellows==0.21.0

ERROR: Command errored out with exit status 1: /volume1/@appstore/homeassistant/env/bin/python3 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-c7js1a8q/pycryptodome_3f088c13dbdd469b94ac2014c2fd1622/setup.py'"'"'; __file__='"'"'/tmp/pip-install-c7js1a8q/pycryptodome_3f088c13dbdd469b94ac2014c2fd1622/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-jnsf99eb/install-record.txt --single-version-externally-managed --compile --install-headers /volume1/@appstore/homeassistant/env/include/site/python3.8/pycryptodome

[ggens]

+1

[qyz232]

I have been re-installing the Python and Home Assistant packages a number of times. Trying to find and remove any left overs before doing a new install. But I feel I'm not making any progress.

I did notice however that also some other integrations are no longer working like my SolarEdge.
ZHA just being the most important one as it handles the comm's of my light plan.

2021-09-06 22:04:45 INFO (SyncWorker_0) [homeassistant.util.package] Attempting install of solaredge==0.0.2
2021-09-06 22:05:02 INFO (SyncWorker_6) [homeassistant.util.package] Attempting install of stringcase==1.2.0
2021-09-06 22:05:20 ERROR (MainThread) [homeassistant.config_entries] Error occurred loading configuration flow for integration solaredge: No module named 'solaredge'
2021-09-06 22:44:06 ERROR (MainThread) [homeassistant.config_entries] Error occurred loading configuration flow for integration samsungtv: No module named 'getmac'
2021-09-06 22:44:06 ERROR (MainThread) [homeassistant] Error doing job: Task exception was never retrieved
Traceback (most recent call last):
  File "/volume1/@appstore/homeassistant/env/lib/python3.8/site-packages/homeassistant/config_entries.py", line 696, in async_create_flow
    integration.get_platform("config_flow")
  File "/volume1/@appstore/homeassistant/env/lib/python3.8/site-packages/homeassistant/loader.py", line 498, in get_platform
    cache[full_name] = self._import_platform(platform_name)
  File "/volume1/@appstore/homeassistant/env/lib/python3.8/site-packages/homeassistant/loader.py", line 503, in _import_platform
    return importlib.import_module(f"{self.pkg_path}.{platform_name}")
  File "/var/packages/python38/target/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 961, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 843, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/volume1/@appstore/homeassistant/env/lib/python3.8/site-packages/homeassistant/components/samsungtv/__init__.py", line 5, in <module>
    import getmac
ModuleNotFoundError: No module named 'getmac'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/volume1/@appstore/homeassistant/env/lib/python3.8/site-packages/homeassistant/data_entry_flow.py", line 153, in async_init
    flow, result = await task
  File "/volume1/@appstore/homeassistant/env/lib/python3.8/site-packages/homeassistant/data_entry_flow.py", line 171, in _async_init
    flow = await self.async_create_flow(handler, context=context, data=data)
  File "/volume1/@appstore/homeassistant/env/lib/python3.8/site-packages/homeassistant/config_entries.py", line 703, in async_create_flow
    raise data_entry_flow.UnknownHandler
homeassistant.data_entry_flow.UnknownHandler

The Samsung integration in the log above must have triggered by HA because it detected the tv on the network. It was not initiated by me manually.

Any clues as to where to search for the root cause of all this?

[rs1gg]

@hgy59 any chance for a workaround similar to #4458 (comment) ?

I updated the workaround from #4458 to reflect the homeassistant core 2021.8.8 dependencies (from https://github.com/home-assistant/core/blob/2021.8.8/requirements_all.txt):

./pip install bellows==0.26.0 zha-quirks==0.0.59 zigpy-cc==0.5.2 zigpy-deconz==0.12.1 zigpy-xbee==0.13.0 zigpy-zigate==0.7.3 zigpy-znp==0.5.3 zigpy==0.36.1

However, this fails compiling the pycryptodome 3.11.0 dependency (downloads .tar.gz and tries to compile it).

[hgy59]

@rs1gg the zha dependencies should be installed on demand. Unfortunately bellows does not work with pycryptodomex (any more?). With the latest commit on #4919 I have included both the pycryptodome and pycryptodomex and successfully added the zha integration in HA on aarch64.
This will need another round of testing. When the github build action on #4919 is successfully terminated you can download and test this new release. Any feedback is welcome.

I am working on HA update to 2021.9.7 and will not update the 2021.8.8 package, as there are issues with cross compilation of python wheels that got recently fixed by @th0ma7.

[th0ma7]

@hgy59 updated python3 + python38 packages now online.

[rs1gg]

Thanks @hgy59 and @th0ma7! I can see the updated python 3.8 (v3.8.12-6) on https://synocommunity.com/packages, however my NAS does not offer to update it yet - still not after removing and adding the synocommunity repo. I will give it some hours and test as soon as the update is available.

[hgy59]

@rs1gg you don't need the updated python38, I am still testing with python38 3.8.11-4.
The latest commit in #4919 includes fixes for armv7 archs.
I have tested on DS-115j and zigbee home automation loads all its dependencies.

on DS-115j with only 256MB RAM it took about 1.5 hours (with 100% cpu load) to install the homeassistant package.

[hgy59]

ZHA should run with the latest update to 21.9.7 #4919

[qyz232]

ZHA working nicely indeed, many thx