Broadcast on ESP8266

[veryalien]

I cannot get broadcasting to work on esp8266.
Adding individual peer addresses works and I can send/receive to/from registered peers which are esp01s boards.
Trying to add the all 'ff' peer broadcast address gives an error.
Broadcasts to ff:ff... appear to be sent, but are apparently never received by any other board.
Sorry I don't have the hardware directly up and running to tell you the exact error I was getting.
Instead of hard-coding a list of peer addresses, I was trying to esp-now broadcast from each peer to try to dynamically build a list of peers for a mesh-like network.
Any ideas if broadcasting should just work on esp8266 out of the box, or is there something wrong here, or anything else I can try?
Issue raised here first, I can move this to the esp-now pull request if that's more appropriate.

[glenn20]

Thanks for the report. Sorry it took a while to respond. I'll look into it this arvo. The docs on the impacts of the various operation modes for the esp8266 are rather opaque so I may well have misunderstood something here. In fact the esp8266 espnow doc contradicts itself on whether broadcast is supported at all.

[moefear85]

From what I've read, esp8266 truly doesn't support espnow broadcasting, be it to a broadcast address (ff:ff:...) or to a bunch of peers (multicast). I was surprised when I read your images supported esp8266 broadcasting and was hoping I myself was wrong and was about to test it. But I see now there isn't much point.

[glenn20]

Thanks for the prompt - I forgot to close this one off. In my tests on my devices:

• ESP8266 can send messages to the broadcast address (b'\xff'*6) and they will be received by any active ESP32 devices.
• ESP8266 devices will NOT receive (or ack) any messages sent to a broadcast address (by an ESP32 or ESP8266 device).

A disappointing limitation - but at least we know that ESP8266 devices can announce themselves via broadcast to any available ESP32 listeners. I use this for the esp8266 devices I have in my home - but I must have at least one esp32 listener.

[moefear85]

Hi... I stumbled upon this: espressif/ESP8266_NONOS_SDK#8

So it seems, there was a bug in the ESP8266 SDK for a long time, but that it's been fixed for a long time now too, meaning (although not explicitly fully stated) the ESP8266 can send and recieve broadcasts, by any other ESP device. The catch is that it has to be built using NON-OS-SDK, and has to use a version greater than 3.0.0. IE it probably doesn't work with ESP8266_RTOS_SDK, and it is not clear which SDK type micropython for the ESP8266 was built with. Since your tests showed it can't recieve broadcasts, then most likely the ESP8266_RTOS_SDK is being used.

[glenn20]

Ooh. Excellent pickup.

It'd be interesting to see if we can build espnow images against that SDK.

[glenn20]

@moefear85 - Actually - on further reading of that issue, it seems that users are confirming that it enables broadcast send on esp8266 but not receive - which is the same as the behaviour I reported above.

EDIT: Actually - on re-reading some users are claiming success with send and receive - other are not. Wil lbe interesting to try it out.

[glenn20]

I haven't had any success building a micropython compatible sdk environment for the esp8266 based on v3+ of the non-os sdk so it seems we are stuck with this limtation for the meantime.

I will keep this issue open as placeholder to revisit this in the future if a micropython compatible/supported build env for the v3+ of the sdk becomes available.

[moefear85]

Hi, I've been (and still am) preoccupied with never ending todos. But I've become aware of a few things during this time. Hence:

Why use non-os? It is effectively abandoned by esp-idf (long ago), and probably doesn't support cmake. I think MP uses esp8266-rtos-sdk. It's API is almost identical to esp-idf, which makes everything easy.

More importantly, espressif provides 3 crucial functions, that together, fully supercede espnow.

esp_wifi_809211_tx(...)
esp_wifi_set_promiscuous(...) // and its related functions
esp_wifi_config_80211_tx_rate(...)

The 3rd is not available on esp8266, but on esp32, it allows setting the speed all the way up to 72Mbps. IE, together they circumvent both the 1Mbit speed limit of espnow, as well as the 255 byte packet size. Moreover, properly wrapped, they make it easy to send/recieve to/from any other esp8266/esp32 chip simply by specifying the mac address. There is no need to pre-register, nor any limit on the number of nodes that can simultaneously communicate together (compared to max 20 in espnow).

I'm very happy with it, and working out the kinks in my own esp-idf library. Thus it will be a while before I can update my github page to share what I've accomplished. But in the future I hope to port it to micropython and post it. I highly recommend focusing on it instead of espnow, or atleast take priority over that. Most people don't want espnow for its own sake, rather because they want to avoid the need to setup/configure a network and want instant communication, which this would provide, without any softAP/espnow node limits.

[glenn20]

@moefear85: Thanks for the post. This is quite interesting. Some time ago I was interested in how we might build our own fully open source protocol that offered the same advantages as espnow (and removed the limitations), but ran into some dead ends at the time. I'd be very interested in your work when you are ready to share.

Re. the data rate limit, I have run espnow at higher rates using esp_wifi_internal_set_fix_rate() and esp_wifi_config_espnow_rate(), however the other limitations still apply (including the need to register peers before send and the encryption protocol limits). The espnow protocol also provides some additional features, including automatic retransmission if no ACK is received. However, you have prompted me to add the rate feature to the public api as this would be useful to users :-).

Re. my esp8266 builds, I have used at various times the https://github.com/jepler/esp-open-sdk (used in the micropython CI builds), https://github.com/pfalcon/esp-open-sdk (on which the jepler sdk is based) and newer variants including https://github.com/thierer/esp-open-sdk (there are a myriad of unsupported forks of pfalcons original sdk). As far as I know they all use the non-os sdk (but I may be misunderstanding something). If MP is indeed using rtos sdk I'd happily shift to that. Any pointers welcome.

Hi, I've been (and still am) preoccupied with never ending todos. But I've become aware of a few things during this time. Hence:

Why use non-os? It is effectively abandoned by esp-idf (long ago), and probably doesn't support cmake. I think MP uses esp8266-rtos-sdk. It's API is almost identical to esp-idf, which makes everything easy.

More importantly, espressif provides 3 crucial functions, that together, fully supercede espnow.

esp_wifi_809211_tx(...)
esp_wifi_set_promiscuous(...) // and its related functions
esp_wifi_config_80211_tx_rate(...)

The 3rd is not available on esp8266, but on esp32, it allows setting the speed all the way up to 72Mbps. IE, together they circumvent both the 1Mbit speed limit of espnow, as well as the 255 byte packet size. Moreover, properly wrapped, they make it easy to send/recieve to/from any other esp8266/esp32 chip simply by specifying the mac address. There is no need to pre-register, nor any limit on the number of nodes that can simultaneously communicate together (compared to max 20 in espnow).

I'm very happy with it, and working out the kinks in my own esp-idf library. Thus it will be a while before I can update my github page to share what I've accomplished. But in the future I hope to port it to micropython and post it. I highly recommend focusing on it instead of espnow, or atleast take priority over that. Most people don't want espnow for its own sake, rather because they want to avoid the need to setup/configure a network and want instant communication, which this would provide, without any softAP/espnow node limits.

[moefear85]

The espnow protocol also provides some additional features, including automatic retransmission if no ACK is received.

I wasn't aware of this. I remember having to manually code ACK packets to inform the sender that the packet has arrived and been processed. Otherwise any attempt to run SLIP (ie run esptool and flash remotely over wireless), would fail so often it was unusable. Infact, I migrated to esp-mdf exactly for the sake of guarantees and larger packets, yet anything larger than 1500 would fail, silently or otherwise. That is what prompted me to migrate to the above mentioned functions.
However, esp_wifi_80211_tx also attempts to resend up to 10x waiting for an ACK (which it consumes silently and doesn't appear in the promiscuous callback), which turned out to be a problem because I needed to know for sure whether it was delivered, and it would cause a huge slowdown if the target was offline. These are now solved problems for me as I've written handlers & wrappers around those functions, that abstract away the manual fragmentation & sequencing/acking. it's basically a manual tcp library for esp_wifi_80211_tx. I will open source as soon as I can... but I'm genuinely drowning in todos, so it will take some time. It's still unstable, as I'm still learning esp-idf while using it.

if micropython uses esp-open-sdk, then it is very misleading with its continuous version increments, as these give the impression the latest sdks are being used, whereas the last commit on esp-open-sdk is dated June 10,2018 (5 years ago). Then it's even a security issue to use it.

The above mentioned functions are well documented. I was hoping you could beat me to implementing them for micropython, instead of espnow, to avoid duplication of efforts. There is practically no difference to exposing them in micropython relative to espnow functions, besides the name changes, which can be wrapped into nice names, even those of espnow for easy migration. Anything I push to github won't be a dependency to accomplish this, as I'm effectively building on top of esp_wifi_80211_tx (or equivalently the somewhat more restricted espnow).

[moefear85]

Ok, it seems micropython for esp8266 definately uses esp-open-sdk port page

In my view that's a no-go. Either somebody has to go through the effort of bumping it to esp8266-rtos-sdk, or honestly, it's time to deprecate esp8266 and tell people to move on. It's barely cheaper or smaller than esp32 these days, it no longer has the relevance it once did. I haven't used one in over a year now except to replace preexisting burned-out modules. But knowing how outdated it is, I wouldn't write a new project, not even in micropython, using it.

[glenn20]

Closing as it seems unlikely a fix will be availablle to fix this (ie. unlikely that micropython will move to a newer SDK which may (or may not) address this limitation).