Feedback about the removal of `broadcast_enable` in 3.7.0+

[matesh]

Versions

• Python: 3.11
• OS: Yocto Linux
• Pymodbus: 3.7.x
• Modbus Hardware (if used):

Pymodbus Specific

• Client: tcp/rtu - sync

Description

The changes in the API introduced in 3.7.0 regarding broadcasting are in my opinion problematic. The below is some feedback of my experience with hundreds of different devices and complex systems using modbus TCP/RTU in the energy sector.

The change assumes that messages sent to slave==0 are intended to be broadcast, however, this is not always the case. I deal with several devices that act as a gateway into a system of multiple devices, which have slave==0 by default for their internal operational data, and all other unit IDs are devices addressed behind this gateway device. Previously, using the broadcast_enable flag upon instantiation of the client, it could be clearly defined how the client has to behave when slave==0:

• When disabled, it was handled as if it was a standard device on the bus, expecting a response to any requests.
• When enabled, requests were handled as a broadcast message, not expecting a response.

The behaviour is now solely determined by the slave ID which results in undesired behaviour in case when responses are indeed expected from slave==0. When sending a read request to slave==0, the client doesn't take timeout in consideration, not expecting a response and thus requests fail, even though these requests are legitimate.

The assumption made in the documentation, that "experience shows that modern devices do not allow broadcast, mostly because it is inheriently dangerous" is in my opinion incorrect. In industrial control, broadcasting is rather important. We need to write setpoints in the same registers of multiple identical devices on the same RS485 bus (or RS485 bus behind a TCP/RTU gateway), and writing registers individually into each device one by one takes too much time. Broadcasting is thus important in terms of timely delivery of register writes simultaneously into all devices on the same bus.

The assumption "With slave=0 the application can get upto 254 responses on a single request" is in my opinion incorrect: Devices (even modern ones) with modbus implemented properly, ignore read commands on slave==0, only receive write commands (if they support broadcasting at all, otherwise they ignore those messages too), and regardless whether they accepted broadcast write commands or not, they do not respond/confirm the failure or success of any write commands, not even by means of exceptions. I have not yet worked with a device that responded to write broadcast commands, and definitely not read commands on slave==0, unless of course as described above, when they had a slave==0 on purpose, in a well established case.

I believe, that because of the above reasons, the previous implementation, where the client behaviour for slave==0 could be set in the client for the specific use case, was a much more elegant and functional solution than what was introduced in 3.7.0 onwards, therefore I'd recommend either reinstating this option in the clients (more elegant) or handling slave==0 differently between read and write commands (less elegant). If the behaviour introduced in 3.7.0 is preferred to be the default, broadcast_enable could be set to True by default to maintain compatibility with 3.7.0 and above.

I won't be able to use 3.7.0 <= pymodbus on hundreds of systems because of this new behaviour. This is not a problem right now, but it might be in a few years time when I might have to move on to a more recent version due to python compatibility, etc. A small proportion of these gateways allow to be reconfigured to a different slave ID other than the default 0 for the internal data, but not all of them, and reconfiguring the ones that do allow, would require a site visit by a technician causing logistical difficulties. I'd also prefer not having to manage different versions of libraries on different installations based on what type of devices these systems need to interact with. I don't really see a reasonable/straightforward workaround to this without having to change code significantly, potentially causing backwards compatibility issues.

[janiversen]

First of all:

slave=0 is broadcast according to the modbus standard !

Secondly:

With v3.7.3 pymodbus should return the first response when sending a request with slave=0. Problem is that on a serial line, ALL slaves are deemed to respond to a request with slave=0.

So in total, I am not sure what the problem really is ??

We saw that there was really no need for the broadcast_enable= parameter, since it allowed silly combinations like broadcast_enable=True and slave=3.

[janiversen]

And as usual Pull requests are welcome, if you want a different behavior.

[janiversen]

I think you miss an important point, write commands also have responses !

And thus sending a write command with slave=0, means that all connected devices respond. Something you would need to handle by calling execute, and do whatever you want to do with the responses.

According to the standard, there are no requests without a response! However I too am aware that some devices do not respond.

Maybe the best option would be to add ignore_responses= to all request calls, that would solve both no response and multiple responses.

Looking forward to see a pull request.

[matesh]

I know the standard, and you are absolutely correct, as per chapter 2.1 of "MODBUS over serial line specification and implementation guide V1.02":

The address 0 is reserved to identify a broadcast exchange.

Unfortunately, there are a lot of developers (especially ones from over the big rock candy mountain) who think "outside of the box" and these different ideas need to be accommodated. Developers of modbus interfaces twist and turn the modbus standard so much, that the space here wouldn't be enough to list all my experience...

What I was trying to point out, was that the previous behaviour was sensible and usable. broadcast_enable was only affecting the behaviour when slave ID 0 was used on a case-by-case basis as a legit slave, to define whether response was expected to requests/writes on that address or not, and this option gave the ability to handle most cases out there.

Problem is that on a serial line, ALL slaves are deemed to respond to a request with slave=0
According to the standard, there are no requests without a response!

I think you might have misunderstood something. According to chapter 2.1 of the "MODBUS over serial line specification and implementation guide V1.02" and my personal experience working with modbust devices for decades:

No response is returned to broadcast requests sent by the master. The broadcast requests are necessarily writing commands. All devices must accept the broadcast for writing function.

Chapter 2.4.1:

When a broadcast request is sent on the serial bus, no response is returned from the slaves. Nevertheless a delay is respected by the Master in order to allow any slave to process the current request before sending a new one. This delay is called "Turnaround delay".

With v3.7.3 pymodbus should return the first response when sending a request with slave=0

Unfortunately, this doesn't seem to be the case. If unit ID 0 is used, the transaction returns immediately after sending the packet (https://github.com/pymodbus-dev/pymodbus/blob/dev/pymodbus/transaction.py#L308) without even attempting to receive. This implementation is as per the specification, and yes, we are dealing with an outlier here...

"We saw that there was really no need for the broadcast_enable= parameter, since it allowed silly combinations like broadcast_enable=True and slave=3"

Previously, the broadcast argument for _transact was determined by the combination of self.client.params.broadcast_enable and not request.slave_id now it is only based solely on not request.slave_id. So broadcast_enable was only affecting behaviour when slave was 0, not in any other case. It was basically an override for the client to indeed receive in case unit ID 0 was a legitimate device, not an intention to broadcast.

I'm happy to open a PR, though I don't know what solution you'd find more fitting to your library: reinstating the broadcast_enable argument for the client with a default of true, or add an argument to receive_slave_0 or similar that perhaps represents the intention better, either into the client or the read/write methods.

[janiversen]

broadcast_enable= is wrong because it allows the combination brodcast_enable=true and slave=3, which makes no sense.

My preference would be a parameter "no_response_expected=" with default false. I am open to suggestions. The parameter needs to be at call request level and not at client level.

I know the standard very well (I have been implementing clients and masters for 30+ years as well as working on the standard), and yes we all twist the standard. slave=0 is actually used in many modbus testers, to detect which slaves are connected.

[matesh]

Ok, I have to point out however, that pymodbus 3.7.0+ currently works as per the modbus specification I referred to above, and it is not even trying to receive after sending a broadcast message. Thus defaults to the behaviour opposing to what you are suggesting, no_response_expected=True. This is actually part of the problem I have.

I am not aware of what protocol modbus testers use to detect devices on the bus. In a previous implmentation of my software, I was going miles to try to find a way to detect slaves on a bus without sweeping unit IDs and probing requests, but I couldn't find a universally used protocol with broadcasting, certainly not one that was supported by any of the devices I had to work with. If you have any standard you can refer me to, that would be much appreciated.

[janiversen]

With no_response_expected=False (which is the default) you tell that you want a response, that allows you to expect a response when slave=0 and actually works as the old "broadcast_enable=False"

The current version will actually never wait for a response, the fix for that is only on dev, sorry for the confusion.

this parameter would also solve another issue, which is currently solved badly. Some Chinese devices, do not respond to specific requests (like write requests).

[janiversen]

FYI: v3.7.4 is due in 1-2 weeks and then there will probably be a bigger pause

Would be nice to have a solution to your problem in that release.