Capability to send PINGREQ manually

[tylerjroach]

Describe the feature

Customers are asking for a bit more control over when PINGREQ keep-alives are sent.

We understand that this is handled internally, but a public facing API that allowed triggering a PINGREQ request would improve the customer experience, especially on Android.

The Android OS has many power saving features (ex: Doze) that may hinder the devices ability to send timely PINGREQs.

Use Case

To prevent unexpected keep-alive timeouts (especially in implementations that retain connections when the screen is turned off and entering doze mode), the app developer can prevent timeouts through Alarm scheduling and manually sending the PINGREQ.

Proposed Solution

No response

Other Information

No response

Acknowledgements

• I may be able to implement this feature request
• This feature might incur a breaking change

SDK version used

Latest

Environment details (OS name and version, etc.)

Android (All)

[bretambrose]

We have an intake process for internal feature requests. I'll DM the details when I have a moment to dig them up.

We would need a lot more details on specific, common use cases with typical numbers. My first instinct is that any problem that would be truly solved by a manual ping API can also be solved without one.

[gurjit-stripe]

Being able to schedule pings or more correctly keep-alive checks is important for platforms where the default loop provided from the library is not guaranteed to run.

For example on Android the app can be placed on standby or the device can enter Doze mode where the thread responsible for keeping the connection open by periodically sending PINGREQ if there is no other activity within the keep-alive period is suspended.
https://developer.android.com/training/monitoring-device-state/doze-standby

The Paho library designs for this issue by exposing checkPing as a public method, and with the following javadoc. link

/*
 * (non-Javadoc) Check and send a ping if needed. <p>By default, client sends
 * PingReq to server to keep the connection to server. For some platforms which
 * cannot use this mechanism, such as Android, developer needs to handle the
 * ping request manually with this method. </p>
 *
 * @throws MqttException for other errors encountered while publishing the
 * message.
 */
/*

Finally there is also the problem of timing. The Paho library uses System.nanoTime() when calculating the time delta between when the next ping needs to be sent and when the last activity on the channel was. It only sends a PINGREQ if

(time - lastOutboundActivity >= keepAlive - delta)

But this calculation requires System.nanoTime() to be monotonically increasing. On Android System.nanoTime() is "guaranteed to be monotonic, and is suitable for interval timing when the interval does not span device sleep." doc

This means the clock is monotonic but not necessarily increasing. For this reason, using this provided method to keep the connection open does not work when in sleep/Doze mode. It would work if elapsedRealtimeNanos() was used. Which is "the time since the system was booted, and include deep sleep. This clock is guaranteed to be monotonic, and continues to tick even when the CPU is in power saving modes, so is the recommend basis for general purpose interval timing."

Paho has an Android service variant for their MQTT library which keeps the connection alive using Android's AlarmManager which can wake the device even in low power states. However, do to the timing issues described above, this is not guaranteed to work since System.nanoTime() is not guaranteed to continue to tick even when the CPU is in power saving modes.
https://github.com/eclipse-paho/paho.mqtt.android/blob/master/org.eclipse.paho.android.service/src/main/java/org/eclipse/paho/android/service/AlarmPingSender.java

[bretambrose]

An empty QoS 1 publish has roughly the same costs as a ping roundtrip. If a manually triggered ping keeps the connection alive, why wouldn't an empty QoS 1 publish do the same?

[gurjit-stripe]

From the MQTT Spec : https://docs.oasis-open.org/mqtt/mqtt/v5.0/os/mqtt-v5.0-os.html#_Toc3901045

The Keep Alive is a Two Byte Integer which is a time interval measured in seconds. It is the maximum time interval that is permitted to elapse between the point at which the Client finishes transmitting one MQTT Control Packet and the point it starts sending the next. It is the responsibility of the Client to ensure that the interval between MQTT Control Packets being sent does not exceed the Keep Alive value.

To be compliant with the spec, the client should be responsible for this operation. Also, as mentioned above, its not that we need to send a PING every keep alive period but rather we need to check if the connection is still healthy, use a PINGREQ if necessary to do so, and then preform the proper action. If the code to check the connection is not healthy is not running then it doesn't matter if we are sending a qos 1 message instead. It is not guaranteed that the check and publish times are aligned and therefore the publish frequency would need to be double that of the keep-alive frequency.
https://github.com/eclipse-paho/paho.mqtt.java/blob/master/org.eclipse.paho.mqttv5.client/src/main/java/org/eclipse/paho/mqttv5/client/internal/ClientState.java#L694

Finally, theres the matter of pricing. PINGREQ/PINGRESP are free using AWS IoT Core but there is a cost for regular messages

To maintain connectivity, devices can send keep-alive (Ping) messages at frequencies ranging from 20 minutes to every 30 seconds, and you do not incur any additional cost for these messages.

https://aws.amazon.com/iot-core/pricing/

[bretambrose]

Cost would certainly nix that approach, true.

A manual ping requires the app to have unthrottled CPU/network access for several seconds. Assuming the client is using the proper internal clock (ie. sleep aware), it ought to be sufficient to just trigger the alarm/maintenance window at a rate that is a calculated function of the keep alive interval (something less than half keep alive). Then you're guaranteed to have a full, unthrottled client service to run sometime between the keep alive timer and 1.5 * the keep alive timer. This would trigger a ping cycle automatically without an API invocation.

We are currently using CLOCK_MONOTONIC/CLOCK_MONOTONIC_RAW (https://github.com/awslabs/aws-c-common/blob/main/source/posix/clock.c#L12-L16) which doesn't factor sleep into account. I feel like switching to CLOCK_BOOTTIME for Android builds could address that, address the problem, and leave the API unchanged.

In fact, from a fundamental correctness standpoint I feel like we have to switch to CLOCK_BOOTTIME. There are many invariants/expectations in the code base that the high res clock is truly monotonic.

[uzairkhan-stripe]

@bretambrose So can you confirm whether you will actually explore making the changes you mentioned? If so, do you have any timeline?

[bretambrose]

There's a PR in progress for the timer change. One reviewer is out for a few weeks so it may not get merged and propagated up until March.

[sbSteveK]

The PR mentioned here: #608 (comment) has been merged and the changes have propagated into aws-iot-device-sdk-java-v2.

If available, the sleep-aware clock mentioned here: #608 (comment) should be used automatically.

[github-actions]

This issue is now closed. Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one.