isStarted on RemoteVideoTrack is intermittently false

[funayamateppei]

• I have verified that the issue occurs with the latest twilio-video.js release and is not marked as a known issue in the CHANGELOG.md.
• I reviewed the Common Issues and open GitHub issues and verified that this report represents a potentially new issue.
• I verified that the Quickstart application works in my environment.
• I am not sharing any Personally Identifiable Information (PII)
  or sensitive account information (API keys, credentials, etc.) when reporting this issue.

Code to reproduce the issue:

About one out of every twenty times the RemoteVideoTrack.isStarted is false and no video is displayed.
On the other occasions when it works, isStarted = true

When the bug reoccurs
When the RemoteParticipant publishes a video, the trackSubscribed event is detected, but the RemoteVideoTrack.isStarted returned at that time is false.
In such cases, the trackUnpublished event is immediately detected.
On the publishing side, unpublish is not being called.
→ no video is displayed because there is no video to attach.

Under what conditions does isStarted return false?
Additionally, why does the trackUnpublished event fire in such cases, immediately after the trackSubscribed event is detected?
(It has also been confirmed that the trackUnsubscribed event fires.)

    // Set when RemoteParticipant is joined
    // participant: RemoteParticipant
    participant.on("trackSubscribed", (t) => {
      console.log("🔥: received trackSubscribed event\n")
      console.dir(t, {depth: null})
    })

    participant.on("trackUnpublished", (t) => {
      console.log("💧: received trackUnpublished event\n")
      console.dir(t, {depth: null})
    })

The isStarted value of the track on the publishing side was true at the time of reproduction, and it was published without any issues.
I also subscribed to the trackPublicationFailed event, but it did not fire.

    console.log(track.isStarted)
    await localParticipant.publishTrack(track)

Log

The RemoteTrack output from trackSubscribed

{
    "sid": "MT8a92a894e7b116512de39af0bf1c4e50",
    "priority": null,
    "isSwitchedOff": false,
    "isEnabled": true,
    "processor": null,
    "dimensions": {
        "height": null,
        "width": null
    },
    "processedTrack": null,
    "mediaStreamTrack": {},
    "isStarted": false,
    "name": "acdfa356-f5b0-4c5d-b9df-52b2adfa9f1f",
    "kind": "video"
}

The RemoteTrackPublication output from trackUnpublished

{
    "track": null,
    "publishPriority": "standard",
    "kind": "video",
    "isTrackEnabled": true,
    "isSubscribed": false,
    "trackSid": "MT8a92a894e7b116512de39af0bf1c4e50",
    "trackName": "acdfa356-f5b0-4c5d-b9df-52b2adfa9f1f"
}

Expected behavior:

• RemoteVideoTrack.isStarted = true

Actual behavior:

• RemoteVideoTrack.isStarted = false

Software versions:

• Browser(s): chrome(mobile)
• Operating System: iPhone15, iOS18.1.1
• twilio-video.js: 2.27.0
• Third-party libraries (e.g., Angular, React, etc.): React

[funayamateppei]

This is the value when the problem does not occur.

The RemoteTrack output from trackSubscribed

{
    "sid": "MT42da65465848686dc9713084c4ff4784",
    "priority": null,
    "isSwitchedOff": false,
    "isEnabled": true,
    "processor": null,
    "dimensions": {
        "height": 640,
        "width": 480
    },
    "processedTrack": null,
    "mediaStreamTrack": {},
    "isStarted": true,
    "name": "169493c5-38a3-4ae6-81b0-2e5818c76fd1",
    "kind": "video"
}

[funayamateppei]

Additionally, why does the trackUnpublished event fire in such cases, immediately after the trackSubscribed event is detected?

I noticed some related logic on the Android SDK side that might be relevant to this issue.

Regarding the PREDICTED mode in TrackSwitchOffMode, the following is mentioned:
"In this mode, Tracks are pro-actively switched off when network congestion is predicted by the bandwidth estimation mechanism."

Could this have any impact on the issue?

https://twilio.github.io/twilio-video-android/docs/latest/com/twilio/video/TrackSwitchOffMode.html

[luisrivas]

Hi @funayamateppei. Twilo-video could be turning off the video track based on network conditions. For more details, see https://www.twilio.com/docs/video/tutorials/using-bandwidth-profile-api. This behavior generally improves the quality of your calls. However, as the guide mentions, you can adjust the behavior to your application's needs depending on the use cases.

Please let me know if that helped you. Here is also a resource about turning on the logs for troubleshooting https://www.twilio.com/docs/video/troubleshooting/javascript-logger.

[funayamateppei]

Hi @luisrivas,

Thank you for your response. I’ll check if the suggested approach resolves the issue and get back to you once I’ve confirmed it. (Try changing clientTrackSwitchOffControl and trackSwitchOffMode)
(I’d like to note that the communication environment was stable when this issue occurred.)

Regarding the network instability mentioned, could you clarify whether it refers to the network conditions on the publishing side or the receiving side?
For additional context, when multiple participants joined the room, none of them were able to receive the video stream.

I also have an additional question: Is there a listener to detect when a VideoTrack I’ve published is turned off? The reason I’m asking is that while the video appears to be streaming on my device, the recipient doesn’t receive the video, and nothing is displayed on their end.

If there’s a listener to detect this, I could detach my VideoTrack to prevent it from being shown. Additionally, I could inform users with a message like, “The network is unstable, and the video cannot be shared.”

Looking forward to your advice.

Best regards,

[luisrivas]

You are welcome!

could you clarify whether it refers to the network conditions on the publishing side or the receiving side?

Regarding the bandwidth profile api, when you use it, you configure how your available downlink will be distributed among your subscribed RemoteVideoTracks. So this would configure your experience as a receiver during the call.

Is there a listener to detect when a VideoTrack I’ve published is turned off? The reason I’m asking is that while the video appears to be streaming on my device, the recipient doesn’t receive the video, and nothing is displayed on their end.

That's a great question! Since the media server's estimation algorithm considers your downlink bandwidth, only RemoteVideoTracks have the switchedOff and switchedOn events and the switchOff and switchOn methods.

I see your point about providing feedback to users regarding their connections, so I recommend you look at the quality API, too: https://www.twilio.com/docs/video/using-network-quality-api. Also, here is a set of good practices that would be helpful to get the most out of the Twilio video services: https://www.twilio.com/docs/video/tutorials/developing-high-quality-video-applications.

[funayamateppei]

Thank you for your response, @luisrivas !

I tried the suggested approach and here’s what I tested:
I changed clientTrackSwitchOffControl to manual and also set trackSwitchOffMode to disabled.
However, the same issue persists.
The VideoTrack is successfully published.
I also subscribed to the trackPublicationFailed event, but it did not fire.

Additionally, I registered a listener to detect the trackSwitchedOff event for the RemoteVideoTrack, but no events were triggered.

Could this possibly be a bug? I would greatly appreciate it if you could kindly look into this.

await connect(token, {
            tracks: [new LocalDataTrack()],
            networkQuality: {local: 1, remote: 1},
            bandwidthProfile: {
              video: {
                trackSwitchOffMode: "disabled",
                clientTrackSwitchOffControl: "manual",
              },
            },
          }

    // participant: RemoteParticipant
    participant.on("trackSwitchedOff", (t) => {
      console.dir(t, {depth: null})
    })

[luisrivas]

I just assigned this issue to myself to investigate if that is a bug. Could you confirm if this is the environment where you are experiencing the problem:

Software versions:
Browser(s): chrome(mobile)
Operating System: iPhone15, iOS18.1.1
twilio-video.js: 2.27.0
Third-party libraries (e.g., Angular, React, etc.): React

Also, can you reproduce this with the react demo app?

[funayamateppei]

Thank you for looking into this issue.

Yes, the described environment matches where I am experiencing the problem.

Specifically:
The side sharing the video is using an iPhone 15 with iOS 18.1.1 and Chrome (mobile).
The side receiving the video is using a Mac and an iPhone.

Also, can you reproduce this with the react demo app?

I will test this with the React demo app and will follow up with you once I have the results.

It seems that deployment is required for confirmation. That would incur some costs, wouldn't it?

[funayamateppei]

I tried to run the react-demo-app locally, but I couldn't get it started. It seems the demo app is broken. Even after pulling the master branch, the result was the same. Therefore, I wasn't able to test it with the demo app.
https://github.com/twilio/twilio-video-app-react?tab=readme-ov-file#deeper-dive

node: v22.1.0
npm: 10.7.0

[luisrivas]

Hi @funayamateppei! Are you receiving a specific error while running the demo app locally? I just tested it using node v22.11.0 and npm 10.9.0, and it is working fine.

[funayamateppei]

@luisrivas

Sorry for the delayed response.

I was able to successfully launch the demo application. However, while the demo app allows toggling the camera on and off, it does not support publishing and unpublishing tracks.

The issue occurs when using the publishTrack method instead of enabling and disabling the LocalVideoTrack.

Additionally, I was unable to launch the demo application on a smartphone. The issue reproduces on an iPhone 15 when using the publishTrack method with LocalVideoTrack, so I am unable to verify this issue using the demo app.

The reason for using publishTrack and unpublishTrack is to reduce users' data usage.

[funayamateppei]

I would like to provide additional information.
As I mentioned before, when the issue occurs, there is a difference in the isStarted status of the RemoteTrack output from trackSubscribed compared to the normal case.
To address this, I waited for the isStarted flag to become true before passing the LocalVideoTrack to publishTrack, but the issue was not resolved.
I am using the started listener to check whether isStarted is true.

[funayamateppei]

Hi @luisrivas,

Just following up on this issue.

The bug is still occurring, and our customers are experiencing significant difficulties. Could you please provide an update on its status? Your prompt attention to this would be greatly appreciated.