GStreamer can't open stream RTP

[Progressia]

Hello,

I have a problem opening an RTP stream in flutter-pi. I am using a RaspberryPI 3B+ (Buster).

My device is streaming a live camera to rtp://<host-ip>:PORT
I correctly installed GStreamer on the Raspberry Pi and used the command:

$ gst-launch-1.0 udp://<raspberryPi-IP>:PORT ! application/x-rtp, media=video, clock-rate=90000, encoding-name=H264, payload=96 ! rtph264depay ! avdec_h264 ! autovideosink

GStreamer respond

Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Redistribute latency...
Redistribute latency...
WARNING: from element /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstKMSSink:autovideosink0-actual-sink-kms: Pipeline construction is invalid, please add queues.
Additional debug info:
../libs/gst/base/gstbasesink.c(1258): gst_base_sink_query_latency (): /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstKMSSink:autovideosink0-actual-sink-kms:
Not enough buffering available for  the processing deadline of 0:00:00.015000000, add enough queues to buffer  0:00:00.015000000 additional data. Shortening processing latency to 0:00:00.000000000.
WARNING: from element /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstKMSSink:autovideosink0-actual-sink-kms: Pipeline construction is invalid, please add queues.
Additional debug info:
../libs/gst/base/gstbasesink.c(1258): gst_base_sink_query_latency (): /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstKMSSink:autovideosink0-actual-sink-kms:
Not enough buffering available for  the processing deadline of 0:00:00.015000000, add enough queues to buffer  0:00:00.015000000 additional data. Shortening processing latency to 0:00:00.000000000.

and that's work - camera view is showed with minimal latency but that form pipeline for URI in video-player in flutter-pi isn't correct.

So I try using as URI:
udp://<raspberryPi-IP>:PORT
or
rtp://<raspberryPi-IP>:PORT

Widget code

  @override
  void initState() {
    super.initState();
    _controller = VideoPlayerController.network("rtp://<raspberryPi-IP>:PORT");
    _controller.addListener(() {
      setState(() {});
    });
    _controller.setLooping(true);
    _controller.initialize().then((_) => setState(() {}));

    _controller.play();
  }

but I got error:
[gstreamer video_player] Could not fetch duration. (gst_element_query_duration)

and white screen and displays nothing else. Other elements of the application (like buttons) work. Only the video-player does not show a frame from the camera.

[ardera]

How do you launch the rtp server? Do you use gstreamer there too? Can you give me the command for that so I can try to reproduce this on my own pi?

Can you run the rtp client with debug messages:
GST_DEBUG=rtpsrc:5 gst-play-1.0 rtp://<ip>:5000 and send me the complete output?

Seems like the rtpsrc plugin (the one which does the URI handling) is in the gst-plugins-bad directory, so probably not that reliable. Maybe you can try a different protocol, like rtmp, rtsp, or something.

[Progressia]

RTSP Server I started from this example:
https://github.com/GStreamer/gst-rtsp-server/blob/1.14.5/examples/test-launch.c

But not work in video-player. Client have to read on rtsp://:PORT

If I run GST_DEBUG=rtpsrc:5 gst-play-1.0 rtp://:5000
I get debug in terminal like that:

..
0:00:06.086803123   679  0x176ce90 DEBUG                 rtpsrc gstrtpsrc.c:166:gst_rtp_src_rtpbin_request_pt_map_cb:<source> Could not determine caps based on pt and the encoding-name was not set.
0:00:06.086949789   679  0x176ce90 DEBUG                 rtpsrc gstrtpsrc.c:126:gst_rtp_src_rtpbin_request_pt_map_cb:<source> Requesting caps for session-id 0x0 and pt 96.
0:00:06.087013487   679  0x176ce90 DEBUG                 rtpsrc gstrtpsrc.c:166:gst_rtp_src_rtpbin_request_pt_map_cb:<source> Could not determine caps based on pt and the encoding-name was not set.
..

Ok, If I run that command

GST_DEBUG=rtpsrc:5 gst-play-1.0 "rtp://192.168.0.4:5000?encoding-name=H264&payload=96&rtph264depay&avdec_h264&autovideosink"

I got in terminal something like that and nothing happend

Press 'k' to see a list of keyboard shortcuts.
Now playing rtp://192.168.0.4:5000?encoding-name=H264&payload=96&rtph264depay&avdec_h264&autovideosink
0:00:00.211207187   633   0x961dc0 INFO                  rtpsrc gstrtpsrc.c:518:gst_rtp_src_rtpbin_pad_added_cb:<GstRtpSrc@0x96e080> Element <rtp_recv_rtpbin0> added pad <rtp_recv_rtpbin0:recv_rtp_sink_0>with caps application/x-rtp.
0:00:00.211730729   633   0x961dc0 INFO                  rtpsrc gstrtpsrc.c:518:gst_rtp_src_rtpbin_pad_added_cb:<GstRtpSrc@0x96e080> Element <rtp_recv_rtpbin0> added pad <rtp_recv_rtpbin0:recv_rtcp_sink_0>with caps application/x-rtcp.
0:00:00.212027447   633   0x961dc0 INFO                  rtpsrc gstrtpsrc.c:518:gst_rtp_src_rtpbin_pad_added_cb:<GstRtpSrc@0x96e080> Element <rtp_recv_rtpbin0> added pad <rtp_recv_rtpbin0:send_rtcp_src_0>with caps application/x-rtcp; application/x-srtcp.
0:00:00.212270624   633   0x961dc0 DEBUG                 rtpsrc gstrtpsrc.c:126:gst_rtp_src_rtpbin_request_pt_map_cb:<source> Requesting caps for session-id 0x0 and pt 96.
0:00:00.212340052   633   0x961dc0 DEBUG                 rtpsrc gstrtpsrc.c:160:gst_rtp_src_rtpbin_request_pt_map_cb:<source> Decided on caps application/x-rtp, encoding-name=(string)H264, clock-rate=(int)90000, media=(string)video
0:00:00.212640989   633   0x961dc0 DEBUG                 rtpsrc gstrtpsrc.c:771:gst_rtp_src_change_state:<source> Changing state: NULL => READY
0:00:00.214037291   633   0x961dc0 DEBUG                 rtpsrc gstrtpsrc.c:771:gst_rtp_src_change_state:<source> Changing state: READY => PAUSED
Pipeline is live.
0:00:00.215078697   633   0x961dc0 DEBUG                 rtpsrc gstrtpsrc.c:771:gst_rtp_src_change_state:<source> Changing state: PAUSED => PLAYING
Redistribute latency...
0:00:00.277633385   633   0x95c490 INFO                  rtpsrc gstrtpsrc.c:596:gst_rtp_src_rtpbin_on_new_ssrc_cb:<source> Detected a new SSRC: session-id 0x0, ssrc 0x43160991.
0:00:00.277785729   633   0x95c490 DEBUG                 rtpsrc gstrtpsrc.c:126:gst_rtp_src_rtpbin_request_pt_map_cb:<source> Requesting caps for session-id 0x0 and pt 96.
0:00:00.277891197   633   0x95c490 DEBUG                 rtpsrc gstrtpsrc.c:160:gst_rtp_src_rtpbin_request_pt_map_cb:<source> Decided on caps application/x-rtp, encoding-name=(string)H264, clock-rate=(int)90000, media=(string)video
0:00:00.478048854   633 0x75401b50 INFO                  rtpsrc gstrtpsrc.c:518:gst_rtp_src_rtpbin_pad_added_cb:<source> Element <rtp_recv_rtpbin0> added pad <rtp_recv_rtpbin0:recv_rtp_src_0_1125517713_96>with caps application/x-rtp, encoding-name=(string)H264, clock-rate=(int)90000, media=(string)video, payload=(int)96, ssrc=(uint)1125517713.
0:00:00.478322656   633 0x75401b50 DEBUG                 rtpsrc gstrtpsrc.c:554:gst_rtp_src_rtpbin_pad_added_cb:<source> Adding new pad: src_96
Redistribute latency...

[Progressia]

If You could run on Your RaspPi

Server (NVIDIA Jetson)
gst-launch-1.0 nvarguscamerasrc ! 'video/x-raw(memory:NVMM), format=NV12, width=800, height=480' ! nvvidconv flip-method=0 ! nvv4l2h264enc insert-sps-pps=true ! h264parse ! rtph264pay pt=96 ! udpsink host=192.168.0.4 port=5000

Client (RaspberryPi 3B+)
gst-launch-1.0 udpsrc port=5000 ! application/x-rtp,media=video,clock-rate=90000,encoding-name=H264,payload=96 ! rtph264depay ! avdec_h264 ! autovideosink sync=false

and that format server and client is correct for me.

[ardera]

I think I'll just add a way to launch the video player with a custom gstreamer pipeline. I mean, it definitely should be possible to launch the player with just some special rtp:// URL, don't think it should be that hard to set up. But if it's that hard to troubleshoot, maybe being able to do VideoPlayerController.fromGstreamerPipeline is more straightforward.

[Progressia]

It's great news. Last time, I try add/modify player but not successful. I think player should open gstreamer player like autovideosink.
If VideoPlayerController.fromGstreamerPipeline could be play stream using arguments like

udpsrc port=5000 ! application/x-rtp,media=video,clock-rate=90000,encoding-name=H264,payload=96 ! rtph264depay ! avdec_h264 ! autovideosink sync=false

It will be awsome.

[ardera]

@Progressia working on it right now

[ardera]

@Progressia Sorry for the delay, can you try the feature/gstreamer-player-from-pipeline flutter-pi branch with the flutterpi_gstreamer_video_player package? I could successfully view the camera preview using

FlutterpiVideoPlayerController.withGstreamerPipeline('libcamerasrc ! queue ! appsink name="sink")

there (the last element must always be an appsink with name sink), your pipeline should work too

[Progressia]

Hi @ardera . It's good news. I will check in a day.

[Progressia]

Hi @ardera.
Today I try open stream and I got error.

[gstreamer video_player] Couldn't find "src" element to configure additional HTTP headers.

Maybe I did something wrong. I set up the pipeline like this

String pipeline = 'udpsrc udp://192.168.0.4:5000 caps = "application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96" ! rtph264depay ! decodebin ! videoconvert ! libcamerasrc ! queue ! appsink name="sink" ';

If I run this command in the console, I can see the frames from the remote camera.

gst-launch-1.0 udp://192.168.0.4:5000 ! application/x-rtp, media=video, clock-rate=90000, encoding-name=H264, payload=96 ! rtph264depay ! avdec_h264 ! autovideosink

In this command I have autovideosink as last element because if I have appsink name="sink" I can't see anything from the camera. Maybe I don't have some plugin in gstreamer for appsink ?

The error appears to relate to the absence of the src element. Are you specifying something specific as src ?

[ardera]

Today I try open stream and I got error.

[gstreamer video_player] Couldn't find "src" element to configure additional HTTP headers.

That's actually not an error, more like a warning :) the video player will look for an element called src and try to configure any HTTP headers that were specified using VideoPlayerController.httpHeaders. If it doesn't find one that's not a problem. Maybe I should improve the log message so it doesn't look like an error.

Maybe I did something wrong. I set up the pipeline like this

String pipeline = 'udpsrc udp://192.168.0.4:5000 caps = "application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96" ! rtph264depay ! decodebin ! videoconvert ! libcamerasrc ! queue ! appsink name="sink" ';

Try this:

udpsrc udp://192.168.0.4:5000 caps = "application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96" ! rtph264depay ! decodebin ! queue ! appsink name="sink"

libcamerasrc is a source element (sources video from an attached camera), so you can't just put it somewhere in the middle, and you probably don't want it anyway since you want to playback from UDP. videoconvert is probably unncessary, but if it doesn't work, try putting it in again.

gst-launch-1.0 udp://192.168.0.4:5000 ! application/x-rtp, media=video, clock-rate=90000, encoding-name=H264, payload=96 ! rtph264depay ! avdec_h264 ! autovideosink

In this command I have autovideosink as last element because if I have appsink name="sink" I can't see anything from the camera. Maybe I don't have some plugin in gstreamer for appsink ?

appsink means a custom, application specified sink. Basically if you use gstreamer as a library instead of as a console application, this allows you to do completely custom video playback, and e.g. upload the video frames to a flutter OpenGL texture, like flutter-pi does. If you use it from console it'll just do nothing.

[Progressia]

Thanks for answer

I try
udpsrc udp://192.168.0.4:5000 caps = "application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96" ! rtph264depay ! decodebin ! queue ! appsink name="sink"

but got this same Warning

I went back to the beginning and use Your full example (https://pub.dev/packages/flutterpi_gstreamer_video_player/example) and I only modified part in CameraViewPage

_controller = FlutterpiVideoPlayerController.withGstreamerPipeline(
      'udpsrc udp://192.168.0.4:5000 caps = "application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96" ! rtph264depay ! decodebin ! queue ! appsink name="sink"',
    ); 

Now I got Warning and after that I got error:

[gstreamer video_player] Couldn't find "src" element to configure additional HTTP headers.
[gstreamer video_player] gstreamer error: code: 5, domain: gst-resource-error-quark, msg: Could not establish connection to server. (debug info: gstsouphttpsrc.c(1375): gst_soup_http_src_parse_status (): /GstPipeline:pipeline0/GstURIDecodeBin:src/GstSoupHTTPSrc:source:
Could not connect: Network is unreachable (4), URL: https://flutter.github.io/assets-for-api-docs/assets/videos/bee.mp4, Redirect to: (NULL))
[gstreamer video_player] gstreamer error: code: 1, domain: gst-stream-error-quark, msg: Internal data stream error. (debug info: gstbasesrc.c(3055): gst_base_src_loop (): /GstPipeline:pipeline0/GstURIDecodeBin:src/GstSoupHTTPSrc:source:
streaming stopped, reason error (-5))
[gstreamer video_player] gstreamer error: code: 4, domain: gst-stream-error-quark, msg: Stream doesn't contain enough data. (debug info: gsttypefindelement.c(991): gst_type_find_element_chain_do_typefinding (): /GstPipeline:pipeline0/GstURIDecodeBin:src/GstTypeFindElement:typefindelement0:
Can't typefind stream) 

Next I cut part of example with ExampleVideoPage and run just CameraView and I got just Warning

 [gstreamer video_player] Couldn't find "src" element to configure additional HTTP headers.

I received a warning every time I clicked the refresh icon.

[ardera]

fixed by #323