bcm2835-codec bcm2835-codec: bcm2835_codec_start_streaming: Failed enabling i/p port, ret -3

[bugsyb]

Every time trying to run use v4l2h264enc via gstreamer, below kernel crash is visible:

GST_DEBUG=2 gst-launch-1.0 -vvv v4l2src device=/dev/video2 ! 'image/jpeg,framerate=30/1,width=1920,height=1080'! jpegdec ! videoconvert ! v4l2h264enc ! fakesink

5.10.63-v8+ #1459 SMP PREEMPT Wed Oct 6 16:42:49 BST 2021 aarch64 GNU/Linux

Mar 08 01:12:17 pihost kernel: bcm2835-codec bcm2835-codec: bcm2835_codec_start_streaming: Failed enabling i/p port, ret -3
Mar 08 01:12:17 pihost kernel: ------------[ cut here ]------------
Mar 08 01:12:17 pihost kernel: WARNING: CPU: 2 PID: 86308 at drivers/media/common/videobuf2/videobuf2-core.c:1548 vb2_start_streaming+0x10c/0x160 [videobuf2_common]
Mar 08 01:12:17 pihost kernel: Modules linked in: snd_usb_audio snd_hwdep snd_usbmidi_lib snd_rawmidi snd_seq_device uvcvideo veth xt_nat xt_tcpudp xt_conntrack nft_chain_nat xt_MASQUERADE nf_nat nf_conntrack_netlink nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 nft_counter xt_addrtype nft_compat nf_tables nfnetlink br_netfilter bridge rfcomm algif_hash algif_skcipher af_alg bnep hci_uart btbcm bluetooth ecdh_generic ecc 8021q imx219 garp stp llc overlay snd_soc_hdmi_codec vc4 brcmfmac cec brcmutil bcm2835_codec(C) v3d gpu_sched bcm2835_isp(C) drm_kms_helper cfg80211 v4l2_mem2mem bcm2835_v4l2(C) bcm2835_unicam rfkill videobuf2_dma_contig bcm2835_mmal_vchiq(C) v4l2_dv_timings videobuf2_vmalloc v4l2_fwnode raspberrypi_hwmon videobuf2_memops i2c_mux_pinctrl videobuf2_v4l2 i2c_brcmstb i2c_mux videobuf2_common snd_soc_core i2c_bcm2835 snd_compress videodev snd_bcm2835(C) snd_pcm_dmaengine mc vc_sm_cma(C) snd_pcm snd_timer snd nvmem_rmem syscopyarea sysfillrect sysimgblt fb_sys_fops rpivid_mem uio_pdrv_genirq uio
Mar 08 01:12:17 pihost kernel:  i2c_dev aes_neon_bs sha256_generic aes_neon_blk crypto_simd cryptd drm fuse drm_panel_orientation_quirks backlight ip_tables x_tables ipv6
Mar 08 01:12:17 pihost kernel: CPU: 2 PID: 86308 Comm: v4l2src0:src Tainted: G        WC        5.10.63-v8+ #1459
Mar 08 01:12:17 pihost kernel: Hardware name: Raspberry Pi 4 Model B Rev 1.4 (DT)
Mar 08 01:12:17 pihost kernel: pstate: 60000005 (nZCv daif -PAN -UAO -TCO BTYPE=--)
Mar 08 01:12:17 pihost kernel: pc : vb2_start_streaming+0x10c/0x160 [videobuf2_common]
Mar 08 01:12:17 pihost kernel: lr : vb2_start_streaming+0x74/0x160 [videobuf2_common]
Mar 08 01:12:17 pihost kernel: sp : ffffffc0134d3b40
Mar 08 01:12:17 pihost kernel: x29: ffffffc0134d3b40 x28: ffffffc008e19eb0 
Mar 08 01:12:17 pihost kernel: x27: ffffff813774cf00 x26: 0000000000000000 
Mar 08 01:12:17 pihost kernel: x25: ffffff81045c1e30 x24: 0000000040045612 
Mar 08 01:12:17 pihost kernel: x23: 0000000000000000 x22: ffffffc011288948 
Mar 08 01:12:17 pihost kernel: x21: ffffff815216ac70 x20: ffffff815216aa98 
Mar 08 01:12:17 pihost kernel: x19: 00000000fffffffd x18: 0000000000000001 
Mar 08 01:12:17 pihost kernel: x17: 0000000000000000 x16: 0000000000000000 
Mar 08 01:12:17 pihost kernel: x15: ffffffffffffffff x14: 6c696146203a676e 
Mar 08 01:12:17 pihost kernel: x13: 696d61657274735f x12: ffffffc0113154b8 
Mar 08 01:12:17 pihost kernel: x11: 0000000000000003 x10: ffffffc0112fd478 
Mar 08 01:12:17 pihost kernel: x9 : ffffffc0100e62b8 x8 : 0000000000017fe8 
Mar 08 01:12:17 pihost kernel: x7 : c0000000ffffefff x6 : ffffff81f6fc28d8 
Mar 08 01:12:17 pihost kernel: x5 : ffffff81f6fc28d8 x4 : ffffffc1e6009000 
Mar 08 01:12:17 pihost kernel: x3 : 0000000000000001 x2 : cb3f4eaff923b800 
Mar 08 01:12:17 pihost kernel: x1 : 0000000000000000 x0 : 0000000000000001 
Mar 08 01:12:17 pihost kernel: Call trace:
Mar 08 01:12:17 pihost kernel:  vb2_start_streaming+0x10c/0x160 [videobuf2_common]
Mar 08 01:12:17 pihost kernel:  vb2_core_streamon+0x98/0x198 [videobuf2_common]
Mar 08 01:12:17 pihost kernel:  vb2_streamon+0x24/0x78 [videobuf2_v4l2]
Mar 08 01:12:17 pihost kernel:  v4l2_m2m_streamon+0x34/0x88 [v4l2_mem2mem]
Mar 08 01:12:17 pihost kernel:  v4l2_m2m_ioctl_streamon+0x20/0x30 [v4l2_mem2mem]
Mar 08 01:12:17 pihost kernel:  v4l_streamon+0x2c/0x38 [videodev]
Mar 08 01:12:17 pihost kernel:  __video_do_ioctl+0x18c/0x3f0 [videodev]
Mar 08 01:12:17 pihost kernel:  video_usercopy+0x154/0x618 [videodev]
Mar 08 01:12:17 pihost kernel:  video_ioctl2+0x20/0x68 [videodev]
Mar 08 01:12:17 pihost kernel:  v4l2_ioctl+0x48/0x68 [videodev]
Mar 08 01:12:17 pihost kernel:  __arm64_sys_ioctl+0xb0/0xf0
Mar 08 01:12:17 pihost kernel:  el0_svc_common.constprop.3+0xc0/0x1f8
Mar 08 01:12:17 pihost kernel:  do_el0_svc+0x2c/0x98
Mar 08 01:12:17 pihost kernel:  el0_svc+0x20/0x30
Mar 08 01:12:17 pihost kernel:  el0_sync_handler+0x90/0xb8
Mar 08 01:12:17 pihost kernel:  el0_sync+0x180/0x1c0
Mar 08 01:12:17 pihost kernel: ---[ end trace e0b1acf3d280da16 ]---

But running this, works fine
PARAM=" --width=1920 --heigh=1080 --codec yuv420 --framerate 15" libcamera-vid -t 0 $PARAM -o - | gst-launch-1.0 -vvvv fdsrc fd=0 ! rawvideoparse use-sink-caps=false width=1920 height=1080 format=i420 ! v4l2convert ! v4l2h264enc extra-controls="controls,video_bitrate_mode=0,video_bitrate=1000000,repeat_sequence_header=1" ! 'video/x-h264,profile=high,level=(string)4.2' ! h264parse ! rtph264pay ! udpsink host=myhost port=5001

Could it be related to: #1673 ?

[6by9]

Totally unrelated to #1673 which is related to decode.
It is a duplicate of raspberrypi/linux#3974 (comment) - you haven't set the H264 level, and it'll default to level 1.0 (max 128x96 @ 30.9fps).

[bugsyb]

Thank you @6by9.

The test with videotestsrc worked:
gst-launch-1.0 videotestsrc ! v4l2h264enc ! 'video/x-h264,level=(string)3' ! fakesink

but am still struggling with getting v4l2src working (no crash anymore) and is most probably lack of my experience, but can't find.
Did check:
https://gitlab.freedesktop.org/gstreamer/gst-plugins-good/-/issues/849
And this workflow reported as working:
gst-launch-1.0 v4l2src device=/dev/video0 ! 'video/x-raw,framerate=30/1,format=UYVY' ! v4l2h264enc ! 'video/x-h264,level=(string)4' ! filesink location = test_video.h264

fails on me with:

ST_DEBUG=2 gst-launch-1.0 -vvvvv v4l2src device=/dev/video0 ! 'video/x-raw,framerate=30/1,format=UYVY' ! v4l2h264enc ! 'video/x-h264,level=(string)4' ! fakesink
Setting pipeline to PAUSED ...
0:00:00.049506042 688755   0x559dcc7640 WARN                    v4l2 gstv4l2object.c:4406:gst_v4l2_object_probe_caps:<v4l2h264enc0:src> Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Invalid argument
Pipeline is live and does not need PREROLL ...
0:00:00.050786122 688755   0x559dcc1920 WARN                 basesrc gstbasesrc.c:3127:gst_base_src_loop:<v4l2src0> error: Internal data stream error.
0:00:00.050833065 688755   0x559dcc1920 WARN                 basesrc gstbasesrc.c:3127:gst_base_src_loop:<v4l2src0> error: streaming stopped, reason not-negotiated (-4)
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
Additional debug info:
../libs/gst/base/gstbasesrc.c(3127): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)
ERROR: pipeline doesn't want to preroll.
Setting pipeline to NULL ...
Freeing pipeline ...

What am I doing wrong? One or two working example with v4l2src and v4l2h264enc would be of great use - if you could please kindly put together.

Thank you in advance.

[6by9]

What is your V4L2 source device?
What formats does it support? Output of v4l2-ctl --list-formats-ext and v4l2-ctl -V please.

I suspect you're now hitting the GStreamer v4l2*enc plugins advertising a limited set of colorimetry caps, as is being discussed in #1673.
My USB webcam advertises what GStreamer views as colorimetry=(string)2:4:16:1, which isn't a standard.

gst-launch-1.0 -vvvvv v4l2src device=/dev/video0 ! 'video/x-raw,width=640,height=480,framerate=5/1,format=YUY2' ! capssetter caps="video/x-h264, colorimetry=bt709" ! v4l2h264enc ! 'video/x-h264,level=(string)4' ! fakesink should work, but my webcam seems to be upsetting the USB bus and triggering a reset every time it starts up (reset high-speed USB device number 3 using xhci_hcd in the kernel logs).

[bugsyb]

Thank you. I did loads of reading (many nights as it is DIY project for me) and got lost with all these parameters, dependencies, etc. and it looks like some real issues.
Thank you for your help.

Below are details, one is Pi cam2 and the other is usb cam (Microdia Webcam Vitade AF) marketed as "Wide-Angle 1080p UVC-Compliant USB Camera Module with Metal Case - Arducam" sold on The Pi Hut.
Pi cam:

v4l2-ctl  -d /dev/video0  --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
	Type: Video Capture

	[0]: 'pBAA' (10-bit Bayer BGBG/GRGR Packed)
		Size: Discrete 3280x2464
		Size: Discrete 1920x1080
		Size: Discrete 1640x1232
		Size: Discrete 640x480
	[1]: 'BG10' (10-bit Bayer BGBG/GRGR)
		Size: Discrete 3280x2464
		Size: Discrete 1920x1080
		Size: Discrete 1640x1232
		Size: Discrete 640x480
	[2]: 'BA81' (8-bit Bayer BGBG/GRGR)
		Size: Discrete 3280x2464
		Size: Discrete 1920x1080
		Size: Discrete 1640x1232
		Size: Discrete 640x480

v4l2-ctl -d /dev/video0 -V
Format Video Capture:
	Width/Height      : 1920/1080
	Pixel Format      : 'pBAA' (10-bit Bayer BGBG/GRGR Packed)
	Field             : None
	Bytes per Line    : 2400
	Size Image        : 2592000
	Colorspace        : sRGB
	Transfer Function : sRGB
	YCbCr/HSV Encoding: ITU-R 601
	Quantization      : Full Range
	Flags             : 

USB cam:

v4l2-ctl -d /dev/video0 -V
Format Video Capture:
	Width/Height      : 1920/1080
	Pixel Format      : 'pBAA' (10-bit Bayer BGBG/GRGR Packed)
	Field             : None
	Bytes per Line    : 2400
	Size Image        : 2592000
	Colorspace        : sRGB
	Transfer Function : sRGB
	YCbCr/HSV Encoding: ITU-R 601
	Quantization      : Full Range
	Flags             : 

v4l2-ctl -d /dev/video2 --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
	Type: Video Capture

	[0]: 'MJPG' (Motion-JPEG, compressed)
		Size: Discrete 1920x1080
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 1280x720
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 640x480
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 320x240
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 1920x1080
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.033s (30.000 fps)
	[1]: 'YUYV' (YUYV 4:2:2)
		Size: Discrete 640x480
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 320x240
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 640x480
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.033s (30.000 fps)

[6by9]

The Pi camera configured for libcamera needs libcamera to run in order to handle exposure, gain, white balance, lens shading, demosaicing, denoising, etc. The video encoder does not support Bayer formats. You need to be looking at libcamerasrc instead of v4l2src.

UVC (USB Video Class) should work fine, though may well need the capssetter override on the colorimetry.
As I said my webcam seems to throw a reset each time it starts streaming, but that looks to be my camera as it fails on Pi4, CM4, and my x86_64 laptop (ie 3 different USB host controllers).

[bugsyb]

Thank you @6by9.

Managed to get pipeline which didn't error out, but v4l2h264 doesn't seem to respect bit rate settings. At least I can't manage to get that done.
Used video_bitrate_mode=0 and video_bitrate=25000, where _video_bitrate_mode has been tested with 0 & 1 and with additional settings of video_bitrate. Regardless of settings am getting ~5Mb/s even with completely black content on the screen.
Seems like the v4l2h264enc is not that good. probably due to limitations of bmc. vp8 (cpu based sends same content at ~100kb/s). It has some hiccups due to cpu, but is of a huge difference.

Is there a chance to have a control over bitrate/compression flags/levels?

To repeat some success and for others to get a working pipeline (below what worked for me):

gst-launch-1.0 -vvv v4l2src device=/dev/video2 ! 'image/jpeg,framerate=30/1,width=1920,height=1080'! v4l2jpegdec ! videoconvert ! v4l2h264enc !  'video/x-h264,level=(string)4,c' ! h264parse ! rtph264pay ! udpsink

[6by9]

For a repeatable test:

gst-launch-1.0 -vvv videotestsrc  num-buffers=300 ! video/x-raw,framerate=30/1,width=1280,height=720,format=I420 ! capssetter caps="video/x-h264, colorimetry=bt709" ! v4l2h264enc extra-controls="controls,video_bitrate=100000,video_bitrate_mode=1" !  'video/x-h264,level=(string)4' ! filesink location=100k.h264
gst-launch-1.0 -vvv videotestsrc  num-buffers=300 ! video/x-raw,framerate=30/1,width=1280,height=720,format=I420 ! capssetter caps="video/x-h264, colorimetry=bt709" ! v4l2h264enc extra-controls="controls,video_bitrate=1000000,video_bitrate_mode=1" !  'video/x-h264,level=(string)4' ! filesink location=1M.h264
gst-launch-1.0 -vvv videotestsrc  num-buffers=300 ! video/x-raw,framerate=30/1,width=1280,height=720,format=I420 ! capssetter caps="video/x-h264, colorimetry=bt709" ! v4l2h264enc extra-controls="controls,video_bitrate=10000000,video_bitrate_mode=1" !  'video/x-h264,level=(string)4' ! filesink location=10M.h264

ls -l *.h264
-rw-r--r--  1 pi pi   498464 Mar  9 14:34 100k.h264
-rw-r--r--  1 pi pi  1251538 Mar  9 14:34 1M.h264
-rw-r--r--  1 pi pi  9303021 Mar  9 14:35 10M.h264

Repeat with video_bitrate_mode=0 and I get

-rw-r--r--  1 pi pi   314284 Mar  9 14:36 100k.h264
-rw-r--r--  1 pi pi  1251024 Mar  9 14:36 1M.h264
-rw-r--r--  1 pi pi 12500280 Mar  9 14:36 10M.h264

1MBit/s is 125kbytes/sec, so recording for 300 frames at 30fps is 10 seconds.
Getting an output file pretty much bang on 1250kB is pretty reasonable rate control in my book.

[6by9]

Minor caveat: those were done against Gstreamer 1.21 main branch instead of the stock Raspberry Pi OS 1.18. I don't believe anything significant has changed with v4l2h264enc between those two points.

[talksik]

Thanks for the hints everyone. For any newcomers, I was able to solve this with the following guide that I put together myself after days of experimentation.

• headless Raspbian 32 bit
• using gstreamer from apt

The v4l2src element shoots an error about not enough memory allocated and it errors out at that first element of the pipeline.

I was able to solve it with this:

$ v4l2-ctl --list-formats
ioctl: VIDIOC_ENUM_FMT
        Type: Video Capture

        [0]: 'YU12' (Planar YUV 4:2:0)
        [1]: 'YUYV' (YUYV 4:2:2)
        [2]: 'RGB3' (24-bit RGB 8-8-8)
        [3]: 'JPEG' (JFIF JPEG, compressed)
        [4]: 'H264' (H.264, compressed)
        [5]: 'MJPG' (Motion-JPEG, compressed)
        [6]: 'YVYU' (YVYU 4:2:2)
        [7]: 'VYUY' (VYUY 4:2:2)
        [8]: 'UYVY' (UYVY 4:2:2)
        [9]: 'NV12' (Y/UV 4:2:0)
        [10]: 'BGR3' (24-bit BGR 8-8-8)
        [11]: 'YV12' (Planar YVU 4:2:0)
        [12]: 'NV21' (Y/VU 4:2:0)
        [13]: 'RX24' (32-bit XBGR 8-8-8-8)

$ v4l2-ctl --set-fmt-video=width=640,height=480,pixelformat=9
$ v4l2-ctl -V
Format Video Capture:
        Width/Height      : 640/480
        Pixel Format      : 'NV12' (Y/UV 4:2:0)
        Field             : None
        Bytes per Line    : 640
        Size Image        : 460800
        Colorspace        : SMPTE 170M
        Transfer Function : Default (maps to Rec. 709)
        YCbCr/HSV Encoding: Default (maps to ITU-R 601)
        Quantization      : Default (maps to Limited Range)
        Flags             :

And now this works:

$ gst-launch-1.0 -e v4l2src ! video/x-raw,width=640,height=480,framerate=30/1 ! videoconvert ! fakesink

Notice how the capabilities match those which we set with v4l2-ctl.

Encoding

This works for test video:

$ gst-launch-1.0 videotestsrc ! v4l2h264enc ! 'video/x-h264,level=(string)3' ! fakesink

Encoding from camera:

$ gst-launch-1.0 -e v4l2src ! video/x-raw,width=640,height=480,framerate=30/1,format=NV12  ! v4l2h264enc ! 'video/x-h264,level=(string)4' ! fakesink
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Redistribute latency...
0:00:10.5 / 99:99:99.

Make sure your v4l2 controls are set properly.

The format much be something that gst-inspect-1.0 v4l2src has, AS WELL AS something that gst-inspect-1.0 v4l2h264enc accepts. In my system, I chose to go with NV12.

$ v4l2-ctl -V
Format Video Capture:
        Width/Height      : 640/480
        Pixel Format      : 'NV12' (Y/UV 4:2:0)
        Field             : None
        Bytes per Line    : 640
        Size Image        : 460800
        Colorspace        : SMPTE 170M
        Transfer Function : Default (maps to Rec. 709)
        YCbCr/HSV Encoding: Default (maps to ITU-R 601)
        Quantization      : Default (maps to Limited Range)
        Flags             :