Just moozing

Before you can check your notes, you must make them…

gstreamer and MJPEG

with 4 comments

I had a lot of trouble getting gstreamer to accept the mjpeg stream and convert it to something HTML5 understands. In the following, I will try to describe how I got it working.

Accessing the IP camera

I had this running already as part of getting to the virtual camera device, it is described in this blog entry.

To show the mjpeg stream I used the following command

$ gst-launch souphttpsrc location=http://trendnetcam/video.cgi timeout=5 ! jpegdec ! autovideosink

The parts

  • souphttpsrc location=http://trendnetcam/video.cgi timeout=5: uses http to retrieve data. It has a 5 seconds timeout before it issues an error.
  • jpegdec: decodes jpeg images to YUV or other relevant format.
  • autovideosink: outputs the video to screen

This also worked with the v4l2sink, so I assumed that converting to ogg/theora would be trivial.

Encoding to something HTML5 may understand

The examples I found use theora (I lost the links, sorry)

$ gst-launch videotestsrc num-buffers=100 ! theoraenc ! oggmux ! filesink location=testmovie.ogg

The parts

  • videotestsrc num-buffers=100: This is the gstreamer test signal. It is good to know for debugging. The num-buffers parameter means that we terminate after 100 images. Again, good for debugging.
  • theoraenc: Converts YUV to a theora stream.
  • oggmux: This was in the example I found, but it seems to be related to muxing audio and video.
  • filesink location=testmovie.ogg: Output result to specified file. autovideosink does not support theora, otherwise I would have used that one.

Mjpeg to theora

Combining the above pipes.

$ gst-launch souphttpsrc location=http://trendnetcam/video.cgi timeout=5 num-buffers=100 ! jpegdec ! theoraenc ! oggmux ! filesink location=testmovie.ogg

Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
ERROR: from element /GstPipeline:pipeline0/GstSoupHTTPSrc:souphttpsrc0: Internal data flow error.
Additional debug info:
gstbasesrc.c(2582): gst_base_src_loop (): /GstPipeline:pipeline0/GstSoupHTTPSrc:souphttpsrc0:
streaming task paused, reason not-negotiated (-4)
ERROR: pipeline doesn't want to preroll.
Setting pipeline to NULL ...
Freeing pipeline ...
 

So I get an Internal data flow error. That is highly non-informative.

Sometimes I get

$ gst-launch souphttpsrc location=http://trendnetcam/video.cgi timeout=5 num-buffers=100 ! jpegdec ! theoraenc ! oggmux ! filesink location=testmovie.ogg
...
ERROR: from element /GstPipeline:pipeline0/GstSoupHTTPSrc:souphttpsrc0: Cannot connect to destination (trendnetcam)
...

Apparently, when playing with gstreamer and souphttpsrc I am able to crash the IP camera and make it unavailable. I am considering a firmware upgrade (or a newer camera).

I started googling, and apparently I am not the only one trying to use gstreamer with an IP camera.  Actually, there are several things wrong with the above command – both the mjpeg stream is flawed and the pipeline itself.

Making the MJPEG stream conform

Inspired by this forum post, I concluded that I was “lucky” to have jpegdec working directly without using multipartdemux. It takes the stream, detects the boundary between the parts and splits the stream into images. This works for some IP cameras.

$ gst-launch souphttpsrc location=http://trendnetcam/video.cgi timeout=5 num-buffers=100 ! multipartdemux ! jpegdec ! autovideosink
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
[and it waits forever here...]

To debug, I save some mjpeg frames from the camera.

$ gst-launch souphttpsrc location=http://trendnetcam/video.cgi timeout=5 num-buffers=100 ! filesink location=trendnet.mjpeg

Searching some more let me here, and to the conclusion that the stream that my IP camera is not multipart conform. It has some irregularities that multipartdemux cannot handle.

Mime/multipart is defined in RFC1341, and describes how boundaries should be handled. Two things are wrong with my cameras implementation of multipart.

Firstly, the initial boundary does not end with a new line.

$ head trendnet.mjpeg -n 3
--video boundary--Content-length: 58329
Content-type: image/jpeg

Secondly, all the boundaries are “end-of-multipart” boundaries, i.e. they are “–video boundary–” instead of “–video boundary”.

I don’t which problem breaks multipartdemux, but the solution was presented in the above forum entry.

$ (echo "--video boundary--"; curl -s http://trendnetcam/video.cgi;) | gst-launch fdsrc do-timestamp=true ! multipartdemux boundary="video boundary--" ! jpegdec ! autovideosink

The parts

  • (echo “–video boundary–“; curl -s http://trendnetcam/video.cgi;): Prepends a boundary line and fetches data from the camera and pipes it to stdout.
  • fdsrc do-timestamp=true: Receives data from stdin. A gstreamer time stamp is added also (this was in the example, so I kept it)
  • multipartdemux boundary=”video boundary–“: Demultiplex the multipart input and use the specified boundary. The trailing “–” is probably the difference between the boundary to use and the auto detected one.
  • jpegdec ! autovideosink: convert from jpeg to YUV and show on screen.

Mjpeg to theora (take 2)

Now that the mjpeg stream is correctly handled, we try again.

$ $ (echo "--video boundary--"; curl -s http://trendnetcam/video.cgi;) | gst-launch fdsrc do-timestamp=true ! multipartdemux boundary="video boundary--" ! jpegdec ! theoraenc ! oggmux !  filesink location=testvideo.ogg
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
ERROR: from element /GstPipeline:pipeline0/GstFdSrc:fdsrc0: Internal data flow error.
Additional debug info:
gstbasesrc.c(2582): gst_base_src_loop (): /GstPipeline:pipeline0/GstFdSrc:fdsrc0:
streaming task paused, reason not-negotiated (-4)
ERROR: pipeline doesn't want to preroll.
Setting pipeline to NULL ...
Freeing pipeline ...

Great, still Internal data flow error. Still non-informative.

Debugging the pipeline

gstreamer has some built-in debugging possibilities as described here. I use –gst-debug-level=2, but that might change if my pipeline get big.

$ (echo "--video boundary--"; curl -s http://trendnetcam/video.cgi;) | gst-launch --gst-debug-level=2 fdsrc do-timestamp=true ! multipartdemux boundary="video boundary--" ! jpegdec ! theoraenc ! oggmux !  filesink location=testvideo.ogg
0:00:00.100388113 12792  0x8233050 WARN               theoraenc gsttheoraenc.c:310:gst_theora_enc_class_init: Failed to determine settings for the speed-level property.
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
0:00:00.910899563 12792  0x837d800 WARN                 basesrc gstbasesrc.c:2582:gst_base_src_loop: error: Internal data flow error.
0:00:00.911039106 12792  0x837d800 WARN                 basesrc gstbasesrc.c:2582:gst_base_src_loop: error: streaming task paused, reason not-negotiated (-4)
ERROR: from element /GstPipeline:pipeline0/GstFdSrc:fdsrc0: Internal data flow error.
Additional debug info:
gstbasesrc.c(2582): gst_base_src_loop (): /GstPipeline:pipeline0/GstFdSrc:fdsrc0:
streaming task paused, reason not-negotiated (-4)
ERROR: pipeline doesn't want to preroll.
Setting pipeline to NULL ...
Freeing pipeline ...

Three warnings and two of them converted to errors. Odd, and looking into theoraencs speed-level property doesn’t tell me much. My guess is that it is frame rate related.

Adding a format specification into the stream

$ (echo "--video boundary--"; curl -s http://trendnetcam/video.cgi;) | gst-launch --gst-debug-level=2 --gst-debug=theora:2 fdsrc do-timestamp=true ! multipartdemux boundary="video boundary--" ! jpegdec ! video/x-raw-yuv,framerate=4/1 ! theoraenc ! oggmux !  filesink location=nn.ogg
0:00:00.091995704 13004  0x8e6f050 WARN               theoraenc gsttheoraenc.c:310:gst_theora_enc_class_init: Failed to determine settings for the speed-level property.
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
0:00:00.868368027 13004  0x8f9fe98 WARN           basetransform gstbasetransform.c:1211:gst_base_transform_setcaps: transform could not transform video/x-raw-yuv, format=(fourcc)I420, width=(int)640, height=(int)480, framerate=(fraction)0/1 in anything we support
0:00:00.868774364 13004  0x8f9fe98 WARN                 basesrc gstbasesrc.c:2582:gst_base_src_loop: error: Internal data flow error.
0:00:00.868894002 13004  0x8f9fe98 WARN                 basesrc gstbasesrc.c:2582:gst_base_src_loop: error: streaming task paused, reason not-negotiated (-4)
ERROR: from element /GstPipeline:pipeline0/GstFdSrc:fdsrc0: Internal data flow error.
Additional debug info:
gstbasesrc.c(2582): gst_base_src_loop (): /GstPipeline:pipeline0/GstFdSrc:fdsrc0:
streaming task paused, reason not-negotiated (-4)
ERROR: pipeline doesn't want to preroll.
Setting pipeline to NULL ...
Freeing pipeline ...

We get at new warning saying that the the capsfilter we just introduced cannot handle the input it is getting. I notice the “framerate=(fraction)0/1” part, and conclude that we need something to convert from a framerate of 0 fps to my requested of 4 fps.

$ (echo "--video boundary--"; curl -s http://trendnetcam/video.cgi;) | gst-launch --gst-debug-level=2 fdsrc do-timestamp=true ! multipartdemux boundary="video boundary--" ! jpegdec ! videorate ! video/x-raw-yuv,framerate=4/1 ! theoraenc ! oggmux !  filesink location=nn.ogg

The parts

  • (echo “–video boundary–“; curl -s http://trendnetcam/video.cgi;): get the mjpeg stream
  • fdsrc do-timestamp=true ! multipartdemux boundary=”video boundary–” ! jpegdec: Receive data, split into images and convert from jpeg
  • videorate: Handles the framerate conversion.
  • video/x-raw-yuv,framerate=4/1: Caps filter to specify the format to be send to theoraenc
  • theoraenc ! oggmux ! filesink location=nn.ogg: Convert to theora and save to file

And it works.

Success!

It took me a while get it all working and I suspect there are still some quirks, but now I can make theora movies that may be shown using HTML5 and video tags. More to come in a future blog post.

Advertisements

Written by moozing

December 28, 2011 at 09:00

Posted in Tech

Tagged with ,

4 Responses

Subscribe to comments with RSS.

  1. […] The command on line 18 is described in a previous blog. […]

  2. […] "jqmodal", embeddedHeight: "400", embeddedWidth: "425", themeCSS: "" }); . gstreamer and MJPEG « Just moozing . Mjpeg sourceforge net macos mpeg2enc intel « riedabergee […]

    Mjpeg | TagHall

    March 28, 2012 at 17:49

  3. […] is a recap of an old blog post of mine on gstreamer. It has a section on debugging (hint: […]

  4. Timberland Vente
    timberland shoes uk http://timberland6inch.blogspot.com

    timberland shoes uk

    November 2, 2016 at 15:36


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: