Using Gstreamer with OpenMax in a Raspberry Pi

This minipost shows a subgroup of commands tested in a Raspberry Pi to evaluate the possibilities of a usage of this hardware as a domestic TV Headend.

  • From UDP/TS source with video MPEG2 to another UDP multicast group transcoding the video
    stream with H264:gst-launch-1.0 -v  udpsrc uri=udp://239.123.123.3:1234 ! tsdemux ! queue ! mpegvideoparse ! omxmpeg2videodec ! videoconvert ! omxh264enc ! video/x-h264,stream-format=byte-stream,profile=high ! h264parse ! mpegtsmux ! udpsink host=239.123.124.3 port=1234 auto-multicast=true

    The Gstreamer pipeline doesn’t break/end but there is a bug in the h264parse: it sends not regularly the needed SPS / PPS information with it (http://www.raspberrypi.org/forums/viewtopic.php?f=70&t=59412). Then, resulting stream is only playable if you get the stream from the beginning.

  • From UDP/TS source with video MPEG2 and MP2 to another UDP multicast group transcoding the video stream with H264 and video with AAC:
    gst-launch-1.0 -v udpsrc uri=udp://239.123.123.1:1234 ! queue ! tsdemux name=dem \
    dem. ! queue ! mpegvideoparse ! mpeg2dec ! videoconvert ! omxh264enc control-rate=1 target-bitrate=1000000 ! video/x-h264,stream-format=byte-stream,profile=high ! h264parse config-interval=2 ! queue ! muxer. \
    dem. ! queue ! mpegaudioparse ! mpg123audiodec ! audioconvert ! faac ! queue ! muxer. \
    flvmux name=muxer ! queue ! rtmpsink location="rtmp://rtmp.server:1935/rtmp/test2 live=test2"

    The Gstreamer pipeline breaks for some unkknown reason.

  • From UDP/TS source with video MPEG2 to a RTMP server transcoding to H264:
    gst-launch-1.0 -v udpsrc uri=udp://239.123.123.1:1234 ! queue ! tsdemux name=dem \
    dem. ! queue ! mpegvideoparse ! mpeg2dec ! videoconvert ! omxh264enc control-rate=1 target-bitrate=1000000 ! video/x-h264,stream-format=byte-stream,profile=high ! h264parse config-interval=2 ! queue ! muxer. \
    flvmux name=muxer ! queue ! rtmpsink location="rtmp://rtmp.server:1935/rtmp/test2 live=test2"

    Works fine and smooth. Source is a MPEG/TS SD channel.

  • From UDP/TS source with audio MP2 to a RTMP server transcoding audio channel to AAC:
    gst-launch-1.0 -v udpsrc uri=udp://239.123.123.1:1234 ! queue ! tsdemux name=dem \
    dem. ! queue ! mpegaudioparse ! mpg123audiodec ! audioconvert ! faac ! queue ! muxer. \
    flvmux name=muxer ! queue ! rtmpsink location="rtmp://rtmp.server:1935/rtmp/test2 live=test2"

    Works fine and smooth.

Tips about FFserver & FFmpeg

FFmpeg

Today, I want to share one tip about ffmpeg and ffserver multimedia video tools and  server. FFmpeg is a open source project that produces libraries and programs for handling multimedia data. FFserver is a HTTP and RTSP multimedia streaming server for live broadcasts. It can also time shift live broadcast.

All the settings used in this article have been tested on AMD64 Debian Squeeze OS using
FFmpeg Debian packages of the Debian-Multimedia repositories:

ffmpeg 5:0.6.1+svn20101128-0.2

You can get the Debian Multimedia repositories adding this lines to your APT sources.list
file:

deb http://www.debian-multimedia.org squeeze main
deb-src http://www.debian-multimedia.org squeeze main

Note that, with this same version, I’ve observe a problem trying to run ffserver:

Mon Apr 25 13:29:09 2011 Aspect ratio mismatch between encoder and muxer layer

To work ffserver in this version of ffmpeg is neccesary to hack the source code:

    1. Install DPKG development tools:  apt-get install dpkg-dev
    2. Get sources: apt-get source ffmpeg
    3. Go to sources directory: cd ffmpeg-dmo-0.6.1+svn20101128/
    4. Apply the patch:
      Index: libavutil/rational.h
      ===================================================================
      --- libavutil/rational.h    (revision 25549)
      +++ libavutil/rational.h    (working copy)
      @@ -29,7 +29,6 @@
      #define AVUTIL_RATIONAL_H
      
      #include <stdint.h>
      -#include <limits.h>
      #include "attributes.h"
      
      /**
      @@ -44,16 +43,13 @@
      * Compare two rationals.
      * @param a first rational
      * @param b second rational
      - * @return 0 if a==b, 1 if a>b, -1 if a<b, and INT_MIN if one of the
      - * values is of the form 0/0
      + * @return 0 if a==b, 1 if a>b and -1 if a<b
      */
      static inline int av_cmp_q(AVRational a, AVRational b){
      const int64_t tmp= a.num * (int64_t)b.den - b.num * (int64_t)a.den;
      
      if(tmp) return ((tmp ^ a.den ^ b.den)>>63)|1;
      -    else if(b.den && a.den) return 0;
      -    else if(a.num && b.num) return (a.num>>31) - (b.num>>31);
      -    else                    return INT_MIN;
      +    else    return 0;
      }
      
      /**
  • Install all the dependences neccesaries to build the package:
    apt-get install  debhelper libmp3lame-dev zlib1g-dev libvorbis-dev libsdl-dev libfaac-dev quilt texi2html libxvidcore4-dev liblzo2-dev libx264-dev  libtheora-dev libgsm1-dev ccache libbz2-dev libxvmc-dev libdc1394-22-dev libdirac-dev   libschroedinger-dev libspeex-dev yasm libopenjpeg-dev libopencore-amrwb-dev libvdpau-dev libopencore-amrnb-dev libxfixes-dev libasound-dev libva-dev libjack-dev libvpx-dev  librtmp-dev doxygen
  • Build the packages: dpkg-buildpackage -rfakeroot
  • Finally you’ll have the new *deb packages:
    # ls ../*.deb
    ffmpeg_0.6.1+svn20101128-0.2_amd64.deb         libavfilter-dev_0.6.1+svn20101128-0.2_amd64.deb
    ffmpeg-dbg_0.6.1+svn20101128-0.2_amd64.deb     libavformat52_0.6.1+svn20101128-0.2_amd64.deb
    ffmpeg-doc_0.6.1+svn20101128-0.2_all.deb     libavformat-dev_0.6.1+svn20101128-0.2_amd64.deb
    libavcodec52_0.6.1+svn20101128-0.2_amd64.deb     libavutil50_0.6.1+svn20101128-0.2_amd64.deb
    libavcodec-dev_0.6.1+svn20101128-0.2_amd64.deb     libavutil-dev_0.6.1+svn20101128-0.2_amd64.deb
    libavcore0_0.6.1+svn20101128-0.2_amd64.deb     libpostproc51_0.6.1+svn20101128-0.2_amd64.deb
    libavcore-dev_0.6.1+svn20101128-0.2_amd64.deb     libpostproc-dev_0.6.1+svn20101128-0.2_amd64.deb
    libavdevice52_0.6.1+svn20101128-0.2_amd64.deb     libswscale0_0.6.1+svn20101128-0.2_amd64.deb
    libavdevice-dev_0.6.1+svn20101128-0.2_amd64.deb  libswscale-dev_0.6.1+svn20101128-0.2_amd64.deb
    libavfilter1_0.6.1+svn20101128-0.2_amd64.deb

After to install the ffmpeg packages, you’ll can to run ffserver adjusted like you want. for this aim, you can run as follow: ffserver -f your_ffserver_settings.conf. The ffserver configuration file should have this structuration:

  • Main settings:
     # Port on which the server is listening. You must select a different
     # port from your standard HTTP web server if it is running on the same
     # computer.
     Port 8090
     # Address on which the server is bound. Only useful if you have
     # several network interfaces.
     BindAddress 0.0.0.0
     RTSPPort 554
     RTSPBindAddress 0.0.0.0
     # Number of simultaneous HTTP connections that can be handled. It has
     # to be defined *before* the MaxClients parameter, since it defines the
     # MaxClients maximum limit.
     MaxHTTPConnections 2000
     # Number of simultaneous requests that can be handled. Since FFServer
     # is very fast, it is more likely that you will want to leave this high
     # and use MaxBandwidth, below.
     MaxClients 1000
     # This the maximum amount of kbit/sec that you are prepared to
     # consume when streaming to clients.
     MaxBandwidth 1000
     # Access log file (uses standard Apache log file format)
     # '-' is the standard output.
     CustomLog -
     # Suppress that if you want to launch ffserver as a daemon.
     NoDaemon
  • Definition of the live feeds. Each live feed contains one video and/or audio sequence coming from an ffmpeg encoder or another ffserver. This sequence may be encoded simultaneously with several codecs at several resolutions. You must use ffmpeg to send a live feed to ffserver. In this example, you can type ffmpeg http://localhost:8090/feed1.ffm  or ffmpeg   -f alsa   -i hw:1   -f video4linux2 -r 25 -s 352x288  -i /dev/video0   http://localhost:8090/feed1.ffm:
     ################################################################################
     <Feed feed1.ffm>
     # ffserver can do time shifting. It means that it can stream any
     # previously recorded live stream. The request should contain:
     # "http://xxxx?date=[YYYY-MM-DDT][[HH:]MM:]SS[.m...]".You must specify
     # a path where the feed is stored on disk. You also specify the
     # maximum size of the feed, where zero means unlimited. Default:
     # File=/tmp/feed_name.ffm FileMaxSize=5M
     File /tmp/feed1.ffm
     FileMaxSize 100M
     # You could specify
     # ReadOnlyFile /saved/specialvideo.ffm
     # This marks the file as readonly and it will not be deleted or updated.
     # Specify launch in order to start ffmpeg automatically.
     # First ffmpeg must be defined with an appropriate path if needed,
     # after that options can follow, but avoid adding the http:// field
     # Launch ffmpeg
     # Only allow connections from localhost to the feed.
     ACL allow 127.0.0.1
     </Feed>
  • Setting a RTSP/RTP stream:
     ################################################################################
     # It's a lot of important the .sdp extension to allow RTP working well.
     #
     # Note that AVOptionVideo is only interesting for libx264 video codec:
     # For RTSP:
     # ffplay  rtsp://10.121.55.148:554/live.sdp
     #
     # For SDP (RTP):
     #   vlc  http://10.121.55.148:8090/live.sdp
     #
     <Stream live.sdp>
     Format rtp
     Feed feed1.ffm
     ### MulticastAddress 224.124.0.1
     ### MulticastPort 5000
     ### MulticastTTL 16
     # NoLoop
     VideoSize 352x288
     VideoFrameRate 15
     VideoBitRate 200
     # Alternative video codecs:
     # VideoCodec h263p
     # VideoCodec h263
     # VideoCodec libxvid
     # VideoQMin 10
     # VideoQMax 31
     VideoCodec libx264
     AVOptionVideo me_range 16
     AVOptionVideo i_qfactor .71
     AVOptionVideo qmin 30
     AVOptionVideo qmax 51
     AVOptionVideo qdiff 4
     # AVOptionVideo coder 0
     # AVOptionVideo flags +loop
     # AVOptionVideo cmp +chroma
     # AVOptionVideo partitions +parti8x8+parti4x4+partp8x8+partb8x8
     # AVOptionVideo me_method hex
     # AVOptionVideo subq 7
     # AVOptionVideo g 50
     # AVOptionVideo keyint_min 5
     # AVOptionVideo sc_threshold 0
     # AVOptionVideo b_strategy 1
     # AVOptionVideo qcomp 0.6
     # AVOptionVideo bf 3
     # AVOptionVideo refs 3
     # AVOptionVideo directpred 1
     # AVOptionVideo trellis 1
     # AVOptionVideo flags2 +mixed_refs+wpred+dct8x8+fastpskip
     # AVOptionVideo wpredp 2
     ## AVOptionVideo flags +global_header+loop
     # NoAudio
     AudioCodec libmp3lame
     AudioBitRate 32
     AudioChannels 1
     AudioSampleRate 24000
     ## AVOptionAudio flags +global_header
     </Stream>
  • Setting a FLV stream ouput:
     ################################################################################
     # FLV output - good for streaming
     <Stream test.flv>
     # the source feed
     Feed feed1.ffm
     # the output stream format - FLV = FLash Video
     Format flv
     VideoCodec flv
     # this must match the ffmpeg -r argument
     VideoFrameRate 15
     # generally leave this is a large number
     VideoBufferSize 80000
     # another quality tweak
     VideoBitRate 200
     # quality ranges - 1-31 (1 = best, 31 = worst)
     VideoQMin 1
     VideoQMax 5
     VideoSize 352x288
     # this sets how many seconds in past to start
     PreRoll 0
     # wecams don't have audio
     Noaudio
     </Stream>
  • Setting a ASF stream ouput:
     ################################################################################
     # ASF output - for windows media player
     <Stream test.asf>
     # the source feed
     Feed feed1.ffm
     # the output stream format - ASF
     Format asf
     VideoCodec msmpeg4
     # this must match the ffmpeg -r argument
     VideoFrameRate 15
     # transmit only intra frames (useful for low bitrates, but kills frame rate).
     # VideoIntraOnly
     # if non-intra only, an intra frame is transmitted every VideoGopSize
     # frames. Video synchronization can only begin at an intra frame.
     VideoGopSize 40
     # generally leave this is a large number
     VideoBufferSize 1000
     # another quality tweak
     VideoBitRate 200
     # quality ranges - 1-31 (1 = best, 31 = worst)
     VideoQMin 1
     VideoQMax 15
     VideoSize 352x288
     # this sets how many seconds in past to start
     PreRoll 0
     # generally, webcams don't have audio
     # Noaudio
     AudioCodec libmp3lame
     AudioBitRate 32
     AudioChannels 1
     AudioSampleRate 24000
     </Stream>
  • Other streams availables:
     # Multipart JPEG
     #<Stream test.mjpg>
     #Feed feed1.ffm
     #Format mpjpeg
     #VideoFrameRate 2
     #VideoIntraOnly
     #NoAudio
     #Strict -1
     #</Stream>
     # Single JPEG
     #<Stream test.jpg>
     #Feed feed1.ffm
     #Format jpeg
     #VideoFrameRate 2
     #VideoIntraOnly
     ##VideoSize 352x240
     #NoAudio
     #Strict -1
     #</Stream>
     # Flash
     #<Stream test.swf>
     #Feed feed1.ffm
     #Format swf
     #VideoFrameRate 2
     #VideoIntraOnly
     #NoAudio
     #</Stream>
     # MP3 audio
     #<Stream test.mp3>
     #Feed feed1.ffm
     #Format mp2
     #AudioCodec mp3
     #AudioBitRate 64
     #AudioChannels 1
     #AudioSampleRate 44100
     #NoVideo
     #</Stream>
     # Ogg Vorbis audio
     #<Stream test.ogg>
     #Feed feed1.ffm
     #Title "Stream title"
     #AudioBitRate 64
     #AudioChannels 2
     #AudioSampleRate 44100
     #NoVideo
     #</Stream>
     # Real with audio only at 32 kbits
     #<Stream test.ra>
     #Feed feed1.ffm
     #Format rm
     #AudioBitRate 32
     #NoVideo
     #NoAudio
     #</Stream>
     # Real with audio and video at 64 kbits
     #<Stream test.rm>
     #Feed feed1.ffm
     #Format rm
     #AudioBitRate 32
     #VideoBitRate 128
     #VideoFrameRate 25
     #VideoGopSize 25
     #NoAudio
     #</Stream>
  • Other special streams:
     # Server status
     <Stream stat.html>
     Format status
     # Only allow local people to get the status
     ACL allow localhost
     ACL allow 192.168.0.0 192.168.255.255
     #FaviconURL http://pond1.gladstonefamily.net:8080/favicon.ico
     </Stream>
    
     # Redirect index.html to the appropriate site
     <Redirect index.html>
     URL http://www.ffmpeg.org/
     </Redirect>

Extra references: