gstreamer應用筆記

gstreamer官網

https://gstreamer.freedesktop.org/

應用手冊

https://gstreamer.freedesktop.org/documentation/index.html

 

一、getreamer安裝(ubuntu)

gstreamer0.10和gstreamer1.0兩個版本容易混淆

sudo add-apt-repository ppa:mc3man/trusty-media

sudo apt-get update

sudo apt-get install build-essential dpkg-dev flex bison autotools-dev automake liborc-dev autopoint libtool gtk-doc-tools libgstreamer1.0-dev

sudo apt-get install libgstreamer0.10-0 libgstreamer0.10-dev gstreamer0.10-tools gstreamer0.10-plugins-base libgstreamer-plugins-base0.10-dev gstreamer0.10-plugins-good gstreamer0.10-plugins-ugly gstreamer0.10-plugins-bad gstreamer0.10-ffmpeg

sudo apt-get install libgstreamer0.10-dev gstreamer-tools gstreamer0.10-tools gstreamer0.10-doc

sudo apt-get install gstreamer0.10-plugins-base gstreamer0.10-plugins-good gstreamer0.10-plugins-ugly gstreamer0.10-plugins-bad gstreamer0.10-plugins-bad-multiverse

若有需要還可以再安裝如下gst插件:
gstreamer0.10-tools
gstreamer0.10-x
gstreamer0.10-plugins-base
gstreamer0.10-plugins-good
gstreamer0.10-plugins-ugly
gstreamer0.10-plugins-bad
gstreamer0.10-ffmpeg
gstreamer0.10-alsa
gstreamer0.10-schroedinger
gstreamer0.10-pulseaudio

有可能需要安裝的軟件：
sudo apt-get install bison
sudo apt-get install flex
sudo apt-get install zlib1g
mad解碼插件
apt-get install libmad0-dev
apt-get install gstreamer0.10-plugins-ugly

安裝音頻庫

sudo apt-get install gstreamer1.0-alsa

安裝ffmpeg多媒體庫

gst-ffmpeg
--enable-liba52       enable GPLed liba52 support [default=no]
--enable-liba52bin    open liba52.so.0 at runtime [default=no]
--enable-libamr-nb    enable libamr-nb floating point audio codec
--enable-libamr-wb    enable libamr-wb floating point audio codec
--enable-libfaac      enable FAAC support via libfaac [default=no]
--enable-libfaad      enable FAAD support via libfaad [default=no]
--enable-libfaadbin   open libfaad.so.0 at runtime [default=no]
--enable-libgsm       enable GSM support via libgsm [default=no]
--enable-libmp3lame   enable MP3 encoding via libmp3lame
--enable-libvorbis    enable Vorbis encoding via libvorbis,
                     native implementation exists [default=no]
--enable-libx264      enable H.264 encoding via x264 [default=no]
--enable-libxvid      enable Xvid encoding via xvidcore,
                      native MPEG-4/Xvid encoder exists [default=no]

 

插件太多了，幾百上千個

List of Elements and Plugins

https://gstreamer.freedesktop.org/documentation/plugins.html

 

一勞永逸

sudo apt-get install gstreamer-plugins-*

sudo apt-get install gstreamer-*

 

二、 千里之行，始於"hello,world"

Tutorials

Basic tutorials

Basic tutorial 1: Hello world!

Basic tutorial 2: GStreamer concepts

Basic tutorial 3: Dynamic pipelines

Basic tutorial 4: Time management

Basic tutorial 5: GUI toolkit integration

Basic tutorial 6: Media formats and Pad Capabilities

Basic tutorial 7: Multithreading and Pad Availability

Basic tutorial 8: Short-cutting the pipeline

Basic tutorial 9: Media information gathering

Basic tutorial 10: GStreamer tools

Basic tutorial 11: Debugging tools

Basic tutorial 12: Streaming

Basic tutorial 13: Playback speed

Basic tutorial 14: Handy elements

Basic tutorial 16: Platform-specific elements

 

除了這16個入門samples，后面的pipelines Command line tools 和 Plugin 插件開發也是多媒體類應用極好的教材

官網才是最應該多關注的地方

https://gstreamer.freedesktop.org/documentation/tutorials/basic/hello-world.html

wiki手冊也非常全面，幾乎所有應用方向都說明

http://wiki.oz9aec.net/index.php?title=Gstreamer_cheat_sheet

IBM社區gstreamer教程

https://www.ibm.com/developerworks/cn/linux/l-gstreamer/

 

 

三、 gstreamer 進階...

1、播放視頻文件

   以MP4格式為例，其它格式可以 通過gst-inspect-1.0 |  grep  查找對應的demux，decode，sink等插件，當然也可以使用auto開頭的插件，或者playbin會自動選擇播放，只是沒有自己指定那么靈活，方便調試和驗證一些功能。

    1）硬解(vaapi)播放MP4文件：

     gst-launch-1.0 filesrc location=FilePath/test.mp4 ! qtdemux ! vaapidecode ! vaapisink

    2） 軟解，只要將解碼器vaapidecode換成avdec_h264，播放器vaapisink換成 ximagesink即可

2、播放RTSP視頻流

    1） 硬解。

     gst-launch-1.0 rtspsrc location=rtsp://username:passwd@ipaddr:port  latency=0 ! rtph264depay  !  capsfilter caps="video/x-h264"  ! h264parse  ! vaapidecode  !  vaapipostproc  width=800 height=600  !  vaapisink sync=false

    2）軟解。

   gst-launch-1.0 rtspsrc location=rtsp://username:passwd@ipaddr:port  latency=0 ! rtph264depay ! capsfilter caps="video/x-h264" ! h264parse ! avdec_h264 ! videoconvert ! videoscale ! video/x-raw,width=800,height=600 ! ximagesink

3、 播放Udp視頻流

    Udp播放需要根據發送端數據源封裝格式來決定采用哪些Gstreamer插件，如果進行了RTP封裝，則需要先用rtph264depay進行解包，如果包含自定義幀頭的情況，應該編程對幀頭進行處理，不然會顯示異常，比如部分花屏現象，以下是對裸流進行播放。    

1）硬解

     gst-launch-1.0 udpsrc port=2101 ! h264parse ! vaapidecode ! vaapisink

2）軟解

     gst-launch-1.0  udpsrc port=2101 ! h264parse ! avdec_h264 !  autovideosink

 

參考https://blog.csdn.net/manjiao4651538/article/details/80227966

 

4、gstreamer rtsp推流/拉流

    1）gstreamer rtsp拉流播放

    https://blog.csdn.net/yang_quan_yang/article/details/78846134

    2）gstereamer rtsp推流

    https://blog.csdn.net/zhuwei622/article/details/80348916

    3) On the Raspberry:

 $ gst-launch-1.0 rtspsrc location=rtsp://192.168.2.112:8080/stream.sdp ! rtph264depay ! h264parse ! omxh264dec ! autovideosink

5、rtpbin Network/RTP

send

Encode and payload H263 video captured from a v4l2src. Encode and payload AMR audio generated from audiotestsrc. The video is sent to session 0 in rtpbin and the audio is sent to session 1. Video packets are sent on UDP port 5000 and audio packets on port 5002. The video RTCP packets for session 0 are sent on port 5001 and the audio RTCP packets for session 0 are sent on port 5003. RTCP packets for session 0 are received on port 5005 and RTCP for session 1 is received on port 5007. Since RTCP packets from the sender should be sent as soon as possible and do not participate in preroll, sync=false and async=false is configured on udpsink

gst-launch-1.0 rtpbin name=rtpbin \
        v4l2src ! videoconvert ! ffenc_h263 ! rtph263ppay ! rtpbin.send_rtp_sink_0 \
                  rtpbin.send_rtp_src_0 ! udpsink port=5000                            \
                  rtpbin.send_rtcp_src_0 ! udpsink port=5001 sync=false async=false    \
                  udpsrc port=5005 ! rtpbin.recv_rtcp_sink_0                           \
        audiotestsrc ! amrnbenc ! rtpamrpay ! rtpbin.send_rtp_sink_1                   \
                  rtpbin.send_rtp_src_1 ! udpsink port=5002                            \
                  rtpbin.send_rtcp_src_1 ! udpsink port=5003 sync=false async=false    \
                  udpsrc port=5007 ! rtpbin.recv_rtcp_sink_1

recv

Receive H263 on port 5000, send it through rtpbin in session 0, depayload, decode and display the video. Receive AMR on port 5002, send it through rtpbin in session 1, depayload, decode and play the audio. Receive server RTCP packets for session 0 on port 5001 and RTCP packets for session 1 on port 5003. These packets will be used for session management and synchronisation. Send RTCP reports for session 0 on port 5005 and RTCP reports for session 1 on port 5007.

gst-launch-1.0 -v rtpbin name=rtpbin                                          \
    udpsrc caps="application/x-rtp,media=(string)video,clock-rate=(int)90000,encoding-name=(string)H263-1998" \
            port=5000 ! rtpbin.recv_rtp_sink_0                                \
        rtpbin. ! rtph263pdepay ! ffdec_h263 ! xvimagesink                    \
     udpsrc port=5001 ! rtpbin.recv_rtcp_sink_0                               \
     rtpbin.send_rtcp_src_0 ! udpsink port=5005 sync=false async=false        \
    udpsrc caps="application/x-rtp,media=(string)audio,clock-rate=(int)8000,encoding-name=(string)AMR,encoding-params=(string)1,octet-align=(string)1" \
            port=5002 ! rtpbin.recv_rtp_sink_1                                \
        rtpbin. ! rtpamrdepay ! amrnbdec ! alsasink                           \
     udpsrc port=5003 ! rtpbin.recv_rtcp_sink_1                               \
     rtpbin.send_rtcp_src_1 ! udpsink port=5007 sync=false async=false

參考：

https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gst-plugins-good-plugins/html/gst-plugins-good-plugins-rtpbin.html

GStreamer RTP Streaming

https://community.nxp.com/docs/DOC-94646 

 

6、錄音

錄音:

gst-launch -e pulsesrc ! audioconvert ! lamemp3enc target=1 bitrate=64 cbr=true ! filesink location=audio.mp3

gst-launch -e pulsesrc device="alsa_input.pci-0000_02_02.0.analog-stereo" ! audioconvert ! \
   lamemp3enc target=1 bitrate=64 cbr=true ! filesink location=audio.mp3

 播放錄音：

gst-launch-1.0 filesrc location=audio.mp3 ! decodebin ! audioconvert ! audioresample ! autoaudiosink

還是上面的wiki

http://wiki.oz9aec.net/index.php?title=Gstreamer_cheat_sheet

 

7、錄視頻

命令：gst-launch-1.0 -e rtspsrc location=rtsp://admin:admin@192.168.1.2 ! rtph264depay ! "video/x-h264, stream-format=byte-stream" ! filesink location=test.264

說明：主要是用gst-lanuch工具連接相關插件將rtsp video stream 保存為.264文件，然后可以利用相關播放器（如：kmpplayer）進行播放，亦可以供live555MediaServer生成rtsp stream；（"video/x-h264, stream-format=byte-stream"這個caps一定要連接才行）
原文：https://blog.csdn.net/u010005508/article/details/52710302

8、視頻收發(監控，預覽)

send:

gst-launch v4l2src ! video/x-raw-yuv,width=128,height=96,format='(fourcc)'UYVY ! ffmpegcolorspace ! ffenc_h263 ! video/x-h263 ! rtph263ppay pt=96 ! udpsink host=127.0.0.1 port=5000 sync=false

recv:
gst-launch  udpsrc  port=5000 ! application/x-rtp, clock-rate=90000,payload=96 ! rtph263pdepay queue-delay=0 ! ffdec_h263 ! xvimagesink

 

以Freescale平台為例，實時碼流收發命令行如下：

Server側（發送方）：

gst-launch -v videotestsrc ! video/x-raw-yuv,width=640,height=480 ! vpuenc codec=avc ! rtph264pay pt=96 ! udpsink host=127.0.0.1 port=1234

Client側（接收方）：

gst-launch -vvv udpsrc port=1234 caps="application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264" ! rtph264depay ! vpudec ! mfw_isink

 

9、音頻收發(語音對講)

模擬聲音數據

1）send.sh

gst-launch-1.0 rtpbin name=rtpbin \
        audiotestsrc ! amrnbenc ! rtpamrpay ! rtpbin.send_rtp_sink_1                   \
                  rtpbin.send_rtp_src_1 ! udpsink port=5002                            \
                  rtpbin.send_rtcp_src_1 ! udpsink port=5003 sync=false async=false    \
                  udpsrc port=5007 ! rtpbin.recv_rtcp_sink_1

2)recv.sh

gst-launch-1.0 -v rtpbin name=rtpbin                                          \
    udpsrc caps="application/x-rtp,media=(string)audio,clock-rate=(int)8000,encoding-name=(string)AMR,encoding-params=(string)1,octet-align=(string)1" \
            port=5002 ! rtpbin.recv_rtp_sink_1                                \
        rtpbin. ! rtpamrdepay ! amrnbdec ! alsasink                           \
     udpsrc port=5003 ! rtpbin.recv_rtcp_sink_1                               \
     rtpbin.send_rtcp_src_1 ! udpsink port=5007 sync=false async=false

 

真實聲卡

1) send.sh

gst-launch-1.0 rtpbin name=rtpbin \
        pulsesrc ! amrnbenc ! rtpamrpay ! rtpbin.send_rtp_sink_1                   \
                  rtpbin.send_rtp_src_1 ! udpsink port=5002                            \
                  rtpbin.send_rtcp_src_1 ! udpsink port=5003 sync=false async=false    \
                  udpsrc port=5007 ! rtpbin.recv_rtcp_sink_1

2) recv.sh

gst-launch-1.0 -v rtpbin name=rtpbin                                          \
    udpsrc caps="application/x-rtp,media=(string)audio,clock-rate=(int)8000,encoding-name=(string)AMR,encoding-params=(string)1,octet-align=(string)1" \
            port=5002 ! rtpbin.recv_rtp_sink_1                                \
        rtpbin. ! rtpamrdepay ! amrnbdec ! alsasink                           \
     udpsrc port=5003 ! rtpbin.recv_rtcp_sink_1                               \
     rtpbin.send_rtcp_src_1 ! udpsink port=5007 sync=false async=false

模擬聲音和實際聲卡只有發送端采集程序不同，模擬采集是audiotestsrc，實際聲卡采集是pulsesrc

 

中國移動和對講amr實時語音解碼播放

gst-launch-1.0 udpsrc port=6000 caps="application/x-rtp, media=(string)audio, clock-rate=(int)8000, encoding-name=(string)AMR, payload=(int)106" ! rtpamrdepay ! decodebin name=decoder ! queue ! audioconvert ! autoaudiosink

 tcpdump -i eth0 -w dump.pcap

 

gstreamer中通過UDP(RTP)遠程播放MP3

send.sh

gst-launch-1.0 -v filesrc location = Hopy_Always.mp3 ! decodebin ! audioconvert ! rtpL16pay ! udpsink host=127.0.0.1 port=6000

recv.sh

gst-launch-1.0 udpsrc port=6000 caps='application/x-rtp, media=(string)audio, clock-rate=(int)44100, channels=(int)2' ! rtpjitterbuffer latency=400 ! rtpL16depay ! pulsesink

 

Gstreamer 測試udpsink udpsrc播放mp3文件

https://blog.csdn.net/zhujinghao_09/article/details/8513962

 

10、gstreamer 播放mp3源碼(播放器) ，入門開發極好的samples

https://blog.csdn.net/fireroll/article/details/49126827

https://www.cnblogs.com/274914765qq/p/5090299.html

 

11、Gstreamer的音視頻同步

https://blog.csdn.net/maeom/article/details/7729840

 

12、播放音頻

gst-launch-1.0 playbin uri=file:///home/dong/Hopy_Always.mp3
gst-launch-1.0 filesrc location=Hopy_Always.mp3 ! decodebin ! audioconvert ! audioresample ! autoaudiosink

 

13、Gstreamer視頻傳輸測試gst-launch

https://blog.csdn.net/meng_tianshi/article/details/80142005

 

14、How to listen to the pulseaudio RTP Stream and play

https://www.freedesktop.org/wiki/Software/PulseAudio/Documentation/User/Network/RTP/

 

15、Gstreamer cheat sheet —— Picture in Picture / Video Wall / Text Overlay / Time Overlay ... ... ..

http://wiki.oz9aec.net/index.php?title=Gstreamer_cheat_sheet

 

16、用樹莓派做 RTMP 流直播服務器，可推送至斗魚直播

http://shumeipai.nxez.com/2017/11/01/build-rtmp-stream-live-server-with-raspberry-pi.html

 

17、gstreamer學習筆記：通過gst-launch工具抓取播放的音頻數據並通過upd傳輸

gst-launch數據轉換(pcm,aac,ts), rtp收發

https://blog.csdn.net/u010312436/article/details/53335579

 

18、gstreamer實現攝像頭的遠程采集，udp傳輸，本地顯示和保存為AVI文件 發送端

send

https://blog.csdn.net/zhujinghao_09/article/details/8528802

recv

https://blog.csdn.net/zhujinghao_09/article/details/8528879

 

19、QtGStreamer dvr

https://blog.csdn.net/lg1259156776/article/details/53413877

https://blog.csdn.net/xueyeguiren8/article/details/54581536

 

20、基於Gstreamer的實時視頻流的分發

https://blog.csdn.net/sdjhs/article/details/51444934

 

21、gstreamer學習筆記：將音視頻合成MPEG2-TS流並打包通過rtp傳輸

https://blog.csdn.net/u010312436/article/details/53668083

 

22、gstreamer之RTSP Server一個進程提供多路不同視頻

https://blog.csdn.net/quantum7/article/details/82999132

 

23、GStreamer資料整理(包括攝像頭采集,視頻保存,遠程監控,流媒體RTP傳輸)

https://blog.csdn.net/wzwxiaozheng/article/details/6099397

 

24、使用GStreamer作v4l2攝像頭采集和輸出到YUV文件及屏幕的相關測試

https://blog.csdn.net/shallon_luo/article/details/5400708

 

25、Gstreamer中添加x265編解碼器

https://blog.csdn.net/songwater/article/details/34855883

 

26、Gstreamer One Liners

https://metalab.at/wiki/Gstreamer_One_Liners

 

ARM平台基於嵌入式Linux Gstreamer 使用

https://www.eefocus.com/toradex/blog/16-05/379143_e4fcb.html

 

常見gstreamer pipeline 命令—— TI 3730 dvsdk

https://blog.csdn.net/songwater/article/details/34800017

 

gstreamer中的好東西，appsink和appsrc

https://blog.csdn.net/jack0106/article/details/5909935

 

基於DM3730平台的gstreamer音視頻傳輸調試

https://blog.csdn.net/goalietech/article/details/24887955

 

gstreamer appsrc appsink應用

gstreamer向appsrc發送幀畫面的代碼

https://blog.csdn.net/quantum7/article/details/82226608

gstreamer向appsrc發送編碼數據的代碼

https://blog.csdn.net/quantum7/article/details/82250524

gstreamer學習筆記：分享幾個appsink和appsrc的example

https://blog.csdn.net/u010312436/article/details/53610599

 

Here are two basic send/receive  h264 video stream pipelines:

gst-launch-0.10 v4l2src ! ffmpegcolorspace ! videoscale ! video/x-raw-yuv,width=640,height=480 ! vpuenc ! h264parse ! rtph264pay ! udpsink host=localhost port=5555

gst-launch-0.10 udpsrc port=5555 ! application/x-rtp,encoding-name=H264,payload=96 ! rtph264depay ! h264parse ! ffdec_h264 ! videoconvert ! ximagesink

 

gstreamer使用進階

https://blog.csdn.net/jack0106/article/details/5592557

 

 

# 整理了這么多，梳理一下指令，組織一下模塊代碼，應付常規的多媒體應用綽綽有余了！