從RTSP協議的直播到 HTTP TS(APPLE的Live streaming方案)轉換工作終於取得技術勝利,中間的過程有些經驗值得留存.
基本代碼完成后,存在幾個較為嚴重的問題. 首先是轉換之后的文件,VLC播放不了;其二是Mplayer能夠播放視頻,但是沒有音頻.
經過分析,因為TS流中的載體是H264的視頻和AAC的音頻;對這種情況TS流的描述,只是依照ISO-13818-1的2000版的顯然不夠,根據資料顯示,只有2007版的13818才能滿足需要. 因為2007版的13818涉及到版權和費用,獲取到它也費了不少時間和周折.不過新標准最后終於拿到了.
就是這一部分:
2.14 Carriage of ITU-T Rec. H.264 | ISO/IEC 14496-10 video
2.14.1 Introduction
This Specification defines the carriage of ITU-T Rec. H.264 | ISO/IEC 14496-10 elementary stream within ITU-T Rec. H.222.0 | ISO/IEC 13818-1 systems, both for program and transport streams. Typically, an ITU-T Rec. H.264 | ISO/IEC 14496-10 stream will be an element of an ITU-T Rec. H.222.0 | ISO/IEC 13818-1 program, as defined by the PMT in a Transport Stream and the PSM in a Program Stream. The carriage and buffer management of AVC video streams is defined using existing parameters from this Recommendation | International Standard such as PTS and DTS,as well as information present within an AVC video stream. Carriage of AVC video streams in an ITU-T Rec. H.222.0 | ISO/IEC 13818-1 stream defines accurate mapping between STD parameters and HRD parameters that may be present in an AVC video stream. Requirements are defined for the presence of HRD parameters in the AVC video stream, to ensure that it can be verified whether each STD requirement is met for each AVC video stream carried in a transport stream or a program stream. NOTE 1 – Though the timing information present in the AVC video stream may not use a 90-kHz clock, the PTS and DTS timestamps need to be expressed in units of 90 kHz. When an ITU-T Rec. H.264 | ISO/IEC 14496-10 stream is carried in an ITU-T Rec. H.222.0 | ISO/IEC 13818-1 stream, the ITU-T Rec. H.264 | ISO/IEC 14496-10 coded data shall be contained in PES packets. The ITU-T Rec. H.264 | ISO/IEC 14496-10 coded data shall comply with the byte stream format defined in Annex B of ITU-T Rec. H.264 | ISO/IEC 14496-10, with the following constraints:
• Each AVC access unit shall contain an access unit delimiter NAL Unit; NOTE 2 – ITU-T Rec. H.264 | ISO/IEC 14496-10 requires that an access unit delimiter NAL Unit, if present, is the first NAL Unit within an AVC access unit. Access unit delimiter NAL Units simplify the ability to detect the boundary between pictures; they avoid the need to process the content of slice headers, and they are particularly useful for the Baseline and Extended profiles where slice order can be arbitrary.
• Each byte stream NAL Unit that carries the access unit delimiter shall contain exactly one zero_byte syntax element. NOTE 3 – The syntax and semantics of byte stream NAL units are defined in Annex B of ITU-T Rec. H.264 |ISO/IEC 14496-10.
• All Sequence and Picture Parameter Sets (SPS and PPS) necessary for decoding the AVC video stream shall be present within that AVC video stream. NOTE 4 – ITU-T Rec. H.264 | ISO/IEC 14496-10 also allows delivery of SPS and PPS by external means. This Specification does not provide support for such delivery, and therefore requires SPS and PPS to be carried within the AVC video stream.
• Each AVC video sequence that contains hrd_parameters() with the low_delay_hrd_flag set to '1', shall carry VUI parameters in which the timing_info_present_flag shall be set to '1'.
NOTE 5 – If the low_delay_hrd_flag is set to '1', then buffer underflow is allowed to occur in the STD model; see 2.14.3 and 2.14.4. Setting the timing_info_present_flag to '1' ensures that the AVC video stream contains sufficient information to determine the DPB output time and the CPB removal time of AVC access units, also in case of underflow.
To provide display specific information such as aspect_ratio, it is strongly recommended that each AVC video stream carries VUI parameters with sufficient information to ensure that the decoded AVC video stream can be displayed correctly by receivers.
對着標准,發現了ADU的問題,標准要求ADU在每個訪問單元都出現一次.然后根據要求更新了代碼之后,VLC仍然播放不出.
打開VLC的日志,顯示CRC,PMT的錯誤. 這次終於在vlc上看到了視頻了,然而音頻不是沒有聲音就是不正常,上到IPHONE上實驗,索性一點聲音都沒有.
那么音頻的問題在哪?
從以下幾點開始懷疑:
1) ADTS頭的問題, 確實發現了問題,頻率等信息是hardcode進去的,改正這個bug后,可以在VLC上正常播放了.但是在IPHONE上依然保持沉默.
TS流結構的問題: 對比可正常在IPHONE上播放,TS流,發現了有的TS流PCR專門用一個流來傳送,但有的卻直接使用視頻的攜帶的PCR.
2,3天過去了音頻的在IPHONE上的問題依然沒有解決.
最后終於懷疑到音頻的時戳上了,通過跟正確的對比,發現音頻從RTSP收下來的是傳竟然幾幀都相同,但是可以正確播放的音頻時間戳每次都有變化.時戳的變化也是有規律的.按照這個規律更改了時戳.終於聽到了久違的聲音.
事后回頭查了TS的標准,確認時戳是要變化的.
但是RTSP為什么產生了這種時戳呢?誰知道呢,或許是所謂deepth 導致的吧