一、RTP視頻數據包讀取並組幀
jitter_buffer.cc webrtc::FrameList::InsertFrame(webrtc::VCMFrameBuffer * frame) 行 58 C++
jitter_buffer.cc webrtc::VCMJitterBuffer::InsertPacket receiver.cc webrtc::VCMReceiver::InsertPacket video_receiver.cc webrtc::vcm::VideoReceiver::IncomingPacket rtp_stream_receiver.cc webrtc::RtpStreamReceiver::OnReceivedPayloadData rtp_receiver_video.cc webrtc::RTPReceiverVideo::ParseRtpPacket rtp_receiver_impl.cc webrtc::RtpReceiverImpl::IncomingRtpPacket rtp_stream_receiver.cc webrtc::RtpStreamReceiver::ReceivePacket rtp_stream_receiver.cc webrtc::RtpStreamReceiver::OnRecoveredPacket ulpfec_receiver_impl.cc webrtc::UlpfecReceiverImpl::ProcessReceivedFec rtp_stream_receiver.cc webrtc::RtpStreamReceiver::ParseAndHandleEncapsulatingHeader rtp_stream_receiver.cc webrtc::RtpStreamReceiver::ReceivePacket rtp_stream_receiver.cc webrtc::RtpStreamReceiver::OnRtpPacket video_receive_stream.cc webrtc::internal::VideoReceiveStream::OnRtpPacket call.cc webrtc::internal::Call::DeliverRtp call.cc webrtc::internal::Call::DeliverPacket webrtcvideoengine2.cc cricket::WebRtcVideoChannel2::OnPacketReceived channel.cc cricket::BaseChannel::OnPacketReceived rtc::MethodFunctor3 rtc::FireAndForgetAsyncClosure<rtc::MethodFunctor3 rtc::AsyncInvoker::OnMessage(rtc::Message * msg) rtc::MessageQueue::Dispatch(rtc::Message * pmsg) rtc::Thread::ProcessMessages(int cmsLoop) rtc::Thread::Run() rtc::Thread::PreRun(void * pv)
1. VideoReceiveStream和AudioReceiveStream都派生自Syncable;
2. VCMJitterBuffer視頻幀管理類,是VCMReceiver的成員變量,負責接收網絡到來的RTP數據包,並組幀,包含幾個frame list:
decodable_frames_:可解碼的幀
incomplete_frames_:不完整的幀
當jitterbuffer InsertPacket時,它會首先找到該packet所屬於的frame。
3. VCMFrameBuffer代表一幀視頻幀
4. VCMSessionInfo成員變量,表示該幀的狀態,每接收一幀都會UpdateCompleteSession更新該幀的狀態,判斷幀完整的依據是:是否有首包和尾包,並且首包和尾包之間的序號是否連續;
5. RTPReceiverVideo類負責解析RTP;
二、RTP音頻數據包讀取
packet_buffer.cc webrtc::PacketBuffer::InsertPacket packet_buffer.cc webrtc::PacketBuffer::InsertPacketList neteq_impl.cc webrtc::NetEqImpl::InsertPacketInternal neteq_impl.cc webrtc::NetEqImpl::InsertPacket acm_receiver.cc webrtc::acm2::AcmReceiver::InsertPacket audio_coding_module.cc webrtc::AudioCodingModuleImpl::IncomingPacket channel.cc webrtc::voe::Channel::OnReceivedPayloadData rtp_receiver_audio.cc webrtc::RTPReceiverAudio::ParseAudioCodecSpecific rtp_receiver_audio.cc webrtc::RTPReceiverAudio::ParseRtpPacket rtp_receiver_impl.cc webrtc::RtpReceiverImpl::IncomingRtpPacket channel.cc webrtc::voe::Channel::ReceivePacket channel.cc webrtc::voe::Channel::ReceivedRTPPacket channel_proxy.cc webrtc::voe::ChannelProxy::ReceivedRTPPacket audio_receive_stream.cc webrtc::internal::AudioReceiveStream::DeliverRtp call.cc webrtc::internal::Call::DeliverRtp webrtc::internal::Call::DeliverPacket cricket::WebRtcVoiceMediaChannel::OnPacketReceived cricket::BaseChannel::OnPacketReceived rtc::MethodFunctor3 rtc::FireAndForgetAsyncClosure<rtc::MethodFunctor3 rtc::AsyncInvoker::OnMessage rtc::MessageQueue::Dispatch rtc::Thread::ProcessMessages rtc::Thread::Run() rtc::Thread::PreRun
1,音頻包讀取和視頻包讀取是一個線程;
三、音頻數據解碼及渲染線程
packet_buffer.cc webrtc::PacketBuffer::GetNextPacket neteq_impl.cc webrtc::NetEqImpl::ExtractPackets neteq_impl.cc webrtc::NetEqImpl::GetDecision neteq_impl.cc webrtc::NetEqImpl::GetAudioInternal neteq_impl.cc webrtc::NetEqImpl::GetAudio acm_receiver.cc webrtc::acm2::AcmReceiver::GetAudio webrtc::AudioCodingModuleImpl::PlayoutData10Ms webrtc::voe::Channel::GetAudioFrameWithMuted webrtc::voe::Channel::GetAudioFrameWithInfo webrtc::voe::ChannelProxy::GetAudioFrameWithInfo webrtc::AudioReceiveStream::GetAudioFrameWithInfo webrtc::AudioMixerImpl::GetAudioFromSources webrtc::AudioMixerImpl::Mix webrtc::AudioTransportProxy::NeedMorePlayData webrtc::AudioDeviceBuffer::RequestPlayoutData webrtc::AudioDeviceWindowsCore::DoRenderThread webrtc::AudioDeviceWindowsCore::WSAPIRenderThread
1. 該線程主要是從NetEQ模塊里取出原始的RTP數據,解碼后進行一系列操作,最終渲染出去,可以參考文檔;
四、視頻數據解碼線程
video_render_frames.cc webrtc::VideoRenderFrames::AddFrame incoming_video_stream.cc webrtc::IncomingVideoStream::OnFrame video_stream_decoder.cc webrtc::VideoStreamDecoder::FrameToRender generic_decoder.cc webrtc::VCMDecodedFrameCallback::Decoded generic_decoder.cc webrtc::VCMDecodedFrameCallback::Decoded generic_decoder.cc webrtc::VCMDecodedFrameCallback::Decoded webrtc::VP8DecoderImpl::ReturnFrame webrtc::VP8DecoderImpl::Decode webrtc::VCMGenericDecoder::Decode webrtc::vcm::VideoReceiver::Decode webrtc::vcm::VideoReceiver::Decode webrtc::VideoReceiveStream::Decode webrtc::VideoReceiveStream::DecodeThreadFunction rtc::PlatformThread::Run() rtc::PlatformThread::StartThread
1. 解碼后的frame調用VideoRenderFrames的AddFrame函數,存放在incoming_frames_變量;
2. 渲染線程調用VideoRenderFrames的FrameToRender函數,取出incoming_frames_中的解碼后的frame;