WebRTC VideoEngine綜合應用示例（一）——視頻通話的基本流程（轉）

本系列目前共三篇文章，后續還會更新

WebRTC VideoEngine綜合應用示例（一）——視頻通話的基本流程

WebRTC VideoEngine綜合應用示例（二）——集成OPENH264編解碼器

WebRTC VideoEngine綜合應用示例（三）——集成X264編碼和ffmpeg解碼

WebRTC技術的出現改變了傳統即時通信的現狀，它是一套開源的旨在建立瀏覽器端對端的通信標准的技術，支持瀏覽器平台，使用P2P架構。WebRTC所采用的技術都是當前VoIP先進的技術，如內部所采用的音頻引擎是Google收購知名GIPS公司獲得的核心技術：視頻編解碼則采用了VP8。

大家都說WebRTC好，是未來的趨勢，但是不得不說這個開源項目對新手學習實在是太不友好，光是windows平台下的編譯就能耗費整整一天的精力，還未必能成功，關於這個問題在我之前的文章中有所描述。編譯成功之后打開一看，整個solution里面有215個項目，絕對讓人當時就懵了，而且最重要的是，google方面似乎沒給出什么有用的文檔供人參考，網絡上有關的資料也多是有關於web端開發的，和Native API開發有關的內容少之又少，於是我決定把自己這兩天學習VideoEngine的成果分享出來，供大家參考，有什么問題也歡迎大家指出，一起學習一起進步。

首先需要說明的是，webrtc項目的all.sln下有一個vie_auto_test項目，里面包含了一些針對VideoEngine的測試程序，我這里的demo就是基於此修改得到的。

先來看一下VideoEngine的核心API，基本上就在以下幾個頭文件中了。

具體來說

ViEBase用於

- 創建和銷毀 VideoEngine 實例

- 創建和銷毀 channels - 將 video channel 和相應的 voice channel 連接到一起並同步 - 發送和接收的開始與停止

ViECapture用於

- 分配capture devices. - 將 capture device 與一個或多個 channels連接起來. - 啟動或停止 capture devices. - 獲得capture device 的可用性.

ViECodec用於

- 設置發送和接收的編解碼器.

- 設置編解碼器特性.

- Key frame signaling.

- Stream management settings.

ViEError即一些預定義的錯誤消息

ViEExternalCodec用於注冊除VP8之外的其他編解碼器

ViEImageProcess提供以下功能

- Effect filters - 抗閃爍 - 色彩增強

ViENetwork用於

- 配置發送和接收地址. - External transport support. - 端口和地址過濾. - Windows GQoS functions and ToS functions. - Packet timeout notification. - Dead‐or‐Alive connection observations.

ViERender用於

- 為輸入視頻流、capture device和文件指定渲染目標. - 配置render streams.

ViERTP_RTCP用於

- Callbacks for RTP and RTCP events such as modified SSRC or CSRC.

- SSRC handling. - Transmission of RTCP reports. - Obtaining RTCP data from incoming RTCP sender reports. - RTP and RTCP statistics (jitter, packet loss, RTT etc.). - Forward Error Correction (FEC). - Writing RTP and RTCP packets to binary files for off‐line analysis of the call quality. - Inserting extra RTP packets into active audio stream.

 

下面將以實現一個視頻通話功能為實例詳細介紹VideoEngine的使用，在文末將附上相應源碼的下載地址

第一步是創建一個VideoEngine實例，如下

webrtc::VideoEngine* ptrViE = NULL;
ptrViE = webrtc::VideoEngine::Create();
if (ptrViE == NULL)
{
    printf("ERROR in VideoEngine::Create\n");
    return -1;
}

然后初始化VideoEngine並創建一個Channel

webrtc::ViEBase* ptrViEBase = webrtc::ViEBase::GetInterface(ptrViE);
if (ptrViEBase == NULL)
{
    printf("ERROR in ViEBase::GetInterface\n");
    return -1;
}

error = ptrViEBase->Init();//這里的Init其實是針對VideoEngine的初始化
if (error == -1)
{
    printf("ERROR in ViEBase::Init\n");
    return -1;
}

webrtc::ViERTP_RTCP* ptrViERtpRtcp = webrtc::ViERTP_RTCP::GetInterface(ptrViE);
if (ptrViERtpRtcp == NULL)
{
    printf("ERROR in ViERTP_RTCP::GetInterface\n");
    return -1;
}

int videoChannel = -1;
error = ptrViEBase->CreateChannel(videoChannel);
if (error == -1)
{
    printf("ERROR in ViEBase::CreateChannel\n");
    return -1;
}

列出可用的capture devices等待用戶進行選擇, 然后進行allocate和connect，最后start選中的capture device

webrtc::ViECapture* ptrViECapture = webrtc::ViECapture::GetInterface(ptrViE);
if (ptrViEBase == NULL)
{
    printf("ERROR in ViECapture::GetInterface\n");
    return -1;
}

const unsigned int KMaxDeviceNameLength = 128;
const unsigned int KMaxUniqueIdLength = 256;
char deviceName[KMaxDeviceNameLength];
memset(deviceName, 0, KMaxDeviceNameLength);
char uniqueId[KMaxUniqueIdLength];
memset(uniqueId, 0, KMaxUniqueIdLength);

printf("Available capture devices:\n");
int captureIdx = 0;
for (captureIdx = 0;
    captureIdx < ptrViECapture->NumberOfCaptureDevices();
    captureIdx++)
{
    memset(deviceName, 0, KMaxDeviceNameLength);
    memset(uniqueId, 0, KMaxUniqueIdLength);

    error = ptrViECapture->GetCaptureDevice(captureIdx, deviceName, KMaxDeviceNameLength, uniqueId, KMaxUniqueIdLength);
    if (error == -1)
    {
        printf("ERROR in ViECapture::GetCaptureDevice\n");
        return -1;
    }
    printf("\t %d. %s\n", captureIdx + 1, deviceName);
}
printf("\nChoose capture device: ");

if (scanf("%d", &captureIdx) != 1)
{
    printf("Error in scanf()\n");
    return -1;
}
getchar();
captureIdx = captureIdx - 1; // Compensate for idx start at 1.

error = ptrViECapture->GetCaptureDevice(captureIdx, deviceName, KMaxDeviceNameLength, uniqueId, KMaxUniqueIdLength);
if (error == -1)
{
    printf("ERROR in ViECapture::GetCaptureDevice\n");
    return -1;
}

int captureId = 0;
error = ptrViECapture->AllocateCaptureDevice(uniqueId, KMaxUniqueIdLength, captureId);
if (error == -1)
{
    printf("ERROR in ViECapture::AllocateCaptureDevice\n");
    return -1;
}

error = ptrViECapture->ConnectCaptureDevice(captureId, videoChannel);
if (error == -1)
{
    printf("ERROR in ViECapture::ConnectCaptureDevice\n");
    return -1;
}

error = ptrViECapture->StartCapture(captureId);
if (error == -1)
{
    printf("ERROR in ViECapture::StartCapture\n");
    return -1;
}

設置RTP/RTCP所采用的模式

error = ptrViERtpRtcp->SetRTCPStatus(videoChannel, webrtc::kRtcpCompound_RFC4585);
if (error == -1)
{
    printf("ERROR in ViERTP_RTCP::SetRTCPStatus\n");
    return -1;
}

設置接收端解碼器出問題的時候，比如關鍵幀丟失或損壞，如何重新請求關鍵幀的方式

error = ptrViERtpRtcp->SetKeyFrameRequestMethod(videoChannel, webrtc::kViEKeyFrameRequestPliRtcp);
if (error == -1)
{
    printf("ERROR in ViERTP_RTCP::SetKeyFrameRequestMethod\n");
    return -1;
}

設置是否為當前channel使用REMB（Receiver Estimated Max Bitrate）包，發送端可以用它表明正在編碼當前channel

接收端用它來記錄當前channel的估計碼率

error = ptrViERtpRtcp->SetRembStatus(videoChannel, true, true);
if (error == -1)
{
    printf("ERROR in ViERTP_RTCP::SetTMMBRStatus\n");
    return -1;
}

設置rendering用於顯示

webrtc::ViERender* ptrViERender = webrtc::ViERender::GetInterface(ptrViE);
if (ptrViERender == NULL)
{
    printf("ERROR in ViERender::GetInterface\n");
    return -1;
}

顯示本地攝像頭數據，這里的window1和下面的window2都是顯示窗口，更詳細的內容后面再說

error = ptrViERender->AddRenderer(captureId, window1, 0, 0.0, 0.0, 1.0, 1.0);
if (error == -1)
{
    printf("ERROR in ViERender::AddRenderer\n");
    return -1;
}

error = ptrViERender->StartRender(captureId);
if (error == -1)
{
    printf("ERROR in ViERender::StartRender\n");
    return -1;
}

顯示接收端收到的解碼數據

error = ptrViERender->AddRenderer(videoChannel, window2, 1, 0.0, 0.0, 1.0, 1.0);
if (error == -1)
{
    printf("ERROR in ViERender::AddRenderer\n");
    return -1;
}

error = ptrViERender->StartRender(videoChannel);
if (error == -1)
{
    printf("ERROR in ViERender::StartRender\n");
    return -1;
}

設置編解碼器

webrtc::ViECodec* ptrViECodec = webrtc::ViECodec::GetInterface(ptrViE);
if (ptrViECodec == NULL)
{
    printf("ERROR in ViECodec::GetInterface\n");
    return -1;
}

VideoCodec videoCodec;
int numOfVeCodecs = ptrViECodec->NumberOfCodecs();
for (int i = 0; i<numOfVeCodecs; ++i)
{
    if (ptrViECodec->GetCodec(i, videoCodec) != -1)
    {
        if (videoCodec.codecType == kVideoCodecVP8)
        {
            break;
        }            
    }
}

videoCodec.targetBitrate = 256;
videoCodec.minBitrate = 200;
videoCodec.maxBitrate = 300;
videoCodec.maxFramerate = 25;

error = ptrViECodec->SetSendCodec(videoChannel, videoCodec);
assert(error != -1);

error = ptrViECodec->SetReceiveCodec(videoChannel, videoCodec);
assert(error != -1);

設置接收和發送地址，然后開始發送和接收

webrtc::ViENetwork* ptrViENetwork = webrtc::ViENetwork::GetInterface(ptrViE);
if (ptrViENetwork == NULL)
{
    printf("ERROR in ViENetwork::GetInterface\n");
    return -1;
}
//VideoChannelTransport是由我們自己定義的類，后面將會詳細介紹
VideoChannelTransport* video_channel_transport = NULL;

video_channel_transport = new VideoChannelTransport(ptrViENetwork, videoChannel);

const char* ipAddress = "127.0.0.1";
const unsigned short rtpPort = 6000;
std::cout << std::endl;
std::cout << "Using rtp port: " << rtpPort << std::endl;
std::cout << std::endl;

error = video_channel_transport->SetLocalReceiver(rtpPort);
if (error == -1)
{
    printf("ERROR in SetLocalReceiver\n");
    return -1;
}
error = video_channel_transport->SetSendDestination(ipAddress, rtpPort);
if (error == -1)
{
    printf("ERROR in SetSendDestination\n");
    return -1;
}

error = ptrViEBase->StartReceive(videoChannel);
if (error == -1)
{
    printf("ERROR in ViENetwork::StartReceive\n");
    return -1;
}

error = ptrViEBase->StartSend(videoChannel);
if (error == -1)
{
    printf("ERROR in ViENetwork::StartSend\n");
    return -1;
}

設置按下回車鍵即停止通話

printf("\n call started\n\n");
printf("Press enter to stop...");
while ((getchar()) != '\n')
{

}

停止通話后的各種stop

error = ptrViEBase->StopReceive(videoChannel);
if (error == -1)
{
    printf("ERROR in ViEBase::StopReceive\n");
    return -1;
}

error = ptrViEBase->StopSend(videoChannel);
if (error == -1)
{
    printf("ERROR in ViEBase::StopSend\n");
    return -1;
}

error = ptrViERender->StopRender(captureId);
if (error == -1)
{
    printf("ERROR in ViERender::StopRender\n");
    return -1;
}

error = ptrViERender->RemoveRenderer(captureId);
if (error == -1)
{
    printf("ERROR in ViERender::RemoveRenderer\n");
    return -1;
}

error = ptrViERender->StopRender(videoChannel);
if (error == -1)
{
    printf("ERROR in ViERender::StopRender\n");
    return -1;
}

error = ptrViERender->RemoveRenderer(videoChannel);
if (error == -1)
{
    printf("ERROR in ViERender::RemoveRenderer\n");
    return -1;
}
error = ptrViECapture->StopCapture(captureId);
if (error == -1)
{
    printf("ERROR in ViECapture::StopCapture\n");
    return -1;
}

error = ptrViECapture->DisconnectCaptureDevice(videoChannel);
if (error == -1)
{
    printf("ERROR in ViECapture::DisconnectCaptureDevice\n");
    return -1;
}

error = ptrViECapture->ReleaseCaptureDevice(captureId);
if (error == -1)
{
    printf("ERROR in ViECapture::ReleaseCaptureDevice\n");
    return -1;
}

error = ptrViEBase->DeleteChannel(videoChannel);
if (error == -1)
{
    printf("ERROR in ViEBase::DeleteChannel\n");
    return -1;
}

delete video_channel_transport;

int remainingInterfaces = 0;
remainingInterfaces = ptrViECodec->Release();
remainingInterfaces += ptrViECapture->Release();
remainingInterfaces += ptrViERtpRtcp->Release();
remainingInterfaces += ptrViERender->Release();
remainingInterfaces += ptrViENetwork->Release();
remainingInterfaces += ptrViEBase->Release();
if (remainingInterfaces > 0)
{
    printf("ERROR: Could not release all interfaces\n");
    return -1;
}

bool deleted = webrtc::VideoEngine::Delete(ptrViE);
if (deleted == false)
{
    printf("ERROR in VideoEngine::Delete\n");
    return -1;
}

return 0;

以上就是VideoEngine的基本使用流程，下面說一下顯示窗口如何創建

這里使用了webrtc已經為我們定義好的類ViEWindowCreator，它有一個成員函數CreateTwoWindows可以直接創建兩個窗口，只需實現定義好窗口名稱、窗口大小以及坐標即可，如下

ViEWindowCreator windowCreator;
ViEAutoTestWindowManagerInterface* windowManager = windowCreator.CreateTwoWindows();
VideoEngineSample(windowManager->GetWindow1(), windowManager->GetWindow2());

這里的VideoEngineSample就是我們在前面所寫的包含全部流程的示例程序，它以兩個窗口的指針作為參數。 

至於前面提到的VideoChannelTransport定義如下

class VideoChannelTransport : public webrtc::test::UdpTransportData
{
public:
    VideoChannelTransport(ViENetwork* vie_network, int channel);

    virtual  ~VideoChannelTransport();

    // Start implementation of UdpTransportData.
    virtual void IncomingRTPPacket(const int8_t* incoming_rtp_packet,
        const int32_t packet_length,
        const char* /*from_ip*/,
        const uint16_t /*from_port*/) OVERRIDE;

    virtual void IncomingRTCPPacket(const int8_t* incoming_rtcp_packet,
        const int32_t packet_length,
        const char* /*from_ip*/,
        const uint16_t /*from_port*/) OVERRIDE;
    // End implementation of UdpTransportData.

    // Specifies the ports to receive RTP packets on.
    int SetLocalReceiver(uint16_t rtp_port);

    // Specifies the destination port and IP address for a specified channel.
    int SetSendDestination(const char* ip_address, uint16_t rtp_port);

private:
    int channel_;
    ViENetwork* vie_network_;
    webrtc::test::UdpTransport* socket_transport_;
};

VideoChannelTransport::VideoChannelTransport(ViENetwork* vie_network,
    int channel)
    : channel_(channel),
    vie_network_(vie_network)
{
    uint8_t socket_threads = 1;
    socket_transport_ = webrtc::test::UdpTransport::Create(channel, socket_threads);
    int registered = vie_network_->RegisterSendTransport(channel,
        *socket_transport_);
}

VideoChannelTransport::~VideoChannelTransport()
{
    vie_network_->DeregisterSendTransport(channel_);
    webrtc::test::UdpTransport::Destroy(socket_transport_);
}

void VideoChannelTransport::IncomingRTPPacket(
    const int8_t* incoming_rtp_packet,
    const int32_t packet_length,
    const char* /*from_ip*/,
    const uint16_t /*from_port*/)
{
    vie_network_->ReceivedRTPPacket(
        channel_, incoming_rtp_packet, packet_length, PacketTime());
}

void VideoChannelTransport::IncomingRTCPPacket(
    const int8_t* incoming_rtcp_packet,
    const int32_t packet_length,
    const char* /*from_ip*/,
    const uint16_t /*from_port*/)
{
    vie_network_->ReceivedRTCPPacket(channel_, incoming_rtcp_packet, packet_length);
}

int VideoChannelTransport::SetLocalReceiver(uint16_t rtp_port)
{
    int return_value = socket_transport_->InitializeReceiveSockets(this, rtp_port);
    if (return_value == 0)
    {
        return socket_transport_->StartReceiving(500);
    }
    return return_value;
}

int VideoChannelTransport::SetSendDestination(const char* ip_address, uint16_t rtp_port)
{
    return socket_transport_->InitializeSendSockets(ip_address, rtp_port);
}

繼承自UdpTransportData類，主要重寫了IncomingRTPPacket和IncomingRTCPPacket兩個成員函數，分別調用了vie_network的ReceivedRTPPacket和ReceivedRTCPPacket方法，當需要將接收到的RTP和RTCP包傳給VideoEngine時就應該使用這兩個函數。

 

該示例程序最后效果如下，我這里是幾個虛擬攝像頭，然后會有兩個窗口，一個是攝像頭畫面，一個是解碼的畫面。

源碼地址在這里，這是一個可以脫離webrtc那個大項目而獨立運行的工程。

 

原文轉自 http://blog.csdn.net/nonmarking/article/details/47375849#