⑤NuPlayer播放框架之GenericSource源碼分析

[時間：2017-01] [狀態：Open]
[關鍵詞：android，nuplayer，開源播放器，播放框架，GenericSource]

0 導讀

GenericSource是NuPlayer::Source的一個子類，主要功能是負責本地多媒體文件的讀取解析，功能類似FFmpeg的libavformt。
通常GenericSource有以下功能：

• 多媒體文件格式探測
• 多媒體文件解析

本文是我的NuPlayer播放框架的第五篇。

1 GenericSource對外接口及主要成員

NuPlayer::Source抽象類

先從父類開始，接口如下：

// code from ~/frameworks/av/media/libmediaplayerservice/nuplayer/NuPlayerSource.h
struct NuPlayer::Source : public AHandler {
    enum Flags {
        FLAG_CAN_PAUSE          = 1,
        FLAG_CAN_SEEK_BACKWARD  = 2,  // the "10 sec back button"
        FLAG_CAN_SEEK_FORWARD   = 4,  // the "10 sec forward button"
        FLAG_CAN_SEEK           = 8,  // the "seek bar"
        FLAG_DYNAMIC_DURATION   = 16,
        FLAG_SECURE             = 32,
        FLAG_PROTECTED          = 64,
    };

    // The provides message is used to notify the player about various events.
    Source(const sp<AMessage> &notify): mNotify(notify) {}

    virtual void prepareAsync() = 0; // 常規接口，播放、停止、暫停、恢復
    virtual void start() = 0;
    virtual void stop() {}
    virtual void pause() {}
    virtual void resume() {}

    // Explicitly disconnect the underling data source
    virtual void disconnect() {}

    // Returns OK iff more data was available, an error or ERROR_END_OF_STREAM if not.
    virtual status_t feedMoreTSData() = 0;
	// 獲取音視頻格式
    virtual sp<AMessage> getFormat(bool audio);
    virtual sp<MetaData> getFormatMeta(bool /* audio */) { return NULL; }
    virtual sp<MetaData> getFileFormatMeta() const { return NULL; }

    virtual status_t dequeueAccessUnit(bool audio, sp<ABuffer> *accessUnit) = 0;

    virtual status_t getDuration(int64_t * /* durationUs */) {...}
	// 返回實際解析之后的節目數目及信息，這里將每個節目稱為一個Track
    virtual size_t getTrackCount() const {return 0;}
    virtual sp<AMessage> getTrackInfo(size_t /* trackIndex */) const {...}
    virtual ssize_t getSelectedTrack(media_track_type /* type */) const {...}
    virtual status_t selectTrack(size_t /* trackIndex */, bool /* select */, int64_t /* timeUs*/) {...}
	// seek操作的主要執行函數
    virtual status_t seekTo(int64_t /* seekTimeUs */) {...}

    virtual status_t setBuffers(bool /* audio */, Vector<MediaBuffer *> &/* buffers */) {...}

    virtual bool isRealTime() const {return false;}

    virtual bool isStreaming() const {return true;}

    virtual void setOffloadAudio(bool /* offload */) {}

protected:
    virtual ~Source() {}

    virtual void onMessageReceived(const sp<AMessage> &msg);

    sp<AMessage> dupNotify() const { return mNotify->dup(); }

    void notifyFlagsChanged(uint32_t flags);
    void notifyVideoSizeChanged(const sp<AMessage> &format = NULL);
    void notifyInstantiateSecureDecoders(const sp<AMessage> &reply);
    void notifyPrepared(status_t err = OK);

private:
    sp<AMessage> mNotify;
};

從類層次結構來看，NuPlayer::Source的子類有：GenericSource、HTTPLiveSource、RTSPSource、StreamingSource。
本文先從最簡單的GenericSource開始分析。

GenericSource對外接口及主要成員

鑒於存在繼承關系，這里僅給出GenericSource特有的接口，及其主要成員函數：

// 注意這里有部分代碼刪減，並不是全部
struct NuPlayer::GenericSource : public NuPlayer::Source {
    GenericSource(const sp<AMessage> &notify, bool uidValid, uid_t uid);

    status_t setDataSource(const sp<IMediaHTTPService> &httpService,
            const char *url, const KeyedVector<String8, String8> *headers);
    status_t setDataSource(int fd, int64_t offset, int64_t length);
    status_t setDataSource(const sp<DataSource>& dataSource);
private:
	struct Track {
        size_t mIndex;
        sp<IMediaSource> mSource;
        sp<AnotherPacketSource> mPackets;
    };
	// Helper to monitor buffering status. The polling happens every second.
    // When necessary, it will send out buffering events to the player.
    struct BufferingMonitor : public AHandler { ... };

	Vector<sp<IMediaSource> > mSources;
    Track mAudioTrack; // 音頻流
    int64_t mAudioTimeUs;
    int64_t mAudioLastDequeueTimeUs;
    Track mVideoTrack; // 視頻流
    int64_t mVideoTimeUs;
    int64_t mVideoLastDequeueTimeUs;
    Track mSubtitleTrack; // 字幕流
    Track mTimedTextTrack;

    sp<IMediaHTTPService> mHTTPService;
    AString mUri;
    KeyedVector<String8, String8> mUriHeaders;
    int mFd;
    int64_t mOffset;
    int64_t mLength;

    sp<DataSource> mDataSource;
    sp<NuCachedSource2> mCachedSource;
    sp<DataSource> mHttpSource;
    sp<WVMExtractor> mWVMExtractor;
    sp<MetaData> mFileMeta;
    DrmManagerClient *mDrmManagerClient;
    sp<DecryptHandle> mDecryptHandle;
    bool mStarted;
    bool mStopRead;
    int64_t mBitrate;
    sp<BufferingMonitor> mBufferingMonitor;
    uint32_t mPendingReadBufferTypes;
    sp<ABuffer> mGlobalTimedText;

    sp<ALooper> mLooper;
    sp<ALooper> mBufferingMonitorLooper;
};

從這里可以看到GenericSource添加了setDataSource接口，並包含多個Track和各種DataSource/NuCachedSource2。

2 NuPlayer中調用的Source接口

先回顧下NuPlayer源碼解析中的調用的NuPlayer::Source接口。

• 構建部分接口——構造/析構函數/setDataSource
• 基本播放控制接口——prepareAsync/stop/start/pause/resume/seekTo/disconnect
• 節目信息相關——getTrackCount/getTrackInfo/getSelectedTrack/selectTrack/getFormat
• 輔助信息獲取及設置——getDuration/isRealTime/getFormatMeta/isStreaming/setOffloadAudio/setBuffers/setBuffers/feedMoreTSData

一個典型的GenericS初始化邏輯如下：

    sp<AMessage> notify = new AMessage(kWhatSourceNotify, this);
    sp<GenericSource> source = new GenericSource(notify, mUIDValid, mUID);
    status_t err = source->setDataSource(fd, offset, length);

3 構建部分接口

構造函數

GenericSource的構造函數相對簡單，代碼如下：

void NuPlayer::GenericSource::resetDataSource() {
    mHTTPService.clear();
    mHttpSource.clear();
    mUri.clear();
    mUriHeaders.clear();
    if (mFd >= 0) {
        close(mFd);
        mFd = -1;
    }
    mOffset = 0;
    mLength = 0;
    setDrmPlaybackStatusIfNeeded(Playback::STOP, 0);
    mDecryptHandle = NULL;
    mDrmManagerClient = NULL;
    mStarted = false;
    mStopRead = true;

    if (mBufferingMonitorLooper != NULL) {
        mBufferingMonitorLooper->unregisterHandler(mBufferingMonitor->id());
        mBufferingMonitorLooper->stop();
        mBufferingMonitorLooper = NULL;
    }
    mBufferingMonitor->stop();
}
NuPlayer::GenericSource::GenericSource(
        const sp<AMessage> &notify,
        bool uidValid,
        uid_t uid)
    : Source(notify),
      mAudioTimeUs(0),
      mAudioLastDequeueTimeUs(0),
      mVideoTimeUs(0),
      mVideoLastDequeueTimeUs(0),
      mFetchSubtitleDataGeneration(0),
      mFetchTimedTextDataGeneration(0),
      mDurationUs(-1ll),
      mAudioIsVorbis(false),
      mIsWidevine(false),
      mIsSecure(false),
      mIsStreaming(false),
      mUIDValid(uidValid),
      mUID(uid),
      mFd(-1),
      mDrmManagerClient(NULL),
      mBitrate(-1ll),
      mPendingReadBufferTypes(0) {
    mBufferingMonitor = new BufferingMonitor(notify);
    resetDataSource();
    DataSource::RegisterDefaultSniffers(); // 這部分注意下，各種格式的探測鏈就是在這里初始化的
}

邏輯比較簡單都是一些變量及參數的初始化。比較有意思的是關於Sniffer的注冊。我們看一下對應代碼：

// code from ~/frameworks/av/media/libstagefright/DataSource.cpp
Mutex DataSource::gSnifferMutex;
List<DataSource::SnifferFunc> DataSource::gSniffers;
bool DataSource::gSniffersRegistered = false;

// static
void DataSource::RegisterSniffer_l(SnifferFunc func) {
    for (List<SnifferFunc>::iterator it = gSniffers.begin();
         it != gSniffers.end(); ++it) {
        if (*it == func) {
            return;
        }
    }

    gSniffers.push_back(func);
}

// static
void DataSource::RegisterDefaultSniffers() {
    Mutex::Autolock autoLock(gSnifferMutex);
    if (gSniffersRegistered) {
        return;
    }

    RegisterSniffer_l(SniffMPEG4); // mpeg4
    RegisterSniffer_l(SniffMatroska); // mkv
    RegisterSniffer_l(SniffOgg); // ogg
    RegisterSniffer_l(SniffWAV); // wav
    RegisterSniffer_l(SniffFLAC); // flac
    RegisterSniffer_l(SniffAMR); // amr
    RegisterSniffer_l(SniffMPEG2TS); // mpeg-ts
    RegisterSniffer_l(SniffMP3); // mp3
    RegisterSniffer_l(SniffAAC); // aac
    RegisterSniffer_l(SniffMPEG2PS); // mpeg-ps
    if (getuid() == AID_MEDIA) {
        // WVM only in the media server process
        RegisterSniffer_l(SniffWVM);
    }
    RegisterSniffer_l(SniffMidi);

    char value[PROPERTY_VALUE_MAX];
    if (property_get("drm.service.enabled", value, NULL)
            && (!strcmp(value, "1") || !strcasecmp(value, "true"))) {
        RegisterSniffer_l(SniffDRM);
    }
    gSniffersRegistered = true;
}

上面代碼就是將所有支持的容器格式放到一個List中。后面會用到的。

析構函數

析構函數相對簡單，直接銷毀Looper，重置DataSource即可，代碼如下：

NuPlayer::GenericSource::~GenericSource() {
    if (mLooper != NULL) {
        mLooper->unregisterHandler(id());
        mLooper->stop();
    }
    resetDataSource();
}

setDataSource接口

設置數據源的接口，有三個重載函數，這里以file_descriptor的接口為例給出，實現非常簡單，就是保存下參數就算完成了。代碼如下：

status_t NuPlayer::GenericSource::setDataSource(
        int fd, int64_t offset, int64_t length) {
    resetDataSource();

    mFd = dup(fd);
    mOffset = offset;
    mLength = length;

    // delay data source creation to prepareAsync() to avoid blocking
    // the calling thread in setDataSource for any significant time.
    return OK;
}

4 基本播放控制接口

這一部分的接口可以認為是一個多媒體文件播放必然會調用的接口。

prepareAsync

代碼里邊主要是發送kWhatPrepareAsync消息，如下：

void NuPlayer::GenericSource::prepareAsync() {
    sp<AMessage> msg = new AMessage(kWhatPrepareAsync, this);
    msg->post();
}

具體消息處理是在onPrepareAsync中。這個函數完成了格式探測和Metadata提取等操作，處理代碼如下：

// 注意代碼有刪減
void NuPlayer::GenericSource::onPrepareAsync() {
    // delayed data source creation 創建DataSource
    if (mDataSource == NULL) {
		{
            mIsWidevine = false;

            mDataSource = new FileSource(mFd, mOffset, mLength);
            mFd = -1;
        }

        if (mDataSource == NULL) {
            ALOGE("Failed to create data source!");
            notifyPreparedAndCleanup(UNKNOWN_ERROR);
            return;
        }
    }

    if (mDataSource->flags() & DataSource::kIsCachingDataSource) {
        mCachedSource = static_cast<NuCachedSource2 *>(mDataSource.get());
    }

    // init extractor from data source
    status_t err = initFromDataSource();

    if (err != OK) {
        ALOGE("Failed to init from data source!");
        notifyPreparedAndCleanup(err); // 上報處理結果
        return;
    }

    if (mVideoTrack.mSource != NULL) {
        sp<MetaData> meta = doGetFormatMeta(false /* audio */);
        sp<AMessage> msg = new AMessage;
        err = convertMetaDataToMessage(meta, &msg);
        if(err != OK) {
            notifyPreparedAndCleanup(err);
            return;
        }
        notifyVideoSizeChanged(msg); // 上報視頻分辨率
    }
	// 上報流狀態
    notifyFlagsChanged(
            (mIsSecure ? FLAG_SECURE : 0)
            | (mDecryptHandle != NULL ? FLAG_PROTECTED : 0)
            | FLAG_CAN_PAUSE
            | FLAG_CAN_SEEK_BACKWARD
            | FLAG_CAN_SEEK_FORWARD
            | FLAG_CAN_SEEK);

   	finishPrepareAsync();// 上報函數調用正常結束
}

這里重點關注下initFromDataSource函數，因為這里面包含多媒體文件格式探測，代碼如下：

status_t NuPlayer::GenericSource::initFromDataSource() {
    sp<IMediaExtractor> extractor;
    String8 mimeType;
    float confidence;
    sp<AMessage> dummy;
  
    CHECK(mDataSource != NULL);

 	{
        extractor = MediaExtractor::Create(mDataSource,
                mimeType.isEmpty() ? NULL : mimeType.string());
    }

    if (extractor == NULL) {
        return UNKNOWN_ERROR;
    }

    if (extractor->getDrmFlag()) {
        checkDrmStatus(mDataSource);
    }

    mFileMeta = extractor->getMetaData();
    if (mFileMeta != NULL) {
        int64_t duration;
        if (mFileMeta->findInt64(kKeyDuration, &duration)) {
            mDurationUs = duration;
        }
    }

    int32_t totalBitrate = 0;
    size_t numtracks = extractor->countTracks();
    if (numtracks == 0) {
        return UNKNOWN_ERROR;
    }
	// 讀取多媒體文件的全部Track信息
    for (size_t i = 0; i < numtracks; ++i) {
        sp<IMediaSource> track = extractor->getTrack(i);
        if (track == NULL) {
            continue;
        }

        sp<MetaData> meta = extractor->getTrackMetaData(i);
        if (meta == NULL) {
            ALOGE("no metadata for track %zu", i);
            return UNKNOWN_ERROR;
        }

        const char *mime;
        CHECK(meta->findCString(kKeyMIMEType, &mime));

        // 處理下音視頻信息，並保存 ... 省略部分代碼
        mSources.push(track);
        int64_t durationUs;
        if (meta->findInt64(kKeyDuration, &durationUs)) {
            if (durationUs > mDurationUs) {
                mDurationUs = durationUs;
            }
        }

        int32_t bitrate;
        if (totalBitrate >= 0 && meta->findInt32(kKeyBitRate, &bitrate)) {
            totalBitrate += bitrate;
        } else {
            totalBitrate = -1;
        }
    }

    if (mSources.size() == 0) {
        ALOGE("b/23705695");
        return UNKNOWN_ERROR;
    }

    mBitrate = totalBitrate;

    return OK;
}

最終通過MediaExtractor::CreateFromService調用DataSource::sniff函數來判斷具體類型。sniff的實現代碼如下：

bool DataSource::sniff(String8 *mimeType, float *confidence, sp<AMessage> *meta) {
    *mimeType = "";
    *confidence = 0.0f;
    meta->clear();
	// 遍歷，找得分最高的，注意需要遍歷全部支持的格式
    for (List<SnifferFunc>::iterator it = gSniffers.begin();
         it != gSniffers.end(); ++it) {
        String8 newMimeType;
        float newConfidence;
        sp<AMessage> newMeta;
        if ((*it)(this, &newMimeType, &newConfidence, &newMeta)) {
            if (newConfidence > *confidence) {
                *mimeType = newMimeType;
                *confidence = newConfidence;
                *meta = newMeta;
            }
        }
    }

    return *confidence > 0.0;
}

stop/start

stop函數實現相對簡單，直接修改當前狀態。代碼如下：

void NuPlayer::GenericSource::stop() {
    // nothing to do, just account for DRM playback status
    setDrmPlaybackStatusIfNeeded(Playback::STOP, 0);
    mStarted = false;
	// ...
}

start函數主要發送kWhatStart消息，代碼如下：

void NuPlayer::GenericSource::start() {
    mStopRead = false;
    if (mAudioTrack.mSource != NULL) { // 啟動音頻包讀取
        postReadBuffer(MEDIA_TRACK_TYPE_AUDIO);
    }

    if (mVideoTrack.mSource != NULL) { // 啟動視頻包讀取
        postReadBuffer(MEDIA_TRACK_TYPE_VIDEO);
    }

    setDrmPlaybackStatusIfNeeded(Playback::START, getLastReadPosition() / 1000);
    mStarted = true;

    (new AMessage(kWhatStart, this))->post();
}

實際消息響應函數比較簡單，如下：

case kWhatStart:
case kWhatResume:
{
  mBufferingMonitor->restartPollBuffering();
  break;
}

pause/resume

這兩個函數跟start/pause類似，直接設置狀態值，代碼如下：

void NuPlayer::GenericSource::pause() {
    // nothing to do, just account for DRM playback status
    setDrmPlaybackStatusIfNeeded(Playback::PAUSE, 0);
    mStarted = false;
}

void NuPlayer::GenericSource::resume() {
    // nothing to do, just account for DRM playback status
    setDrmPlaybackStatusIfNeeded(Playback::START, getLastReadPosition() / 1000);
    mStarted = true;

    (new AMessage(kWhatResume, this))->post();
}

seekTo

seekTo是實現多媒體文件seek的主要函數，其實現跟kWhatSeek消息有關，代碼如下：

status_t NuPlayer::GenericSource::seekTo(int64_t seekTimeUs) {
    sp<AMessage> msg = new AMessage(kWhatSeek, this);
    msg->setInt64("seekTimeUs", seekTimeUs);

    sp<AMessage> response;
    status_t err = msg->postAndAwaitResponse(&response);
    if (err == OK && response != NULL) {
        CHECK(response->findInt32("err", &err));
    }

    return err;
}

實際消息響應函數在onSeek中，代碼如下：

void NuPlayer::GenericSource::onSeek(sp<AMessage> msg) {
    int64_t seekTimeUs;
    CHECK(msg->findInt64("seekTimeUs", &seekTimeUs));

    sp<AMessage> response = new AMessage;
    status_t err = doSeek(seekTimeUs); // 這是實際作seek的
    response->setInt32("err", err);

    sp<AReplyToken> replyID;
    CHECK(msg->senderAwaitsResponse(&replyID));
    response->postReply(replyID);
}

status_t NuPlayer::GenericSource::doSeek(int64_t seekTimeUs) {
    mBufferingMonitor->updateDequeuedBufferTime(-1ll);

    // If the Widevine source is stopped, do not attempt to read any more buffers.
    if (mStopRead) {
        return INVALID_OPERATION;
    }
    if (mVideoTrack.mSource != NULL) { // 調整視頻讀取時間
        int64_t actualTimeUs;
        readBuffer(MEDIA_TRACK_TYPE_VIDEO, seekTimeUs, &actualTimeUs);

        seekTimeUs = actualTimeUs;
        mVideoLastDequeueTimeUs = seekTimeUs;
    }

    if (mAudioTrack.mSource != NULL) { // 調整音頻讀取時間
        readBuffer(MEDIA_TRACK_TYPE_AUDIO, seekTimeUs);
        mAudioLastDequeueTimeUs = seekTimeUs;
    }

    setDrmPlaybackStatusIfNeeded(Playback::START, seekTimeUs / 1000);
    if (!mStarted) {
        setDrmPlaybackStatusIfNeeded(Playback::PAUSE, 0);
    }

    // If currently buffering, post kWhatBufferingEnd first, so that
    // NuPlayer resumes. Otherwise, if cache hits high watermark
    // before new polling happens, no one will resume the playback.
    mBufferingMonitor->stopBufferingIfNecessary();
    mBufferingMonitor->restartPollBuffering();

    return OK;
}

disconnect

這個函數主要斷開DataSource和GenericSource之間的關聯，保證后續可用，代碼如下：

void NuPlayer::GenericSource::disconnect() {
    sp<DataSource> dataSource, httpSource;
    {
        Mutex::Autolock _l(mDisconnectLock);
        dataSource = mDataSource;
        httpSource = mHttpSource;
    }

    if (dataSource != NULL) {
        // disconnect data source
        if (dataSource->flags() & DataSource::kIsCachingDataSource) {
            static_cast<NuCachedSource2 *>(dataSource.get())->disconnect();
        }
    } else if (httpSource != NULL) {
        static_cast<HTTPBase *>(httpSource.get())->disconnect();
    }
}

5 節目信息相關

getTrackCount、getTrackInfo、selectTrack和getSelectedTrack

這幾個函數都是跟節目選擇有關的，getTrackCount返回當前Source中包含的Track數目（一個Track可以是音頻、視頻、字幕或者文本），getTrackInfo則返回對應索引的詳細信息。getSelectedTrack則返回當前選擇或者正在播放的Track信息。selectTrack則用於選定特定的讀取Track，也可用於取消讀取。

getFormat

這個接口用於獲取音頻或者視頻格式，實現如下：

sp<AMessage> NuPlayer::Source::getFormat(bool audio) {
    sp<MetaData> meta = getFormatMeta(audio);

    if (meta == NULL) {
        return NULL;
    }

    sp<AMessage> msg = new AMessage;

    if(convertMetaDataToMessage(meta, &msg) == OK) {
        return msg;
    }
    return NULL;
}

也就是說可以看看getFormatMeta的實現邏輯，代碼如下：

sp<MetaData> NuPlayer::GenericSource::getFormatMeta(bool audio) {
    sp<AMessage> msg = new AMessage(kWhatGetFormat, this);
    msg->setInt32("audio", audio);

    sp<AMessage> response;
    sp<RefBase> format;
    status_t err = msg->postAndAwaitResponse(&response);
    if (err == OK && response != NULL) {
        CHECK(response->findObject("format", &format));
        return static_cast<MetaData*>(format.get());
    } else {
        return NULL;
    }
}

主要是發送kWhatGetFormat消息，然后交給DataSource處理。

6 輔助信息獲取及設置

這里面幾個函數都比較簡單，多數信息都是在prepareAsync函數中讀取的。這里僅列出代碼：

getDuration

status_t NuPlayer::GenericSource::getDuration(int64_t *durationUs) {
    *durationUs = mDurationUs;
    return OK;
}

isRealTime、isStreaming

virtual bool isRealTime() const {
    return false;
}

bool NuPlayer::GenericSource::isStreaming() const {
    return mIsStreaming;
}

setOffloadAudio/setBuffers

void NuPlayer::GenericSource::setOffloadAudio(bool offload) {
    mBufferingMonitor->setOffloadAudio(offload);
}

status_t NuPlayer::GenericSource::setBuffers(
        bool audio, Vector<MediaBuffer *> &buffers) {
    if (mIsSecure && !audio && mVideoTrack.mSource != NULL) {
        return mVideoTrack.mSource->setBuffers(buffers);
    }
    return INVALID_OPERATION;
}

feedMoreTSData

// 用於測試是否處理完所有數據
status_t NuPlayer::GenericSource::feedMoreTSData() {
    return OK;
}

7 小結

斷斷續續的把本文整理完成，算是基本整清楚了針對文件讀取的解析過程和調用邏輯。本文代碼居多，有點亂。不建議參考，如果有問題，還是直接閱讀代碼吧。