安卓視頻處理總結

 

 

04-04 16:55:57.099 28015-28195/com.rockylearnstorock.testcamera D/MediaHelper: {csd-1=java.nio.ByteArrayBuffer[position=0,limit=8,capacity=8], mime=video/avc, frame-rate=30, rotation-degrees=270, height=144, width=176, max-input-size=5755, durationUs=21107833, csd-0=java.nio.ByteArrayBuffer[position=0,limit=20,capacity=20]}
04-04 16:55:57.106 28015-28195/com.rockylearnstorock.testcamera D/MediaHelper: {mime=audio/mpeg, durationUs=256835918, encoder-delay=576, channel-count=2, encoder-padding=1681, sample-rate=44100, bit-rate=128000}
04-04 16:55:57.106 28015-28195/com.rockylearnstorock.testcamera E/MPEG4Writer: Unsupported mime 'audio/mpeg'
04-04 16:55:57.107 28015-28195/com.rockylearnstorock.testcamera W/System.err: java.lang.IllegalStateException: Failed to add the track to the muxer
04-04 16:55:57.111 28015-28195/com.rockylearnstorock.testcamera W/System.err: at android.media.MediaMuxer.nativeAddTrack(Native Method)
04-04 16:55:57.111 28015-28195/com.rockylearnstorock.testcamera W/System.err: at android.media.MediaMuxer.addTrack(MediaMuxer.java:294)
04-04 16:55:57.112 28015-28195/com.rockylearnstorock.testcamera W/System.err: at com.rockylearnstorock.testcamera.MediaHelper.combineAudioVideo(MediaHelper.java:181)
04-04 16:55:57.112 28015-28195/com.rockylearnstorock.testcamera W/System.err: at com.rockylearnstorock.testcamera.MainActivity$2.run(MainActivity.java:399)
04-04 16:55:57.112 28015-28195/com.rockylearnstorock.testcamera W/System.err: at java.lang.Thread.run(Thread.java:818)

 

Google "Unsupported mime 'audio/mpeg'， 找到StackOverflow： Impossible to mix audio file and video file using MediaMuxer?

里面有個解釋是

MediaMuxer does not transcode. If you write out an MPEG4 file, it will expect the video file to be MPEG4/AAC and the audio file to be an AAC file (m4a) as well.
Once you feed it with an m4a, muxing will succeed.

嘗試將MP3換成m4a文件，合成，Negtive。

這個貼子里面還有個人發了一個github 例子，嘗試， 合成的視頻m4a的聲音根本就沒有加進去。Negtive.

 

Google " java.lang.IllegalStateException: Failed to add the track to the muxer", 返回的結果，都是" java.lang.IllegalStateException: Failed to stop the track to the muxer"。這個顯然不是我要問的。

再看報錯行

int audioTrackIndex = muxer.addTrack(audioFormat); //MediaHelper.java:181 報錯行

顯示是將audioFormat添加到muxer時報錯了。

關於MP4和MP3的MediaFormat，打印結果見上面黃色部分：

MP4:{csd-1=java.nio.ByteArrayBuffer[position=0,limit=8,capacity=8], mime=video/avc, frame-rate=30, rotation-degrees=270, height=144, width=176, max-input-size=5755, durationUs=21107833, csd-0=java.nio.ByteArrayBuffer[position=0,limit=20,capacity=20]}

MP3:{mime=audio/mpeg, durationUs=256835918, encoder-delay=576, channel-count=2, encoder-padding=1681, sample-rate=44100, bit-rate=128000}

兩者的編碼一個是avc, 另一個是mpeg. 

根據我的理解， 不同的編碼可能造成這個方法報錯的原因。因為，“Unsupported mime 'audio/mpeg'”。

 

MediaExtractor audioExtractor = new MediaExtractor();
        try {
            audioExtractor.setDataSource("/storage/emulated/0/DCIM/TestCamera/kisstherain.mp3");
        } catch (IOException e) {
            e.printStackTrace();
            Log.d("MediaHelper", "mp3 file can not be found");
            return;
        }

        MediaFormat audioFormat = null;
        for(int i = 0; i < audioExtractor.getTrackCount(); i++){
            audioFormat = audioExtractor.getTrackFormat(i);
            String mime = audioFormat.getString(MediaFormat.KEY_MIME);
            if(mime.startsWith("audio/")) {
                break;
            }
        }

        if(audioFormat == null){
            Log.d("MediaHelper", "audioFormat is null");
            return;
        }else{
            Log.d("MediaHelper", audioFormat.toString());
        }

        int audioTrackIndex = muxer.addTrack(audioFormat); //MediaHelper.java:181 報錯行

 

嘗試將MP3改為m4a文件，進行合並。

1.參考：Android中如何提取和生成mp4文件 （這是目前為止找到的最接近的答案）

按照文中的思路，提取video和audio， 添加到MediaMuxer。生成的視頻只有m4a中的聲音，沒有video原來的聲音。

public static void combineAudioVideo(){

        MediaMuxer muxer;
        try {
            muxer = new MediaMuxer("/storage/emulated/0/DCIM/TestCamera/temp.mp4", MediaMuxer.OutputFormat.MUXER_OUTPUT_MPEG_4);
        } catch (IOException e) {
            e.printStackTrace();

            Log.d("MediaHelper", "MediaMuxer init failed");
            return;
        }
//         More often, the MediaFormat will be retrieved from MediaCodec.getOutputFormat()
//         or MediaExtractor.getTrackFormat().
        MediaExtractor mMediaExtractor = new MediaExtractor();
        try {
            mMediaExtractor.setDataSource(FileHelper.getOutputMediaFile(FileHelper.MEDIA_TYPE_VIDEO).getPath());
        } catch (IOException e) {
            e.printStackTrace();
        }

        int framerate = 0;
        MediaFormat videoFormat = null;
        for(int i = 0; i < mMediaExtractor.getTrackCount(); i++){
            videoFormat = mMediaExtractor.getTrackFormat(i);
            String mime = videoFormat.getString(MediaFormat.KEY_MIME);
            framerate = videoFormat.getInteger(MediaFormat.KEY_FRAME_RATE);
            if(mime.startsWith("video/")) {
                mMediaExtractor.selectTrack(i);
                break;
            }
        }

        if(videoFormat == null){

            Log.d("MediaHelper", "videoFormat is null");
            return;
        }else {
            Log.d("MediaHelper", videoFormat.toString());
        }

        MediaExtractor audioExtractor = new MediaExtractor();
        try {
            audioExtractor.setDataSource("/storage/emulated/0/DCIM/TestCamera/1.m4a");
        } catch (IOException e) {
            e.printStackTrace();
            Log.d("MediaHelper", "mp3 file can not be found");
            return;
        }

        int sampleRate = 0;
        MediaFormat audioFormat = null;
        for(int i = 0; i < audioExtractor.getTrackCount(); i++){
            audioFormat = audioExtractor.getTrackFormat(i);
            String mime = audioFormat.getString(MediaFormat.KEY_MIME);
            sampleRate = audioFormat.getInteger(MediaFormat.KEY_SAMPLE_RATE);
            if(mime.startsWith("audio/")) {
                audioExtractor.selectTrack(i);
                break;
            }
        }

        if(audioFormat == null){
            Log.d("MediaHelper", "audioFormat is null");
            return;
        }else{
            Log.d("MediaHelper", audioFormat.toString());
        }

        int audioTrackIndex = muxer.addTrack(audioFormat);
        int videoTrackIndex = muxer.addTrack(videoFormat);

        muxer.start();

        MediaCodec.BufferInfo info = new MediaCodec.BufferInfo();
        MediaCodec.BufferInfo audioInfo = new MediaCodec.BufferInfo();
        info.presentationTimeUs = 0;
        ByteBuffer buffer = ByteBuffer.allocate(500*1024);
        ByteBuffer audioBuffer = ByteBuffer.allocate(500 * 1024);

        while(true) {
            int sampleSize = mMediaExtractor.readSampleData(buffer, 0);
            if(sampleSize < 0) {
                break;
            }

            int audioSampleSize = audioExtractor.readSampleData(audioBuffer, 0);
            if(audioSampleSize < 0){
                break;
            }

            mMediaExtractor.advance();
            audioExtractor.advance();

            info.offset = 0;
            info.size = sampleSize;
            info.flags = MediaCodec.BUFFER_FLAG_SYNC_FRAME;
            info.presentationTimeUs += 1000*1000/framerate;

            audioInfo.offset = 0;
            audioInfo.size = audioSampleSize;
            audioInfo.flags = MediaCodec.BUFFER_FLAG_SYNC_FRAME;
//            audioInfo.presentationTimeUs += 1000 * 1000 / sampleRate;  //暫提不明白sampleRate有什么作用，如果加了這句，生在的視頻就無法播放
 muxer.writeSampleData(videoTrackIndex, buffer, info);
            muxer.writeSampleData(audioTrackIndex, audioBuffer, audioInfo);
        }

        mMediaExtractor.release();
        audioExtractor.release();

        muxer.stop();
        muxer.release();

    }

 

可能的原因：

 int audioTrackIndex = muxer.addTrack(audioFormat); //添加audio時，只有m4a文件audio，所以原來mp4文件中的就被覆蓋了。

嘗試添加多個audio track 到MediaMuxer, google結果：Add second audio track to MediaMuxer