JavaCV 采集攝像頭和麥克風數據推流直播

越來越覺得放棄JavaCV FFmpeg native API，直接使用JavaCV二次封裝的API開發是很明智的選擇，使用JavaCV二次封裝的API開發避免了各種內存操作不當引起的crash。

上一次介紹了 JavaCV 采集攝像頭及桌面視頻數據，這次介紹一下如何采集攝像頭和麥克風數據推送到流媒體服務器。

其他關於JavaCV的文章，可以通過下面的鏈接查看：
JavaCV-開發系列文章匯總篇(https://www.cnblogs.com/itqn/p/14696221.html)

引入依賴

跟上一次一樣，這里使用的還是最新的JavaCV庫（1.5.5）.

<properties>
  <javacpp.version>1.5.5</javacpp.version>
</properties>
<dependencies>
  <dependency>
    <groupId>org.bytedeco</groupId>
    <artifactId>javacv</artifactId>
    <version>${javacpp.version}</version>
  </dependency>
  <dependency>
    <groupId>org.bytedeco</groupId>
    <artifactId>javacv-platform</artifactId>
    <version>${javacpp.version}</version>
  </dependency>
</dependencies>

視頻采集錄制

在 JavaCV 采集攝像頭及桌面視頻數據 介紹了兩種采集攝像頭數據的方法，這里采用第一種，即使用 OpencvFrameGrabber 采集攝像頭數據。

public class VideoRecorder implements Runnable {

  private static final int VIDEO_DEVICE_INDEX = 0;
  private FFmpegFrameRecorder recorder;
  private int width, height;
  public VideoRecorder(FFmpegFrameRecorder recorder, int width, int height) {
    this.recorder = recorder;
    this.width = width;
    this.height = height;
  }

  @Override
  public void run() {
    try {
      OpenCVFrameGrabber grabber = new OpenCVFrameGrabber(VIDEO_DEVICE_INDEX);
      grabber.setImageWidth(width);
      grabber.setImageHeight(height);
      grabber.start();

      long startTS = 0, videoTS = 0;
      Frame frame = null;
      while (!Thread.interrupted() && (frame = grabber.grab()) != null) {
        if (startTS == 0) {
          startTS = System.currentTimeMillis();
        }
        videoTS = 1000 * (System.currentTimeMillis() - startTS);
        if (videoTS > recorder.getTimestamp()) {
          recorder.setTimestamp(videoTS);
        }
        recorder.record(frame);
      }

      grabber.close();
    } catch (Exception e) {
      e.printStackTrace();
    }
  }
}

音頻采集錄制

音頻采集直接使用JDK的API即可，直接讀麥克風的數據塞給錄制器，采樣率為44100，16bit，小端模式。

public class AudioRecoder implements Runnable {

  private FFmpegFrameRecorder recorder;
  private int channels;
  private int sampleRate;
  public AudioRecoder(FFmpegFrameRecorder recorder, int sampleRate, int channels) {
    this.recorder = recorder;
    this.sampleRate = sampleRate;
    this.channels = channels;
  }

  @Override
  public void run() {
    try {
      AudioFormat format = new AudioFormat(Float.valueOf(sampleRate), 16, channels, true, false);
      TargetDataLine line = (TargetDataLine) AudioSystem.getLine(new DataLine.Info(TargetDataLine.class, format));
      line.open(format);
      line.start();

      int sampleRate = (int) format.getSampleRate();
      int numChannels = format.getChannels();
      byte[] buffer = new byte[sampleRate * numChannels];

      while (!Thread.interrupted()) {
        int nBytesRead = 0;
        while (nBytesRead == 0) {
          nBytesRead = line.read(buffer, 0, line.available());
        }
        int nSamplesRead = nBytesRead / 2;
        short[] samples = new short[nSamplesRead];
        ByteBuffer.wrap(buffer).order(ByteOrder.LITTLE_ENDIAN).asShortBuffer().get(samples);
        ShortBuffer sBuff = ShortBuffer.wrap(samples, 0, nSamplesRead);
        recorder.recordSamples(sampleRate, numChannels, sBuff);
      }
    } catch (Exception e) {
      e.printStackTrace();
    }
  }

}

錄制推流

這里的錄制推流涉及音頻和視頻，音頻和視頻分開線程采集並錄制，這樣可以降低延遲，在錄制推流方面還可以通過設置錄制參數降低延遲：

recorder.setVideoOption("tune", "zerolatency");
recorder.setVideoOption("preset", "ultrafast");

啟動錄制推流器，視頻采集錄制線程和音頻采集錄制線程，開始推流：

recorder.start();
Thread vt = new Thread(new VideoRecorder(recorder, width, height));
Thread at = new Thread(new AudioRecoder(recorder, sampleRate, channels));
		
vt.start();
at.start();
		
vt.join();
at.join();

效果展示

可以使用VLC拉取程序的推流查看效果：

拉流的音視頻流信息：

錄制推流完整源碼可以在公眾號上獲取

=========================================================
關注公眾號 “HiIT青年” 發送 “javacv-recoder” 獲取。

關注公眾號，閱讀更多文章。