基於OpenCv和swing的圖片/視頻展示Java實現

基於OpenCv和swing實現圖片/視頻的展示

圖片的展示

swing展示圖片，多為操作BufferedImage，這里要關注的核心是將Mat轉為BufferedImage。

代碼如下：

    public Image toBufferedImage(Mat matrix) {
        int type = BufferedImage.TYPE_BYTE_GRAY;
        if (matrix.channels() > 1) {
            type = BufferedImage.TYPE_3BYTE_BGR;
        }
        int bufferSize = matrix.channels() * matrix.cols() * matrix.rows();
        byte[] buffer = new byte[bufferSize];
        matrix.get(0,0,buffer); // get all the pixels
        BufferedImage image = new BufferedImage(matrix.cols(),matrix.rows(),type);
        final byte[] targetPixels = ((DataBufferByte)image.getRaster().getDataBuffer()).getData();
        System.arraycopy(buffer,0,targetPixels,0,buffer.length);
        return image;
    }

comment 1：OpenCV Mat --> BufferedImage AWT ， 創建一個byte array用以保存mat 矩陣的像素信息。數組大小為通道數和圖片寬/高之積。其中，Mat.get()將所有的元素導入byte數組。最終，圖片的光柵信息通過 getDataBuffer()和getDate()組成接收數組，並通過System.arraycopy方法完成填充。實現最終的類型與數據的轉移。

comment 2：圖片/視頻，最終展示的都為BufferedImage，並在JFrame中展示，這里可將toBufferedImage與swing組件配置以展示的部分抽取為類ImageReader。如下：

import org.opencv.core.Mat;

import javax.swing.*;
import java.awt.*;
import java.awt.image.BufferedImage;
import java.awt.image.DataBufferByte;

/**
 * @Author: nya
 * @Description: 圖片展示類
 * @Date: Created in 10:55 2018/9/21
 * @Modify by:
 */
public class ImageViewer {
    private JLabel imageView;

    public void show(Mat image,String windowName) {
        setSystemLookAndFeel();
        JFrame frame = createJFrame(windowName);
        Image loadedImage = toBufferedImage(image);
        imageView.setIcon(new ImageIcon(loadedImage));

        frame.pack();
        frame.setLocationRelativeTo(null);
        frame.setVisible(true);
    }

    private JFrame createJFrame(String windowName) {
        JFrame frame = new JFrame(windowName);
        imageView = new JLabel();
        final JScrollPane imageScrollPane = new JScrollPane(imageView);
        imageScrollPane.setPreferredSize(new Dimension(640,480));
        frame.add(imageScrollPane,BorderLayout.CENTER);
        frame.setDefaultCloseOperation(WindowConstants.EXIT_ON_CLOSE);
        return frame;
    }

    private void setSystemLookAndFeel() {
        try {
            UIManager.setLookAndFeel(UIManager.getSystemLookAndFeelClassName());
        } catch (ClassNotFoundException e) {
            e.printStackTrace();
        } catch (InstantiationException e) {
            e.printStackTrace();
        } catch (IllegalAccessException e) {
            e.printStackTrace();
        } catch (UnsupportedLookAndFeelException e) {
            e.printStackTrace();
        }
    }

    public Image toBufferedImage(Mat matrix) {
        int type = BufferedImage.TYPE_BYTE_GRAY;
        if (matrix.channels() > 1) {
            type = BufferedImage.TYPE_3BYTE_BGR;
        }
        int bufferSize = matrix.channels() * matrix.cols() * matrix.rows();
        byte[] buffer = new byte[bufferSize];
        matrix.get(0,0,buffer); // get all the pixels
        BufferedImage image = new BufferedImage(matrix.cols(),matrix.rows(),type);
        final byte[] targetPixels = ((DataBufferByte)image.getRaster().getDataBuffer()).getData();
        System.arraycopy(buffer,0,targetPixels,0,buffer.length);
        return image;
    }
}

image Mat operate code：

        String path = "src/main/java/com/opencv/simpleopencvsample/sample/1.jpg"; 
        Mat openFile = null;
        try {
　　　　　　　// openFile just is path to mat 
            openFile = openFile(path);
            Mat clone = openFile.clone();
            System.out.println(openFile);
            Imgproc.resize(openFile,clone,new Size(640,480));
            System.out.println(clone);
            ImageViewer imageViewer = new ImageViewer();
            imageViewer.show(clone,"Loaded image");

        } catch (Exception e) {
            e.printStackTrace();
        } finally {
            // comment ： never forget to release the matrix
            if (openFile != null ) {
                openFile.release();
            }
        }
        System.out.println(openFile);

視頻的展示

有了圖片展示為藍本，視頻的操作關鍵在於VideoCapture類的使用，展示部分不過是捕獲視頻的每一幀轉為Mat，基於swing循環順序展示即可。獲取mat推薦采用read()，此方法為grab()/retrieve()的結合體。

為適應視頻處理的JFrame寬高，此處自定義設置，主要采用toBufferedImage方法。

VideoCapture -> Mat -> display code:

import org.opencv.core.Core;
import org.opencv.core.Mat;
import org.opencv.videoio.VideoCapture;
import org.opencv.videoio.Videoio;

import javax.swing.*;
import java.awt.*;

/**
 * @Author: nya
 * @Description: 視頻捕獲相關操作類VideoCapture使用
 * @Date: Created in 13:50 2018/9/21
 * @Modify by:
 */
public class VideoCaptureSample {
    static {
        System.loadLibrary(Core.NATIVE_LIBRARY_NAME);
    }
    private JFrame frame;
    private JLabel imageLabel;

    public static void main(String[] args) {
        VideoCaptureSample sample = new VideoCaptureSample();
        sample.initGUI();
        sample.runMainLoop(args);
    }

    private void initGUI(){
        frame = new JFrame("Camera Input Example");
        frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
        frame.setSize(555,970);
        imageLabel = new JLabel();
        frame.add(imageLabel);
        frame.setVisible(true);
    }

    private void runMainLoop(String[] args) {
        ImageViewer viewer = new ImageViewer();
        Mat webcamMatImage = new Mat();
        Image tempImage;
        VideoCapture capture = new VideoCapture("src/main/java/com/opencv/simpleopencvsample/sample/1.mp4");
        capture.set(Videoio.CAP_PROP_FRAME_WIDTH,550);
        capture.set(Videoio.CAP_PROP_FRAME_HEIGHT,960);
        if (capture.isOpened()) {
            while (true) {
                capture.read(webcamMatImage);
                if (!webcamMatImage.empty()) {
                    tempImage = viewer.toBufferedImage(webcamMatImage);
                    ImageIcon imageIcon = new ImageIcon(tempImage,"Captured video");
                    imageLabel.setIcon(imageIcon);
                    frame.pack();
                } else {
                    System.out.println(" --- Frame not captured -- Break !");
                    break;
                }
            }
        } else {
            System.out.println("Couldn't open capture.");
        }
    }
}

常見異常 VideoCapture-isOpened返回false

　　實際測試中，存在視頻路徑正常，isOpened()一直返回false的問題。

　　這是因為視頻處理類VideoCapture位於opencv_videoio模塊，使用該類時需在運行時加載預先構建的opencv_ffmpeg * .dll / so。該模塊如果加載成功，ffmpeg可用於解碼/編碼視頻；否則，使用其它API。在排除路徑問題后，仍然無法讀取視頻則多為此情況。

　　解決方法很簡單，將動態庫dll/so導入 管理員/root 權限下配置的java.library.path路徑下即可。