Camera 使用
MIPI CSI用法
RK3566/RK3568平台僅有一個標准物理mipi csi2 dphy,可以工作在兩個模式: full mode 和split mode, 拆分為csi2_dphy0/csi2_dphy1/csi2_dphy2三個邏輯dphy(參見rk3568.dtsi)
Full Mode
-
僅使用csi2_dphy0,csi2_dphy0與csi2_dphy1/csi2_dphy2互斥,不可同時使用;
-
data lane最大4 lanes;
-
最大速率2.5Gbps/lane;
Split Mode
-
僅使用csi2_dphy1和csi2_dphy2, 與csi2_dphy0互斥,不可同時使用;
-
csi2_dphy1和csi2_dphy2可同時使用;
-
csi2_dphy1和csi2_dphy2各自的data lane最大是2 lanes;
-
csi2_dphy1對應物理dphy的lane0/lane1;
-
csi2_dphy2對應物理dphy的lane2/lane3;
-
最大速率2.5Gbps/lane
簡單點來講，如果用單目攝像頭我們可以配置full mode，若使用雙目攝像頭我們可以配置split mode
Full Mode配置
鏈接關系: sensor->csi2_dphy0->isp
Full Mode設備樹配置要點
配置sensor端
我們需要根據板子原理圖的MIPI CSI接口找到sensor是掛在哪個I2C總線上，然后在對應的I2C節點配置camera節點，正確配置camera模組的I2C設備地址、引腳等屬性。如下Core-3568J的xc7160配置：
&i2c4 { status = "okay"; XC7160: XC7160b@1b{ status = "okay"; compatible = "firefly,xc7160"; reg = <0x1b>; clocks = <&cru CLK_CIF_OUT>; clock-names = "xvclk"; power-domains = <&power RK3568_PD_VI>; pinctrl-names = "default"; pinctrl-0 = <&cif_clk>; power-gpios = <&pca9555 PCA_IO0_4 GPIO_ACTIVE_LOW>; reset-gpios = <&pca9555 PCA_IO0_0 GPIO_ACTIVE_HIGH>; pwdn-gpios = <&pca9555 PCA_IO0_1 GPIO_ACTIVE_HIGH>; firefly,clkout-enabled-index = <0>; rockchip,camera-module-index = <0>; rockchip,camera-module-facing = "back"; rockchip,camera-module-name = "NC"; rockchip,camera-module-lens-name = "NC"; port { xc7160_out: endpoint { remote-endpoint = <&mipi_in_ucam4>; data-lanes = <1 2 3 4>; }; }; }; };
csi2_dphy0相關配置
csi2_dphy0與csi2_dphy1/csi2_dphy2互斥,不可同時使用。另外需要使能csi2_dphy_hw節點
&csi2_dphy0 { status = "okay"; /* * dphy0 only used for full mode, * full mode and split mode are mutually exclusive */ ports { #address-cells = <1>; #size-cells = <0>; port@0 { reg = <0>; #address-cells = <1>; #size-cells = <0>; ... mipi_in_ucam4: endpoint@5 { reg = <5>; remote-endpoint = <&xc7160_out>; data-lanes = <1 2 3 4>; }; }; port@1 { reg = <1>; #address-cells = <1>; #size-cells = <0>; csidphy_out: endpoint@0 { reg = <0>; remote-endpoint = <&isp0_in>; }; }; }; }; &csi2_dphy_hw { status = "okay"; }; &csi2_dphy1 { status = "disabled"; }; &csi2_dphy2 { status = "disabled"; };
isp相關配置
其中rkisp_vir0節點的remote-endpoint指向csidphy_out
&rkisp { status = "okay"; }; &rkisp_mmu { status = "okay"; }; &rkisp_vir0 { status = "okay"; port { #address-cells = <1>; #size-cells = <0>; isp0_in: endpoint@0 { reg = <0>; remote-endpoint = <&csidphy_out>; }; }; };
Split Mode配置
鏈接關系:
sensor1->csi_dphy1->isp_vir0
sensor2->csi_dphy2->mipi_csi2->vicap->isp_vir1
Split Mode設備樹配置要點
配置sensor端
我們需要根據板子原理圖的MIPI CSI接口找到兩個sensor是掛在哪個I2C總線上，然后在對應的I2C節點配置兩個camera節點，正確配置camera模組的I2C設備地址、引腳等屬性。如下Core-3568J的gc2053/gc2093配置：
&i2c4 { status = "okay"; gc2053: gc2053@37 { //IR status = "okay"; compatible = "galaxycore,gc2053"; reg = <0x37>; avdd-supply = <&vcc_camera>; power-domains = <&power RK3568_PD_VI>; clock-names = "xvclk"; pinctrl-names = "default"; clocks = <&pmucru CLK_WIFI>; pinctrl-0 = <&refclk_pins>; power-gpios = <&pca9555 PCA_IO0_0 GPIO_ACTIVE_HIGH>; //IR_PWR_EN pwdn-gpios = <&pca9555 PCA_IO0_4 GPIO_ACTIVE_LOW>; firefly,clkout-enabled-index = <1>; rockchip,camera-module-index = <0>; rockchip,camera-module-facing = "back"; rockchip,camera-module-name = "YT-RV1109-2-V1"; rockchip,camera-module-lens-name = "40IR-2MP-F20"; port { gc2053_out: endpoint { remote-endpoint = <&dphy1_in>; data-lanes = <1 2>; }; }; }; gc2093: gc2093b@7e{ //RGB status = "okay"; compatible = "galaxycore,gc2093"; reg = <0x7e>; avdd-supply = <&vcc_camera>; power-domains = <&power RK3568_PD_VI>; clock-names = "xvclk"; pinctrl-names = "default"; flash-leds = <&flash_led>; pwdn-gpios = <&pca9555 PCA_IO0_1 GPIO_ACTIVE_HIGH>; firefly,clkout-enabled-index = <0>; rockchip,camera-module-index = <1>; rockchip,camera-module-facing = "front"; rockchip,camera-module-name = "YT-RV1109-2-V1"; rockchip,camera-module-lens-name = "40IR-2MP-F20"; port { gc2093_out: endpoint { remote-endpoint = <&dphy2_in>; data-lanes = <1 2>; }; }; }; };
csi2_dphy1/csi2_dphy2相關配置
csi2_dphy0與csi2_dphy1/csi2_dphy2互斥,不可同時使用
&csi2_dphy0 { status = "disabled"; }; &csi2_dphy1 { status = "okay"; /* * dphy1 only used for split mode, * can be used concurrently with dphy2 * full mode and split mode are mutually exclusive */ ports { #address-cells = <1>; #size-cells = <0>; port@0 { reg = <0>; #address-cells = <1>; #size-cells = <0>; dphy1_in: endpoint@1 { reg = <1>; remote-endpoint = <&gc2053_out>; data-lanes = <1 2>; }; }; port@1 { reg = <1>; #address-cells = <1>; #size-cells = <0>; dphy1_out: endpoint@1 { reg = <1>; remote-endpoint = <&isp0_in>; }; }; }; }; &csi2_dphy2 { status = "okay"; /* * dphy2 only used for split mode, * can be used concurrently with dphy1 * full mode and split mode are mutually exclusive */ ports { #address-cells = <1>; #size-cells = <0>; port@0 { reg = <0>; #address-cells = <1>; #size-cells = <0>; dphy2_in: endpoint@1 { reg = <1>; remote-endpoint = <&gc2093_out>; data-lanes = <1 2>; }; }; port@1 { reg = <1>; #address-cells = <1>; #size-cells = <0>; dphy2_out: endpoint@1 { reg = <1>; remote-endpoint = <&mipi_csi2_input>; }; }; }; }; &csi2_dphy_hw { status = "okay"; }; &mipi_csi2 { status = "okay"; ports { #address-cells = <1>; #size-cells = <0>; port@0 { reg = <0>; #address-cells = <1>; #size-cells = <0>; mipi_csi2_input: endpoint@1 { reg = <1>; remote-endpoint = <&dphy2_out>; data-lanes = <1 2>; }; }; port@1 { reg = <1>; #address-cells = <1>; #size-cells = <0>; mipi_csi2_output: endpoint@0 { reg = <0>; remote-endpoint = <&cif_mipi_in>; data-lanes = <1 2>; }; }; }; }; &rkcif_mipi_lvds { status = "okay"; port { cif_mipi_in: endpoint { remote-endpoint = <&mipi_csi2_output>; data-lanes = <1 2>; }; }; }; &rkcif_mipi_lvds_sditf { status = "okay"; port { mipi_lvds_sditf: endpoint { remote-endpoint = <&isp1_in>; data-lanes = <1 2>; }; }; };
isp相關配置
其中rkisp_vir0節點的remote-endpoint指向dphy1_out
&rkisp { status = "okay"; }; &rkisp_mmu { status = "okay"; }; &rkisp_vir0 { status = "okay"; port { #address-cells = <1>; #size-cells = <0>; isp0_in: endpoint@0 { reg = <0>; remote-endpoint = <&dphy1_out>; }; }; }; &rkisp_vir1 { status = "okay"; port { reg = <0>; #address-cells = <1>; #size-cells = <0>; isp1_in: endpoint@0 { reg = <0>; remote-endpoint = <&mipi_lvds_sditf>; }; }; }; &rkcif_mmu { status = "okay"; }; &rkcif { status = "okay"; };
軟件相關目錄
Linux Kernel-4.19
|-- arch/arm/boot/dts #DTS配置文件 |-- drivers/phy/rockchip |-- phy-rockchip-mipi-rx.c #mipi dphy驅動 |-- phy-rockchip-csi2-dphy-common.h |-- phy-rockchip-csi2-dphy-hw.c |-- phy-rockchip-csi2-dphy.c |-- drivers/media |-- platform/rockchip/cif #RKCIF驅動 |-- platform/rockchip/isp #RKISP驅動 |-- dev #包含 probe、異步注冊、clock、pipeline、 iommu及media/v4l2 framework |-- capture #包含 mp/sp/rawwr的配置及 vb2,幀中斷處理 |-- dmarx #包含 rawrd的配置及 vb2,幀中斷處理 |-- isp_params #3A相關參數設置 |-- isp_stats #3A相關統計 |-- isp_mipi_luma #mipi數據亮度統計 |-- regs #寄存器相關的讀寫操作 |-- rkisp #isp subdev和entity注冊 |-- csi #csi subdev和mipi配置 |-- bridge #bridge subdev,isp和ispp交互橋梁 |-- platform/rockchip/ispp #rkispp驅動 |-- dev #包含 probe、異步注冊、clock、pipeline、 iommu及media/v4l2 framework |-- stream #包含 4路video輸出的配置及 vb2,幀中斷處理 |-- rkispp #ispp subdev和entity注冊 |-- params #TNR/NR/SHP/FEC/ORB參數設置 |-- stats #ORB統計信息 |-- i2
單目CAM-8MS1M/雙目CAM-2MS2MF攝像頭的使用
firefly已經配置好相應的dts，單目攝像頭CAM-8MS1M和雙目攝像頭CAM-2MS2MF使用互斥，只需包含相應的dtsi文件即可使用單目攝像頭CAM-8MS1M或雙目攝像頭CAM-2MS2MF
使用單目攝像頭CAM-8MS1M
dts的配置默認使用單目攝像頭
diff --git a/kernel/arch/arm64/boot/dts/rockchip/rk3568-firefly-aioj.dts b/kernel/arch/arm64/boot/dts/rockchip/rk3568-firefly-aioj.dts
index 7e2a8b2..14fa027 100755
--- a/kernel/arch/arm64/boot/dts/rockchip/rk3568-firefly-aioj.dts
+++ b/kernel/arch/arm64/boot/dts/rockchip/rk3568-firefly-aioj.dts
@@ -7,6 +7,15 @@
+#include "rk3568-firefly-aioj-cam-8ms1m.dtsi" +//#include "rk3568-firefly-aioj-cam-2ms2m.dtsi"
使用雙目攝像頭CAM-2MS2MF
diff --git a/kernel/arch/arm64/boot/dts/rockchip/rk3568-firefly-aioj.dts b/kernel/arch/arm64/boot/dts/rockchip/rk3568-firefly-aioj.dts
index 7e2a8b2..14fa027 100755
--- a/kernel/arch/arm64/boot/dts/rockchip/rk3568-firefly-aioj.dts
+++ b/kernel/arch/arm64/boot/dts/rockchip/rk3568-firefly-aioj.dts
@@ -7,6 +7,15 @@
- #include "rk3568-firefly-aioj-cam-8ms1m.dtsi" +//#include "rk3568-firefly-aioj-cam-8ms1m.dtsi" - //#include "rk3568-firefly-aioj-cam-2ms2m.dtsi" + #include "rk3568-firefly-aioj-cam-2ms2m.dtsi"
Camera底層調試
使用v4l2-ctl抓取camera數據幀
v4l2-ctl --verbose -d /dev/video0 --set-fmt-video=width=1920,height=1080,pixelformat='NV12' --stream-mmap=4 --set-selection=target=crop,flags=0,top=0,left=0,width=1920,height=1080 --stream-to=/data/out.yuv
把out.yuv文件拷貝出來通過ubuntu去查看
ffplay -f rawvideo -video_size 1920x1080 -pix_fmt nv12 out.yuv
Android系統使用camera應用
Android系統使用camera的apk打開攝像頭需要配置camera3_profiles*.xml，具體可參考Android SDK hardware/rockchip/camera/etc/camera目錄下的文件
Linux系統預覽攝像頭
Buildroot直接使用qcamera打開攝像頭，可進行拍攝與錄制，詳細參考 Buildroot使用手冊
Ubuntu 單目攝像頭預覽可以使用如下腳本：
#!/bin/bash
export DISPLAY=:0 export XAUTHORITY=/home/firefly/.Xauthority WIDTH=1920 HEIGHT=1080 SINK=gtksink gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw,format=NV12,width=${WIDTH},height=${HEIGHT}, framerate=30/1 ! videoconvert ! $SINK & wait
也可以使用之前提到的抓幀方法，抓到數據后用mpv播放：
mpv test.yuv --demuxer=rawvideo --demuxer-rawvideo-w=1920 --demuxer-rawvideo-h=1080
對於雙目攝像頭預覽，則使用如下腳本：
#!/bin/bash
export DISPLAY=:0 export XAUTHORITY=/home/firefly/.Xauthority WIDTH=640 HEIGHT=480 SINK=gtksink gst-launch-1.0 v4l2src device=/dev/video14 ! video/x-raw,format=NV12,width=${WIDTH},height=${HEIGHT}, framerate=30/1 ! videoconvert ! $SINK & gst-launch-1.0 v4l2src device=/dev/video5 ! video/x-raw,format=NV12,width=${WIDTH},height=${HEIGHT}, framerate=30/1 ! videoconvert ! $SINK & wait
IQ文件
raw攝像頭支持的iq文件路徑external/camera_engine_rkaiq/iqfiles/isp21, 與以前不一樣的地方是iq文件不再采用.xml的方式，而是采用.json的方式。雖有提供xml轉json的工具, 但isp20的xml配置轉換后也不適用isp21。
若使用raw攝像頭sensor，請留意isp21目錄所支持的iq文件