bayer轉dng實現過程記錄

前言

　　項目中需要將imx185出來的raw數據轉成dng格式，一開始認為很簡單的事情，后面才發現還是挺復雜的！！！首先考慮的是不寫任何代碼，直接用adobe提供的轉換工具來轉，結果發現，不僅是adobe的工具，網上能夠找到的工具（所謂的raw轉dng）都是針對各大廠商的raw，而不能支持最原始的bayer raw。於是又考慮了兩種解決方式，第一種就是將bayer raw隨便轉換成某一個廠商的raw(CR2, CRW, DNG, MRW, NEF, ORF, RAF, DCR, ARW中的一種)，然后用所謂的raw轉dng工具再轉dng，結果發現廠商的raw一般都是不開放的，網上雖然有一些逆向工程，但不完整，沒辦法，只好用第二種方案了。第二種方案是考慮基於adobe提供的dng_sdk開發一個bayer轉dng的程序，但是這個工作量真的很大，首先得完全弄懂dng格式，其次，adobe的sdk沒有任何文檔(html文件夾下有點點，但那個沒多大作用)，網上也找到不任何關於該sdk的說明！沒辦法，只好邊看dng_spec_1.4.0.0邊一步步解決。幸運的時，偶然在論壇上發現了兩個關於bayer轉dng的討論，根據他們的代碼，實現了轉換。

dng_sdk linux移植具體步驟

　　adobe官方提供的dng_sdk只有windows和mac版本，而我用的是linux系統，所以得想辦法將其移植到linux里去。網上有有兩種方案可以參考，方案一，一步一步按照他說的移植就ok了，方案二，也是一步一步根據他里面的文檔並結合dng_sdk文檔來移植。兩種方案我都有試過，都是可行的，下面我主要基於第二種方案來說說具體的實踐過程。

安裝依賴jpeg

在ubuntu下，要安裝jpeg依賴很簡單，一條命令搞定

sudo apt-get install cmake libjpeg8-dev uuid-dev

安裝依賴xmp sdk

下載xmp_sdk

wget http://download.macromedia.com/pub/developer/xmp/sdk/XMP-Toolkit-SDK-CC-201306.zip
unzip XMP-Toolkit-SDK-CC-201306.zip
mv XMP-Toolkit-SDK-CC201306 xmp_sdk

然后根據xmp_sdk/third-party下各子目錄里的ReadMe.txt安裝其他一些依賴

zlib

cd xmp_sdk/third-party/zlib
wget http://zlib.net/zlib-1.2.8.tar.gz
tar xzf zlib-1.2.8.tar.gz
cp zlib-1.2.8/*.h zlib-1.2.8/*.c .

expat

cd xmp_sdk/third-party/expat
Download here the source at http://sourceforge.net/projects/expat/files/expat/2.1.0/expat-2.1.0.tar.gz/download.
tar xzf expat-2.1.0.tar.gz
cp -R expat-2.1.0/lib .

dng sdk編譯

在xmp_sdk文件夾所在的目錄下執行以下命令獲取源碼

git clone https://github.com/yanburman/dng_sdk.git

進入目錄projects/linux/，執行make即可編譯生成dng_validate，該工具用於校驗dng文件，后面生成的文件可以通過該工具來檢測。

bayer轉dng編譯

拷貝下面的代碼並命名為bayer2dng.cpp，

#include <iostream>

#include "dng_color_space.h"
#include "dng_date_time.h"
#include "dng_exceptions.h"
#include "dng_file_stream.h"
#include "dng_globals.h"
#include "dng_host.h"
#include "dng_ifd.h"
#include "dng_image_writer.h"
#include "dng_info.h"
#include "dng_linearization_info.h"
#include "dng_mosaic_info.h"
#include "dng_negative.h"
#include "dng_preview.h"
#include "dng_render.h"
#include "dng_simple_image.h"
#include "dng_tag_codes.h"
#include "dng_tag_types.h"
#include "dng_tag_values.h"
#include "dng_xmp.h"
#include "dng_xmp_sdk.h"
#include "dng_camera_profile.h"
// --------------------------------------------------------------------------------
//
// MakeDNGSample
//
int main(int argc, char *argv[])
{
	// Sample BAYER image at ISO100 and tEXP 1/10 on f/4.0 and focal length 10mm
	std::string m_szInputFile = "pic6";
	uint16 m_unISO = 100;
	double m_dExposureTime = 0.1;
	double m_dLensAperture = 4.0;
	double m_dFocalLength = 10.0;

	// SETTINGS: 12-Bit RGGB BAYER PATTERN
	uint8 m_unColorPlanes = 3;
	uint8 m_unBitDepth = 12;
	uint16 m_unBayerType = 1;    // RGGB
	uint32 m_ulWidth = 1920;
	uint32 m_ulHeight = 1225;
	uint32 m_ulBlackLevel = 200;

	// SETTINGS: Whitebalance D65, Orientation "normal"
	dng_orientation m_oOrientation = dng_orientation::Normal();
	dng_xy_coord m_oWhitebalanceDetectedXY = D65_xy_coord();

	// SETTINGS: Names
	std::string m_szMake = "My Company";
	std::string m_szCameraModel = "My Camera";
	std::string szProfileName = m_szCameraModel;
	std::string szProfileCopyright = m_szMake;

	// Calculate bit limit
	uint16 m_unBitLimit = 0x01 << m_unBitDepth;

	// Form output filenames
	std::string szBaseFilename = "";
	std::string m_szOutputFile = "";
	std::string m_szRenderFile = "";
	std::string m_szPathPrefixInput = "";
	std::string m_szPathPrefixOutput = "";
	std::string m_szPathPrefixProfiles = "";
	size_t unIndex = m_szInputFile.find_last_of(".");
	if (unIndex == std::string::npos)
	{
		szBaseFilename = m_szInputFile;
	}
	else
	{
		szBaseFilename = m_szInputFile.substr(0, unIndex);
	}
	m_szInputFile = m_szPathPrefixInput + m_szInputFile;
	m_szOutputFile = m_szPathPrefixOutput + szBaseFilename + ".dng";
	m_szRenderFile = m_szPathPrefixOutput + szBaseFilename + ".tiff";

	// Create DNG
	try
	{
		dng_host oDNGHost;

		// -------------------------------------------------------------
		// Print settings
		// -------------------------------------------------------------

		printf("\n");
		printf("===============================================================================\n");
		printf("Simple DNG converter\n");
		printf("===============================================================================\n\n");
		printf("\n");
		printf("BAYER:\n");
		printf("%s\n", m_szInputFile.c_str());

		printf("\nConverting...\n");

		// -------------------------------------------------------------
		// BAYER input file settings
		// -------------------------------------------------------------
		dng_file_stream oBayerStream(m_szInputFile.c_str());
        //oBayerStream.SetLittleEndian(false);

		uint32 ullBayerSize = (uint32)oBayerStream.Length();

		AutoPtr<dng_memory_block> oBayerData(oDNGHost.Allocate(ullBayerSize));

		oBayerStream.SetReadPosition(0);

		oBayerStream.Get(oBayerData->Buffer(), ullBayerSize);

		// -------------------------------------------------------------
		// DNG Host Settings
		// -------------------------------------------------------------

		// Set DNG version
		// Remarks: Tag [DNGVersion] / [50706]
		oDNGHost.SetSaveDNGVersion(dngVersion_SaveDefault);

		// Set DNG type to RAW DNG
		// Remarks: Store Bayer CFA data and not already processed data
		oDNGHost.SetSaveLinearDNG(false);

		// -------------------------------------------------------------
		// DNG Image Settings
		// -------------------------------------------------------------

		dng_rect vImageBounds(m_ulHeight, m_ulWidth);

		AutoPtr<dng_image> oImage(oDNGHost.Make_dng_image(vImageBounds, m_unColorPlanes, ttShort));

		dng_pixel_buffer oBuffer;

		oBuffer.fArea      = vImageBounds;
		oBuffer.fPlane     = 0;
		oBuffer.fPlanes    = 1;
		oBuffer.fRowStep   = oBuffer.fPlanes * m_ulWidth;
		oBuffer.fColStep   = oBuffer.fPlanes;
		oBuffer.fPlaneStep = 1;
		oBuffer.fPixelType = ttShort;
		oBuffer.fPixelSize = TagTypeSize(ttShort);
		oBuffer.fData      = oBayerData->Buffer();

		oImage->Put(oBuffer);


		// -------------------------------------------------------------
		// DNG Negative Settings
		// -------------------------------------------------------------

		AutoPtr<dng_negative> oNegative(oDNGHost.Make_dng_negative());

		// Set camera model
		// Remarks: Tag [UniqueCameraModel] / [50708]
		oNegative->SetModelName(m_szCameraModel.c_str());

		// Set localized camera model
		// Remarks: Tag [UniqueCameraModel] / [50709]
		oNegative->SetLocalName(m_szCameraModel.c_str());

		// Set bayer pattern information
		// Remarks: Tag [CFAPlaneColor] / [50710] and [CFALayout] / [50711]
		oNegative->SetColorKeys(colorKeyRed, colorKeyGreen, colorKeyBlue);

		// Set bayer pattern information
		// Remarks: Tag [CFAPlaneColor] / [50710] and [CFALayout] / [50711]
		oNegative->SetBayerMosaic(m_unBayerType);

		// Set bayer pattern information
		// Remarks: Tag [CFAPlaneColor] / [50710] and [CFALayout] / [50711]
		oNegative->SetColorChannels(m_unColorPlanes);

		// Set linearization table
		// Remarks: Tag [LinearizationTable] / [50712]
		AutoPtr<dng_memory_block> oCurve(oDNGHost.Allocate(sizeof(uint16)*m_unBitLimit));
		for ( int32 i=0; i<m_unBitLimit; i++ )
		{
			uint16 *pulItem = oCurve->Buffer_uint16() + i;
			*pulItem = (uint16)(i);
		}
		oNegative->SetLinearization(oCurve);

		// Set black level to auto black level of sensor
		// Remarks: Tag [BlackLevel] / [50714]
		oNegative->SetBlackLevel(m_ulBlackLevel);

		// Set white level
		// Remarks: Tag [WhiteLevel] / [50717]
		oNegative->SetWhiteLevel(m_unBitLimit-1);

		// Set scale to square pixel
		// Remarks: Tag [DefaultScale] / [50718]
		oNegative->SetDefaultScale(dng_urational(1,1), dng_urational(1,1));

		// Set scale to square pixel
		// Remarks: Tag [BestQualityScale] / [50780]
		oNegative->SetBestQualityScale(dng_urational(1,1));

		// Set pixel area
		// Remarks: Tag [DefaultCropOrigin] / [50719]
		oNegative->SetDefaultCropOrigin(0, 0);

		// Set pixel area
		// Remarks: Tag [DefaultCropSize] / [50720]
		oNegative->SetDefaultCropSize(m_ulWidth, m_ulHeight);

		// Set base orientation
		// Remarks: See Restriction / Extension tags chapter
		oNegative->SetBaseOrientation(m_oOrientation);

		// Set camera white XY coordinates
		// Remarks: Tag [AsShotWhiteXY] / [50729]
		oNegative->SetCameraWhiteXY(m_oWhitebalanceDetectedXY);

		// Set baseline exposure
		// Remarks: Tag [BaselineExposure] / [50730]
		oNegative->SetBaselineExposure(0);

		// Set if noise reduction is already applied on RAW data
		// Remarks: Tag [NoiseReductionApplied] / [50935]
		oNegative->SetNoiseReductionApplied(dng_urational(0,1));

		// Set baseline sharpness
		// Remarks: Tag [BaselineSharpness] / [50732]
		oNegative->SetBaselineSharpness(1);

		// -------------------------------------------------------------
		// DNG EXIF Settings
		// -------------------------------------------------------------

		dng_exif *poExif = oNegative->GetExif();

		// Set Camera Make
		// Remarks: Tag [Make] / [EXIF]
		poExif->fMake.Set_ASCII(m_szMake.c_str());

		// Set Camera Model
		// Remarks: Tag [Model] / [EXIF]
		poExif->fModel.Set_ASCII(m_szCameraModel.c_str());

		// Set ISO speed
		// Remarks: Tag [ISOSpeed] / [EXIF]
		poExif->fISOSpeedRatings[0] = m_unISO;
		poExif->fISOSpeedRatings[1] = 0;
		poExif->fISOSpeedRatings[2] = 0;

		// Set WB mode
		// Remarks: Tag [WhiteBalance] / [EXIF]
		poExif->fWhiteBalance = 0;

		// Set metering mode
		// Remarks: Tag [MeteringMode] / [EXIF]
		poExif->fMeteringMode = 2;

		// Set metering mode
		// Remarks: Tag [ExposureBiasValue] / [EXIF]
		poExif->fExposureBiasValue = dng_srational(0, 0);

		// Set aperture value
		// Remarks: Tag [ApertureValue] / [EXIF]
		poExif->SetFNumber(m_dLensAperture);

		// Set exposure time
		// Remarks: Tag [ExposureTime] / [EXIF]
		poExif->SetExposureTime(m_dExposureTime);

		// Set focal length
		// Remarks: Tag [FocalLength] / [EXIF]
		poExif->fFocalLength.Set_real64(m_dFocalLength, 1000);

		// Set lens info
		// Remarks: Tag [LensInfo] / [EXIF]
		poExif->fLensInfo[0].Set_real64(m_dFocalLength, 10);
		poExif->fLensInfo[1].Set_real64(m_dFocalLength, 10);
		poExif->fLensInfo[2].Set_real64(m_dLensAperture, 10);
		poExif->fLensInfo[3].Set_real64(m_dLensAperture, 10);

		// -------------------------------------------------------------
		// DNG Profile Settings: Simple color calibration
		// -------------------------------------------------------------

		// Camera space RGB to XYZ matrix with D65 illumination
		// Remarks: Derived from MATLAB using least square linear regression with
		//          MacBeth ColorChecker classic
		dng_matrix_3by3 oCameraRGB_to_XYZ_D65 = dng_matrix_3by3( 2.3150, 0.0711, 0.1455,
				0.9861, 0.7815,-0.2192,
				0.2082,-0.3349, 1.6432 );
		uint32 ulCalibrationIlluminant1 = lsD65;

		// Camera space RGB to XYZ matrix with StdA illumination
		// Remarks: Derived from MATLAB using least square linear regression with
		//          MacBeth ColorChecker classic
		dng_matrix_3by3 oCameraRGB_to_XYZ_A = dng_matrix_3by3(   1.6335, 0.4718,-0.0656,
				0.5227, 1.0298,-0.3416,
				0.0475,-0.2020 ,1.2522 );

		uint32 ulCalibrationIlluminant2 = lsStandardLightA;

		AutoPtr<dng_camera_profile> oProfile(new dng_camera_profile);

		// Set first illuminant color calibration if available
		if ( ulCalibrationIlluminant1!=0 )
		{
			// Set calibration illuminant 1
			// Remarks: Tag [CalibrationIlluminant1] / [50778]
			oProfile->SetCalibrationIlluminant1(ulCalibrationIlluminant1);

			// Set color matrix 1
			// Remarks: Tag [ColorMatrix1] / [50721]
			oProfile->SetColorMatrix1(Invert(oCameraRGB_to_XYZ_D65));
		}

		// Set second illuminant color calibration if available
		if ( ulCalibrationIlluminant2!=0 )
		{
			// Set calibration illuminant 2
			// Remarks: Tag [CalibrationIlluminant2] / [50779]
			oProfile->SetCalibrationIlluminant2(ulCalibrationIlluminant2);

			// Set color matrix 1
			// Remarks: Tag [ColorMatrix2] / [50722]
			oProfile->SetColorMatrix2(Invert(oCameraRGB_to_XYZ_A));
		}

		// Set name of profile
		// Remarks: Tag [ProfileName] / [50936]
		oProfile->SetName(szProfileName.c_str());

		// Set copyright of profile
		// Remarks: Tag [ProfileCopyright] / [50942]
		oProfile->SetCopyright(szProfileCopyright.c_str());

		// Force flag read from DNG to make sure this profile will be embedded
		oProfile->SetWasReadFromDNG(true);

		// Set policy for profile
		// Remarks: Tag [ProfileEmbedPolicy] / [50941]
		oProfile->SetEmbedPolicy(pepAllowCopying);

		// Add camera profile to negative
		oNegative->AddProfile(oProfile);

		// -------------------------------------------------------------
		// Write DNG file
		// -------------------------------------------------------------

		// Assign Raw image data.
		oNegative->SetStage1Image(oImage);

		// Compute linearized and range mapped image
		oNegative->BuildStage2Image(oDNGHost);

		// Compute demosaiced image (used by preview and thumbnail)
		oNegative->BuildStage3Image(oDNGHost);

		// Update XMP / EXIF
		oNegative->SynchronizeMetadata();

		// Update IPTC
		oNegative->RebuildIPTC(true);

		// Create stream writer for output file
		dng_file_stream oDNGStream(m_szOutputFile.c_str(), true);
        //oDNGStream.SetSwapBytes(1);

		// Write DNG file to disk
		AutoPtr<dng_image_writer> oWriter(new dng_image_writer());
		oWriter->WriteDNG(oDNGHost, oDNGStream, *oNegative.Get(), NULL, ccUncompressed);

 #if 0
		// -------------------------------------------------------------
		// Write TIFF file
		// -------------------------------------------------------------

		// Create stream writer for output file
		dng_file_stream oTIFFStream(m_szRenderFile.c_str(), true);

		// Create render object
		dng_render oRender (oDNGHost, *oNegative);

		// Set exposure compensation
		oRender.SetExposure(0.0);

		// Create final image
		AutoPtr<dng_image> oFinalImage;

		// Render image
		oFinalImage.Reset (oRender.Render ());
		oFinalImage->Rotate(oNegative->Orientation());

		// Write TIFF file to disk
		oWriter->WriteTIFF(oDNGHost, oTIFFStream, *oFinalImage.Get(), piRGB, ccUncompressed, oNegative.Get(), &oRender.FinalSpace ());
#endif

	}

	catch (const dng_exception &except)
	{

		return except.ErrorCode ();

	}

	catch (...)
	{

		return dng_error_unknown;

	}

	printf ("Conversion complete\n");

	return 0;
}

保存，然后執行下面命令編譯

g++ bayer2dng.cpp -I../../../xmp_sdk/public/include -I../../source libdng_sdk.a -ljpeg -lz -lpthread -ldl ../../../xmp_sdk/public/libraries/i80386linux_x64/release//staticXMPFiles.ar ../../../xmp_sdk/public/libraries/i80386linux_x64/release//staticXMPCore.ar -ldl -o bayer2dng

即可生成bayer轉rgb的工具bayer2dng了。

總結

　　上面的bayer2dng.cpp僅僅是將1920 1225分辨率 12bit RGGB格式的raw轉為dng格式的文件，里面很多參數都需要根據實際情況調整，比如曝光時間、iso、光圈大小、焦距、線性表等等，所以繼續研究dng格式是避免不了的^_

引用

• Supported camera models
• MRW File Format
• How to get Raw data from CR2 format.
• Understanding What is stored in a Canon RAW .CR2 file, How and Why
• Digital negative
• MRW Camera Raw Signature Format: Specification & MRW Recovery Example
• Canon's CR2 Raw File Format Specification
• The Canon RAW (CRW) File Format
  
• Digital Cameras - Canon EOS-1D X Test Images
• TIFF圖像文件格式詳解
• TIFF
• TIFF圖像文件格式詳解
• TIFF Tag Reference
• Intel格式與Motorola格式的區別
• SubIFD1 tags not updating
• How to import a real raw CFA image into a DNG ?
• ChromaSoft: dcpTool and the DNG SDK on Linux
• DNG tools in LINUX | shalamander
• dcraw-float command line options

完！
2016年12月