項目需要進行大文件的讀寫,調查測試的結果使我決定使用MappedByteBuffer及相關類進行文件的操作,效果不是一般的高。
網上參考資源很多,如下兩篇非常不錯:
1、花1K內存實現高效I/O的RandomAccessFile類
2、Java中Stream和Memory-mapped File的I/O性能對比
小結:
1、RandomAccessFile本身不帶緩沖讀寫,和FileInputStream、FileOutputStream等一樣,直接按字節讀寫時,性能不可接受;
2、使用MappedByteBuffer讀寫,固然性能會得到極大提升;其實只要自己處理緩沖,性能都會有非常大的提升,比如以下兩種方式(見下記代碼)中第一種使用了MappedByteBuffer,第二種自己進行緩沖,對於12M的文件,后者的效率甚至高於前者。
3、BufferedXXXX之類的緩沖流,如果僅使用默認的buffer size,性能不一定最優,要權衡不同情況各種因素設置大小。
package helloword.helloword; import java.io.BufferedInputStream; import java.io.BufferedOutputStream; import java.io.FileInputStream; import java.io.FileOutputStream; import java.io.IOException; import java.io.RandomAccessFile; import java.nio.MappedByteBuffer; import java.nio.channels.FileChannel; public class Test { public static void main(String[] args) { // TODO Auto-generated method stub } public void test() throws IOException { /* * 測試結果與Buffer size有關 */ // 1、使用MappedByteBuffer: 0.7s String srcFile = "D://Noname1.txt"; String destFile = "D://copy.txt"; RandomAccessFile rafi = new RandomAccessFile(srcFile, "r"); RandomAccessFile rafo = new RandomAccessFile(destFile, "rw"); FileChannel fci = rafi.getChannel(); FileChannel fco = rafo.getChannel(); long size = fci.size(); MappedByteBuffer mbbi = fci.map(FileChannel.MapMode.READ_ONLY, 0, size); MappedByteBuffer mbbo = fco.map(FileChannel.MapMode.READ_WRITE, 0, size); long start = System.currentTimeMillis(); for (int i = 0; i < size; i++) { byte b = mbbi.get(i); mbbo.put(i, b); } fci.close(); fco.close(); rafi.close(); rafo.close(); System.out.println("Spend: " + (double) (System.currentTimeMillis() - start) / 1000 + "s"); } public void test2() throws IOException { // 2、自己處理Buffer(RandomAccessFile): 0.13s String srcFile = "D://Noname1.txt"; String destFile = "D://copy.txt"; RandomAccessFile rafi = new RandomAccessFile(srcFile, "r"); RandomAccessFile rafo = new RandomAccessFile(destFile, "rw"); byte[] buf = new byte[1024 * 8]; long start = System.currentTimeMillis(); int c = rafi.read(buf); while (c > 0) { if (c == buf.length) { rafo.write(buf); } else { rafo.write(buf, 0, c); } c = rafi.read(buf); } rafi.close(); rafo.close(); System.out.println("Spend: " + (double) (System.currentTimeMillis() - start) / 1000 + "s"); } public void test3() throws IOException { // 3、BufferedInputStream&BufferedOutputStream: 3.02s String srcFile = "D://Noname1.txt"; String destFile = "D://copy.txt"; FileInputStream rafi = new FileInputStream(srcFile); FileOutputStream rafo = new FileOutputStream(destFile); BufferedInputStream bis = new BufferedInputStream(rafi, 8192); BufferedOutputStream bos = new BufferedOutputStream(rafo, 8192); long size = rafi.available(); long start = System.currentTimeMillis(); for (int i = 0; i < size; i++) { byte b = (byte) bis.read(); bos.write(b); } rafi.close(); rafo.close(); System.out.println("Spend: " + (double) (System.currentTimeMillis() - start) / 1000 + "s"); } }
