Java NIO

Java NIO 是Java新的IO類庫(相對於舊IO來說),它的目的是提高速度.雖然舊IO已經使用NIO重新實現過,但是顯示使用NIO對於文件IO和網絡IO的速度還是有很大提升.

NIO的體系結構比較簡單,主要圍繞的是FileChannel和ByteBuffer來使用

• FileChannel相當於IO讀取的內容數據源,可以通過InputStream,OutputStream和RandomAccessFile獲得
• ByteBuffer則是存儲當前讀取出來的數據或者等待寫入數據源FileChannel的數據的媒介,FileChannel數據的讀取和寫入必須通過ByteBuffer來實現

1. 簡單示例

public class SimpeTest {
    private static final String FILE_PATH = "e:\\in.test";
    private static final int BSIZE = 1024;

    public static void main(String[] args) throws IOException {
        // OutputStream
        FileChannel fc = new FileOutputStream(FILE_PATH).getChannel();
        fc.write(ByteBuffer.wrap("Some data ".getBytes()));
        fc.close();

        // RandomAccessFile
        fc = new RandomAccessFile(FILE_PATH, "rw").getChannel();
        fc.position(fc.size()); // 移動到文件尾
        fc.write(ByteBuffer.wrap("some more data".getBytes()));
        fc.close();

        // InputStream
        ByteBuffer bb = ByteBuffer.allocate(BSIZE);
        System.out.println(bb);
        fc = new FileInputStream(FILE_PATH).getChannel();
        fc.read(bb);
        bb.flip(); // 將limit設為position並將position設為0
        while (bb.hasRemaining()) {
            System.out.print((char) bb.get());
        }
        fc.close();
    }
}

 

 

要理解ByteBuffer的用法,理解清楚ByteBuffer的內部結構很重要,下面是示意圖

其中

mark: 標記值,使用mark()函數可以標記當前position,在使用reset()后會將position重置為mark值,默認為-1,即沒有mark值

position: 當前位置

limit: 限制值

capacity: buffer總容量

常用方法:

mark(): 將mark設為position

get(): 讀取position位置數據,使用后position后移

put(): 在position位置寫入數據,使用后position后移

remaining(): 獲取position和limit之間的元素個數

flip(): 將limit設為position,並將position設為0,mark置為-1,一般來講在准備讀取buffer數據前會調用此方法

rewind(): 將position設為0,並丟棄標志位

reset(): 將position設為mark

clear(): 將position設為 0,並將limit設為capacity,不要誤會這個方法的作用,它並沒有清空buffer內的數據,一般來講在寫入buffer數據前會調用此方法

 

 

 2. 對FileChannel做數據轉移

 

public class TransferTest {
    private static final int BSIZE = 1024;
    private static final String IN_FILE_PATH = "e:\\in.test";
    private static final String OUT_FILE_PATH = "e:\\out.test";

    public static void main(String[] args) throws IOException {
        FileChannel in = new FileInputStream(IN_FILE_PATH).getChannel();
        FileChannel out = new FileOutputStream(OUT_FILE_PATH).getChannel();
        in.transferTo(0, in.size(), out);
        // or
        // out.transferFrom(0, in.size(), in);
        in.close();
        out.close();
    }
}

 

 

對文件的數據轉移的最佳方式是使用一個通道直接與另一個通道直接相連,調用FileChannel.transferTo()或FileChannel.transferFrom()方法直接解決問題

 

 

3. 對文件編碼

public class EncodeTest {

    private static final int BSIZE = 1024;
    private static final String OUT_FILE_PATH = "e:\\out.test";

    public static void main(String[] args) throws IOException {
        FileChannel fc = new FileOutputStream(OUT_FILE_PATH).getChannel();
        fc.write(ByteBuffer.wrap("test data".getBytes()));
        fc.close();

        ByteBuffer bb = ByteBuffer.allocate(BSIZE);
        fc = new FileInputStream(OUT_FILE_PATH).getChannel();
        fc.read(bb);
        fc.close();

        // 編碼有問題情況:寫入的時候使用的是UTF-8,而ByteBuffer.asCharBuffer()的解碼編碼是UTF16-BE
        bb.flip();
        System.out.println("Bad encoding : " + bb.asCharBuffer());

        // 使用指定編碼UTF-8 decode
        String encoding = System.getProperty("file.encoding");
        System.out.println("Using charset '" + encoding + "' : " + Charset.forName(encoding).decode(bb));

        // 或者使用指定編碼UTF-16寫入文件
        fc = new FileOutputStream(OUT_FILE_PATH).getChannel();
        fc.write(ByteBuffer.wrap("test data".getBytes("UTF-16BE")));
        fc.close();

        bb.clear();
        fc = new FileInputStream(OUT_FILE_PATH).getChannel();
        fc.read(bb);
        fc.close();

        bb.flip();
        System.out.println("Using charset 'UTF-16BE' to write file : " + bb.asCharBuffer());

        // 或者選擇直接使用CharBuffer寫入
        bb.clear();
        bb.asCharBuffer().put("test data 2");
        fc = new FileOutputStream(OUT_FILE_PATH).getChannel();
        fc.write(bb);
        fc.close();

        fc = new FileInputStream(OUT_FILE_PATH).getChannel();
        bb.clear();
        fc.read(bb);
        bb.flip();
        System.out.println("Use CharBuffer to write file : " + bb.asCharBuffer());
    }
}

 

輸出:

Bad encoding : 瑥獴⁤慴
Using charset 'UTF-8' : test data
Using charset 'UTF-16BE' to write file : test data
Use CharBuffer to write file : test data 2

ByteBuffer可以在寫入或讀取的時候指定編碼,默認的編碼式Buffer編碼是"UTF-16BE"(Big Endian) 

 

3. View Buffer

使用ViewBuffer可以實現多種不同的基本數據類型Buffer的讀取與寫入,例子

public class ViewBufferTest {

    public static void main(String[] args) {
        ByteBuffer bb = ByteBuffer.wrap(new byte[] { 0, 0, 0, 0, 0, 0, 0, 'a' });

        // ByteBuffer
        bb.rewind();
        System.out.print("ByteBuffer: ");
        while (bb.hasRemaining()) {
            System.out.print(bb.position() + " -> " + bb.get() + ", ");
        }
        System.out.println();

        // CharBuffer
        bb.rewind();
        CharBuffer cb = bb.asCharBuffer();
        System.out.print("CharBuffer: ");
        while (cb.hasRemaining()) {
            System.out.print(cb.position() + " -> " + cb.get() + ", ");
        }
        System.out.println();

        // ShortBuffer
        bb.rewind();
        ShortBuffer sb = bb.asShortBuffer();
        System.out.print("ShortBuffer: ");
        while (sb.hasRemaining()) {
            System.out.print(sb.position() + " -> " + sb.get() + ", ");
        }
        System.out.println();

        // IntBuffer
        bb.rewind();
        IntBuffer ib = bb.asIntBuffer();
        System.out.print("IntBuffer: ");
        while (ib.hasRemaining()) {
            System.out.print(ib.position() + " -> " + ib.get() + ", ");
        }
        System.out.println();

        // LongBuffer
        bb.rewind();
        LongBuffer lb = bb.asLongBuffer();
        System.out.print("LongBuffer: ");
        while (lb.hasRemaining()) {
            System.out.print(lb.position() + " -> " + lb.get() + ", ");
        }
        System.out.println();

        // FloatBuffer
        bb.rewind();
        FloatBuffer fb = bb.asFloatBuffer();
        System.out.print("FloatBuffer: ");
        while (fb.hasRemaining()) {
            System.out.print(fb.position() + " -> " + fb.get() + ", ");
        }
        System.out.println();

        // DoubleBuffer
        bb.rewind();
        DoubleBuffer db = bb.asDoubleBuffer();
        System.out.print("DoubleBuffer: ");
        while (db.hasRemaining()) {
            System.out.println(db.position() + " -> " + db.get() + ", ");
        }
        System.out.println();
    }
}

 

輸出

可以看出不同的ViewBuffer每次讀取數據的長度不一樣,其讀取原理如下

 

4. Big Endian 與 Little Endian

ByteBuffer可以使用Big Endian的存儲方式,也可以使用Little Endian的存儲方式,默認是Big Endian

public class EndiansTest {

    public static void main(String[] args) {
        ByteBuffer bb = ByteBuffer.wrap(new byte[12]);
        bb.asCharBuffer().put("abcdef");
        System.out.println(Arrays.toString(bb.array()));

        // Big Endian
        bb.rewind();
        bb.order(ByteOrder.BIG_ENDIAN);
        bb.asCharBuffer().put("abcdef");
        System.out.println(Arrays.toString(bb.array()));

        // Little Endian
        bb.rewind();
        bb.order(ByteOrder.LITTLE_ENDIAN);
        bb.asCharBuffer().put("abcdef");
        System.out.println(Arrays.toString(bb.array()));
    }
}

 

輸出:

[0, 97, 0, 98, 0, 99, 0, 100, 0, 101, 0, 102]
[0, 97, 0, 98, 0, 99, 0, 100, 0, 101, 0, 102]
[97, 0, 98, 0, 99, 0, 100, 0, 101, 0, 102, 0]

由於Char在Java中占用兩個字節,所以使用Big Endian和Little Endian的區別就是兩個字節對調.

 

5. 更為精細的IO操作

下面例子為使用ByteBuffer的操作方法實現字符的兩兩調換的操作

public class UsingBuffers {
    private static void symmetricsScamble(CharBuffer buffer) {
        while (buffer.hasRemaining()) {
            buffer.mark();
            char c1 = buffer.get();
            char c2 = buffer.get();
            buffer.reset();
            buffer.put(c2).put(c1);
        }
    }

    public static void main(String[] args) throws UnsupportedEncodingException {
        // 此處也可以使用CharBuffer來寫入數據,從而避免亂碼問題
        byte[] data = "UsingBuffers".getBytes("UTF-16BE");
        ByteBuffer bb = ByteBuffer.allocate(data.length * 2);
        bb.put(data);

        bb.rewind();
        System.out.println(bb.asCharBuffer());

        bb.rewind();
        symmetricsScamble(bb.asCharBuffer());
        System.out.println(bb.asCharBuffer());

        bb.rewind();
        symmetricsScamble(bb.asCharBuffer());
        System.out.println(bb.asCharBuffer());
    }
}

 

輸出

UsingBuffers

sUniBgfuefsr

UsingBuffers

 

6. 內存映射文件

內存映射文件允許你讀入整個太大而不能讀入內存的的大文件,並假定整個文件都已經在內存中,將其當做一個大數組來操作,這種方式簡化了文件修改的代碼.

他可以將整個或部分文件映射到虛擬內存, 用這種方式我們讀取到整個文件的內容, 從而減少磁盤 IO, 提升讀取性能

下面是一個讀取文件的性能對比

public class MappedIOTest {
    private static final String TEST_FILE = "e:\\test.file";
    private static int numOfInts = 4000000;
    private static int numOfUbuffInts = 200000;

    private abstract static class Tester {
        private String name;

        public Tester(String name) {
            this.name = name;
        }

        // 效率測試模板方法
        public void runTest() {
            System.out.println(name + " : ");
            try {
                long start = System.nanoTime();
                test();
                double duration = System.nanoTime() - start;
                System.out.format("%.2f\n", duration / 1.0e9);
            } catch (IOException e) {
                throw new RuntimeException(e);
            }
        }

        public abstract void test() throws IOException;
    }

    private static Tester[] testers = { new Tester("Stream Write") {

        @Override
        public void test() throws IOException {
            DataOutputStream out = new DataOutputStream(new BufferedOutputStream(new FileOutputStream(TEST_FILE)));
            int i = 0;
            while (i++ < numOfInts) {
                out.writeInt(i);
            }
            out.close();
        }
    }, new Tester("Mapped Write") {

        @Override
        public void test() throws IOException {
            FileChannel fc = new RandomAccessFile(TEST_FILE, "rw").getChannel();
            IntBuffer ib = fc.map(FileChannel.MapMode.READ_WRITE, 0, fc.size()).asIntBuffer();
            int i = 0;
            while (i++ < numOfInts) {
                ib.put(i);
            }
            fc.close();
        }
    }, new Tester("Stream Read") {

        @Override
        public void test() throws IOException {
            DataInputStream in = new DataInputStream(new BufferedInputStream(new FileInputStream(TEST_FILE)));
            int i = 0;
            while (i++ < numOfInts) {
                in.readInt();
            }
            in.close();
        }
    }, new Tester("Mapped Read") {

        @Override
        public void test() throws IOException {
            FileChannel fc = new FileInputStream(TEST_FILE).getChannel();
            IntBuffer ib = fc.map(FileChannel.MapMode.READ_ONLY, 0, fc.size()).asIntBuffer();
            ib.rewind();
            while (ib.hasRemaining()) {
                ib.get();
            }
            fc.close();
        }
    }, new Tester("Stream Read/Write") {

        @Override
        public void test() throws IOException {
            RandomAccessFile raf = new RandomAccessFile(TEST_FILE, "rw");
            raf.writeInt(1);
            int i = 0;
            while (i++ < numOfUbuffInts) {
                raf.seek(raf.length() - 4);
                raf.writeInt(raf.readInt());
            }
            raf.close();
        }
    }, new Tester("Mapped Read/Write") {

        @Override
        public void test() throws IOException {
            FileChannel fc = new RandomAccessFile(TEST_FILE, "rw").getChannel();
            IntBuffer ib = fc.map(FileChannel.MapMode.READ_WRITE, 0, fc.size()).asIntBuffer();
            ib.put(0);
            int i = 0;
            while (i++ < numOfUbuffInts) {
                ib.put(ib.get(i - 1));
            }
            fc.close();
        }
    } };

    public static void main(String[] args) throws IOException {
        for (Tester tester : testers) {
            tester.runTest();
        }
    }
}

 

輸出:

Stream Write :
0.45
Mapped Write :
0.03
Stream Read :
0.43
Mapped Read :
0.03
Stream Read/Write :
8.08
Mapped Read/Write :
0.00

 

7. 文件鎖

NIO引入了文件鎖,文件鎖跟線程鎖有個很大的區別,對文件的操作有可能是在不同的JVM的兩個線程,甚至有可能是一個是Java線程,另一個是操作系統的其他本地線程.而文件鎖對操作系統進程是可見的,因為java的文件鎖直接映射到本地操作系統的加鎖工具,所以它的作用於是整個系統的,而非在JVM內.

FileChannel有tryLock()和lock()方法,他們分別對應非阻塞鎖和阻塞鎖

tryLock()在無法獲得鎖時不會被阻塞,而是直接往下執行代碼

lock()則會在無法獲得鎖時進入等待階段

此外,Java的FileChannel文件鎖還支持對文件的部分加鎖,就像Mapped文件映射一樣,下面是一個例子

public class FileLockTest {
    private static final String LOCK_FILE = "e:\\lock.file";
    private static final int LENGTH = 0x8FFFFFF; // 128M
    private static FileChannel fc;

    public static void main(String[] args) throws IOException {
        fc = new RandomAccessFile(LOCK_FILE, "rw").getChannel();
        MappedByteBuffer mbb = fc.map(FileChannel.MapMode.READ_WRITE, 0, LENGTH);
        for (int i = 0; i < LENGTH; i++) {
            mbb.put((byte) 'x');
        }
        new LockAndModify(mbb, 0, LENGTH / 3);
        new LockAndModify(mbb, LENGTH / 2, LENGTH / 2 + LENGTH / 4);
    }

    private static class LockAndModify extends Thread {
        private ByteBuffer buffer;
        private int start, end;

        public LockAndModify(ByteBuffer bb, int start, int end) {
            this.start = start;
            this.end = end;
            bb.limit(end);
            bb.position(start); // position必須要比limit小,所以position()必須在limit()之后
            buffer = bb.slice();
            start();
        }

        @Override
        public void run() {
            try {
                // 非共享鎖
                FileLock fl = fc.lock(start, end, false);
                System.out.println("Locked: " + start + " to " + end);
                // 修改鎖內文件數據
                while (buffer.position() < buffer.limit() - 1) {
                    buffer.put((byte) (buffer.get() + 1));
                }
                System.out.println("Released: " + start + " to " + end);
            } catch (IOException e) {
                throw new RuntimeException(e);
            }
        }
    }
}

 

總的來說,Java NIO的優勢有:

1. 效率高

2. 文件寫入讀取的編碼控制

3. 文件映射

4. 文件鎖

5. ByteBuffer的靈活性

6. 基礎數據類型Buffer的支持