JAVA導出數據到excel中大數據量的解決方法

最近在做項目功能時 ，發現有20萬以上的數據。要求導出時直接導出成壓縮包。原來的邏輯是使用poi導出到excel，他是操作對象集合然后將結果寫到excel中。

使用poi等導出時，沒有考慮數據量的問題，大數據量無法滿足，有個幾千行jvm就哭了。更別提幾萬行幾百萬行數據了。

經過一天的研究發現一種不會消耗過多內存的方法：

導出成csv格式

大數據量的導出成csv格式分為以下幾步：

1.首先引入需要的jar包 一下是我maven的配置方式

	<dependency>
			<groupId>org.mvel</groupId>
			<artifactId>mvel2</artifactId>
			<version>2.2.8.Final</version>
		</dependency>
		<dependency>
			<groupId>net.sourceforge.javacsv</groupId>
			<artifactId>javacsv</artifactId>
			<version>2.0</version>
		</dependency>

 2.以下是具體的執行代碼，我是用的是jdbcTemplate

public class DownloadVehicleRepair extends AbstractJob {

    @Autowired
    private JdbcTemplate jdbcTemplate;

    @Override
    protected void executeBusiness(Long aLong) {
        System.out.println("開始執行！！！！！！！！！！");
        final String fileName = "車輛維修清單.csv";//壓縮包里面的文件
        final String[] header = {"序號", "第三方機構代碼", "機構名稱", "分公司", "合作機構", "單位類別", "主品牌", "品牌名稱",
                "被投訴", "涉及欺詐", "黑名單", "審核狀態", "維護時間", "維護人員代碼"};
        final String sql = "您需要執行sql”;

        jdbcTemplate.execute(new PreparedStatementCreator() {
            @Override
            public PreparedStatement createPreparedStatement(Connection connection) throws SQLException {
                PreparedStatement pstmt = connection.prepareStatement(sql);
                return pstmt;
            }
        }, new PreparedStatementCallback<Integer>() {
            @Override
            public Integer doInPreparedStatement(PreparedStatement preparedStatement) throws SQLException, DataAccessException {
                ResultSet rs = preparedStatement.executeQuery();
                try {
                    CsvUtil.writeCsv(RuntimeEnvironmentUtil.getValue(SysConstent.code,SysConstent.path) + "\\VehicleRepairDetail.zip",
                            fileName, header, rs);//RuntimeEnvironmentUtil.getValue()是為了獲取你導出到服務器的路徑
                } catch (Exception e) {
                    e.printStackTrace();
                }
                return 0;
            }
        });
        System.out.println("導出完成！！！！！！！！！！！");

    }
}

 3.以下是幫助類

public class CsvUtil {
    // 編碼類型
    public static final Charset CHARSET = Charset.forName("GBK");

    // 分隔符
    public static final char DELIMITER = ',';

    // 文件后綴
    public static final String SUFFIX = ".csv";


    public static void writeCsv(OutputStream out, String[] header, ResultSet rs)
            throws IOException, SQLException {
        CsvWriter writer = null;
        try {
            writer = new CsvWriter(out, CsvUtil.DELIMITER, CsvUtil.CHARSET);
            writeCsv(writer, header, rs);
        } finally {
            if (writer != null)
                writer.close();
        }
    }

    public static void writeCsv(CsvWriter writer, String[] header, ResultSet rs)
            throws IOException, SQLException {
        if (header != null)
            writer.writeRecord(header);
        ResultSetMetaData md = rs.getMetaData();
        int columnCount = md.getColumnCount();
        while (rs.next()) {
            for (int i = 1; i <= columnCount; i++)
                writer.write(rs.getString(i));
            writer.endRecord();
        }
    }

    public static void writeCsv(File file, String[] header, ResultSet rs)
            throws IOException, SQLException {
        BufferedOutputStream out = null;
        FileOutputStream fileOutputStream = null;
        try {
            fileOutputStream = new FileOutputStream(file);
            out = new BufferedOutputStream(fileOutputStream);
            writeCsv(out, header, rs);
        } finally {
            if (out != null) {
                out.flush();
                out.close();
            }
            if (fileOutputStream != null) {
                fileOutputStream.close();
            }
        }
    }

    public static void writeCsv(String csvFilePath, String[] header,
                                ResultSet rs) throws IOException, SQLException {
        writeCsv(new File(csvFilePath), header, rs);
    }

    public static void writeCsv(String zipFilePath, String csvName, String[] header, ResultSet rs)
            throws IOException, SQLException {
        FileOutputStream fos = null;
        BufferedOutputStream bos = null;
        ZipOutputStream zos = null;
        try {
            fos = new FileOutputStream(zipFilePath);
            bos = new BufferedOutputStream(fos);
            zos = new ZipOutputStream(bos);
            zos.putNextEntry(new ZipEntry(csvName));
            writeCsv(zos, header, rs);
        } finally {
            StreamUtil.flush(zos);
            StreamUtil.close(zos);
            //StreamUtil.flush(bos);
            StreamUtil.close(bos);
            //StreamUtil.flush(fos);
            StreamUtil.close(fos);
        }
    }

}

 

public class StreamUtil {
    public static void flush(Flushable flushable) {
        if (flushable != null) {
            try {
                flushable.flush();
            } catch (IOException e) {
                e.printStackTrace();
            }
        }
    }
    public static void close(Closeable closeable){
        if(closeable!=null){
            try {
                closeable.close();
            } catch (IOException e) {
                e.printStackTrace();
            }
        }
    }
}

 4.下面是下載時的action

 @RequestMapping(value = "/downloadVehicleRepair", method = RequestMethod.POST)
    public ResponseEntity<byte[]> download() throws IOException {
        String path = RuntimeEnvironmentUtil.getValue(SysConstent.code,SysConstent.path)+"\\VehicleRepairDetail.zip";
        File file = new File(path);
        HttpHeaders headers = new HttpHeaders();
        String fileName = new String("車輛維修清單.zip".getBytes("UTF-8"), "iso-8859-1");//為了解決中文名稱亂碼問題
        headers.setContentDispositionFormData("attachment", fileName);
        headers.setContentType(MediaType.APPLICATION_OCTET_STREAM);
        return new ResponseEntity<byte[]>(FileUtils.readFileToByteArray(file), headers, HttpStatus.OK);
    }

 總結：以上只是關鍵代碼。使用時只需要稍加改變就可以運行。