Apache POI是Apache软件基金会的开源函式库,提供跨平台的Java API实现Microsoft Office格式档案读写。但是存在如下一些问题:
对POI有过深入了解的应该知道POI有SAX模式(Dom解析模式)。但SAX模式相对比较复杂,excel有03(xls文件)和07(xlsx文件)两种版本,两个版本数据存储方式截然不同,sax解析方式也各不一样。
POI的SAX模式的API可以一定程度的解决一些内存溢出的问题,但是POI还是有一些缺陷,比如07版Excel解压缩以及解压后存储都是在内存中完成的,内存消耗依然很大,一个3M的excel用POI的SAX解析,依然需要100M左右内存。
大部分使用POI都是使用它的userModel模式。userModel的好处是上手容易、使用简单,然而userModel模式最大的问题是在于非常大的内存消耗,一个几兆的文件解析要用掉几百兆的内存。现在很多应用采用这种模式,之所以还正常在跑一定是并发不大,并发上来后一定会OOM或者频繁的full gc。
总体来说,简单写法重度依赖内存,复杂写法学习成本高。
重写了POI对07版excel的解析。
easyexcel重写了POI对07版excel的解析,可以把内存消耗从100M左右降低到10M以内,并且再大的excel不会出现内存溢出,03版仍依赖POI的SAX模式。
下图为64M内存1分钟内读取75M(46万行25列)的excel(当然还有急速模式能更快,但是内存占用会在100M多一点)。
在上层做了模型转换的封装,让使用者更加简单方便。
特点
pom.xml
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0modelVersion>
<groupId>com.qrxqrxgroupId>
<artifactId>easyexcelartifactId>
<version>1.0-SNAPSHOTversion>
<dependencies>
<dependency>
<groupId>org.projectlombokgroupId>
<artifactId>lombokartifactId>
<version>1.18.8version>
<scope>providedscope>
dependency>
<dependency>
<groupId>junitgroupId>
<artifactId>junitartifactId>
<version>4.12version>
<scope>testscope>
dependency>
<dependency>
<groupId>com.alibabagroupId>
<artifactId>easyexcelartifactId>
<version>2.1.6version>
dependency>
<dependency>
<groupId>org.slf4jgroupId>
<artifactId>slf4j-log4j12artifactId>
<version>1.7.25version>
dependency>
dependencies>
project>
log4j.properties
### direct log messages to stdout ###
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.Target=System.out
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%d{ABSOLUTE} %5p %c{1}:%L - %m%n
### direct message to file mylog.log ###
log4j.appender.file=org.apache.log4j.FileAppender
log4j.appender.file.File=d:/mylog.log
log4j.appender.file.layout=org.apache.log4j.PatternLayout
log4j.appender.file.layout.ConversionPattern=%d{ABSOLUTE} %5p %c{1}:%L - %m%n
### set log levels - for more verbose logging change 'info' to 'debug' ###
log4j.rootLogger=debug, stdout
Student
package com.qrxqrx.easyexcel.domain;
import lombok.Data;
import java.util.Date;
@Data
public class Student {
private String name;
private String gender;
private Date birthday;
private String id;
}
StudentListener
package com.qrxqrx.easyexcel.listener;
import com.alibaba.excel.context.AnalysisContext;
import com.alibaba.excel.event.AnalysisEventListener;
import com.qrxqrx.easyexcel.domain.Student;
//读取文档的监听器类
public class StudentListener extends AnalysisEventListener<Student> {
// 每读一行内容,都会调用一次该对象的invoke方法,在invoke方法中可以操作使用读取到的数据
// 参数student:每次读取到的数据封装到的对象
@Override
public void invoke(Student student, AnalysisContext analysisContext) {
System.out.println(student);
}
// 读取完整个文档之后调用的方法
@Override
public void doAfterAllAnalysed(AnalysisContext analysisContext) {
}
}
ExcelTest
package com.qrxqrx.easyexcel;
import com.alibaba.excel.EasyExcel;
import com.alibaba.excel.read.builder.ExcelReaderBuilder;
import com.alibaba.excel.read.builder.ExcelReaderSheetBuilder;
import com.qrxqrx.easyexcel.domain.Student;
import com.qrxqrx.easyexcel.listener.StudentListener;
import org.junit.Test;
public class ExcelTest {
/*
* 工作簿:一个excel文件就是一个工作簿
* 工作表:一个工作簿中可以有多个工作表(sheet)
* */
@Test
public void test01() {
// 获得工作簿对象
//输入参数:要读的文件的路径、文件中每一行数据要存储到的实体的类型的class、
// 读监听器(每读一行内容,都会调用一次该对象的invoke方法,在invoke方法中可以操作使用读取到的数据)
ExcelReaderBuilder readWorkBook = EasyExcel.read("data.xlsx", Student.class, new StudentListener());
// 获得工作表对象
ExcelReaderSheetBuilder sheet = readWorkBook.sheet();
// 读取工作表中内容
sheet.doRead();
}
}
data.xlsx:
Student
package com.qrxqrx.easyexcel.domain;
import com.alibaba.excel.annotation.ExcelProperty;
import com.alibaba.excel.annotation.write.style.ColumnWidth;
import com.alibaba.excel.annotation.write.style.ContentRowHeight;
import lombok.Data;
import java.util.Date;
@Data
// @ContentRowHeight() // 内容行高
public class Student {
@ColumnWidth(20)
@ExcelProperty(value = "学生姓名", index = 1)
private String name;
@ExcelProperty(value = "学生性别", index = 3)
private String gender;
@ColumnWidth(20)
@ExcelProperty(value = "学生生日", index = 2)
private Date birthday;
@ExcelProperty(value = "ID", index = 4)
private String id;
}
ExcelTest
package com.qrxqrx.easyexcel;
import com.alibaba.excel.EasyExcel;
import com.alibaba.excel.read.builder.ExcelReaderBuilder;
import com.alibaba.excel.read.builder.ExcelReaderSheetBuilder;
import com.alibaba.excel.write.builder.ExcelWriterBuilder;
import com.alibaba.excel.write.builder.ExcelWriterSheetBuilder;
import com.qrxqrx.easyexcel.domain.Student;
import com.qrxqrx.easyexcel.listener.StudentListener;
import org.junit.Test;
import java.util.ArrayList;
import java.util.Date;
public class ExcelTest {
/*
* 工作簿:一个excel文件就是一个工作簿
* 工作表:一个工作簿中可以有多个工作表(sheet)
* */
@Test
public void test01() {
// 获得工作簿对象
//输入参数:要读的文件的路径、文件中每一行数据要存储到的实体的类型的class、
// 读监听器(每读一行内容,都会调用一次该对象的invoke方法,在invoke方法中可以操作使用读取到的数据)
ExcelReaderBuilder readWorkBook = EasyExcel.read("data.xlsx", Student.class, new StudentListener());
// 获得工作表对象
ExcelReaderSheetBuilder sheet = readWorkBook.sheet();
// 读取工作表中内容
sheet.doRead();
}
@Test
public void test02() {
// 工作簿对象
ExcelWriterBuilder writeWorkBook = EasyExcel.write("data-w.xlsx", Student.class);
// 工作表对象
ExcelWriterSheetBuilder sheet = writeWorkBook.sheet();
ArrayList<Student> list = new ArrayList<>();
for (int i = 0; i < 10; i++) {
Student student = new Student();
student.setName("qrx"+i);
student.setGender("男");
student.setBirthday(new Date());
list.add(student);
}
sheet.doWrite(list);
}
}
data-w.xlsx:
使用位置:标准作用在成员变量上。
可选属性:
属性名 | 含义 | 说明 |
---|---|---|
index | 对应excel表中的列数 | 默认-1,建议从0开始 |
value | 对应excel表中的列头 | |
converter | 成员变量转换器 | 自定义转换器需要实现Converter接口 |
使用效果:index属性可以指定当前字段对应excel中的哪一列,可以根据列名value去匹配,也可以不写。如果不使用@ExcelProperties注解,成员变量从上到下的顺序,对应表格中从左到右的顺序。
使用建议:要么全部不写,要么全部用index,要么全部用value去匹配,尽量不要三个混着用。
标注在成员变量上,默认所有字段都会和excel去匹配,加了这个注解会忽略该字段
标注在成员变量上,日期转换,代码中用String类型的成员变量去接收excel中日期格式的数据会调用这个注解。里面的value参照java.text.SimpleDateFormat。
@DateTimeFormat("yyyy-MM-dd")
private Date birthday;
标注在成员变量上,数字转换,代码中用String类型的成员变量去接收excel数字格式的数据会调用这个注解,里面的value参照java.text.DecimalFormat。
标注在类上。
不标注该注解时,默认类中所有成员变量都会参与读写,无论是否在成员变量上加了@ExcelProperty的注解。
标注该注解后,类中的成员变量如果没有标注@ExcelProperty注解将不会参与读写。
ReadWorkbook,ReadSheet都会有的参数,如果为空,默认使用上级。
使用位置:标准作用在成员变量上
可选属性:
属性名 | 含义 | 说明 |
---|---|---|
index | 对应excel表中的列数 | 默认-1,指定时建议从0开始 |
value | 对应excel表中的列头 | |
converter | 成员变量转换器 | 自定义转换器需要实现Converter接口 |
使用效果:index指定写到第几列,如果不指定则根据成员变量位置排序;value指定写入的列头,如果不指定则使用成员变量的名字作为列头;如果要设置复杂的头,可以为value指定多个值。
基本和读取时一致:
writeworkbook、writesheet都会有的参数,如果为空,默认使用上级。
pom.xml
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0modelVersion>
<groupId>com.qrxqrxgroupId>
<artifactId>easyexcelartifactId>
<version>1.0-SNAPSHOTversion>
<packaging>warpackaging>
<dependencies>
<dependency>
<groupId>org.projectlombokgroupId>
<artifactId>lombokartifactId>
<version>1.18.8version>
<scope>providedscope>
dependency>
<dependency>
<groupId>junitgroupId>
<artifactId>junitartifactId>
<version>4.12version>
<scope>testscope>
dependency>
<dependency>
<groupId>com.alibabagroupId>
<artifactId>easyexcelartifactId>
<version>2.1.6version>
dependency>
<dependency>
<groupId>org.slf4jgroupId>
<artifactId>slf4j-log4j12artifactId>
<version>1.7.25version>
dependency>
<dependency>
<groupId>org.springframeworkgroupId>
<artifactId>spring-webmvcartifactId>
<version>5.0.5.RELEASEversion>
dependency>
<dependency>
<groupId>javax.servletgroupId>
<artifactId>javax.servlet-apiartifactId>
<version>4.0.1version>
<scope>providedscope>
dependency>
<dependency>
<groupId>commons-fileuploadgroupId>
<artifactId>commons-fileuploadartifactId>
<version>1.4version>
dependency>
dependencies>
project>
web.xml
<web-app xmlns="http://xmlns.jcp.org/xml/ns/javaee"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://xmlns.jcp.org/xml/ns/javaee http://xmlns.jcp.org/xml/ns/javaee/web-app_4_0.xsd"
version="4.0">
<listener>
<listener-class>org.springframework.web.context.ContextLoaderListenerlistener-class>
listener>
<context-param>
<param-name>contextConfigLocationparam-name>
<param-value>classpath:beans.xmlparam-value>
context-param>
<filter>
<filter-name>characterEncodingFilterfilter-name>
<filter-class>org.springframework.web.filter.CharacterEncodingFilterfilter-class>
<init-param>
<param-name>encodingparam-name>
<param-value>UTF-8param-value>
init-param>
filter>
<filter-mapping>
<filter-name>characterEncodingFilterfilter-name>
<url-pattern>/*url-pattern>
filter-mapping>
<servlet>
<servlet-name>dispatcherServletservlet-name>
<servlet-class>org.springframework.web.servlet.DispatcherServletservlet-class>
<init-param>
<param-name>contextConfigLocationparam-name>
<param-value>classpath:spring-mvc.xmlparam-value>
init-param>
<load-on-startup>1load-on-startup>
servlet>
<servlet-mapping>
<servlet-name>dispatcherServletservlet-name>
<url-pattern>/url-pattern>
servlet-mapping>
web-app>
spring-mvc.xml
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:context="http://www.springframework.org/schema/context"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd">
<context:component-scan base-package="com.qrxqrx.easyexcel.web.controller"/>
<bean class="org.springframework.web.multipart.commons.CommonsMultipartResolver" id="multipartResolver"/>
beans>
beans.xml
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:context="http://www.springframework.org/schema/context"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd">
<context:component-scan base-package="com.qrxqrx.easyexcel">
<context:exclude-filter type="annotation" expression="org.springframework.stereotype.Controller"/>
context:component-scan>
beans>
log4j.properties
### direct log messages to stdout ###
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.Target=System.out
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%d{ABSOLUTE} %5p %c{1}:%L - %m%n
### direct message to file mylog.log ###
log4j.appender.file=org.apache.log4j.FileAppender
log4j.appender.file.File=d:/mylog.log
log4j.appender.file.layout=org.apache.log4j.PatternLayout
log4j.appender.file.layout.ConversionPattern=%d{ABSOLUTE} %5p %c{1}:%L - %m%n
### set log levels - for more verbose logging change 'info' to 'debug' ###
log4j.rootLogger=debug, stdout
Student
package com.qrxqrx.easyexcel.domain;
import com.alibaba.excel.annotation.ExcelProperty;
import com.alibaba.excel.annotation.write.style.ColumnWidth;
import com.alibaba.excel.annotation.write.style.ContentRowHeight;
import lombok.Data;
import java.util.Date;
@Data
// @ContentRowHeight() // 内容行高
public class Student {
@ColumnWidth(20)
@ExcelProperty(value = "学生姓名", index = 1)
private String name;
@ExcelProperty(value = "学生性别", index = 3)
private String gender;
@ColumnWidth(20)
@ExcelProperty(value = "学生生日", index = 2)
private Date birthday;
@ExcelProperty(value = "ID", index = 4)
private String id;
}
StudentService
package com.qrxqrx.easyexcel.service;
import com.qrxqrx.easyexcel.domain.Student;
import java.util.List;
public interface StudentService {
void readExcel(List<Student> students);
}
StudentServiceImpl
package com.qrxqrx.easyexcel.service.impl;
import com.qrxqrx.easyexcel.domain.Student;
import com.qrxqrx.easyexcel.service.StudentService;
import org.springframework.stereotype.Service;
import java.util.List;
@Service
public class StudentServiceImpl implements StudentService {
@Override
public void readExcel(List<Student> students) {
for (Student student : students) {
System.out.println(student);
}
}
}
WebStudentListener
package com.qrxqrx.easyexcel.web.listener;
import com.alibaba.excel.context.AnalysisContext;
import com.alibaba.excel.event.AnalysisEventListener;
import com.qrxqrx.easyexcel.domain.Student;
import com.qrxqrx.easyexcel.service.StudentService;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Scope;
import org.springframework.stereotype.Component;
import java.util.ArrayList;
@Component
@Scope("prototype")
public class WebStudentListener extends AnalysisEventListener<Student> {
@Autowired
StudentService studentService;
ArrayList<Student> students = new ArrayList<>();
@Override
public void invoke(Student student, AnalysisContext analysisContext) {
students.add(student);
if (students.size() % 5 == 0) {
studentService.readExcel(students);
students.clear();
}
}
@Override
public void doAfterAllAnalysed(AnalysisContext analysisContext) {
}
}
StudentController
package com.qrxqrx.easyexcel.web.controller;
import com.alibaba.excel.EasyExcel;
import com.alibaba.excel.read.builder.ExcelReaderBuilder;
import com.alibaba.excel.write.builder.ExcelWriterBuilder;
import com.alibaba.excel.write.builder.ExcelWriterSheetBuilder;
import com.qrxqrx.easyexcel.domain.Student;
import com.qrxqrx.easyexcel.service.StudentService;
import com.qrxqrx.easyexcel.web.listener.WebStudentListener;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.ResponseBody;
import org.springframework.web.multipart.MultipartFile;
import javax.servlet.ServletOutputStream;
import javax.servlet.http.HttpServletResponse;
import java.io.IOException;
import java.net.URLEncoder;
import java.util.ArrayList;
import java.util.Date;
@Controller
@RequestMapping("student")
public class StudentController {
@Autowired
WebStudentListener webStudentListener;
// 上传
@RequestMapping("read")
@ResponseBody
public String readExcel(MultipartFile uploadExcel) {
try {
// 工作簿
ExcelReaderBuilder readWorkBook = EasyExcel.read(uploadExcel.getInputStream(), Student.class, webStudentListener);
// 工作表
readWorkBook.sheet().doRead();
return "success";
} catch (IOException e) {
e.printStackTrace();
return "fail";
}
}
// 下载
@RequestMapping("write")
@ResponseBody
public void writeExcel(HttpServletResponse httpServletResponse) throws IOException {
httpServletResponse.setContentType("application/vnd.ms-excel");
httpServletResponse.setCharacterEncoding("utf-8");
String filename = URLEncoder.encode("data2", "UTF-8");
httpServletResponse.setHeader("Content-Disposition","attachment; filename*=UTF-8''"+filename+".xlsx");
ServletOutputStream outputStream = httpServletResponse.getOutputStream();
ExcelWriterBuilder writeWorkBook = EasyExcel.write(outputStream, Student.class);
ExcelWriterSheetBuilder sheet = writeWorkBook.sheet();
ArrayList<Student> list = new ArrayList<>();
for (int i = 0; i < 10; i++) {
Student student = new Student();
student.setName("qrx"+i);
student.setGender("男");
student.setBirthday(new Date());
list.add(student);
}
sheet.doWrite(list);
}
}
excel表格中用{}来包裹要填充的变量,如果单元格文本中本来就有{、}左右大括号,需要在括号前面使用斜杠转义\{、\}。
代码中被填充数据的实体对象的成员变量名或被填充map集合的key需要和excel中被{}包裹的变量名称一致。
FillData
package com.qrxqrx.easyexcel.domain;
import lombok.Data;
@Data
public class FillData {
private String name;
private int age;
}
ExcelTest
@Test
public void test03() {
String template = "fill-data-template01.xlsx";
ExcelWriterBuilder writeWorkBook = EasyExcel.write("data-fill01.xlsx", FillData.class).withTemplate(template);
ExcelWriterSheetBuilder sheet = writeWorkBook.sheet();
// 准备数据
FillData fillData = new FillData();
fillData.setName("qrx");
fillData.setAge(25);
// 准备map数据
HashMap<String, String> map = new HashMap<>();
map.put("name","qrxqrx");
map.put("age","19");
// sheet.doFill(fillData);
sheet.doFill(map);
}
@Test
public void test04() {
String template = "fill-data-template02.xlsx";
ExcelWriterBuilder writeWorkBook = EasyExcel.write("data-fill02.xlsx", FillData.class).withTemplate(template);
ExcelWriterSheetBuilder sheet = writeWorkBook.sheet();
// 准备数据
List<FillData> list = new ArrayList<>();
for (int i = 0;i < 9;i++) {
FillData fillData = new FillData();
fillData.setName("abc"+i);
fillData.setAge(i+10);
list.add(fillData);
}
sheet.doFill(list);
}
@Test
public void test05() {
String template = "fill-data-template03.xlsx";
ExcelWriter build = EasyExcel.write("data-fill03.xlsx", FillData.class).withTemplate(template).build();
WriteSheet sheet = EasyExcel.writerSheet().build();
// 组合数据填充,因为多组填充的数据量不确定,需要在多组填充完之后另起一行
FillConfig config = FillConfig.builder().forceNewRow(true).build();
// 准备数据
List<FillData> list = new ArrayList<>();
for (int i = 0;i < 9;i++) {
FillData fillData = new FillData();
fillData.setName("abc"+i);
fillData.setAge(i+10);
list.add(fillData);
}
HashMap<String, String> map = new HashMap<>();
map.put("date",new Date().toString());
map.put("total","1w");
build.fill(list, config, sheet);
build.fill(map, sheet);
build.finish();
}
@Test
public void test06() {
String template = "fill-data-template04.xlsx";
ExcelWriter build = EasyExcel.write("data-fill04.xlsx", FillData.class).withTemplate(template).build();
WriteSheet sheet = EasyExcel.writerSheet().build();
FillConfig config = FillConfig.builder().direction(WriteDirectionEnum.HORIZONTAL).build();
// 准备数据
List<FillData> list = new ArrayList<>();
for (int i = 0;i < 9;i++) {
FillData fillData = new FillData();
fillData.setName("abc"+i);
fillData.setAge(i+10);
list.add(fillData);
}
build.fill(list, config, sheet);
build.finish();
}