java文件复制的几种方式比较

ゝ一纸荒年。 2023-02-08 12:49 7阅读 0赞

# java文件复制的几种方式比较 #

## 1.文件复制的几种实现方式 ##

### 1.1使用java文件工具类Files ###

/**
     * 文件复制 使用java文件工具类Files
     *
     * @param srcFile  源文件
     * @param destFile 目标文件
     * @throws IOException IO异常
     */
    public static void copyByJavaFiles(File srcFile, File destFile) throws IOException {
        
        Files.copy(srcFile.toPath(), new BufferedOutputStream(new FileOutputStream(destFile)));
    }

### 1.2使用FileInputStream , FileOutputStream ###

/**
     * 文件复制 使用FileInputStream , FileOutputStream
     *
     * @param srcFile  源文件
     * @param destFile 目标文件
     * @throws IOException IO异常
     */
    public static void copyByInOutStream(File srcFile, File destFile) throws IOException {
        
        byte[] bytes = new byte[24 * 1024];
        try (InputStream in = new FileInputStream(srcFile);
             OutputStream out = new FileOutputStream(destFile)) {
        
            int count;
            while ((count = in.read(bytes)) > 0) {
        
                out.write(bytes, 0, count);
            }
        }
    }

### 1.3使用BufferedInputStream , BufferedOutputStream ###

/**
     * 文件复制 使用BufferedInputStream , BufferedOutputStream
     *
     * @param srcFile  源文件
     * @param destFile 目标文件
     * @throws IOException IO异常
     */
    public static void copyByBufferedInOutStream(File srcFile, File destFile) throws IOException {
        
        byte[] bytes = new byte[24 * 1024];
        try (InputStream in = new BufferedInputStream(new FileInputStream(srcFile));
             OutputStream out = new BufferedOutputStream(new FileOutputStream(destFile))) {
        
            int count;
            while ((count = in.read(bytes)) > 0) {
        
                out.write(bytes, 0, count);
            }
        }
    }

### 1.4使用nio FileChannel + ByteBuffer ###

/**
     * 文件复制 使用nio FileChannel + ByteBuffer
     *
     * @param srcFile  源文件
     * @param destFile 目标文件
     * @throws IOException IO异常
     */
    public static void copyByNioByteBuffer(File srcFile, File destFile) throws IOException {
        
        ByteBuffer buffer = ByteBuffer.allocate(24 * 1024);
        try (FileChannel inChannel = new FileInputStream(srcFile).getChannel();
             FileChannel outChannel = new FileOutputStream(destFile).getChannel()) {
        
            while (inChannel.read(buffer) != -1) {
        
                buffer.flip();
                while (buffer.hasRemaining()) {
        
                    outChannel.write(buffer);
                }
                buffer.clear();
            }
        }
    }

知识补充:

缓冲区(Buffer)就是在内存中预留指定大小的存储空间用来对输入/输出(I/O)的数据作临时存储，ByteBuffer是缓冲区的一种

缓冲区都有4个属性：capacity、limit、position、mark

<table> 
 <thead> 
  <tr> 
   <th align="left">属性</th> 
   <th align="left">描述</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <td align="left">capacity</td> 
   <td align="left">容量，即可以容纳的最大数据量；在缓冲区创建时被设定并且不能改变</td> 
  </tr> 
  <tr> 
   <td align="left">limit</td> 
   <td align="left">表示缓冲区的当前终点，不能对缓冲区超过极限的位置进行读写操作。且极限是可以修改的</td> 
  </tr> 
  <tr> 
   <td align="left">position</td> 
   <td align="left">位置，下一个要被读或写的元素位置索引，每次读写缓冲区数据时都会改变改值，为下次读写作准备</td> 
  </tr> 
  <tr> 
   <td align="left">mark</td> 
   <td align="left">标记，调用mark()来设置mark=position，再调用reset()可以让position恢复到标记的位置</td> 
  </tr> 
 </tbody> 
</table>

ByteBuffer的方法

`buffer.flip()` 翻转 , 执行后将属性 limit = position;position = 0;mark = -1;position和limit之间即为可以读取到的有效数据，总的作用是将一个处于存数据状态的缓冲区变为一个处于准备取数据的状态的缓冲区；

`buffer.clear()` 属性初始, 执行后将属性position = 0;limit = capacity;mark = -1; 初始化, 但是并不影响底层byte数组的内容；

### 1.5使用nio FileChannel + transferTo (超过2g的文件需要分批) ###

/**
     * 文件复制 使用nio FileChannel + transferTo (超过2g的文件需要分批)
     *
     * @param srcFile  源文件
     * @param destFile 目标文件
     * @throws IOException IO异常
     */
    public static void copyByNioTransferTo(File srcFile, File destFile) throws IOException {
        
        try (FileChannel inChannel = new FileInputStream(srcFile).getChannel();
             FileChannel outChannel = new FileOutputStream(destFile).getChannel()) {
        
            long size = inChannel.size();
            long pos = 0;
            long count;
            while (pos < size) {
        
                count = Math.min(size - pos, 2 * 1024 * 1024 * 1024L - 1);
    //                pos += outChannel.transferFrom(inChannel, pos, count);
                pos += inChannel.transferTo(pos, count, outChannel);
            }
        }
    }

### 1.6.使用common-io 中的FileUtils ###

需要在pom中引入commons-io

<dependency>
        <groupId>commons-io</groupId>
        <artifactId>commons-io</artifactId>
        <version>2.4</version>
    </dependency>

/**
     * 文件复制 使用common-io 中的FileUtils
     *
     * @param srcFile  源文件
     * @param destFile 目标文件
     * @throws IOException IO异常
     */
    public static void copyByCommonIoFileUtils(File srcFile, File destFile) throws IOException {
        
        FileUtils.copyFile(srcFile, destFile);
    }

## 2.完整代码以及测试 ##

### 2.1工具类代码 ###

/**
     * File的工具类
     */
    public final class FileUtil {
        
        /**
         * 缓存byte数组的默认的大小
         */
        private static final int DEFAULT_BUFFER_BYTES_LENGTH = 24 * 1024;
    
        /**
         * transferTo 或者 transferFrom 最大的限制 2g - 1
         */
        private static final long CHANNEL_TRANSFER_BUFFER_SIZE = 2 * 1024 * 1024 * 1024L - 1;
    
        /**
         * 单位
         */
        private static final String[] UNITS = new String[]{
        "B", "KB", "MB", "GB", "TB", "PB", "EB", "ZB", "YB", "BB"};
    
        /**
         * 点号
         */
        private static final String POINT = ".";
    
        private FileUtil() {
        
        }
    
        /**
         * 字节 转换 最大单位
         *
         * @param bytesCount 文件大小 默认是字节
         * @return String
         */
        public static String convertUnit(long bytesCount) {
        
            int index = 0;
            long k = bytesCount;
            if (bytesCount >= 1024) {
        
                do {
        
                    k = k / 1024;
                    index++;
                } while (!(k < 1024));
            }
            return k + UNITS[index];
        }
    
        /**
         * 检测文件参数是否满足复制的条件
         *
         * @param srcFile  源文件
         * @param destFile 目标文件
         * @throws FileNotFoundException fileNotFoundException
         */
        private static void requireCanCopyFile(File srcFile, File destFile) throws FileNotFoundException {
        
            Objects.requireNonNull(srcFile, "the params 'srcFile' is must not null");
            if (!srcFile.exists()) {
        
                throw new FileNotFoundException("the 'srcFile' is not found");
            }
            Objects.requireNonNull(destFile, "the params 'destFile' is must not null");
        }
    
        /**
         * 文件复制 使用java文件工具类Files
         *
         * @param srcFile  源文件
         * @param destFile 目标文件
         * @throws IOException IO异常
         */
        public static void copyByJavaFiles(File srcFile, File destFile) throws IOException {
        
            requireCanCopyFile(srcFile, destFile);
            Files.copy(srcFile.toPath(), new BufferedOutputStream(new FileOutputStream(destFile)));
        }
    
        /**
         * 文件复制 使用FileInputStream , FileOutputStream
         *
         * @param srcFile  源文件
         * @param destFile 目标文件
         * @throws IOException IO异常
         */
        public static void copyByInOutStream(File srcFile, File destFile) throws IOException {
        
            requireCanCopyFile(srcFile, destFile);
            byte[] bytes = new byte[DEFAULT_BUFFER_BYTES_LENGTH];
            try (InputStream in = new FileInputStream(srcFile);
                 OutputStream out = new FileOutputStream(destFile)) {
        
                int count;
                while ((count = in.read(bytes)) > 0) {
        
                    out.write(bytes, 0, count);
                }
            }
        }
    
        /**
         * 文件复制 使用BufferedInputStream , BufferedOutputStream
         *
         * @param srcFile  源文件
         * @param destFile 目标文件
         * @throws IOException IO异常
         */
        public static void copyByBufferedInOutStream(File srcFile, File destFile) throws IOException {
        
            requireCanCopyFile(srcFile, destFile);
            byte[] bytes = new byte[DEFAULT_BUFFER_BYTES_LENGTH];
            try (InputStream in = new BufferedInputStream(new FileInputStream(srcFile));
                 OutputStream out = new BufferedOutputStream(new FileOutputStream(destFile))) {
        
                int count;
                while ((count = in.read(bytes)) > 0) {
        
                    out.write(bytes, 0, count);
                }
            }
        }
    
        /**
         * 文件复制 使用nio FileChannel + ByteBuffer
         *
         * @param srcFile  源文件
         * @param destFile 目标文件
         * @throws IOException IO异常
         */
        public static void copyByNioByteBuffer(File srcFile, File destFile) throws IOException {
        
            requireCanCopyFile(srcFile, destFile);
            ByteBuffer buffer = ByteBuffer.allocate(DEFAULT_BUFFER_BYTES_LENGTH);
            try (FileChannel inChannel = new FileInputStream(srcFile).getChannel();
                 FileChannel outChannel = new FileOutputStream(destFile).getChannel()) {
        
                while (inChannel.read(buffer) != -1) {
        
                    buffer.flip();
                    while (buffer.hasRemaining()) {
        
                        outChannel.write(buffer);
                    }
                    buffer.clear();
                }
            }
        }
    
        /**
         * 文件复制 使用nio FileChannel + transferTo (超过2g的文件需要分批)
         *
         * @param srcFile  源文件
         * @param destFile 目标文件
         * @throws IOException IO异常
         */
        public static void copyByNioTransferTo(File srcFile, File destFile) throws IOException {
        
            requireCanCopyFile(srcFile, destFile);
            try (FileChannel inChannel = new FileInputStream(srcFile).getChannel();
                 FileChannel outChannel = new FileOutputStream(destFile).getChannel()) {
        
                long size = inChannel.size();
                long pos = 0;
                long count;
                while (pos < size) {
        
                    count = Math.min(size - pos, CHANNEL_TRANSFER_BUFFER_SIZE);
    //                pos += outChannel.transferFrom(inChannel, pos, count);
                    pos += inChannel.transferTo(pos, count, outChannel);
                }
            }
        }
    
        /**
         * 文件复制 使用common-io 中的FileUtils
         *
         * @param srcFile  源文件
         * @param destFile 目标文件
         * @throws IOException IO异常
         */
        public static void copyByCommonIoFileUtils(File srcFile, File destFile) throws IOException {
        
            FileUtils.copyFile(srcFile, destFile);
        }
    }

### 2.2测试类代码 ###

/**
     * 文件工具类测试
     */
    public class FileUtilTest {
        
        private static final Logger slf4jLog = LoggerFactory.getLogger(FileUtilTest.class);
    
        public static void testCopy(File sourceFile) {
        
            try {
        
                // 测试使用FileInputStream ,FileOutputStream
                long startTime1 = System.currentTimeMillis();
                File destFile = new File(sourceFile.getCanonicalPath().replaceAll("(\\.[A-Za-z]+)$",
                        "copyByInOutStream-$1"));
                FileUtil.copyByInOutStream(sourceFile, destFile);
                slf4jLog.debug("copyByInOutStream() take time : " + (System.currentTimeMillis() - startTime1) + " ms");
            } catch (IOException e) {
        
                slf4jLog.error(e.getMessage());
                e.printStackTrace();
            }
    
            try {
        
                // 测试使用BufferedInputStream , BufferedOutputStream
                long startTime2 = System.currentTimeMillis();
                File destFile = new File(sourceFile.getCanonicalPath().replaceAll("(\\.[A-Za-z]+)$",
                        "copyByBufferedInOutStream-$1"));
                FileUtil.copyByBufferedInOutStream(sourceFile, destFile);
                slf4jLog.debug("copyByBufferedInOutStream() take time : " + (System.currentTimeMillis() - startTime2) + " ms");
            } catch (IOException e) {
        
                slf4jLog.error(e.getMessage());
                e.printStackTrace();
            }
    
            try {
        
                // 测试使用java工具类Files
                long startTime3 = System.currentTimeMillis();
                File destFile = new File(sourceFile.getCanonicalPath().replaceAll("(\\.[A-Za-z]+)$",
                        "copyByJavaFiles-$1"));
                FileUtil.copyByJavaFiles(sourceFile, destFile);
                slf4jLog.debug("copyByJavaFiles() take time : " + (System.currentTimeMillis() - startTime3) + " ms");
            } catch (IOException e) {
        
                slf4jLog.error(e.getMessage());
                e.printStackTrace();
            }
    
            try {
        
                // 测试使用nio FileChannel + ByteBuffer
                long startTime4 = System.currentTimeMillis();
                File destFile = new File(sourceFile.getCanonicalPath().replaceAll("(\\.[A-Za-z]+)$",
                        "copyByNioByteBuffer-$1"));
                FileUtil.copyByNioByteBuffer(sourceFile, destFile);
                slf4jLog.debug("copyByNioByteBuffer() take time : " + (System.currentTimeMillis() - startTime4) + " ms");
            } catch (IOException e) {
        
                slf4jLog.error(e.getMessage());
                e.printStackTrace();
            }
    
            try {
        
                // 测试使用nio FileChannel + TransferTo()/transferFrom() ps:注意大于2g的文件不能用这种方式
                long startTime5 = System.currentTimeMillis();
                File destFile = new File(sourceFile.getCanonicalPath().replaceAll("(\\.[A-Za-z]+)$",
                        "copyByNioTransferTo-$1"));
                FileUtil.copyByNioTransferTo(sourceFile, destFile);
                slf4jLog.debug("copyByNioTransferTo() take time : " + (System.currentTimeMillis() - startTime5) + " ms");
            } catch (IOException e) {
        
                slf4jLog.error(e.getMessage());
                e.printStackTrace();
            }
    
            try {
        
                // 测试使用common-io FileUtils.copyFile()工具包
                long startTime6 = System.currentTimeMillis();
                File destFile = new File(sourceFile.getCanonicalPath().replaceAll("(\\.[A-Za-z]+)$",
                        "copyByCommonIoFileUtils-$1"));
                FileUtil.copyByCommonIoFileUtils(sourceFile, destFile);
                slf4jLog.debug("copyByCommonIoFileUtils() take time : " + (System.currentTimeMillis() - startTime6) + " ms");
            } catch (IOException e) {
        
                slf4jLog.error(e.getMessage());
                e.printStackTrace();
            }
        }
    
        public static void main(String[] args) throws IOException {
        
            // 创建一些测试文件
    //        Runtime.getRuntime().exec("fsutil file createnew D:\\download\\测试文件读写\\0-200MB.doc " + 200 * 1024* 1024L);
    //        Runtime.getRuntime().exec("fsutil file createnew D:\\download\\测试文件读写\\1-400MB.doc " + 400 * 1024* 1024L);
    //        Runtime.getRuntime().exec("fsutil file createnew D:\\download\\测试文件读写\\3-600MB.doc " + 600 * 1024* 1024L);
    //        Runtime.getRuntime().exec("fsutil file createnew D:\\download\\测试文件读写\\4-800MB.doc " + 800 * 1024* 1024L);
    //
    //        Runtime.getRuntime().exec("fsutil file createnew D:\\download\\测试文件读写\\5-2GB.doc " + 2 * 1024 * 1024* 1024L);
    //        Runtime.getRuntime().exec("fsutil file createnew D:\\download\\测试文件读写\\6-3GB.doc " + 3 * 1024 * 1024 * 1024L);
    //        Runtime.getRuntime().exec("fsutil file createnew D:\\download\\测试文件读写\\7-10GB.doc " + 10 * 1024 * 1024 * 1024L);
    //
            slf4jLog.debug("testCopy start...");
            File dir = new File("D:\\download\\测试文件读写\\");
            for (File file : Objects.requireNonNull(dir.listFiles())) {
        
                slf4jLog.debug("=======================================分割线==============================================");
                slf4jLog.debug("file size : " + FileUtil.convertUnit(file.length()));
                testCopy(file);
            }
        }
    }

### 2.3测试结果 ###

测试`byte[] bytes = new byte[1 * 1024]`时的测试结果

testCopy start...
    =======================================分割线==============================================
    file size : 200MB
    copyByInOutStream() take time : 4182 ms
    copyByBufferedInOutStream() take time : 693 ms
    copyByJavaFiles() take time : 720 ms
    copyByNioByteBuffer() take time : 4013 ms
    copyByNioTransferTo() take time : 501 ms
    copyByCommonIoFileUtils() take time : 1979 ms
    =======================================分割线==============================================
    file size : 400MB
    copyByInOutStream() take time : 8243 ms
    copyByBufferedInOutStream() take time : 1374 ms
    copyByJavaFiles() take time : 1364 ms
    copyByNioByteBuffer() take time : 8167 ms
    copyByNioTransferTo() take time : 537 ms
    copyByCommonIoFileUtils() take time : 398 ms
    =======================================分割线==============================================
    file size : 600MB
    copyByInOutStream() take time : 12087 ms
    copyByBufferedInOutStream() take time : 2084 ms
    copyByJavaFiles() take time : 1992 ms
    copyByNioByteBuffer() take time : 11969 ms
    copyByNioTransferTo() take time : 851 ms
    copyByCommonIoFileUtils() take time : 549 ms
    =======================================分割线==============================================
    file size : 800MB
    copyByInOutStream() take time : 16175 ms
    copyByBufferedInOutStream() take time : 2822 ms
    copyByJavaFiles() take time : 2671 ms
    copyByNioByteBuffer() take time : 15952 ms
    copyByNioTransferTo() take time : 1110 ms
    copyByCommonIoFileUtils() take time : 594 ms
    =======================================分割线==============================================
    file size : 2GB
    copyByInOutStream() take time : 41318 ms
    copyByBufferedInOutStream() take time : 6771 ms
    copyByJavaFiles() take time : 6597 ms
    copyByNioByteBuffer() take time : 40908 ms
    copyByNioTransferTo() take time : 2165 ms
    copyByCommonIoFileUtils() take time : 2619 ms
    =======================================分割线==============================================
    file size : 3GB
    copyByInOutStream() take time : 62443 ms
    copyByBufferedInOutStream() take time : 10344 ms
    copyByJavaFiles() take time : 10297 ms
    copyByNioByteBuffer() take time : 63058 ms
    copyByNioTransferTo() take time : 3052 ms
    copyByCommonIoFileUtils() take time : 5407 ms
    
    Process finished with exit code -1

测试`byte[] bytes = new byte[24 * 1024]`时的测试结果

testCopy start...
    =======================================分割线==============================================
    file size : 200MB
    copyByInOutStream() take time : 932 ms
    copyByBufferedInOutStream() take time : 772 ms
    copyByJavaFiles() take time : 731 ms
    copyByNioByteBuffer() take time : 714 ms
    copyByNioTransferTo() take time : 174 ms
    copyByCommonIoFileUtils() take time : 187 ms
    =======================================分割线==============================================
    file size : 400MB
    copyByInOutStream() take time : 1583 ms
    copyByBufferedInOutStream() take time : 1537 ms
    copyByJavaFiles() take time : 1433 ms
    copyByNioByteBuffer() take time : 1356 ms
    copyByNioTransferTo() take time : 346 ms
    copyByCommonIoFileUtils() take time : 376 ms
    =======================================分割线==============================================
    file size : 600MB
    copyByInOutStream() take time : 2020 ms
    copyByBufferedInOutStream() take time : 2116 ms
    copyByJavaFiles() take time : 2054 ms
    copyByNioByteBuffer() take time : 2331 ms
    copyByNioTransferTo() take time : 919 ms
    copyByCommonIoFileUtils() take time : 1137 ms
    =======================================分割线==============================================
    file size : 800MB
    copyByInOutStream() take time : 3465 ms
    copyByBufferedInOutStream() take time : 3328 ms
    copyByJavaFiles() take time : 3159 ms
    copyByNioByteBuffer() take time : 3106 ms
    copyByNioTransferTo() take time : 1268 ms
    copyByCommonIoFileUtils() take time : 1348 ms
    =======================================分割线==============================================
    file size : 2GB
    copyByInOutStream() take time : 8118 ms
    copyByBufferedInOutStream() take time : 7953 ms
    copyByJavaFiles() take time : 7924 ms
    copyByNioByteBuffer() take time : 7900 ms
    copyByNioTransferTo() take time : 3481 ms
    copyByCommonIoFileUtils() take time : 3291 ms
    
    Process finished with exit code 0

大文件10G测试结果

=======================================分割线==============================================
    file size : 10GB
    copyByInOutStream() take time : 16928 ms
    copyByBufferedInOutStream() take time : 17053 ms
    copyByJavaFiles() take time : 34347 ms
    copyByNioByteBuffer() take time : 17218 ms
    copyByNioTransferTo() take time : 17226 ms
    copyByCommonIoFileUtils() take time : 18619 ms

## 3. 总结 ##

1.`bytes`缓存的大小对`copyByInOutStream()`的速度影响很大，选择合适的大小`copyByInOutStream()`的速度可以和`copyByBufferedInOutStream()`基本相当；

2.在文件在一定的大小内（2G左右\[不准确\]，具体的数值需要大量的测试）`copyByNioTransferTo()` 和 `copyByCommonIoFileUtils()`无疑是最快的；

3.文件很大的情况下（例如10G）各种复制方式速度基本趋于一致；