封装了支持S3协议的文件服务器(如minio、阿里云OSS、腾讯云)的相关操作，支持分片上传、断点续传、不经过后端服务上传

包含的主要功能和示例

• 创建bucket
• 删除bucket
• 文件上传
• 拷贝文件
• 删除文件
• 文件下载
• 设置文件标签
• 上传文件指定时间自动删除
• 上传文件并加密
• 分片上传
• 断点续传
• 生成预签名url，直接前端上传不经过后端

源码地址

源码地址

使用demo地址

demo地址

前置测试环境

首先使用docker-compose安装了最新的minio用于测试

version: '3'

  minio:
    image: minio/minio
    container_name: minio
    restart: always
    ports:
      - "9000:9000" # api端口
      - "9001:9001" # 控制台端口
    environment:
      MINIO_ROOT_USER: "miniouser"      # 设置你的访问账户(用于控制台访问)
      MINIO_ROOT_PASSWORD: "123456789"  # 设置你的访问密钥(用于控制台访问)
    volumes:
      - /mnt/minio/files:/data                   # 将容器中的 /data 文件夹挂载到主机上的指定文件夹 我使用的是/mnt/minio/files
    command: server /data --console-address ":9001"

项目引入依赖

这里说明一下cn.allbs这个group用于jdk1.8，com.alltobs用于jdk17+，但是cn.allbs中有一点点jdk17，遇到的话降级点版本。

<dependency>
  <groupId>com.alltobs</groupId>
  <artifactId>alltobs-oss</artifactId>
  <version>1.0.0</version>
</dependency>

项目配置

配置文件

oss:
  endpoint: http://xxx.xxx.xxx.xxx:9000
  # 所在地区
  region: cn-north-1
  # minio账号或者
  access-key: adadmin
  # 密码
  secret-key: 123456778
  # 设置一个默认的文件桶，比如不同项目使用同一个文件库，以项目为文件桶分隔
  bucket-name: test
  # 设置会过期的子文件桶
  expiring-buckets:
    temp-bucket-1: 30  # 生命周期30天
    temp-bucket-2: 60  # 生命周期60天

• endpoint就是安装minio或者腾讯云、阿里云之类的对象储存地址
• region所在地区
• access-key可以直接使用控制台账号，但是建议在控制台中生成账号和密钥
• secret-key可以直接使用控制台密钥，但是建议在控制台中生成账号和密钥
• bucket-name 设置一个默认的文件桶，比如不同项目使用同一个文件库，以项目为文件桶分隔
• expiring-buckets 里面设置的是包含生命周期的文件夹，创建位置位于bucket-name下，比如bucket-name叫test,那么会在test目录下创建一个生命周期为30天的文件夹temp-bucket-1，生命周期为60天的文件夹temp-bucket-2

启动类注解@EnableAllbsOss

使用引入

@Resource
private OssTemplate ossTemplate;

使用演示

以下所有方法都遵循一个原则，当bucket-name不为空时，所有操作都在这个bucket下进行。

自动创建主目录和带过期时间的子目录

就会自动创建文件桶如下，一个位于test目录下且十天后会自动删除的expire-bucket-1

创建bucket

@PostMapping("/createBucket")  
public ResponseEntity<String> createBucket(@RequestParam String bucketName) {  
    ossTemplate.createBucket(bucketName);  
    return ResponseEntity.ok("Bucket created: " + bucketName);  
}

查询所有bucket

@GetMapping("/getAllBuckets")  
public R<List<String>> getAllBuckets() {  
    return R.ok(ossTemplate.getAllBuckets());  
}

删除指定bucket

@DeleteMapping("/removeBucket")  
public R<String> removeBucket(@RequestParam String bucketName) {  
    ossTemplate.removeBucket(bucketName);  
    return R.ok("Bucket removed: " + bucketName);  
}

上传文件

@PostMapping("/putObject")  
public R<String> putObject(@RequestParam String bucketName, @RequestParam MultipartFile file) {  
    String fileName = file.getOriginalFilename();  
    try {  
        String uuid = UUID.randomUUID() + "." + FileUtil.getFileType(fileName);  
        ossTemplate.putObject(bucketName, uuid, file.getInputStream());  
        return R.ok("File uploaded: " + uuid);  
    } catch (IOException e) {  
        return R.fail("Failed to upload file: " + fileName);  
    }  
}

查询指定目录下指定前缀的文件

@GetMapping("/getAllObjectsByPrefix")  
public R<Set<String>> getAllObjectsByPrefix(@RequestParam String bucketName, @RequestParam String prefix) {   
    return R.ok(ossTemplate.getAllObjectsByPrefix(bucketName, prefix).stream().map(S3Object::key).collect(Collectors.toSet()));  
}

查看文件(字节数组)

@GetMapping("/getObject")  
public R<byte[]> getObject(@RequestParam String bucketName, @RequestParam String objectName) {  
    try (var s3Object = ossTemplate.getObject(bucketName, objectName)) {  
        return R.ok(s3Object.readAllBytes());  
    } catch (IOException e) {  
        return R.fail(e.getLocalizedMessage());  
    }  
}

下载文件

@GetMapping("/download")  
public void download(@RequestParam String bucketName, @RequestParam String objectName, HttpServletResponse response) {  
    try (ResponseInputStream<GetObjectResponse> inputStream = ossTemplate.getObject(bucketName, objectName);  
         OutputStream outputStream = response.getOutputStream()) {  
  
        // 获取文件的Content-Type  
        String contentType = inputStream.response().contentType();  
        response.setContentType(contentType);  
  
        // 设置响应头：Content-Disposition，用于浏览器下载文件时的文件名  
        response.setHeader("Content-Disposition", "attachment; filename=\"" + objectName + "\"");  
  
        // 直接将输入流中的数据传输到输出流  
        inputStream.transferTo(outputStream);  
  
        // 刷新输出流，确保所有数据已写出  
        outputStream.flush();  
  
    } catch (IOException e) {  
        // 如果出错，设置响应状态为404  
        response.setStatus(HttpServletResponse.SC_NOT_FOUND);  
    }  
}

查看文件链接带过期时间

@GetMapping("/getObjectURL")  
public R<String> getObjectURL(@RequestParam String bucketName, @RequestParam String objectName, @RequestParam int minutes) {  
    return R.ok(ossTemplate.getObjectURL(bucketName, objectName, minutes));  
}

删除文件

@DeleteMapping("/removeObject")  
public R<String> removeObject(@RequestParam String bucketName, @RequestParam String objectName) {  
    ossTemplate.removeObject(bucketName, objectName);  
    return R.ok("Object removed: " + objectName);  
}

文件复制

@PostMapping("/copyObject")  
public R<String> copyObject(@RequestParam String sourceBucketName, @RequestParam String sourceKey,  
                            @RequestParam String destinationBucketName, @RequestParam String destinationKey) {  
    ossTemplate.copyObject(sourceBucketName, sourceKey, destinationBucketName, destinationKey);  
    return R.ok("Object copied from " + sourceKey + " to " + destinationKey);  
}

查询指定文件的访问权限

@GetMapping("/getObjectAcl")  
public R<String> getObjectAcl(@RequestParam String bucketName, @RequestParam String objectName) {  
    return R.ok(ossTemplate.getObjectAcl(bucketName, objectName).toString());  
}

设置指定文件的访问权限

@PostMapping("/setObjectAcl")  
public R<String> setObjectAcl(@RequestParam String bucketName, @RequestParam String objectName, @RequestParam String acl) {  
    ossTemplate.setObjectAcl(bucketName, objectName, acl);  
    return R.ok("ACL set for object: " + objectName);  
}

启用或者关闭指定bucket版本控制

@PostMapping("/setBucketVersioning")  
public R<String> setBucketVersioning(@RequestParam String bucketName, @RequestParam boolean enable) {  
    ossTemplate.setBucketVersioning(bucketName, enable);  
    return R.ok("Versioning set to " + (enable ? "Enabled" : "Suspended") + " for bucket: " + bucketName);  
}

给指定文件打标签

@PostMapping("/setObjectTags")  
public R<String> setObjectTags(@RequestParam String bucketName, @RequestParam String objectName, @RequestBody Map<String, String> tags) {  
    ossTemplate.setObjectTags(bucketName, objectName, tags);  
    return R.ok("Tags set for object: " + objectName);  
}

获取指定文件的标签

@GetMapping("/getObjectTags")  
public R<Map<String, String>> getObjectTags(@RequestParam String bucketName, @RequestParam String objectName) {  
    return R.ok(ossTemplate.getObjectTags(bucketName, objectName));  
}

上传一个会定时删除的文件

@PostMapping("putObjectWithExpiration")  
public R<String> putObjectWithExpiration(@RequestParam String bucketName, @RequestParam MultipartFile file, @RequestParam int days) {  
    String fileName = file.getOriginalFilename();  
    try {  
        String uuid = UUID.randomUUID() + "." + FileUtil.getFileType(fileName);  
        ossTemplate.putObjectWithExpiration(bucketName, uuid, file.getInputStream(), days);  
        return R.ok("File uploaded: " + uuid);  
    } catch (IOException e) {  
        return R.fail("Failed to upload file: " + fileName);  
    }  
}

上传加密文件

需要在服务端配置KMS，这里使用的是默认的AES256，其他可以调putObjectWithEncryption方法

@PostMapping("/uploadWithEncryption")  
public R<String> uploadWithEncryption(@RequestParam String bucketName,  
                                      @RequestParam("file") MultipartFile file) {  
    String uuid = UUID.randomUUID() + "." + FileUtil.getFileType(file.getOriginalFilename());  
    try (InputStream inputStream = file.getInputStream()) {  
        PutObjectResponse response = ossTemplate.uploadWithEncryption(  
                bucketName,  
                uuid,                inputStream,                file.getSize(),  
                file.getContentType()  
        );  
        return R.ok("File uploaded with encryption: " + response.toString());  
    } catch (IOException e) {  
        return R.fail("File upload failed: " + e.getMessage());  
    }  
}

分片上传

@PostMapping("/uploadMultipart")  
public R<String> uploadMultipart(@RequestParam String bucketName,  
                                 @RequestParam MultipartFile file) throws IOException, InterruptedException {  
    String objectName = file.getOriginalFilename();  
    // 初始化分片上传  
    String uploadId = ossTemplate.initiateMultipartUpload(bucketName, objectName);  
  
    // 将文件按部分大小（5MB）分块上传  
    long partSize = 5 * 1024 * 1024;  
    long fileSize = file.getSize();  
    int partCount = (int) Math.ceil((double) fileSize / partSize);  
  
    // 用于存储已上传的部分  
    List<CompletedPart> completedParts = Collections.synchronizedList(new ArrayList<>());  
  
    // 创建线程池  
    ExecutorService executor = Executors.newFixedThreadPool(Math.min(partCount, 10));  
  
    for (int i = 0; i < partCount; i++) {  
        final int partNumber = i + 1;  
        long startPos = i * partSize;  
        long size = Math.min(partSize, fileSize - startPos);  
  
        executor.submit(() -> {  
            try (InputStream inputStream = file.getInputStream()) {  
                inputStream.skip(startPos);  
                byte[] buffer = new byte[(int) size];  
                int bytesRead = inputStream.read(buffer, 0, (int) size);  
  
                if (bytesRead > 0) {  
                    CompletedPart part = ossTemplate.uploadPart(bucketName, objectName, uploadId, partNumber, buffer);  
                    completedParts.add(part);  
                }  
            } catch (IOException e) {  
                e.printStackTrace();  
            }  
        });  
    }  
  
    executor.shutdown();  
    executor.awaitTermination(Long.MAX_VALUE, TimeUnit.MILLISECONDS);  
  
    // 检查是否成功上传了所有部分  
    if (completedParts.size() == partCount) {  
        // 在完成上传之前，按 partNumber 升序排序  
        completedParts.sort(Comparator.comparing(CompletedPart::partNumber));  
  
        // 完成分片上传  
        ossTemplate.completeMultipartUpload(bucketName, objectName, uploadId, completedParts);  
        return R.ok("Upload completed successfully uploadId: " + uploadId);  
    } else {  
        // 如果有部分上传失败，取消上传  
        ossTemplate.abortMultipartUpload(bucketName, objectName, uploadId);  
        return R.fail("Upload failed, some parts are missing.");  
    }  
}

断点续传

uploadId是上一步分片上传获取到的，可以做个的记录，方便断点续传时使用。我这边测试方法是分片上传过程中直接终止了服务。

@PostMapping("/resumeMultipart")  
public R<String> resumeMultipart(@RequestParam String bucketName,  
                                 @RequestParam MultipartFile file,  
                                 @RequestParam String uploadId) throws IOException, InterruptedException {  
    String objectName = file.getOriginalFilename();  
  
    // 将文件读入内存  
    byte[] fileBytes = file.getBytes();  
  
    // 获取已经上传的部分  
    List<CompletedPart> completedParts = ossTemplate.listParts(bucketName, objectName, uploadId);  
  
    // 继续上传未完成的部分  
    long partSize = 5 * 1024 * 1024;  
    long fileSize = fileBytes.length;  
    int partCount = (int) Math.ceil((double) fileSize / partSize);  
  
    ExecutorService executor = Executors.newFixedThreadPool(Math.min(partCount, 10));  
  
    for (int i = completedParts.size(); i < partCount; i++) {  
        final int partNumber = i + 1;  
        long startPos = i * partSize;  
        long size = Math.min(partSize, fileSize - startPos);  
  
        executor.submit(() -> {  
            try {  
                byte[] buffer = Arrays.copyOfRange(fileBytes, (int) startPos, (int) (startPos + size));  
                CompletedPart part = ossTemplate.uploadPart(bucketName, objectName, uploadId, partNumber, buffer);  
                synchronized (completedParts) {  
                    completedParts.add(part);  
                }  
            } catch (Exception e) {  
                e.printStackTrace();  
            }  
        });  
    }  
  
    executor.shutdown();  
    executor.awaitTermination(Long.MAX_VALUE, TimeUnit.MILLISECONDS);  
  
    // 按 partNumber 升序排序  
    completedParts.sort(Comparator.comparing(CompletedPart::partNumber));  
  
    // 完成分片上传  
    ossTemplate.completeMultipartUpload(bucketName, objectName, uploadId, completedParts);  
  
    return R.ok("Upload resumed and completed successfully");  
}

前端不通过后台服务器使用预签名的表单上传数据

使用这种方式可以让客户端能够直接与 S3 进行交互，减少了服务器的负担，并且可以利用 S3 的上传能力进行大文件的处理。

@GetMapping("/generatePreSignedUrl")  
public R<String> generatePreSignedUrl(@RequestParam String bucketName,  
                                      @RequestParam String objectName,  
                                      @RequestParam int expiration) {  
    String preSignedUrl = ossTemplate.generatePreSignedUrlForPut(bucketName, objectName, expiration);  
    return R.ok(preSignedUrl);  
}

前面的demo

<!DOCTYPE html>  
<html lang="en">  
<head>  
    <meta charset="UTF-8">  
    <meta name="viewport" content="width=device-width, initial-scale=1.0">  
    <title>文件上传</title>  
    <style>        body {  
            font-family: Arial, sans-serif;  
            margin: 20px;  
        }  
  
        .upload-container {  
            max-width: 500px;  
            margin: 0 auto;  
            padding: 20px;  
            border: 2px solid #ccc;  
            border-radius: 10px;  
            text-align: center;  
        }  
  
        .file-input {  
            margin-bottom: 20px;  
        }  
  
        .progress-bar {  
            width: 100%;  
            background-color: #f3f3f3;  
            border-radius: 5px;  
            overflow: hidden;  
            margin-bottom: 10px;  
        }  
  
        .progress {  
            height: 20px;  
            background-color: #4caf50;  
            width: 0;  
        }  
    </style>  
</head>  
<body>  
<div class="upload-container">  
    <h2>上传文件到 S3</h2>  
    <input type="file" id="fileInput" class="file-input"/>  
    <div class="progress-bar">  
        <div class="progress" id="progressBar"></div>  
    </div>  
    <button onclick="uploadFile()">上传文件</button>  
    <p id="statusText"></p>  
</div>  
  
<script>  
    async function uploadFile() {  
        const fileInput = document.getElementById('fileInput');  
        const file = fileInput.files[0];  
  
        if (!file) {  
            alert('请选择一个文件进行上传');  
            return;  
        }  
  
        // 获取预签名 URL        const response = await fetch(`/oss/generatePreSignedUrl?bucketName=myBucket&objectName=${file.name}&expiration=15`);  
        const result = await response.json();  
  
        if (result.code !== 200) {  
            document.getElementById('statusText').innerText = '获取预签名URL失败';  
            return;  
        }  
  
        const presignedUrl = result.data;  // 从返回的 JSON 数据中提取预签名的 URL  
        const xhr = new XMLHttpRequest();  
        xhr.open('PUT', presignedUrl, true);  
        xhr.setRequestHeader('Content-Type', file.type);  
  
        // 更新进度条  
        xhr.upload.onprogress = function(event) {  
            if (event.lengthComputable) {  
                const percentComplete = (event.loaded / event.total) * 100;  
                document.getElementById('progressBar').style.width = percentComplete + '%';  
            }  
        };  
  
        // 处理上传完成后的事件  
        xhr.onload = function() {  
            if (xhr.status === 200) {  
                document.getElementById('statusText').innerText = '文件上传成功！';  
            } else {  
                document.getElementById('statusText').innerText = '文件上传失败，请重试。';  
            }  
        };  
  
        // 错误处理  
        xhr.onerror = function() {  
            document.getElementById('statusText').innerText = '文件上传过程中出现错误。';  
        };  
  
        xhr.send(file);  
    }  
</script>  
</body>  
</html>