Java GenericObjectPool 对象池化技术--SpringBoot sftp 连接池工具类

Java BasePooledObjectFactory 对象池化技术

通常一个对象创建、销毁非常耗时的时候，我们不会频繁的创建和销毁它，而是考虑复用。复用对象的一种做法就是对象池，将创建好的对象放入池中维护起来，下次再用的时候直接拿池中已经创建好的对象继续用，这就是池化的思想。

Apache Commons Pool是一个对象池的框架，他提供了一整套用于实现对象池化的API。它提供了三种对象池：GenericKeyedObjectPool，SoftReferenceObjectPool和GenericObjectPool，其中GenericObjectPool是我们最常用的对象池，内部实现也最复杂。

GenericObjectPool

GenericObjectPool 是一个通用对象池框架，我们可以借助它实现一个健壮的对象池，UML图如下所示：

GenericObjectPool 实现了ObjectPool接口，而ObjectPool就是对象池的核心接口，它定义了一个对象池应该实现的行为。

public interface ObjectPool<T> extends Closeable {
    /**
     * 从池中借走到一个对象
     */
    T borrowObject() throws Exception, NoSuchElementException, IllegalStateException;
    /**
     * 把对象归还给对象池
     */
    void returnObject(T var1) throws Exception;
    /**
     * 验证对象的有效性
     */
    void invalidateObject(T var1) throws Exception;

    /**
     * 往池中添加一个对象
     */
    void addObject() throws Exception, IllegalStateException, UnsupportedOperationException;
    /**
     * 返回对象池中有多少对象是空闲的，也就是能够被借走的对象的数量。
     */
    int getNumIdle();
    /**
     * 返回对象池中有对象对象是活跃的，也就是已经被借走的，在使用中的对象的数量。
     */
    int getNumActive();
    /**
     * 清理对象池。注意是清理不是清空，该方法要求的是，清理所有空闲对象，释放相关资源。
     */
    void clear() throws Exception, UnsupportedOperationException;
    /**
     * 关闭对象池。这个方法可以达到清空的效果，清理所有对象以及相关资源。
     */
    void close();
}

BasePooledObjectFactory

Java BasePooledObjectFactory 对象池化技术

使用GenericObjectPool只需要创建一个对象工厂类，继承BasePooledObjectFactory并重写它的create()和destroyObject()。
如下文中的：SftpPool.java

public interface PooledObjectFactory<T> {
    /**
     * 创建一个可由池提供服务的实例，并将其封装在由池管理的PooledObject中。
     */
    PooledObject makeObject() throws Exception;

    /**
     *  销毁池不再需要的实例
     */
    void destroyObject(PooledObject var1) throws Exception;

    /**
     * 确保实例可以安全地由池返回
     */
    boolean validateObject(PooledObject var1);

    /**
     * 重新初始化池返回的实例
     */
    void activateObject(PooledObject var1) throws Exception;

    /**
     * 取消初始化要返回到空闲对象池的实例
     */
    void passivateObject(PooledObject var1) throws Exception;
}

配置类GenericObjectPoolConfig

GenericObjectPoolConfig是封装GenericObject池配置的简单“结构”，此类不是线程安全的；它仅用于提供创建池时使用的属性。大多数情况，可以使用GenericObjectPoolConfig提供的默认参数就可以满足日常的需求。

工作原理流程

1. 构造方法
   当我们执行构造方法时，主要工作就是创建了一个存储对象的LinkedList类型容器，也就是概念意义上的“池”
2. 从对象池中获取对象
   获取池中的对象是通过borrowObject()命令，源码比较复杂，简单而言就是去LinkedList中获取一个对象，如果不存在的话，要调用构造方法中第一个参数Factory工厂类的makeObject()方法去创建一个对象再获取，获取到对象后要调用validateObject方法判断该对象是否是可用的，如果是可用的才拿去使用。LinkedList容器减一
3. 归还对象到线程池
   简单而言就是先调用validateObject方法判断该对象是否是可用的，如果可用则归还到池中，LinkedList容器加一，如果是不可以的则调用destroyObject方法进行销毁

上面三步就是最简单的流程，由于取和还的流程步骤都在borrowObject和returnObject方法中固定的，所以我们只要重写Factory工厂类的makeObject()和validateObject以及destroyObject方法即可实现最简单的池的管理控制，通过构造方法传入该Factory工厂类对象则可以创建最简单的对象池管理类。这算是比较好的解耦设计模式，借和还的流程如下图所示：

使用Demo

<dependency>
    <groupId>org.apache.commonsgroupId>
    <artifactId>commons-pool2artifactId>
    <version>2.7.0version>
dependency>


<dependency>
    <groupId>com.jcraftgroupId>
    <artifactId>jschartifactId>
    <version>0.1.55version>
dependency>

点击查看代码

<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <parent>
        <artifactId>vipsoft-parentartifactId>
        <groupId>com.vipsoft.bootgroupId>
        <version>1.0-SNAPSHOTversion>
    parent>
    <modelVersion>4.0.0modelVersion>

    <artifactId>vipsoft-sftpartifactId>
    <version>1.0-SNAPSHOTversion>

    <dependencies>

        <dependency>
            <groupId>org.apache.commonsgroupId>
            <artifactId>commons-pool2artifactId>
            <version>2.7.0version>
        dependency>

        
        <dependency>
            <groupId>com.jcraftgroupId>
            <artifactId>jschartifactId>
            <version>0.1.55version>
        dependency>

        <dependency>
            <groupId>org.eclipse.pahogroupId>
            <artifactId>org.eclipse.paho.client.mqttv3artifactId>
            <version>1.2.5version>
        dependency>


        <dependency>
            <groupId>cn.hutoolgroupId>
            <artifactId>hutool-allartifactId>
            <version>5.3.6version>
        dependency>
        <dependency>
            <groupId>org.springframework.bootgroupId>
            <artifactId>spring-boot-starter-webartifactId>
        dependency>
        <dependency>
            <groupId>org.springframework.bootgroupId>
            <artifactId>spring-boot-starter-actuatorartifactId>
        dependency>
        <dependency>
            <groupId>org.springframework.bootgroupId>
            <artifactId>spring-boot-starter-testartifactId>
            <scope>testscope>
            <exclusions>
                <exclusion>
                    <groupId>org.junit.vintagegroupId>
                    <artifactId>junit-vintage-engineartifactId>
                exclusion>
            exclusions>
        dependency>

    dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.bootgroupId>
                <artifactId>spring-boot-maven-pluginartifactId>
            plugin>
        plugins>
    build>
project>

application.yaml

server:
  port: 8088
  application:
    name: sftp Demo


sftp:
  host: 172.16.3.88 # 服务器ip
  port: 22 # ssh端口
  username: root # 用户名
  password: root # 密码
  # 连接池参数
  pool:
    max-total: 10
    max-idle: 10
    min-idle: 5

SftpPoolException.java

package com.vipsoft.sftp.exception;


/**
 * sftp连接池异常
 */
public class SftpPoolException extends RuntimeException {

    private static final long serialVersionUID = 1L;

    /**
     * Constructs a new runtime exception with {@code null} as its
     * detail message.  The cause is not initialized, and may subsequently be
     * initialized by a call to {@link #initCause}.
     */
    public SftpPoolException() {
    }

    /**
     * Constructs a new runtime exception with the specified detail message.
     * The cause is not initialized, and may subsequently be initialized by a
     * call to {@link #initCause}.
     *
     * @param message the detail message. The detail message is saved for
     *                later retrieval by the {@link #getMessage()} method.
     */
    public SftpPoolException(String message) {
        super(message);
    }

    /**
     * Constructs a new runtime exception with the specified detail message and
     * cause.  
Note that the detail message associated with
     * {@code cause} is not automatically incorporated in
     * this runtime exception's detail message.
     *
     * @param message the detail message (which is saved for later retrieval
     *                by the {@link #getMessage()} method).
     * @param cause   the cause (which is saved for later retrieval by the
     *                {@link #getCause()} method).  (A null value is
     *                permitted, and indicates that the cause is nonexistent or
     *                unknown.)
     * @since 1.4
     */
    public SftpPoolException(String message, Throwable cause) {
        super(message, cause);
    }

    /**
     * Constructs a new runtime exception with the specified cause and a
     * detail message of (cause==null ? null : cause.toString())
     * (which typically contains the class and detail message of
     * cause).  This constructor is useful for runtime exceptions
     * that are little more than wrappers for other throwables.
     *
     * @param cause the cause (which is saved for later retrieval by the
     *              {@link #getCause()} method).  (A null value is
     *              permitted, and indicates that the cause is nonexistent or
     *              unknown.)
     * @since 1.4
     */
    public SftpPoolException(Throwable cause) {
        super(cause);
    }

    /**
     * Constructs a new runtime exception with the specified detail
     * message, cause, suppression enabled or disabled, and writable
     * stack trace enabled or disabled.
     *
     * @param message            the detail message.
     * @param cause              the cause.  (A {@code null} value is permitted,
     *                           and indicates that the cause is nonexistent or unknown.)
     * @param enableSuppression  whether or not suppression is enabled
     *                           or disabled
     * @param writableStackTrace whether or not the stack trace should
     *                           be writable
     * @since 1.7
     */
    public SftpPoolException(String message, Throwable cause, boolean enableSuppression, boolean writableStackTrace) {
        super(message, cause, enableSuppression, writableStackTrace);
    }

}

config

SftpConfig.java

package com.vipsoft.sftp.config;

import com.vipsoft.sftp.pool.SftpFactory;
import com.vipsoft.sftp.pool.SftpPool;
import com.vipsoft.sftp.utils.SftpUtil;
import org.springframework.boot.context.properties.EnableConfigurationProperties;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
@EnableConfigurationProperties(SftpProperties.class)
public class SftpConfig {
    // 工厂
    @Bean
    public SftpFactory sftpFactory(SftpProperties properties) {
        return new SftpFactory(properties);
    }

    // 连接池
    @Bean
    public SftpPool sftpPool(SftpFactory sftpFactory) {
        return new SftpPool(sftpFactory);
    }

    // 辅助类
    @Bean
    public SftpUtil sftpUtil(SftpPool sftpPool) {
        return new SftpUtil(sftpPool);
    }
}

SftpProperties.java

package com.vipsoft.sftp.config;

import com.jcraft.jsch.ChannelSftp;
import org.apache.commons.pool2.impl.GenericObjectPoolConfig;
import org.springframework.boot.context.properties.ConfigurationProperties;

@ConfigurationProperties(prefix = "sftp")
public class SftpProperties {

    private String host;
    private int port = 22;
    private String username = "root";
    private String password = "root";
    private Pool pool = new Pool();

    public String getHost() {
        return host;
    }

    public void setHost(String host) {
        this.host = host;
    }

    public int getPort() {
        return port;
    }

    public void setPort(int port) {
        this.port = port;
    }

    public String getUsername() {
        return username;
    }

    public void setUsername(String username) {
        this.username = username;
    }

    public String getPassword() {
        return password;
    }

    public void setPassword(String password) {
        this.password = password;
    }

    public Pool getPool() {
        return pool;
    }

    public void setPool(Pool pool) {
        this.pool = pool;
    }

    public static class Pool extends GenericObjectPoolConfig<ChannelSftp> {

        private int maxTotal = DEFAULT_MAX_TOTAL;
        private int maxIdle = DEFAULT_MAX_IDLE;
        private int minIdle = DEFAULT_MIN_IDLE;

        public Pool() {
            super();
        }
        @Override
        public int getMaxTotal() {
            return maxTotal;
        }
        @Override
        public void setMaxTotal(int maxTotal) {
            this.maxTotal = maxTotal;
        }
        @Override
        public int getMaxIdle() {
            return maxIdle;
        }
        @Override
        public void setMaxIdle(int maxIdle) {
            this.maxIdle = maxIdle;
        }
        @Override
        public int getMinIdle() {
            return minIdle;
        }
        @Override
        public void setMinIdle(int minIdle) {
            this.minIdle = minIdle;
        }
    }
}

Pool

SftpFactory.java

package com.vipsoft.sftp.pool;

import com.jcraft.jsch.ChannelSftp;
import com.jcraft.jsch.JSch;
import com.jcraft.jsch.JSchException;
import com.jcraft.jsch.Session;
import com.vipsoft.sftp.config.SftpProperties;
import com.vipsoft.sftp.exception.SftpPoolException;
import org.apache.commons.pool2.BasePooledObjectFactory;
import org.apache.commons.pool2.PooledObject;
import org.apache.commons.pool2.impl.DefaultPooledObject;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import java.util.Properties;

public class SftpFactory extends BasePooledObjectFactory {

    private  final Logger logger = LoggerFactory.getLogger(this.getClass());

    private SftpProperties properties;

    public SftpProperties getProperties() {
        return properties;
    }

    public void setProperties(SftpProperties properties) {
        this.properties = properties;
    }

    public SftpFactory(SftpProperties properties) {
        this.properties = properties;
    }

    @Override
    public ChannelSftp create() {
        try {
            JSch jsch = new JSch();
            Session sshSession = jsch.getSession(properties.getUsername(), properties.getHost(), properties.getPort());
            sshSession.setPassword(properties.getPassword());
            Properties sshConfig = new Properties();
            sshConfig.put("StrictHostKeyChecking", "no");
            sshSession.setConfig(sshConfig);
            sshSession.connect();
            ChannelSftp channel = (ChannelSftp) sshSession.openChannel("sftp");
            channel.connect();
            return channel;
        } catch (JSchException e) {
            throw new SftpPoolException("连接sfpt失败", e);
        }
    }

    @Override
    public PooledObject wrap(ChannelSftp channelSftp) {
        return new DefaultPooledObject<>(channelSftp);
    }

    // 销毁对象
    @Override
    public void destroyObject(PooledObject p) {
        ChannelSftp channelSftp = p.getObject();
        channelSftp.disconnect();
    }

}

SftpPool.java

package com.vipsoft.sftp.pool;

import com.jcraft.jsch.ChannelSftp;
import org.apache.commons.pool2.impl.GenericObjectPool;

public class SftpPool extends GenericObjectPool {

    public SftpPool(SftpFactory factory) {
        super(factory,factory.getProperties().getPool());
    }

    /**
     * 获取一个sftp连接对象
     * @return sftp连接对象
     */
    @Override
    public ChannelSftp borrowObject() throws Exception {
        return super.borrowObject();
    }

    /**
     * 归还一个sftp连接对象
     * @param channelSftp sftp连接对象
     */
    @Override
    public void returnObject(ChannelSftp channelSftp) {
        if (channelSftp!=null) {
            super.returnObject(channelSftp);
        }
    }

}

Utils

ByteUtil.java

package com.vipsoft.sftp.utils;

import com.jcraft.jsch.ChannelSftp;
import com.jcraft.jsch.SftpException;
import com.vipsoft.sftp.exception.SftpPoolException;
import com.vipsoft.sftp.pool.SftpPool;

import java.io.InputStream;

public class SftpUtil {

    private SftpPool pool;

    public SftpUtil(SftpPool pool) {
        this.pool = pool;
    }

    /**
     * 下载文件
     *
     * @param dir  远程目录
     * @param name 远程文件名
     * @return 文件字节数组
     */
    public byte[] download(String dir, String name) {
        ChannelSftp sftp = null;
        try {
            sftp = pool.borrowObject();
            sftp.cd(dir);
            InputStream in = sftp.get(name);
            return ByteUtil.inputStreamToByteArray(in);
        } catch (Exception e) {
            throw new SftpPoolException("sftp下载文件出错", e);
        } finally {
            pool.returnObject(sftp);
        }
    }

    /**
     * 上传文件
     *
     * @param dir  远程目录
     * @param name 远程文件名
     * @param in   输入流
     */
    public void upload(String dir, String name, InputStream in) {
        ChannelSftp sftp = null;
        try {
            sftp = pool.borrowObject();
            mkdirs(sftp, dir);
            sftp.cd(dir);
            sftp.put(in, name);
        } catch (Exception e) {
            throw new SftpPoolException("sftp上传文件出错", e);
        } finally {
            pool.returnObject(sftp);
        }
    }

    /**
     * 删除文件
     *
     * @param dir  远程目录
     * @param name 远程文件名
     */
    public void delete(String dir, String name) {
        ChannelSftp sftp = null;
        try {
            sftp = pool.borrowObject();
            sftp.cd(dir);
            sftp.rm(name);
        } catch (Exception e) {
            throw new SftpPoolException("sftp删除文件出错", e);
        } finally {
            pool.returnObject(sftp);
        }
    }

    /**
     * 递归创建多级目录
     *
     * @param dir 多级目录
     */
    private void mkdirs(ChannelSftp sftp, String dir) {
        String[] folders = dir.split("/");
        try {
            sftp.cd("/");
            for (String folder : folders) {
                if (folder.length() > 0) {
                    try {
                        sftp.cd(folder);
                    } catch (Exception e) {
                        sftp.mkdir(folder);
                        sftp.cd(folder);
                    }
                }
            }
        } catch (SftpException e) {
            throw new SftpPoolException("sftp创建目录出错", e);
        }
    }

}

Test

SftpTest.java

package com.vipsoft.sftp;

import com.vipsoft.sftp.utils.SftpUtil;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;

@SpringBootTest
public class SftpTest {

    @Autowired
    private SftpUtil sftpUtil;

    @Test
    void downloadTest() {
        byte[] dockerfiles = sftpUtil.download("/opt/demo/", "Dockerfile");
        System.out.println("FileSize =>" + dockerfiles.length);
    }

}