聊聊httpclient的evict操作

序

本文主要研究一下httpclient的evict操作

evictExpiredConnections

org/apache/http/impl/client/HttpClientBuilder.java

public class HttpClientBuilder {
	private boolean evictExpiredConnections;

	/**
     * Makes this instance of HttpClient proactively evict expired connections from the
     * connection pool using a background thread.
     * 

     * One MUST explicitly close HttpClient with {@link CloseableHttpClient#close()} in order
     * to stop and release the background thread.
     * 

     * Please note this method has no effect if the instance of HttpClient is configuted to
     * use a shared connection manager.
     * 

     * Please note this method may not be used when the instance of HttpClient is created
     * inside an EJB container.
     *
     * @see #setConnectionManagerShared(boolean)
     * @see org.apache.http.conn.HttpClientConnectionManager#closeExpiredConnections()
     *
     * @since 4.4
     */
    public final HttpClientBuilder evictExpiredConnections() {
        evictExpiredConnections = true;
        return this;
    }
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

HttpClientBuilder提供了evictExpiredConnections方法，该方法会设置evictExpiredConnections为true

evictIdleConnections

public class HttpClientBuilder {

	private boolean evictIdleConnections;
	private long maxIdleTime;
    private TimeUnit maxIdleTimeUnit;

    /**
     * Makes this instance of HttpClient proactively evict idle connections from the
     * connection pool using a background thread.
     * 

     * One MUST explicitly close HttpClient with {@link CloseableHttpClient#close()} in order
     * to stop and release the background thread.
     * 

     * Please note this method has no effect if the instance of HttpClient is configuted to
     * use a shared connection manager.
     * 

     * Please note this method may not be used when the instance of HttpClient is created
     * inside an EJB container.
     *
     * @see #setConnectionManagerShared(boolean)
     * @see org.apache.http.conn.HttpClientConnectionManager#closeExpiredConnections()
     *
     * @param maxIdleTime maximum time persistent connections can stay idle while kept alive
     * in the connection pool. Connections whose inactivity period exceeds this value will
     * get closed and evicted from the pool.
     * @param maxIdleTimeUnit time unit for the above parameter.
     *
     * @since 4.4
     */
    public final HttpClientBuilder evictIdleConnections(final long maxIdleTime, final TimeUnit maxIdleTimeUnit) {
        this.evictIdleConnections = true;
        this.maxIdleTime = maxIdleTime;
        this.maxIdleTimeUnit = maxIdleTimeUnit;
        return this;
    }
}    
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

HttpClientBuilder提供了evictIdleConnections方法，该方法会设置evictIdleConnections为true，同时设置maxIdleTime及maxIdleTimeUnit

build

		if (!this.connManagerShared) {
            if (closeablesCopy == null) {
                closeablesCopy = new ArrayList(1);
            }
            final HttpClientConnectionManager cm = connManagerCopy;

            if (evictExpiredConnections || evictIdleConnections) {
                final IdleConnectionEvictor connectionEvictor = new IdleConnectionEvictor(cm,
                        maxIdleTime > 0 ? maxIdleTime : 10, maxIdleTimeUnit != null ? maxIdleTimeUnit : TimeUnit.SECONDS,
                        maxIdleTime, maxIdleTimeUnit);
                closeablesCopy.add(new Closeable() {

                    @Override
                    public void close() throws IOException {
                        connectionEvictor.shutdown();
                        try {
                            connectionEvictor.awaitTermination(1L, TimeUnit.SECONDS);
                        } catch (final InterruptedException interrupted) {
                            Thread.currentThread().interrupt();
                        }
                    }

                });
                connectionEvictor.start();
            }
            closeablesCopy.add(new Closeable() {

                @Override
                public void close() throws IOException {
                    cm.shutdown();
                }

            });
        }
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34

HttpClientBuilder的build方法会在connManagerShared为false的前提下判断是否开启evictExpiredConnections或者evictIdleConnections，是则创建IdleConnectionEvictor，往closeablesCopy注册shutdown及awaitTermination，最后执行connectionEvictor.start()。如果只是设置了evictExpiredConnections，则默认sleepTime为10s，否则sleepTime及maxIdleTime都为设置的值(>0)

IdleConnectionEvictor

org/apache/http/impl/client/IdleConnectionEvictor.java

/**
 * This class maintains a background thread to enforce an eviction policy for expired / idle
 * persistent connections kept alive in the connection pool.
 *
 * @since 4.4
 */
public final class IdleConnectionEvictor {

    private final HttpClientConnectionManager connectionManager;
    private final ThreadFactory threadFactory;
    private final Thread thread;
    private final long sleepTimeMs;
    private final long maxIdleTimeMs;

    private volatile Exception exception;

    public IdleConnectionEvictor(
            final HttpClientConnectionManager connectionManager,
            final ThreadFactory threadFactory,
            final long sleepTime, final TimeUnit sleepTimeUnit,
            final long maxIdleTime, final TimeUnit maxIdleTimeUnit) {
        this.connectionManager = Args.notNull(connectionManager, "Connection manager");
        this.threadFactory = threadFactory != null ? threadFactory : new DefaultThreadFactory();
        this.sleepTimeMs = sleepTimeUnit != null ? sleepTimeUnit.toMillis(sleepTime) : sleepTime;
        this.maxIdleTimeMs = maxIdleTimeUnit != null ? maxIdleTimeUnit.toMillis(maxIdleTime) : maxIdleTime;
        this.thread = this.threadFactory.newThread(new Runnable() {
            @Override
            public void run() {
                try {
                    while (!Thread.currentThread().isInterrupted()) {
                        Thread.sleep(sleepTimeMs);
                        connectionManager.closeExpiredConnections();
                        if (maxIdleTimeMs > 0) {
                            connectionManager.closeIdleConnections(maxIdleTimeMs, TimeUnit.MILLISECONDS);
                        }
                    }
                } catch (final Exception ex) {
                    exception = ex;
                }

            }
        });
    }

    public IdleConnectionEvictor(
            final HttpClientConnectionManager connectionManager,
            final long sleepTime, final TimeUnit sleepTimeUnit,
            final long maxIdleTime, final TimeUnit maxIdleTimeUnit) {
        this(connectionManager, null, sleepTime, sleepTimeUnit, maxIdleTime, maxIdleTimeUnit);
    }

    public IdleConnectionEvictor(
            final HttpClientConnectionManager connectionManager,
            final long maxIdleTime, final TimeUnit maxIdleTimeUnit) {
        this(connectionManager, null,
                maxIdleTime > 0 ? maxIdleTime : 5, maxIdleTimeUnit != null ? maxIdleTimeUnit : TimeUnit.SECONDS,
                maxIdleTime, maxIdleTimeUnit);
    }

    public void start() {
        thread.start();
    }

    public void shutdown() {
        thread.interrupt();
    }

    public boolean isRunning() {
        return thread.isAlive();
    }

    public void awaitTermination(final long time, final TimeUnit timeUnit) throws InterruptedException {
        thread.join((timeUnit != null ? timeUnit : TimeUnit.MILLISECONDS).toMillis(time));
    }

    static class DefaultThreadFactory implements ThreadFactory {

        @Override
        public Thread newThread(final Runnable r) {
            final Thread t = new Thread(r, "Connection evictor");
            t.setDaemon(true);
            return t;
        }

    };


}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88

IdleConnectionEvictor创建了一个thread，使用while循环，每次循环sleep指定的sleepTimeMs时间，然后执行connectionManager.closeExpiredConnections()；对于maxIdleTimeMs大于0的，执行connectionManager.closeIdleConnections(maxIdleTimeMs, TimeUnit.MILLISECONDS)

closeExpired

org/apache/http/pool/AbstractConnPool.java

    /**
     * Closes expired connections and evicts them from the pool.
     */
    public void closeExpired() {
        final long now = System.currentTimeMillis();
        enumAvailable(new PoolEntryCallback() {

            @Override
            public void process(final PoolEntry entry) {
                if (entry.isExpired(now)) {
                    entry.close();
                }
            }

        });
    }
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

closeExpired主要是遍历available，挨个判断是否expired(取决于connTimeToLive值)，是则执行close

closeIdle

org/apache/http/pool/AbstractConnPool.java

    /**
     * Closes connections that have been idle longer than the given period
     * of time and evicts them from the pool.
     *
     * @param idletime maximum idle time.
     * @param timeUnit time unit.
     */
    public void closeIdle(final long idletime, final TimeUnit timeUnit) {
        Args.notNull(timeUnit, "Time unit");
        long time = timeUnit.toMillis(idletime);
        if (time < 0) {
            time = 0;
        }
        final long deadline = System.currentTimeMillis() - time;
        enumAvailable(new PoolEntryCallback() {

            @Override
            public void process(final PoolEntry entry) {
                if (entry.getUpdated() <= deadline) {
                    entry.close();
                }
            }

        });
    }
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

closeIdle方法遍历enumAvailable，挨个判断最近的更新时间+idletime是否小于等于当前时间，是则执行close

小结

HttpClientBuilder提供了evictExpiredConnections、evictIdleConnections方法，在build方法会在connManagerShared为false的前提下判断是否开启evictExpiredConnections或者evictIdleConnections，是则创建IdleConnectionEvictor并执行start方法。IdleConnectionEvictor创建了一个thread，使用while循环，每次循环sleep指定的sleepTimeMs时间，然后执行connectionManager.closeExpiredConnections()；对于maxIdleTimeMs大于0的，执行connectionManager.closeIdleConnections(maxIdleTimeMs, TimeUnit.MILLISECONDS)。