OkHttp Source Code Analysis
About the principle of OkHttp framework
Process analysis
We start with a simple HTTP request:
The above code will initiate two simple HTTP requests. The request flow is shown in the following figure.
The above flow chart only depicts the Chain of Responsibility. After the previous introduction, the principle of the Chain of Responsibility and each interceptor will be introduced separately.
OkHttpClient
We use new OkHttpClient() to create a default OkHttpClient, and we can also use OkHttpClient.Builder to construct a client with custom parameters.
Later, when we use network requests, we will use this client. It can be understood as the core class of the entire OkHttp. It encapsulates the overall OkHttp and provides external request initiation and some parameter configuration interfaces. We can use OkHttpClient .Builder to set, responsible for coordinating the operation of each class internally, it does not actually contain too much code.
Request
Request is well understood and responsible for assembling the request.
Call(RealCall)
Then call the client.newCall(request) method, which means to create a new request to be executed, and obtain a Call object (implemented as RealCall) through the newCall method. At this time, we use Call’s execute/enqueue to initiate a synchronization /async request.
So each Request will eventually be encapsulated into a RealCall object. RealCall and Request have a one-to-one correspondence. Call is used to describe a request that can be executed and interrupted. We create a RealCall object every time we initiate a request.
Finally, RealCall#getResponseWithInterceptorChain() is called to initiate the request, and this method will return a response result Response.
Dispatcher
Dispatcher is used to manage all requests of its corresponding OkHttpClient. As can be seen from the above flowchart, when using asynchronous requests, the request will be delegated to the Dispatcher object for processing, and the Dispatcher object is created with the creation of OkHttpClient.
In fact, Dispatcher is not only used to manage asynchronous requests, but also responsible for managing synchronous requests.
When we initiate a request, whether it is asynchronous or synchronous, it will be recorded by Dispatcher.
We can obtain the Dispatcher object through OkHtpClient#dispatcher() for unified control of requests, such as ending all requests, obtaining thread pools, and so on.
The Dispatcher contains three queues:
readyAsyncCalls: A new asynchronous request will first be added to the queuerunningAsyncCalls: the currently running asynchronous requestsrunningSyncCalls: currently running sync requests
Dispatcher contains a default thread pool for executing all asynchronous requests. You can also specify a thread pool through the constructor, and all asynchronous requests will be executed through this thread pool.
Like synchronous requests, asynchronous requests will eventually call RealCall#getResponseWithInterceptorChain() to initiate requests, but one is called directly and the other is called in the thread pool.
Through the above introduction, it has been found that the key point is that the method with a long name, as long as it is called, it can return a Response, and this method begins to involve the well-known OkHttp Chain of Responsibility model.
OkHttp Chain of Responsibility
It all starts with the method with a very long name. We know that in any case, RealCall#getResponseWithInterceptorChain() will be called to initiate the request and get the final Response.
This method will assemble the Interceptor list according to the Interceptor set by the user and several default Interceptors, and then create a Chain of Responsibility.
After the Chain of Responsibility is created, its process method will be called to get the Response and return it, which involves two concepts: Interceptor and Chain.
Interceptor
As an abstract concept of an interceptor, the Interceptor interface is designed as a unit node on the Chain of Responsibility for observing, intercepting, and processing requests, such as adding headers, redirecting, data processing, and so on.
Interceptors are independent of each other, and each Interceptor is only responsible for the tasks it focuses on and does not contact other Interceptors.
The Interceptor interface contains only one method (OkHttp is now rewritten in Kotlin):
The intercept method receives a Chain as a parameter and returns a Response.
In RealCall, the following default Interceptors will be added to the Chain of Responsibility in order to complete the basic functions:
- User-set Interceptor
RetryAndFollowUpInterceptor: Retry and redirect on failure BridgeInterceptor: handles network headers, cookies, gzip, etc.CacheInterceptor: manage cacheConnectInterceptor: connect to the server- If it is a WebSocket request, add the corresponding Interceptors
CallServerInterceptor: data send/receive
The specific meaning and principle of these Interceptors will be introduced in detail later.
The Chain of Responsibility will execute these Interceptors in the order in which they were added, so the order is very important.
Through the processing of these Interceptors, a perfect Response will eventually be returned to the method with a long name in RealCall, and then returned to downstream users. At this point, a complete request has come to an end.
Chain
Chain is used to describe the Chain of Responsibility, through which the process method starts to execute each node on the chain in turn, and returns the processed Response.
The only implementation of Chain is RealInterceptorChain (hereinafter referred to as RIC), RIC can be called Interceptor Responsibility Chain, and the nodes in it are composed of Interceptors added in RealCall. Due to the independence of Interceptors, RIC also contains some common parameters and shared objects.
Interceptor and Chain depend on each other, call each other, and develop together, forming a perfect call chain. Let’s take a look at their call relationship diagram:
It can be clearly seen from the above figure that when we call the Chain#process method in an Interceptor to obtain the Response, the request will be processed according to the Interceptor after calling the current position.
After the processing is completed, the Response will be returned to the current Interceptor, and then after processing, return to the upper level until the end of the traversal.
Network connection and data sending and receiving
The basic concepts, basic configuration, thread control, and chain of responsibility of OkHttp have been introduced above. Let’s talk about the soul of a network framework: the establishment of network requests and the sending and receiving of data.
Several different Interceptors added in RealCall cooperate with each other to complete these functions. As long as you understand these basic interceptors, you will understand the soul of OkHttp.
In fact, I don’t recommend paying too much attention to the implementation details when reading the source code. As long as you understand the design ideas, the general implementation is almost the same, otherwise it is easy to be confused by the responsible details.
So before introducing these interceptors, let’s introduce some basic concepts in OkHttp.
How the connection is established
Many of the network request frameworks we saw before, such as Volley, etc., are connected to the server through HTTPURLConnection at the bottom layer, and OkHttp is better. Because the HTTP protocol is based on the TCP/IP protocol, and the bottom layer is still using Socket, OkHttp directly uses Socket to complete HTTP requests.
Route
route is the specific route used to connect to the server. It contains parameters such as IP address, port, proxy, etc.
The same interface address may correspond to multiple routes due to the fact that the proxy or DNS may return multiple IP addresses.
The Route will be used instead of the IP address directly when creating the Connection.
RouteSelector
Route selector, which stores all available routes, and gets the next route through the RouteSelector#next method when ready to connect.
It is worth noting that the RouteSelector contains a routeDatabase object, which stores the Routes that failed to connect, and the RouteSelector will store the last route that failed to connect at the end to improve the connection speed.
RealConnection
RealConnection implements the Connection interface, which uses Socket to establish HTTP/HTTPS connections and obtain I/O streams. The same Connection may carry multiple HTTP requests and responses.
In fact, it can be roughly understood as the encapsulation of Socket, I/O stream, and some protocols. This involves a lot of computer network-related knowledge, such as TLS handshake, HTTPS verification and so on.
RealConnectionPool
This is the pool used to store the RealConnection, and internally a double-ended queue is used for storage.
In OkHttp, a connection (RealConnection) will not be closed and released immediately after it is used up, but will be stored in the connection pool (RealConnectionPool).
In addition to caching connections, the cache pool is also responsible for regularly cleaning up expired connections. A field is maintained in RealConnection to describe the idle time of the connection.
Every time a new connection is added to the connection pool, detection is performed, traversing all Connect to find the connection that is currently unused and has the longest idle time.
If the connection idle time exceeds the threshold, or the connection pool is full, the connection will be closed.
In addition, RealConnection also maintains a weak reference list of Transmitter to store the Transmitter currently using the connection. When the list is empty it means that the connection is not in use.
ExchangeCodec
ExchangeCodec is responsible for encoding and decoding the Response, that is, writing the request and reading the response. Our request and response data are read and written through it.
So Connection is responsible for establishing the connection, and ExchangeCodec is responsible for sending and receiving data.
There are two implementation classes of the ExchangeCodec interface: Http1ExchangeCodec and Http2ExchangeCodec, corresponding to two protocol versions respectively.
Exchange
The Exchange function is similar to ExchangeCodec, but it corresponds to a single request, which is responsible for some connection management and event distribution functions on the basis of ExchangeCodec.
Specifically, Exchange corresponds to Request one by one. When a new request is created, an Exchange is created. The Exchange is responsible for sending the request and reading the response data, and the ExchangeCodec is used for sending and receiving data.
Transmitter
Transmitter is the bridge of the OkHttp network layer. The concepts we mentioned above are ultimately integrated through Transmitter and provide external functions.
Ok, now that the basic concepts are introduced, let’s start looking at interceptors.
RetryAndFollowUpInterceptor
This interceptor, as the name suggests, is responsible for failed retries and redirects.
Conditions that may trigger a retry or redirect are as follows:
- 401: Unauthorized
- 407: Proxy not authorized
- 503: Service Unauthorized
- 3xx: request redirection
- 408: Request timed out
- And some I/O exceptions and other connection failures
As we mentioned above, due to proxy and DNS reasons, there may be multiple IP addresses for the same URL. When connecting, select the appropriate Route through RouteSelector to connect, so the failed retry here does not refer to multiple IP addresses for the same IP address. The retries are to try the addresses in the routing table one by one.
If the response code is 401 or 407, it means that the request is not authenticated. At this time, the request is re-authenticated, and then the authenticated Request is returned.
The response code is 3xx, which means redirection. At this time, the redirection address is in the Location field of the response Header, and then a new Request is constructed through this new address and the previous Request and returned.
The response code 503 indicates a server error, but this is temporary and may be restored soon, so the previous request will be returned directly.
BridgeInterceptor
BridgeInterceptor is a bridge between users and the network, responsible for converting user requests into network requests, that is, forming network headers and setting response data based on Request information.
In fact, BridgeInterceptor is responsible for setting cookies and gzip.
Before starting a network request, BridgeInterceptor will first judge whether there is a cookie through the URL and if so, it will bring the cookie.
After the request ends, it will also judge whether the response header contains the Set-Cookie field, and it will be saved next time use. However, the operation of storing cookies will be delegated to CookieJar.
OkHttp provides an empty CookieJar object by default, which means that no operation is performed by default, but you can specify your own CookieJar to use when creating OkHttp.
If the Accept-Encoding and Range fields are not included in the Request request header, an Accept-Encoding: gzip request header will be added to it. After receiving the response data, if the response indicates that gzip is used, the response data will be handed over to okio’s GzipSource decoding.
CacheInterceptor
CacheInterceptor is responsible for caching response data.
This method first tries to obtain the cached data through the Cache object, and then obtains the cache strategy through CacheStrategy.
Through the calculation result of this strategy, we can obtain two nullable objects: networkRequest and cacheResponse.
Where networkRequest is the original Request but may be empty. Whether it is empty or not is controlled by CacheStrategy.
cacheResponse is the Response obtained through Cache, same as above, and may also be empty.
Then you can handle the cache by judging the nullability of the two objects, the logic is as follows:
- If both are empty, it means that neither the use of network requests nor the use of caches or cache misses is allowed, and a 504 error is returned directly.
- If only the
networkRequestis empty, it means that the network request is forbidden, and the Response hit from the cache is directly returned. - If neither is empty, start making requests and get response data.
- If the
cacheResponseis not empty at this time and the response code is 304, thecacheResponseis returned directly, and the cache is updated with the response data. - If
cacheResponseis empty, the response data will be stored inCache.
Return response data.
It should be noted that the Cache object mentioned above is empty by default. If it is empty, the operations related to it will not be executed, and the cacheResponse must be empty.
We can set Cache in OkHttpClient.
ConnectInterceptor
ConnectInterceptor is used to open a connection to the server.
The code is very simple. It will create an Exchange object through the Transmitter#newExchange method and call the Chain#process method.
In the newExchange method, it will first try to find an existing connection in the RealConnectionPool through ExchangeFinder.
If it is not found, it will re-create a RealConnection and start the connection, and then store it in the RealConnectionPool.
At this time, the RealConnection object has been prepared, and then created through the request protocol Different ExchangeCodec and return. The specific details have been mentioned above and will not be introduced in detail here.
After creating the ExchangeCodec through the above steps, create an Exchange object based on it and other parameters and return it.
ConnectInterceptor calls the Chain#process method with the Exchange object returned by the newExchange method as a parameter.
CallServerInterceptor
CallServerInterceptor is responsible for reading and writing data.
This is the last interceptor. Everything that should be prepared here is ready. Through it, the data in the Request will be sent to the server, and the obtained data will be written into the Response.
So the source code of OkHttp is almost analyzed here. In fact, there are many things that have not been mentioned. OkHttp is a huge framework, which involves too many things, and includes a lot of basic knowledge of computer networks. thanks!
