Since a servlet is blocking, a normal HTTP communication makes the thread wait. So there is a possibility that all the threads in the pool can be used up. By using the Asynchronous HTTP Client framework (AHC javadocs), the thread can be released back into the pool and be made available for other purposes until the answer comes back. When the answer comes back, a callback is fired which then relays the response back to the client.

The async http client is to be used in conjunction with web containers that are configured for non blocking communication. CometProcesor with Tomcat and Continuation with Jetty are some good examples. This example uses Tomcat.

This document is organized in the following sections:

About this example

This example seeks to demonstrate the usage of the async http client by first configuring the Geronimo server. It then deploys an app which we shall call http-local-app on this configured server.  Another webapp call http-remote-app is deployed on any other server. The servlet on the http-local-app delegates requests to the async http client and returns immediately. The async http client connects to the remote app and when it recieves a response, a callback is fired which relays the message back to the original response stream.

The http-local-app

The AsyncServlet in this app uses the AHC and implements the CometProcessor interface. The CometProcessor will make it's event method invoked rather than the usual service method, according to the event which occurred. The event object gives access to the usual request and response objects, which may be used in the usual way. More information about this can be read here.

The servlet also generates a random delay time and a random http status code. Using the async client, it connects to a remote url and passes these randomly generated values to it. The remote url is initialized from the servlet's <init-param> value.  Once the async client sends a request to the remote app, the servlet returns without waiting for a response from the remote app.  This app has a callback listener which holds the CometEvent object. When it receives the answer from the remote app, it gets the original response from the event object and passes the message to it.

The http-remote-app

A simple servlet in this app reads the two request parameters delay and code.  It sleeps for the amount of time specified in the delay. If the code is 200, it serves a file called dummy.html. For the status code 500, it does a divide by zero to throw a servlet exception. For other status codes, it sets the appropriate response code in the response.sendError(). It then returns. This simulates varous response times and return status codes from an external app. This app can be run on any other server and machine.

Configuring the server

Use one of the following methods to configure your server. Little-G assemblies do not have a console.

Using console

Install a geronimo server with tomcat webcontainer and start it. Log onto the console by using system/manager as your credentials. Click on the "Web Server" link on the left hand navigation frame. Under "Add new:" section, click on "Tomcat NIO HTTP connector".  Specify a uniquename for the connector and take all the default values. Save this configuration. Your new connector must be listed in the Network Listeners.

Now delete the TomcatWebConnector in the Network Listeners list so that all connections are made to the new Tomcat NIO HTTP connector.

Editing the config.xml

1. Stop the server.
2. edit geronimo_home/var/config/config.xml
3. add load="false" to the TomcatWebConnector gbean. The line would now look like this
   <gbean load="false" name="TomcatWebConnector">

1. for the module org.apache.geronimo.configs/tomcat6/<version>/car, add the following gbean
2. The gbean uses version 2.0.1. Replace that with the appropriate version of your server
3. Start the server.

Building the sample

1. Download the zip file which contains both the apps discussed above.
2. Unzip the file into a directory of your choice, say samples.
3. cd into samples\async-http directory\http-local-app\src\main\webapp\WEB-INF directory.
4. edit the web.xml there and change the value of the <remoteUrl> initi-param. This should be set to the machine where your remote app will be deployed.
5. cd back to async-http directory.
6. execute mvn. This will build the 2 WARs.

Testing the sample with a browser

1. Deploy samples\async-http\http-local-app\target\http-local-app-2.0-SNAPSHOT.war on the Geronimo server we configured earlier using either the consoleor the command line
   $geronimo_home/bin>./deploy.sh deploy samples\async-http\http-local-app\target\http-local-app-2.0-SNAPSHOT.war
2. Deploy samples\async-http directory\http-remote-app\target\http-remote-app-2.0-SNAPSHOT.war on any server on the remote machine.
3. Point your browser to http://localhost:8080/localApp/async. If you wish to override the remoteUrl set in the <init-param> of the web.xml, you may pass it as a request param (http://localhost:8080/localApp/async?remoteUrl=http://foo:8080)
4. You should see either a dummy html or one of the popular HTTP errors.

Running the sample under heavy load

The true powers of the AHC can be seen only when the app using it is stressed under a heavy load. The client stressing the app can be any multi-threaded simulating simultaneous HTTP requests.

To leverage the full potential of AHC, the Geronimo server running it and the 3 machines (machines running the client, the local app with AHC and remote server) in the test setup must be tuned. This is necessary because the AHC starts devouring sockets and making remote connections at the same pace at which it gets the requests. If more sockets are made available to it, the AHC spends less time waiting for existing ones to free up. The bottleneck now has been transferred to establishing connections rather than at a normal blocking servlet.

Tuning

Here are some of the parameters that can be tuned.

1. acceptCount This attribute can be adjusted in TomcatNIOConnector gbean that you created above. Each incoming request requires a thread for the duration of that request. If more simultaneous requests are received than can be handled by the currently available request processing threads, additional threads will be created up to the configured maximum (the value of the maxThreads attribute). If still more simultaneous requests are received, they are stacked up inside the server socket created by the Connector, up to the configured maximum (the value of the acceptCount attribute). Any further simultaneous requests will receive "connection refused" errors, until resources are available to process them. So set the value of this attribute to be atleast equal to the number of the client threads pounding the AHC.
2. TIME_WAIT (TcpTimedWaitDelay) This setting determines the length of time that a connection will stay in the TIME_WAIT state before being closed
3. SYN_COOKIES This detects syn attacks. This should be turned off/disabled.
4. SOCKET_QUEUE (same as MaxUserPort ?) The Windows system limits the maximum port number assigned to outgoing connections. By default this value is 5000. You may want to increase that value to 20,000 or more
5. TcpNumConnections On Windows, this parameter limits the maximum number of connections that TCP can have open at the same time.

Stress Test

1. Download the client python script. You will need python installed on your machine to run this script. You can download Python from here.
2. The script takes 3 arguments - the name of the machine running http-local-app (AHC), the url of the server on the remote app and the number of threads.
   ahcrequest.py local_machine http://remote_machine:8080 200

Test Results

The following test results show a comparison between an AHC enhanced servlet and a normal blocking servlet. Each of these servlets are pounded by 16, 200 and 500 simultaneous requests. The output captured from the python script shows the response time for each request. These tests also demonstrate the significance of tuning the IP stacks. When none of the machines are tuned, the response time starts increasing as the number of threads start increasing. Executing a netstat command shows that most sockets are either in TIME_WAIT or CLOSED_WAIT state.

Code location and References

Browse the sample code at http://svn.apache.org/viewvc/geronimo/samples/trunk/samples/async-http/

Check out code from svn using
svn co https://svn.apache.org/repos/asf/geronimo/samples/trunk/samples/async-http async-http

Browse the AHC code at http://svn.apache.org/viewvc/geronimo/sandbox/AsyncHttpClient

Check out code from svn using
svn co https://svn.apache.org/repos/asf/geronimo/sandbox/AsyncHttpClient ahc

View AHC's javadocs at AHC javadocs