docs/src/main/docbook/async.xml

<?xml version="1.0"?>
<!--

    Copyright (c) 2012, 2021 Oracle and/or its affiliates. All rights reserved.

    This program and the accompanying materials are made available under the
    terms of the Eclipse Public License v. 2.0, which is available at
    http://www.eclipse.org/legal/epl-2.0.

    This Source Code may also be made available under the following Secondary
    Licenses when the conditions for such availability set forth in the
    Eclipse Public License v. 2.0 are satisfied: GNU General Public License,
    version 2 with the GNU Classpath Exception, which is available at
    https://www.gnu.org/software/classpath/license.html.

    SPDX-License-Identifier: EPL-2.0 OR GPL-2.0 WITH Classpath-exception-2.0

-->

<!DOCTYPE chapter [<!ENTITY % ents SYSTEM "jersey.ent" > %ents;]>
<chapter xmlns="http://docbook.org/ns/docbook"
         version="5.0"
         xml:lang="en"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xmlns:xi="http://www.w3.org/2001/XInclude"
         xmlns:xlink="http://www.w3.org/1999/xlink"
         xsi:schemaLocation="http://docbook.org/ns/docbook http://docbook.org/xml/5.0/xsd/docbook.xsd
                             http://www.w3.org/1999/xlink http://www.w3.org/1999/xlink.xsd"
         xml:id="async">
    <title>Asynchronous Services and Clients</title>
    <para>
        This chapter describes the usage of asynchronous API on the client and server side. The term <emphasis>async</emphasis>
        will be sometimes used interchangeably with the term <emphasis>asynchronous</emphasis> in this chapter.
    </para>
    <section>
        <title>Asynchronous Server API</title>
        <para>
            Request processing on the server works by default in a synchronous processing mode, which means that a client
            connection of a request is processed in a single I/O container thread. Once the thread processing the request
            returns to the I/O container, the container can safely assume that the request processing is finished and that
            the client connection can be safely released including all the resources associated with the connection. This model
            is typically sufficient for processing of requests for which the processing resource method execution takes
            a relatively short time. However, in cases where a resource method execution is known to take a long time to compute
            the result, server-side asynchronous processing model should be used. In this model, the association between a
            request processing thread and client connection is broken. I/O container that handles incoming request may no longer
            assume that a client connection can be safely closed when a request processing thread returns. Instead a facility for
            explicitly suspending, resuming and closing client connections needs to be exposed.
            Note that the use of server-side asynchronous processing model will not improve the request processing time perceived
            by the client. It will however increase the throughput of the server, by releasing the
            initial request processing thread back to the I/O container while the request may still be waiting in a queue for
            processing or the processing may still be running on another dedicated thread. The released I/O container thread
            can be used to accept and process new incoming request connections.
        </para>
        <para>
            The following example shows a simple asynchronous resource method defined using the new JAX-RS async API:
            <example>
                <title>Simple async resource</title>
                <programlisting language="java" linenumbering="numbered">@Path("/resource")
public class AsyncResource {
    @GET
    public void asyncGet(@Suspended final AsyncResponse asyncResponse) {

        new Thread(new Runnable() {
            @Override
            public void run() {
                String result = veryExpensiveOperation();
                asyncResponse.resume(result);
            }

            private String veryExpensiveOperation() {
                // ... very expensive operation
            }
        }).start();
    }
}</programlisting>
            </example>
            In the example above, a resource <literal>AsyncResource</literal>
            with one <literal>GET</literal> method <literal>asyncGet</literal> is defined. The <literal>asyncGet</literal> method
            injects a JAX-RS &jaxrs.container.AsyncResponse; instance using a JAX-RS &jaxrs.container.Suspended; annotation.
            Please note that &lit.jaxrs.container.AsyncResponse; must be injected by the &lit.jaxrs.container.Suspended;
            annotation and not by &jaxrs.core.Context; as &lit.jaxrs.container.Suspended; does not only inject response but also
            says that the method is executed in the asynchronous mode. By the &lit.jaxrs.container.AsyncResponse; parameter into
            a resource method we tell the Jersey runtime that the method is supposed to be invoked using the asynchronous
            processing mode, that is the client connection should not be automatically closed by the underlying I/O container
            when the method returns. Instead, the injected &lit.jaxrs.container.AsyncResponse; instance (that represents the
            suspended client request connection) will be used to explicitly send the response back to the client using some other
            thread. In other words, Jersey runtime knows that when the <literal>asyncGet</literal> method completes, the response
            to the client may not be ready yet and the processing must be suspended and wait to be explicitly resumed with a
            response once it becomes available. Note that the method <literal>asyncGet</literal> returns <literal>void</literal>
            in our example. This is perfectly valid in case of an asynchronous JAX-RS resource method, even for a &jaxrs.GET;
            method, as the response is never returned directly from the resource method as its return value. Instead, the response
            is later returned using &lit.jaxrs.container.AsyncResponse; instance as it is demonstrated in the example. The
            <literal>asyncGet</literal> resource method starts a new thread and exits from the method. In that state the
            request processing is suspended and the container thread (the one which entered the resource method) is returned back
            to the container's thread pool and it can process other requests. New thread started in the resource method may
            execute an expensive operation which might take a long time to finish. Once a result is ready it is resumed using
            the <literal>resume()</literal> method on the &lit.jaxrs.container.AsyncResponse; instance.
            The resumed response is then processed in the new thread by Jersey in a same way as any other synchronous response,
            including execution of filters and interceptors, use of exception mappers as necessary and sending the response
            back to the client.
        </para>
        <para>
            It is important to note that the asynchronous response (<literal>asyncResponse</literal> in the example)
            does not need to be resumed from the thread started from the resource method. The asynchronous
            response can be resumed even from different request processing thread as it is shown in the
            the example of the &jaxrs.container.AsyncResponse; javadoc. In the javadoc example the
            async response suspended from the &lit.http.GET; method is resumed later on from
            the &lit.http.POST; method. The suspended async response is passed between requests using
            a static field and is resumed from the other resource method running on a different request processing thread.
        </para>
        <para>
            Imagine now a situation when there is a long delay between two requests and you would not like to let
            the client wait for the response "forever" or at least for an unacceptable long time. In asynchronous processing
            model, occurrences of such situations should be carefully considered with client connections not being automatically
            closed when the processing method returns and the response needs to be resumed explicitly based on an event that
            may actually even never happen. To tackle these situations asynchronous <emphasis>timeouts</emphasis> can be used.
        </para>
        <para>
            The following example shows the usage of timeouts:
            <example>
                <title>Simple async method with timeout</title>
                <programlisting language="java" linenumbering="numbered">@GET
public void asyncGetWithTimeout(@Suspended final AsyncResponse asyncResponse) {
    asyncResponse.setTimeoutHandler(new TimeoutHandler() {

        @Override
        public void handleTimeout(AsyncResponse asyncResponse) {
            asyncResponse.resume(Response.status(Response.Status.SERVICE_UNAVAILABLE)
                    .entity("Operation time out.").build());
        }
    });
    asyncResponse.setTimeout(20, TimeUnit.SECONDS);

    new Thread(new Runnable() {

        @Override
        public void run() {
            String result = veryExpensiveOperation();
            asyncResponse.resume(result);
        }

        private String veryExpensiveOperation() {
            // ... very expensive operation that typically finishes within 20 seconds
        }
    }).start();
}</programlisting>
            </example>
            By default, there is no timeout defined on the suspended &lit.jaxrs.container.AsyncResponse; instance.
            A custom timeout and timeout event handler may be defined using <literal>setTimeoutHandler(TimeoutHandler)</literal>
            and <literal>setTimeout(long, TimeUnit)</literal> methods. The <literal>setTimeoutHandler(TimeoutHandler)</literal>
            method defines the handler that will be invoked when timeout is reached. The handler resumes the response with the
            response code 503 (from Response.Status.<literal>SERVICE_UNAVAILABLE</literal>).
            A timeout interval can be also defined without specifying a custom timeout handler (using just the
            <literal>setTimeout(long, TimeUnit)</literal> method).
            In such case the default behaviour of Jersey runtime is to throw a <literal>ServiceUnavailableException</literal>
            that gets mapped into 503, "Service Unavailable" HTTP error response, as defined by the JAX-RS specification.
        </para>

        <section>
            <title>Asynchronous Server-side Callbacks</title>
            <para>
                As operations in asynchronous cases might take long time and they are not always finished within
                a single resource method invocation, JAX-RS offers facility to register callbacks to be invoked
                based on suspended async response state changes. In Jersey you can register two JAX-RS callbacks:
                <itemizedlist>
                    <listitem><simpara>
                        &jaxrs.container.CompletionCallback; that is executed when request finishes or fails, and
                    </simpara></listitem>
                    <listitem><simpara>
                        &jaxrs.container.ConnectionCallback; executed when a connection to a client is closed or lost.
                    </simpara></listitem>
                </itemizedlist>
                <example>
                    <title>CompletionCallback example</title>
                    <programlisting language="java" linenumbering="numbered">@Path("/resource")
public class AsyncResource {
    private static int numberOfSuccessResponses = 0;
    private static int numberOfFailures = 0;
    private static Throwable lastException = null;

    @GET
    public void asyncGetWithTimeout(@Suspended final AsyncResponse asyncResponse) {
        asyncResponse.register(new CompletionCallback() {
            @Override
            public void onComplete(Throwable throwable) {
                if (throwable == null) {
                    // no throwable - the processing ended successfully
                    // (response already written to the client)
                    numberOfSuccessResponses++;
                } else {
                    numberOfFailures++;
                    lastException = throwable;
                }
            }
        });

        new Thread(new Runnable() {
            @Override
            public void run() {
                String result = veryExpensiveOperation();
                asyncResponse.resume(result);
            }

            private String veryExpensiveOperation() {
                // ... very expensive operation
            }
        }).start();
    }
}</programlisting>
                </example>
                A completion callback is registered using <literal>register(...)</literal> method
                on the &lit.jaxrs.container.AsyncResponse; instance. A registered completion callback is bound only
                to the response(s) to which it has been registered. In the example the &lit.jaxrs.container.CompletionCallback;
                is used to calculate successfully processed responses, failures and to store last exception. This is only a
                simple case demonstrating the usage of the callback. You can use completion callback to release the resources,
                change state of internal resources or representations or handle failures. The method has an argument
                <literal>Throwable</literal> which is set only in case of an error. Otherwise the parameter
                will be <literal>null</literal>, which means that the response was successfully written. The callback is executed
                only after the response is written to the client (not immediately after the response is resumed).
            </para>
            <para>
                The &lit.jaxrs.container.AsyncResponse; <literal>register(...)</literal> method is overloaded and offers options
                to register a single callback as an <literal>Object</literal> (in the example), as a <literal>Class</literal> or
                multiple callbacks using varags.
            </para>
            <para>
                As some async requests may take long time to process the client may decide to terminate its connection to the
                server before the response has been resumed or before it has been fully written to the client. To deal with these
                use cases a &lit.jaxrs.container.ConnectionCallback; can be used. This callback will be executed only if the
                connection was prematurely terminated or lost while the response is being written to the back client. Note that
                this callback will not be invoked when a response is written successfully and the client connection is closed
                as expected. See javadoc of &jaxrs.container.ConnectionCallback; for more information.
            </para>
        </section>

        <section xml:id="chunked-output">
            <title>Chunked Output</title>
            <para>Jersey offers a facility for sending response to the client in multiple more-or-less independent chunks using a
                <emphasis>chunked output</emphasis>. Each response chunk usually takes some (longer) time to prepare before
                sending it to the client. The most important fact about response chunks is that you want
                to send them to the client immediately as they become available without waiting for the remaining chunks to become
                available too. The first bytes of each chunked response consists of the HTTP headers that are sent to the client.
                As noted above, the entity of the response is then sent in chunks as they become available.
                Client knows that the response is going to be chunked, so it reads each chunk of the response separately,
                processes it, and waits for more chunks to arrive on the same connection. After some time, the server generates
                another response chunk and send it again to the client. Server keeps on sending response chunks until
                it closes the connection after sending the last chunk when the response processing is finished.
            </para>
            <para>
                In Jersey you can use &jersey.server.ChunkedOutput; to send response to a client in chunks. Chunks are strictly
                defined pieces of a response body can be marshalled as a separate entities using Jersey/JAX-RS
                &jaxrs.ext.MessageBodyWriter; providers. A chunk can be String, Long or JAXB bean serialized to XML or JSON or
                any other defined custom type for which a &lit.jaxrs.ext.MessageBodyWriter; is available.
            </para>
            <para>
                The resource method
                that returns &lit.jersey.server.ChunkedOutput; informs the Jersey runtime that the response will be chunked
                and that the processing works asynchronously as such. You do not need to inject
                &lit.jaxrs.container.AsyncResponse; to start the asynchronous processing mode in this case.
                Returning a &lit.jersey.server.ChunkedOutput; instance from the method is enough to indicate the asynchronous
                processing. Response headers will be sent to a client when the resource method returns and the client will wait
                for the stream of chunked data which you will be able to write from different thread using the same
                &lit.jersey.server.ChunkedOutput; instance returned from the resource method earlier. The following example
                demonstrates this use case:
                <example>
                    <title>ChunkedOutput example</title>
                    <programlisting language="java" linenumbering="numbered"><![CDATA[@Path("/resource")
public class AsyncResource {
    @GET
    public ChunkedOutput<String> getChunkedResponse() {
        final ChunkedOutput<String> output = new ChunkedOutput<String>(String.class);

        new Thread() {
            public void run() {
                try {
                    String chunk;

                    while ((chunk = getNextString()) != null) {
                        output.write(chunk);
                    }
                } catch (IOException e) {
                    // IOException thrown when writing the
                    // chunks of response: should be handled
                } finally {
                    output.close();
                        // simplified: IOException thrown from
                        // this close() should be handled here...
                }
            }
        }.start();

        // the output will be probably returned even before
        // a first chunk is written by the new thread
        return output;
    }

    private String getNextString() {
        // ... long running operation that returns
        //     next string or null if no other string is accessible
    }
}]]></programlisting>
                </example>
                The example above defines a &lit.http.GET; method that returns a &lit.jersey.server.ChunkedOutput; instance.
                The generic type of &lit.jersey.server.ChunkedOutput; defines the chunk types (in this case chunks are Strings).
                Before the instance is returned a new thread is started that writes individual chunks into
                the chunked output instance named <literal>output</literal>. Once the original
                thread returns from the resource method, Jersey runtime writes headers to the container response but does not
                close the client connection yet and waits for the response data to be written to the chunked
                <literal>output</literal>.
                New thread in a loop calls the method <literal>getNextString()</literal> which returns a
                next String or &lit.null; if no other String exists (the method could for example load latest data
                from the database). Returned Strings are written to the chunked <literal>output</literal>. Such a written
                chunks are internally written to the container response and client can read them. At the end the
                chunked output is closed which determines the end of the chunked response. Please note that you must close
                the output explicitly in order to close the client connection as Jersey does not implicitly know when
                you are finished with writing the chunks.
            </para>
            <para>
                A chunked output can be processed also from threads created from another request as it is explained in the
                sections above. This means that one resource method may e.g. only return a &lit.jersey.server.ChunkedOutput;
                instance and other resource method(s) invoked from another request thread(s) can write data into the chunked
                output and/or close the chunked response.
            </para>
        </section>
    </section>
    <section>
        <title>Client API</title>
        <para>The client API supports asynchronous processing too. Simple usage of asynchronous client API is shown in the
        following example:
            <example>
                <title>Simple client async invocation</title>
<programlisting language="java" linenumbering="numbered"><![CDATA[final AsyncInvoker asyncInvoker = target().path("http://example.com/resource/")
        .request().async();
final Future<Response> responseFuture = asyncInvoker.get();
System.out.println("Request is being processed asynchronously.");
final Response response = responseFuture.get();
    // get() waits for the response to be ready
System.out.println("Response received.");]]></programlisting>
            </example>
            The difference against synchronous invocation is that the http method call <literal>get()</literal>
            is not called on &jaxrs.client.SyncInvoker; but on &jaxrs.client.AsyncInvoker;. The
            &lit.jaxrs.client.AsyncInvoker; is returned from the call of method
            <literal>Invocation.Builder.async()</literal> as shown above. &lit.jaxrs.client.AsyncInvoker;
            offers methods similar to &lit.jaxrs.client.SyncInvoker; only these methods do not return a response
            synchronously. Instead a <literal>Future&lt;...&gt;</literal> representing response data is returned.
            These method calls also return immediately without waiting for the actual request to complete.
            In order to get the response of the invoked <literal>get()</literal> method, the
            <literal>responseFuture.get()</literal> is invoked which waits for the response to be finished
            (this call is blocking as defined by the Java SE <literal>Future</literal> contract).
        </para>
        <para>
            Asynchronous Client API in JAX-RS is fully integrated in the fluent JAX-RS Client API flow, so that
            the async client-side invocations can be written fluently just like in the following example:
            <example>
                <title>Simple client fluent async invocation</title>
                <programlisting language="java" linenumbering="numbered"><![CDATA[final Future<Response> responseFuture = target().path("http://example.com/resource/")
        .request().async().get();]]></programlisting>
            </example>
        </para>

        <para>
            To work with asynchronous results on the client-side, all standard <literal>Future</literal> API facilities
            can be used. For example, you can use the <literal>isDone()</literal> method
            to determine whether a response has finished to avoid the use of a blocking call to <literal>Future.get()</literal>.
        </para>

        <section>
            <title>Asynchronous Client Callbacks</title>
            <para>Similarly to the server side, in the client API you can register asynchronous callbacks too. You can use
                these callbacks to be notified when a response arrives instead of waiting for the
                response on <literal>Future.get()</literal> or checking the status by <literal>Future.isDone()</literal> in
                a loop.
                A client-side asynchronous invocation callback can be registered as shown in the following example:
                <example>
                    <title>Client async callback</title>
                    <programlisting language="java" linenumbering="numbered"><![CDATA[final Future<Response> responseFuture = target().path("http://example.com/resource/")
        .request().async().get(new InvocationCallback<Response>() {
            @Override
            public void completed(Response response) {
                System.out.println("Response status code "
                        + response.getStatus() + " received.");
            }

            @Override
            public void failed(Throwable throwable) {
                System.out.println("Invocation failed.");
                throwable.printStackTrace();
            }
        });]]></programlisting>
                </example>

                The registered callback is expected to implement the &jaxrs.client.InvocationCallback; interface that defines
                two methods.
                First method <literal>completed(Response)</literal> gets invoked when an invocation successfully
                finishes. The result response is passed as a parameter to the callback method. The second method
                <literal>failed(Throwable)</literal> is invoked in case the invocation fails and the exception describing
                the failure is passed to the method as a parameter. In this case since the callback generic type is
                &lit.jaxrs.core.Response;, the <literal>failed(Throwable)</literal> method would  only invoked in case
                the invocation fails because of an internal client-side processing error. It would not be invoked
                in case a server responds with an HTTP error code, for example if the requested resource
                is not found on the server and HTTP <literal>404</literal> response code is returned. In such case
                <literal>completed(Response)</literal> callback method would be invoked and the response passed to the method
                would contain the returned error response with HTTP <literal>404</literal> error code. This is a special
                behavior in case the generic callback return type is &lit.jaxrs.core.Response;. In the next example an
                exception is thrown (or <literal>failed(Throwable)</literal> method on the invocation callback is invoked)
                even in case a non-<literal>2xx</literal> HTTP error code is returned.
            </para>
            <para>
                As with the synchronous client API, you can retrieve the response entity as a Java type directly without
                requesting a &lit.jaxrs.core.Response; first. In case of an &lit.jaxrs.client.InvocationCallback;, you need
                to set its generic type to the expected response entity type instead of using the &lit.jaxrs.core.Response;
                type as demonstrated in the example below:
                <example>
                    <title>Client async callback for specific entity</title>
                    <programlisting language="java" linenumbering="numbered"><![CDATA[final Future<String> entityFuture = target().path("http://example.com/resource/")
        .request().async().get(new InvocationCallback<String>() {
            @Override
            public void completed(String response) {
                System.out.println("Response entity '" + response + "' received.");
            }

            @Override
            public void failed(Throwable throwable) {
                System.out.println("Invocation failed.");
                throwable.printStackTrace();
            }
        });
System.out.println(entityFuture.get());]]></programlisting>
                </example>
                Here, the generic type of the invocation callback information is used to unmarshall the HTTP response content
                into a desired Java type.
            </para>
            <important>
                <para>
                    Please note that in this case the method <literal>failed(Throwable throwable)</literal> would be invoked even
                    for cases when a server responds with a non HTTP-<literal>2xx</literal> HTTP error code. This is because in this
                    case the user does not have any other means of finding out that the server returned an error response.
                </para>
            </important>
        </section>

        <section>
            <title>Chunked input</title>
            <para>
                In an <link linkend="chunked-output">earlier section</link> the &lit.jersey.server.ChunkedOutput; was
                described. It was shown how to use a chunked output on the server. In order to read chunks on the client the
                &jersey.client.ChunkedInput; can be used to complete the story.
            </para>
            <para>
                You can, of course, process input on the client as a standard input stream but if you would like to
                leverage Jersey infrastructure to provide support of translating message chunk data into Java types
                using a &lit.jersey.client.ChunkedInput; is much more straightforward. See the usage of the
                &lit.jersey.client.ChunkedInput; in the following example:

                <example>
                    <title>ChunkedInput example</title>
                    <programlisting language="java" linenumbering="numbered"><![CDATA[final Response response = target().path("http://example.com/resource/")
        .request().get();
final ChunkedInput<String> chunkedInput =
        response.readEntity(new GenericType<ChunkedInput<String>>() {});
String chunk;
while ((chunk = chunkedInput.read()) != null) {
    System.out.println("Next chunk received: " + chunk);
}
]]></programlisting>
                </example>

                The response is retrieved in a standard way from the server. The entity is read as a
                &lit.jersey.client.ChunkedInput; entity. In order to do that the &jaxrs.core.GenericEntity; is used to preserve
                a generic information at run time. If you would not use &lit.jaxrs.core.GenericEntity;, Java language generic type
                erasure would cause that the generic information would get lost at compile time and an exception would be thrown
                at run time complaining about the missing chunk type definition.
            </para>
            <para>
                In the next lines in the example, individual chunks are being read from the response. Chunks can come with some
                delay, so they will be written to the console as they come from the server. After receiving last chunk the
                &lit.null; will be returned from the <literal>read()</literal> method. This will mean that the server has sent
                the last chunk and closed the connection. Note that the <literal>read()</literal> is a blocking operation and the
                invoking thread is blocked until a new chunk comes.
            </para>
            <para>
                Writing chunks with &lit.jersey.server.ChunkedOutput; is simple, you only call method <literal>write()</literal>
                which writes exactly one chunk to the output. With the input reading it is slightly more complicated. The
                &lit.jersey.client.ChunkedInput; does not know how to distinguish chunks in the byte stream unless being told by
                the developer. In order to define custom chunks boundaries,
                the &lit.jersey.client.ChunkedInput; offers possibility to register a &jersey.client.ChunkParser; which
                reads chunks from the input stream and separates them. Jersey provides several chunk parser implementations and
                you can implement your own parser to separate your chunks if you need. In our example above the default parser
                provided by Jersey is used that separates chunks based on presence of a <literal>\r\n</literal> delimiting
                character sequence.
            </para>
            <para>
                Each incoming input stream is firstly parsed by the &lit.jersey.client.ChunkParser;, then each chunk is processed
                by the proper &jaxrs.ext.MessageBodyReader;.
                You can define the media type of chunks to aid the selection of a proper &lit.jaxrs.ext.MessageBodyReader; in
                order to read chunks correctly into the requested entity types (in our case into Strings).
            </para>
        </section>
    </section>
</chapter>