神刀安全网

Why Async?

At therecent Yext Offsite, I gave a deep-dive on CompletableFutures and their ability to maintain the semantics of an algorithm while ‘mixing in’ asynchrony. This makes the transition from sync to async intuitive and clean.

One question that inevitably pops up is “Why bother making your interface asynchronous, since the thing that implements that interface will have to block anyway?” This blog post answers that question.

Why Async?

Why is this:

httpClient.getAsync(url)           .then(content -> {               doSomethingWith(content);           }); 

better than this:

String content = httpClient.getSync(url); doSomethingWith(content); 

?

Isn’t httpClient just going to do something like:

CompletableFuture<String> getAsync(String url) {     return CompletableFuture.supplyAsync(() -> {         return getSync(url);     }); } 

?

We unblocked one thread but blocked another.

Net result: same number of blocked threads. Right?

Well, maybe, maybe not. Either way, there are the following reasons to consider the async version:

Natural Parallelism

Let’s extend our above example. Say that now, instead of one HTTP operation, we need to do two .

If we already have the first (async) form, the extension to parallelism is natural:

httpClient.getAsync(url)           .then(content -> {               doSomethingWith(content);           });  httpClient.getAsync(url2)           .then(content -> {               doSomethingElseWith(content);           }); 

We just add the new code, and it runs in parallel.

To extend the sync version, we’d first have to rewrite it to be async. No one will actually bother to do that, and you’ll end up with a sequential function instead:

String content = httpClient.getSync(url); doSomethingWith(content); String content2 = httpClient.getSync(url2); doSomethingElseWith(content2); 

Now this function takes twice as long as it needs to.

This situation comes up all the time. Just look at any non-trivial web service – how many parallelizable operations does a typical request handler do in sequence?

Going async-first allows developers to fall into the pit of success when it comes to parallelism.

Special Contexts

Some threads are special.

  • UI Thread – If you are working on a client application with a UI, such as iOS or Android, the UI thread is very special. If you block it, the UI will stop updating and become unresponsive to user input.

  • Request Handlers – Web frameworks like Play! often have a pool of threads they use to serve requests. If you block one, you’re preventing further requests from being handled. Here’s a great writeup for Play 2.x. Play 1.x (which we use at Yext) is less async-focused but the limited thread pool situation is similar

  • Message Handlers – If you block a message handler thread (responding to updates from RabbitMQ), you prevent further messages from being processed.

  • RPC Handlers – If you block an RPC request handler thread, you prevent further requests from being handled.

You should be seeing a pattern here. Threads and thread pools generally have a purpose. Unless it’s the common fork/join pool.

If you’re intentionally rate-limiting requests, then do so, but rate-limiting implicitly by blocking handler threads is a bad way to do it. It confuses limitations.

The limitation on number of RabbitMQ messages that can be handled concurrently is not equivalent to the limit of outgoing HTTP requests that a single server should be doing concurrently. So why are you running HTTP requests on the RabbitMQ thread pool? Better let some shared HTTP library handle that, so it can impose whatever limit there is on HTTP requests.

Non-Thread Asynchrony

Did you know: you can have asynchrony without threads?

If you use nio , such as AsynchronousFileChannel , Java will use OS primitives to do the I/O efficiently. Threads are still involved, but very special purpose-built threads, like I/O Completion Ports on Windows.

Going further, you really don’t need threads at all. Consider this service:

public interface IService {     Task<Result> DoRequestAsync(Request arg); } 

Our first implementation of this service was what you would expect:

class Service : IService {     public Task<Result> DoRequestAsync(Request arg) {         // uses a thread pool, like CompletableFuture.supplyAsync         return Task.Run(() => internalService.DoRequestSync(arg));     } } 

However, we eventually replaced this with a version where the other side of the service (which is running in another process) will call into our code directly to notify completion:

class Service : IService {     public Task<Result> DoRequestAsync(Request arg) {         TaskCompletionSource<Result> tcs = new TaskCompletionSource<Result>();         int requestId = internalService.StartRequest(arg);         InFlight[requestId] = tcs;         return tcs.Task;     }      // called directly by the service impl, using IPC     public void RequestFinished(int requestId, Result result) {         // like CompletableFuture.complete         TaskCompletionSource<Result> tcs = InFlight[requestId];         tcs.TrySetResult(result);     } } 

We were able to make this change completely transparently because our asynchronous interface abstracts over the mechanism of asynchrony. If we left it to the client to schedule the request on a thread pool, we would have had to update every caller.

In Java, CompletableFuture allows you to tie together diverse asynchronous operations. It doesn’t matter how each producer implements its asynchrony, it’s the same interface to the caller.

Separation of Concerns

The previous two points imply this one, but let’s underscore it.

If you are a programmer that wants to make an asynchronous HTTP request, the user of this functionality, you don’t know what you’re doing . And it’s not your responsibility to know.

Do you use a thread pool and a blocking Stream , or nio and a Channel ? How many requests are okay to do in parallel? What about requests happening concurrently in other parts of the system? Maybe you can rate limit your use case, but what if some completely separate daemon is making an additional n requests per second?

As the user , the one who just wants to do some useful operation with the result of an HTTP call, it is almost certainly not in your domain to answer these questions. Leave it to the provider of the HTTP service.

Even if their solution is to create exactly the same thread pool you would create as the naive user-side solution, it’s better to leave that decision to the one most capable of making it.

Further reading

I’ve tried to demonstrate the power of expressing the units of computation using the primitives that CompletableFuture provides. Essentially, the great strength of CompletableFuture is that it’s a monadic type, allowing you to take an operation in the domain of basic types like function from string to integer and ‘elevate’ it to a higher domain like function from asynchronous string to asynchronous integer

For further reading, I’d recommend reading these articles in this order:

转载本站任何文章请注明:转载至神刀安全网,谢谢神刀安全网 » Why Async?

分享到:更多 ()

评论 抢沙发

  • 昵称 (必填)
  • 邮箱 (必填)
  • 网址
分享按钮