Guide of Virtual Threads – Lightweight threads in Java

No comments
Advertisements

Since the very beginning – back to 1997 – Java provided an easy way to work with multi-thread applications. Yes, we are talking about the Thread class and the Runnable interface. With new Java versions, the core included more a more ways to simplify working with concurrency in Java. For example: The Executor Service, ForkJoinPool, Future, CompletableFuture (which are very similar to promises in Java Script), the parallel stream, the java.concurrent.* package with collections, utilities, locks and more.

All those features, makes Java a rich ecosystem to work with multi-thread application, however they have been limited to the OS threads. In big applications with hundreds of concurrent process might not be efficient enough, and might not scale easily, requiring to add more cpu to provide more available threads. This is primarily due to the shared state concurrency model used by default.

That’s why Project Loom started in 2017, and an initiative to provide lightweight threads that are not tied to OS threads but are managed by the JVM. A lot of changes and proposals from the first draft, but it seems to be ready in Java 21.

Project Loom and Virtual Threads

As mentioned before, project Loom started as an internal initiative to provide lightweight threads in Java, passing through different drafts and proposals about how to do it, concepts like Fiber and Channels renamed to the actual concept of Virtual Threads. Project loom was incubating some of those features in previous versions of Java – added in Java 19 in preview mode available to developers to provide feedback-, the preview stage finished and now the features are available as part of Java 21 (released in September 2023).

Virtual Threads are a new kind of threads added to Java, which are not tied to the OS threads like the “Platform Threads”. They are lightweight because it is possible to have thousands and even millions of Virtual Threads with minimum of resources, as they are managed by the JVM, the limit is the memory in the JVM.

If you have worked with Kotlin or Go Lang, you will notice that Virtual Threads are in deed very similar to Go routines or Kotlin coroutines. Some developers move to those languages because of those features, it seems Java is now providing the same in a backward compatible approach – For the Java Lovers, this is one more reason to continue using Java -.

Thread.startVirtualThread(() -> {
    System.out.println("Hello, Virtual Thread!");
});
go func() {
    println("Hello, Goroutines!")
}()
runBlocking {
    launch {
        println("Hello, Kotlin coroutines!")
    }
}


How to use virtual threads

The Thread class provides some new builder methods that can be used to create a VirtualThread. For example: startVirtualThread, which creates and starts a Virtual thread in one single operation. 

var task = () -> System.out.println("Hello World!");
// Creating and starting the thread in the same operation
Thread.startVirtualThread(task);

By using the ofVirtual() which returns a Builder instance to create step by step the desired thread. 

//Creating the Virtual thread, and then Running the start method. 
var thread = Thread.ofVirtual().start(task);
var thread2 = Thread.ofVirtual().name("My Thread").start(task); 

//If need to create the thread but not start it yet
var unstartedThread = Thread.ofVirtual().unstarted();
//... some other logic... 
unstartedThread.started(); //Now starts the thread

In addition to that, the Builder can return a factory instance from the can we can build VirtualThreads on demand. 

// Virtual thread created with a factory
ThreadFactory factory = Thread.ofVirtual().factory();
Thread virtualThreadFromAFactory = factory.newThread(task);
virtualThreadFromAFactory.start();

Executor Service and Virtual Threads

Thread Pools are a valid strategy to reuse the expensive Platform Threads for multiple tasks, but with Virtual Threads -which are lightweight and cheap – creating a pool of them is not necessary. The ExecutorService is an API providing some thread pools in Java, and extensively used in many applications. Therefore, wasting resources in pooling very cheap objects usually with a short life time does not make sense, that’s why it’s not recommend to pool Virtual Threads.

If you want to take advantage of the benefits of Virtual Threads but without refactoring your existing code using the ExecutorService , then you can use the implementation VirtualThreadPerTaskExecutor . As it name indicates, it creates a Executor which creates VirtualThread per Task, in other words it’s not reusing or “pooling” the virtual threads.

Runnable task = () -> System.out.println("Hello, world");
ExecutorService executorService = Executors.newVirtualThreadPerTaskExecutor();
executorService.execute(task);

Or by adding a ThreadFactory instance which returns VirtualThreads to any other Executors methods. 

ThreadFactory factory = Thread.ofVirtual().factory();
Executors.newThreadPerTaskExecutor(factory); // Same as newVirtualThreadPerTaskExecutor
Executors.newSingleThreadExecutor(factory);
Executors.newCachedThreadPool(factory);
Executors.newFixedThreadPool(1, factory);
Executors.newScheduledThreadPool(1, factory);
Executors.newSingleThreadScheduledExecutor(factory);

That’s the recommend first step to start using Virtual Threads if using the Executor Service. As you can see minimal changes to the existing code, and backward-compatible. Then, a more deep refactor in the code can be done if needed.


Show me the numbers

You will see in a lot of places (included this article) that now, it’s possible to have millions of threads at the same moment with virtual threads, with minimum resource usage. Let’s look at some numbers collected from different articles that support that.

Scenario 1

On this article A simple benchmark for Virtual Threads, the author created some code running 1000 simple tasks, using threads and virtual threads, each task is basically just a sleep for 100ms. He ran different scenarios for example: one single thread for all the tasks, one executor service with a pool of 4 threads running all the tasks, and also running one single Virtual Thread per each task (recommend usage of Virtual threads). These are the numbers:

Advertisements
one-thread-for-all     109.435 ± 0.023   sec/operation
fixed-pool              25.363 ± 1.721   sec/operation
virt-thread-per-task     0.117 ± 0.010   sec/op

As you can see, it took just 117 ms to run all the 1000 tasks, while in the other scenarios it took 25-100 seconds to do it… Why? because in the fixed pool only 4 threads where running at the time, doing some parallelism, but still running them in sequence.

Scenario 2

Maybe it can look that numbers might not be correct because the task just does a simple sleep, and real applications are more complex than it. In this article, Java virtual threads millions within grasp, the author create a task which is doing a request to one of the url for 250 top Alexa’s sites ranking, request are I/O Blocking operations, therefore good candidates for using Virtual Threads.

Two scenarios, one with a Fixed Pool Executor with size as the number of processors, the other with an Executor with a Virtual Thread per task. And the results, he noticed between 25%-50% shorter execution using Virtual Threads. Which a good improvements in a more real scenario.

Scenario 3

What about having a Web Application accepting request with Virtual Threads, because of using one virtual thread per request (per task), it might be possible to accept more request than with a fixed pool of Platform Threads. Well, that was the exercise tested on this article, Virtual thread: performance gain for microservices, where the author modified a Spring Boot application to support virtual threads for the requests.

The result shows how with the traditional threading, with more than 200 requests concurrently (like 1000 or 2000), only 200 were processed (the top of threads). The other ones were “queued” are the average response time increased in time.

On the other side, for Virtual Threads, regardless of the number of concurrency of request, almost all of them were processed immediately having a similar number of transactions per second. Reducing the average response time of the operations.

Scenario 4

But, is it true that it’s possible to have up to 1 000 000 virtual threads at the same time? The answer is YES.

There are benchmarks where someone just starts running 1 000 000 threads, doing some sleep for some minutes, in one case with Platform threads, in the other using Virtual Threads. In the case of Platform threads the JVM crashed with OutOfMemoryError with close to 30k threads created. While with virtual threads the one million threads were created, with minimum usage of memory (resources) and the application never crashed.

Another example is this article, Running 1 Million Threads in Java, the author run a similar scenario like the explained before, and monitor the CPU in the Operative System, to realize that Virtual Threads created 1 000 000 tasks quicker, and uses less resources in the Operative System.

Scenario 5

In this scenario (Project Loom & Virtual Threads), the benchmark is comparing the number of OS threads required for some logic in Java. In one side, it’s using Java Platform Threads to run 1000 tasks, and in the other it’s using Virtual Threads to run also 1000 tasks.

For Platform Threads, it created around 1000 Platform Threads during the test, using a 2-3% of CPU, and 150 Mb of memory. On the other side with virtual Threads, it only used 30 “Platform threads” (as workers), CPU usage was less than 2%, and memory usage was between 20Mb to 60Mb.

Therefore definitely, virtual threads were lightweight than platform threads in memory, and also the virtual threads required less platform threads to work.

Considerations

There are considerations about using Virtual Threads instead of Platform Threads.

  • When using CPU extensive tasks, it’s better to use Platform Threads. Because Virtual threads take advantage of CPU idle time (like sleep, I/O connections, etc) to switch and execute other threads on the same worker, but with CPU extensive tasks, there is not idle time, the CPU is always busy.
  • Use Virtual Threads, when the task is more about, doing some other logic, like connecting another service, http request, socket connection, moving data between objects, etc.

Regarding how to setup a Virtual Thread (comparing with Platform threads), these are other important considerations.

  • A virtual thread can be scheduled on different carriers over the course of its lifetime; in other words, the scheduler does not maintain affinity between a virtual thread and any particular platform thread. It means a virtual thread which is running over the Platform Thread A, and sleep for some time, when it continues, it might run Platform Thread B, or C, D.
  • Virtual threads are always daemon threads. The Thread.setDaemon(boolean) method cannot change a virtual thread to be a non-daemon thread.
  • Virtual threads have a fixed priority of Thread.NORM_PRIORITY. The Thread.setPriority(int) method has no effect on virtual threads.
  • Virtual threads have no permissions when running with a SecurityManager set.


Conclusion

Even though Virtual Threads are compatible to “Platform Threads” developer are not going to start refactoring their code to use it – If it’s working, don’t touch it – but new developments can take advantage of them. Also, third party libraries for reactive or asynchronous programming can start using them under the hood to provide a more scalable solution.

And I hope to see how lightweight Java web-servers (like tomcat, netty, jetty, and others) can start using Virtual Threads to provide an scalable solution to provide thousands of concurrent request. For us, the developers using those web-servers, there should be not change in the code, just take advantage of the new performance enhancements on them.

References

Post Views: 4,685

Advertisements
Tags

Leave a Reply

Your email address will not be published. Required fields are marked *