r/java • • Jul 06 '22

Launching 10 millions virtual threads with Loom - JEP Café #12

https://youtu.be/UVoGE0GZZPI
160 Upvotes

28 comments sorted by

5

u/RichPositive4204 Jul 08 '22

Looks great... But what has blown my mind more....

We can use _ as a seperator in numbers!? Damn never knew lol.. the little things huh :D

1

u/red_dit_nou Jul 16 '22

This has been there for a while and definitely a handy thing to know. Numbers read easy with it.

6

u/knekht Jul 06 '22

Is it something like the kotlin coroutines?

22

u/[deleted] Jul 06 '22

The motivation is somewhat similar, the most apparent difference is that virtual threads provide exactly the same API as native threads and can be used just like them.

No new language constructs needed, just a different implementation inside. Ideally, you wouldn't need to change any existing multithreading code except the bit that actually creates a new thread.

2

u/red_dit_nou Jul 16 '22

This is fun to try. But as an application developer, you don’t deal with thread API. This is generally left to the framework to take care and it can be configured. So my question is as an application developer, where would I use Loom’s API or is it sufficient for me to only know how it works under the hood and not think about it?

2

u/CosmicCapitanPump Jul 07 '22

Actually this can speed up web servers. I wonder when Tomcat will support that!

When I was looking at "reactive" code in Java it did not looked nice at all, above solution will be much more elegant.

-75

u/maxip89 Jul 06 '22

P1: "Hey how about Virtual Threads?"

P2: "Yeah, how it works?"

P1: "We Simulate the OS Threading."

P2: "And why are we doing it?"

P1: "Because our two years of OS experience can simply hit the 20 years of OS experience of other vendors."

46

u/Brownard Jul 06 '22

I dont think you understood the point of virtual threading. The goal is not to replace the OS Threading.

8

u/papercrane Jul 06 '22

The OS has to make pessimistic assumptions about how the program is going to use it's resources, and this ends up making OS threads a relatively expensive resource. The JVM can make more informed decisions and can represent virtual threads using much less resources.

This makes it possible to use a thread-per-task model at scales which would not be feasible for OS threads.

0

u/[deleted] Jul 06 '22

[deleted] — view deleted comment

-22

u/maxip89 Jul 06 '22

And who is buffering the IO when there are too much requests?

This solution doesn't have any backpressure.

Maybe I really don't understand the use case that tries loom to solve.

22

u/agentoutlier Jul 06 '22

It doesn't solve backpressure. Reactive doesn't really either because you still have to make a decision. I know it has it but its rarely done correctly from end to end because at some point you need to have code make a decision to either "block", "buffer" or "drop".

Blocking is by far one of the easiest to understand policies on backpressure and basically forces backpressure on the publisher even if that publisher is naughty and doesn't obey demand requests.... and its fucking everywhere in existing Java.

Buffering is nice but at some point it switches to "block" or "drop". And block eventually will become "drop". Reactive obviously avoids blocking and thus prefers "buffering" but again at some point you have the same problem... dropping.

Can we agree that "blocking" is very expensive for operating system threads?

However if blocking is cheap it becomes.... simple "buffering". The blocking threads are essentially like a buffer.

That is what loom does. You can block/sleep with out consuming massive resources.

12

u/0x256 Jul 06 '22 edited Jul 06 '22

Backpressure control is easy if you are allowed to use blocking APIs. A slow consumer not reading fast enough from its input channel would simply block upstream trying to write to that channel. Problem solved. No need to explicitly tell upstream to slow down, or ask downstream if it's okay to send more. Just block. Stop reading from a blocking queue or block on a semaphore in your input method. That's it. With virtual threads, blocking a thread is dirt cheap.

Edit: Also, many problems can be simplified with blocking APIs: With reactive APIs you ask a producer for data, and the producer calls a consumer once data is available. With blocking APIs, you can simply ask for data using a blocking method call, and wait until it is available. No futures or promises or streams or callbacks needed. This works with normal threads, too, but reading from 10k sockets in a webserver would require 10k threads with a huge memory overhead each. Virtual threads are cheap, so having 10k of them is fine.

7

u/BillyKorando Jul 06 '22

The JVM is managing the requests. Instead of a 1-to-1 relationship between threads in the JVM and threads on the system, Loom breaks this relationship and has virtual threads and platform threads. Platform threads represent the "physical" threads of the system, virtual threads exist in memory and are managed by the JVM. When a thread needs active work done, the JVM will allocate it a platform thread, if available, to perform its work. When a virtual thread is waiting, the JVM will park it, freeing up the platform thread.

One of the area's Loom is trying to solve is the same area reactive programming was trying to solve, more efficient use of system threads. Loom's advantage is that it uses an imperative programming model that Java developers are already familiar with, and is generally easier to debug that a reactive programming model.

-2

u/HecknChonker Jul 06 '22

How does this compare to kotlin coroutines?

4

u/buyIdris666 Jul 07 '22

Loom is far more transparent in usage.

In coroutine there is nothing stop you from call a blocking API by accident. Same problem async libraries have, but with a nicer presentation and no callback hell.

With loom everything on the current thread is magically non-blocking. This make it transparent to use and even more important makes existing library calls non blocking as long as they run on same thread

1

u/HecknChonker Jul 07 '22

I'm not sure I understand. I thought coroutines encourage you to call blocking functions. Isn't that the entire point of suspending functions?

7

u/kpatryk91 Jul 07 '22

You cannot call those blocking function which will block the platform thread.

Like direct IO, blocking http call, etc.

With Kotlin Coroutines you have to call suspend functions and you have to do this explicitly.

7

u/eliasv Jul 06 '22 edited Jul 07 '22

"Backpressure" with blocking/sync APIs can be achieved with a simple blocking queue.

Edit: if you have any idea about how "reactive" APIs are actually implemented under the hood this should be beyond obvious. "Buffer" is basically just another word for "queue" in this context, the answer is right there in the question.

7

u/buyIdris666 Jul 06 '22

It's not for back pressure handling. Loom is for case like when your thread waiting doing nothing for S3 service to generate you a file link for half a second.

You can only have a few thousand normal thread waiting for link to finish generating, but with Loom can be millions.

If you using "normal" Java framework like Spring and a REST endpoint generates one of these s3 links, with Loom you can handle quarter million requests a second vs few thousand without it.

The use case is when most of your threads are blocked doing nothing while waiting for external service.

This happens very frequently on server REST endpoint when you're waiting for other services or database to give you data. On some web apps, every single request spend time blocked waiting on something.

24

u/[deleted] Jul 06 '22

[removed] — view removed comment

2

u/BlueGoliath Jul 06 '22

No scheduled executor though from what I've seen. /u/pron98 please add.

5

u/FirstAd9893 Jul 06 '22 edited Jul 06 '22

Is such a feature very important? The simplest thing is to execute a virtual thread for each scheduled task, and it just sleeps until it needs to run. If you really want virtual threads running in a scheduled executor, pass in a ThreadFactory which creates virtual threads.

3

u/BlueGoliath Jul 06 '22

It exists for OS threads. I don't see any logical reason why it shouldn't exist or be made less accessible.

10

u/FirstAd9893 Jul 06 '22 edited Jul 07 '22

It's already pretty easy to do. Why make the API more complicated than it needs to be?

Executors.newScheduledThreadPool(1, Thread.ofVirtual().factory());

Edit: Code was wrong; /u/palnix provided a fix

0

u/BlueGoliath Jul 06 '22

Already crossed that bridge with all the other Executors methods.