I mean of course Java is slower if you're including JVM start up time. A more modern version of the problem would ask for a service which performs the same function and averages time taken over N requests. I suspect modern java would look pretty close to C++/rust there
This is always an interesting question, about including the JVM startup time or not. In this context doesn't it seem like the JVM startup time should be included?
During JVM startup, one thing it's generally doing is allocating space for the heap from the OS. For the C/C++/Rust application, this could be implemented either as a large arena allocation, or allocations inline during the program. That is a programming choice, but the allocation cost is still there. It would be unfair to not include that allocation for Java whereas always including it (whether inline or upfront) in the native C/C++/Rust versions.
For long lived programs, like server-side things, the JVM startup time probably isn't relevant, but for short lived programs, it seems reasonable to include the startup time as that would be part of the time running the program from the CLI.
> for short lived programs, it seems reasonable to include the startup time
but then one can say, if the whole problem is so trivial that startup time is a significant chunk of the execution then is it really a problem worth optimising in the first place? The only situation where it would matter is if you are putting this in a tight loop on the command line or something like that (which is a thing, but its a reasonably small thing).
There are actually important cases where the JVM startup time is a huge problem. For example, my team chose Go instead of Java for a microservice solution because, among others, the startup time of a microservice after a scaling event or crash matters a lot for user perception. The difference between a 100ms startup time and a 500ms startup time (or more) is often the difference between a system that seems to 'just work' and a system that seems to 'constantly stutter'.
When my deployments are incremental I barely care about startup time. Even if a service takes ten seconds to come up healthy (which is mostly pubsub and polling backends, not classloading) we can upgrade the entire production tier 1% at a time in under twenty minutes, which is about as fast as I’d be comfortable going with automatic plus manual monitoring. And I don’t really expect sub-second response times and image load times from cluster schedulers like YARN or Apache Mesos or EC2.
I admit that Java is doing a lot of redundant bytecode verification and typechecking work at classload time, but nothing would really change if it were free. I certainly don’t want to give up plugins and AspectJ the way Go does.
I am by no means a devops expert, but it seems strange that 400ms would matter all that much, when the vm itself takes similar amounts of time for a new instance.
Either your application crashes way too much, or the scaling is not implemented properly.
That depends on what you're trying to evaluate. If you want a language that can do CLI tools, you should absolutely include the startup time. If you're doing a long lived service, you should benchmark a warmed-up JVM. Different environment require different performance characteristics.
With exception of Assembly all languages have a runtime, even C.
Who do you think calls main(), processes the command line arguments, runs functions registered via atexit(), emulates floating point when not available on CPU, processes signals on non-POSIX OSes, handles CPU faults?
The JVM startup time is discussed, and it mentions that a pre-compiled version using Graal might make for an interesting comparison. It wasn't done, because as mentioned in the blog post, things were getting out of hand with all the extra testing.
