Did you consider running different VMs? One of the biggest improvements in the Java ecosystem, IMO, is the development of GraalVM. It looks like these tests are using the standard VMs, but in the benchmarks I've seen GraalVM is generally faster and might have a different comparison between 8 and 17.