benchmarks? java vs .net

java-programmer, benchmarks? java vs .net On Tue, 03 Jun 2008 16:18:34 +0100, Jon Harrop <email***@***.com>
wrote:

>There is no merit in comparing unoptimized code on .NET with optimized Java.

That's why posted it here so you can "optimize it". You fixed
mandelbrot but C# is still twice slower in three other benchmarks:

binarytrees
(the command line argument should be -server -Xms64m binarytrees)

http://shootout.alioth.debian.org/gp4sandbox/benchmark.php?test=binarytrees&lang=javaxx&id=2
vs
http://shootout.alioth.debian.org/gp4sandbox/benchmark.php?test=binarytrees&lang=csharp&id=0

revcomp
http://shootout.alioth.debian.org/gp4sandbox/benchmark.php?test=revcomp&lang=javaxx&id=4
vs
http://shootout.alioth.debian.org/gp4sandbox/benchmark.php?test=revcomp&lang=csharp&id=2

sumcol
http://shootout.alioth.debian.org/gp4sandbox/benchmark.php?test=sumcol&lang=javaxx&id=4
vs
http://shootout.alioth.debian.org/gp4sandbox/benchmark.php?test=sumcol&lang=csharp&id=0

also, C# significantly slower in recursion

http://shootout.alioth.debian.org/gp4sandbox/benchmark.php?test=recursive&lang=javaxx&id=0
vs
http://shootout.alioth.debian.org/gp4sandbox/benchmark.php?test=recursive&lang=csharp&id=0

>I have seen is on the Mersenne Twister PRNG where Java is 2x slower than .NET.

Post the benchmark. Let me see...

java-programmer >> benchmarks? java vs .net On Tue, 03 Jun 2008 16:18:34 +0100, Jon Harrop <email***@***.com>
wrote:

>There is no merit in comparing unoptimized code on .NET with optimized Java.

That's why posted it here so you can "optimize it". You fixed
mandelbrot but C# is still twice slower in three other benchmarks:

binarytrees
(the command line argument should be -server -Xms64m binarytrees)

http://shootout.alioth.debian.org/gp4sandbox/benchmark.php?test=binarytrees&lang=javaxx&id=2
vs
http://shootout.alioth.debian.org/gp4sandbox/benchmark.php?test=binarytrees&lang=csharp&id=0

revcomp
http://shootout.alioth.debian.org/gp4sandbox/benchmark.php?test=revcomp&lang=javaxx&id=4
vs
http://shootout.alioth.debian.org/gp4sandbox/benchmark.php?test=revcomp&lang=csharp&id=2

sumcol
http://shootout.alioth.debian.org/gp4sandbox/benchmark.php?test=sumcol&lang=javaxx&id=4
vs
http://shootout.alioth.debian.org/gp4sandbox/benchmark.php?test=sumcol&lang=csharp&id=0

also, C# significantly slower in recursion

http://shootout.alioth.debian.org/gp4sandbox/benchmark.php?test=recursive&lang=javaxx&id=0
vs
http://shootout.alioth.debian.org/gp4sandbox/benchmark.php?test=recursive&lang=csharp&id=0

>I have seen is on the Mersenne Twister PRNG where Java is 2x slower than .NET.

Post the benchmark. Let me see...

java-programmer >> benchmarks? java vs .net On Tue, 3 Jun 2008 09:19:22 -0700 (PDT), "Jon Skeet [C# MVP]"
<email***@***.com> wrote:

>If Razii is genuinely comparing Java with Mono (I haven't looked at
>any of the figures) it's a silly test to start with (unless you're
>specifically interested in Mono, of course).

If you have no clue what is this thread about, why post at all?

java-programmer >> benchmarks? java vs .net Razii <email***@***.com> wrote:
> >If Razii is genuinely comparing Java with Mono (I haven't looked at
> >any of the figures) it's a silly test to start with (unless you're
> >specifically interested in Mono, of course).
>
> If you have no clue what is this thread about, why post at all?

Well gee, you keep post URLs comparing Mono with Java. That's *not*
comparing .NET with Java - and is thus relatively pointless IMO. (It's
also made more pointless by you not mentioning in your blog post which
version of Java you're using, or which version of .NET.)

To make the conversation significant (well, as significant as this kind
of thing can be):

1) Don't bother posting about the Mono benchmarks at all. Keep it to
.NET and Java.
2) Explain the precise runtime environments.

The IO suggestions so far seem to be an indication of the general merit
of the benchmark, mind you.

--
Jon Skeet - <email***@***.com>
Web site: http://www.pobox.com/~skeet
Blog: http://www.msmvps.com/jon.skeet
C# in Depth: http://csharpindepth.com

java-programmer >> benchmarks? java vs .net Razii wrote:
> On Tue, 03 Jun 2008 16:08:17 +0100, Jon Harrop <email***@***.com>
> wrote:
>
>> BufferedStream s = new BufferedStream(Console.OpenStandardOutput());
>>
>>and the entire program becomes 2.5x faster.
>
> Yes, that was faster. I submitted it to shootout site. I am not sure
> if it will be faster on Mono too but I will let Gouy test it.

It makes no difference on Mono and the shootout does not test .NET.

--
Dr Jon D Harrop, Flying Frog Consultancy
http://www.ffconsultancy.com/products/?u

java-programmer >> benchmarks? java vs .net On Tue, 3 Jun 2008 18:51:10 +0100, Jon Skeet [C# MVP]
<email***@***.com> wrote:

>Well gee, you keep post URLs comparing Mono with Java. That's *not*
>comparing .NET with Java - and is thus relatively pointless IMO. (It's
>also made more pointless by you not mentioning in your blog post which
>version of Java you're using, or which version of .NET.)

The link I posted was this:

http://kingrazi.blogspot.com/

It's clear what it is about.

java-programmer >> benchmarks? java vs .net On Tue, 3 Jun 2008 18:51:10 +0100, Jon Skeet [C# MVP]
<email***@***.com> wrote:

>The IO suggestions so far seem to be an indication of the general merit
>of the benchmark, mind you.

The criteria is same for both sides. For now C# is slower by wide
margin in these benchmarks. How about fixing them? If you can't, I
will conclude C# is just slower.

binarytrees
(the command line argument should be -server -Xms64m binarytrees)

http://shootout.alioth.debian.org/gp4sandbox/benchmark.php?test=binarytrees&lang=javaxx&id=2
vs
http://shootout.alioth.debian.org/gp4sandbox/benchmark.php?test=binarytrees&lang=csharp&id=0

revcomp
http://shootout.alioth.debian.org/gp4sandbox/benchmark.php?test=revcomp&lang=javaxx&id=4
vs
http://shootout.alioth.debian.org/gp4sandbox/benchmark.php?test=revcomp&lang=csharp&id=2

sumcol
http://shootout.alioth.debian.org/gp4sandbox/benchmark.php?test=sumcol&lang=javaxx&id=4
vs
http://shootout.alioth.debian.org/gp4sandbox/benchmark.php?test=sumcol&lang=csharp&id=0

also, C# significantly slower in recursion

http://shootout.alioth.debian.org/gp4sandbox/benchmark.php?test=recursive&lang=javaxx&id=0
vs
http://shootout.alioth.debian.org/gp4sandbox/benchmark.php?test=recursive&lang=csharp&id=0

java-programmer >> benchmarks? java vs .net Jon Harrop wrote:
> Mark Thornton wrote:
>> Rather amusing really as unintentional use of unbuffered IO is a
>> frequent cause of Java benchmarks running more slowly than they should.
>> It seems that .NET copied that characteristic as well.
>
> Yes. I've no idea why they do that. Isn't buffered IO a better default?!
>

I think the reasoning is simplicity --- you create what you want through
composition of a smaller number of classes. Otherwise you need more
classes or extra options on constructors to provide for all possible
cases. Unfortunately they spoiled this a bit by providing a few
composites (java.io.FileReader for example). It is also a bit confusing
that InputStreamReader includes some buffering, but adding buffering to
an already buffered stream doesn't usually increase the cost by much.

Mark Thornton

java-programmer >> benchmarks? java vs .net Razii wrote:
> The criteria is same for both sides. For now C# is slower by wide
> margin in these benchmarks. How about fixing them?

I did.

> If you can't, I will conclude C# is just slower.

You had drawn that conclusion before you even tried to measure anything.

--
Dr Jon D Harrop, Flying Frog Consultancy
http://www.ffconsultancy.com/products/?u

java-programmer >> benchmarks? java vs .net Mark Thornton wrote:
> Jon Skeet [C# MVP] wrote:
>>> Does Java have anything comparable to .NET's Task Parallel Library?
>>
>> I haven't used it, but I've heard about Doug Lea's Fork/Join
>> framework:
>> http://gee.cs.oswego.edu/dl/papers/fj.pdf
>>
> I have used it and find it works quite well even without closures. I
> haven't used .NET so can't compare them.

The Task Parallel Library is awesome, even though it is only a CTP for now.
I highly recommend it. For example, read the article:

"Surviving in the multicore era with Microsoft's ParallelFX" - The F#.NET
Journal, 30th April 2008
http://www.ffconsultancy.com/products/fsharp_journal/?cs

--
Dr Jon D Harrop, Flying Frog Consultancy
http://www.ffconsultancy.com/products/?u

java-programmer >> benchmarks? java vs .net Jon Skeet [C# MVP] wrote:
>
>> Does Java have anything comparable to .NET's Task Parallel Library?
>
> I haven't used it, but I've heard about Doug Lea's Fork/Join
> framework:
> http://gee.cs.oswego.edu/dl/papers/fj.pdf
>
I have used it and find it works quite well even without closures. I
haven't used .NET so can't compare them.

For more information and downloads of the package:
http://g.oswego.edu/dl/concurrency-interest/

Mark Thornton

java-programmer >> benchmarks? java vs .net Razii <email***@***.com> wrote:
> On Tue, 3 Jun 2008 18:51:10 +0100, Jon Skeet [C# MVP]
> <email***@***.com> wrote:
>
> >Well gee, you keep post URLs comparing Mono with Java. That's *not*
> >comparing .NET with Java - and is thus relatively pointless IMO. (It's
> >also made more pointless by you not mentioning in your blog post which
> >version of Java you're using, or which version of .NET.)
>
> The link I posted was this:
>
> http://kingrazi.blogspot.com/
>
> It's clear what it is about.

That was the link you posted *initially*. Since then you have posted
multiple links to http://shootout.alioth.debian.org which is about
Mono.

--
Jon Skeet - <email***@***.com>
Web site: http://www.pobox.com/~skeet
Blog: http://www.msmvps.com/jon.skeet
C# in Depth: http://csharpindepth.com

java-programmer >> benchmarks? java vs .net Razii wrote:
> On Tue, 03 Jun 2008 16:08:17 +0100, Jon Harrop <email***@***.com>
> wrote:
>> Java is still over 2x slower on both partialsums and
>>the Mersenne Twister.
>
> The partialsums is only slower due to trig accuracy required by Java's
> specification. Your C# results are not accurate enough to satisfy
> Java's specification. It's easy to solve this problem. Try this Java
> version of partialsums. It's two times faster.

But still 2x slower than everything else and now 2x more bloated as well:

C 1.300s
C# 1.363s
Java 2.750s
Java 3.840s (original)

> In binarytrees C# is twice slower. You haven't posted any workaround
> yet.

Optimizing binarytrees is trivial because the benchmark is fundamentally
flawed. I described this in detail on the shootout mailing list and urged
the maintainers to persue well defined benchmarks but they did not take
heed.

--
Dr Jon D Harrop, Flying Frog Consultancy
http://www.ffconsultancy.com/products/?u

java-programmer >> benchmarks? java vs .net Razii wrote:
> On Tue, 03 Jun 2008 19:37:57 +0100, Jon Harrop <email***@***.com>
> wrote:
>>But still 2x slower than everything else and now 2x more bloated as well:
>>
>>C 1.300s
>>C# 1.363s
>>Java 2.750s
>
> that's not what I get
>
> time java -server partialsums 2500000
>
> 0m1.697s
>
> and for Mono I get
>
> 0m1.525s

Yes. Mono is extremely slow. Almost as slow as Java on this benchmark. As we
have seen, .NET is much faster...

--
Dr Jon D Harrop, Flying Frog Consultancy
http://www.ffconsultancy.com/products/?u

java-programmer >> benchmarks? java vs .net On Tue, 03 Jun 2008 19:37:57 +0100, Jon Harrop <email***@***.com>
wrote:

>But still 2x slower than everything else and now 2x more bloated as well:
>
>C 1.300s
>C# 1.363s
>Java 2.750s

that's not what I get

time java -server partialsums 2500000

0m1.697s

and for Mono I get

0m1.525s

java-programmer >> benchmarks? java vs .net Razii <email***@***.com> wrote:
> >The IO suggestions so far seem to be an indication of the general merit
> >of the benchmark, mind you.
>
> The criteria is same for both sides.

Um, the C# version (as posted on the site you've linked to below) uses
unbuffered IO while the Java version doesn't. They're just not doing
the same thing. Even if they were, that wouldn't give any meaningful
data about which language/platform "in general". It might help if you
happen to know that your code is going to require a huge amount of
recursion - to the extent that it's going to be the performance
bottleneck - *and* performance is your primary concern.

> For now C# is slower by wide margin in these benchmarks. How about
> fixing them? If you can't, I will conclude C# is just slower.

Feel free to draw that conclusion, while the rest of us instead
conclude that you can't claim that C# (which is a *language* after all,
not a platform) is slower or faster than another language.

Even running .NET rather than Mono will only go so far - you'd need to
run it on both 64 bit and 32 bit platforms, as the runtimes do
different things. (IIRC the 64 bit CLR uses tail recursion more heavily
than the 32 bit CLR, for instance.)

If I were to find a really slow JVM, would that prove that Java is a
slower *language* than C#?

I can easily write a benchmark where Java (under Hotspot) "beats" .NET:
just call a very small virtual method which is never overridden.
Hotspot will inline the call aggressively, as it is able to "undo" the
optimisation later on. .NET, with a single-pass JIT, won't do that.
Does that prove that Java is a faster language? No, not at all. It just
shows one area situation where it works better. It pretty much has to,
given that far more methods tend to be virtual in Java code than in
.NET code. I can probably find similar situations where .NET stomps
over Java (being able to create custom value types would probably be a
good starting point - force heap allocation and garbage collection in
Java but not .NET; even though it's fast, it's not free) - but that
wouldn't prove that .NET is faster than Java, either.

--
Jon Skeet - <email***@***.com>
Web site: http://www.pobox.com/~skeet
Blog: http://www.msmvps.com/jon.skeet
C# in Depth: http://csharpindepth.com

java-programmer >> benchmarks? java vs .net Mark Thornton <email***@***.com> wrote:
> > I haven't used it, but I've heard about Doug Lea's Fork/Join
> > framework:
> > http://gee.cs.oswego.edu/dl/papers/fj.pdf
>
> I have used it and find it works quite well even without closures.

Cool - that's good to hear. I'd be very surprised if closures didn't
make it even simpler to use (possibly with a bit of refactoring towards
single-method interfaces if necessary).

Given the rest of Doug Lea's work, I'd expect it to be good :)

> I haven't used .NET so can't compare them.

Here's a little example from a Mandelbrot benchmark I was writing a few
weeks ago (oh the irony).

Simple initial code:

public override void Generate()
{
int index = 0;
for (int row = 0; row < Height; row++)
{
for (int col = 0; col < Width; col++)
{
Data[index++] = ComputeMandelbrotIndex(row, col);
}
}
}

Now using Parallel Extensions, and the lamdba expressions of C# 3:

public override void Generate()
{
Parallel.For(0, Height, row =>
{
int index = row * Width;
for (int col = 0; col < Width; col++)
{
Data[index++] = ComputeMandelbrotIndex(row, col);
}
});
}

I can't imagine many transformations being simpler than that - and the
results are great.

See http://preview.tinyurl.com/58vfav for rather more :)

> For more information and downloads of the package:
> http://g.oswego.edu/dl/concurrency-interest/

Thanks, will take a look.

--
Jon Skeet - <email***@***.com>
Web site: http://www.pobox.com/~skeet
Blog: http://www.msmvps.com/jon.skeet
C# in Depth: http://csharpindepth.com

java-programmer >> benchmarks? java vs .net On Tue, 03 Jun 2008 19:33:41 +0100, Jon Harrop <email***@***.com>
wrote:

>> If you can't, I will conclude C# is just slower.
>
>You had drawn that conclusion before you even tried to measure anything.

What the heck? I measured everything and posted the linhk. That's how
the thread started.

BufferedStream didn't help in sumcol

using (StreamReader r = new StreamReader(new
BufferedStream(Console.OpenStandardInput())))

makes no difference.

The only fix you have made so far is to mandelbrot. What about the 4
other I mentioned?

java-programmer >> benchmarks? java vs .net Jon Harrop <email***@***.com> wrote:
> Mark Thornton wrote:
> > Jon Skeet [C# MVP] wrote:
> >>> Does Java have anything comparable to .NET's Task Parallel Library?
> >>
> >> I haven't used it, but I've heard about Doug Lea's Fork/Join
> >> framework:
> >> http://gee.cs.oswego.edu/dl/papers/fj.pdf
> >>
> > I have used it and find it works quite well even without closures. I
> > haven't used .NET so can't compare them.
>
> The Task Parallel Library is awesome, even though it is only a CTP for now.

Agreed. Now is a particularly good time to get into it, as the second
CTP has just been released:

http://blogs.msdn.com/pfxteam/archive/2008/06/02/8567093.aspx

> I highly recommend it. For example, read the article:
>
> "Surviving in the multicore era with Microsoft's ParallelFX" - The F#.NET
> Journal, 30th April 2008
> http://www.ffconsultancy.com/products/fsharp_journal/?cs

Or not, given that that's subscription only...

--
Jon Skeet - <email***@***.com>
Web site: http://www.pobox.com/~skeet
Blog: http://www.msmvps.com/jon.skeet
C# in Depth: http://csharpindepth.com

java-programmer >> benchmarks? java vs .net On Tue, 3 Jun 2008 19:37:47 +0100, Jon Skeet [C# MVP]
<email***@***.com> wrote:

>That was the link you posted *initially*. Since then you have posted
>multiple links to http://shootout.alioth.debian.org which is about
>Mono.

Huh? I will be polite for now and just say you didn't read the thread
carefully. The benchmark I tested are at that site. I don't have Mono.
I only have .NET.

java-programmer >> benchmarks? java vs .net On Tue, 03 Jun 2008 19:37:57 +0100, Jon Harrop <email***@***.com>
wrote:

>But still 2x slower than everything else and now 2x more bloated as well:

The FastMath class can be separated and used instead of java's Math in
all cases. The bloat is ZERO.

java-programmer >> benchmarks? java vs .net On Tue, 03 Jun 2008 19:48:16 +0100, Jon Harrop <email***@***.com>
wrote:

>Yes. Mono is extremely slow. Almost as slow as Java on this benchmark. As we
>have seen, .NET is much faster...

That was a typo. I don't have Mono. I get that with .NET.

java-programmer >> benchmarks? java vs .net Razii wrote:
> On Tue, 03 Jun 2008 19:33:41 +0100, Jon Harrop <email***@***.com>
> wrote:
>>> If you can't, I will conclude C# is just slower.
>>
>>You had drawn that conclusion before you even tried to measure anything.
>
> What the heck? I measured everything...

You only measured unoptimized C#. I have optimized the C# for you but your
response has only been to cherry pick results that back the conclusion you
wanted to draw.

> The only fix you have made so far is to mandelbrot.

Have you already forgotten that you were not compiling the regular
expressions in the C# implementation of regexdna?

> What about the 4 other I mentioned?

You said ".NET is twice slower in four benchmarks: binarytrees, mandelbrot,
regexdna, sumcol.". I have already corrected two of your results (mandelbrot
and regexdna) and provided two more counter examples where .NET is >2x
faster than Java and explained why binarytrees is flawed.

--
Dr Jon D Harrop, Flying Frog Consultancy
http://www.ffconsultancy.com/products/?u

java-programmer >> benchmarks? java vs .net On Tue, 3 Jun 2008 19:50:21 +0100, Jon Skeet [C# MVP]
<email***@***.com> wrote:

>Um, the C# version (as posted on the site you've linked to below) uses
>unbuffered IO while the Java version doesn't. They're just not doing
>the same thing.

What benchmark are you talking about? Changing the line to

using (StreamReader r = new StreamReader(new
BufferedStream(Console.OpenStandardInput())))

didn't make a difference in sum-file benchmark.

By the way, StreamTokenizer is not buffered. It reads a byte each time
from the stream. It maybe that System.in is buffered by default in
Java.

java-programmer >> benchmarks? java vs .net Razii wrote:
> On Tue, 03 Jun 2008 19:48:16 +0100, Jon Harrop <email***@***.com>
> wrote:
>>Yes. Mono is extremely slow. Almost as slow as Java on this benchmark. As
>>we have seen, .NET is much faster...
>
> That was a typo. I don't have Mono. I get that with .NET.

Java is 2x slower than .NET on my AMD64 for this benchmark. What CPU are you
using?

--
Dr Jon D Harrop, Flying Frog Consultancy
http://www.ffconsultancy.com/products/?u

java-programmer >> benchmarks? java vs .net Razii wrote:
> On Tue, 03 Jun 2008 19:37:57 +0100, Jon Harrop <email***@***.com>
> wrote:
>>But still 2x slower than everything else and now 2x more bloated as well:
>
> The FastMath class can be separated and used instead of java's Math in
> all cases. The bloat is ZERO.

So having to write your own basic trig functions in Java so that you can be
only 2x slower than other languages doesn't bother you?

--
Dr Jon D Harrop, Flying Frog Consultancy
http://www.ffconsultancy.com/products/?u