57

I spent the last 18 months getting the grip of functional programming, starting with learning OCaml and for some weeks now Haskell. Now I want to take the next step and implement some actual application: A simple realtime terrain editor. I've written numerous realtime terrain rendering engines, so this is a familiar topic. And the used recursive algorithms and data structures seem very fit for a functional implementation.

With this being a realtime application I'm naturally looking for the best performance I can get. Now some proponents of OCaml quite frequently complain about Haskell being slow compared to OCaml or F#. But according to the The Computer Language Benchmarks Game Haskell oftenly beats OCaml, if only by rather small fractions. There remains the problem, that this benchmark takes only very specific samples.

The right thing to do would be to, of course, implement the program in both languages and compare them, but I simply don't want to do double the work.

But maybe other people did comparable applications in OCaml and Haskell and give some statistics?

Naman Goel
  • 1,525
  • 11
  • 16
datenwolf
  • 159,371
  • 13
  • 185
  • 298
  • 4
    @Trufa, I'd dare say that the DRY principle applies at smaller "granularities" than a whole application; it would make more sense at the method or class level. I agree with the OP that you can only directly compare two languages and platforms when both are essentially doing the exact same thing -- but of course this is a rather extreme way of finding out and not very practical in most cases. – stakx - no longer contributing Nov 29 '10 at 21:18
  • 4
    @Trufa: Writing two equivalent programs to compare their performance characteristics has nothing to do with the principle of DRY. DRY is about having one canonical source for a piece of information, not many independent sources. In this case, neither program by itself contains the necessary information, so DRY would only come into play if you had already written the comparison and were considering writing it again for some mysterious reason. Two is the minimum number of programs required for a comparison, so I don't see what it could possibly be "plain inefficient" in comparison to. – Chuck Nov 29 '10 at 21:19
  • @stakx, ok you are actually right (deleted my first comment to avoid confusions), Maybe it doest apply, I should have gone with dont repet yourself and no with the dont repet yourself principle :) I don´t know, but unless it is a very small piece of software, it does't make sense to me to build twice, just wanted to point that out. – Trufa Nov 29 '10 at 21:23
  • 1
    What do you mean by "realtime"? Depending on your response, Haskell might not be well-suited for your task. (And possibly not OCaml either.) – ephemient Nov 29 '10 at 21:26
  • "realtime" like in: Tesselated elevation grid with unlimited (well, only limited by memory) levels of detail, interactively editable with immediate response, i.e. update rate should be >30Hz. BTDT using C. Think along the lines of what flight simulators do to render terrain. – datenwolf Nov 29 '10 at 21:35
  • 2
    The language benchmarks game has been the subject of some controversy on the OCaml mailing list recently - some of the rules seem to affect OCaml more than others. For example the OCaml binary tree code runs in 47s. The OCaml tree code found in the "interesting alternative" section runs in 12.67s which would put it in second place behind C. The alt code tunes the garbage collector, which I guess goes against the rules for the binary tree code. I can't really comment on whether Haskell is faster than OCaml or not, but some of the code used for the benchmarks are sub-optimal. – Niki Yoshiuchi Nov 29 '10 at 21:54
  • Odd that I (and I'm sure quite a few other people) know exactly who you are talking about. Anyways, inb4 some drivel about hashmaps and the floor function in Haskell, and a subsequent statement that the whole language is useless because of it. – Paul Nov 29 '10 at 22:08
  • 2
    @Niki Yoshiuchi, the binary tree benchmark affects Haskell quite badly too since lazy tree is not allowed. – HaskellElephant Nov 29 '10 at 22:21
  • @Niki Yoshiuchi >> For example the OCaml binary tree code runs in 47s. The OCaml tree code found in the "interesting alternative" section runs in 12.67s << You seem to have ignored the OCaml binary tree code that runs in 15s? http://shootout.alioth.debian.org/u32q/program.php?test=binarytrees&lang=ocaml&id=2 – igouy Nov 29 '10 at 23:17
  • @igouy - I didn't ignore it, I wasn't aware of it. I am not trying to restart the controversy on the mailing list, I'm merely pointing out that there are rules to the benchmarks that don't necessarily apply to a terrain editor. – Niki Yoshiuchi Nov 29 '10 at 23:28
  • @Niki Yoshiuchi - Sorry but what you wrote wasn't about whether the benchmarks game rules apply to a terrain editor. What you wrote was about OCaml and the benchmarks game. – igouy Nov 30 '10 at 21:05
  • @datenwolf: If you want to study the shootout then make sure you read the source code. You may also find the Haskell wiki pages about it interesting: http://www.haskell.org/haskellwiki/Great_language_shootout – J D Dec 19 '10 at 12:43
  • 2
    The main point of interest is surely that OCaml's incremental GC incurs pauses of 10ms on average up to 30ms max (for well behaved code) whereas GHC's stop-the-world GC incurs arbitrarily-long pauses. Amazingly, nobody else has mentioned this except me and I got 3 downvotes for my troubles... – J D Mar 02 '11 at 20:25

4 Answers4

64

By all accounts, both OCaml and Haskell have sufficiently performant compilers and runtimes for almost anything. Picking between them on the basis of pure performance seems silly to me. You've come this far -- moving away from the obviously most low-level and performant languages (C, C++, etc.) in the name of clearer, more succinct, more expressive, higher-level code. So why, when the performance differences involved are much smaller, switch to that criteria now?

I'd go with some broader criteria -- if you want pervasive parallelism, then Haskell's the better choice. If you want genuinely pervasive mutation, then OCaml is better.

If you want only very coarse parallelism at best, and you intend to stick with mostly functional structures, then pick based on something else, like syntax (I think Haskell is much nicer here, but that's subjective) or available libraries (Haskell wins on quantity/availability, but OCaml might edge it out in the graphics department nonetheless).

I don't think you'll go wrong either way

sclv
  • 38,665
  • 7
  • 99
  • 204
  • 7
    "Picking between them on the basis of pure performance seems silly to me". Even if the performance difference can be orders of magnitude? – J D Dec 18 '10 at 15:07
  • 18
    But the benchmarks show them to be neck and neck. Not orders of magnitude off. – Theo Belaire Dec 19 '10 at 13:53
  • 13
    @Tyr: Depends which benchmarks you look at. This question is about soft real-time performance which means latency as well as throughput. GHC's stop-the-world GC will incur prohibitively long pause times for this kind of application, orders of magnitude longer than the pause times of OCaml's incremental GC. Everything else is superfluous. – J D Feb 28 '11 at 10:15
  • @Tyr where are these benchmarks you speak of? – pyCthon Aug 27 '12 at 16:35
  • I assume, these: http://shootout.alioth.debian.org/u32q/benchmark.php?test=all&lang=ocaml&lang2=ghc – sclv Aug 28 '12 at 14:33
  • 2
    C++11 isn't low level. – kirbyfan64sos Feb 11 '14 at 16:56
39

With help from two very smart colleagues, I've written a dataflow-optimization library in both Objective Caml and Haskell. The Haskell version is a bit more polymorphic, has more compile-time type checking, and therefore has less run-time checking. The OCaml version uses mutable state to accumulate dataflow facts, which might be faster or slower this week, depending on the phase of the moon. The key fact is that in their intended applications, both libraries are so fast that they are not worth fooling with. That is, in the respective compilers (Quick C-- and GHC), so little time is spent in dataflow optimization that the code is not worth improving.

Benchmarking is hell.

Norman Ramsey
  • 198,648
  • 61
  • 360
  • 533
  • 9
    You're talking solely about throughput and he wants latency. – J D Feb 27 '11 at 23:25
  • 2
    The link for C-- seems to be dead and now automatically redirects to a blog about skincare industry... – xji Aug 19 '15 at 08:35
25

I've written numerous realtime terrain rendering engines, so this is a familiar topic.

Familiar enough to know where most time will be spent?

If so then maybe you can write code for just that part in different languages and compare.

But according to the The Computer Language Benchmarks Game Haskell often beats OCaml, if only by rather small fractions — there remains the problem, that this benchmark takes only very specific samples.

The benchmarks game reports 4 sets of results - one core and quad core, 32 or 64 bit Ubuntu - and you may find that the OCaml or Haskell benchmark programs perform better or worse depending on the platform.

All a benchmark can do is take very specific samples, and of course you should disregard comparisons on tasks that are unlike where most time will be spent in your application - large integer arithmetic? regex? strings? - and look at the comparisons that are most like what you intend to do.

igouy
  • 2,547
  • 17
  • 16
  • Very familiar when it comes to do this kind of stuff imperatively. If it were just about terrain _rendering_ then a purely functional approach comes quite naturally, since all you do is basically running down some tree structure, emiting vertices and indices into lists. The interesting part is, when the data structures get changed… a lot. – datenwolf Nov 29 '10 at 23:56
  • 2
    >> when the data structures get changed… a lot << So can you take just that "interesting part" as the basis for performance experiments? Would that be enough to resolve whatever risks are associated with choosing between those implementation languages? – igouy Nov 30 '10 at 00:54
21

Based on all the data I've seen, they are roughly comparable. The code you write will make a bigger difference than the language itself.

Chuck
  • 234,037
  • 30
  • 302
  • 389
  • 1
    "Based on all the data I've seen". Would be great if you could link to some data because almost all of the data I've seen shows Haskell running significantly slower than OCaml. And I'm not talking about Haskell with the FFI hacks you see on the Great Computer Language Shootout. – J D Jan 31 '17 at 20:52