13

Now that Chez Scheme is open-source, I wonder how it compares to Racket and other Schemes or languages in terms of performance, so that one could make informed choices about using them in one's projects.

Unfortunately, I couldn't find any relevant benchmarks.

I found the following:

https://ecraven.github.io/r7rs-benchmarks/benchmark.html

Problem: no Racket, or other languages (Update 10/13/18: Chez is now included in some of the benchmarks)

http://www.larcenists.org/benchmarksGenuineR6Linux.html

Problem: no Chez Scheme, or other languages

https://benchmarksgame-team.pages.debian.net/benchmarksgame/

Problem: only Racket, questionable comparisions (For example, Python is not allowed to use Numpy where it would clearly help, while Racket is making FFI calls to GMP)

So, none of the benchmarks I found allow you to compare Racket to Chez, for example, or Chez to SBCL, or Java. Are there Chez benchmarks that give you a sense of how fast it is?

Chez Scheme is often said to be the fastest Scheme/Lisp around. We should know if it's faster than, say, Java for your typical business logic application.

MWB
  • 11,740
  • 6
  • 46
  • 91
  • So what is it ou want to test R6RS, R7RS? Have you considered just downloading the top level test, compile it in the two and run them? – Sylwester Aug 01 '17 at 00:31
  • >>while Racket is making FFI calls to GMP<< Is Python allowed to make FFI calls to GMP? – igouy Aug 01 '17 at 16:28
  • @igouy Why are you asking me? Have you specified the rules of the benchmark, before renaming this thing into a "Game"? Why would you forbid using popular idiomatic libraries like Numpy and, at the same time, let people call any external C/Asm libraries? – MWB Aug 01 '17 at 21:17
  • @MaxB Just like Racket, Python is allowed to make FFI calls to GMP; and there's a Python program that does. – igouy Aug 01 '17 at 23:14
  • There is a bunch of Common Lisp benchmarks. Some of them should have been ported to Scheme. This project assembles https://common-lisp.net/project/cl-bench/ some of the benchmarks. The early Lisp benchmarks were the 'Gabriel Benchmarks', here included. Those are widely translated. – Rainer Joswig Aug 04 '17 at 17:34
  • @MaxB >>and, at the same time, let people call any external C/Asm libraries?<< There were language implementations that implemented arbitrary precision by bundling GMP, so it seemed unfair not to allow other language implementations to use GMP if they could. Level playing field. – igouy Sep 11 '18 at 15:09
  • It should be noted that the first link of this question **now includes Racket** benchmarks. – logc Sep 12 '18 at 17:28
  • @logc good to know. I guess one could use that to compare Racket to Chez, and the "not shootout" project to compare Racket to Java, C and SBCL. – MWB Sep 12 '18 at 19:26
  • all, please take the data shown on benchmarksgame site with a lot of salt (not just a pinch). I have it on very good authority that many of those results are not true: try out the codes by yourselves to make sure. –  Jul 20 '21 at 13:04

2 Answers2

4

It's anecdotal, but Matthew Flatt, the lead developer of Racket, thinks Chez is pretty good. You can read more about it here. He cites a regular expression matcher in which Chez is twice as fast as Racket and comparable to C.

user448810
  • 17,381
  • 4
  • 34
  • 59
  • I'd seen those. "Everyone" thinks Chez is pretty good, but is it in the same league as Java, or at least SBCL? – MWB Aug 04 '17 at 16:02
  • I don't think you'll find a controlled study, only anecdotes. But Matthew Flatt is hardly "everyone." If he thinks Chez is sufficiently better than Racket to abandon the native compiler in Racket in favor of Chez, I'm inclined to believe him. My experience is that Chez is as fast as anything else, and also rock-solid, in terms of not having bugs, which is at least as important as raw speed. – user448810 Aug 04 '17 at 16:15
4

Kent Dybvig has written articles on the implementation Chez Scheme. They'll often have comparisons with other implementations:

https://www.cs.indiana.edu/~dyb/

MWB
  • 11,740
  • 6
  • 46
  • 91
soegaard
  • 30,661
  • 4
  • 57
  • 106