Python vs. Ruby for metaprogramming

Question

I'm currently primarily a D programmer and am looking to add another language to my toolbox, preferably one that supports the metaprogramming hacks that just can't be done in a statically compiled language like D.

I've read up on Lisp a little and I would love to find a language that allows some of the cool stuff that Lisp does, but without the strange syntax, etc. of Lisp. I don't want to start a language flame war, and I'm sure both Ruby and Python have their tradeoffs, so I'll list what's important to me personally. Please tell me whether Ruby, Python, or some other language would be best for me.

Important:

1. Good metaprogramming. Ability to create classes, methods, functions, etc. at runtime. Preferably, minimal distinction between code and data, Lisp style.
2. Nice, clean, sane syntax and consistent, intuitive semantics. Basically a well thought-out, fun to use, modern language.
3. Multiple paradigms. No one paradigm is right for every project, or even every small subproblem within a project.
4. An interesting language that actually affects the way one thinks about programming.

Somewhat important:

1. Performance. It would be nice if performance was decent, but when performance is a real priority, I'll use D instead.
2. Well-documented.

Not important:

1. Community size, library availability, etc. None of these are characteristics of the language itself, and all can change very quickly.
2. Job availability. I am not a full-time, professional programmer. I am a grad student and programming is tangentially relevant to my research.
3. Any features that are primarily designed with very large projects worked on by a million code monkeys in mind.

Answer 1

I've read up on Lisp a little and I would love to find a language that allows some of the cool stuff that Lisp does, but without the strange syntax, etc. of Lisp.

Wouldn't we all.

minimal distinction between code and data, Lisp style

Sadly, the minimal distinction between code and data and "strange" syntax are consequences of each other.

If you want easy-to-read syntax, you have Python. However, the code is not represented in any of the commonly-used built-in data structures. It fails—as most languages do—in item #1 of your 'important' list. That makes it difficult to provide useful help.

You can't have it all. Remember, you aren't the first to have this thought. If something like your ideal language existed, we'd all be using it. Since the real world falls short of your ideals, you'll have to re-prioritize your wish list. The "important" section has to be rearranged to identify what's really important to you.

Answer 2

Honestly, as far as metaprogramming facilities go, Ruby and Python are a lot more similar than some of their adherent like to admit. This review of both language offers a pretty good comparison/review:

• http://regebro.wordpress.com/2009/07/12/python-vs-ruby/

So, just pick one based on some criteria. Maybe you like Rails and want to study that code. Maybe SciPy is your thing. Look at the ecosystem of libraries, community, etc, and pick one. You certainly won't lose out on some metaprogramming nirvana based on your choice of either.

Answer 3

Disclaimer: I only dabble in either language, but I have at least written small working programs (not just quick scripts, for which I use Perl, bash or GNU make) in both.

Ruby can be really nice for the "multiple paradigms" point 3, because it works hard to make it easy to create domain-specific languages. For example, browse online and look at a couple of bits of Ruby on Rails code, and a couple of bits of Rake code. They're both Ruby, and you can see the similarities, but they don't look like what you'd normally think of as the same language.

Python seems to me to be a bit more predictable (possibly correlated to 'clean' and 'sane' point 2), but I don't really know whether that's because of the language itself or just that it's typically used by people with different values. I have never attempted deep magic in Python. I would certainly say that both languages are well thought out.

Both score well in 1 and 4. [Edit: actually 1 is pretty arguable - there is "eval" in both, as common in interpreted languages, but they're hardly conceptually pure. You can define closures, assign methods to objects, and whatnot. Not sure whether this goes as far as you want.]

Personally I find Ruby more fun, but in part that's because it's easier to get distracted thinking of cool ways to do things. I've actually used Python more. Sometimes you don't want cool, you want to get on with it so it's done before bedtime...

Neither of them is difficult to get into, so you could just decide to do your next minor task in one, and the one after that in the other. Or pick up an introductory book on each from the library, skim-read them both and see what grabs you.

Answer 4

There's not really a huge difference between python and ruby at least at an ideological level. For the most part, they're just different flavors of the same thing. Thus, I would recommend seeing which one matches your programming style more.

Answer 5

Have you considered Smalltalk? It offers a very simple, clear and extensible syntax with reflectivity and introspection capabilities and a fully integrated development environment that takes advantage of those capabilities. Have a look at some of the work being done in Squeak Smalltalk for instance. A lot of researchers using Squeak hang out on the Squeak mailing list and #squeak on freenode, so you can get help on complex issues very easily.

Other indicators of its current relevance: it runs on any platform you'd care to name (including the iPhone); Gilad Bracha is basing his Newspeak work on Squeak; the V8 team cut their teeth on Smalltalk VMs; and Dan Ingalls and Randal Schwartz have recently returned to Smalltalk work after years in the wilderness.

Best of luck with your search - let us know what you decide in the end.

Answer 6

Lisp satisfies all your criteria, including performance, and it is the only language that doesn't have (strange) syntax. If you eschew it on such an astoundingly ill-informed/wrong-headed basis and consequently miss out on the experience of using e.g. Emacs+SLIME+CL, you'll be doing yourself a great disservice.

Answer 7

Your 4 "important" points lead to Ruby exactly, while the 2 "somewhat important" points ruled by Python. So be it.

Answer 8

You are describing Ruby.

• Good metaprogramming. Ability to create classes, methods, functions, etc. at runtime. Preferably, minimal distinction between code and data, Lisp style.

It's very easy to extend and modify existing primitives at runtime. In ruby everything is an object, strings, integers, even functions.

You can also construct shortcuts for syntactic sugar, for example with class_eval.

• Nice, clean, sane syntax and consistent, intuitive semantics. Basically a well thought-out, fun to use, modern language.

Ruby follows the principle of less surprise, and when comparing Ruby code vs the equivalent in other language many people consider it more "beautiful".

• Multiple paradigms. No one paradigm is right for every project, or even every small subproblem within a project.

You can follow imperative, object oriented, functional and reflective.

• An interesting language that actually affects the way one thinks about programming.

That's very subjective, but from my point of view the ability to use many paradigms at the same time allows for very interesting ideas.

I've tried Python and it doesn't fit your important points.

Answer 9

Compare code examples that do the same thing (join with a newline non-empty descriptions of items from a myList list) in different languages (languages are arranged in reverse-alphabetic order):

Ruby:

myList.collect { |f| f.description }.select { |d| d != "" }.join("\n")

Or

myList.map(&:description).reject(&:empty?).join("\n")

Python:

descriptions = (f.description() for f in mylist)
"\n".join(filter(len, descriptions)) 

Or

"\n".join(f.description() for f in mylist if f.description())

Perl:

join "\n", grep { $_ } map { $_->description } @myList;

Or

join "\n", grep /./, map { $_->description } @myList;

Javascript:

myList.map(function(e) e.description())
      .filter(function(e) e).join("\n")

Io:

myList collect(description) select(!="") join("\n")

Here's an Io guide.

Answer 10

Ruby would be better than Lisp in terms of being "mainstream" (whatever that really means, but one realistic concern is how easy it would be to find answers to your questions on Lisp programming if you were to go with that.) In any case, I found Ruby very easy to pick up. In the same amount of time that I had spent first learning Python (or other languages for that matter), I was soon writing better code much more efficiently than I ever had before. That's just one person's opinion, though; take it with a grain of salt, I guess. I know much more about Ruby at this point than I do Python or Lisp, but you should know that I was a Python person for quite a while before I switched.

Lisp is definitely quite cool and worth looking into; as you said, the size of community, etc. can change quite quickly. That being said, the size itself isn't as important as the quality of the community. For example, the #ruby-lang channel is still filled with some incredibly smart people. Lisp seems to attract some really smart people too. I can't speak much about the Python community as I don't have a lot of firsthand experience, but it seems to be "too big" sometimes. (I remember people being quite rude on their IRC channel, and from what I've heard from friends that are really into Python, that seems to be the rule rather than the exception.)

Anyway, some resources that you might find useful are:

1) The Pragmatic Programmers Ruby Metaprogramming series (http://www.pragprog.com/screencasts/v-dtrubyom/the-ruby-object-model-and-metaprogramming) -- not free, but the later episodes are quite intriguing. (The code is free, if you want to download it and see what you'd be learning about.)

2) On Lisp by Paul Graham (http://www.paulgraham.com/onlisp.html). It's a little old, but it's a classic (and downloadable for free).

Answer 11

@Jason I respectively disagree. There are differences that make Ruby superior to Python for metaprogramming - both philosophical and pragmatic. For starters, Ruby gets inheritance right with Single Inheritance and Mixins. And when it comes to metaprogramming you simply need to understand that it's all about the self. The canonical difference here is that in Ruby you have access to the self object at runtime - in Python you do not!

Unlike Python, in Ruby there is no separate compile or runtime phase. In Ruby, every line of code is executed against a particular self object. In Ruby every class inherits from both object and a hidden metaclass. This makes for some interesting dynamics:

class Ninja
  def rank
    puts "Orange Clan"
  end

  self.name #=> "Ninja"
end

Using self.name accesses the Ninja classes' metaclass name method to return the class name of Ninja. Does metaprogramming flower so beautiful in Python? I sincerely doubt it!

Answer 12

I am using Python for many projects and I think Python does provide all the features you asked for.

important:

1. Metaprogramming: Python supports metaclasses and runtime class/method generation etc
2. Syntax: Well thats somehow subjective. I like Pythons syntax for its simplicity, but some People complain that Python is whitespace-sensitive.
3. Paradigms: Python supports procedural, object-oriented and basic functional programming.
4. I think Python has a very practical oriented style, it was very inspiring for me.

Somewhat important:

1. Performance: Well its a scripting language. But writing C extensions for Python is a common optimization practice.
2. Documentation: I cannot complain. Its not that detailed as someone may know from Java, but its good enough.

As you are grad student you may want to read this paper claiming that Python is all a scientist needs. Unfortunately I cannot compare Python to Ruby, since I never used that language.

Regards, Dennis

Answer 13

Well, if you don't like the lisp syntax perhaps assembler is the way to go. :-)

It certainly has minimal distinction between code and data, is multi-paradigm (or maybe that is no-paradigm) and it's a mind expanding (if tedious) experience both in terms of the learning and the tricks you can do.

Answer 14

Io satisfies all of your "Important" points. I don't think there's a better language out there for doing crazy meta hackery.

Answer 15

one that supports the metaprogramming hacks that just can't be done in a statically compiled language

I would love to find a language that allows some of the cool stuff that Lisp does

Lisp can be compiled.

Answer 16

Did you try Rebol?

Answer 17

My answer would be neither. I know both languages, took a class on Ruby and been programming in python for several years. Lisp is good at metaprogramming due to the fact that its sole purpose is to transform lists, its own source code is just a list of tokens so metaprogramming is natural. The three languages I like best for this type of thing is Rebol, Forth and Factor. Rebol is a very strong dialecting language which takes code from its input stream, runs an expression against it and transforms it using rules written in the language. Very expressive and extremely good at dialecting. Factor and Forth are more or less completely divorced from syntax and you program them by defining and calling words. They are generally mostly written in their own language. You don't write applications in traditional sense, you extend the language by writing your own words to define your particular application. Factor can be especially nice as it has many features I have only seen in smalltalk for evaluating and working with source code. A really nice workspace, interactive documents, etc.

Answer 18

There isn't really a lot to separate Python and Ruby. I'd say the Python community is larger and more mature than the Ruby community, and that's really important for me. Ruby is a more flexible language, which has positive and negative repercussions. However, I'm sure there will be plenty of people to go into detail on both these languages, so I'll throw a third option into the ring. How about JavaScript?

JavaScript was originally designed to be Scheme for the web, and it's prototype-based, which is an advantage over Python and Ruby as far as multi-paradigm and metaprogramming is concerned. The syntax isn't as nice as the other two, but it is probably the most widely deployed language in existence, and performance is getting better every day.

Answer 19

If you like the lisp-style code-is-data concept, but don't like the Lispy syntax, maybe Prolog would be a good choice.

Whether that qualifies as a "fun to use, modern language", I'll leave to others to judge. ;-)

Answer 20

I've use Python a very bit, but much more Ruby. However I'd argue they both provide what you asked for.

If I see all your four points then you may at least check: http://www.iolanguage.com/

And Mozart/Oz may be interesting for you also: http://mozart.github.io/

Regards Friedrich

Answer 21

Ruby is my choice after exploring Python, Smalltalk, and Ruby.

Answer 22

What about OCaml ?

OCaml features: a static type system, type inference, parametric polymorphism, tail recursion, pattern matching, first class lexical closures, functors (parametric modules), exception handling, and incremental generational automatic garbage collection.

I think that it satisfies the following:

Important:

2. Nice, clean, sane syntax and consistent, intuitive semantics. Basically a well thought-out, fun to use, modern language.
3. Multiple paradigms. No one paradigm is right for every project, or even every small subproblem within a project.
4. An interesting language that actually affects the way one thinks about programming.

Somewhat important:

1. Performance. It would be nice if performance was decent, but when performance is a real priority, I'll use D instead.
2. Well-documented.

Answer 23

For python-style syntax and lisp-like macros (macros that are real code) and good DSL see converge.

Answer 24

I'm not sure that Python would fulfill all things you desire (especially the point about the minimal distinction between code and data), but there is one argument in favour of python. There is a project out there which makes it easy for you to program extensions for python in D, so you can have the best of both worlds. http://pyd.dsource.org/celerid.html

Answer 25

if you love the rose, you have to learn to live with the thorns :)

Answer 26

Do not to mix Ruby Programming Language with Ruby Implementations, thinking that POSIX threads are not possible in ruby.

You can simply compile with pthread support, and this was already possible at the time this thread was created, if you pardon the pun.

The answer to this question is simple. If you like lisp, you will probably prefer ruby. Or, whatever you like.

Answer 27

I would recommend you go with Ruby.

When I first started to learn it, I found it really easy to pick up.

Answer 28

Concerning your main-point (meta-programming): Version 1.6 of Groovy has AST (Abstract Syntax Tree) programming built-in as a standard and integrated feature. Ruby has RubyParser, but it's an add-on.

Answer 29

I suggest that you try out both languages and pick the one that appeals to you. Both Python and Ruby can do what you want.

Also read this thread.

Answer 30

I recommend Haskell. It isn't much of a contender for "important" criteria 1 (unless you include Template Haskell) or 3 (Haskell is a firmly Functional language, though you'd be surprised how easy it actually is to code imperatively in Haskell if you want to). You'll certainly enjoy Haskell's first class functions, though.

Haskell is a rock solid choice for "important" criterion 2 and 4. Especially #4: "An interesting language that actually affects the way one thinks about programming." Anyone who has learned a significant amount of Haskell can attest to its ability to expand your mind.

Answer 31

Go with JS just check out AJS (Alternative JavaScript Syntax) at my github http://github.com/visionmedia it will give you some cleaner looking closures etc :D