How To Use a Wildcard in robots.txt

Question

Is it possible to:

User-agent: *
Disallow: /apps/abc*/

In a robots.txt file to disallow abc123, abc-xyz, etc.?

Answer 1

Quoting Wikipedia:

The Robot Exclusion Standard does not mention anything about the "*" character in the Disallow: statement. Some crawlers like Googlebot and Slurp recognize strings containing "*", while MSNbot and Teoma interpret it in different ways.

Additional research may be found in the cited source.