0

I just asked the support guys on GitHub why AspectJ (*.aj) files are not syntax-highlighted. The answer was that they are using Pygments, but are unaware of any existing lexer for AspectJ. I did a quick web search and did not find any either. Has anyone here written one or can point me to a link for an existing one?

Long ago I have written a lexer for Kconfig (Linux kernel configuration) files, but it was rather hard for me because I do not speak Python. So before I start torturing my brain again, I thought I should better ask first instead of possibly re-inventing the wheel.

kriegaex
  • 63,017
  • 15
  • 111
  • 202

2 Answers2

2

After having created a "copy, paste & modify" solution of JavaLexer initially because I really do not speak Python, I managed to hack another quick'n'dirty solution which subclasses JavaLexer and delegates lexing to it for the most part. Exceptions are

  • AspectJ-specific keywords,
  • handling of inter-type declarations followed by colons without a space not as Java labels, but as AspectJ keywords plus ":" operator and
  • handling of inter-type annotation declarations as AspectJ keywords and not as Java name decorators.

I am sure my little heuristic solution misses some details, but as Andrew Eisenberg said: an imperfect, but working solution is better than a non-existent perfect one:

class AspectJLexer(JavaLexer):
    """
    For `AspectJ <http://www.eclipse.org/aspectj/>`_ source code.
    """

    name = 'AspectJ'
    aliases = ['aspectj']
    filenames = ['*.aj']
    mimetypes = ['text/x-aspectj']

    aj_keywords = [
        'aspect', 'pointcut', 'privileged', 'call', 'execution',
        'initialization', 'preinitialization', 'handler', 'get', 'set',
        'staticinitialization', 'target', 'args', 'within', 'withincode',
        'cflow', 'cflowbelow', 'annotation', 'before', 'after', 'around',
        'proceed', 'throwing', 'returning', 'adviceexecution', 'declare',
        'parents', 'warning', 'error', 'soft', 'precedence', 'thisJoinPoint',
        'thisJoinPointStaticPart', 'thisEnclosingJoinPointStaticPart',
        'issingleton', 'perthis', 'pertarget', 'percflow', 'percflowbelow',
        'pertypewithin', 'lock', 'unlock', 'thisAspectInstance'
    ]
    aj_inter_type = ['parents:', 'warning:', 'error:', 'soft:', 'precedence:']
    aj_inter_type_annotation = ['@type', '@method', '@constructor', '@field']

    def get_tokens_unprocessed(self, text):
        for index, token, value in JavaLexer.get_tokens_unprocessed(self, text):
            if token is Name and value in self.aj_keywords:
                yield index, Keyword, value
            elif token is Name.Label and value in self.aj_inter_type:
                yield index, Keyword, value[:-1]
                yield index, Operator, value[-1]
            elif token is Name.Decorator and value in self.aj_inter_type_annotation:
                yield index, Keyword, value
            else:
                yield index, token, value
kriegaex
  • 63,017
  • 15
  • 111
  • 202
  • I have created a Pygments fork + [pull request](https://bitbucket.org/birkenfeld/pygments-main/pull-request/90/add-aspectjlexer-as-subclass-of-javalexer) for this code. – kriegaex Aug 09 '12 at 08:52
  • Update: The pull request for my lexer has been accepted, so AspectJ highlighting is to be expected in the next version of Pygments. :) – kriegaex Aug 21 '12 at 12:22
1

Syntax highlighting for aspectj should be quite straight forward to implement if you start with a Java lexer. The lexer would be identical to Java's with some extra keywords.

See here for a list of the AspectJ-specific keywords: http://git.eclipse.org/c/ajdt/org.eclipse.ajdt.git/tree/org.eclipse.ajdt.core/src/org/eclipse/ajdt/core/AspectJPlugin.java

And here for the Java keywords: http://git.eclipse.org/c/ajdt/org.eclipse.ajdt.git/tree/org.eclipse.ajdt.ui/src/org/eclipse/ajdt/internal/ui/editor/AspectJCodeScanner.java

Andrew Eisenberg
  • 28,387
  • 9
  • 92
  • 148
  • There is more to syntax highlighting than just filtering keywords. AspectJ has special syntax for pointcuts and advice. But your links are a good start because I would agree that a heuristic, imperfect lexer which only extends the Java lexer by a few keywords would be better than none. So I might give it a shot when I have some leisure time. Thanks for the links. P.S.: I have no intention of lecturing you about AspectJ syntax, knowing that you are an AJDT committer. You know much better than I do. ;-) – kriegaex Aug 07 '12 at 21:37
  • Simple and good enough is much better than perfect and non-existant. Even just copying the java lexer with no changes would be better than nothing. There is only so much you can do with just a lexer. To get things perfect, you would need a complete parser. – Andrew Eisenberg Aug 07 '12 at 22:47
  • I just did it. It was simple and is looking good. I sent the result to the guys at GitHub. I might also create a fork + pull request for the Pygments maintainers later. – kriegaex Aug 08 '12 at 10:23
  • This is a bit OT, but when looking into your class `AspectJCodeScanner` I noticed that you list `@type`, `@method`, `@field` as keywords, but not `@constructor`. Is this a bug or have you done it on purpose? – kriegaex Aug 08 '12 at 14:33
  • Ha! Nice catch. That code was written well before I arrived at AJDT and I never noticed it. And seeing as how no one has raised a bug about it, I don't think @constructor is used very often. – Andrew Eisenberg Aug 08 '12 at 16:24