Is there any way go get GNU xgettext (or something similar) to work with scala files?

Question

We are using the GNU gettext utilities for creating PO files directly from our code base. The problem is - however - that some of our code base is written in Scala (most of it is Java). We are using gradle for building and deployment.

As far as I can see, Scala is not a supported language according to the xgettext documentation:

5.1.3 Choice of input file language

‘-L name’

‘--language=name’ Specifies the language of the input files.

The supported languages are C, C++, ObjectiveC, PO, Shell, Python, Lisp, EmacsLisp, librep, Scheme, Smalltalk, Java, JavaProperties, C#, awk, YCP, Tcl, Perl, PHP, GCC-source, NXStringTable, RST, RSJ, Glade, Lua, JavaScript, Vala, GSettings, Desktop.

Is there any way to use xgettext for Scala files? I have found this sbt compiler plugin that does what we need, but unfortunately we are using gradle instead of sbt.

score 2 · Answer 1 · answered May 16 '20 at 06:44

I use a very simple solution, I specify Vala as the language for xgettext. It is close enough to Scala in terms of escaping literals and function call syntax. It is not a rigorous solution, but in my practice it has worked well enough for me.

I have a Po class holding the message catalog, and allowing me to perform the translations via a couple of simple methods:

def t(singular: String): String = lookupSingular(None, singular)
def tf(singular: String, parameters: Any*): String = format(t(singular), Array(parameters: _*))
def tc(ctx: String, singular: String): String = lookupSingular(Some(ctx), singular)
def tcf(ctx: String, singular: String, parameters: Any*): String = format(lookupSingular(Some(ctx), singular), Array(parameters: _*))
def tn(singular: String, plural: String, n: Long): String = lookupPlural(None, singular, plural, n)
def tcn(ctx: String, singular: String, plural: String, n: Long): String = lookupPlural(Some(ctx), singular, plural, n)

t stands for Translate, f for Formatted (has parameters), c for Context (meaning the gettext context, and additional information to the messages distinguishing it from other messages with the same text), n stands for plural with N specified. You can make up your own names.

These methods acts both as a means of performing a translation, and as a marker for xgettext to extract the .pot files from the source code.

I use those like this:

new JMenuItem(po.t("Copy to clipboard"))
new JButton(po.tc("errorDialog", "Copy to clipboard"))
po.tf("Camera {0}", number),

etc.

With this setup, I run xgettext like this:

xgettext \
    --package-name="xxx" \
    -ktc:1c,2 \
    -ktcf:1c,2 \
    -ktnc:1c,2,3 \
    -ktf \
    -kt \
    -ktn:1,2 \
    -o "output.po" \
    $(find "DIRECTORY" -name *.scala) 2>&1 | grep -v "extension 'scala' is unknown; will try C"

The important part is the -k parameters, that maps custom xgettext keywords directly to names of my methods. This way, xgettext can be configured to use custom keywords.

Scala's triple quotes are not supported, though. xgettext warns about "unterminated literal" when encountering them, but progresses anyway.

IIRC, even the -c command line parameter can be made to work to support comments about the translated message, that can be very useful for the translators.

A couple of more observations from practice that do not directly answer the original question:

If I was implementing this again, the methods t, tc, ... would not return directly a translated String, but some LocalizableMessage encapsulating the original string and expansion parameters, that could be translated later lazily via an appropriate catalog.
I don't use the methods tn (gettext way of handling plurals). I recommend using the more general ICU messsage expansion libraries or something similar.

Guido Flohr · Accepted Answer · 2020-01-16T21:11:54.480

The Perl module Locale::XGettext allows you to write your own xgettext variant for pretty much every text format. You only have to supply a parser for the language/format that is able to extract the strings.

It is possible to write the parser in Java with minimal Perl boilerplate code around it. I have described the process in a blog post Extending Xgettext With Locale::XGettext. A sample extractor in Java is available at https://github.com/gflohr/Locale-XGettext/tree/master/samples/Java

Disclaimer: I am the author of all the above-mentioned software and documents!

If interfacing with Perl code sounds too complicated, you can also just write a parser for Scala in a language of your choice, run that parser before xgettext, and dump all translatable strings into a format that your version of xgettext understands, for example C:

gettext("Hello, world!");
gettext("Another string.");

And then you simply use the generated dummy source file as input to xgettext along with the other source files inside your build setup.

It may be worth noting that xgettext will always recognize .po and .pot files as input, even if you had invoked it with --language=java. That means, that .po or .pot files are suited best for dumping the translatable strings from your Scala file, provided that you find a way to generate .po/.pot files.

Finally, the sources of the gettext tools are designed to be extensible. You can add the Scala language if you are able to write a string extractor and format checker for it in C. You cannot, however, add support for another language at runtime, but you have to compile your own version of xgettext or contribute your extractor to the GNU gettext project.

Thank you very much for this detailed answer. We will need to discuss if the effort is worth the result but at least I now know what can be done. Greatly appreciated! — Florian Baierl, Jan 19 '20 at 12:29

Is there any way go get GNU xgettext (or something similar) to work with scala files?

2 Answers2