Ok, here is how I solved it:
First I created one require.js bootstrap file for each language. Then I modified my maven pom.xml to perform the javascript optimization twice, one for each locale. I used requirejs-maven-plugin with two execution elements, one for each locale. Each of the elements have a configFile value which points to the respective r.js build files. (One for each language). The build file for a locale then points to the bootstrap file for that language.
After running the optimization you will now have a uglified js-file of each locale. But chances are that none of these are included in the final war file. You have to copy the uglified files from the r.js build_out folders into the war. I did this by adding configuration elements to the maven-war-plugin; warSourceDirectory and webResources to copy my two locales. I also used webSourceInclude and webSourceExlude to remove the files I did not want.
The last step was to load the correct require bootstrap file when loading the site. To do this simply create a javascript snippet in your index.html which first determines your locale (from a cookie for example), and then dynamically load the bootstrap file
document.write("<script data-main='bootstrap_" + getLocaleString() + "' src='/js/lib/require.js'><\/script>");
Thanks to the author of https://github.com/SlexAxton/require-handlebars-plugin for helping me figure this out.