5

Update:

My problem has evolved after trying various things detailed in the comments to this question. The problems previously described in this question may indeed have been due to bad JSON, as suggested by @apple16, or related to my "$PATH" configuration, as discussed with @RandyLai.

I may have identified the underlying problem: character encoding. Using my build variant in Sublime Text with diacritics in the YAML front-matter of my .Rmd document (which I had been doing when asking the original question here) results in an error different from those described initially, which I no longer seem to get:

Error en yaml::yaml.load(front_matter)

Interestingly, this error is not thrown with the same YAML in RStudio or in Terminal (or from a shell script), where everything renders perfectly. I get no error in ST when I remove the diacritics from the YAML; however, if I put diacritics in the main document, these are rendered as NA in the resulting HTML. This kind of resembles what was happening previously that I described below as "wackily wrong" HTML output.

In the comments to this question I attempt to do various things to my "$PATH" which now, as they were for @RandyLai, appear unnecessary; I am referring specifically to adding the path to pandoc to /etc/launchd.config: after having restored the default path /usr/bin:/bin:/usr/sbin:/sbin, everything works in ST as I am describing it now, whether I use my build variant in SublimeKnitr-knitr's .sublime-build or in R Box-R Markdown's .sublime-build.

Here is my locale information, in case it is of use:

Mi-Mac:~ LeGastronome$ locale
LANG=
LC_COLLATE="C"
LC_CTYPE="UTF-8"
LC_MESSAGES="C"
LC_MONETARY="C"
LC_NUMERIC="C"
LC_TIME="C"
LC_ALL=

I am trying to find out how Sublime Text executes its .sublime-build files.

In Sublime Text 3 on my Mac (v10.9.3) I have created a custom build variant for both SublimeKnitr's knitr-Markdown and R Box's R Markdown syntaxes; here is my .sublime-build file for the former, which has the fully functional original/default build followed by my faulty variant:

{
  "selector": "text.html.markdown.knitr",
  "working_dir": "${project_path:${folder}}",
  "cmd": [ "Rscript -e \"library(knitr); knit('$file', output='$file_path/$file_base_name.md')\"" ],
  "shell": true,

  "variants":
  [
    {
      "name": "Render",
      "working_dir": "$file_path",
      "shell_cmd": "Rscript -e \"rmarkdown::render(input = '$file')\""
    }
  ]
}

And here is my .sublime-build for R Box's R Markdown, which is quite similar to SublimeKnitr's knitr-Markdown build:

{
    "selector": "text.html.rmarkdown",
    "working_dir": "${project_path:${folder}}",
    "cmd": [ "Rscript -e \"library(knitr); knit('$file', output='$file_path/$file_base_name.md')\"" ],
    "shell": true,

  "variants":
  [
    {
      "name": "Render",
      "working_dir": "$file_path",
      "shell_cmd": "Rscript -e \"rmarkdown::render(input = '$file')\""
    }
  ]
}

In my custom build variants I have tried using both the pair of keys cmd and shell, as well as shell_cmd as cited in the code above, which I believe is new to ST3. In any case, both do undesirable things that happen neither when I run, in Terminal:

Rscript -e "rmarkdown::render(input = 'path/to/file.Rmd')"

nor when I run, in RStudio:

rmarkdown::render(input = 'path/to/file.Rmd')

I get the same results using SublimeKnitr's build as I do using R Box's build. The output I get is the following:

  1. with the mappings:

    "cmd": [ "Rscript -e \"rmarkdown::render(input = '$file')\"" ],
    "shell": true
    

    I get:

    Error: pandoc version 1.12.3 or higher is required and was not found.
    Ejecuci'on interrumpida
    [Finished in 0.4s with exit code 1]
    [cmd: ['Rscript -e "rmarkdown::render(input = \'/path/to/file.Rmd\')"']]
    [dir: /path/to/pertinent/directory]
    [path: /usr/bin:/bin:/usr/sbin:/sbin]
    

    Note that I do indeed have pandoc installed and fully operative, e.g.:

    Mi-Mac:~ LeGastronome$ pandoc --version
    pandoc 1.12.4.2
    Compiled with texmath 0.6.6.1, highlighting-kate 0.5.8.1.
    Syntax highlighting is supported for the following languages:
        actionscript, ada, apache, asn1, asp, awk, bash, bibtex, boo, c, changelog,
        clojure, cmake, coffee, coldfusion, commonlisp, cpp, cs, css, curry, d,
        diff, djangotemplate, doxygen, doxygenlua, dtd, eiffel, email, erlang,
        fortran, fsharp, gcc, gnuassembler, go, haskell, haxe, html, ini, isocpp,
        java, javadoc, javascript, json, jsp, julia, latex, lex, literatecurry,
        literatehaskell, lua, makefile, mandoc, markdown, matlab, maxima, metafont,
        mips, modelines, modula2, modula3, monobasic, nasm, noweb, objectivec,
        objectivecpp, ocaml, octave, pascal, perl, php, pike, postscript, prolog,
        python, r, relaxngcompact, restructuredtext, rhtml, roff, ruby, rust, scala,
        scheme, sci, sed, sgml, sql, sqlmysql, sqlpostgresql, tcl, texinfo, verilog,
        vhdl, xml, xorg, xslt, xul, yacc, yaml
    Default user data directory: /Users/LeGastronome/.pandoc
    Copyright (C) 2006-2014 John MacFarlane
    Web:  http://johnmacfarlane.net/pandoc
    This is free software; see the source for copying conditions.  There is no
    warranty, not even for merchantability or fitness for a particular purpose.
    
  2. with the mapping:

    "shell_cmd": "Rscript -e \"rmarkdown::render(input = '$file')\""
    

    I get the desired type of document, HTML, but the HTML is wackily wrong and different from what I get when running the aforementioned code from Terminal and RStudio; additionally, exclusively in Sublime Text I get the following warning:

    pandoc: YAML header is not an object "source" (line 1, column 1)
    

    Note that if, as in the original build configurations cited above, I use brackets "[" like so:

    "shell_cmd": [ "Rscript -e \"rmarkdown::render(input = '$file')\"" ]
    

    Sublime does not build nor do anything other than save my file when I attempt to run the build variant.

My problem seems rather user-specific for which reason I am looking for more precise information as to how .sublime-build files are executed. I understand from some unofficial documentation that by default a bit of Python is run; however, how can it be possible that, as mentioned above, when I run the same bit of code from Terminal as is included in my build variant, i.e.:

Rscript -e "rmarkdown::render(input = 'path/to/file.Rmd')"

I get such undesirably different results from Sublime Text? Thanks for any help.

user109114
  • 403
  • 4
  • 10
  • Have you tried replacing the original/main command with your falty variant? The command looks good (I use sublime 2 and cmd:"" for everything but it shouldn't make a difference). – apple16 Jun 16 '14 at 00:50
  • And also try splitting up the command into different items in the list: '"shell_cmd": [ "Rscript", "-e", "\"rmarkdown::render(input = '$file')\"" ]' – apple16 Jun 16 '14 at 00:53
  • @apple16: Thanks for your suggestions; I have just tried both. Using my variant as the only build option gives the exact same results as detailed in my question. Using brackets with the values of `"shell_cmd"` also had the same results as I mention in my question, i.e., ST only saves the file and does nothing else. Using `"cmd"`, listing its values within brackets, as you suggest, and `"shell": true` throws a syntax error of the sort seen at the command line, i.e., `Usage: /path/to/Rscript [--options] [-e expr [-e expr2 ...] | file] [args]` ... – user109114 Jun 16 '14 at 01:13
  • I think only saving the file that means JSON syntax error in the sublime build. And if you are getting a command line syntax error I would put echo in front of the entire command so you can see what is being ran. – apple16 Jun 16 '14 at 02:05
  • Where is your pandoc installed? make sure that the path to pandoc binary is specified. Check also http://sublimetext.info/docs/en/reference/build_systems.html – Randy Lai Jun 16 '14 at 03:38
  • 1
    [This](https://www.sublimetext.com/forum/viewtopic.php?f=2&t=14585) will be also helpful. – Randy Lai Jun 16 '14 at 03:38
  • @RandyLai : Thanks for your suggestions. I have pandoc v1.12.3, which shipped with RStudio, in `/Applications/R/RStudio/Contents/MacOs/pandoc/pandoc` and I have a symlink in `/Users/LeGastronome/bin` to v1.12.4.2, whose original location is in `/Users/LeGastronome/Library/Haskell/ghc-7.6.3/lib/pandoc-1.12.4.2/bin/pandoc`. – user109114 Jun 16 '14 at 05:59
  • The second link you posted has helped: I seem to get a successful build after opening ST from Terminal. On the other hand, I tried int3h's [Fix Mac Path](https://github.com/int3h/SublimeFixMacPath) and setting `$PATH` in `/etc/launchd.conf`, as per the accepted answer [here](http://superuser.com/questions/682260/how-can-i-set-environment-variables-for-gui-apps-in-os-x-mavericks); neither worked. – user109114 Jun 16 '14 at 06:02
  • Note: I also already had `/Users/LeGastronome/bin` included in my `$PATH` in `/etc/paths.d`, as per, e.g., [this question](http://apple.stackexchange.com/questions/106355/setting-the-system-wide-path-environment-variable-in-mavericks) but evidently that doesn't suffice. – user109114 Jun 16 '14 at 06:02
  • Try to put to path of pandoc under the `path` variable of the `.sublime-build`. http://sublimetext.info/docs/en/reference/build_systems.html – Randy Lai Jun 16 '14 at 06:10
  • @RandyLai : I have tried that, unsuccessfully, using both the symlink and the path to the original v.1.12.4.2. – user109114 Jun 16 '14 at 06:20
  • Your `.sublime-build` file works for me. However, I installed `pandoc` from `cabal` and I symlinked `/Users/Randy/.cabal/bin/pandoc` to `/usr/local/bin/pandoc` – Randy Lai Jun 16 '14 at 06:45
  • So this command works from the terminal? Does it work from a bash file? Can you put the command in a bash file and have sublime run the bash file? If that dosent work, I'm not sure – apple16 Jun 16 '14 at 11:59
  • @apple16 : Yes, everything seems to work perfectly from both Terminal and a shell script. Given the way that I can get ST to cooperate by having opened it from Terminal, I feel like it must be an issue with ST's interpretation of `"$PATH"`, no? – user109114 Jun 16 '14 at 14:52
  • @RandyLai : Are you on a Mac and running Mavericks? I too got `pandoc` via `cabal`. What do you get with `launchctl getenv PATH`? Have you done anything special with your `"$PATH"` in order that ST interprets it correctly, i.e., something akin to what is suggested in the links you originally signaled me? – user109114 Jun 16 '14 at 15:07
  • @user3143179: Yes, I am also on Mavericks. I have done nothing for my $PATH variable and Sublime except that I symlinked `/Users/Randy/.cabal/bin/pandoc` to `/usr/local/bin/pandoc`. And Sublime is able to run the `Render` variant with this pandoc. My `launchctl getenv PATH` is of its default value: `/usr/bin:/bin:/usr/sbin:/sbin` – Randy Lai Jun 17 '14 at 02:52
  • @user3143179: When I ran `import os; os.environ['PATH']` in the ST console, I got `'/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin'` – Randy Lai Jun 17 '14 at 02:56
  • @user3143179: by reading the [source](https://github.com/rstudio/rmarkdown/blob/master/R/pandoc.R#L388) code of `Rmarkdown`, I believe that the problem can be solved by symlinking your pandoc binary to `/usr/local/bin/pandoc`. – Randy Lai Jun 17 '14 at 03:02
  • @RandyLai : Thanks for your continued attention. I have updated my original question. – user109114 Jun 17 '14 at 04:58
  • Try to change your document to utf8. I can reproduce the same error for a file with diacritics which is not in utf8 but the problem is solved when the file is encoded in utf8. – Randy Lai Jun 17 '14 at 07:39
  • Can you also run `import subprocess; print(subprocess.check_output("R -q -e 'Sys.getlocale()'", shell=True).decode('utf8'))` in your ST console and run `R -q -e 'Sys.getlocale()'` in your terminal to see if they have the same? – Randy Lai Jun 17 '14 at 08:00
  • My default and fallback encodings in my `.sublime-settings` are set to UTF-8. If I save or re-open the `.Rmd` file with a different encoding, I get the same results as described in my update. Running `import subprocess; print(subprocess.check_output("R -q -e 'Sys.getlocale()'", shell=True).decode('utf8'))` in ST gives `"C"`, while `R -q -e 'Sys.getlocale()'` from Terminal gives `"C/UTF-8/C/C/C/C"`. – user109114 Jun 17 '14 at 14:54
  • It seems that your system is not in English. When you run R in terminal, since "LC_CTYPE" is utf-8, R will read documents as UTF-8 documents. However, if the command is launched inside Sublime, "LC_CTYPE" in Sublime is empty, thus it doesn't know what encoding to use. The easiest but less attractive solution is to switch your system language to English. Another possible way is to change your R default locale to utf8 by adding `LANG=en_US.UTF-8` to `~/.Renviron`. – Randy Lai Jun 17 '14 at 23:51
  • 1
    Another possible solution is to add the variable LANG in the .sublime-build file. Check the option env [here](http://sublimetext.info/docs/en/reference/build_systems.html). – Randy Lai Jun 17 '14 at 23:53
  • Or specify UTF8 encoding in the render function. – Randy Lai Jun 18 '14 at 01:26
  • @RandyLai : Fabulous. Adding `"env": { "LANG": "en_US.UTF-8" }` to the variant build seems to have solved the problem completely. Do you want to put that as an official answer to my question? If not, I'll do so in order that it be clear what really is necessary to resolve the issue. I am also thinking of adding the exact build variant JSON in an update to the question. And I'll update the comment I left on GitHub. – user109114 Jun 18 '14 at 04:26
  • I'll try your other suggestions too. – user109114 Jun 18 '14 at 04:35

1 Answers1

2

It seems that your system is not in English. When you run R in terminal, since "LC_CTYPE" is utf-8, R will read documents as UTF-8 documents. However, if the command is launched inside Sublime, "LC_CTYPE" is empty, thus R doesn't know what encoding to use.

There are several solutions.

  1. The easiest but less attractive solution is to switch your system language to English.

  2. Another possible way is to change your R default locale to utf8 by adding LANG=en_US.UTF-8 to ~/.Renviron

  3. Another possible solution is to add the variable LANG in the .sublime-build file. Check the option env here

  4. Or specify UTF8 encoding in the rmarkdown::render function.

Randy Lai
  • 3,084
  • 2
  • 22
  • 23
  • - **Solution 1**: Not attempted. - **Solution 2**: It seems to work just fine. - **Solution 3**: Adding `"env": { "LANG": "en_US.UTF-8" }"` works swell. - **Solution 4**: `"shell_cmd": "Rscript -e \"rmarkdown::render(input = '$file', encoding = 'UTF-8')\""` seems to work fine, i.e., it produces the desired characters and HTML, but on my machine gives the following warning in Sublime's console: `Mensajes de aviso perdidos In native_encode(text) : some characters may not work under the current locale` – user109114 Jun 18 '14 at 05:31