2

I'm writing an article with papaja package (an RMarkdown variant), which contains some Japanese characters. I would like to write its shorttitle in Japanese as follows:

title             : "ここは日本語もOK: You have a Japanese title"
shorttitle        : "日本語が書けません: A Japanese short title causes an error"

However, the characters cause a Pandoc's error, even though I write and save my file with UTF-8 encoding: pandoc.exe: Cannot decode byte '\x93': Data.Text.Internal.Encoding.decodeUtf8: Invalid UTF-8 stream Error: pandoc document conversion failed with error 1 Execution halted. At this stage, neither tex file nor PDF was produced.

Nevertheless, when I removed the CJK characters from the short title and included other CJK ones in the main text as below, I was able to get both tex and PDF file.

title             : "ここは日本語もOK: You have a Japanese title"
shorttitle        : "A Japanese short title causes an error"

Then, how can I fix the problem? Or, at least, where is the cause of the promblem?

MWE

---
title             : "ここは日本語もOK: You have a Japanese title"
shorttitle        : "日本語が書けません: A Japanese short title causes an error"

author: 
  - name          : "First Author"
    affiliation   : "1"
    corresponding : yes    # Define only one corresponding author
    address       : "Postal address"
    email         : "my@email.com"

affiliation:
  - id            : "1"
    institution   : "Hoge-Huga-University"

bibliography      : ["r-references.bib"]

floatsintext      : no
figurelist        : no
tablelist         : no
footnotelist      : no
linenumbers       : no
mask              : no
draft             : no

CJKmainfont: MS Mincho

#CJKsansfont: MS Gothic

documentclass     : "apa6"
classoption       : "doc"
output            : 
  papaja::apa6_pdf:
    number_sections: true
    latex_engine: xelatex
    keep_tex: true
    toc: false
    fig_caption: TRUE
    dev: cairo_pdf

csl-hanging-indent: true
---

```{r setup, include = FALSE}
library("papaja")
r_refs("r-references.bib")

```

```{r analysis-preferences}
# Seed for random number generation
set.seed(42)
knitr::opts_chunk$set(cache.extra = knitr::rand_seed)
```

# 日本語desu・yo!
お早う

\textsf{こんにちは}

This is the \texttt{document 書類} for 資料 and I'm \textsf{stuck}...

$$
\log(5) = 1.609438
$$


# Methods
We report how we determined our sample size, all data exclusions (if any), all manipulations, and all measures in the study. <!-- 21-word solution (Simmons, Nelson & Simonsohn, 2012; retrieved from http://ssrn.com/abstract=2160588) -->

## Participants

## Material

## Procedure

## Data analysis
We used `r cite_r("r-references.bib")` for all our analyses.


# Results

# Discussion


\newpage

# References

\begingroup
\setlength{\parindent}{-0.5in}
\setlength{\leftskip}{0.5in}

<div id="refs" custom-style="Bibliography"></div>
\endgroup

Session Info

> sessionInfo()
R version 4.0.2 (2020-06-22)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 18362)

Matrix products: default

locale:
[1] LC_COLLATE=Japanese_Japan.932  LC_CTYPE=Japanese_Japan.932   
[3] LC_MONETARY=Japanese_Japan.932 LC_NUMERIC=C                  
[5] LC_TIME=Japanese_Japan.932    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods  
[7] base     

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.5         compiler_4.0.2     RColorBrewer_1.1-2
 [4] prettyunits_1.1.1  remotes_2.2.0      tools_4.0.2       
 [7] extrafont_0.17     digest_0.6.25      packrat_0.5.0     
[10] pkgbuild_1.1.0     jsonlite_1.7.0     evaluate_0.14     
[13] rlang_0.4.7        cli_2.0.2          rstudioapi_0.11   
[16] curl_4.3           yaml_2.2.1         parallel_4.0.2    
[19] xfun_0.16          Rttf2pt1_1.3.8     withr_2.2.0       
[22] DiagrammeR_1.0.6.1 knitr_1.29         htmlwidgets_1.5.1 
[25] rprojroot_1.3-2    glue_1.4.1         R6_2.4.1          
[28] DiagrammeRsvg_0.1  processx_3.4.3     fansi_0.4.1       
[31] rmarkdown_2.3      papaja_0.1.0.9997  callr_3.4.3       
[34] extrafontdb_1.0    magrittr_1.5       backports_1.1.8   
[37] ps_1.3.4           htmltools_0.5.0    assertthat_0.2.1  
[40] rsvg_2.1           V8_3.2.0           visNetwork_2.0.9  
[43] crayon_1.3.4 

Pandoc version

> rmarkdown::pandoc_version()
[1] ‘2.9.2.1’
Carlos Luis Rivera
  • 3,108
  • 18
  • 45
  • Quick drive-by question: did you ask on the pandoc issue tracker first with your [mcve]? Because as useful as SO is, it should be your last resort, not your first port of call. If there's an actual problem with pandoc over this, they should be made aware of that so they can fix it for everyone, rather than you personally getting a fix just for your own codebase via Stackoverflow. – Mike 'Pomax' Kamermans Aug 14 '20 at 01:06
  • Hi, this could be a bug in `papaja`, but I'm having some trouble reproducing the problem on my Mac (are you on a Windows machine?). I have prepared a potential fix that you can install via `remotes::install_github("crsh/papaja@cjk")`. Could you try it out and let me know if this resolves the problem? – crsh Aug 21 '20 at 07:11
  • @crsh I reinstalled papaja via remote my Win 10 machine (see also my edited post for the session info). RMarkdown still returns an error `pandoc.exe: Cannot decode byte '\x8e': Data.Text.Internal.Encoding.decodeUtf8: Invalid UTF-8 stream Error: pandoc document conversion failed with error 1 Execution halted` – Carlos Luis Rivera Aug 21 '20 at 12:30
  • 1
    The issue should be fixed in the latest develoment version. `remotes::install_github("crsh/papaja@devel")` – crsh Nov 22 '21 at 16:43

0 Answers0