Regex with Chinese characters

Question

I'm searching text_ which is: 本周（3月25日-3月31日），国内油厂开机率继续下降，全国各地油厂大豆压榨总量1456000吨（出粕1157520吨，出油262080吨)，较上周的...[continued]

  crush <- str_extract(string = text_, pattern = perl("(?<=量).*(?=吨（出粕)"))
  meal <- str_extract(string = text_, pattern = perl("(?<=粕).*(?=吨，出)"))
  oil <-  str_extract(string = text_, pattern = perl("(?<=出油).*(?=吨）)"))

prints

[1] "1456000"   ## correct
[1] "1157520"   ## correct
[1] NA          ## looking for 262080 here

Why do the first two match but not the last one? I'm using the stringr library.

Use `"(?<=出油)\\d+(?=吨)"` to get the digits between the substrings. The third pattern does not work because the round bracket is an ASCII one, not the Chinese one. — Wiktor Stribiżew, Apr 04 '17 at 15:45
Ah, the troubles with look-a-like characters. So easy to be tricked by those. — LukStorms, Apr 04 '17 at 15:55
@WiktorStribiżew solution didn't work for me, I posted what I ended up using. Still curious why you're regex didn't return the digits on r when it worked on regex101.com — Rafael, Apr 04 '17 at 16:04

score 2 · Accepted Answer · answered Apr 04 '17 at 16:21

Note that current version of stringr package is based on ICU regex library, and using perl() is deprecated.

Note that lookbehind patterns are fixed-width, and it seems that there is a problem with how ICU parses the first letter in your lookbehind pattern (it cannot calculate its width for some unknown reason).

Since you are using stringr, you may just rely on capturing that can be achieved with str_match, to extract a part of the pattern:

> match <- str_match(s, "出油(\\d+)吨")
> match[,2]
[1] "262080"

This way, you will avoid any eventual issues in the future. Also, these regexps are executed faster since there is no unanchored lookbehind in the pattern that is executed at every location in the searched string.

Also, you may just use your PCRE regex with base R:

> regmatches(s, regexpr("(?<=出油)\\d+(?=吨)", s, perl=TRUE))
[1] "262080"

Rafael · Answer 2 · 2017-04-04T16:36:40.980

1

For some reason, still don't know, I wasn't able to use @WiktorStribiżew 's commented solution, but this ended up working:

oil <-  str_extract(string = text_, pattern = perl("(?<=吨).*(?=吨)"))
# [1] "（出粕1157520吨，出油262080吨），较
oil <- str_extract(string = oil, pattern = perl("(?<=油)\\d+(?=吨)"))
# [1] 262080

edited Apr 04 '17 at 16:36

answered Apr 04 '17 at 16:02

Rafael

3,096
1
23
61

Please see my answer, you do not even need stringr, you may use the PCRE regex with base R. – Wiktor Stribiżew Apr 04 '17 at 20:08

score 0 · Answer 3 · answered Apr 04 '17 at 15:39

0

Try this:

  oil <-  str_extract(string = text_, pattern = perl("(?<=出油).*(?=吨)，较上周的）)"))

Because simple 吨 could appear later again your text again, cannot precise locate which part, may exceed the data length or cause the data type issue.

answered Apr 04 '17 at 15:39

LONG

4,490
2
17
35

Still returns `NA`. With some trouble shooting I've found that the problem is with matching `出油`. If I replace this with just `油`, it returns, predictably, a much longer string, but correctly identifies the end of the desired string right after `262080`. Any idea why `出油` isn't matching? – Rafael Apr 04 '17 at 15:42
Has `出油` appeared before your pasted text? – LONG Apr 04 '17 at 15:44

score 0 · Answer 4 · answered Dec 03 '21 at 07:48

Did you check your ICU version. I met this problem before, ICU version of stringi is 55 that time, I tried to recompile stringi with ICU 58, and then stringr works fine with Chinese characters. Now stringi's new version is compiled with ICU which's version is newer than 60, the problem should be fixed.

Regex with Chinese characters

4 Answers4