Trying to split string into single words or "quoted words", and want to keep the quotes in the resulting array

Question

I'm trying to split a string like Presentation about "Test Driven Development" into an array like this:

[ 'Presentation',
  'about',
  '"Behavior Driven Development"' ]

I have tried CSV::parse_line(string, col_sep: ' '), but this results in

[ 'Presentation',
  'about',
  'Behavior Driven Development' ] # I'm missing the quotes here

I also tried some regexp magic, but I'm still a beginner and didn't succeed. I guess this is quite simple for a pro, so maybe someone could point me into the right direction? Thanks.

I fixed my answer. It should work for you now. – Linuxios Jul 19 '12 at 17:40 — Linuxios, Jul 19 '12 at 17:40
Ha! CSV was actually exactly what I needed. – Chris Dec 17 '19 at 13:02 — Chris, Dec 17 '19 at 13:02

Howard · Accepted Answer · 2012-07-19T18:40:27.653

22

You may use the following regular expression split:

str = 'Presentation about "Test Driven Development"'
p str.split(/\s(?=(?:[^"]|"[^"]*")*$)/)
# => ["Presentation", "about", "\"Test Driven Development\""]

It splits if there is a space but only if the text following until the end contains an even number of ". Be aware that this version will only work if all your strings are properly quoted.

An alternative solution uses scan to read the parts of the string (besides spaces):

p str.scan(/(?:\w|"[^"]*")+/)
# => ["Presentation", "about", "\"Test Driven Development\""]

edited Jul 19 '12 at 18:40

answered Jul 19 '12 at 17:33

Howard

38,639
9
64
83

Thank you, works like a charm! Regexes are magic... I will try to dissect them and understand it. – Joshua Muheim Jul 19 '12 at 18:06
For your reference, I used this to remove empty elements and strip quotes and spaces: "a b 'c d '".split(/\s(?=(?:[^'"]|'[^']*'|"[^"]*")*$)/).select {|s| not s.empty? }.map {|s| s.gsub(/(^ +)|( +$)|(^["']+)|(["']+$)/,'')} – Keymon Jan 02 '15 at 12:11

score 4 · Answer 2 · answered Jan 02 '15 at 12:19

Just to extend the previous answer from Howard, you can add this method:

class String
  def tokenize
    self.
      split(/\s(?=(?:[^'"]|'[^']*'|"[^"]*")*$)/).
      select {|s| not s.empty? }.
      map {|s| s.gsub(/(^ +)|( +$)|(^["']+)|(["']+$)/,'')}
  end
end

And the result:

> 'Presentation      about "Test Driven Development"  '.tokenize
=> ["Presentation", "about", "Test Driven Development"]

Linuxios · Answer 3 · 2012-07-19T17:40:38.233

0

Here:

"Presentation about \"Test Driven Development\"".scan(/\s?\w+\s?|"[\w\s]*"/).map {|s| s.strip}

edited Jul 19 '12 at 17:40

answered Jul 19 '12 at 17:26

Linuxios

34,849
13
91
116

Fine, removed my -1. Please note that now the spaces are included in the result. – Howard Jul 19 '12 at 17:42
What about `.scan(/(?:\S|"[^"]*")+/)`? – Howard Jul 19 '12 at 17:45
Thank you! It's interesting how many different ways there are to attack a problem. – Joshua Muheim Jul 19 '12 at 18:06

Trying to split string into single words or "quoted words", and want to keep the quotes in the resulting array

3 Answers3

Linked