Remove all punctuation from string, except if it's between digits

Question

I have a text that contains words and numbers. I'll give a representative example of the text:

string = "This is a 1example of the text. But, it only is 2.5 percent of all data"

I'd like to convert it to something like:

"This is a  1 example of the text But it only is  2.5  percent of all data"

So removing punctuation (can be . , or any other in string.punctuation) and also put a space between digits and words when it is concatenated. But keep the floats like 2.5 in my example.

I used the following code:

item = "This is a 1example of the text. But, it only is 2.5 percent of all data"
item = ' '.join(re.sub( r"([A-Z])", r" \1", item).split())
# This a start but not there yet !
#item = ' '.join([x.strip(string.punctuation) for x in item.split() if x not in string.digits])
item = ' '.join(re.split(r'(\d+)', item) )
print item

The result is :

 >> "This is a  1 example of the text. But, it only is  2 . 5  percent of all data"

I'm almost there but can't figure out that last peace.

In your example output you still have punctuation? do you want to remove that or not? — Roelant, Mar 31 '17 at 14:13
@Roelant I just updated the post I made a mistake. Please take a look at it. I want to remove punctuation but not from `2.5` and likewise. — deltascience, Mar 31 '17 at 14:14

score 8 · Accepted Answer · answered Mar 31 '17 at 14:59

You can use regex lookarounds like this:

(?<!\d)[.,;:](?!\d)

Working demo

The idea is to have a character class gathering the punctuation you want to replace and use lookarounds to match punctuation that does not have digits around

regex = r"(?<!\d)[.,;:](?!\d)"

test_str = "This is a 1example of the text. But, it only is 2.5 percent of all data"

result = re.sub(regex, "", test_str, 0)

Result is:

This is a 1example of the text But it only is 2.5 percent of all data

Thanks @Federico looks like it's the best answer so far ! – deltascience Apr 02 '17 at 12:59 — deltascience, Apr 02 '17 at 12:59

deltascience · Answer 2 · 2017-03-31T15:04:53.550

1

Okay folks, here is an answer (the best ? I don't know but it seems to work) :

item = "This is a 1example 2Ex of the text.But, it only is 2.5 percent of all data?"
#if there is two strings contatenated with the second starting with capital letter
item = ' '.join(re.sub( r"([A-Z])", r" \1", item).split())
#if a word starts with a digit like "1example"
item = ' '.join(re.split(r'(\d+)([A-Za-z]+)', item) )
#Magical line that removes punctuation apart from floats
item = re.sub('\S+', lambda m: re.match(r'^\W*(.*\w)\W*$', m.group()).group(1), item)
item = item.replace("  "," ")
print item

edited Mar 31 '17 at 15:04

answered Mar 31 '17 at 14:56

deltascience

3,321
5
42
71

This takes a lot of overhead. – Mar 31 '17 at 15:55
@Federico's answer is better, have a look at it. – deltascience Apr 04 '17 at 09:11

score 0 · Answer 3 · answered Mar 31 '17 at 14:15

I am out of touch with Python, but have some insight into the regexps. My I suggest the usage of or? I would use this regexp: "(\d+)([a-zA-Z])|([a-zA-Z])(\d+)", and then as the replacement string use: "\1 \2"
If some corner cases plague you, you can pass the back-reference to a procedure, and then deal 1-by-1, probably by checking if your "\1\2" can translate to float. TCL has such built-in functionality, Python should too.

score 0 · Answer 4 · answered Mar 31 '17 at 14:30

0

I tried this and it worked very well.

a = "This is a 1example of the text. But, it only is 2.5 percent of all data" a.replace(". ", " ").replace(", "," ")

Notice that, in replace function there is space after punctuation. I just replaced punctuation and space with only space.

answered Mar 31 '17 at 14:30

Shivam Pandya

251
3
10

This solution applies only for `.` and `,` but in `string.punctuation` there is a lot of characters. I need something that can work for all the punctuation... – deltascience Mar 31 '17 at 14:36

dot.Py · Answer 5 · 2017-03-31T14:54:59.333

0

Code:

from itertools import groupby

s1 = "This is a 1example of the text. But, it only is 2.5 percent of all data"
s2 = [''.join(g) for _, g in groupby(s1, str.isalpha)]
s3 = ' '.join(s2).replace("   ", "  ").replace("  ", " ")

#you can keep adding a replace for each ponctuation
s4 = s3.replace(". ", " ").replace(", "," ").replace("; "," ").replace(", "," ").replace("- "," ").replace("? "," ").replace("! "," ").replace(" ("," ").replace(") "," ").replace('" '," ").replace(' "'," ").replace('... '," ").replace('/ '," ").replace(' “'," ").replace('” '," ").replace('] '," ").replace(' ['," ")

s5 = s4.replace("  ", " ")
print(s5)

Output:

'This is a 1 example of the text But it only is 2.5 percent of all data'

P.s.: You can take a look at Punctuation Marks and keep adding them inside the .replace() function.

edited Mar 31 '17 at 14:54

answered Mar 31 '17 at 14:36

dot.Py

5,007
5
31
52

same comment as for @Shivam 's answer... The ponctuation doesn't contain only `.` and `,` – deltascience Mar 31 '17 at 14:39
I know it's not the best solution, but you know that you can keep adding `.replace()`, right? I've added a few more in my example to contemplate more punctuation marks.. But I think the best approach would be using a regex pattern. – dot.Py Mar 31 '17 at 14:50

score 0 · Answer 6 · answered Mar 31 '17 at 15:53

This is a regex approach

([^ ]?)(?:[^\P{punct}.]|(?<!\d)\.(?!\d))([^ ]?)

Replace in a callback:

if $1 length > 0 and $2 length > 0
replace with $1 + space + $2
else replace with $1$2

Expanded

 ( [^ ]? )                     # (1)
 (?:
      [^\P{punct}.] 
   |  
      (?<! \d )
      \.
      (?! \d )
 )
 ( [^ ]? )                     # (2)

If you don't want to use logic for chars adjacent to punct
Use (?:[^\P{punct}.]|(?<!\d)\.(?!\d)) and replace with nothing.

Remove all punctuation from string, except if it's between digits

6 Answers6

Linked