Python Map, Lambda, and string.replace

Question

I know how to solve my problem without using lambda/map (and I can't use regex or other python libraries for this exercise), using a for-loop with string.replace()....but I am really trying to see if I can achieve the same result using a combination of map/lambda and string.replace\

My goal here is to read in a txt file, and then substitute every instance of a non-standard 'e' (like éêèÉÊÈ) with the letter 'e'

My main issue now is that i get 6 lists, (e.g. I have 6 strings in newFile / newFileListComprehension and each string has updated the original string, based on the 1 iterable that was evaluated

e.g. newFile[0] = output of .replace('é'), newFile[1] = output of .replace('ê') etc.

So what I would like, is to return 1 copy of the formatted string, with all of the .replace() iterated over it.

Link to the text file I am referencing below can be accessed https://easyupload.io/s7m0zj

import string

def file2str(filename):
    with open(filename, 'r', encoding="utf8") as fid:
        return fid.read()

def count_letter_e(filename, ignore_accents, ignore_case):
    eSToAvoid = 'éêèÉÊÈ'
    textFile = file2str("Sentence One.txt")
    newFileListComprehension = [textFile.replace(e,'e') for e in eSToAvoid if ignore_accents == 1]
    value = textFile.count('e')
    #newFile = list((map(lambda element: (textFile.replace(element, 'e') if ignore_accents == 1 else textFile.count('e')), eSToAvoid)))
    return 0

numberOfEs = count_letter_e("Sentence One.txt", 1, 1)```

list comprehension is very easy to adapt from your line of code. Is that what you want? — Jean-François Fabre, Mar 14 '20 at 20:23
Honestly JF, I tried to use list comprehension but that seemed to work even less....and I assumed list comprehension would be more difficult because by its nature it will create an iterable list for "each" value in the list....but maybe I am mistaken? — Pardon_me, Mar 14 '20 at 20:25
"and I assumed list comprehension would be more difficult because by its nature it will create an iterable list for "each" value in the list." I don't know what that means, but a list comprehension is essentially a convenient way to write `map`/`filter` operations that produce lists. — juanpa.arrivillaga, Mar 14 '20 at 20:27
In any case, you can provide an example, it is generally not well received to have these sorts of vague descriptions. Note, the fact that the data is coming from a text file isn't really relevant. Just give inputs and expected outputs, and the code you've used already. Note, just by reading your description, it doesn't sound like `map` is the correct solution to begin with. It *sounds* like it should be a `reduce`. But really, in Python, you'd just write a for-loop. Even `map` / `filter` is generally avoided where they would be appropriate. — juanpa.arrivillaga, Mar 14 '20 at 20:29
Sure juanpa, I updated with my "attempt" at a list comprehension of this problem....i tried it, but got stuck with more or less the same results as map/reduce — Pardon_me, Mar 14 '20 at 20:32
No I mean your attempt with a loop. Again, **it isn't clear at all what you are even trying to accomplish**. You have to keep in mind, *we don't know what you assignment is*. A lot of what your saying doesn't make a lot of sense. file-objects don't even have those methods, so you are just working with a `str` object, presumably. That's fine, but again, what is your *exact* input, and what is your *exact, expected* output. — juanpa.arrivillaga, Mar 14 '20 at 20:34
Ok, I thought the code was clear and my explanation, but sure i'll try to clarify in the original post — Pardon_me, Mar 14 '20 at 20:34
Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/209641/discussion-between-pardon-me-and-juanpa-arrivillaga). — Pardon_me, Mar 14 '20 at 20:43
Give an example that we can copy-n-paste and run. Then we can test both your function and our own changes. Your filename and read is useless to us. I don't like constructing my own examples - I've done it, but it's more work, and often requires guesses. — hpaulj, Mar 14 '20 at 20:52
Thanks @hpaulj I just updated the original post, and included a link to the sample text file as well — Pardon_me, Mar 14 '20 at 21:02

score 1 · Answer 1 · answered Mar 14 '20 at 21:04

1

You can use str.translate for replacing multiple characters at once. str.maketrans helps you create the required mapping:

eSToAvoid = 'éêèÉÊÈ'
textFile.translate(str.maketrans(eSToAvoid, 'e' * len(eSToAvoid)))

answered Mar 14 '20 at 21:04

a_guest

34,165
12
64
118

oh my.....thank you for sharing this method. This is super clean, I need to learn a little more about how this works, but it runs well on my end :) Thanks a_guest! – Pardon_me Mar 14 '20 at 21:11

hpaulj · Accepted Answer · 2020-03-15T04:58:53.497

While the str.replace can only replace one substring with another, re.sub can replace a pattern.

In [55]: eSToAvoid = 'éêèÉÊÈ' 
In [58]: import re

test cases:

In [61]: re.sub(r'[éêèÉÊÈ]', 'e', 'foobar')                                                                          
Out[61]: 'foobar'
In [62]: re.sub(r'[éêèÉÊÈ]', 'e', eSToAvoid)                                                                         
Out[62]: 'eeeeee'
In [63]: re.sub(r'[éêèÉÊÈ]', 'e', 'testingè,É  foobar  è É')                                                         
Out[63]: 'testinge,e  foobar  e e'

The string replace approach is:

In [70]: astr = 'testingè,É  foobar  è É' 
    ...: for e in eSToAvoid: 
    ...:     astr = astr.replace(e,'e') 
    ...:                                                                                                             
In [71]: astr                                                                                                        
Out[71]: 'testinge,e  foobar  e e'

the replace is applied sequentially to astr. This can't be expressed as a list comprehension (or map). A list comprehensions most naturally replaces a loop that collects its results in a list (with list.append).

There's nothing wrong with the for loop. It's actually faster:

In [72]: %%timeit 
    ...: astr = 'testingè,É  foobar  è É' 
    ...: for e in eSToAvoid: 
    ...:     astr = astr.replace(e,'e') 
    ...:  
    ...:                                                                                                             
1.37 µs ± 8.96 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

In [73]: timeit re.sub(r'[éêèÉÊÈ]', 'e', 'testingè,É  foobar  è É')                                                  
2.79 µs ± 15.3 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)

In [77]: timeit astr.translate(str.maketrans(eSToAvoid, 'e' * len(eSToAvoid)))                                       
2.56 µs ± 14.5 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)

reduce

In [93]: from functools import reduce  
In [96]: reduce(lambda s,e: s.replace(e,'e'),eSToAvoid, 'testingè,É  foobar  è É' )                                  
Out[96]: 'testinge,e  foobar  e e'
In [97]: timeit reduce(lambda s,e: s.replace(e,'e'),eSToAvoid, 'testingè,É  foobar  è É' )                           
2.11 µs ± 32.1 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)

For fun you could also explore some of the idea presented here:

Cleanest way to combine reduce and map in Python

You want to 'accumulate' changes, and to do that, you need some sort of accumulator, something that hangs on to the last replace. itertools has an accumulate function, and Py 3.8 introduced a := walrus operator.

generator

In [110]: def foo(astr, es): 
     ...:     for e in es: 
     ...:         astr = astr.replace(e,'e') 
     ...:         yield astr 
     ...:                                                                                                            
In [111]: list(foo(astr, eSToAvoid))                                                                                 
Out[111]: 
['testingè,É  foobar  è É',
 'testingè,É  foobar  è É',
 'testinge,É  foobar  e É',
 'testinge,e  foobar  e e',
 'testinge,e  foobar  e e',
 'testinge,e  foobar  e e']

Or [s for s in foo(astr, eSToAvoid)] in place of the list(). This highlights that fact that a list comprehension returns a list of strings, even if the strings accumulate the changes.

great suggestion (one of my conditions was not to use regular expressions...) that being said is it fair to say that my "attempt" at using either list comprehension or lambda/map combo simply doesn't won't "work" to get the output i desire? — Pardon_me, Mar 15 '20 at 01:33
A list comprehension produces a list. That's not what you want, is it? You want a chain application, `astr.replace(...).replace(...).replace(...)`. There may be some trickery that would accumulate such changes, but why? Even in straightforward cases, a list comprehension is only mildly faster than the equivalent loop. — hpaulj, Mar 15 '20 at 04:16
thanks hpaulj....the "why" is just a reflection of me learning more about python...so i thank you for your insight here — Pardon_me, Mar 15 '20 at 07:23

Python Map, Lambda, and string.replace

2 Answers2

reduce

generator