Is there a way to perform edit distance between two string columns in a dataframe?

Question

I have two datasets: dataset1 & dataset2 (image link provided), which have a common column called SAX which is a string object.

dataset1=
         SAX
0    glngsyu
1    zicobgm
2    eerptow
3    cqbsynt
4    zvmqben
..       ...
475  rfikekw
476  bnbzvqx
477  rsuhgax
478  ckhloio
479  lbzujtw

480 rows × 2 columns

and

dataset2=
         SAX
0    glngsyu
1    zicobgm
2    eerptow
3    cqbsynt
4    zvmqben
..       ...
475  rfikekw
476  bnbzvqx
477  rsuhgax
478  ckhloio
479  lbzujtw

480 rows × 2 columns

I need the output to be a column of minimum number of edits (operations) required to convert "SAX" column of dataset1 to "SAX" of dataset2. Is there a way to accomplish that?

Thanks.

Answer 1

Using Levenshtein distance from the textdistance module:

from textdistance import levenshtein

# Merge the two columns in one dataframe
df = dataset1[['SAX']].merge(dataset2[['SAX']], left_index=True, right_index=True, suffixes=('_1', '_2'))

# Compute the Levenshtein distance
df['distance'] = df.apply(lambda x: levenshtein.distance(x['SAX_1'],  x['SAX_2']), axis=1)

Answer 2

I would use pylev module for this task. It is pure-python so it should work in any operating system just after doing pip install pylev. For comparison of pair-wise elements I would harness zip, consider following simple example

import pylev
import pandas as pd
df1 = pd.DataFrame({'col1':['some','values','here']})
df2 = pd.DataFrame({'col1':['same','virtues','there']})
dfdist = pd.DataFrame({'dist':[pylev.levenshtein(*i) for i in zip(df1.col1,df2.col1)]})
print(dfdist)

output

   dist
0     1
1     3
2     1