Edit Distance (Dynamic Programming): Aren't insertion and deletion the same thing?

Question

In looking through the dynamic programming algorithm for computing the minimum edit distance between two strings I am having a hard time grasping one thing. To me it seems like given the two strings s and t inserting a character into s would be the same as deleting a character from t. Why then do we need to consider these operations separately when computing the edit distance? I always have a hard time computing the indices in the recurrence relation because I can't intuitively understand this part.

I've read through Skiena and some other sources but they all don't explain this part well. This SO link explains the insert and delete operations better than elsewhere in terms of understanding what string is being inserted into or deleted from but I still can't figure out why they aren't one and the same.

Edit: Ok, I didn't do a very good job of detailing the source of my confusion.

The way Skiena explains computing the minimum edit distance m(i,j) of the first i characters of a string s and the first j characters of a string t based on already having computed solutions to the subproblems is as follows. m(i,j) will be the minimum of the following 3 possibilities:

opt[MATCH] = m[i-1][j-1].cost + match(s[i],t[j]);
opt[INSERT] = m[i][j-1].cost + indel(t[j]);
opt[DELETE] = m[i-1][j].cost + indel(s[i]);

The way I understand it the 3 operations are all operations on the string s. An INSERT means you have to insert a character at the end of string s to get the minimum edit distance. A DELETE means you have to delete the character at the end of string s to get the minimum edit distance.

Given s = "SU" and t = "SATU" INSERT and DELETE would be as follows:

Insert:
 SU_
SATU

Delete:
   SU
SATU_

My confusion was that an INSERT into s is the same as a DELETION from t. I'm probably confused on something basic but it's not intuitive to me yet.

Edit 2: I think this link kind of clarifies my confusion but I'd love an explanation given my specific questions above.

Answer 1

They aren't the same thing any more than < and > are the same thing. There is of course a sort of duality and you are correct to point it out. a < b if and only if b > a so if you have a good algorithm to test for b > a then it makes sense to use it when you need to test if a < b.

It is much easier to directly test if s can be obtained from t by deletion rather than to directly test if t can be obtained from s by insertion. It would be silly to randomly insert letters to s and see if you get t. I can't imagine that any implementation of edit-distance actually does that. Still, it doesn't mean that you can't distinguish between insertion and deletion.

More abstractly. There is a relation, R on any set of strings defined by

s R t <=> t can be obtained from s by insertion

deletion is the inverse relation. Closely related, but not the same.

Answer 2

The problem of edit distance can be restated as a problem of converting the source string into target string with minimum number of operations (including insertion, deletion and replacement of a single character).

Thus, in the process of converting a source string into a target string, if inserting a character from target string or deleting a character from the source string or replacing a character in the source string with a character from the target string yields the same (minimum) edit distance, then, well, all the operations can be said to be equivalent. In other words, it does not matter how you arrive at the target string as long as you have done minimum number of edits.

This is realized by looking at how the cost matrix is calculated. Consider a simpler problem where source = AT (represented vertically) and target = TA (represented horizontally). The matrix is then constructed as (coming from west, northwest, north in that order):

  |    ε       |    T             |      A            |
  |            |                  |                   |
ε |    0       |    1             |      2            |
  |            |                  |                   |
A |    1       | min(2, 1, 2) = 1 | min(2, 1, 3) = 1  |
  |            |                  |                   |
T |    2       | min(3, 1, 3) = 1 | min(2, 2, 2) = 2  |

The idea of filling this matrix is:

1. If we moved east, we insert the current target string character.
2. If we moved south, we delete the current source string character.
3. If we moved southeast, we replace the current source character with current target character.

If all or any two of these impart the same cost in terms of editing, then they can be said to be equivalent and you can break the ties arbitrarily.

One of the first experiences with this comes when we find c(2, 2) in the cost matrix (c(0, 0) through c(0, 2) -- minimum costs of converting an empty string to "T", "TA" respectively, and c(0, 0) to c(2,0) -- costs of converting "A", "AT" respectively to empty string are clear).

Value of c(2, 2), can be realized either by:

1. inserting the current character in target, 'A' (we move east from c(2,1)) -- cost is 1 + 1 = 2, or
2. replacing the current character 'T' in source by current character in target 'A' -- cost is `1 + 1 = 2
3. deleting the current character in source, 'T' (we move south from c(1, 2)) -- cost is 1 + 1 = 2

Since all values are the same, which one are you going to choose?

If you choose to move from west, your alignment could be:

A   T   -
-   T   A

(one deletion, one 0-cost replacement, one insertion)

If you choose to move from north, your alignment could be:

 -   A   T 
 T   A   -

(one insertion, one 0-cost replacement, one deletion)

If you choose to move from northwest, your alignment could be:

 A   T
 T   A

(Two 1-cost replacements).

All these edit graphs are equivalent in terms of given edit distance (under given cost function).

Answer 3

Edit distance is only interested in the minimum number of operations required to transform one sequence into another; it is not interested in the uniqueness of the transformation. In practice, there are often multiple ways to transform one string into another, that all have the minimum number of operations.