0

I wrote the below code using recursion to do CYK algorithm so that the Grammar G can generate any words having same number of a's followed by any number of b's but for some reason it is very slow, not sure why? When I use s2 it works, but if I used s1 which is a longer string it takes forever. Could anyone please advise as I can't figure out where is the bottleneck?

def fn(G, w, i, j, T):
    if T[i, j]:
        print '1'
        return T[i, j]
    elif i == j:
        for r in G:
            print '2'
            if r.endswith(w[i]) and r[0] not in T[i, j]:    
                print '3'
                T[i, j].append(r[0])
    else:
        print '4'
        for k in range(i, j):
            print '5'
            for a in fn(G, w, i, k, T):
                print '6'
                for b in fn(G, w, k+1, j, T):
                    print '7'
                    for r in G:
                        print '8'
                        if r.endswith(a+b) and r[0] not in T[i, j]:
                            print '9'
                            T[i, j].append(r[0])
    return T[i, j]


def fnmain(G, S, w):
    dict = {}
    for x in range(0, len(w)):
        for y in range(x, len(w)):
            dict[x,y] = []
    print dict        
    v = fn(G, w, 0, len(w) - 1, dict)
    print (w, v)
    if S in v:
        print ("T")
        return True
    else:
        print ("F")
        return False

G = ["S->AB", "S->XB", "T->AB", "T->XB", "X->AT", "A->a", "B->b"]

s1 = "aaaaabbbbb"
s1 = "aaaaaaaaabbbbbbbbb"

fnmain(G, "S", s1)

I've used cProfile as suggested by @wwii and below is the result:

When s1 = "aaaaabbbbb" enter image description here when s1= "aaaaaaaaabbbbbbbbb" enter image description here

Tak
  • 3,536
  • 11
  • 51
  • 93
  • 1
    Have you [Profiled](https://docs.python.org/3/library/profile.html) it?? Look for nested loops - the deeper the nesting the longer it will take. Try to refactor. – wwii Oct 21 '17 at 15:39
  • @wwii Thank you. I've updated my question to include the cProfile results and included a screenshot of the results. If you could please advise. – Tak Oct 21 '17 at 15:47
  • In the `else` suite of `fn` , there are nested loops three deep. The two inner loops iterate over a recursive call to the same function. It seems that the complexity is increasing exponentially. Looking at the link you posted, the pseudocode has many nested loops - so maybe you cannot get away with it. Try to see if your solution has **more** nesting than the algorithm in the link. – wwii Oct 21 '17 at 15:55
  • Just a quick comment. There is no need to call the function `fnmain`. The `def` makes us aware that it is a function, So you can just call it `main`. – tread Oct 21 '17 at 16:20
  • The program might get bogged down with garbage collecting. Try to avoid passing too many parameters, e.g. G could be kept in a global context. – yacc Oct 21 '17 at 18:46
  • @yacc I changed the G to be global but still very slow – Tak Oct 21 '17 at 21:42
  • Maybe there's more insight if you profile a heavy load. I mean 0.026s isn't very much. – yacc Oct 21 '17 at 21:51
  • 1
    @yacc I've updated the question modifying the input string which is a profile heavy load and updated the screenshot – Tak Oct 21 '17 at 21:55
  • I'd move G, w and T to global context and then check again. Most of the time the function checks T[i,j] and returns it since defined. That means, it's just a hell of a lot invocations. – yacc Oct 21 '17 at 22:30
  • I just noticed you had two strings s1 and s2 originally included which seem to have vanished. I suggest to update your question with a few strings and their respective processing time to illustrate the situation. – yacc Oct 21 '17 at 22:39
  • @yacc I've updated my question with two different input strings as s1. – Tak Oct 22 '17 at 11:24
  • Why is this recursive? The pseudo-code on the Wiki page doesn't look like that. Where is P? – yacc Oct 23 '17 at 02:38
  • @yacc I already developed it normally and it works fast but now trying to make it with recursion but looks like there is something wrong that is making it too slow – Tak Oct 23 '17 at 08:35

0 Answers0