0

I'm trying to implent this network (https://arxiv.org/pdf/1604.03505) (source code file - https://gist.github.com/kanihal/8b981b75cb6f22264c0af03be79b243a) Is this the correct implementation of network given below using functional api?

I'm stuct on how to pass input to bi-lstm? should I use a merge layer? If yes then how?

How to do this using sequential model? using fork??

image

Here the code snippet for the model that I have written,

N = 3
fc7 = np.ndarray(shape=(1,4096), dtype=float) # dummy vgg fc7 is 1 row of 4096 numbers
cells_fc7 = np.array([[fc7 for j in range(N)] for i in range(N)])
#getfc7 for cells code here - dont run this for now
for i in range(N):
    for j in range(N):
        cells_fc7[i][j] = getfc7(cells[i][j])


#model code starts here
#list of inputs
cells_in=[]
for i in range(N):
    for j in range(N):
        # cells_rf[i][j] = sharedfc(cells_fc7[i][j])
        cells_in.append(Input(shape=(1,4096)))


#shared fully connected layer1
sharedfc = Dense(output_dim=500,input_dim=4096,activation='relu')
cells_rf=[]
# cells_rf = np.array([[rf for j in range(N)] for i in range(N)])
for i in range(N):
    for j in range(N):
        # cells_rf[i][j] = sharedfc(cells_fc7[i][j])
        cells_rf.append(sharedfc(cells_in[i][j]))

#replace with merge layer????
t = np.array(cells_rf)
top1=Bidirectional(LSTM(500, return_sequences=True), input_shape=(N*N, 500)) #input shape(num of vectors in the sequence, size of vector)
t1=top1(t)
top2=Bidirectional(LSTM(1000,return_sequences=True))
t_out=top2(t1)

b = np.array(cells_rf)
btm1=Bidirectional(LSTM(500, return_sequences=True), input_shape=(N*N, 500)) #input shape(num of vectors in the sequence, size of vector)
b1=btm1(a)
btm2=Bidirectional(LSTM(1000,return_sequences=True))
b_out=btm2(t1)


# we can then concatenate the two vectors:
merged_vector = merge([t_out, b_out], mode='concat')
n_classes = 80
sharedfc_out = Dense(output_dim=n_classes,input_dim=4000,activation='relu')

#partial counts
pc = np.ndarray(shape=(1,n_classes), dtype=float) 
cells_pc = np.array([[pc for j in range(N)] for i in range(N)])
outpc=[]
for i in range(N):
    for j in range(N):
        cells_pc[i][j] = sharedfc_out(merged_vector[N*i+j])
        outpc.append(sharedfc_out(merged_vector[N*i+j]))


model = Model(input=cells_in, output=outpc)
jaggi
  • 357
  • 1
  • 4
  • 17
  • Source code link gives 404 error! – SilverSurfer Mar 11 '17 at 18:23
  • Yes there is a mistake in the code, but I am not sure of the solution, you are passing a numpy array to the Dense layer of your model, and instead you should be passing it the Input layer. – Dr. Snoopy Mar 11 '17 at 18:48
  • @MatiasValdenegro fixed by changing cells_rf.append(sharedfc(cells_fc7[i][j])) to cells_rf.append(sharedfc(cells_in[i][j])) – jaggi Mar 11 '17 at 19:00
  • @MatiasValdenegro https://gist.github.com/kanihal/8b981b75cb6f22264c0af03be79b243a can you please look into this full implementation code, during merge whether concat axis=1 to be used? – jaggi Mar 11 '17 at 19:00
  • @MatiasValdenegro how do I make sequence of 9 vectors each of dimention 500 to feed into bi-lstms (line 89 in the gist link above), write now I'm using np.array(t) but that won't work? can you suggest an alternative? – jaggi Mar 11 '17 at 19:46
  • @MatiasValdenegro I have edited the question, I can see that you have answered lot of keras related questions here, can you please give a detailed answer for this? – jaggi Mar 11 '17 at 19:55

0 Answers0