1

I have images from a video and a look-up table with indexes. The goal is to make a new image from each frame by using the indexes in the look-up table. My code work using my laptop, but bit slow. My aim is to make it to work in TX2. Below is my code, not sure why it is slow. 

    cap1 = cv2.VideoCapture(2)
    with open('lookup_table.pkl', 'rb') as f:
        lut_idx = pickle.load(f)
    channels = 3 # Colored images
    dim = 480 # Square image 480 x480
    while (cap1.isOpened()):
        ret1, left = cap1.read()
        if ret1 == True:
            newImg = np.reshape(np.reshape(together, (dim * dim , channels))[lut_idx],
                                     (dim, dim, channels))  
            cv2.imshow('newImage', newImg)  # Show image

        if cv2.waitKey(1) & 0xFF == ord('q'):
            break

    cap1.release()
    cv2.destroyAllWindows()

What can I do to make it faster? Should I look into parallel processing? Any help is welcome. Thanks

Drona
  • 107
  • 11
  • First you should benchmark and see what's taking the time. Run your video reading code with no processing and time it. Then run your processing on a static frame without reading the video. Then you'll have a better idea what your maximum possible speed is and what you should optimise. – Mark Setchell Dec 05 '19 at 20:23
  • I used python profiler to identify which process takes the most time. It seems like reading image data and pickle table is the biggest one. But is there a way to make the indexing part faster?. – Drona Dec 05 '19 at 20:28
  • What are the times for acquiring the image and doing the lookup please? – Mark Setchell Dec 05 '19 at 20:46
  • What exactly do you want to achieve? Permutate pixels with lut_idx? Could provide some screenshots with exemplary input and expected output? – tstanisl Dec 05 '19 at 21:05
  • So I have two fisheye camera input with 235 Degree FOV. And I am trying to dewarp it. After wasting many months I have found a Master's work where they used a mathematical function to dewarp. It gets the job done to pretty well. The only issue is that it is made to dewarp images not video. So there are lot of for loops. So I decided to make a for-loop and an indexing look-up table. Now I can dewarp in real time, the only issue is that its not fast enough for two fisheye camera and TX2. I need the video feed to drive my robot. – Drona Dec 05 '19 at 21:27
  • Using profile from Pycharm, "waitkey" takes 3.0 % time, "resize" 1.54 %, "pickle_load" takes 1.1 % , imshow 0.4%. Rest is below 0.0% – Drona Dec 05 '19 at 21:29
  • I know in my code above I am missing the resize line. Basically I resize the image to 480 * 480 from 2880 x 2880. – Drona Dec 05 '19 at 21:33
  • Your profiling only accounts for 6% of the time - you need to be improving the performance of the other 94% which are unexplained, surely? If you don't give proper timings, nor any indication of what your lookup table looks like or what your input and output look like, I'm not very hopeful you'll get optimal help from the SO community. – Mark Setchell Dec 06 '19 at 22:51

1 Answers1

0

So, I was doing some image manipulation using a predefined lookup table. The table had the index values of where to grab pixels from the image. Doing this in a TX2 was difficult as TX2 has a weak CPU. The way to do this faster is by using GPU since TX2 has a huge GPU.CUPY or Numba can be used to solve the speed issue.

Drona
  • 107
  • 11