This Medium article, while in Python might offer some further reading on the topic. It seems to indicate that once you have obtained the K and D values from fisheye::calibrate you can then use fisheye::initUndistortRectifyMap (note that D can be empty, as it seems to be in your case) to generate two Maps, which can then be used with remap to transform the warped input image into a dewarped output image (further reading on remapping here).
In python (for anyone looking at this later) this is done as such:
ap1, map2 = cv2.fisheye.initUndistortRectifyMap(K, D, np.eye(3),
K, DIM, cv2.CV_16SC2)
undistorted_img = cv2.remap(img, map1, map2,
interpolation=cv2.INTER_LINEAR,
borderMode=cv2.BORDER_CONSTANT)