I don't speak English well, but I think it may be enough to explain this.
A translation in UIPanGestureRecognizer stands for a vector from where you started dragging to your current finger location, though the origin of this vector is {0, 0}. So all you need to determine the distance you dragged is another point of this vector. You get this point by calling :
recognizer.translationInView(self.view)
Then this point helped you setting a new location of your view.
But UIPanGestureRecognizer is indeed a "continuous" reporter, she will not forget the state after the last report. she didn't know that you have used up that part of translation(to re-locate your view), so the next time when "handlePan" is called, the translation is not calculated from previous location of your finger , it is from the original place where started your finger dragging!!
That's why you have to call:
recognizer.setTranslation(CGPointZero, inView: self.view)
every-time you used that translation to re-locate your view, as if you are telling the recognizer that you are going to start a new drag gesture.