OpenGL is not a scene graph (gah, it seems every other question on OpenGL I answer begins with this statement). After you've drawn something, OpenGL no longer has any recollection of what you actually sent it. The old OpenGL selection mode is technically just testing if the geometry submitted is within the projections clip space range. On most OpenGL implementations selection mode falls back into software rendering mode, so you'll get a major performance hit.
There are several better ways to do selection (that's why selection mode has been removed from OpenGL after all). If it's just single vertices in specific geometry (like a selection rubberband) you're after, then you should perform the whole transformation of those points into normalized device coordinates yourself, sort them into some screen space spatial subdivision structure (2d Kd tree, quadtree, etc.) so that you can determine the point clicked on in O(log n) time – in contrast to the O(n) you'd have with selection mode, in which you have to "draw" the whole rubberband, so that all points are tested.
EDIT/Update
Since OpenGL is (just) a drawing API you also can't "drag around" things. You'll have to redraw them. Technically you should redraw the whole scene, or when starting a drag, draw the scene without the object about to be dragged into a texture (color and maybe depth), and then for each dragging step clear the view to the cached contents in the texture and then add the dragged objects in its updated position.