OpenGL does not store what you draw. If you draw a line in OpenGL, then OpenGL will take that line, perform various math operations on it, and write pixels into a framebuffer that makes the shape of a line. OpenGL does not remember that you drew a line; all OpenGL can do is write pixels to the framebuffer.
The general idea is that it is up to the user of OpenGL to remember what they drew. So if you draw two lines, you should remember the coordinates you gave for those two lines. Therefore, if you want to "erase" a line, what you do is clear the screen and redraw everything except that line.
This isn't as silly as it may sound. Many OpenGL applications are constantly redrawing the screen. They show a frame, draw a new frame, then show that frame, etc. This provides the possibility for animation: changing what gets drawn and where it gets drawn from frame to frame. This creates the illusion of movement.