I had a similar issue, where I had 1 folder which despite being deleted from Hard drive, was still showing as in Source Control (with a green + by the folder). It also showed me the content but if I tried to open any, an error message informed me the file wasn't there.
Clicking on any folder didn't show the delete button. I then realised that selecting a file did! If you don't have a file, create a text file and add it to the folder you want to delete (via Source Control). Click this file, note the delete shows. Select a folder and note that the delete disappears but only after half a second. This small delay is actually usable.
I clicked the file, so the delete button is enabled and hovered the mouse above the delete button. Using the keyboard, I did a CTRL + A and as soon as all files and folders were selected I clicked left mouse button. And it worked! Everything was removed, including (and this is the odd thing) the parent folder.