The questions how to open a selected file in a file explorer has been asked and answered thouroughly but nobody ever explained how to do it without downloading a plugin. At least as far as I know.
So the question I had was: How can I add "Open in Explorer" to Eclipse without downloading or installing an extra tool?
Turns out there is a quite simple solution to this problem. All it needs is the following line:
explorer /select,%1
Write this into a empty file and save as Explorer.bat. Now go to Preferences - General - Editors - File Association, search or create your favorite file extension and add an associated editor. After clicking the "Add" button, select "External program", click "Browse...", search and select your Explorer.bat file, confirm everything and you are done. Go to a file you just associated, right click, select "Open with", then click "Explorer" and the containing directory should open up.