On Windows, some binary files e.g. .obj are in COFF format, and others e.g. .exe and .dll are PE, which is COFF with an extra header. It seems you need to look at the file extension to know which to expect.
Is there a comprehensive list of which file extensions use COFF and which use PE?
There is NO "comprehensive list of which file extensions use COFF and which use PE".
Extensions are only used to help the system to "map" applications that will be used to start the selected file (e.g. *.jpg will be mapped to be started by MSPAINT.exe, *.doc will be mapped to be started by MSWORD.EXE...etc). This logical mapping is made through the registry.
The Microsoft Portable Executable (PE) Specification never mentions anything about the file extension.
There is NO "comprehensive list", there is NO list at all, because there is no need for such a list. A PE file is a file whose CONTENT corresponds to the specification. Period (which is what David Heffernan commented above)
You have executable with PDF extension (very common in malware), or TXT, ....or whatever you want, or even WITHOUT any extension! These "unusual" extensions for executable will of course have "side-effects" where you won't be able to double-click on these files. But, you could cmd>start these files, or programatically start these using many API or shell functions.
Since the system is able to start executable independently of their extensions, the parsing presents no problem at all!
Hope that helps! :-)
Look for the magic: a PE file will always start MZ due to its MS-DOS heritage. See the 'MS-DOS Header' section of An In-Depth Look into the Win32 Portable Executable File Format.
MZ was how DOS knew that this was an EXE file rather than COM - for historical reasons, some Windows console-mode utilities have a .com file extension but are actually PE-format files.
I cant comment but to add to Mike Dimmick's answer. Magic Numbers is how the executable file on linux is able to tell file types.
Extensions are used in windows to define default behaviors for parsing a binary. This can get confusing/dangerous, for example, if Program A knows how to handle/parse multiple file types (think Image Preview), a file could have the extension of one type, and be parsed as another. Windows will open Program A feeding it the file. Program A will parse the file and usually disregard the extension all together to determine how to handle the file.
This can be used in order to social engineer (trick) users into opening "familiar" file formats but hit code execution for more obscure formats.
