It's totally down to hhc.exe, nothing else. The trick is to look at the %ERRORLEVEL% after it runs. It returns "1" despite success. This could be used in a custom command to warn the user it's spurious, if the hhc.exe run is isolated from other stuff.
HHC.exe is using HHA.dll. About HHA.dll info has not been published. Microsoft grant the HHA interface information under non-disclosure agreement (NDA) to approved help ISV's.
D:\_working>"C:\Program Files (x86)\HTML Help Workshop\hhc" foobar.hhp
Microsoft HTML Help Compiler 4.74.8702
Compiling d:\_working\foobar.chm
...
Compile time: 0 minutes, 1 second
22 Topics
87 Local links
2 Internet links
0 Graphics
Created d:\_working\foobar.chm, 305,338 bytes
Compression decreased file by 53,639 bytes.
D:\_working>echo %errorlevel%
1
To go around this and continue you need to add if not %errorlevel% 1 exit /B 1
in a batch file.
@echo off
REM -----------------------------------------
REM batch file is located in D:\_batch
REM HH project file is located in D:\_working
REM -----------------------------------------
cd ..\_working
echo '//--- HH Compiler start --------------------------------------
"C:\Program Files (x86)\HTML Help Workshop\hhc" foobar.hhp
echo '//--- HH Compiler end --------------------------------------
echo '//--- errorlevel -------------------------------------------
echo %errorlevel%
echo '//------------------------------------------------------------
if not %errorlevel% 1 exit /B 1
And a python script calling this batch:
print ("*******************************")
print ("We compile a CHM help file ...")
print ("*******************************")
# First import the 'ctypes' module. ctypes module provides C compatible data types and allows calling functions in DLLs or shared libraries.
import ctypes # An included library with Python install.
# ctypes.windll.user32.MessageBoxW(0, "Open CHM", "Your title", 1) # OK only
messageBox = ctypes.windll.user32.MessageBoxA
# documentation: https://msdn.microsoft.com/en-us/library/windows/desktop/ms645505(v=vs.85).aspx
returnValue = messageBox(None,"Compile Help Module (CHM) now?","CHM and Python",1) # 1=OK Cancel, 2=Cancel, Retry, Ignore
if returnValue == 1:
print("Pressed OK")
# How to compile a chm file in Python?
# ---------------------------------
import os
os.system("D:/_batch/run-hhc.bat")
elif returnValue == 2:
print("user pressed cancel button!")

You may be interested in calling a CHM from a python script:
# How to open a chm file in Python?
# ---------------------------------
# os.system("hh.exe D:/UserData-QGIS-Python/Projekte/ConnectChm/CHM-example.chm")
import os
os.system("hh.exe D:/UserData-QGIS-Python/Projekte/ConnectChm/CHM-example.chm::/garden/garden.htm")