If I'm not mistaken, this is what you'll need to do:
The virtual disk does need to be 80GB or greater.
You need to set the SCSI controller to BusLogic
During the first phase of the WHS install (initial GUI phase), if you're prompted that a compatible hard drive can not be found, you'll need to load the BusLogic driver from a virtual floppy (vmscsi-1.2.0.4.flp)
http://download3.vmware.com/software/vmscsi-1.2.0.4.flp
During the second phase of WHS install, during the "blue screen" phase, you need to press F6 to load the driver from floppy again.
During the "name your server" phase of set up, you need to disconnect the virtual NIC, otherwise you'll get errors stating that a duplicate name already exists on the network.
I just installed WHS on ESXi 3.5 U4 yesterday and these are the steps I had to take to get it installed.