In some cases when you install ESX 3.5 on a system with a SATA controller, the install will complete but will fail to boot with the error "mounting root failed". The problem is that ESX does not recognize the specific SATA but you may be able to edit the PCI id database files to get ESX to load a driver that is compatible with the SATA controller that you are using. You are not able to add Linux drivers to an ESX system, but ESX does install a number of drivers for SATA controllers including models from Intel, nVidia and Promise Technology.
Reinhard Partmann posted a method on this VMware forum thread using a boot CD to edit the PCI id files. The below method uses the service console of the ESX install. The below example was done on a system with a ECS GEFORCE6100SM-M2 /AMD Sempron 3000+ which uses the NVIDIA Geforce6100 chipset for storage.
1) Setup a single SATA with no RAID.
2) Installed an Intel Pro/1000 NIC (the onboard is not supported)
3) Installed ESX
4) On the first reboot, I selected the Service Console only (troubleshooting mode) boot option.
5) Let the host boot up and logged in.
6) Edited the file /etc/vmware/pciid/sata_nv.xml - changed the last entry to use a device ID of 03f6 (and I updated the device name to MCP61 SATA Controller). I was able to determine the device ID by running lspci and looking for the device ID of an unknown IDE / SATA controller.
7) After saving the file, I ran esxcfg-pciid
8) Rebooted and the controller came up fine.
I did have to re-format the default vmfs partition with vmkfstools -C vmfs vmhba32:0:0:x, as the VI client did not see any free space on the drive. In hindsight I would not create a VMFS partition during the install next time I do this.