Hi,

I'm not sure where to post this so I'll post here. I've noticed a lot of ESX white box builds, but I'm looking to try to build one that I can also test out iSCSI and fibre cards. I would like to try to do it the proper way by having:

(2) dual port iSCSI HBA cards
(2) dual port FC HBA cards

But I noticed most of the motherboards on newegg.com and tigerdirect.com don't come with that many PCI cards anymore. They are usually 1-2 PCI cards.

I was wondering if someone can give me a little push in the right direction on what kind of mobo, I can purchase to build a whitebox ESX server to test iSCSI/fibre SAN's?

My design is to have (2) ESXi 4.0 servers connecting to an iSCSI SAN and a fibre SAN at the same time utilizing different switches(obviously) to isolate traffic. I want to also be able to test Vmotion/DRS/HA.

I looked at vm-help's HCL list and quite frankly, with all the motherboards listed, I can't tell which one has 4+ PCI available slots.

Could anyone give me a little direction or am I stuck having to buy some used brands like HP or Dell?

TIA!!

I just read this article from 2008, it's a little old. It talks about the possible death of iSCSI HBA cards.

http://blogs.netapp.com/storage_nuts_n_ ... -is-t.html


I'm wondering if this article is for VMware or Microsoft OS point of view.

It talks about how MS's iSCSI SW initiator works very well but how do you run Microsoft's SW initiator using VMware? I don't think you do...

I know I am still NEW to the game, but do people really implement iSCSI HBA's or is that just something talked about in books as best practice, but not really practiced....?

With iSCSI and ESXi you have 3 options
1) Use a hardware iSCSI HBA (which is what the article refers to).
2) Use the ESX(i) software iSCSI initiator
3) Use an iSCSI initiator within the guest OS

Aside - you can also do HA / vMotion on NFS.

With all three options you would want a dedicated storage vSwitch and network ports for a production roll out. So lets say you had 4 NIC ports in your host, 2 would be in a vSwitch for VM and management traffic and the other two would be setup for iSCSI. If you used an HBA, then you would just need 2 NIC ports in the host for VM / management traffic and the HBA ports would provide network access for iSCSI.

Why the MS initiator over VMware's software. The MS one has performed better in the past, but ESX(i) 4 has made significant improvements so this would be an item to test in your environment. Using the ESXi initiator is easier to setup and manage (i.e. one iSCSI connection verus one for each VM accessing the iSCSI LUNs) and your VMs won't have to be setup with network access on the iSCSI network which has security implications.

If you do plan to pick up iSCSI or FC HBAs make sure to stick with the official HCL (www.vmware.com/go/hcl). For servers look at SuperMicro MBs. They have both server and workstation chipset MBs that have 4+ PCI / PCI-E slots.

What sort of load do you expect to test? I'd stick with the ESXi software initiator and if you go with FC just a single dual port.

Thanks Dave,

For testing and learning purposes, I'd like to actually try all three options.

I'm having a hard time with VMware's HCL website. Maybe I'm searching wrong but it seems kinda hard to find stuff.

For example, I have a Gigabyte GA-8SIMLH motherboard, it's an old P4 socket 478 mobo, when I search for Gigabyte, I only get a handful of results.

Does VMware limit it's results to just recent hardware?

Another example, I have:

(3) Pentium 4 3.0 ghz
(2) Pentium 4 2.6 ghz

I thought it would be pretty easy to look up on VMware's HCL to see if they support these old CPU's, but I don't see anything for that.

But back to your question, I don't expect any load for my testing/learning, although it would be nice to create load so I can play with the DRS/HA and force VM to move my high load VM's to other ESX server. That would be pretty fun.

As for vmotiom/drs/ha, I didn't realize that you can do that with NAS. I thought that was one of the downfalls of using NAS, but after looking at the features matrix, it looks like I can. (with other pitfalls of NAS of course)

I wanted to utilized my spare Pentium 4's so I can use the extra money to buy HBA cards, fibre switch etc...so I'm hoping that I can squeak by with them.

The other great thing about my spare P4's is that three are the same speed, and two are the same speed. So I can play with vmotion.

If you think they are too old, then I suppose I will just have to find a way to fork out some more cash to build the proper sandbox.

I'm also pretty sure that the P4 2.6 ghz doesn't support HT, and definitely doesn't support VT.

But the P4 3.0 ghz supports HT, but not VT....

VMware's HCL just lists server class machines. I don't think you'll even find any workstation class PCs on it.

For vSphere you'll need 64 bit CPUs. 32 are Ok for ESXi 3.5. For either version you'll need Intel VT to support 64 bit guests.

I'm not sure if I need any 64-bit guest OS's for learning/testing purposes. Is there something different or unique with them that is worth the time to install them?

Are there 64-bit "features" that are only activated or used when you install a 64-bit OS? Just curious. I do have quite a few 64-bit Intel's laying around but none are the same CPU clock, so they would have to be standalone ESX builds.

Windows 2008 R2 is 64 bit only, but for the rest of the OSes you can use 32 or 64 bit.

With ESX(i) you can setup a CPU mask on a VM to be able to vmotion between similar CPUs - http://kb.vmware.com/selfservice/micros ... &stateId=0 0 45580359

This is great, going to print this out! You really are an expert, you have links and answers to everything! LOL...

I suggest Supermicro X8SAX

This board has 3 PCI-e (2x16 and 1x4), 1 pci and 2 pci-x, the only downside with this board is it doesn't have onboard video, otherwise it's a perfect server board for ESX. It also supports all the latest VT features, VT-x, VT-d, VMDirectPath, etc.

It has 2 onboard 82574L gb nic, you can get very cheap Intel Pro 1000 MT dual port nics for less than $20, Qlogic 2340 FC HBA for $20 to test all the features you want. You still have 3 PCI-e to add new cards such as RAID, new 4Gb FC HBA, etc.

Fantastic! I will look this up now. The Qlogic FC HBA for $20 seems very cheap no? Even on eBay those Qlogic's go for quite a bit....

I've also never heard of a motherboard that comes with NO on-board video, that's pretty funny...I'll take a look now...