View unanswered posts | View active topics It is currently Sun Oct 20, 2019 4:17 am



Reply to topic  [ 13 posts ]  Go to page 1, 2  Next
 PCI Passthrough - BioStar TA890FXE & Sil3132 
Author Message

Joined: Thu Jan 20, 2011 7:49 pm
Posts: 7
Reply with quote
Post PCI Passthrough - BioStar TA890FXE & Sil3132
Good morning! Here is a breakdown of my current situation:

I am trying to put unRaid (not fully supported, but workable), WHS & Win7 on my ESXi box.

My current MB doesn't support PCI Passthrough. I was able to use a custom OEM.tgz to get ESXi to see the Sil3132, but it only sees the first hard drive in my external enclosure, not all four drives. If I boot directly into unRaid on the same hardware, while the SIL3132 card bios doesn't see all four drives, unRaid does once it's loaded. So I know unRaid works.

My current thought is, if I had a MB that supported PCI passthrough, I just pass through the SIL3132 eSATA card to the unRaid VM, and even if ESXi can't see all four drives, unRaid will when it boots.

Does that jive with how PCI passthrough works? If so, I'm going to pull the trigger on the BioStar TA890FXE as that has a confirmed IOMMU bios. Before doing so, I just wanted to bounce it off the community and see if my plan was sound.

Thank you in advance for any help provided!

- Cha


Mon Jan 31, 2011 5:44 am
Profile
Site Admin

Joined: Mon Mar 16, 2009 10:13 pm
Posts: 3880
Reply with quote
Post Re: PCI Passthrough - BioStar TA890FXE & Sil3132
With VMDirectpath the VM has full control over the PCI(e) device so in this case the VM would be able to see all disks attached. There are no driver dependecies for ESXi that would limit how the device would perform or function.

If you're looking at a motherboard that hasn't been tested with VMdirectpath on ESXi specifically I would check to see if it's been tested for IOMMU with other hypervisors like Xen.

ESXi has to enable all devices on the same PCI bus for VMdirectpath due to bus sharing so ideally look for a MB that will seperate the PCI devices onto seperate buses.

_________________
Dave Mishchenko
VMware vExpert 2009-2013
Image
Now available - VMware ESXi: Planning, Implementation, and Security
Also available - vSphere Quick Start Guide


Mon Jan 31, 2011 9:31 am
Profile

Joined: Thu Jan 20, 2011 7:49 pm
Posts: 7
Reply with quote
Post Re: PCI Passthrough - BioStar TA890FXE & Sil3132
Dave, thank you for the response! I found the BioStar board tested with Xen, so I suppose that should suffice.

Does anyone have any recommendations, like the MSI 890FXA-GD70? According to http://forums.amd.com/forum/messageview.cfm?catid=383&threadid=134410 it's another IOMMU confirmed board.

Thanks!


Mon Jan 31, 2011 11:18 am
Profile

Joined: Sat Feb 05, 2011 9:27 pm
Posts: 5
Reply with quote
Post Re: PCI Passthrough - BioStar TA890FXE & Sil3132
I was actually about to ask about this specific board as well.

I need to upgrade my current NAS/ VM host from OpenSolaris/ XVM (can't justify the $1,000/ socket just for patches from Oracle). Since the number of allowed DirectPath devices in ESXi just got bumped up, it has now a feasible alternative.

The hardware I'm looking at getting is:
    - Biostar TA890FXE (confirmed IOMMU and unbuffered-ECC support by manufacturer)
    - Some Phenom II chip (cheapest X6 probably)
    - 16GB (4 x 4) ECC DDR3

Existing hardware consists of:
    - 2 x LSI 1068e SAS HBAs (pci-e)
    - 1 x Realtek 8169-based NIC (pci)
    - 1 x Realtek 8169-based NIC (pci-e)
    - lots of disk...

Environment would be:
    - ESXi 4.1 host on CF (IDE to CF adapter, hardware mirrored) - on-board NIC assigned as management
    - VMFS storage for VM images on an old SSD (fairly light images, ~1G ea)
    - FreeBSD NAS VM (with 2 x HBA, 1 x e-sata and pci-e NIC pasthru), ZFS-backed storage from the HBAs
    - Linux Media Server/ VoIP VM (with 1 pci NIC passthru)

My concerns are around:
    - whether the onboard pci devices will get fuddled if i try to allocated the PCI nic to a VM (i.e. I'm not sure how the devices are on the bus)
    - whether I can actually configure a single e-sata port as passthru to a VM, since it's simply one of the 6 ports provided by the southbridge. Will all the ports get mapped to the VM/ 5 internal ports blocked?

I haven't been able to put my hands on a block diagram yet of the board, so any information/ insight would be appreciated.

<edit> To be more precise, if anyone has this board, can they post the output of 'lspci -tvkbQ', which would assist greatly in determining the structure of the bus. Thanks<edit>

~peace
/trini


Sat Feb 05, 2011 10:03 pm
Profile

Joined: Thu Jan 20, 2011 7:49 pm
Posts: 7
Reply with quote
Post Re: PCI Passthrough - BioStar TA890FXE & Sil3132
Okay, so I bought the ASRock 890FX Deluxe 4 motherboard, with confirmed IOMMU support. I've enabled IOMMU support, and selected the Sil3132 card for passthrough. I passed it through to my unRaid VM. The unRaid VM doesn't see any of the drives attached to it.

When I boot to unRaid directly, it sees the drives no problem.
In ESXi, when I run lspci I see the Sil3132 card, but none of the drives attached to it.

What am I missing? My plan is falling apart! lol Thanks!


Thu Mar 10, 2011 8:45 am
Profile
Site Admin

Joined: Mon Mar 16, 2009 10:13 pm
Posts: 3880
Reply with quote
Post Re: PCI Passthrough - BioStar TA890FXE & Sil3132
Quote:
I passed it through to my unRaid VM. The unRaid VM doesn't see any of the drives attached to it.


Does the Sil3132 appear to be functioning OK within the VM (besides the lack of disks)?


Quote:
In ESXi, when I run lspci I see the Sil3132 card, but none of the drives attached to it.


That's not necessarily a problem. ESXi is going to pass the entire card up to the VM and it doesn't have to "work" at the kernel level for ESXi.

_________________
Dave Mishchenko
VMware vExpert 2009-2013
Image
Now available - VMware ESXi: Planning, Implementation, and Security
Also available - vSphere Quick Start Guide


Thu Mar 10, 2011 9:54 am
Profile

Joined: Thu Jan 20, 2011 7:49 pm
Posts: 7
Reply with quote
Post Re: PCI Passthrough - BioStar TA890FXE & Sil3132
Hard to tell, but i believe so. When i run lspci from the VM, it sees the Sil3132 so i assume so.

Edit: is there a good way to check if its working?


Thu Mar 10, 2011 9:59 am
Profile

Joined: Thu Jan 20, 2011 7:49 pm
Posts: 7
Reply with quote
Post Re: PCI Passthrough - BioStar TA890FXE & Sil3132
i am really at a loss here. I can't seem to figure out why this Sil3132, which is properly set to passthrough, won't work in the unRaid VM, but does work in straight up unRaid.

The MB supports IOMMU, ESXi allows me to pass it through, there are no error messages, etc.

What am I missing? Please tell me it's something obvious! lol Thank you!

EDIT: How do I tell if something is being passed through? The other item ESXi doesn't recognize natively is the Marvel 9120 Sata Controller on the Motherboard. It adds an extra 2 SATA ports. I passed both that and the Sil3132 through to unRaid, but none of the drives attached to the Sil3132 or the Marvel 9120 show up. I don't get it... what am I missing? Thank you!


Fri Mar 11, 2011 6:20 am
Profile
Site Admin

Joined: Mon Mar 16, 2009 10:13 pm
Posts: 3880
Reply with quote
Post Re: PCI Passthrough - BioStar TA890FXE & Sil3132
Have you tried another guest OS just to eliminate that possibility?

_________________
Dave Mishchenko
VMware vExpert 2009-2013
Image
Now available - VMware ESXi: Planning, Implementation, and Security
Also available - vSphere Quick Start Guide


Fri Mar 11, 2011 2:12 pm
Profile

Joined: Thu Jan 20, 2011 7:49 pm
Posts: 7
Reply with quote
Post Re: PCI Passthrough - BioStar TA890FXE & Sil3132
Good call. I installed Windows 7 and again, it sees the Sil3132 as a working, valid device in Device Manager, but no drives. So it's not unRaid that's not passing it through, it's something to do w/ ESXi. I mean, pass through means just that, pass through, right? So it should just expose the device and whatever is attached to it.

Also, the Marvel 9120 I'm passing through doesn't even appear in Windows Device Manager... very strange. I do see it as a device in unRaid when I run lspci, but again, not the drive attached to it. Thanks again for your help!


Fri Mar 11, 2011 5:38 pm
Profile
Display posts from previous:  Sort by  
Reply to topic   [ 13 posts ]  Go to page 1, 2  Next

Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group.
Designed by STSoftware for PTF.