[Home] [Forum]

ESXi provides the ability to use raw device mapping (RDM) as a method to provide a VM with direct access to a LUN on a Fibre CHannel or iSCSI storage device. RDMs are useful should you have to share a LUN with a physical server or have SAN utilities running in the VM that will be able to access the LUN directly.

You can also use a RDM to provide a VM direct access to a local SATA drive that is connected a SATA controller. This method was first posted by Mário Simões in the forums here as a method to run RAID 5 in a VM on a SATA controller that did not support RAID. This method would also be useful for importing data from existing servers, but one should note that this is not officially supported. In the below example, the host is an Asus P5E-VM DO with a Seagate drive connected to an Intel ICH9 controller.

1) The first step of the process is to determine the disk that you'll want to use for the RDM. You can run fdisk -l and you'll be able to see all the disks that ESXi has access to. As shown below the system has a WD drive upon which ESXi is installed and hold the main datastore. An existing VMFS datastore is required as a pointer VMDK file will be created for the RDM and this must be stored on a VMFS datastore. The system also has a Seagate drive that is identified as t10.ATA_____ST3500630AS_________________________________________9QG3CC60 and is unused.

~ # fdisk -l

Disk /dev/disks/t10.ATA_____ST3500630AS_________________________________________9QG3CC60: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System

Disk /dev/disks/t10.ATA_____WDC_WD5000AAKS2D00YGA0________________________WD2DWCAS82934992: 500.1 GB, 500107862016 bytes
64 heads, 32 sectors/track, 476940 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes

Device Boot Start End Blocks Id System
/dev/disks/t10.ATA_____WDC_WD5000AAKS2D00YGA0________________________WD2DWCAS82934992p1 5 900 917504 5 Extended
/dev/disks/t10.ATA_____WDC_WD5000AAKS2D00YGA0________________________WD2DWCAS82934992p2 901 4995 4193280 6 FAT16
/dev/disks/t10.ATA_____WDC_WD5000AAKS2D00YGA0________________________WD2DWCAS82934992p3 4996 476941 483271704 fb VMFS
/dev/disks/t10.ATA_____WDC_WD5000AAKS2D00YGA0________________________WD2DWCAS82934992p4 * 1 4 4080 4 FAT16 <32M
/dev/disks/t10.ATA_____WDC_WD5000AAKS2D00YGA0________________________WD2DWCAS82934992p5 5 254 255984 6 FAT16
/dev/disks/t10.ATA_____WDC_WD5000AAKS2D00YGA0________________________WD2DWCAS82934992p6 255 504 255984 6 FAT16
/dev/disks/t10.ATA_____WDC_WD5000AAKS2D00YGA0________________________WD2DWCAS82934992p7 505 614 112624 fc VMKcore
/dev/disks/t10.ATA_____WDC_WD5000AAKS2D00YGA0________________________WD2DWCAS82934992p8 615 900 292848 6 FAT16

2) The next step is to run the command ls /dev/disks/ -l. This is used to determine the VML identifier for the disk. The VML ID will be used in the command to create the RDM.

ls /dev/disks/ -l
-rw------- 1 root root 500107862016 Oct 22 07:15 t10.ATA_____ST3500630AS_________________________________________9QG3CC60
-rw------- 1 root root 500107862016 Oct 22 07:15 t10.ATA_____WDC_WD5000AAKS2D00YGA0________________________WD2DWCAS82934992
-rw------- 1 root root 939524096 Oct 22 07:15 t10.ATA_____WDC_WD5000AAKS2D00YGA0________________________WD2DWCAS82934992:1
-rw------- 1 root root 4293918720 Oct 22 07:15 t10.ATA_____WDC_WD5000AAKS2D00YGA0________________________WD2DWCAS82934992:2
-rw------- 1 root root 494870224896 Oct 22 07:15 t10.ATA_____WDC_WD5000AAKS2D00YGA0________________________WD2DWCAS82934992:3
-rw------- 1 root root 4177920 Oct 22 07:15 t10.ATA_____WDC_WD5000AAKS2D00YGA0________________________WD2DWCAS82934992:4
-rw------- 1 root root 262127616 Oct 22 07:15 t10.ATA_____WDC_WD5000AAKS2D00YGA0________________________WD2DWCAS82934992:5
-rw------- 1 root root 262127616 Oct 22 07:15 t10.ATA_____WDC_WD5000AAKS2D00YGA0________________________WD2DWCAS82934992:6
-rw------- 1 root root 115326976 Oct 22 07:15 t10.ATA_____WDC_WD5000AAKS2D00YGA0________________________WD2DWCAS82934992:7
-rw------- 1 root root 299876352 Oct 22 07:15 t10.ATA_____WDC_WD5000AAKS2D00YGA0________________________WD2DWCAS82934992:8
l--------- 0 root root 1984 Jan 1 1970 vml.01000000002020202020202020202020203951473358423630535433353030 -> t10.ATA_____ST3500630AS_________________________________________9QG3CC60
l--------- 0 root root 1984 Jan 1 1970 vml.0100000000202020202057442d574341533832393031393932574443205744 -> t10.ATA_____WDC_WD5000AAKS2D00YGA0________________________WD2DWCAS82934992
l--------- 0 root root 1984 Jan 1 1970 vml.0100000000202020202057442d574341533832393031393932574443205744:1 -> t10.ATA_____WDC_WD5000AAKS2D00YGA0________________________WD2DWCAS82934992:1
l--------- 0 root root 1984 Jan 1 1970 vml.0100000000202020202057442d574341533832393031393932574443205744:2 -> t10.ATA_____WDC_WD5000AAKS2D00YGA0________________________WD2DWCAS82934992:2
l--------- 0 root root 1984 Jan 1 1970 vml.0100000000202020202057442d574341533832393031393932574443205744:3 -> t10.ATA_____WDC_WD5000AAKS2D00YGA0________________________WD2DWCAS82934992:3
l--------- 0 root root 1984 Jan 1 1970 vml.0100000000202020202057442d574341533832393031393932574443205744:4 -> t10.ATA_____WDC_WD5000AAKS2D00YGA0________________________WD2DWCAS82934992:4
l--------- 0 root root 1984 Jan 1 1970 vml.0100000000202020202057442d574341533832393031393932574443205744:5 -> t10.ATA_____WDC_WD5000AAKS2D00YGA0________________________WD2DWCAS82934992:5
l--------- 0 root root 1984 Jan 1 1970 vml.0100000000202020202057442d574341533832393031393932574443205744:6 -> t10.ATA_____WDC_WD5000AAKS2D00YGA0________________________WD2DWCAS82934992:6
l--------- 0 root root 1984 Jan 1 1970 vml.0100000000202020202057442d574341533832393031393932574443205744:7 -> t10.ATA_____WDC_WD5000AAKS2D00YGA0________________________WD2DWCAS82934992:7
l--------- 0 root root 1984 Jan 1 1970 vml.0100000000202020202057442d574341533832393031393932574443205744:8 -> t10.ATA_____WDC_WD5000AAKS2D00YGA0________________________WD2DWCAS82934992:8

3) To create the RDM you'll use the command vmkfstools. In the below example I've created a folder called RDMs on a datastore to hold the VMDK mapping file that will be created for the RDM. The vmkfstools command is run the the -r switch and I've also specified that the controller should be set to be LSILogic (instead of the default BusLogic). The command created the mapping files RDM1.vmdk and RDM1-flat.vmdk. The contents of RDM1.vmdk are shown below and while the RDM1-flat.vmdk file appears to be 500 GB in size it actually takes up next to no disk space.

/vmfs/volumes/4a7cf921-017eb919-bb32-000423e540c6/RDMs # vmkfstools -r /vmfs/devices/disks/vml.01000000002020202020202020202020203951473358423630535433353030 RDM1.vmdk -a lsilogic
/vmfs/volumes/4a7cf921-017eb919-bb32-000423e540c6/RDMs # ls -l
-rw------- 1 root root 500107862016 Oct 22 07:28 RDM1-rdm.vmdk
-rw------- 1 root root 459 Oct 22 07:28 RDM1.vmdk
/vmfs/volumes/4a7cf921-017eb919-bb32-000423e540c6/RDMs # cat RDM1.vmdk
# Disk DescriptorFile

# Extent description
RW 976773168 VMFSRDM "RDM1-rdm.vmdk"

# The Disk Data Base

ddb.virtualHWVersion = "7"
ddb.longContentID = "99893594518fe8d6d9454db3ef5ed87c"
ddb.uuid = "60 00 C2 9f 7d 63 e6 e1-21 a4 da 24 ef f6 af 0b"
ddb.geometry.cylinders = "60801"
ddb.geometry.heads = "255"
ddb.geometry.sectors = "63"
ddb.adapterType = "lsilogic"

4) Once the RDM has been created, you'll be able to add it to a VM. As shown in the below image, a VM was edited to add an existing disk file and the RDM1.vmdk file was chosen. The VM was then powered on and Windows was installed to the RDM. After the install the VM displayed the drive as a VMware Virtual Disk. At this point it would have been possible to remove the drive from the server, plug it into a physical server and access the data written by the OS install onto the disk as there was no VMFS layer present as is the case when a VMDK is stored on datastore.



In the above example a virtual RDM was created which is useful if you want to take snapshots of the VM. You can also create physical RDMs which are required if you plan to share a RDM with a physical host or if you need to run SAN management software within the VM. With a physical RDM nearly all SCSI commands are passed directly to the disk rather being handled by the VMkernel. With a physical RDM you are not able to use snapshots so it is not too useful for local storage, but the below process describes what you need to do.

1) You'll first need to determine the VML ID for the disk as shown in steps 1 and 2 above. Once that is done you'll be able to run vmkfstools to create the physical RDM. But instead of using the -r switch you'll use -z (or --createrdmpassthru) to create the VMDK mapping file. As with the virtual RDM, a VMDK file is created with the relevant data for the RDM, but in this case the other file is in the format of rdm.vmdk instead of flat.vmdk. As with the flat.vmdk file, the rdm.vmdk file shows the size of the raw physical disk but takes up next to no space on the VMFS datastore.

/vmfs/volumes/4a7cf921-017eb919-bb32-000423e540c6/RDMs # vmkfstools -z /vmfs/devices/disks/vml.01000000002020202020202020202020203951473358423630535433353030 RDM1P.vmdk -a lsilogic
/vmfs/volumes/4a7cf921-017eb919-bb32-000423e540c6/RDMs # ls -l
-rw------- 1 root root 500107862016 Oct 22 07:28 RDM1-rdm.vmdk
-rw------- 1 root root 485 Oct 22 19:32 RDM1.vmdk
-rw------- 1 root root 500107862016 Oct 22 19:33 RDM1P-rdmp.vmdk
-rw------- 1 root root 472 Oct 22 19:33 RDM1P.vmdk
/vmfs/volumes/4a7cf921-017eb919-bb32-000423e540c6/RDMs # cat RDM1P.vmdk
# Disk DescriptorFile

# Extent description
RW 976773168 VMFSRDM "RDM1P-rdmp.vmdk"

# The Disk Data Base

ddb.virtualHWVersion = "7"
ddb.longContentID = "8291253e77a09bcf3ea5eb21068a56bb"
ddb.uuid = "60 00 C2 9a f6 7b 03 7f-b3 ab 00 d0 6e 32 95 eb"
ddb.geometry.cylinders = "60801"
ddb.geometry.heads = "255"
ddb.geometry.sectors = "63"
ddb.adapterType = "lsilogic"
/vmfs/volumes/4a7cf921-017eb919-bb32-000423e540c6/RDMs #

2) Once you have created the physical RDM you can add it to a VM. This is where I ran into my first problem with SATA RDMs. I found that a VM would not boot from a physical RDM (it would hang after the main BIOS screen) but it would be just fine if I recreated a virtual RDM with the same physical disk. I was able to add the physical RDM as a data disk to existing servers and those would boot without any issues. In the first image below you'll notice that the Compatibility Mode for the RDM is listed as Physical. In the second image, you can see the the VM sees the actual hard drive model (Seagate - ATA ST3500630AS) instead of as a VMware virtual disk as was the case in the virtual RDM example above.



Name (required)

Web Site (optional)

Email address (required - will not be displayed)

Comment (required)

Please enter code


Copyright © 2011 - Dave Mishchenko