View unanswered posts | View active topics It is currently Wed May 22, 2013 9:00 am



Reply to topic  [ 4 posts ] 
 ESXi 4.1 U1 on P5NTWS crashes under heavy I/O load 
Author Message

Joined: Thu Apr 21, 2011 6:32 am
Posts: 3
Post ESXi 4.1 U1 on P5NTWS crashes under heavy I/O load
Hello,

first post, hoping to get some thoughts. :)

I'm running ESXi 4.1 U1 on a P5NT WS based machine.

I have a WinXP Pro SP3 system (and some other VMs) running on that host to do some administrative tasks.
That works without any hazzle. The system runs straight and stable for several weeks.

Now I decided to add fileserver functionality to it and shared some space on the HDD. Therefore I added a new datastore and created a shared folder on it.

When I copy many files (total 6GB in around 4.800 files) from another Windows host to the shared folder, the complete host gets stuck after some time and the HDD-activity LED doesn't turn off anymore. Administration screen doesn't respond anymore, too.

This happens on any SATA port on the board. Also with additional installed fixed disks. RAID functionality is turned off in BIOS.
I also tested it with the Turnkey FileServer virtual appliance - same behaviour.

Does anyone have an idea what may cause this error? I would greatly appreciate it.

Thanks in advance.
Robert


Thu Apr 21, 2011 6:47 am
Profile
Site Admin

Joined: Mon Mar 16, 2009 10:13 pm
Posts: 3874
Post Re: ESXi 4.1 U1 on P5NTWS crashes under heavy I/O load
Hi Robert, what sort of NIC is the system using? I would suggest using a tool like iperf - http://sourceforge.net/projects/iperf/ to stress test the network between the VM and the physical host. iperf doesn't rely on storage so it may help to isolate the issue.

Do you have an issue copying files between your VMs? If inter-VM copies work fine then it may point to a network issue.

_________________
Dave Mishchenko
VMware vExpert 2009-2012
Image
Now available - VMware ESXi: Planning, Implementation, and Security
Also available - vSphere Quick Start Guide


Thu Apr 21, 2011 9:06 am
Profile

Joined: Thu Apr 21, 2011 6:32 am
Posts: 3
Post Re: ESXi 4.1 U1 on P5NTWS crashes under heavy I/O load
Hello Dave,

thanks for your fast reply.

Will try your suggestions on Tuesday.

I noticed some errormessages in the vmkernel log view (Alt+F12):

The problems start with this line written in red on the log screen:
Code:
ALERT: APIC: 1823: APICID 0x00000001 - ESR =0x40


Then some lines later:

Code:
(6)ata4: ata_scsi_abort: cmd 0x41000e013600, code 2a
(6)ata3: ata_scsi_abort: cmd 0x41000e00e000, code 2a
(6)ata4: ata_port_reset: 1 cmds aborted, freezing port
(3)ata4.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action...
(3)ata4.00: (BMDMA stat 0x4)
(6)ata3: ata_port_reset: 1 cmds aborted, freezing port
(3)ata3.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action...
(3)ata3.00: (BMDMA stat 0x24)
(6)ata4: soft resetting port
(6)ata3: soft resetting port
(6)ata4: SATA link up 3.0 Gbps (SStatus 123 SControl 30...
(6)ata3: SATA link up 3.0 Gbps (SStatus 123 SControl 30...
ScsiDeviceIO: 1672: Command 0x2a to device "t10.ATA_____
FS3: 7412: Waiting for timed-out heartbeat [HB state ab....
ScsiDeviceIO: 1672: Command 0x28 to device "t10.ATA_____
BC: 3837: Failed to flish 1 buffers of size 8192 each.....
(4)ata4.00: qc timeout (cmd 0xec)
(4)ata4.00: failed to IDENTIFY (I/O error, err_mask=0x4
(3)ata4.00: revalidation failed (errno=-5)
(4)ata4: failed to recover some devices, retrying in 5...
(4)ata3.00: qc timeout (cmd 0xec)
(4)ata3.00: failed to IDENTIFY (I/O error, err_mask=0x4
(3)ata3.00: revalidation failed (errno=-5)
(4)ata3: failed to recover some devices, retrying in 5...
(6)ata4: hard resetting port
(6)ata3: hard resetting port
(4)ata4: port is slow to respond, please be patient
(4)ata3: port is slow to respond, please be patient
WARNING: VSCSI: 3258: handle 8193(vscsi4:1) WaitForCIF:...
...
(3)ata4: port failed to respond (30 secs, Status 0x80)
(3)ata4: COMRESET failed (device not ready)
(4)ata4: hardreset failed, retrying in 5 secs
(3)ata3: port failed to respond (30 secs, Status 0x80)
(3)ata3: COMRESET failed (device not ready)
(4)ata3: hardreset failed, retrying in 5 secs


and there it goes into an mess of warnings from NMP: nmpDeviceAttemptFailover...an the box is stuck.

These messages are not written to syslog. I snapshotted them with my phone, so they may not be complete. Sorry.

Thanks for your great help.
Robert


Fri Apr 22, 2011 1:04 am
Profile

Joined: Thu Apr 21, 2011 6:32 am
Posts: 3
Post Re: ESXi 4.1 U1 on P5NTWS crashes under heavy I/O load
Maybe this helps somehow:


esxcfg-scsidevs -l
Code:
~ # esxcfg-scsidevs -l
mpx.vmhba0:C0:T0:L0
   Device Type: CD-ROM
   Size: 0 MB
   Display Name: Local PIONEER CD-ROM (mpx.vmhba0:C0:T0:L0)
   Multipath Plugin: NMP
   Console Device: /vmfs/devices/cdrom/mpx.vmhba0:C0:T0:L0
   Devfs Path: /vmfs/devices/cdrom/mpx.vmhba0:C0:T0:L0
   Vendor: PIONEER   Model: DVD-RW  DVR-112D  Revis: 1.15
   SCSI Level: 5  Is Pseudo: false Status: on
   Is RDM Capable: false Is Removable: true
   Is Local: true
   Other Names:
      vml.0005000000766d686261303a303a30
   VAAI Status: unknown
   
t10.ATA_____WDC_WD2500AAJS2D55RYA0________________________WD2DWCAPZ0384370
   Device Type: Direct-Access
   Size: 238475 MB
   Display Name: Local ATA Disk (t10.ATA_____WDC_WD2500AAJS2D55RYA0________________________WD2DWCAPZ0384370)
   Multipath Plugin: NMP
   Console Device: /vmfs/devices/disks/t10.ATA_____WDC_WD2500AAJS2D55RYA0________________________WD2DWCAPZ0384370
   Devfs Path: /vmfs/devices/disks/t10.ATA_____WDC_WD2500AAJS2D55RYA0________________________WD2DWCAPZ0384370
   Vendor: ATA       Model: WDC WD2500AAJS-5  Revis: 12.0
   SCSI Level: 5  Is Pseudo: false Status: on
   Is RDM Capable: false Is Removable: false
   Is Local: true
   Other Names:
      vml.0100000000202020202057442d574341505a30333834333730574443205744
   VAAI Status: unknown
   
t10.ATA_____WDC_WD5000AAKS2D75TMA0________________________WD2DWCAPW2076339
   Device Type: Direct-Access
   Size: 476940 MB
   Display Name: Local ATA Disk (t10.ATA_____WDC_WD5000AAKS2D75TMA0________________________WD2DWCAPW2076339)
   Multipath Plugin: NMP
   Console Device: /vmfs/devices/disks/t10.ATA_____WDC_WD5000AAKS2D75TMA0________________________WD2DWCAPW2076339
   Devfs Path: /vmfs/devices/disks/t10.ATA_____WDC_WD5000AAKS2D75TMA0________________________WD2DWCAPW2076339
   Vendor: ATA       Model: WDC WD5000AAKS-7  Revis: 12.0
   SCSI Level: 5  Is Pseudo: false Status: on
   Is RDM Capable: false Is Removable: false
   Is Local: true
   Other Names:
      vml.0100000000202020202057442d574341505732303736333339574443205744
   VAAI Status: unknown
~ #


esxcfg-nics -l
Code:
~ # esxcfg-nics -l
Name    PCI           Driver      Link Speed     Duplex MAC Address       MTU    Description
vmnic0  0000:00:11.00 forcedeth   Down 0Mbps     Half   xx:xx:xx:xx:xx:xx 1500   nVidia Corporation nVidia NForce Network Controller
vmnic1  0000:00:12.00 forcedeth   Up   1000Mbps  Full   xx:xx:xx:xx:xx:xx 1500   nVidia Corporation nVidia NForce Network Controller
~ #


Fri Apr 22, 2011 1:52 am
Profile
Display posts from previous:  Sort by  
Reply to topic   [ 4 posts ] 

Who is online

Users browsing this forum: No registered users and 0 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group.
Designed by STSoftware for PTF.