View unanswered posts | View active topics It is currently Mon Oct 15, 2018 11:20 am



Reply to topic  [ 7 posts ] 
 Major issue: hard drives not detected anymore 
Author Message

Joined: Sun Oct 23, 2011 8:37 am
Posts: 13
Reply with quote
Post Major issue: hard drives not detected anymore
The hard drives are not detected anymore on my ESXi 4.1 host, hence its datastores went missing after booting on a custom oem.tgz.

This happened subsequently to installing an additional driver for the RTL8169 NIC.
I placed the relevant oem.tgz into /bootbank (carefully backing up the previous oem.tgz), the booting process went through although it took longer than it used to before using the new driver. So the network card was indeed assigned a vmnic, however... both /bootbank and /altbookbank went awol (they're just completely missing) and my two datastores are missing.

Note that all networking features are working fine, I can access the host via the management network, connect with vSphere, SSH it... etc.
Note that I have to reenable SSH after each reboot, which I never had to do. This is yet again another change that I noticed since booting on this new oem.tgz file. So the inetd.conf doesn't seem to be permanently taking the changes I do to it when I enable SSH.

What steps would you suggest me to take now in order to get back this ESXi host to work again ?
Any troubleshooting guidelines from here would be greatly appreciated as I am very worried about the current interruption.
Can I resinstall without loosing my VMs ? Maybe this could be a chance to upgrade to ESXi 5, would that be possible ?

P.S.
In the worst case scenario where I couldn't get ESXi back to normal quickly, is there a way to recoup the VM files in order to load them up, at least temporarily, within a VMware Workstation (we have only one ESXi server) ?
Could this recouping be done via a Linux live CD my mounting the relevant partitions ?


Fri Jan 27, 2012 11:07 pm
Profile

Joined: Mon Jul 25, 2011 2:22 pm
Posts: 76
Reply with quote
Post Re: Major issue: hard drives not detected anymore
Where do you have ESXi installed? On the hard disks that are missing now, or on a USB drive?
I assume it's on the hard disks. If yes then try to install and boot an additional copy of ESXi on a USB thumb drive (just be sure to choose the option to NOT overwrite existing VMFS datastores during installation!). From there you should be able to access your original installation and restore the oem.tgz with the backup you made.

Good luck!

- Andreas

_________________
Image VMware Front Experience Blog: Home of ESXi-Customizer and ESXi-Customizer-PS, the ESXi5 Community Packaging Tools and the V-Front Online Depot of ESXi software.


Sat Jan 28, 2012 3:57 am
Profile WWW

Joined: Sun Oct 23, 2011 8:37 am
Posts: 13
Reply with quote
Post Re: Major issue: hard drives not detected anymore
Hi Andreas.

Thanks for your suggestion.

Yes you are right, the current ESXi boots up from one of the "missing" hard drives... but, is your suggestion not similar to booting up from a live CD ?
Because, as I said, /bootbank and /altbootbank are missing when I do that... so how can I restore the oem.tgz file again ?

Are you telling me that booting another ESXi from - let's say the USB drive (the CD drive would work as well I suppose) - would let me restore a working oem.tgz ?

I'd be grateful if you could elaborate before I go any further.

Anyway in the meantime I'll be attempting to make a copy of my VMs (hoping they're still there) by following the indications from this post.


Sat Jan 28, 2012 5:55 am
Profile

Joined: Sun Oct 23, 2011 8:37 am
Posts: 13
Reply with quote
Post Re: Major issue: hard drives not detected anymore
Ok I just could run the Ubuntu 11.4 64bit live CD on the server... and the files are still in there.

So this probably just means that my storage controller drivers have gone wrong but the data is preserved.

Tomorrow I will backup the one crucial VM onto a USB drive, and at that stage I will be in place to safely reinstall ESXi without worrying too much.

I'll keep you posted, but anyway I'd be curious to know what you meant with booting ESXi from another drive in order to restore the oem.tgz file that I have backed up.
I think I would still favor that option as it's much less things to do than going through the whole backup, resinstall and reconfiguration again.

Many thanks.


Sat Jan 28, 2012 6:29 am
Profile

Joined: Mon Jul 25, 2011 2:22 pm
Posts: 76
Reply with quote
Post Re: Major issue: hard drives not detected anymore
Hi,

when you look at your ESXi system from a Live boot Linux CD (you are right, this is easier than installing another copy of ESXi), you won't see the directories /bootbank and /altbootbank, because they are only created at ESXi boot time within a RAM disk, and then two of the ESXi installation partitions are mounted on these directories. You only see these partitions with their contents when looking from the live CD.
Just replace the copies of OEM.tgz that you find there with your backup copy of the file or the original copy from the ESXi 4.1 installation CD.
This should do the trick and restore your ESXi installation to a working state.

- Andreas

_________________
Image VMware Front Experience Blog: Home of ESXi-Customizer and ESXi-Customizer-PS, the ESXi5 Community Packaging Tools and the V-Front Online Depot of ESXi software.


Sat Jan 28, 2012 9:00 am
Profile WWW
Site Admin

Joined: Mon Mar 16, 2009 10:13 pm
Posts: 3880
Reply with quote
Post Re: Major issue: hard drives not detected anymore
The process Andreas mentions will get you back to where you were at before the issue. If you hadn't replaced oem.tgz file before it's just an empty file that you can recreate.

_________________
Dave Mishchenko
VMware vExpert 2009-2013
Image
Now available - VMware ESXi: Planning, Implementation, and Security
Also available - vSphere Quick Start Guide


Sat Jan 28, 2012 4:37 pm
Profile

Joined: Sun Oct 23, 2011 8:37 am
Posts: 13
Reply with quote
Post Re: Major issue: hard drives not detected anymore
Ok I sorted out the issue, still simply using the Ubuntu live CD.
I attached the result of my fdisk -l run from Ubuntu.
/dev/sdb5 was in fact was is mounted in VMware as /bootbank and the faulty oem.tgz was in there alongside the healthy backup that I could smoothly restore.
My ESXi 4.1 is now working fine again.

Thanks for your support.


Attachments:
fdisk.txt [1.02 KiB]
Downloaded 257 times
Sun Jan 29, 2012 3:49 am
Profile
Display posts from previous:  Sort by  
Reply to topic   [ 7 posts ] 

Who is online

Users browsing this forum: No registered users and 5 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group.
Designed by STSoftware for PTF.