View unanswered posts | View active topics It is currently Tue Dec 12, 2017 5:52 am



Reply to topic  [ 9 posts ] 
 UNFS server Binary on ESXi 4 
Author Message

Joined: Thu Feb 25, 2010 1:08 am
Posts: 12
Reply with quote
Post UNFS server Binary on ESXi 4
In order to have easier tranferts between 2 ESXi, I've searched to have NFS server directly on the ESXi. It would be very nice for cross snapshot and maybe storage VMotion !

With the great help of Rene Margar (http://rene.margar.fr/2010/03/client-nu ... /#comments), I found that I can use directly the CentOs 5.5 binaries (here ftp://ftp.free.fr/mirrors/ftp.centos.or ... 86/CentOS/) for portmap and unfs, in ESXi ! Yes !

Next, I tried to export /vmfs/volumes, on the ESXi1 and add the NFS storage /vmfs/volumes/Disk1 on the ESXi2. It was fine after few tries ! Cool !
But (there's always a but), it seems unfs can't provide the READDIRPLUS procedure, so you can't browse the NFS storage ! :(
http://communities.vmware.com/message/1149377#1149377

Anyway, I have to test this to store VMs, snapshot from ESXi2 to ESXi1 for example and so on. If it's OK, no need for extra NFS storage like FreeNAS VM, it will be directly available in ESXi : a little revolution, I think.

To continue...


Sat Jul 24, 2010 12:48 am
Profile
Site Admin

Joined: Mon Mar 16, 2009 10:13 pm
Posts: 3880
Reply with quote
Post Re: UNFS server Binary on ESXi 4
Sounds very interesting. How have you made out further with this and have you been able to test the performance of this? Have you tested both hosts using the storage concurrently (and creating files / directories, etc)?

_________________
Dave Mishchenko
VMware vExpert 2009-2013
Image
Now available - VMware ESXi: Planning, Implementation, and Security
Also available - vSphere Quick Start Guide


Tue Jul 27, 2010 1:21 pm
Profile

Joined: Tue Mar 02, 2010 7:49 am
Posts: 5
Reply with quote
Post Re: UNFS server Binary on ESXi 4
Hi all,

After seeing this post, I decided i'd have a go at getting the binaries and seeing what using nfs would be like. It turns out that access is pretty good, i've written a how-to here (http://thebsdbox.co.uk/?p=219). However as mentioned there is an issue with portmap going crackers, which i'm trying to investigate and fix as otherwise all of your vms will become seriously degraded in terms of performance.

It's also recommended to mount with the option -o soft from linux hosts.

--Dan


Tue Aug 17, 2010 12:41 am
Profile

Joined: Thu Feb 25, 2010 1:08 am
Posts: 12
Reply with quote
Post Re: UNFS server Binary on ESXi 4
Hi,
Happy to see some people interest for this.

Here is the nfs.sh I use to launch portmap and unfsd :
Code:
#!  /bin/sh
# Script to start/stop nfs service

[ "$1" = "start" ] && {
ps | grep -q unfsd && echo nfs service already started && exit
[ ! -f /bootbank/exports ] && {
  echo you need to customize the file exports in bootbank
  echo nfs service NOT started
  echo "/vmfs/volumes        192.168.1.0/24(rw,no_root_squash)" > /bootbank/expo
  exit
}

portmap
unfsd -e /bootbank/exports -d >/tmp/nfs 2>&1 &
echo nfs service started
exit
}

[ "$1" = "stop" ] && {
PID=`ps | awk '/unfsd/ {print $1}'`; kill -9 $PID
PID=`ps | awk '/portmap/ {print $1}'`; kill -9 $PID
echo nfs service stopped
exit
}

echo "Syntax : $0 start|stop"

It also create generic exports in /bootbank in order to keep it after reboot. I integrate this script and binaries in a generic oem.tgz that I can post here if needed. All you need is to install oem.tgz, change exports in bootbank and call a "nfs start" in /etc/rc.local.

With one week of test on my 5 free ESXi, I found that portmap overkill CPU after each (re)-connection and a simple restart of portmap & unfsd stop this without stopping NFS Storage.
Here are the options I can see :
* restart nfs every XX minutes.
* scan for more than 90% usage of CPU with portmap (with esxtop ?) and restart nfs.
To me, this issue can be easily overcome.

The second issue is that you can't use "ls" or see in VI-client the content of the Datastore. But in fact, I don't need this, I already have access to this on the nfs server...

My first use is OK : I make snapshot with GhettoVCB of one VM from ESXi1 to ESXi2. Then if ESXi1 is dead, I can use directly the VM in ESXi2, with native local disk performance !

The second use isn't achieved : I try to do storage VMotion with free ESXi 4 and move one vmdk from ESXi1 to the NFS of ESXi2 (After that, Quick Migration will be the next step...)
But without availability of SVMotion plugin for VSphere Client 4, the only other way is to use RCLI with a licence...

If someone has an idea...


Tue Aug 17, 2010 3:16 am
Profile

Joined: Tue Mar 02, 2010 7:49 am
Posts: 5
Reply with quote
Post Re: UNFS server Binary on ESXi 4
Turns out that the majority of this is null and void as far as I can gather, in ESXi4.1 they've changed something in regards to the /vmfs/volumes path. Meaning that none of the paths will work correctly and the NFS mount becomes stale almost immediately. Currently nfs mounting is fine for ESXi4.0 but won't work at all in ESXi4.1.

I've recently built openssh-server from openssh.org a number of times (different versions) and tried to get sshd and sftp working together. At the moment sshd will start and will then seg fault when a user connects to the ssh port.

-Dan


Tue Aug 17, 2010 4:58 am
Profile

Joined: Thu Feb 25, 2010 1:08 am
Posts: 12
Reply with quote
Post Re: UNFS server Binary on ESXi 4
Actually, with all I can already do now with my NFS stuff, I think I'm going to stay in 4.0, if it doesn't work in 4.1 !
And support is not a problem, as I have already activate console on all my ESXi...

For 4.1, maybe you need to try newer CentOS files ?


Tue Aug 17, 2010 11:31 pm
Profile

Joined: Thu Feb 25, 2010 1:08 am
Posts: 12
Reply with quote
Post Re: UNFS server Binary on ESXi 4
With this, nfs is started/stopped and also restarted every time portmap overload the CPU :
Code:
#!  /bin/sh
# Script to start/stop nfs service
# Portmap CPU overload allow only one connection at a time

[ "$1" = "start" ] && {
ps | grep -q unfsd && echo nfs service already started && exit
[ ! -f /bootbank/exports ] && {
  echo you need to customize the file exports in bootbank
  echo nfs service NOT started
  echo "/vmfs/volumes        192.168.1.0/24(rw,no_root_squash)" > /bootbank/exports
  exit
}
grep -q "150.0.0.0" /bootbank/exports || {
  echo -e "portmap: ALL\n" > /etc/hosts.deny
  rm /etc/hosts.allow
  for i in `sed "s/.* \(.*\)\/.*/\1/" /bootbank/exports`; do echo "portmap: $i" >> /etc/hosts.allow; done
  echo "" >> /etc/hosts.allow
}

[ -e /tmp/nfs ] && VAL=running
portmap
unfsd -e /bootbank/exports -d >/tmp/nfs 2>&1 &
echo nfs service started

# Restart portmap/nfs when portmap overload one CPU (after each (de)connection)
[ "$VAL" = "running" ] || (
touch /tmp/nfs
# Restart nfs every 6 hours if not used
{ while [ -e /tmp/nfs ]; do
    sleep 21600
    CPU=`ps -T | awk '/unfsd/ {print int($4*100)}'`; sleep 10
    [ "$CPU" = "`ps -T | awk '/unfsd/ {print int($4*100)}'`" ] && { $0 stop norm; $0 start; }
  done
} &
# Restart first portmap to keep disk access then portmap/nfs at deconnection
while [ -e /tmp/nfs ]; do
  VAL=`ps -T | awk '/portmap/ {print int($4*10000)}'`
  [ $VAL -gt 20000 ] && {
   if [ -z $ONE ]; then
      PID=`ps | awk '/portmap/ {print $1}'`; kill -9 $PID; portmap
      ONE="no"
   else
      ONE=""
      $0 stop norm
      $0 start
   fi
  }
  sleep 60
done
) &

exit
}

[ "$1" = "stop" ] && {
[ "$2" = "norm" ] || rm /tmp/nfs
PID=`ps | awk '/unfsd/ {print $1}'`; kill -9 $PID
PID=`ps | awk '/portmap/ {print $1}'`; kill -9 $PID
echo nfs service stopped
exit
}

echo "Syntax : $0 start|stop"


A+


Last edited by fxmx86 on Thu Feb 24, 2011 5:26 am, edited 2 times in total.



Mon Aug 23, 2010 8:12 am
Profile

Joined: Thu Feb 25, 2010 1:08 am
Posts: 12
Reply with quote
Post Re: UNFS server Binary on ESXi 4
Back after few months of use of this user space nfs server !

It works quite well on ESXi 4.0 but there's the issue of READDIRPLUS procedure not supported. So it's impossible to use it for 4.1 VMs back up (works fine with 4.0 ESXi). And you can't use vmdk on the NFS for live VMs (for exemple, with soft RAID 1, to secure VM or to allow quick VM transfer).

One chance : compile and add the NFS module to ESXi as it includes READDIRPLUS. As others network/storage modules found in vm-help forums, it must be possible to compile it in CentOS. I found this exemple : http://www.bytedynamix.com/?p=77

Did somebody already try this and have some advices ?
Thanks in advance.

Nota : Old ESX already include an NFS server that you could activate, but it's not present in ESXi...


Sat Jan 29, 2011 9:46 am
Profile

Joined: Thu Feb 25, 2010 1:08 am
Posts: 12
Reply with quote
Post Re: UNFS server Binary on ESXi 4
I've updated my script above.

A+


Mon Feb 21, 2011 2:46 am
Profile
Display posts from previous:  Sort by  
Reply to topic   [ 9 posts ] 

Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group.
Designed by STSoftware for PTF.