2011年04月24日

Hi, everybody, long time no see in blog. The last  blog was written more than one year ago, and I found myself reluctant to write a blog, not to mention in English. Recently, I finished one piece of paper in English and found it hard to use this language. As I am free now, I’d like to summarize the past year, which plays the most important role in my whole life.

The reason why the past year is so important for me is that it determines my future. After serveral business trips which are the only things I’ve done during my master, I succeeded in graduating from NUDT. Unfortunately, I failed in the ph.D record examination and had to worry about the problem of the job. Those days were really tough for me and my family and I might lose many brain cells in that time. And finally, with everybody’s help, I was assigned to a nice place where is near to my hometown and very comfortable for work. Someone said it was totally lucky for me to get this job and I would rather say it is all your contribution. I am a grateful man and I would own my indebeted to you for life.

Please do not laugh at my complacency. This place is very nice for work and I am totally satisfied. First of all, this city is close to my hometown. Once I finish my study in driving school, I could drive home in 2.5 hours. The interurban bus is convenient and I also can take a train in the interest of money. The high-speed railway is due in the next year and that’s the fastest way to go home, and I also hope it is the cheapest way. Secondly, I have to admit that living among this quiet hill is very comfortable. The carteen and shop are close and the office is just 10 min on foot. What’s more, this place is not far from the center of the city. At most of 30 min by bus, I can get everything I want to buy. People here are very nice to me and I have full passion in my work. The most exciting thing for me is that I can play soccer every week. Two times in smaller field are regular and one in normal now and again. Like today, I got the first blood in this match to some company and our team beat them badly.

The only thing I need to worry is common to everybody, the house. Well, this involves the marriage problem which bothers me a lot recently. I have been arranged several blind dates recenetly and frankly speaking, I am a little tired of this form to find one to love. Little by little, I develop some basic rule for the one I approve. She should have a steady work in this city or she is going to work in this city. I do not want to see a live seperation after we are married. A good looking is not important but at least I am supposed to have the courage to take her out. Last but not least, filial piety! I am under the mind that those rules are not censorious, are they? I find it hard to love someone when we are older, and I wonder when I can find the one. My family and friends think that I am not old enough to worry this problem, but, as belonging to Libra, I am not the one to bear the loneliness. Since the lover is hard to get, the friend can not miss in my life. Yet, you know how hard to find a friend as you get older, especially at work.

The past year gave me the greatest oppotunity to fulfil my furture and how I can disappoint you, the ones I love. The comming year will determines the direction of my work and future, and I don’t wanna mess it.

Target: work, lover and car.

Please do not forget me, my friends. Give me a call when you miss me~

2010年03月16日

Here’s what I’ve done recently. There’re many articles telling us how to install hadoop, however, few mention NFS and NIS which make this installation easier. Also, I make more effort to enable more users to use hadoop in the same cluster at one time.

System configuration

OS: RedHat AS4
JDK: 1.6.0
Hadoop: 0.20.1
Hardware:
 sun1: 192.168.1.1  
 sun2: 192.168.1.2
 sun3: 192.168.1.3
Prerequisites:
 NFS share folder: 192.168.1.1:/cshare
 NIS server: 192.168.1.1

step 1. modify /etc/hosts on each machine

[root@sun1 ~]$vim /etc/hosts              (do the same thing to sun2-3)

192.168.1.1 sun1
192.168.1.2 sun2
192.168.1.3 sun3

step 2. create user1 with NIS

[root@sun1 ~]$mkdir /cshare/home
[root@sun1 ~]$chmod 1777 /cshare/home
[root@sun1 ~]$mkdir /cshare/user1
[root@sun1 ~]$useradd -p 123456 user1 -d /cshare/user1
[root@sun1 ~]$cd /var/yp
[root@sun1 ~]$make
[root@sun1 ~]$chown -R user1:user1 /cshare/home/user1

step 3. create SSH trusted connection

[user1@sun1 ~]$ssh-keygen -t rsa  (press ‘Enter’ all along)
[user1@sun1 ~]$cd .ssh
[user1@sun1 .ssh]$cp id_rsa.pub authorized_keys
[user1@sun1 .ssh]$ssh sun2   (type ‘yes’)
[user1@sun1 .ssh]$ssh sun3   (type ‘yes’)

step 4. Install Hadoop for user1

step 4.1 create folder for hadoop log and tmp

[root@sun1 ~]$mkdir -p /hadoop/user1
[root@sun1 ~]$chmod 1777 /hadoop
[root@sun1 ~]$chown user1:user1 /hadoop/user1

step 4.2 config Hadoop

[user1@sun1 ~]$tar zxvf hadoop-0.20.1.tar.gz
[user1@sun1 ~]$cd hadoop-0.20.1/conf
[user1@sun1 conf]$vim hadoop-env.sh

# The java implementation to use.  Required.

export JAVA_HOME=/usr/java/jdk_1.6.0

# Where log files are stored.  $HADOOP_HOME/logs by default.

export HADOOP_LOG_DIR=/hadoop/user1

[user1@sun1 conf]$vim core-site.xml

<property>
  <name>hadoop.tmp.dir</name>
  <value>/hadoop/user1</value>
  <description>A base for other temporary directories.</description>
</property>
 
<property>
  <name>fs.default.name</name>
  <value>hdfs://sun1:54310</value>
  <description>The name of the default file system.  A URI whose
  scheme and authority determine the FileSystem implementation.  The
  uri’s scheme determines the config property (fs.SCHEME.impl) naming
  the FileSystem implementation class.  The uri’s authority is used to
  determine the host, port, etc. for a filesystem.</description>
</property>

[user1@sun1 conf]$vim mapred-site.xml

<property>
  <name>mapred.job.tracker</name>
  <value>sun1:54311</value>
  <description>The host and port that the MapReduce job tracker runs
  at.  If “local”, then jobs are run in-process as a single map
  and reduce task.
  </description>
</property>

[user1@sun1 conf]$vim hdfs-site.xml

<property>
  <name>dfs.replication</name>
  <value>3</value>
  <description>Default block replication.
  The actual number of replications can be specified when the file is created.
  The default is used if replication is not specified in create time.
  </description>
</property>

step 4.3 start hadoop

[user1@sun1 conf]$cd ../bin
[user1@sun1 bin]$./hadoop namenode -format (before this, make sure there’s no data in /hadoop/user1 for sun1-3)
[user1@sun1 bin]$./start-all.sh

step 4.4 run grep

[user1@sun1 bin]$./hadoop dfsadmin -report
[user1@sun1 bin]$./hadoop dfs -mkdir grepin
[user1@sun1 bin]$vim /tmp/test.txt
test
test
test
[user1@sun1 bin]$./hadoop dfs -put /tmp/test.txt grepin
[user1@sun1 bin]$./hadoop jar ../hadoop-0.20.1-examples.jar grep grepin grepout test
[user1@sun1 bin]$./hadoop dfs -cat grepout/part-00000
3 test

step 5. install hadoop for user2

step 5.1 create user2 as step 2, 3, 4.1 does

step 5.2 config hadoop for user2  
(notice port config!!! user2 must use different port numbers for each port that hadoop uses.
What’s more, some ports are for master which should use sun1′ ip while others are for slaves which should use 0.0.0.0
As this cluster is not connected to http, I need not config dfs.datanode.https.address and dfs.https.address in hdfs-site.xml)

[root@sun1 ~]$cp /cshare/home/user1/hadoop-0.20.1 /cshare/home/user2
[root@sun1 ~]$chown -R user1:user1 /cshare/home/user1
[user2@sun1 ~]$cd hadoop-0.20.1/conf
[user2@sun1 conf]vim hadoop-env.sh

# The java implementation to use.  Required.

export JAVA_HOME=/usr/java/jdk_1.6.0

# Where log files are stored.  $HADOOP_HOME/logs by default.

export HADOOP_LOG_DIR=/hadoop/user2

[user2@sun1 conf]$vim core-site.xml

<property>
  <name>hadoop.tmp.dir</name>
  <value>/hadoop/user2</value>
  <description>A base for other temporary directories.</description>
</property>
 
<property>
  <name>fs.default.name</name>
  <value>hdfs://sun1:54312</value>
  <description>The name of the default file system.  A URI whose
  scheme and authority determine the FileSystem implementation.  The
  uri’s scheme determines the config property (fs.SCHEME.impl) naming
  the FileSystem implementation class.  The uri’s authority is used to
  determine the host, port, etc. for a filesystem.</description>
</property>

[user2@sun1 conf]$vim mapred-site.xml

<property>
  <name>mapred.job.tracker</name>
  <value>sun1:54313</value>
  <description>The host and port that the MapReduce job tracker runs
  at.  If “local”, then jobs are run in-process as a single map
  and reduce task.
  </description>
</property>

<property>
  <name>mapred.job.tracker.http.address</name>
  <value>sun1:50031</value>
  <description>
    The job tracker http server address and port the server will listen on.
    If the port is 0 then the server will start on a free port.
  </description>
</property>

<property>
  <name>mapred.task.tracker.http.address</name>
  <value>0.0.0.0:50061</value>
  <description>
    The task tracker http server address and port.
    If the port is 0 then the server will start on a free port.
  </description>
</property>

[user2@sun1 conf]$vim hdfs-site.xml

<property>
  <name>dfs.replication</name>
  <value>3</value>
  <description>Default block replication.
  The actual number of replications can be specified when the file is created.
  The default is used if replication is not specified in create time.
  </description>
</property>

<property>
  <name>dfs.secondary.http.address</name>
  <value>sun1:50090</value>
  <description>
    The secondary namenode http server address and port.
    If the port is 0 then the server will start on a free port.
  </description>
</property>

<property>
  <name>dfs.datanode.address</name>
  <value>0.0.0.0:50010</value>
  <description>
    The address where the datanode server will listen to.
    If the port is 0 then the server will start on a free port.
  </description>
</property>

<property>
  <name>dfs.datanode.http.address</name>
  <value>0.0.0.0:50075</value>
  <description>
    The datanode http server address and port.
    If the port is 0 then the server will start on a free port.
  </description>
</property>

<property>
  <name>dfs.datanode.ipc.address</name>
  <value>0.0.0.0:50020</value>
  <description>
    The datanode ipc server address and port.
    If the port is 0 then the server will start on a free port.
  </description>
</property>

<property>
  <name>dfs.http.address</name>
  <value>sun1:50070</value>
  <description>
    The address and the base port where the dfs namenode web ui will listen on.
    If the port is 0 then the server will start on a free port.
  </description>
</property>

step 5.3 start hadoop and run as step 4.3, 4.4 does

Well done!

Notice:
1. Permission: make sure the user can access each folder hadoop needs
2. SSH trusted connection must be created
3. For user2, ports config is vital, or there must be conflict in ports and failures in address assignment
4. before namenode format, make sure there’s no data in /hadoop/user1 for sun1-3, or there may be some conflict
5. After some change to user, for example, passwd user1, you should enter /var/yp and “$make” to take effect

Reference:
1.Running Hadoop On Ubuntu Linux (Single-Node Cluster)  http://www.michael-noll.com/wiki/Running_Hadoop_On_Ubuntu_Linux_(Single-Node_Cluster)
2.Running Hadoop On Ubuntu Linux (Multi-Node Cluster) http://www.michael-noll.com/wiki/Running_Hadoop_On_Ubuntu_Linux_(Multi-Node_Cluster)
3.Hadoop的安装与使用(转) http://jiajiam.spaces.live.com/blog/cns!E9F2928B37455D08!340.entry
4.Hadoop的安装配置 http://www.javaeye.com/topic/365168

2009年10月17日

Yesterday was my 24th birthday, and it really brings me luck and happiness.

When I searched my last delivered blog by accident, I got a definitely happy accident. The blog named "My hard way to install Xen" which is delivered in Sept. 8th. was reprint in here: http://www.diybl.com/course/6_system/linux/Linuxjs/20090925/177098.html by diybl.com. I contacted them and expressed my honor. This is my first time that my blog is reprint as a technical paper which means it receives someone’s accept. i worte the following in my first blog: "As a new comer, adding that I have little knowledge of IT, I can’t accomplish a paper in IT. BUT I believe I can do it soon. My only TASK is to study and study, to have some technical articles come out." Finally, I made it!

P.S. I will get two suns on my QQ in two hours.

Haha, happy!

I should make more efforts!

2009年09月16日

To summarize how to build a virtual cluster and to memorize what I have done in the last half month.
This blog will show you how to build a virtual cluster with Xen step by step.
I almost did the same step as "ClusterMonkey – Building A Virtual Cluster with Xen" did, but you know, there’re much modification. The most important is that I have no internet access. So, I am here, to show you a latest version to build a virtual cluster offline.

My role: a brand new linux user.

structure:
A Xen installation, the creation of 3 virtual machines (one to act as the master and two slaves), shared storage through NFS.

Step 1. Install linux OS
I tried Fedora 8, Fedora 11, Ubuntu 9.04 and CentOS 5.3, and fianlly I choosed CentOS 5.3(final). For some reason, I deleted all the disks and gave CentOS the whole 320GB.

Here’s network configuration
ip  192.168.1.1
netmask  255.255.255.0
gateway  192.168.1.254

Step 2. Install Xen
You may wonder why I tried so many OS. Because, at the first moment, I choosed to install Xen from a source tarball. However, I could not make it due to the version problem. Sometimes, OS lacks of some necessary packages for Xen(Remember: I have no internet access), and sometimes, Xen could not recognize my hard disk, although I had a successful installation. At last, I got useful advices from one senior that CentOS has included Xen(later, I found Fedora also did the same thing).
So, the installation is much easier. Get all xen, xen-kernel and other dependent packages(such as Virtualization-en-US, libvirt, virt-manager, python-virtinst and etc.) from DVD. Like:

[root@daisy ~]# mkdir xen-install
[root@daisy ~]# cp /media/disk/CentOS/xen-3.0.3-80.e15.i386.rpm ./xen-install

Then install Xen.

[root@daisy ~]# cd xen-install
[root@daisy xen-install]# rpm -ivh *.rpm

If you are lack of some packages, copy them from DVD. Finally, you will succeed in installing xen, and then reboot. Then you will check whether the installation is ok by xm command.

modify the enviroment

[root@daisy ~]# cat /etc/bashrc
PATH=$PATH:/usr/sbin
PATH=$PATH:/sbin

[root@daisy ~]# xm list
Name                                      ID Mem(MiB) VCPUs State   Time(s)
Domain-0                                   0     3280     4 r—–    926.5

ok, you have made it.

Step 3. Install guest OS in Xen (Use another user: frank)

Step 3.1. Make a ftp server for guest OS installation source.

[frank@daisy ~]# rpm -ivh vsftpd-2.0.5-12.el5.rpm
[frank@daisy ~]# vsftpd

upload the CentOS 5.3 installation files to ftp
[frank@daisy ~]# cp -rf /media/disk/* /var/ftp/pub/centos5.3

Step 3.2. Create a 10GB img for master installation

[frank@daisy ~]# mkdir vcluster
[frank@daisy ~]# dd if=/dev/zero of=/vcluster/master/master.img bs=4M count=2560

Step 3.3. install master on Xen

[frank@daisy ~]# virt-install
What is the name of your virtual machine? master
How much RAM should be allocated (in megabyte)? 256
What would you like to use as the disk (path)? ./vcluster/master/master.img
Would you like to enable graphics support? (yes or no) no
What is the install location? ftp://192.168.1.1/pub/centos5.3

And then you will find the familiar installation interface. Maybe you are not comfortable without graphics support, but you will soon find it is the same thing. Make sure turn off the firewall and SElinux.

Step 3.4. login master and configure master’s network

[frank@daisy ~]# xm create master
[frank@daisy ~]# xm console master

the following instruction is in master node.
modify /etc/sysconfig/network-scrpits/ifcfg-eth0 to make
ip   192.168.1.2
netmask  255.255.255.0
gateway  192.168.1.254

Use "Virtual system manager" to create a private network eth1, and modify master’s network
ip   192.168.0.2
netmask  255.255.255.0
gateway  192.168.0.254

Step 3.5. install slaves

you can do the similar thing as you install master.

[frank@daisy ~]# dd if=/dev/zero of=/vcluster/slave1/slave1.img bs=4M count=2560
[frank@daisy ~]# virt-install
[frank@daisy ~]# xm create slave1
[frank@daisy ~]# xm console slave1

Use "Virtual system manager" to create a private network eth1, and modify salve1’s network, with ip 192.168.0.3
Do the same thing to slave2 with ip 192.168.0.4

Ok, you will see 3 virtual machine running in xen

[root@daisy ~]# xm list
Name                                      ID Mem(MiB) VCPUs State   Time(s)
Domain-0                                   0     3280     4 r—–    926.5
master                                     1      255     1 -b—-    290.4
slave1                                     2      255     1 -b—-     36.4
slave2                                     3      255     1 -b—-     38.5

Step 4. NFS Configuration

Step 4.1. Master NFS Server Configuration

You should firstly get nfs-utils. So

[root@master ~]# ftp 192.168.1.1
name: anonymous
ftp> cd pub/centos5.3/CentOS
ftp> mget
(remote files) nfs-utils-1,0.9-40.e15.i386.rpm
ftp> quit

Then, you can get nfs-utils. In the following part, I won’t mention how I get rpm package because I always did in this way.

[root@master ~]# rpm -ivh nfs-utils-1,0.9-40.e15.i386.rpm
[root@master ~]# mkdir /cshare
[root@master ~]# chmod 777 /cshare
[root@master ~]# cat /etc/exports
/cshare         192.168.0.0/255.255.255.0(rw,sync)
[root@master ~]# cat /etc/hosts.deny
portmap:ALL
lockd:ALL
mountd:ALL
rquotad:ALL
statd:ALL
[root@master ~]# cat /etc/hosts.allow
portmap: 192.168.0.0/255.255.255.0
lockd: 192.168.0.0/255.255.255.0
mountd: 192.168.0.0/255.255.255.0
rquotad: 192.168.0.0/255.255.255.0
statd: 192.168.0.0/255.255.255.0
[root@master ~]# cat /etc/fstab
none                    /proc/fs/nfsd           nfsd    defaults 0 0
[root@master ~]# cat /etc/hosts
#127.0.0.1             localhost
192.168.0.2            master
192.168.0.3            slave1
192.168.0.4            slave2
[root@master ~]# chkconfig –level 345 nfs on

Step 4.2. Slave NFS Configuration

[root@slave1 ~]# rpm -ivh portmap-4.0-65.2.2.1
[root@slave1 ~]# cat /etc/fstab
192.168.0.2:/cshare    /cshare                 nfs     rw,hard,intr 0 0
[root@slave1 ~]# mount -t nfs 192.168.0.2:/cshare /cshare
[root@slave1 ~]# cat /etc/hosts
#127.0.0.1             localhost
192.168.0.2            master
192.168.0.3            slave1
192.168.0.4            slave2
[root@slave1 ~]# mkdir /cshare

you should do the same thing for slave2.

Step 4.3. Check NFS
[root@slave1 ~]# df -h
[root@slave1 ~]# ping master

Here, you have accomplished the basic building of the virtual cluster. The following is to install cluster softwares. I downloaded C3, MPICH2, Torque and Maui in other places.

Step 5. C3 Installation

For master

[root@master ~]# rpm -ivh rsync-2.6.8-3.1.i386.rpm rsh-0.17-38.e15.i386.rpm rsh-server-0.17-38.e15.i386.rpm xinetd-2.3.14-10.e15.i386.rpm
[root@master ~]# vi /etc/hosts.equiv
master
slave1
slave2
[root@master ~]# cp rsync-2.6.8-3.1.i386.rpm rsh-0.17-38.e15.i386.rpm rsh-server-0.17-38.e15.i386.rpm xinetd-2.3.14-10.e15.i386.rpm /cshare

For slaves

[root@slave1 cshare]# cp rsync-2.6.8-3.1.i386.rpm rsh-0.17-38.e15.i386.rpm rsh-server-0.17-38.e15.i386.rpm xinetd-2.3.14-10.e15.i386.rpm ~
[root@slave1 cshare]# cd ~
[root@slave1 ~]# rpm -ivh rsync-2.6.8-3.1.i386.rpm rsh-0.17-38.e15.i386.rpm rsh-server-0.17-38.e15.i386.rpm xinetd-2.3.14-10.e15.i386.rpm

In the file /etc/xinetd.d/rsh change the line disable = yes to disable = no
In the file /etc/securetty add a line containing rsh
In the file /etc/pam.d/rsh change the line auth required pam_rhosts_auth.so by adding at the end hosts_equiv_rootok

[root@slave1 ~]# vi /etc/hosts.equiv
master
slave1
slave2

Then, (remember, only in the slaves) we start xinetd:
 
[root@slave1 ~]# service xinetd start

Do the same thing to slave2

For master

[root@master ~]# tar -zxf c3-4.0.1.tar.gz
[root@master ~]# cd c3-4.0.1
[root@master c3-4.0.1]# ./Install-c3
[root@master c3-4.0.1]# ln -s /opt/c3-4/c[^0-9]* /usr/local/bin/
[root@master c3-4.0.1]# vi /etc/c3.conf

cluster vcluster {
  master:192.168.0.2
  slave[1-2]
}

[root@master ~]# cat /etc/profile
 
export C3_RSH=rsh

[root@master ~]# cexec uname -a

************************* master *************************
——— slave1———
Linux slave1 2.6.18.128-e15xen #1 SMP Wed Jan 21 11:55:02 EST 2009 i686 i686 i386 GNU/Linux
——— slave2———
Linux slave2 2.6.18.128-e15xen #1 SMP Wed Jan 21 11:55:02 EST 2009 i686 i686 i386 GNU/Linux

Step 6. Install MPI

Step 6.1 create SSH trusted connection

For master

[root@master ~]# ssh-keygen -t rsa (press ‘Enter’ all along)
[root@master ~]# cd .ssh
[root@master .ssh]# cp id_rsa.pub authorized_keys
[root@master .ssh]# cd ..
[root@master ~]# ssh master      (type ‘yes’)

For slaves

[root@slave1 ~]# ssh-keygen -t rsa
[root@master ~]# scp 192.168.0.2:/root/.ssh/* /root/.ssh
[root@master ~]# scp 192.168.0.2:/etc/hosts /etc/hosts
[root@master ~]# ssh slave1  (type ‘yes’)

Do the same thing to slave2
Check the 3 machines have trusted connection built

[root@master ~]# ssh slave1  (type ‘yes’, and no password)
[root@master ~]# ssh slave2
[root@slave1 ~]# ssh master
[root@slave1 ~]# ssh slave2
[root@slave2 ~]# ssh master
[root@slave2 ~]# ssh slave1

Step 6.2. Install MPICH2

[root@master ~]# tar -zxvf mpich2-1.1.tar.gz
[root@master ~]# mkdir /usr/mpich2
[root@master ~]# cd mpich2-1.1
[root@master mpich2-1.1]# ./configure –prefix=/usr/mpich2
[root@master mpich2-1.1]# make
[root@master mpich2-1.1]# make install
[root@master mpich2-1.1]# cat /etc/bashrc

PATH=$PATH:/usr/mpich2/bin

[root@master mpich2-1.1]# logout

[root@master ~]# touch /etc/mpd.conf
[root@master ~]# chmod 600 /etc/mpd.conf
[root@master ~]# vi /etc/mpd.conf

MPD_SECRETWORD=mr45-j9z

[root@master ~]# vi mpd.hosts

master
slave1
slave2

Step 6.3. check installation

[root@master ~]# mpd &
[root@master ~]# mpdtrace
[root@master ~]# mpdallexit

Do the same thing to slave1 and slave2.

Step 6.4. Run MPI

[root@master ~]# vi hello.c

#include "mpi.h"
#include <stdio.h>
#include <math.h>
void main(argc,argv)
int argc;
char *argv[];
{
 int myid, numprocs;
 int namelen;
 char processor_name[MPI_MAX_PROCESSOR_NAME];
 
 MPI_Init(&argc,&argv);
 MPI_Comm_rank(MPI_COMM_WORLD,&myid);
 MPI_Comm_size(MPI_COMM_WORLD,&numprocs);
 MPI_Get_processor_name(processor_name,&namelen);
 fprintf(stderr,"Hello World! Process %d of %d on %s\n", myid, numprocs, processor_name);
 MPI_Finalize();
}

[root@master ~]# mpicc -o hello hello.c
[root@master ~]# cpush ./hello
[root@master ~]# mpdboot -n 3 -f mpd.hosts
[root@master ~]# mpirun -n 3 ./hello

Step 7. Install Torque

Step 7.1 Install Torque on master

[root@master ~]# tar zxvf torque-2.3.7.tar.gz
[root@master ~]# cd torque-2.3.7
[root@master torque-2.3.7]# ./configure -prefix=/opt/torque-2.3.7
[root@master torque-2.3.7]# make
[root@master torque-2.3.7]# make install
[root@master torque-2.3.7]# cat /etc/bashrc

PATH=$PATH:/opt/torque-2.3.7/bin
PATH=$PATH:/opt/torque-2.3.7/sbin

[root@master ~]# logout

[root@master ~]# cd torque-2.3.7
[root@master torque-2.3.7]# ./torque_setup root
[root@master torque-2.3.7]# vi /var/spool/torque/server_priv/nodes

master
slave1
slave2

[root@master torque-2.3.7]# chmod 1777 /var/spool/torque/spool
[root@master torque-2.3.7]# chmod 1777 /var/spool/torque/undelivered

[root@master torque-2.3.7]# pbs_server -t create
[root@master torque-2.3.7]# qmgr -c "set server scheduling=true"
[root@master torque-2.3.7]# qmgr -c "create queue dque queue_type=execution"
[root@master torque-2.3.7]# qmgr -c "set queue dque started=true"
[root@master torque-2.3.7]# qmgr -c "set queue dque enabled=true"
[root@master torque-2.3.7]# qmgr -c "set queue dque resources_default.nodes=1"
[root@master torque-2.3.7]# qmgr -c "set queue dque resources_default.walltime=3600"
[root@master torque-2.3.7]# qmgr -c "set server default_queue=dque"

Check status

shutdown server
[root@master torque-2.3.7]# qterm -t quick

start server
[root@master torque-2.3.7]# pbs_server

verify all queues are properly configured
[root@master torque-2.3.7]# qstat -q

view additional server configuration
[root@master torque-2.3.7]# qmgr -c ‘p s’

verify all nodes are correctly reporting
[root@master torque-2.3.7]# pbsnodes -a
[root@master torque-2.3.7]# pbs_mom

Step 7.2. Install Torque on slaves

[root@master torque-2.3.7]# make packages
[root@master torque-2.3.7]# cp torque-package-clients-linux-i686.sh, torque-package-devel-linux-i686.sh, torque-package-doc-linux-i686.sh, torque-package-mom-linux-i686.sh, torque-package-server-linux-i686.sh /cshare

[root@slave1 ~]# cd /cshare
[root@slave1 cshare]# cp torque-package-clients-linux-i686.sh, torque-package-devel-linux-i686.sh, torque-package-doc-linux-i686.sh, torque-package-mom-linux-i686.sh, torque-package-server-linux-i686.sh ~
[root@slave1 cshare]# cd ~
[root@slave1 ~]# ./torque-package-clients-linux-i686.sh

you should run all the sh files to install

[root@slave1 ~]# vi /var/spool/torque/server_name

master

[root@slave1 ~]# vi /var/spool/torque/mom_priv/config

$pbsserver master
$logevent 255
$usecp master:/cshare /cshare

[root@slave1 ~]# cat /etc/bashrc

PATH=$PATH:/opt/torque-2.3.7/bin
PATH=$PATH:/opt/torque-2.3.7/sbin

[root@slave1 ~]# logout
[root@slave1 ~]# pbs_mom

Do the same thing to slave2

Step 8. Install Maui on master (not in slaves)

[root@master ~]# tar zxvf maui-3.2.6p21.tar.gz
[root@master ~]# cd maui-3.2.6p21
[root@master ~]# ./configure -prefix=/opt/maui-3.2.6p21 -with-pbs=/opt/torque-2.3.7
[root@master ~]# make
[root@master ~]# make install

[root@master ~]# vi /usr/local/maui/maui.cfg

SERVERHOST master
# primary admin must be first in list
ADMIN1 root
# Resource Manager Definition
RMCFG[master] TYPE=PBS@RMNMHOST@
RMTYPE[0] PBS

[root@master ~]# cat /etc/bashrc

PATH=$PATH:/opt/maui-3.2.6p21/bin
PATH=$PATH:/opt/maui-3.2.6p21/sbin

[root@master ~]# logout
[root@master ~]# maui

Notice: do not start pbs_sched on slaves

[root@master ~]# logout

[frank@master ~]# vi hello.pbs

#!/bin/sh
#PBS -N hello
#PBS -o hello.log
#PBS -e hello.err
#PBS -q dque
#PBS -l nodes=3
cd ~/tmp
echo Time is `date`
echo Directory is $PWD
echo This job runs on the following nodes:
cat $PBS NODEFILE
NPROCS=`wc -l<$PBS NODEFILE`
echo This job has allocated $NPROCS nodes
mpiexec -machinefile $PBS NODEFILE -np $NPROCS ./hello

[frank@master ~]# qsub hello.pbs

Then you can check the status

[frank@master ~]# qstat -q
[frank@master ~]# qstat -a

Ok, well done!!!

The above is my steps to build a virtual cluster with Xen offline. I complete this blog in rush, so please tell me if you find any problem.

Reference

ClusterMonkey – Building A Virtual Cluster with Xen
Centos 5 xen 最简单装配置: http://www.diybl.com/course/6_system/linux/Linuxjs/200899/141086.html
Linux下MPICH2集群系统安装手册(采用建立信任ssh)
资源管理软件TORQUE与作业调度软件Maui的安装、设置及使用

2009年09月08日

After I decided to research virtualization technology, my first task is to build a virtual cluster. Since I chould not get the root right from the HPC center, I had to make a try on my pc. First of all, I should install a virtual machine, and this is the start of my nightmare.

I google one paper named "Building a Virtual Cluster with Xen" which described the every detail, so I decided to implement it. I downloaded Ubuntu 9.04 CD img and Wubi.exe to install an Ubuntu in my pc, and I got the latest version of Xen, which is 3.4.1. I have to declare that, I have no internet where I do my work, so, many of you can imagine how bitter I suffered when I checked the prerequisites packages, finding that system lack of some. What’s worse, some packages are not suitable because of its version. To solve it, I download the DVD img which can make me much more convenient to update those packages. However, some packages could not be satisfied in the end. And I found the installation needed internet access which made me very disappointing.

My senior told me to try KVM which is installed in Ubuntu. But, I found it was a false impression. I had to install Qemu. Soon, I found Qemu was not the key problem. The cpu of my pc does not support VT, that is, I could not run KVM. So I turned to VMware. The strange thing is I could not download the software because of the internet problem.

Suddenly, I found a paper named "Install Xen 3.3.1 in Federo 10" which told me how to install Xen offline. Since I have a Fedora 8 in Vmware, I immediately got down to working. I learned the last lesson in Ubuntu, I abondoned the idea to install a lower version of Xen. So I tried Xen 3.4.1. Ironically, this time, the verison of Fedora is too low to satisfy the Xen installation. I asked my another senior to download Fedora 11 DVD, and I hoped to release myself, because I felt so terrible during the course.

Unfortunately, the process of Xen installation on Fedora 11 is much shorter than 8, even if I tried Xen 3.3.0. I am nearly going crazy! Soon, I remembered when I installed Xen 3.0.1 in Fedora 8, the installation was succefully completed, yet I could not boot it, for some fd0 error. Desperately, I started installing Xen 3.3.1 in Fedora 8. "make"… "make install"… My heart flew up into my mouth when some errors appeared, however, the install process is still going, and I realized those are trivial ones.

Successful installation. After modifying the boot loader, I reboot the machine, waiting for the beautiful Xen interface. Yes, I have a Xen Liunx choice in boot, but I soon realized one similar error happened. This time, I believed I made some mistake in /boot/grub/menu.list. With the help of senior, I finally finished the correct modification.

Done! I could not help laughing! It has already taken me more than one week!

Running a xen in vmware is too slow, my next step is to install a Fedora 8 on my pc, and directly install Xen 3.3.0 in it. There’s a long way to finish the virtual cluster, and also a long way to acquaint Linux and virtualization technology. However, after I made a successful installation of Xen, I believe I can do more!

Following is my steps of Xen 3.3.0 installation in Fedora 8, hoping to give you some help.

1.check packages

#cd xen3.3.0/tools/check
#./chk build

2.modify makefile

#vim xen3.3.0/Makefile
comment all lines of codes with ’stubdom’

3.modify hg

#vim xen3.3.0/buildconfigs/src.hg-clone
comment all lines of codes with ‘hg’

4.generate linux-2.6.18-xen.hg

#cp linux-2.6.18-xen-3.3.0.tar ./xen3.3.0/
#tar -xzf linux-2.6.18-xen-3.3.0.tar
#mv linux-2.6.18-xen-3.3.0 linux-2.6.18-xen.hg

5.install

#make
#make install

6.install check. successful installation will give you following changes.

/etc/xen
/lib/modules
/boot

7.generate initrd-2.6.18.8-xen.img

#cd /boot
#mkinitrd -v -f –with=aacraid –with=sd_mod –with=scsi_mod initrd-2.6.18.8-xen.img 2.6.18.8-xen

8.Modify the boot loader

#vim /boot/grub/menu.lst

title  Xen 3.3 / XenLinux 2.6
root(hd0, 0)
kernel  /xen-3.3.gz dom0_mem=262144
module  /vmlinuz-2.6.18.8-xen root=/dev/VolGroup00/LogVol00 ro console=tty0
module  /initrd-2.6.18.8-xen.img

#sudo mv /lib/tls /lib/tls.disabled

9.reboot the machine, and verify that it boots into the Xen kernel, and check that Xen is running correctly by running the command sudo xm list, which should show that Domain-0 is the only machine at the moment.

 

2009年02月05日

I don’t know whether this title is suitable or not.

My new year holiday is one day left and my next return is at least six months later. Since my Internet was cut off and it seems not taking a turn for better. That means you may not see me on the internet untill several months later, so I have to say something before leaving.

I had a really good holiday this time, not only because I had a total drunk on the get-together of classmates of senior high school. I have to confess that make me insensible for one whole day and I would not like to have a taste of wine. However, there’s a long way of practising drinking to go.

2008 is gone, and 2009 is a new start. Usually, as many of you, I have some new year’s wishes. Yet, sometimes, I am thinking of one question, that is, where are we after the earth is destroyed, like by the explosion of the universe or something else? Once I have this question in my mind, I feel so terrible and helpless. Definitely, I think there’s no one can help me figure out this problem. From another perspective, why do we need that much money? The phenomenon that many people bought good cars after the earthquake may explain more clearly what I want to tell. It is a confusing topic.

What I am going to do is to take a train, check the scores, choose the courses and study even harder. I am wondering if my boss would assign some work to me or not.

2009 is my year, that is, my year of birth considered in relation to the Twelve Terrestrial Branches. I wish a good luck. when 24-year-old, David Beckham and his teammates helped Manchester United become a "Triple Crown", and I hope I can do something just like my idol.

What’s more, I’d like to develop a better relationship with her because she is so nice that I am under the impression that she is right for me. I am kind of knowing what the surprise she wants to give me, yet, I am not going to expose it. So, expecting…

2008年12月15日

I am not able to describe the feeling with her, but it is nice.

She has a good character, and she is good-tempered & beautiful. Now, I am sure about her feeling of me and this makes me happy.

The most important is that though it is just three weeks since we first met each other, I have the feeling that she has known me a lot.

I’d like to develop this good relationship and I hope this would last long, to eternal.

Bless~

To good good study.

2008年10月17日

Immediately I logged on the Xiaonei, I received so much bless from many friends which made me so happy that I decided to record my feeling today.

You can’t imagine that I started work at 6:30 in the morining because I was on duty, that is, I would not do anything except watch the west gate, greeting senior officer, answering the phone and making some notes. It was quite a relief as I could read books for reviewing. Everything went well until I called for a short order, yet receiving nothing even after one hour.

2 o’clock p.m. saw my duty handing over to my roommate. Later, I attended a lecture which told us how to apply for studying overseas. My direct thought was to study hard and prepared.

Luck backed to me in the evening especially having lessions. After that, 312 drove outside to dine together for my birthday. The scene which I dreamed before I came here came true. We four ate grilled food with a bottle of beer, talking and laughing. This totally impressed me a lot, and confirmed my confidence to stay.

Back to dorm with birthday cake, we celebrated my birthday with lighting the candles and singing the birthday song. And to thank senior fellow apprentices in 120, we did not forget sending them slices of cake.

Thank you very much for your bless, my friends.

Thank you very much for your care, my bros in 312.

2008年10月13日

Concentrative training is over and my life backs to normal.

Courses

Courses for graduate seem much more difficult and I have to review them after classes. Though I know it does not work well for the final examination, I can take heart from my life of study. You may know that I like teaching in English, yet, frankly speaking, I am sort of disliking the original books which are written in English. Yes, I know, they are the best ones and I may encounter some troubles if only reading Chinese version. Well, I think I have to be used to it. However, it is strange that I like reading papers which are written in English, because I know it is the directest way to get the advanced techniques. Sigh~ I may transplant this good habit to my book reading.

Training

Though concentrative training came to end, we have another tough task. Fortunately, I belong to "the rest", as a result, what I am going to do is to watch "the square" training. Without tired movement, I take it for granted that we, the rest, should be arranged to other useful plans instead of this time-consuming work.

Work

Hard courses lead to one result, that is, we, the new comer, don’t need to consider the work of lab. Thanks to passing PETS5 ahead, I do not have to have English lessions and this makes my spare time a little more sufficient. On my demand, boss  released my first task, a translation of paper of 93 pages. As a rookie of this research area, I believe this would take me a long time to absorb. Well, it is just a start, like the one two years ago. As I can achieve a good result in MDS when I was an undergraduate, I am 100% behind myself that a bright future is waiting for me.

To be good!

2008年06月22日

1-3, completely fail.

With great defeat of Italy and France in the group match, Holland was regarded as the champion of Europe. However, though as a fan of them, I know, they will not have a long way to go which is the worst thing I wanna see. Well, I have to say, Russia deserved to win.

This quter-final match ended at 5 o’clock in the morning, but I did not wanna sleep. With 3 more hours writing, I finished the first vision of my graduation dissertation. I know, it is to be modified, however, I have accomplished most of work. There is a translation of reference left and I believe I am able to manage it soon.

Notified that letter of admission to NUDT is sended and I will receive it soon. Nice!

Cherish the last days of my undergraduate life