Build Power 5 EGATE Prod - Mt Xia: Global System Administration: AIX

Current Location

js
GlobalSysAdmin
AIX

AIX Admin Methodology
Global Consolidation Project
All AIX admins should join
www.aixexpert.com

Join our LinkedIn Group
AIX Advanced Technical Experts
Contract Opportunities
www.LinkedIn.com

	Digg this page
	Post to del.icio.us
	Slashdot it!

LPAR Leasing
Lease an AIX / i5 LPAR
Reduce your costs
www.mtxia.com

Server Leasing
Lease a Server off-site
Reduce your costs
www.mtxia.com

Data Center Automation
Business Continuity and DR
Virtualization/Consolidation
www.mtxia.com

HMC Service
Hardware Management Console
Manage Remote AIX / i5 LPARs
www.siteox.com

Business Web Site Hosting
$3.99 / month includes Tools,
Shopping Cart, Site Builder
www.siteox.com

Disaster Recovery
Small Business Oriented
Off-Site Facilities
www.mtxia.com

IBM pSeries / iSeries
Reduce your Costs
Off-Site Server Hosting
www.mtxia.com

The following document describes the procedure used to move the EGATE Production database environment from the current Power 4 architecture to the Power5/6/7 architecture on the p590's with the Production Disaster Recovery provisions in place.

The first phase of this process is to shutdown the EGATE database environment on the current Power 4 production platform.

The second phase is to acquire the production resources on the Power5/6/7 systems.

The third phase is to start the production EGATE database environment on the Power5/6/7 systems.

The last phase is to verify and implement monitoring.

Phase 1

- Shutdown the EGATE database environment on the production platform.

Phase 1 Start Time: 00:01
Phase 1 Duration: 92m

Wait for database shutdown
mxfmtxdb85:
Start Time: 00:01
Duration: 3m
mxfmtxdb86:
Start Time: 00:01
Duration: 3m

Shutdown HACMP on EGATE database power 4 machines and varyoff all non-"rootvg" volume groups

mxfmtxdb85:

Start Time: 00:10
Duration: 5m

smitty clstop
umount all

for VG in $( lsvg -o )
do
    [[ ${VG} = "rootvg" ]] && continue
    varyoffvg ${VG}
done

mxfmtxdb86:

Start Time: 00:04
Duration: 5m

smitty clstop
umount all

for VG in $( lsvg -o )
do
    [[ ${VG} = "rootvg" ]] && continue
    varyoffvg ${VG}
done

Shutdown EGATE database power 4 machines
mxfmtxdb85:
Start Time: 00:14
Duration: 4m shutdown -Fh
mxfmtxdb86:
Start Time: 00:11
Duration: 4m shutdown -Fh

Contact SAN group to turn off the Tru-Copy mirroring of the production EGATE database drives.
mxfmtxdb85:
Start Time: 00:16
Duration: 5m
mxfmtxdb86:
Start Time: 00:16
Duration: 5m

Contact Doug Lemons to make DNS changes

Start Time: 00:16
Duration: 5m

CNAME: point mxfmtxdb85 to the Power5/6/7 architecture machine mxfapega01
CNAME: point mxfmtxdb85 to the Power5/6/7 architecture machine mxfapega02

Start Power5/6/7 systems
mxfapega01:
Start Time: 00:19
Duration: Already Running
mxfapega02:
Start Time: 00:19
Duration: Already Running

Run configuration manager to pull in all SAN disks

mxfapega01:

Start Time: 00:21
Duration: 4m cfgmgr

mxfapega02:

Start Time: 00:21
Duration: 4m cfgmgr

Clear persistant reserves on all SAN disks

mxfapega01:

Start Time: 00:25
Duration: 10m

cd /usr/DynamicLinkManager/bin
for i in $( lspv | awk '{ print $1 }' | grep "^hdisk" )
do
    print y | ./dlmpr -c ${i}
done

mxfapega02:

Start Time: 00:27
Duration: 10m

cd /usr/DynamicLinkManager/bin
for i in $( lspv | awk '{ print $1 }' | grep "^hdisk" )
do
    print y | ./dlmpr -c ${i}
done

Remove all SAN disks and rediscover to obtain PVID's

mxfapega01:

Start Time: 00:36
Duration: 10m

for i in $( lspv | awk '{ print $1 }' )
do
    rmdev -Rdl ${i}
done
cfgmgr

mxfapega02:

Start Time: 00:36
Duration: 10m

for i in $( lspv | awk '{ print $1 }' )
do
    rmdev -Rdl ${i}
done
cfgmgr

Phase 2

- Acquire the production resources on the Power5/6/7 systems.

Phase 2 Start Time: 01:33
Phase 2 Duration: 100m

Check SAN disks against list of PVID's then import VG's using OCIDR.
mxfapega01:
Start Time: 01:33
Duration: 13m cd /usr/lpp/dr/mxfmtxdb85 ./OCIDR -IGm config
mxfapega02:
Start Time: 01:46
Duration: 9m cd /usr/lpp/dr/mxfmtxdb86 ./OCIDR -IGm config

Restore the /etc/hosts and /.rhosts files from the original Power 4 system from the DR files

mxfapega01:

Start Time: 01:57
Duration: 10m

/usr/lpp/dr/mxfmtxdb85_man/uueConfig/!etc!hosts.uue
/usr/lpp/dr/mxfmtxdb85_man/uueConfig/!.rhosts.uue


vi /etc/hosts

127.0.0.1       loopback localhost
# 
# mxf Apps Cluster
#
146.61.220.152  mxfmtxapp80
146.61.220.158  mxfmtxapp80_boot
192.168.30.152  mxfmtxapp80_stby
#
146.61.220.153  mxfmtxapp81
146.61.220.159  mxfmtxapp81_boot
192.168.30.153  mxfmtxapp81_stby
#
# mxfAPEGA OPS CLUSTER 
#
146.61.220.45   mxfapega01              mxfapega01.tu.com
146.61.220.47   mxfapega01_boot         mxfapega01_boot.tu.com
192.168.30.45   mxfapega01_stby         mxfapega01_stby.tu.com
146.61.242.77   mxfapega01_man          mxfapega01_man.tu.com

146.61.220.46   mxfapega02              mxfapega02.tu.com
146.61.220.48   mxfapega02_boot         mxfapega02_boot.tu.com
192.168.30.46   mxfapega02_stby         mxfapega02_stby.tu.com
146.61.242.78   mxfapega02_man          mxfapega02_man.tu.com

146.61.220.254  mxfxre0a mxfxre0a.tu.com

mxfapega02:

Start Time: 02:08
Duration: 1m

/usr/lpp/dr/mxfmtxdb86_man/uueConfig/!etc!hosts.uue
/usr/lpp/dr/mxfmtxdb86_man/uueConfig/!.rhosts.uue


vi /etc/hosts

127.0.0.1       loopback localhost
# 
# mxf Apps Cluster
#
146.61.220.152  mxfmtxapp80
146.61.220.158  mxfmtxapp80_boot
192.168.30.152  mxfmtxapp80_stby
#
146.61.220.153  mxfmtxapp81
146.61.220.159  mxfmtxapp81_boot
192.168.30.153  mxfmtxapp81_stby
#
# mxfAPEGA OPS CLUSTER 
#
146.61.220.45   mxfapega01              mxfapega01.tu.com
146.61.220.47   mxfapega01_boot         mxfapega01_boot.tu.com
192.168.30.45   mxfapega01_stby         mxfapega01_stby.tu.com
146.61.242.77   mxfapega01_man          mxfapega01_man.tu.com

146.61.220.46   mxfapega02              mxfapega02.tu.com
146.61.220.48   mxfapega02_boot         mxfapega02_boot.tu.com
192.168.30.46   mxfapega02_stby         mxfapega02_stby.tu.com
146.61.242.78   mxfapega02_man          mxfapega02_man.tu.com

146.61.220.254  mxfxre0a mxfxre0a.tu.com

Reconfigure HACMP to use application servers, start/stop scripts, etc. Then verify and synchronize.

mxfapega01:

Start Time: 02:10
Duration: 22m

Node Name: mxfapega01
Node Name: mxfapega02

Resource Group: opsconrg
    Concurrent Resource Volume Groups: 
        exppops01vg exppops2vg exppops3vg exppops4vg exppops5vg 
        exppops6vg exppops7vg exppops08vg exppops09vg exppops10vg 
        exppops11vg exppops12vg exppops13vg

Application Server: Oracle
Application Start Script: /usr/local/hascripts/start_oracle.ksh
Application Start Script: /usr/local/hascripts/stop_oracle.ksh

Application Server: TSM
Application Start Script: /usr/local/hascripts/start_tsm.ksh
Application Start Script: /usr/local/hascripts/stop_tsm.ksh

Resource Group: mxfapega01_ip
    Application Servers: Oracle TSM

Resource Group: mxfapega02_ip
    Application Servers: Oracle TSM

Resource Group Parallel Acquisition Order:
    mxfapega01_ip mxfapega02_ip opsconrg

Resource Group Parallel Release Order:
    mxfapega01_ip mxfapega02_ip opsconrg

Varyon the non-concurrent volume groups and set to automatically varyon at boot time.

mxfapega01:

Start Time: 02:32
Duration: 1m

varyonvg orabackupvg
varyonvg oraarch01vg
varyonvg orabinvg
varyonvg paging01vg

chvg -a y orabackupvg
chvg -a y oraarch01vg
chvg -a y orabinvg
chvg -a y paging01vg

mxfapega02:

Start Time: 02:33
Duration: 1m

varyonvg orabackupvg
varyonvg oraarch01vg
varyonvg orabinvg
varyonvg paging01vg

chvg -a y orabackupvg
chvg -a y oraarch01vg
chvg -a y orabinvg
chvg -a y paging01vg

Set the database filesystems to automatically mount at boot time.

mxfapega01:

Start Time: 02:34
Duration: 1m

chfs -A y /u00901
chfs -A y /u00902
chfs -A y /u00901old
chfs -A y /backup

mxfapega02:

Start Time: 02:35
Duration: 1m

chfs -A y /u00901
chfs -A y /u00902
chfs -A y /u00901old
chfs -A y /backup

Sync and verify, then start the HACMP configuration.
mxfapega01:
Start Time: 02:41
Duration: 4m smitty clstart
mxfapega02:
Start Time: 02:45
Duration: 4m smitty clstart

Change ownership of oracle LV's

mxfapega01:

Start Time: 02:37
Duration: 1m chown oracle:dba /dev/*orarp* chown oracle:dba /dev/*ex*lv

mxfapega02:

Start Time: 02:37
Duration: 1m chown oracle:dba /dev/*orarp* chown oracle:dba /dev/*ex*lv

Change the number of processes per user to 3000
mxfapega01:
Start Time: 02:38
Duration: 1m chdev -l sys0 -a maxuproc=3000
mxfapega02:
Start Time: 02:38
Duration: 1m chdev -l sys0 -a maxuproc=3000

Mount the first Oracle CD from the NIM server and run the "rootpre.sh" script to set permissions and such.

mxfapnim01:

Start Time: 02:49
Duration: Already exported exportfs -i -o 'root=*' /export/cdimages/Oracle8.1.7_Disk1

mxfapega01:

Start Time: 02:49
Duration: 2m

mount mxfapnim01:/export/cdimages/Oracle8.1.7_Disk1 /mnt
cd /mnt
./rootpre.sh
cd /tmp
umount /mnt

mxfapega02:

Start Time: 02:51
Duration: 2m

mount mxfapnim01:/export/cdimages/Oracle8.1.7_Disk1 /mnt
cd /mnt
./rootpre.sh
cd /tmp
umount /mnt

mxfapnim01:

Start Time: 02:53
Duration: exportfs -u /export/cdimages/Oracle8.1.7_Disk1

Reboot both servers and restart HACMP
Start Time: 02:52
Duration: 20m

Phase 3

- The third phase is to start the production EGATE database environment on the Power5/6/7 systems.

Phase 3 Start Time: 03:12
Phase 3 Duration: 33m

Production EGATE database startup
Start Time: 03:12
Duration: 33m

Restore the crontabs from the original Power 4 system from the DR files

mxfapega01:

Start Time: 03:13
Duration: 1m

for CFILE in /usr/lpp/dr/mxfmtxdb85_man/uueConfig/!var!spool!cron!crontabs!*
do
    eval ${CFILE}
done

mxfapega02:

Start Time: 03:14
Duration: 1m

for CFILE in /usr/lpp/dr/mxfmtxdb86_man/uueConfig/!var!spool!cron!crontabs!*
do
    eval ${CFILE}
done

Phase 4

- Verification and Monitoring

Phase 4 Start Time: 03:48
Phase 4 Duration: 37m

Verify connectivity
Start Time: 03:48
Duration: 37m
Verify monitoring
Start Time: 03:48
Duration: 37m

Build Power 5 EGATE Prod

LPAR Leasing
Lease an AIX / i5 LPAR
Reduce your costs
www.mtxia.com

Server Leasing
Lease a Server off-site
Reduce your costs
www.mtxia.com

Data Center Automation
Business Continuity and DR
Virtualization/Consolidation
www.mtxia.com

HMC Service
Hardware Management Console
Manage Remote AIX / i5 LPARs
www.siteox.com

Business Web Site Hosting
$3.99 / month includes Tools,
Shopping Cart, Site Builder
www.siteox.com

FREE Domain Registration
included with Web Site Hosting
Tools, Social Networking, Blog
www.siteox.com

Disaster Recovery
Small Business Oriented
Off-Site Facilities
www.mtxia.com

IBM pSeries / iSeries
Reduce your Costs
Off-Site Server Hosting
www.mtxia.com