Build Power 5 EGATE Prod - Global System Administration: AIX

Current Location

css
GlobalSysAdmin
AIX

Digg this page

Post to del.icio.us

Slashdot it!

Business Web Site Hosting
$3.99 / month includes Tools,
Shopping Cart, Site Builder
www.siteox.com

FREE Domain Registration
included with Web Site Hosting
Tools, Social Networking, Blog
www.siteox.com

The following document describes the procedure used to move the EGATE Production database environment from the current Power 4 architecture to the Power 5 architecture on the p590's with the Production Disaster Recovery provisions in place.

The first phase of this process is to shutdown the EGATE database environment on the current Power 4 production platform.

The second phase is to acquire the production resources on the Power 5 systems.

The third phase is to start the production EGATE database environment on the Power 5 systems.

The last phase is to verify and implement monitoring.

Phase 1

- Shutdown the EGATE database environment on the production platform.

Phase 1 Start Time: 00:01
Phase 1 Duration: 92m

Wait for database shutdown
mdctxudb85:
Start Time: 00:01
Duration: 3m
mdctxudb86:
Start Time: 00:01
Duration: 3m

Shutdown HACMP on EGATE database power 4 machines and varyoff all non-"rootvg" volume groups

mdctxudb85:

Start Time: 00:10
Duration: 5m

smitty clstop
umount all

for VG in $( lsvg -o )
do
    [[ ${VG} = "rootvg" ]] && continue
    varyoffvg ${VG}
done

mdctxudb86:

Start Time: 00:04
Duration: 5m

smitty clstop
umount all

for VG in $( lsvg -o )
do
    [[ ${VG} = "rootvg" ]] && continue
    varyoffvg ${VG}
done

Shutdown EGATE database power 4 machines
mdctxudb85:
Start Time: 00:14
Duration: 4m shutdown -Fh
mdctxudb86:
Start Time: 00:11
Duration: 4m shutdown -Fh

Contact SAN group to turn off the Tru-Copy mirroring of the production EGATE database drives.
mdctxudb85:
Start Time: 00:16
Duration: 5m
mdctxudb86:
Start Time: 00:16
Duration: 5m

Contact Doug Lemons to make DNS changes

Start Time: 00:16
Duration: 5m

CNAME: point mdctxudb85 to the power 5 architecture machine mdcapega01
CNAME: point mdctxudb85 to the power 5 architecture machine mdcapega02

Start Power 5 systems
mdcapega01:
Start Time: 00:19
Duration: Already Running
mdcapega02:
Start Time: 00:19
Duration: Already Running

Run configuration manager to pull in all SAN disks

mdcapega01:

Start Time: 00:21
Duration: 4m cfgmgr

mdcapega02:

Start Time: 00:21
Duration: 4m cfgmgr

Clear persistant reserves on all SAN disks

mdcapega01:

Start Time: 00:25
Duration: 10m

cd /usr/DynamicLinkManager/bin
for i in $( lspv | awk '{ print $1 }' | grep "^hdisk" )
do
    print y | ./dlmpr -c ${i}
done

mdcapega02:

Start Time: 00:27
Duration: 10m

cd /usr/DynamicLinkManager/bin
for i in $( lspv | awk '{ print $1 }' | grep "^hdisk" )
do
    print y | ./dlmpr -c ${i}
done

Remove all SAN disks and rediscover to obtain PVID's

mdcapega01:

Start Time: 00:36
Duration: 10m

for i in $( lspv | awk '{ print $1 }' )
do
    rmdev -Rdl ${i}
done
cfgmgr

mdcapega02:

Start Time: 00:36
Duration: 10m

for i in $( lspv | awk '{ print $1 }' )
do
    rmdev -Rdl ${i}
done
cfgmgr

Phase 2

- Acquire the production resources on the Power 5 systems.

Phase 2 Start Time: 01:33
Phase 2 Duration: 100m

Check SAN disks against list of PVID's then import VG's using OCIDR.
mdcapega01:
Start Time: 01:33
Duration: 13m cd /usr/lpp/dr/mdctxudb85 ./OCIDR -IGm config
mdcapega02:
Start Time: 01:46
Duration: 9m cd /usr/lpp/dr/mdctxudb86 ./OCIDR -IGm config

Restore the /etc/hosts and /.rhosts files from the original Power 4 system from the DR files

mdcapega01:

Start Time: 01:57
Duration: 10m

/usr/lpp/dr/mdctxudb85_man/uueConfig/!etc!hosts.uue
/usr/lpp/dr/mdctxudb85_man/uueConfig/!.rhosts.uue


vi /etc/hosts

127.0.0.1       loopback localhost
# 
# MDC Apps Cluster
#
146.61.220.152  mdctxuapp80
146.61.220.158  mdctxuapp80_boot
192.168.30.152  mdctxuapp80_stby
#
146.61.220.153  mdctxuapp81
146.61.220.159  mdctxuapp81_boot
192.168.30.153  mdctxuapp81_stby
#
# MDCAPEGA OPS CLUSTER 
#
146.61.220.45   mdcapega01              mdcapega01.tu.com
146.61.220.47   mdcapega01_boot         mdcapega01_boot.tu.com
192.168.30.45   mdcapega01_stby         mdcapega01_stby.tu.com
146.61.242.77   mdcapega01_man          mdcapega01_man.tu.com

146.61.220.46   mdcapega02              mdcapega02.tu.com
146.61.220.48   mdcapega02_boot         mdcapega02_boot.tu.com
192.168.30.46   mdcapega02_stby         mdcapega02_stby.tu.com
146.61.242.78   mdcapega02_man          mdcapega02_man.tu.com

146.61.220.254  mdcxre0a mdcxre0a.tu.com

mdcapega02:

Start Time: 02:08
Duration: 1m

/usr/lpp/dr/mdctxudb86_man/uueConfig/!etc!hosts.uue
/usr/lpp/dr/mdctxudb86_man/uueConfig/!.rhosts.uue


vi /etc/hosts

127.0.0.1       loopback localhost
# 
# MDC Apps Cluster
#
146.61.220.152  mdctxuapp80
146.61.220.158  mdctxuapp80_boot
192.168.30.152  mdctxuapp80_stby
#
146.61.220.153  mdctxuapp81
146.61.220.159  mdctxuapp81_boot
192.168.30.153  mdctxuapp81_stby
#
# MDCAPEGA OPS CLUSTER 
#
146.61.220.45   mdcapega01              mdcapega01.tu.com
146.61.220.47   mdcapega01_boot         mdcapega01_boot.tu.com
192.168.30.45   mdcapega01_stby         mdcapega01_stby.tu.com
146.61.242.77   mdcapega01_man          mdcapega01_man.tu.com

146.61.220.46   mdcapega02              mdcapega02.tu.com
146.61.220.48   mdcapega02_boot         mdcapega02_boot.tu.com
192.168.30.46   mdcapega02_stby         mdcapega02_stby.tu.com
146.61.242.78   mdcapega02_man          mdcapega02_man.tu.com

146.61.220.254  mdcxre0a mdcxre0a.tu.com

Reconfigure HACMP to use application servers, start/stop scripts, etc. Then verify and synchronize.

mdcapega01:

Start Time: 02:10
Duration: 22m

Node Name: mdcapega01
Node Name: mdcapega02

Resource Group: opsconrg
    Concurrent Resource Volume Groups: 
        exppops01vg exppops2vg exppops3vg exppops4vg exppops5vg 
        exppops6vg exppops7vg exppops08vg exppops09vg exppops10vg 
        exppops11vg exppops12vg exppops13vg

Application Server: Oracle
Application Start Script: /usr/local/hascripts/start_oracle.ksh
Application Start Script: /usr/local/hascripts/stop_oracle.ksh

Application Server: TSM
Application Start Script: /usr/local/hascripts/start_tsm.ksh
Application Start Script: /usr/local/hascripts/stop_tsm.ksh

Resource Group: mdcapega01_ip
    Application Servers: Oracle TSM

Resource Group: mdcapega02_ip
    Application Servers: Oracle TSM

Resource Group Parallel Acquisition Order:
    mdcapega01_ip mdcapega02_ip opsconrg

Resource Group Parallel Release Order:
    mdcapega01_ip mdcapega02_ip opsconrg

Varyon the non-concurrent volume groups and set to automatically varyon at boot time.

mdcapega01:

Start Time: 02:32
Duration: 1m

varyonvg orabackupvg
varyonvg oraarch01vg
varyonvg orabinvg
varyonvg paging01vg

chvg -a y orabackupvg
chvg -a y oraarch01vg
chvg -a y orabinvg
chvg -a y paging01vg

mdcapega02:

Start Time: 02:33
Duration: 1m

varyonvg orabackupvg
varyonvg oraarch01vg
varyonvg orabinvg
varyonvg paging01vg

chvg -a y orabackupvg
chvg -a y oraarch01vg
chvg -a y orabinvg
chvg -a y paging01vg

Set the database filesystems to automatically mount at boot time.

mdcapega01:

Start Time: 02:34
Duration: 1m

chfs -A y /u00901
chfs -A y /u00902
chfs -A y /u00901old
chfs -A y /backup

mdcapega02:

Start Time: 02:35
Duration: 1m

chfs -A y /u00901
chfs -A y /u00902
chfs -A y /u00901old
chfs -A y /backup

Sync and verify, then start the HACMP configuration.
mdcapega01:
Start Time: 02:41
Duration: 4m smitty clstart
mdcapega02:
Start Time: 02:45
Duration: 4m smitty clstart

Change ownership of oracle LV's

mdcapega01:

Start Time: 02:37
Duration: 1m chown oracle:dba /dev/*orarp* chown oracle:dba /dev/*ex*lv

mdcapega02:

Start Time: 02:37
Duration: 1m chown oracle:dba /dev/*orarp* chown oracle:dba /dev/*ex*lv

Change the number of processes per user to 3000
mdcapega01:
Start Time: 02:38
Duration: 1m chdev -l sys0 -a maxuproc=3000
mdcapega02:
Start Time: 02:38
Duration: 1m chdev -l sys0 -a maxuproc=3000

Mount the first Oracle CD from the NIM server and run the "rootpre.sh" script to set permissions and such.

mdcapnim01:

Start Time: 02:49
Duration: Already exported exportfs -i -o 'root=*' /export/cdimages/Oracle8.1.7_Disk1

mdcapega01:

Start Time: 02:49
Duration: 2m

mount mdcapnim01:/export/cdimages/Oracle8.1.7_Disk1 /mnt
cd /mnt
./rootpre.sh
cd /tmp
umount /mnt

mdcapega02:

Start Time: 02:51
Duration: 2m

mount mdcapnim01:/export/cdimages/Oracle8.1.7_Disk1 /mnt
cd /mnt
./rootpre.sh
cd /tmp
umount /mnt

mdcapnim01:

Start Time: 02:53
Duration: exportfs -u /export/cdimages/Oracle8.1.7_Disk1

Reboot both servers and restart HACMP
Start Time: 02:52
Duration: 20m

Phase 3

- The third phase is to start the production EGATE database environment on the Power 5 systems.

Phase 3 Start Time: 03:12
Phase 3 Duration: 33m

Production EGATE database startup
Start Time: 03:12
Duration: 33m

Restore the crontabs from the original Power 4 system from the DR files

mdcapega01:

Start Time: 03:13
Duration: 1m

for CFILE in /usr/lpp/dr/mdctxudb85_man/uueConfig/!var!spool!cron!crontabs!*
do
    eval ${CFILE}
done

mdcapega02:

Start Time: 03:14
Duration: 1m

for CFILE in /usr/lpp/dr/mdctxudb86_man/uueConfig/!var!spool!cron!crontabs!*
do
    eval ${CFILE}
done

Phase 4

- Verification and Monitoring

Phase 4 Start Time: 03:48
Phase 4 Duration: 37m

Verify connectivity
Start Time: 03:48
Duration: 37m
Verify monitoring
Start Time: 03:48
Duration: 37m

Build Power 5 EGATE Prod

FREE Domain Registration
included with Web Site Hosting
Tools, Social Networking, Blog
www.siteox.com

Business Web Site Hosting
$3.99 / month includes Tools,
Shopping Cart, Site Builder
www.siteox.com

*AIX^tm^mtXIA*	Site Guide \| Contact Us Mt Xia: *Technical Consulting Group*
	Business Continuity / Disaster Recovery / High Availability Data Center Automation / Audit Response / Audit Compliance


	Home About Mt Xia Mt Xia Officers Privacy Policies FAQs Contact Us Events Press Releases Business Continuity Business Continuity Planning BC Auditing BC Policies BC Guidelines BC Standards BC Procedures BC Automated Deployment Disaster Recovery Disaster Recovery Planning DR Plan Template DR Project Implementation DR Auditing DR Automated Deployment High Availability High Availability Planning HA Policies HA Guidelines HA Standards HA Procedures HA Automated Deployment Tech Support/Training AIX 4.3 Tech Support AIX 5.1 Tech Support AIX 5.2 Tech Support AIX 5.3 Tech Support HACMP 4.4.1 Tech Support HACMP 5.2 Tech Support HACMP 5.3 Tech Support Disaster Recovery Unix / Linux Shell Programming E-Business Downloads Scripts Editors Documentation Standards Project Template E-Business Methodology E-Business Model Random Fortune Cookie