Migrating GAD Pairs Non-disruptively using GAD
Introduction
This blog shows how to migrate global-active device (GAD) pairs from a Hitachi Virtual Storage Platform F1500 (VSP F1500) storage system to a VSP E1090 storage system non-disruptively. This migration was tested in a lab environment with microcode 80-06-93-00/80 (VSP F1500) and 93-07-01-80/00 (VSP E1090).
Environment
The migration environment consists of:
· Source GAD: 2x VSP F1500
· Target GAD: 2x VSP E1090
· Quorum: 1x G600
· One RHEL host
· FC switch for Host and Remote connections.
Figure 1 shows the setup used for migrating GAD volumes from a VSP F1500 storage system to a VSP E1090 storage system. A resilient storage connection was established between the two storage systems, allowing them to withstand single controller or single CHB failures. Two quorum disks were created from the VSP G600 storage system for the source and target GAD pairs. A temporary GAD pair is created for data migration with a volume-less quorum device. The migration workflow is illustrated using a single GAD pair.
Figure 1: GAD migration environment
Setup
Create remote paths, quorums, and prepare volumes for testing. For GAD pair operations create four horcm files (one for each storage system), as shown in the following table. In the lab environment, a single GAD pair is used for data migration.
|
Source Secondary
(horcm1.conf)
|
Source Primary
(horcm0.conf)
|
Target Primary
(horcm2.conf)
|
Target Secondary
(horcm3.conf)
|
|
HORCM_MON
#ip_address service poll(10ms) timeout(10ms)
localhost 92000 1000 3000
HORCM_CMD
#dev_name dev_name dev_name
\\.\IPCMD-172.x.x.x-31001
HORCM_LDEV
#GRP DEV SERIAL LDEV# MU#
ora0 dev1 322222 26214 0
HORCM_INSTP
ora0 localhost 91000 1
|
HORCM_MON
#ip_address service poll(10ms) timeout(10ms)
localhost 91000 1000 3000
HORCM_CMD
#dev_name dev_name dev_name
\\.\IPCMD-172.x.x.x-31001
HORCM_LDEV
#GRP DEV SERIAL LDEV# MU#
ora0 dev1 311111 21845 0
ora1 dev2 311111 21845 1
HORCM_INSTP
#GRP IP_ADR PORT#
ora0 localhost 92000 1
ora1 localhost 96000 2
|
HORCM_MON
#ip_address service poll(10ms) timeout(10ms)
localhost 96000 1000 3000
HORCM_CMD
#dev_name dev_name dev_name
\\.\IPCMD-172.x.x.x-31001
HORCM_LDEV
#GRP DEV SERIAL LDEV# MU#
ora1 dev2 733333 21845 1
ora2 dev3 733333 21845 2
HORCM_INSTP
#GRP IP_ADR PORT#
ora1 localhost 91000 2
ora2 localhost 97000 3
|
HORCM_MON
#ip_address service poll(10ms) timeout(10ms)
localhost 97000 1000 3000
HORCM_CMD
#dev_name dev_name dev_name
\\.\IPCMD-172.x.x.x-31001
HORCM_LDEV
#GRP DEV SERIAL LDEV# MU#
ora2 dev3 744444 26214 2
HORCM_INSTP
#GRP IP_ADR PORT#
ora2 localhost 96000 3
|
Remote path
Two initiators and two targets are created between each storage system, and remote paths are designed in such a way that GAD pair operations are not interrupted by a single CTL or CHB failure. A separate path group ID is selected for each remote connection. For example, path group IDs 1, 2, and 3 are created for three remote connections as follows:
Remote paths between Source Primary and Secondary for Source GAD pairs:
# raidcom add rcu -cu_free 311111 R8 1 -mcu_port CL1-F -rcu_port CL7-C -I1
# raidcom add rcu -cu_free 311111 R8 1 -mcu_port CL4-E -rcu_port CL4-D -I1
# raidcom add rcu -cu_free 322222 R8 1 -mcu_port CL8-C -rcu_port CL2-F -I0
# raidcom add rcu -cu_free 322222 R8 1 -mcu_port CL3-D -rcu_port CL1-E -I0
Remote paths between Source Primary and Target Primary for temporary GAD pairs:
# raidcom add rcu -cu_free 733333 M8 2 -mcu_port CL7-D -rcu_port CL5-C -I0
# raidcom add rcu -cu_free 733333 M8 2 -mcu_port CL2-D -rcu_port CL2-B -I0
# raidcom add rcu -cu_free 311111 R8 2 -mcu_port CL6-C -rcu_port CL8-D -I2
# raidcom add rcu -cu_free 311111 R8 2 -mcu_port CL1-B -rcu_port CL1-D -I2
Remote paths between Target Primary and Target Secondary for target GAD pairs:
# raidcom add rcu -cu_free 744444 M8 3 -mcu_port CL7-C -rcu_port CL7-C -I2
# raidcom add rcu -cu_free 744444 M8 3 -mcu_port CL4-B -rcu_port CL4-B -I2
# raidcom add rcu -cu_free 733333 M8 3 -mcu_port CL8-C -rcu_port CL8-C -I3
# raidcom add rcu -cu_free 733333 M8 3 -mcu_port CL3-B -rcu_port CL3-B -I3
Quorums
A G600 storage system is used as an external storage system to provision quorum for source and target GAD pairs. For steps to set a physical quorum device, see the GAD user guide.
The microcode used does not support remote path and quorum operations from Storage Navigator between VSP F1500 and VSP E1090 storage systems. Quorum ID# 1 (volume-less) is created for the temporary GAD pair using the following commands:
# raidcom add quorum -quorum_id 1 -request_id auto -remote_storage 733333 M800 -I0
# raidcom add quorum -quorum_id 1 -request_id auto -remote_storage 311111 R8 -I2
Prepare GAD Volumes
A Virtual Storage Machine (VSM) is created with the Source Primary storage system information on the remaining three storage systems. The following shows creating the resource group and virtualized GAD S-VOLs for each of the three storage systems.
Source Secondary:
# raidcom add resource -resource_name VSM-511111 -virtual_type 311111 R800 -I1
# raidcom add resource -resource_name VSM-511111 -port CL8-C-1 -I1
# raidcom unmap resource -ldev_id 66:66 -virtual_ldev_id 66:66 -I1
# raidcom map resource -ldev_id 66:66 -virtual_ldev_id reserve -I1
# raidcom add resource -resource_name 'VSM-511111' -ldev_id 66:66 -I1
Target Primary:
# raidcom add resource -resource_name VSM-511111 -virtual_type 311111 R800 -I2
# raidcom add resource -resource_name VSM-511111 -port CL2-C-1 -I2
# raidcom unmap resource -ldev_id 55:55 -virtual_ldev_id 55:55 -I2
# raidcom map resource -ldev_id 55:55 -virtual_ldev_id reserve -I2
# raidcom add resource -resource_name 'VSM-511111' -ldev_id 55:55 -I2
Target Secondary:
# raidcom add resource -resource_name VSM-511111 -virtual_type 311111 R800 -I3
# raidcom add resource -resource_name VSM-511111 -port CL2-B-1 -I3
# raidcom unmap resource -ldev_id 66:66 -virtual_ldev_id 66:66 -I3
# raidcom map resource -ldev_id 66:66 -virtual_ldev_id reserve -I3
# raidcom add resource -resource_name 'VSM-511111' -ldev_id 66:66 -I3
Migration Methodology
The high-level workflow for migrating a GAD pair from a storage system to a new storage system using GAD are as follows:
- Setting up the Target Primary and Secondary storage systems.
- Creating the source, temporary, and target GAD pairs.
- Suspending the source GAD pair by specifying the P-VOL, and then enabling the host to recognize the S-VOL of the temporary GAD pair used for data migration in the PAIR status.
- Suspending the temporary GAD pair created for data migration by specifying the S-VOL, and then enabling the host to recognize the S-VOL of the target GAD pair in the PAIR status.
- Deleting the source GAD pair and the temporary GAD pair created for data migration.
- Optionally, removing the physical paths for the source storage systems, and then removing the storage systems.
To start the migration workflow, you must first create a GAD environment by completing the following steps:
1. Create the source GAD pair between two VSP F1500 storage systems by running the following command:
# paircreate -g ora0 -vl -fg never 0 -jq 0 -I0
# pairdisplay -g ora0 -fxce -I0
Group PairVol(L/R) (Port#,TID, LU),Seq#,LDEV#.P/S,Status,Fence, %,P-LDEV# M CTG JID AP EM E-Seq# E-LDEV# R/W QM DM P PR
ora0 dev1(L) (CL1-C-1, 0, 0)311111 5555.P-VOL PAIR NEVER , 100 6666 - 0 0 2 - - - L/M AA D N D
ora0 dev1(R) (CL8-C-1, 0, 0)322222 6666.S-VOL PAIR NEVER , 100 5555 - 0 0 2 - - - L/M AA S N D
2. When the source GAD pair is in PAIR state, create a temporary GAD pair between the Source Primary F1500 storage system and the Target Primary E1090 storage system for data migration. The temporary GAD pair remains in COPY state until the source GAD pair is split.
# paircreate -g ora1 -vl -fg never 1 -jq 1 -I0
# pairdisplay -g ora1 -fxce -I0
Group PairVol(L/R) (Port#,TID, LU),Seq#,LDEV#.P/S,Status,Fence, %,P-LDEV# M CTG JID AP EM E-Seq# E-LDEV# R/W QM DM P PR
ora1 dev2(L) (CL1-C-1, 0, 0)311111 5555.P-VOL COPY NEVER , 99 5555 - 1 1 2 - - - L/M - D N D
ora1 dev2(R) (CL2-C-1, 0, 0)733333 5555.S-VOL COPY NEVER ,----- 5555 - 1 1 2 - - - B/B - S N D
3. Create a target GAD pair between the Target Primary and the Target Secondary storage system. This pair remains in COPY state until the temporary GAD pair is split.
# paircreate -g ora2 -vl -fg never 2 -jq 2 -I2
# pairdisplay -g ora2 -fxce -I2
Group PairVol(L/R) (Port#,TID, LU),Seq#,LDEV#.P/S,Status,Fence, %,P-LDEV# M CTG JID AP EM E-Seq# E-LDEV# R/W QM DM P PR
ora2 dev3(L) (CL2-C-1, 0, 0)733333 5555.P-VOL COPY NEVER , 99 6666 - 2 2 2 - - - L/M AA S N D
ora2 dev3(R) (CL2-B-1, 0, 0)744444 6666.S-VOL COPY NEVER ,----- 5555 - 2 2 2 - - - B/B AA S N D
Migration Workflow
The following describes the workflow to migrate a GAD pair from the source storage system to the target storage system.
When the GAD environment is prepared, the source GAD pair is in PAIR state, the temporary and target GAD pairs are in COPY state, and host I/O is used by both the source F1500 storage systems, as shown in Figure 2.
Figure 2: Status of GAD pairs before migration
1. Suspend the Source GAD pair by specifying the P-VOL. This changes the source GAD pair to PSUS/SSUS, the temporary GAD pair to PAIR/PAIR, and the Target GAD pair remains in COPY/COPY state.
# pairsplit -g ora0 -I0
# pairdisplay -g ora0 -fxce -I0
Group PairVol(L/R) (Port#,TID, LU),Seq#,LDEV#.P/S,Status,Fence, %,P-LDEV# M CTG JID AP EM E-Seq# E-LDEV# R/W QM DM P PR
ora0 dev1(L) (CL1-C-1, 0, 0)311111 5555.P-VOL PSUS NEVER , 100 6666 - 0 0 2 - - - L/L - D N D
ora0 dev1(R) (CL8-C-1, 0, 0)322222 6666.S-VOL SSUS NEVER , 100 5555 - 0 0 2 - - - B/B - S N D
# pairdisplay -g ora1 -fxce -I0
Group PairVol(L/R) (Port#,TID, LU),Seq#,LDEV#.P/S,Status,Fence, %,P-LDEV# M CTG JID AP EM E-Seq# E-LDEV# R/W QM DM P PR
ora1 dev2(L) (CL1-C-1, 0, 0)311111 5555.P-VOL PAIR NEVER , 100 5555 - 1 1 2 - - - L/M - D N D
ora1 dev2(R) (CL2-C-1, 0, 0)733333 5555.S-VOL PAIR NEVER , 100 5555 - 1 1 2 - - - L/M - S N D
# pairdisplay -g ora2 -fxce -I2
Group PairVol(L/R) (Port#,TID, LU),Seq#,LDEV#.P/S,Status,Fence, %,P-LDEV# M CTG JID AP EM E-Seq# E-LDEV# R/W QM DM P PR
ora2 dev3(L) (CL2-C-1, 0, 0)733333 5555.P-VOL COPY NEVER , 99 6666 - 2 2 2 - - - L/M AA S N D
ora2 dev3(R) (CL2-B-1, 0, 0)744444 6666.S-VOL COPY NEVER ,----- 5555 - 2 2 2 - - - B/B AA S N D
2. When the temporary GAD pair is in PAIR state, rescan host paths to recognize the S-VOL from the Target Primary Storage System to allow the host to see I/O paths from both the Source Primary and Target Primary storage systems, as shown in the Figure 3.
Figure 3: Status of GAD pairs after suspending the source GAD pair
3. Suspend the temporary GAD pair created for data migration by specifying the S-VOL. This changes the temporary GAD pair to PSUS/SSWS and the target GAD pair to PAIR/PAIR state.
# pairsplit -g ora1 -RS -I2
# pairdisplay -g ora0 -fxce -I0
Group PairVol(L/R) (Port#,TID, LU),Seq#,LDEV#.P/S,Status,Fence, %,P-LDEV# M CTG JID AP EM E-Seq# E-LDEV# R/W QM DM P PR
ora0 dev1(L) (CL1-C-1, 0, 0)311111 5555.P-VOL PSUS NEVER , 100 6666 - 0 0 2 - - - L/L - D N D
ora0 dev1(R) (CL8-C-1, 0, 0)322222 6666.S-VOL SSUS NEVER , 100 5555 - 0 0 2 - - - B/B - S N D
# pairdisplay -g ora1 -fxce -I0
Group PairVol(L/R) (Port#,TID, LU),Seq#,LDEV#.P/S,Status,Fence, %,P-LDEV# M CTG JID AP EM E-Seq# E-LDEV# R/W QM DM P PR
ora1 dev2(L) (CL1-C-1, 0, 0)311111 5555.P-VOL PSUS NEVER , 100 5555 - 1 1 2 - - - B/B - D N D
ora1 dev2(R) (CL2-C-1, 0, 0)733333 5555.S-VOL SSWS NEVER , 100 5555 - 1 1 2 - - - L/L - S N D
# pairdisplay -g ora2 -fxce -I2
Group PairVol(L/R) (Port#,TID, LU),Seq#,LDEV#.P/S,Status,Fence, %,P-LDEV# M CTG JID AP EM E-Seq# E-LDEV# R/W QM DM P PR
ora2 dev3(L) (CL2-C-1, 0, 0)733333 5555.P-VOL PAIR NEVER , 100 6666 - 2 2 2 - - - L/M AA S N D
ora2 dev3(R) (CL2-B-1, 0, 0)744444 6666.S-VOL PAIR NEVER , 100 5555 - 2 2 2 - - - L/M AA S N D
4. When the target GAD pair is in PAIR state, rescan host paths to recognize the S-VOL from the Target Secondary Storage System to allow the host to receive I/O from both the Target Primary and Secondary storage systems, as shown in Figure 4.
Figure 4: Status of GAD pairs after suspending the temporary GAD pair
5. After the GAD pair is migrated to the Target Primary and Secondary storage system, delete the source GAD pair and the temporary GAD pair created for data migration. Optionally, you can remove the physical paths for the source storage systems and then remove the storage systems.
# pairsplit -g ora0 -S -I0
# pairsplit -g ora1 -R -I2
# pairdisplay -g ora0 -fxce -I0
Group PairVol(L/R) (Port#,TID, LU),Seq#,LDEV#.P/S,Status,Fence, %,P-LDEV# M CTG JID AP EM E-Seq# E-LDEV# R/W QM DM P PR
ora0 dev1(L) (CL1-C-1, 0, 0)311111 5555.SMPL ---- ------,----- ----- - - - - - - - -/- - - - -
ora0 dev1(R) (CL8-C-1, 0, 0)322222 6666.SMPL ---- ------,----- ----- - - - - - - - -/- - - - -
# pairdisplay -g ora1 -fxce -I0
Group PairVol(L/R) (Port#,TID, LU),Seq#,LDEV#.P/S,Status,Fence, %,P-LDEV# M CTG JID AP EM E-Seq# E-LDEV# R/W QM DM P PR
ora1 dev2(L) (CL1-C-1, 0, 0)311111 5555.SMPL ---- ------,----- ----- - - - - - - - -/- - - - -
ora1 dev2(R) (CL2-C-1, 0, 0)733333 5555.SMPL ---- ------,----- ----- - - - - - - - -/- - - - -
# pairdisplay -g ora2 -fxce -I2
Group PairVol(L/R) (Port#,TID, LU),Seq#,LDEV#.P/S,Status,Fence, %,P-LDEV# M CTG JID AP EM E-Seq# E-LDEV# R/W QM DM P PR
ora2 dev3(L) (CL2-C-1, 0, 0)733333 5555.P-VOL PAIR NEVER , 100 6666 - 2 2 2 - - - L/M AA S N D
ora2 dev3(R) (CL2-B-1, 0, 0)744444 6666.S-VOL PAIR NEVER , 100 5555 - 2 2 2 - - - L/M AA S N D
Figure 5: Migration workflow completes, and the source storage systems are disconnected
Failure and Recovery
This section describes the procedure to recover from a scenario where the power fails for a storage system. The storage connectivity must be configured to withstand failures such as single CHB or CTL. However, more critical failures that occur during the migration process can also be recovered. The following example shows the power failure of VSP E1090 storage system (Target Primary storage) after building a GAD environment. In this scenario, the GAD pair state is changed.
After recovering (power restore) the VSP E1090 primary storage system, complete the following steps:
Figure 6: Failure scenario on GAD Target Primary storage system
After failure, the GAD pair states change, as shown in Figure 6.
Source GAD Pair:
# pairdisplay -g ora0 -fxce -I0
Group PairVol(L/R) (Port#,TID, LU),Seq#,LDEV#.P/S,Status,Fence, %,P-LDEV# M CTG JID AP EM E-Seq# E-LDEV# R/W QM DM P PR
ora0 dev1(L) (CL1-C-1, 0, 0)311111 5555.P-VOL PAIR NEVER , 100 6666 - 0 0 2 - - - L/M AA D N D
ora0 dev1(R) (CL8-C-1, 0, 0)322222 6666.S-VOL PAIR NEVER , 100 5555 - 0 0 2 - - - L/M AA S N D
Temporary GAD Pair:
# pairdisplay -g ora1 -fxce -I0
Group PairVol(L/R) (Port#,TID, LU),Seq#,LDEV#.P/S,Status,Fence, %,P-LDEV# M CTG JID AP EM E-Seq# E-LDEV# R/W QM DM P PR
ora1 dev2(L) (CL1-C-1, 0, 0)311111 5555.P-VOL PSUE NEVER , 1 5555 - 1 1 2 - - - L/L - D N D
ora1 dev2(R) (CL2-C-1, 0, 0)733333 5555.S-VOL PSUE NEVER , 100 5555 - 1 1 2 - - - B/B - S N D
Target GAD Pair:
# pairdisplay -g ora2 -fxce -I2
Group PairVol(L/R) (Port#,TID, LU),Seq#,LDEV#.P/S,Status,Fence, %,P-LDEV# M CTG JID AP EM E-Seq# E-LDEV# R/W QM DM P PR
ora2 dev3(L) (CL2-C-1, 0, 0)733333 5555.P-VOL PSUE NEVER , 98 6666 - 2 2 2 - - - L/L - S N D
ora2 dev3(R) (CL2-C-1, 0, 0)744444 6666.S-VOL COPY NEVER ,----- 5555 - 2 2 2 - - - B/B AA S N D
1. To recover from this situation, forcibly delete the target GAD pair by deleting the P-VOL and S-VOL virtual LDEV ID.
# pairsplit -g ora2 -RF -I3
# pairsplit -g ora2 -SF -I2
# pairdisplay -g ora2 -fxce -I2
Group PairVol(L/R) (Port#,TID, LU),Seq#,LDEV#.P/S,Status,Fence, %,P-LDEV# M CTG JID AP EM E-Seq# E-LDEV# R/W QM DM P PR
ora2 dev3(L) (CL2-C-1, 0, 0)733333 5555.SMPL ---- ------,----- ----- - - - - - - - -/- - - - -
ora2 dev3(R) (CL2-C-1, 0, 0)744444 6666.SMPL ---- ------,----- ----- - - - - - - - -/- - - - -
2. Forcibly delete the temporary GAD pair by deleting only the S-VOL virtual LDEV ID.
# pairsplit -g ora1 -RF -I2
# pairsplit -g ora1 -SFV -I0
# pairdisplay -g ora1 -fxce -I0
Group PairVol(L/R) (Port#,TID, LU),Seq#,LDEV#.P/S,Status,Fence, %,P-LDEV# M CTG JID AP EM E-Seq# E-LDEV# R/W QM DM P PR
ora1 dev2(L) (CL1-C-1, 0, 0)311111 5555.SMPL ---- ------,----- ----- - - - - - - - -/- - - - -
ora1 dev2(R) (CL2-C-1, 0, 0)733333 5555.SMPL ---- ------,----- ----- - - - - - - - -/- - - - -
Now, we have the Source GAD pair in PAIR state and the temporary and target GAD pair in SMPL state. Resume the migration operation after creating a temporary and target GAD pair, as mentioned in the Migration Methodology section.
#globalactivedevice #datareplication #businesscontinuity