Hi, it’s time for Part2. I will show you what I did in set up procedure and use case.
Environment Preparation and HIAA installation
I set up the configuration in our Lab. I am going to write some workflow about HIAA with UCP CI particularly.
UCP CI Preparation
- Create a LDEV as Command Device on VSP G600
HIAA Probe communicate with VSP G600 Command Device to retrieve storage information including configuration and Performance statistics. This requires FibreChannel connection between VSP G600 and the server running HIAA Probe server VM.
- Install Brocade Network Advisor
HIAA Probe cannot communicate with Brocade SAN switch directly. If you would like to check Brocade Switch performance, you need Brocade Network Advisor (BNA) to retrieve information.
HIAA Installation & Initial setup
There are two options to install HIAA. One is deployment of OVA virtual machine image. Another option is using installer on a host.
My choice was first one (OVA file deployment). I created three VMs as Fig1. VM1 is for HDCA/HIAA server. VM2 is for Probe server. VM3 is VM to run windows. This is the host OS to install BNA.
Further detail HIAA installation instruction is shown below.
After installing HIAA, I added target probe. Probe is a kind of module to retrieve information from target machines. Probe for Hitachi Storage, BNA, vCenter are available.
If you don’t see any information on HIAA without any error, I recommend you wait an hour. It may take time to hand over. Because retrieved information is hand over across three servers (Probe – HDCA – HIAA).
Grouping of components, “Consumer”
At the time of registering components into HIAA, HIAA recognize them as just individual resources. “Consumer” is a great feature of grouping resources. I made a Consumer which groups UCP CI resources. This provides administrator clear recognition of UCP CI converged system.
I will show you an example of performance analytics use case for UCP CI with HIAA.
1. Finding performance Preblem in vCenter
In most cases, an administrator of virtualized environment manages the infrastructure with vCenter. Let's say the administrator found excessive high latency for storage (fc2) reported on the vCenter. (Fig4)
Fig4. Storage view in vCenter (Latency)
2. Problem Analytics using HIAA
Hand over to HIAA from this step. Login HIAA and check Dashboard.
Fig5. HIAA Dashboard
Alert "Critical" comes up in Dashboard. Then, Jump into E2E view.
Fig6. E2E View
This E2E view shows relationship of VM to LDEV. In this screen, HIAA shows Storage Processor and
LDEV are busy. HIAA E2E view shows VM and Host server is fine. These VMs use LDEV and MP in
G600 which is marked as Critical.
Let's back to vCenter, then open VM Monitor tab. We can see Disk performance mounted on VM.
Disk of VM performance was actually dropped(For example, VM fc-02). Meanwhile, example other
VM fc-18 started Write IO to its disk. I would like to improve performance, but also want to keep
both IO fc-02 and fc-18.
Fig8. Disk of VM fc-18 performance (from VM performance view)
Let’s drill down to find bottleneck.
3. Bottleneck investigation
First, I checked Storage performance. From screen below(Fig9), only MPU-10 is busy. This issue must be occurred uneven assignment of MPUnit. Workload of MPU-10 is above critical line (Red line). The other MPU do not work now. In this example, MPU-10 is primary bottleneck.
Fig9. Sparkling view
4. Performance improvement
Next step, what we can do to solve overload situation in MPU-10? Primary option is offloading workload to other MPUs.
I have to change MPU assignment configuration. This operation makes workloads distributed to other MPU.
Then, we need to identify what LDEV should be moved? Candidate LDEV are shown as below. (Fig10) Click busy MP, HIAA shows LDEVs related with MPU.
Fig10. Relationship LDEV with MPU
I distributed assignment of MP Unit across all MPUnit. Then all MPU started works evenly. (Fig11)
Fig11. After resolving MPU-10 overload
Finally, MP overload was resolved.
Fig12. All resources are fine
I introduced the value of combination HIAA and UCP CI. In the use case section, I showed you one of
the examples to improve performance issue in UCP CI environment.
I hope you can enjoy HIAA and UCP CI solution. Thank you for your time to read.
Please refer further information below,
Videos on YouTube:
"Detecting Performance Bottlenecks using E2E view in Hitachi Infrastructure Analytics Advisor"
"Dynamic Threshold Storage Resource Monitoring With Performance Analytics, Using HIAA"
"Using HIAA to Analyze a Performance Bottleneck in Shared Infrastructure"
"Detecting Performance Bottlenecks Using Sparkline View"