Perform Basic vSphere Storage Troubleshooting
For this objective I used the following resources:
- vSphere Monitoring and Performance
- vSphere Storage
- vSphere Troubleshooting
- VMware KB Article 1008205 (Using ESXTOP to identify storage performance issues)
- VMware KB Article 1003728 (Testing VMkernel network connectivity with the vmkping command)
- VMware KB Article 1020692 (Frequent path state changes indicate storage problem)
- VMware KB Article 1005009 (Analyzing SCSI Reservation conflicts)
- VMware SAN System Design and Deployment Guide
- Duncan Eppings Yellow-Bricks.com Blog
Knowledge
Verify storage configuration
Refer to the vSphere Storage and the SAN System Design and Deployment Guide (not specific to vSphere 5, but worth a read) by VMware. This will cover a lot of areas needed for working with a FC/iSCSI SAN environment with vSphere. Also a good understanding of the hardware you are using on the backend (storage arrays, FC switches, networking, etc) and there “vSphere Best Practices” documents will assist in the proper configuration.
Troubleshoot storage contention issues
When they mention “storage contention” I am taking this as I/O throughput or I/O latency issues. I find the quickest and easiest way of measuring/checking this is via esxtop/resxtop. VMware KB 1008205 and Duncan Eppings esxtop blog post covers this is in more detail.
Metrics to be aware of:
| Disk Metric | Threshold | Description |
DAVG
|
25
| This is the average response time in milliseconds per command being sent to the device. |
GAVG
|
25
| This is the response time as it is perceived by the guest operating system. This number is calculated with the formula: DAVG + KAVG = GAVG. |
KAVG
|
2
| This is the amount of time the command spends in the VMkernel |
Also see pages 47 thru 50 of the vSphere Troubleshooting documentation for further information.
Troubleshoot storage over-commitment issues
Storage over-commitment can occur in several ways:
- As mentioned above with I/O latency
- Datastore usage (Think Thin Provisioning and over subscription)
- SCSI reservation issues (VMware KB 1005009)
- Path thrashing (VMware KB 1020692)
- LUN queue depth
Troubleshoot iSCSI software initiator configuration issues
Things to look out for:
- Setting up a proper iSCSI Network (pg 74 thru 79 of the vSphere Storage document)
- Configuration of Jumbo Frames (pg 80 of the vSphere Storage document)
- Network connectivity using vmkping (See VMware KB 1003728)
For further information refer to pages 107 thru 112 of the vSphere Storage documentation as well as page 51 of the vSphere Troubleshooting documentation.
Troubleshoot Storage Reports and Storage Maps
See pages 29 thru 32 of the vSphere Monitoring and Performance documentation.
Identify the root cause of a storage issue based on troubleshooting information
The vSphere Troubleshooting document covers several issues that you may run into. See Pages 45 thru 51.
Tools
- vSphere Storage guide
- vSphere Troubleshooting guide
No comments:
Post a Comment