NIC Teaming Policies in VMware vSphere

The first point of interest is the load-balancing policy. This is basically how we tell the vSwitch to handle outbound traffic, and there are four choices on a standard vSwitch:

1. Route based on the originating virtual port
2. Route based on IP hash
3. Route based on source MAC hash
4. Use explicit failover order

Keep in mind that we’re not concerned with the inbound traffic because that’s not within our control. Traffic arrives on whatever uplink the upstream switch decided to put it on, and the vSwitch is only responsible for making sure it reaches its destination.

The first option, route based on the originating virtual port, is the default selection for a new vSwitch. Every VM and VMkernel port on a vSwitch is connected to a virtual port. When the vSwitch receives traffic from either of these objects, it assigns the virtual port an uplink and uses it for traffic. The chosen uplink will typically not change unless there is an uplink failure, the VM changes power state, or the VM is migrated around via vMotion.

The second option, route based on IP hash, is used in conjunction with a link aggregation group (LAG), also called an EtherChannel or port channel. When traffic enters the vSwitch, the load-balancing policy will create a hash value of the source and destination IP addresses in the packet. The resulting hash value dictates which uplink will be used.

The third option, route based on source MAC hash, is similar to the IP hash idea, except the policy examines only the source MAC address in the Ethernet frame. To be honest, we have rarely seen this policy used in a production environment, but it can be handy for a nested hypervisor VM to help balance its nested VM traffic over multiple uplinks.

The fourth and final option, use explicit failover order, really doesn’t do any sort of load balancing. Instead, the first Active NIC on the list is used. If that one fails, the next Active NIC on the list is used, and so on, until you reach the Standby NICs. Keep in mind that if you select the Explicit Failover option and you have a vSwitch with many uplinks, only one of them will be actively used at any given time. Use this policy only in circumstances where using only one link rather than load balancing over all links is desired or required.

Credit for this info goes to "Networking for VMware Administrators"

NOTE

In almost all cases, the route based on the originating virtual port is more than adequate. Don’t try to get fancy with an exotic load-balancing policy unless you see an issue where the majority of traffic is being sent down the same uplink and other uplinks are relatively quiet. Remember our motto—the simplest designs are almost always the best designs.

A single VM will not be able to take advantage of more than a single uplink in most circumstances. If you provide a pair of 1 Gb Ethernet uplinks to your vSwitch, a VM will still only use one of those uplinks at a time. There are exceptions to this concept, such as when a VM has multiple virtual NICs attached on a vSwitch with IP hash, but are relatively rare to see in production environments.

Notify Switches Option in VMware vSphere vSwitch

The Notify Switches configuration is a bit mystifying at first. Notify the switches about what, exactly? By default, it’s set to “Yes,” and as we cover here, that’s almost always a good thing.

Remember that all of your upstream physical switches have a MAC address table that they use to map ports to MAC addresses. This avoids the need to flood their ports—which means sending frames to all ports except the port they arrived on (which is the required action when a frame’s destination MAC address doesn’t appear in the switch’s MAC address table).

But what happens when one of your uplinks in a vSwitch fails and all of the VMs begin using a new uplink? The upstream physical switch would have no idea which port the VM is now using and would have to resort to flooding the ports or wait for the VM to send some traffic so it can re-learn the new port. Instead, the Notify Switches option speeds things along by sending Reverse Address Resolution Protocol (RARP) frames to the upstream physical switch on behalf of the VM or VMs so that upstream switch updates its MAC address table. This is all done before frames start arriving from the newly vMotioned VM, the newly powered-on VM, or from the VMs that are behind the uplink port that failed and was replaced.

These RARP announcements are just a fancy way of saying that the ESXi host will send out a special update letting the upstream physical switch know that the MAC address is now on a new uplink so that the switch will update its MAC address table before actually needing to send frames to that MAC address. It’s sort of like ESXi is shouting to the upstream physical switch and saying, “Hey! This VM is over here now!”

Credit for this info goes to "Networking for VMware Administrators"

Traffic Shaping in VMware vSphere

Traffic shaping is the ability to control the quantity of traffic that is allowed to flow across a link. That is, rather than letting the traffic go as fast as it possibly can, you can set limits to how much traffic can be sent.

Within a standard vSwitch, you can only enforce traffic shaping on outbound traffic that is being sent out of an object—such as a VM or VMkernel port—toward another object. This is referred to by VMware as “ingress traffic” and refers to the fact that data is coming into the vSwitch by way of the virtual ports.

 

• Average bandwidth (Kbps): The average amount of bandwidth, measured in kilobits per second (Kbps), that you allow the switch to send. There might be short periods where the traffic slightly exceeds this value, since it is an average over time, but for the most part, it will be enforced and traffic will go no faster than the defined speed limit set here.
• Peak bandwidth (Kbps): The maximum amount of bandwidth that the switch is allowed to let through. The use of the peak bandwidth value is determined by how often we’ve hit the average bandwidth limitation. Whenever the actual traffic volume is lower than the average bandwidth limit, we gain what is called a “burst bonus” which can be any number of bytes up to the limit set by the burst size value (covered next).
  This bonus can be used when there is a pent-up traffic demand to let more traffic flow through the switch using data sizes dictated by the burst size value.
• Burst size (KB): This is an often misunderstood value, so we’ll go into detail. The burst size is the actual amount of “burstable” data that is allowed to be transmitted at the peak bandwidth rate in kilobytes. Think of the burst bonus as a network traffic savings account. And the burst size is the maximum number of bytes that can go into that account. So, when you need to send more traffic than the average bandwidth value allows, you transmit a burst of traffic, which is more than the allowed average bandwidth. But this burst, which always stays at or below the allowable peak bandwidth, will be forced to end when the number of bytes in your traffic savings account, your burst bonus, reaches zero.

Here is an example showing a period of average traffic with a burst of peak bandwidth in the middle. You can determine how long the traffic will be able to burst by taking the burst size (KB) amount divided by the peak bandwidth (kbps).

 

Making changes to the traffic-shaping values will instantly begin enforcing the limitations on the switch—there is no restart or warm-up period.

Traffic Shaping Math

Here’s a concrete example showing how to calculate how long traffic will peak in a “best case” scenario:

• Let’s assume, for easy math, that you set the average bandwidth value to 1,000 Kbps.
• You also set the peak bandwidth to 2,000 Kbps, which is twice the value of the average bandwidth.
• Finally, you configure the burst size to 1,000 kilobytes (KB). Hint—don’t forget that there are 8 bits in a byte, which means that 1,000 KB is 8,000 Kb. Big “B” is for bytes and little “b” is for bits.

If the burst bonus is completely full, which would mean that it’s the full value of the burst size (8,000 Kb), then you could peak for 4 seconds:

8,000 Kb burst size / 2,000 Kbps peak bandwidth = 8 / 2 = 4 seconds

Credit for this info goes to "Networking for VMware Administrators"

 

Network Port Diagram - vSphere 5.x

This article provides a network diagram and reference sheet for vSphere 5.x. Here is the Link to download this PDF for Network Port Diagram - vSphere 5.x.

What is Co-Scheduling in VMware vSphere?

 

ESXi/ESX hosts with visibility to RDM LUNs being used by MSCS nodes with RDMs may take a long time to boot or during LUN rescan (1016106)

Symptoms

• ESXi/ESX 4.x and ESXi 5.x hosts take a long time to boot. This time depends on the number of RDMs that are attached to the ESXi/ESX host.
  
  Note: In a system with 10 RDMs used in an MSCS cluster with two nodes, a reboot of the ESXi/ESX host with the secondary node takes approximately 30 minutes. In a system with less RDMs, the reboot time is less. For example, if only three RDMs are used, the reboot time is approximately 10 minutes.
  
• ESXi intermittently shows an error message "Cannot synchronize host hostname. Operation Timed out. " on the Summary Tab and vSphere Client may not be able to start.
  
• The screen logging shows the boot waiting after this message:
  
  Loading module multiextent.
  
• The cluster is running virtual machines participating in an MSCS using shared RDMs and SCSI Reservations across hosts, and a virtual machine on another host is the active cluster node holding a SCSI Reservation.
  
• Delays appear at these steps:
  
  
  • Starting path claiming and SCSI device discovery
    
    In the VMkernel log of the rebooting ESXi host (check the log file depending on the version of ESXi), you see entries similar to:
    
    Sep 24 12:25:36 cs-tse-d54 vmkernel: 0:00:01:57.828 cpu0:4096)WARNING: ScsiCore: 1353: Power-on Reset occurred on naa.6006016045502500176a24d34fbbdf11
Sep 24 12:25:36 cs-tse-d54 vmkernel: 0:00:01:57.830 cpu0:4096)VMNIX: VmkDev: 2122: Added SCSI device vml0:3:0 (naa.6006016045502500166a24d34fbbdf11)
Sep 24 12:25:36 cs-tse-d54 vmkernel: 0:00:02:37.842 cpu3:4099)ScsiDeviceIO: 1672: Command 0x1a to device "naa.6006016045502500176a24d34fbbdf11" failed H:0x5 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0
    
  • Mounting the partition of the RDM LUNs
    
    In the VMkernel log of the rebooting ESXi/ESX host, you see entries similar to:
    
    Sep 24 12:25:37 cs-tse-d54 vmkernel: 0:00:08:58.811 cpu2:4098)WARNING: ScsiCore: 1353: Power-on Reset occurred on naa.600601604550250083489d914fbbdf11
Sep 24 12:25:37 cs-tse-d54 vmkernel: 0:00:08:58.814 cpu0:4096)VMNIX: VmkDev: 2122: Added SCSI device vml0:9:0 (naa.600601604550250082489d914fbbdf11)
Sep 24 12:25:37 cs-tse-d54 vmkernel: 0:00:09:38.855 cpu2:4098)ScsiDeviceIO: 1672: Command 0x1a to device "naa.600601604550250083489d914fbbdf11" failed H:0x5 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0.
Sep 24 12:25:37 cs-tse-d54 vmkernel: 0:00:09:38.855 cpu1:4111)ScsiDeviceIO: 4494: Could not detect setting of QErr for device naa.600601604550250083489d914fbbdf11. Error Failure.
Sep 24 12:25:37 cs-tse-d54 vmkernel: 0:00:10:08.945 cpu1:4111)WARNING: Partition: 801: Partition table read from device naa.600601604550250083489d914fbbdf11 failed: I/O error
Sep 24 12:25:37 cs-tse-d54 vmkernel: 0:00:10:08.945 cpu1:4111)ScsiDevice: 2200: Successfully registered device "naa.600601604550250083489d914fbbdf11" from plugin "NMP" of type 0

Oct 5 14:21:03 vmkernel: 47:02:52:19.382 cpu17:9624)WARNING: NMP: nmp_IsSupportedPResvCommand: Unsupported Persistent Reservation Command,service action 0 type 4
Oct 5 14:21:03 vmkernel: 47:02:52:19.383 cpu17:9624)WARNING: NMP: nmp_IsSupportedPResvCommand: Unsupported Persistent Reservation Command,service action 0 type 4
Oct 5 14:21:03 vmkernel: 47:02:52:19.383 cpu23:9621)WARNING: NMP: nmp_IsSupportedPResvCommand: Unsupported Persistent Reservation Command,service action 0 type 4
Oct 5 14:21:03 vmkernel: 47:02:52:19.383 cpu17:9624)WARNING: NMP: nmp_IsSupportedPResvCommand: Unsupported Persistent Reservation Command,service action 0 type 4
Oct 5 14:21:03 vmkernel: 47:02:52:19.383 cpu12:4108)WARNING: NMP: nmpUpdatePResvStateSuccess: Parameter List Length 54310000 for service action 0 is beyondthe supported value 18
Oct 5 14:21:03 vmkernel: 47:02:52:19.383 cpu12:4108)WARNING: NMP: nmpUpdatePResvStateSuccess: Parameter List Length 54310000 for service action 0 is beyondthe supported value 18
Oct 5 14:21:03 vmkernel: 47:02:52:19.383 cpu3:5733)WARNING: NMP: nmpUpdatePResvStateSuccess: Parameter List Length 54310000 for service action 0 is beyondthe supported value 18
Oct 5 14:21:03 vmkernel: 47:02:52:19.384 cpu12:9738)WARNING: NMP: nmpUpdatePResvStateSuccess: Parameter List Length 54310000 for service action 0 is beyondthe supported value 18
Oct 5 14:21:05 vmkernel: 47:02:52:21.383 cpu23:9621)WARNING: NMP: nmp_IsSupportedPResvCommand: Unsupported Persistent Reservation Command,service action 0 type 4
    
• If you configure the setting on an existing VMFS LUN, you may see these errors in the vmkernel.log file:
  
  YYYY-MM-DDT13:34:04.247Z cpu4:10169)WARNING: Partition: 1273: Device "naa.XXXXXXXXXXXXXXXXXXXxxxxxxxxxxxxx" with a VMFS partition is marked perennially reserved. This is not supported and may lead to data loss.YYYY-MM-DDT13:34:04.248Z cpu4:10169)WARNING: Partition: 1273: Device "naa.XXXXXXXXXXXXXXXXXXXxxxxxxxxxxxxx" with a VMFS partition is marked perennially reserved. This is not supported and may lead to data loss.YYYY-MM-DDT13:34:04.255Z cpu4:10169)WARNING: Partition: 1273: Device "naa.XXXXXXXXXXXXXXXXXXXxxxxxxxxxxxxx" with a VMFS partition is marked perennially reserved. This is not supported and may lead to data loss.

Purpose

This article describes a specific issue. If you experience all of the above symptoms, consult the sections below.

If you are experiencing only some of the symptoms, search the Knowledge Base for your symptoms or see:

• vSphere ESX/ESXi 4.x stops during start up at the step: multiextent loaded successfully (2003424)
• ESX machines hosting passive MSCS nodes report reservation conflicts during storage operations (1009287)

Source:-

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1016106&src=vmw_so_vex_ragga_1012

Troubleshooting migration compatibility error: Device is a SCSI controller engaged in bus-sharing (1003797)

Symptoms

• VMware vMotion migration fails.
• VMware vMotion migration fails to start.
• When performing a vMotion migration, you see a compatibility error similar to:
  
  Unable to migrate from <source server> to <destination server>: Virtual machine is configured to use a device that prevents migration: Device '<device>' is a SCSI controller engaged in bus-sharing.
  
• VMware Storage vMotion migration fails.
• VMware Storage vMotion migration fails to start.
• When performing a VMware Storage vMotion migration, you see a compatibility error similar to:
  
  Virtual machine is configured to use a device that prevents the operation: Device 'SCSI controller X' is a SCSI controller engaged in bus-sharing.
Device 'SCSI controller X' is a SCSI controller engaged in bus-sharing.

Cause

This issue occurs if you have a SCSI controller in use in a virtual machine, which has the bus-sharing mode changed to either virtual or physical. This configuration is used in virtual machines that are running Microsoft Cluster Services (MSCS).

Resolution

VMware does not permit vMotion or Storage vMotion migrations when a virtual machine is using SCSI bus-sharing. This includes Oracle RAC and MSCS virtual machines located on Storage DRS datastore clusters.

To work around this issue, VMware recommends migrating a virtual machine that is participating in SCSI bus-sharing by powering down the virtual machine that resulted in the error message and performing a cold migration to the new host. For more information on cold migrations, see Cold Migrations from the vSphere 5 documentation Center.

For more information on Oracle RAC, see Appendix A of the Oracle Databases on VMware RAC Deployment Guide.

For information on specific virtual machine configurations which can prevent migration of a virtual machine with vMotion, see Overview of migration compatibility error messages (1003684)

Source:-

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1003797&src=vmw_so_vex_ragga_1012

Change Interval Between DRS Invocations

VirtualCenter activates the DRS algorithm after a fixed interval (the default is five minutes) so DRS can make recommendations based on the past performance metrics of the virtual machines in the DRS cluster. Although we recommend keeping the default value of five minutes, you can change the frequency by adding the following options in the vpxd.cfg file, changing 300 to the desired number of seconds.
<config>
….
 <drm>
 <pollPeriodSec>
300 <!--# of seconds desired between 60 and 3600 -- >
 </pollPeriodSec>
 </drm>
…
</config>

ESXTOP Memory Performance Metrics and Threshold

Esxtop CPU performance metrics

Cores per Socket?

 

http://blogs.vmware.com/vsphere/2013/10/does-corespersocket-affect-performance.html?src=vmw_so_vex_ragga_1012

Understanding CPU Affinity



The Sky's the Limit - Gain the confidence that you need to certify!

Whether you’re in the Cloud, on the Desktop or in the Data Center…we’ve got what you need get VMware Certified! Custom options include discounts on an exam voucher, exam + practice test and a bundle that includes exam insurance!

http://pearsonvue.postclickmarketing.com/VCPExamInsurance

Exam	Exam + Prep	Exam Insurance
Includes: VCP exam voucher at 40% off!	Includes: VCP exam voucher at 40% off! Official Measure Up practice test	Includes: VCP exam voucher at 40% off! Official Measure Up practice test Official VCP Certification Guide Exam Insurance voucher: Gives you a retake opportunity — if you need it!*

$135.00 Save $90 off retail!	$174.50 Save over $139 off retail!	$319.50 Save over $269 off retail!

$135.00 Save $90 off retail!	$174.50 Save over $139 off retail!	$319.50 Save over $269 off retail!

$135.00 Save $90 off retail!	$174.50 Save over $139 off retail!	$319.50 Save over $269 off retail!

Earning VCP certification requires the completion of a qualifying class in addition to passing the appropriate exam. Review the complete certification requirements at vmware.com/certification.

Vouchers expires 6 months from date of purchase. Vouchers are non-refundable and non-returnable. Vouchers are intended for the recipients of this offer only, are for individual purchase, and are not for resale.

*The Exam Insurance voucher may be used twice if the purchaser fails in the first attempt and uses the bundled exam voucher in the second attempt. If the purchaser passes the exam on the first attempt using the bundled exam voucher, a second attempt will not be allowed.

Hot adding a virtual disk in ESXi 5.5 removes the multi-writer flag (2078540)

Symptoms

• When a virtual disk is added while the virtual machine is powered on, the pre-populated multi-writer flag is removed from the.vmx file.
• The hot add fails with file lock errors though the .vmx file contains scsix:x.sharing=Multi-Writer.
• Hot addition of disks to Oracle RAC fails with file lock errors.

Resolution

This is a known issue affecting ESXi 5.5.

 

Currently, there is no resolution.

To work around this issue, power off the virtual machine and then add virtual disks.

 

Source:-

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2078540&src=vmw_so_vex_ragga_1012