Note: altaro.com/vmware – has quite lot of great KBs.

Vmware Networking basics



Vmware powerCLI primer 1


Installing and enabling VMware Update Manager



Vsphere web client:


Vsphere Appliance:


VMware vSphere 5.1 Documentation center


Type 1 and Type 2 Hypervisor

Type 1: VMWare ESXi on Baremetal.

Type 2: VMWare workstation, Fusion etc on an OS.

VMWare Vsphere account management with Windows AD:



Vmware vSphere version comparison PDF from http://www.virten.net/vmware/vsphere-version-comparison/



Vm files


.nvram – Guest OS BIOS file.

.vmdk – Main file where the guest OS changes are written.

-flat.vmdk – A –flat.vmdk file that is one of two files that comprises the base disk. The flat disk contains the raw data for the base disk. This file does not appear as a separate file in the Datastore Browser.

.vmx – Guest OS configuration file. Old VMware uses .cfg extension.

.vmxf – This is a supplemental configuration file for virtual machines that are in a team.

.vswp – swap file

.vmem – paging file

Swapfiles operate by swapping entire processes from system memory into the swapfile. This immediately frees up memory for other applications to use.

Paging files function by moving “pages” of a program from system memory into the paging file. These pages are 4KB in size. The entire program does not get swapped wholesale into the paging file. While swapping occurs when there is heavy demand on the system memory, paging can occur anytime.


.log – Log file

Snapshot files:

When you take a snapshot, you capture the state of the virtual machine settings and the virtual disk. If you are taking a memory snapshot, you also capture the memory state of the virtual machine. These states are saved to files that reside with the virtual machine’s base files.

-delta.vmdk – A delta file is created once we create a snapshot. Once a snapshot is taken, any changes happen in the Guest OS is written in the delta file.

.vmsn – This file is created if we create a snapshot by checking take snapshot vm memory.

.vmsd – It is also known as Snapshot descriptor file. (contains Snapshot metadata)

  • Contains the virtual machine’s snapshot information and is the primary source of information for the Snapshot Manager.
  • This is a centralized file for storing information and metadata about snapshots.
  • This file contains line entries, which define the relationships between snapshots and between child disks for each snapshot.
  • Snapshot file is initially small around 16MB but it grows over time in 16MB increments.


If Vcenter goes down, What will work and what doesn’t ?



Enabling SSH on ESXi servers

To connect to ESXi host via command line, one need to open SSH session. Note that by default the SSH service is disabled.

To enable SSH open the vSphere client and Select your host > Configuration > Security Profile (under software) > Properties.




Duplicate SID


” You can prevent Windows from assigning new virtual machines or templates with the same Security IDs (SIDs) as the original virtual machine. Duplicate SIDs do not cause problems when the computers are part of a domain and only domain user accounts are used. However, if the computers are part of a Workgroup or local user accounts are used, duplicate SIDs can compromise file access controls. “


VMWare Master Glossary. source: http://www.vmware.com/pdf/master_glossary.pdf


Virtual Local Area Network

VLAN’s allow a network manager to logically segment a LAN into different broadcast domains. Since this is a logical segmentation and not a physical one, workstations do not have to be physically located together. Users on different floors of the same building, or even in different buildings can now belong to the same LAN.

More: http://www.cse.wustl.edu/~jain/cis788-97/ftp/virtual_lans/


Logical Unit Number. LUN is used to denote a logical disk. A logical disk is a collection of physical disks.

Each LUN identifies a specific logical unit, which may be a part of a hard disk drive, an entire hard disk or several hard disks in a storage device. So a LUN could reference an entire RAID set, a single disk or partition, or multiple hard disks or partitions. In any case, the logical unit is treated as if it is a single device and is identified by the LUN.


A Datastore is a block of storage that is formated with the VMFS file system. A Datastore can be comprised of one or more LUNs.

VMWare Datastore uses file system called Virtual Machine File System (VMFS).

Virtual representations of combinations of underlying physical storage resources in the datacenter. A
datastore is the storage location (for example, a physical disk, a RAID, or a SAN) for virtual machine files.



A platform that allows multiple operating systems to run on a host computer at the same time.



Is a container for multiple VMs.

vApp offers:

  • Container for multiple virtual machines
  • We can control how much CPU / RAM for the vApp folder
  • Resource controls for the VMs inside the container
  • Network configurations contained inside
  • Portability of the vApp such that everything can be contained and transferred to another virtual infrastructure
  • Entire vApps can be powered on, powered off, suspended, or shutdown
  • Entire vApps can be cloned




Snapshot creation

The snapshot captures the state and data of the virtual machine:

  • State refers ON, OFF or suspended state
  • Data or contents of the virtual machine’s memory. (Disks, Memory and Network interface cards)

When a snapshot creation is initiated, the guest cannot write changes to the vmdk file. A delta disk is created and all further changes in the guest are written into delta disk. If more than one snapshot exists, delta disks can represent the difference (or delta) between each snapshot. For example, a snapshot can be taken, and then the guest could write to every single block of the virtual disk, causing the delta disk to grow as large as the entire virtual disk.

Quiesce guest file system:

Quiescing indicates pausing or altering the state of running processes on a computer, particularly those that might modify information stored on disk during a backup, to guarantee a consistent and usable backup. Quiescing is not necessary for memory snapshots; it is used primarily for backups

-delta.vmdk – A delta file is created once we create a snapshot. Once a snapshot is taken, any changes happen in the Guest OS is written in the delta file.

Delta file will appear as for ex: vmname-00000001.vmdk

.vmsn – This file is created if we create a snapshot by checking take snapshot vm memory.


.vmsd – Database file which is created once we take snapshot and holds snapshot related information. If we have multiple snapshots for a VM this file give proper information to the VM.

Snapshot deletion

When a snapshot is deleted, all the data from the delta disk is written to the parent disk and merges with the base disk. This can involve a large amount of disk input and output. This may reduce the virtual machine performance until consolidation is complete.


Snapshot consolidation is useful when snapshot disks fail to compact after a Delete or Delete all operation or if the disk did not consolidate. This might happen, for example, if you delete a snapshot but its associated disk does not commit back to the base disk.

After consolidation, redundant disks are removed, which improves virtual machine performance and saves storage space.

“Consolidate” and “Delete All” snapshot basically do the same thing, they both merge all deltas into the base disk.


  • The amount of time it takes to commit or delete snapshots depends on how much data the guest operating system has written to the virtual disks since the last snapshot was taken. The required time is directly proportional to the amount of data (committed or deleted) and the amount of RAM allocated to the virtual machine.
  • We can take 32 snapshots for a VM.
  • Memory snapshots takes long time to create due to constant changes happen in the RAM.


VMware recommends the following best practices regarding snapshots:

  • Do not keep a single snapshot for more than 72 hours. While VMware supports up to 32 snapshots in a chain, try to limit chains to three snapshots.
  • Do not rely upon snapshots for I/O intensive VMs with rapid data changes, because significant data inconsistencies will occur when the VM is restored.



VM hardware version

  • We should be aware of the VM Hardware version / VM version while trying to migrate to other VM environment (V2V)
  • If a virtual machine is created on a VMware product that supports a given virtual hardware version and is then migrated to a VMware product that does not support this level of virtual hardware, it does not power on.
  • Virtual machines created by VMware products and versions located higher up in the chart cannot be powered on by products lower on the chart.
  • A VMware product can power on a virtual machine with a virtual hardware version that is lower than what it supports, but functionality may be lost. Lost functionality results in menu items related to virtual machine operations being grayed out and unavailable.






Note: The Option to upgrade VM hardware will only appear if your VM hardware version is low.

Steps to upgrade VM hardware version:



Vmware console help manual


A VMware snapshot is a copy of the virtual machine’s disk file (VMDK) at a given point in time. Snapshots provide a change log for the virtual disk and are used to restore a VM to a particular point in time when a failure or system error occurs. Snapshots alone do not provide backup.

Any data that was writable on a VM becomes read-only when the snapshot is taken. VMware administrators can take multiple snapshots of a VM to create multiple possible point-in-time restore points. When a VM reverts to a snapshot, current disk and memory states are deleted and the snapshot becomes the new parent snapshot for that VM. The snapshot file cannot exceed the size of the original disk file, and it requires some overhead disk space. Snapshots will grow rapidly with high disk-write activity volume. Most snapshots are deleted within an hour and VMware recommends deleting snapshots within 24 hours.

Snapshot file formats include *–delta.vmdk file, *.vmsd file and *.vmsn file. Administrators create snapshots in VMware vSphere’s Snapshot Manager or with the vmware-cmd command line utility. Deleting, or committing, snapshots merges all of the delta files into the VMDK. If delta files remain in the VM’s directory after deletion, the snapshot did not delete properly.


VM – VMotion, DRS and HA difference

Vmotion DRS HA

High Availability (HA)

HA does not use vMotion. HA will restart the vm on the remaining cluster members. The HA master keeps a list of VMs and their power state. If the vCenter server goes down HA will still operate.

HA uses a failure detection interval that is set by default to 15 seconds (15000 milliseconds); you can modify this interval by using an advanced HA setting


DRS can move running VMs between hosts only if they are in same storage.

DRS requires vCenter to function. If vCenter is down, DRS will not work.


vMotion is the migration of live or powered OFF VMs between ESXi or between Datastore. vMotion between Datastore is called Storage vMotion.

vMotion requires vCenter server. If vCenter is down, vMotion will not work.

Fault Tolerance (FT) vs High Availability (HA)

Fault Tolerance eliminates downtime even when hardware goes down by keeping a copy of a critical virtual machine (VM) on another host. For VMs protected by High Availability, if the host running the VM fails, then the VM will get restarted on another server; if the VM fails because of an operating system crash, HA will restart the VM on the same host.

Above from Tech target.

vMotion, Storage vMotion, HA (High Availability) and DRS

  • vMotion migrates a running VM from one ESXi host to another ESXi.
  • Storage vMotion is migration of VM from one datastore to another datastore.
  • High Availability (HA) restarts a VM on another host if the original host disappears for whatever reason. HA does not use VMotion.
  • Dynamic Resource Scheduler (DRS) migrates VM from one ESXi host to another much resource free ESXi host if the current ESXi host experience high resource utilization such as RAM and CPU.

Important: vCenter server is needed for both vMotion and DRS. HA continue to work even if vCenter is down.

DRS (Dynamic Resource Scheduler)

Cluster DRS Settings in VMware.



The DRS cluster will make recommendations to an administrator, but no automated actions will be taken.  The administrator must manually carry out any recommendations.  This is a good setting if you just want to see what impact DRS might have on your environment.

Partially automated

Partially automated DRS clusters are pretty common.  Clusters configured for partial automation will automatically place new virtual machines on existing hosts based on performance and resource criteria.  However, after the initial placement event, which may result in recommendations to move other workloads to accommodate the new virtual machine, DRS operates the same way that it does when Manual DRS is use.

Fully automated

Many administrators are loathe to allow DRS to simply work to its will through the fully automated option.  When this option is selected, DRS will provide initial placement services as described earlier, but it will also move workloads around the cluster without administrator intervention if there is a chance to better balance workloads running inside the cluster.  The administrator is able to specify the level of sensitivity using what is called the Migration Threshold.  You can configure DRS to initiate a migration when there is any associated performance improvement or you can choose to be a bit more conservative and wait until DRS finds that an operation will have a significant positive impact.


vMotion background:

Whenever the ESXi resources are utilized above the threshold value, vCenter will migrate the VMs from the ESXi to another ESXi which has much resource to spare.

  1. vCenter first check whether source VM can be operated on the chosen destination ESXi server.
  2. Then CPU and RAM resources are reserved in destination ESXi server.
  3. In the source ESXi server,  a system memory checkpoint is created. This means all changes to the source VM are written to an extra memory area.
  4. The contents of the checkpoint are transferred to the target VM.
  5. This process is repeated until only the smallest change sets remain in the target VM’s memory.
  6. The CPU of the source VM is stopped.
  7. The vMotion process is ended and a reverse ARP packet is sent to the physical switch (important: Notify Switches must be activated in the properties of the virtual switch).
  8. Hard disk access is taken over by the target ESX.
  9. VM process on the source ESX is deleted.


Disk provisioning types

Thick Provision Lazy Zeroed
Creates a virtual disk in a default thick format. Space required for the virtual disk is allocated when the virtual disk is created. Data remaining on the physical device is not erased during creation, but is zeroed out on demand at a later time on first write from the virtual machine.
Using the default flat virtual disk format does not zero out or eliminate the possibility of recovering deleted files or restoring old data that might be present on this allocated space. You cannot convert a flat disk to a thin disk.

The phrase “zeroed out” means that blocks on the physical storage device are formatted with zeros to overwrite any older data.

In simple English: Allocated area is fully reserved for the VM and the data is not wiped out.

Usage: This can be used in production environment.

Advantage: This is fast compared to Eager Zeroed and can be used in production.

Thick Provision Eager Zeroed
A type of thick virtual disk that supports clustering features such as Fault Tolerance. Space required for the virtual disk is allocated at creation time. In contrast to the flat format, the data remaining on the physical device is zeroed out when the virtual disk is created. It might take much longer to create disks in this format than to create other types of disks.

The phrase “zeroed out” means that blocks on the physical storage device are formatted with zeros to overwrite any older data.

In simple English: Allocated area is fully reserved for the VM and the data is wiped out prior to the VM installation.

Usage: The difference in performance between a thick disk and an Eager Zeroed thick disk is very small, but some applications, such as Microsoft Cluster Services and VMware Fault Tolerance, still require eager zeroed thick provisioned disks.

Disadvantage: This will more time to format depending on the size of the disk you provision for a VM. Larger the size, more time it will take to format and then only we can install OS on it.

Thin provision
Use this format to save storage space. For the thin disk, you provision as much datastore space as the disk would require based on the value that you enter for the disk size. However, the thin disk starts small and at first, uses only as much datastore space as the disk needs for its initial operations.

Usage: Thin provision is used in less critical environment like Development.


vSphere switch


vSphere Distributed Switch



VM clone operation background

Once you initiate a VM clone, a snapshot of the VM is created

Vcenter register the VM to destination host

Next the .VMDK files are copied to destination



One of the practical use of V2V is to reclaim unused datastore space.

Virtual machines usually takes space over time and unused space (if files are not deleted) are not properly returned back to datastore (VM with thin provisioned drives). we have to perform V2V to get the space back. We need VMware Converter to perform V2V operation.

Migration is the process of moving a virtual machine from one host or storage location to another. Copying a virtual machine creates a new virtual machine. It is not a form of migration.

In vCenter Server, you have the following migration options:

Cold Migration

Moving a powered-off virtual machine to a new host or relocating virtual machine configuration and disk files to new storage locations. Cold migration can be used to migrate virtual machines from one datacenter to another.

Migrating a Suspended Virtual Machine

Moving a suspended virtual machine to a new host or relocating configuration and disk files to new storage locations. You can migrate suspended virtual machines from one datacenter to another.

Migration with vMotion

Moving a powered-on virtual machine to a new host. Migration with vMotion allows you to move a virtual machine to a new host without any interruption in the availability of the virtual machine. Migration with vMotion cannot be used to move virtual machines from one datacenter to another.

Migration with Storage vMotion

Moving the virtual disks or configuration file of a powered-on virtual machine to a new datastore. Migration with Storage vMotion allows you to move a virtual machine’s storage without any interruption in the availability of the virtual machine.

Both migration of a suspended virtual machine and migration with vMotion are sometimes referred to as “hot migration”, because they allow migration of a virtual machine without powering it off. Migration with vMotion is sometimes referred to as “live migration”.


ESXi is the master clock for the whole infrastructure. VMs sync their clock with ESXi server using VMware tools installed on them. Hence VMware tool is very important. ESXi uses NTP servers Internet service providers or online sources such as pool.ntp.org. We have to configure ESXi to use any of these Time source.
Vcenter client -> ESXi -> Configuration -> Software -> Time configuration


  • To run Vcenter client in the browser, we first need to install vsphere client integration plug-in.
  • Vcenter server appliance (vCSA) 6 needs a whole 20GB space to install. So the free space should be more than 20GB.


VMware Affinity rules

From techtarget

Affinity rules and anti-affinity rules tell the vSphere hypervisor platform to keep virtual entities together or separated.

If two virtual machines communicate frequently and should share a host, the VMware admin can create a VM-VM affinity rule to keep them together. Conversely, if two resource-hungry VMs would tax a host, an anti-affinity rule will keep those VMs from sharing a host.

How to setup Affinity rules ?

VMware Vcenter -> Right click the cluster -> Edit settings





VMware NICs:

Unlike physical Network Interface Cards Vmware VMs are given virtual NICs. Some of the NIC’s mimic physical NIC’s properties and some don’t. VMs speed depends on the CPU, RAM, HDD and as well the type of (Virtual) NIC used. If you choose a virtual NIC that has higher throughput (10Gb), you also need to have adequate CPU that can handle that much speed. So more CPU are needed for higher NIC speed in general.

E1000 – 1GB Ethernet Speed. E1000 virtual NIC is a software emulation of a 1 GB network card. The hardware card is a long existing, commonly available Intel based device and most operating systems include built in support. Because of that there is no special driver required or any exceptional effort required to make it operate in a virtual environment. The problem is that the virtual device is just as described, a piece of software acting as if it was hardware. That can lead to performance issues as the host’s CPU is required to take care of the processing normally done on a separate ASIC (Application-Specific Integrated Circuit)


  1. VMware tool is not needed.
  2. Compatible with almost all operating systems.


  1. Slow than VMXNET
  2. Uses more CPU cycles
  3. Sometimes VMs could encounter Network drops.



VMXNET3 – 10GB Ethernet Speed.

VMXNET3 virtual NIC is a completely virtualized 10 GB NIC.

With this device the device drivers and network processing are integrated with the ESXi hypervisor. That means there is no additional processing required to emulate a hardware device and network performance is much better. There is no native VMXNET device driver in some operating systems such as Windows 2008 R2 and RedHat/CentOS 5 so VMware Tools is required to obtain the driver. VMware Tools is, however, highly recommended in any case so that should not be an issue.


  1. VMs with VMXNET3 would use Low CPU with high throughput performance.


  1. Needs VMware tools installed on the VM.

There is a huge performance difference between E1000 and VMXNET3.


Above Image from (& for detailed information):

VMware vSphere 5.5 Virtual Network Adapter Performance

Part of the text regarding VMXNET3 and E1000 were copied from:



VMware issues:

VM Migration error between hosts:




Different memory in VMware vSphere

Private: Physical memory on the chip.

Shared: Memory that is provided by transparent page sharing.

Swapped: Memory that has been host swapped – if this number is high it may result in performance issues.

Compressed: Memory that has been zipped – compression provides better performance than swapping.

Ballooned: Memory reclaimed from the guest by the ballooning driver.

Unaccessed: Memory that is untouched by the virtual machine.

Active: Approximate amount of memory being actively used by the virtual machine (read and writes)

Memory Ballooning

VMware ballooning is a memory reclamation technique used when and ESXi host is running low on memory.

Virtual memory ballooning is a computer memory reclamation technique used by a hypervisor to allow the ESXi host to retrieve unused memory from certain guest virtual machines (VMs) and share it with others. Memory ballooning allows the total amount of RAM required by guest VMs to exceed the amount of physical RAM available on the host. When the host system runs low on physical RAM resources, memory ballooning allocates it selectively to VMs.

ESXi don’t have direct access to reclaim memory from the VM. This is done through “Balloon driver”. Balloon driver is installed on a VM if we install VMWare tools. Balloon driver release the unused memory to ESXi host. Which is then shared to other VMs.


Physical network is splitted into vNIC, Uplink, vSwitch and Port Groups in VMware vSphere.

What is vNIC, Uplink, vSwitch and Port Groups ?


VMware vSphere 6 limitations

Virtual SCSI per VM: 4

1024 VM per host

4096 Virtual CPU per host

Max RAM per host is 6TB

Max RAM per VM is 4TB

Some OEM supports up to 12TB


Cluster Maximums / limitations

Hosts per cluster: 64

VM per cluster: 8000

VM per host: 1024


Vcenter limitations

ESXi Hosts per Vcenter servers: 1000

64 Esxi hosts per cluster

Powered ON VM per vCener server: 10000

Registered VM per vCenter server: 15000

Concurrent vSphere web client connections to vCenter server: 180

Number of host per datacenter: 500

4-8 vMotions (4 for 1GB Network and 8 for 10GB network)

8 Storage vMotions

If you have lot of ESXi host then we can have upto 10 linked vCenter servers. We can manage all these with single interface.


ESXi limitations in VMware vSphere 6

480 Logical CPU


1024 VM per ESXi host

64TB LUN or RDM (LUN – Logical Unit Number, RDM – Raw Disk Mapping)

4096 Virtual Switch ports (4088 standard switch ports)

60000 ports per distribuited switch (16 Virtual Distributed Switch per host)



Movement of VMs between ESXi hosts

Storage vMotion

Movement of VMs between Datastores

cold migration – migrating VM that is powered off

Hot migration – migrating VM that is powered on




Platform Services Controller

Provides Single-Sign on (such as AD integrated services), licensing and certificate management.

Ways to access vSphere environment

Desktop client – Use to manage single ESXi host

Web client – Useful to manage multiple ESXi host. Newest – has many features


VMA – VMware management appliance

SDK – Software development Kit


How to restart Management agent in ESXi ?

Method 1: Direct Console User Interface (DCUI):

  1. Connect to the console of your ESXi host.
  2. Press F2 to customize the system.
  3. Log in as root.
  4. Use the Up/Down arrows to navigate to Troubleshooting Options > Restart Management Agents.
  5. Press Enter.
  6. Press F11 to restart the services.
  7. When the service restarts, press Enter.
  8. Press Esc to log out.


Method 2:Restart Management agents in ESXi Using ESXi Shell or Secure Shell (SSH):

  1. Log in to ESXi Shell or SSH as root.
  2. For Enabling ESXi Shell or SSH, see Using ESXi Shell in ESXi 5.x and 6.x (2004746).
  3. Restart the ESXi host daemon and vCenter Agent services using these commands:
  • /etc/init.d/hostd restart
  • /etc/init.d/vpxa restart

When you want to restart the ESXi management agent ?

  • Cannot connect directly to the ESXi host or manage under vCenter server.
  • vCenter Server displays the error:Virtual machine creation may fail because agent is unable to retrieve VM creation options from the host

4. Note: In ESXi 4.x, run this command to restart the vpxa agent:

service vmware-vpxa restart


VMware Tools

VMware Tools is a suite of utilities that enhances the performance of the virtual machines guest operating system and improves management of the virtual machine. Without VMware Tools installed in your guest operating system, guest performance lacks important functionality. Installing VMware Tools eliminates or improves these issues:

  • Unable to reclaim unused memory from VM to ESXi host (Memory Ballooning)
  • Network performance (VMXNET3, Paravirtualized Devices)
  • Guest OS performance
  • Ability to shut down or restart the guest OS from the vSphere client
  • Low video resolution, Inadequate color depth
  • Incorrect display of network speed
  • Restricted movement of the mouse
  • Inability to copy and paste and drag-and-drop files
  • Missing sound
  • Provides the ability to take quiesced snapshots of the guest OS
  • Synchronizes the time in the guest operating system with the time on the host


Swap vs Page file

Swapfiles operate by swapping entire processes from system memory into the swapfile. This immediately frees up memory for other applications to use.

In contrast, paging files function by moving “pages” of a program from system memory into the paging file. These pages are 4KB in size. The entire program does not get swapped wholesale into the paging file.

While swapping occurs when there is heavy demand on the system memory, paging can occur preemptively. This means that the operating system can page out parts of a program when it is minimized or left idle for some time. The memory used by the paged-out portions are not immediately released for use by other applications. Instead, they are kept on standby.


Standalone ESXi patching


How to install patches or updates for your VMware ESX host using Update Manager

Vmware Baseline

A baseline is simply a collection of one or more patches, upgrades or extensions. Different baselines may be combined in what are called baseline groups. Some of the baselines are:

  • Critical Host Patches
  • Non-Critical Host Patches
  • VMware Tools Upgrade to Match Host
  • VM Hardware Upgrade to Match Host
  • VA Upgrade to Latest


VMWARE Vsphere client download links:

vSphere Client Download – Direct Links



Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.