Setting EVC Level on a Cluster with a VCSA and no shared storage


A second host has been added to a one-node vSphere cluster which has no shared storage and EVC cannot be enabled. Details as follows:

  • A vSphere environment consists of an existing cluster containing a single host running ESXi 6. On this host is (amongst other things) the vCenter Server Appliance VM.
  • A second ESXi6 host is added and joined it to the vCenter. This host has older hardware with an older EVC level (rather than the usual situation where the newer box has newer hardware).
  • EVC cannot be enabled on the cluster because the old host does not have the capabilities of the new host. The old host has VMs running which may be using the enhanced capabilities of the higher EVC level, one of these is the vCSA.

    “The host cannot be admitted to the cluster’s current Enhanced vMotion Compatibility mode. Powered-on or suspended virtual machines on the host may be using CPU features hidden by that mode.”

    Therefore the EVC level on the existing cluster cannot be lowered by powering off all the VMs because it cannot be changed without using vCenter.

  • The vCSA cannot be vMotioned to the old host because this requires EVC to be enabled.

    “The virtual machine requires hardware features that are unsupported or disabled on the target host”

  • There is no Shared Storage. VMs are stored on local datastores. The Knowledgebase article “How to enable EVC in vCenter Server (1013111)” has a solution but this doesn’t (as far as I can tell) work without shared storage. Without storage visible from both hosts the VM cannot be disabled in one host and brought back up in a second which is in a new cluster.

Possible Solutions

The problem boils down to needing to cold migrate a VM between ESXi hosts without using shared storage or vCenter. The following solutions came to mind.

  1. Create some shared storage (possibly using an NFS share on a laptop temporarily) and follow the procedure shown in KB1013111
  2. Power down the vCSA, use the host web client to move the files from the datastore to a laptop, then back up to the less-able host. Power it on and set EVC on the cluster.
  3. Dump the vCSA and setup a replacement instance on the less-able host. Reconfigure everything. The “Start Again” option.
  4. Use SCP to do a host-host local datastore transfer of the powered down and unregistered vCSA Virtual Machine files

My Chosen Solution

This is what I tried and tested and it worked in my environment, along with step-by-step instructions if anyone else finds themselves in this predicament (usual disclaimer applies).

Option 4:  Use SCP to do a host-host local datastore transfer of the powered down and unregistered vCSA Virtual Machine files.

Rough steps – “First Host” is the existing box with newer hardware, “Second Host” is the box with older hardware being added:

  1. Using vCSA setup a new cluster containing just the second host (the one with older hardware) and turn on EVC appropriately.
  2. Enable Secure Shell access on both hosts
  3. Shutdown all VMs including the vCSA on the first host
  4. Remove vCSA from inventory (Unregister) using web client on first host
  5. SSH into second host
  6. Enable SCP through the firewall with
    esxcli network firewall ruleset set -e true -r sshClient
    -thanks for that snippet
  7. Use SCP to copy VM files from local datastore on first host to local datastore on second host.
    For example- in SSH session on second host, something like this:
    mkdir /vmfs/volumes/datastore1/LABVC1
    scp ‘root@*.*’ /vmfs/volumes/datastore1/LABVC1/
  8. Connect to web client on second host and register the copy of the VMX file to inventory
  9. Turn on the vCSA VM. When prompted say “I moved it”
  10. Wait for vCSA to spin up then move the first host into the cluster with the second host.
  11. Tidy up: -Remember to go back and delete the old copy of the VCSA from the datastore on the first host and disable SSH on both hosts if it’s not required. The vCSA can be Storage-vMotioned to rethin disks if they inflated during the SCP operation.


VMworld US 2016 Day 1 Keynote

Yesterday saw the 2016 VMworld US formally kick off with the Keynote address. There are 23,000 attendees at the Las Vegas event, and I was amongst the many armchair supporters watching the live online broadcast.

imageThe keynote opened with drumming, poetry, and a lightshow, but it wasn’t long before Pat Gelsinger took the stage for his 5th VMworld as CEO of VMware. His presentation involved plenty of studying the past to predicting the future- “be_Tomorrow” is the tagline for the event.

Comparisons were made between the industrial revolution and todays’ digital revolution, and in more recent history we saw quotes from Eric Schmidt- the birth of the cloud- in 2006, and Ragu Raghuram on the SDDC in 2011. Some stats on the growth of the cloud vs traditional IT since 2006 were shown, workloads on a private/public cloud growing from 2% to 27% of the total in the last 10 years.

VMware’s predictions of “tomorrow” are interesting (and possibly a little too precise). Gelsinger told us to the minute when his team of experts have predicted that 50% of all workloads will be in the cloud.

Next came the real meat of the Keynote- the introduction of VMware’s latest cross cloud architecture- “Any Cloud, Any Device” using the upcoming products “Cloud Foundation” and “Cross Cloud Services”. Last year’s event had the tagline “Ready for Any”, perhaps this is finally being realised?

We heard examples from partner IBM and customer Marriot, before Guido Appenzeller took to the stage to dive deeper into the products.

The new Cloud products will allow IT to move workloads between an on-premises vCenter-based SDDC, AWS, Google Cloud, and Microsoft Azure. This opens up this possibility of bursting into various public cloud offerings when your workload demands it, but also taking that AWS-developed application and siting it appropriately across one or more locations- providing that security and resilience which IT is tasked with providing to the business.


The key here is security- and it’s doubtless no coincidence that the first guest speaker in this segment was from the banking industry- it’s not just the ability to vMotion between different providers “mega-clouds” but also to implement the rest of the SDDC architecture. We saw a demo using NSX to overlay network virtualisation, microsegmentation, and encryption to an AWS application.

One thing that struck me was the way vCloud Air was hardly mentioned. VMware has clearly shifted direction here and is not trying to compete with the AWS/Google Cloud/Azure marketplace but looking more to work with it. This also helps from an IT point of view- we don’t have to turn round to developers and say “Don’t use AWS, learn how to use vCloud Air instead”

imageThe Keynote was rounded off with Pat Gelsinger talking with Michael Dell. The Dell/EMC takeover deal was announced back at VMworld Europe 2015 in October, and currently looks like it’s almost ready to actually happen! As expected, Michael was quick to reassure everyone that it was business as usual for VMware and this Dell ownership wasn’t going to mean the end of the line for those partners who might be Dell competitors. He highlighted how the open partner ecosystem of VMware is critical to it’s continued success and mentioned the new Cloud Foundations and how “only an ecosystem of this size and power could pull this off.”

Both Dell and VMware seem keen to make private clouds easy to deploy, but are also looking to heavily push the public cloud alongside it. The next few years could be interesting, let’s have a look a cloud adoption at about four in the afternoon on June 29th 2021 and see what happened.

Full video of the session can be found on the VMworld website or below.

vExpert 2016

I’ve worn my thesaurus out looking for synonyms of “astounded”, “honoured”, and “proud” for this post, but let’s just say I’m “very happy” find myself amongst the latest 123 individuals to have been awarded the VMware vExpert title following the announcement at the end of last week. Thank you to the VMware community team for this recognition.

What is it?


If you’re new to the VMware community, vExpert is VMware’s global evangelism and advocacy program and the award is given for “significant contributions to the community and a willingness to share .. expertise with others” rather than being a traditional technical certification like the VCA/ VCP/ VCAP/ VCDX etc. Announcements are made twice annually and the award is valid for one year.

What next?

From me, expect more of the same- but hopefully bigger. One of the advantages of the vExpert program is it plugs recipients into non-production software licensing deals, betas, and new sources of information- both from VMware themselves but also through the surrounding partner ecosystem. I’m hoping to make best use of all of those resources to continue to learn and thus improve this blog, my tweets, and hopefully produce some more videos and in-person talks in the community.

VMworld Europe

Sessions to look out for at #VMWorld Europe 2016

imageWith 60 days to go to the Barcelona event, the Content Catalog for VMworld Europe 2016 is filling up and the Schedule Builder will be coming online later this month. Here’s a look at some of the sessions already listed that I’m hoping to get onto my timetable. There’s lots to choose from, and this is by no means a complete or final list for my schedule- just five highlights I’ve spotted in the list so far.

1- The Power Hour: Deep Dive, DevOps, and New Features of PowerCLI [INF8092] with Luc Dekens and Alan Renouf

I saw Alan and Luc in action last year where they were “Automating Everything with PowerCLI Deep Dive” (YouTube link) and it was a thoroughly useful session. This is most definitely in my to-do list for 2016.

2- VMware NSX-Deep Dive [NET9152] with Jacob Rapp

There’s a number of sessions on NSX that I’m interested in, but from experience I’m planning on attending the heavier sessions and catching up on the more intro/overview sessions online. This makes my schedule workable, and personally I believe I learn the most this way. This one is currently scheduled for the Tuesday morning straight after the Keynote, so it’s a case of diving in at the deep end!

3- How to Manage Health, Performance, and Capacity of Your Virtualized Data Center Using vSphere with Operations Management [INF8275] with Kyle Ruddy and Himanshu Singh

vSphere with Operations Management is the core of the virtualised infrastructure, and I try hard not to overlook that in favour of the exciting things happening in the surrounding ecosystem. This talk promises demonstrations “of the latest features and capabilities via various scenarios and use cases”, which sounds promising.

4- What’s New with vSphere [INF8375]

Referred to in the Content Catalog as “A VMworld staple”, this session covers everything that’s new to vSphere in 2016 along with future directions the platform is heading in.  It was filled last year, so one to book in early this year to get a seat. There’s a series of “What’s New” breakout sessions on throughout the week covering vCenter Server, vSphere, Storage, Horizon, vRealize and more.

5- Troubleshooting vSphere 6 Made Easy: Expert Talk [INF9205] with Ragavendra Kumar and Abhilash Kunhappan

Currently scheduled for last thing on the last day (must make sure I can catch the plane home afterwards!)  this session promises insights into the “black art” of vSphere troubleshooting. It will cover the tools and log files available and examples of how to use them.

Not yet signed up for Barcelona in October? You can register here.

Goodbye VMTurbo, Hello turbonomic!

Big news today from workload monitoring and management vendor VMTurbo – The name has gone and the brand is now called “turbonomic”.

turbonomic logo

The name change emphasises several facets of the company’s product- not least that it’s not limited to Virtual Machines, or VMware. At a community briefing by Eric Wright last week the compatibilities with Microsoft Hyper-V, XenServer, RHEL, plus container and cloud platforms were highlighted just for starters.

What’s in a name?
There’s obviously been a lot of thought put into the name change- anyone who has ever worked for a company that has renamed or rebranded will attest to the fact that it’s not something you can do on a whim. So what’s turbonomic about?
turbo– This has been retained (along with the green circle) from the VMTurbo era. The real-time performance of the product is still key to the organisation.
nomic– This part is twofold, firstly the Autonomic Control that their product offers – having self-managing operations where applications collect resources from the platform’s engine in realtime. Secondly the foundation of economic principles are core to this process of matching resources to applications. Together these controls and principles allow QoS management of resources to enable services to meet defined SLAs within the environment- guaranteeing application performance.

Any infrastructure- be it IaaS, containers, or applications does three things- it schedules resources, monitors services, and orchestrates configurations and provisioning. This is where turbonomic is pitched; the platform works across the infrastructure to ensure that applications are configured with the necessary resources and then are monitored to ensure that if the requirements rise or fall over time that the appropriate capacity is provided to the workload in realtime. As a result of this automation we’re told the need for the sysadmin to spend their day balancing services should be removed. turbonomic tell us this leads to happy sysadmins, and that’s a good thing in my book.

A Happy SysAdmin (image via GIPHY)

Interested? If so, have a look at

#VMworld Europe 2016

VMworld is coming back to Europe this autumn, with thousands of delegates attending hundreds of sessions to keep up-to-date in virtualisation technologies. The annual conference has been running since 2004, and since 2008 there’s been a European event alongside the US one. My first VMworld was last year and this year I’m privileged to have been offered an official Bloggers pass to the 2016 event in Barcelona. Many thanks to Corey Romero and the VMware community team for this opportunity.

In my opinion VMworld is an awesome event, showcasing the entire landscape of VMware products from End User Computing through to the Software Defined Datacentre. The week not only profiles the latest and greatest offerings from VMware themselves but also from the other vendors that make up the surrounding ecosystem; Solutions in storage, server hardware, networking, HCI, management and monitoring, End User Computing devices and more fill the Solutions Exchange and some of the sessions.

Hands on Labs at VMworld Europe 2015

Hands on Labs at VMworld Europe 2015. Register Now for 2016

Outside of the formal sessions (primarily lecture style, group discussion, and hands-on-labs) there’s the opportunity to meet and network with fellow IT Professionals from all around the world. You can take the opportunity to discuss your own challenges and solutions with such a wide variety of experts from all manner of countries and industries.

VMworld 2015 Step Count

VMworld 2015 Step Count -Comfy shoes are a must.

So go on, register now then check out my 5 Conference Tips from last year- and I’m not joking about the need for comfortable shoes, as my Fitbit step count from last year shows. If you do make it out to Barcelona this autumn, look me up and say hi!

VMworld 2016

VMworld 2016 Disclaimer: Whilst this blog always strives to be unbiased in any opinions and uninfluenced by an offer of a free T-shirt, readers should be aware that VMware have provided me with a conference pass for this event and also that during the event VMware and their partners may be providing SWAG or entertainment to promote their products. Neither VMware nor their partner companies review what is written here prior to publication, but factual corrections post-release are welcome and will be noted if included. The general disclaimer for this site can be found here but I should reiterate that this is my personal site and not affiliated with my employer in any way.

VMware Certified Associate 6 – Data Center Virtualization Resources

This post contains a list of resources for anyone planning on taking the VMware VCA6-DCV Qualification. This certification is obtained by passing a single, online, exam and unlike the more advanced VMware awards requires no mandatory training course.

London VMUG June 2016

VMUG LogoA rainy day in June saw me back in London for the latest VMware User Group event. For anyone who works with VMware products but has never been to a VMUG I cannot stress enough how great they are. VMUG is a community focussed and community run event filled with a bunch of really friendly, knowledgeable IT Professionals, and- thanks to vendor sponsorship- it’s free of charge so there’s little or no impact on your training/conference/personal budget or wallet.

Simplivity HCI for vSphere
This London event kicked off with a talk from one of the sponsors- Simplivity– discussing their Hyper-converged Infrastructure products for vSphere environments. This HCI solution is available as a turnkey appliance, or with third party hardware, such as Cisco or Lenovo.
The presentation and demo mentioned how crucial deduplication and compression are to this technology. Aside from saving physical disk space, there’s also the added benefits of backup and restore becoming faster and less costly from a network perspective.
For example when restoring a backup the software starts by sending the metadata- block hashes of the deduped backup. The target system can then determine which blocks it already has and which it doesn’t and only those new blocks need to be transmitted across the LAN/WAN to the system where the restore is happening. Additionally, these are the only blocks that need writing to disk so the amount of time spent on disk activity is also cut.

Simplivity Demo

Stuart Gilks of Simplivity demonstrating the power of dedupe and compression in HCI.

Server Hugger or “To the Cloud”
Following this we had a more light-hearted session. As it was the day of the Brexit vote, two teams of speakers attempted to persuade the attendees to be a Server-Hugger and vote to remain in the datacentre, or take the alternate path and vote to Leave for the Cloud. The general consensus was that Hybrid was probably the best option, but as that wasn’t offered it was down to the two choices and the Server-Huggers won the day.
Server Hugger
VMware EUC Update
After a quick break the meeting split into two tracks. I opted for the VMware EUC update with Howard Bliss covering developments in WorkspaceONE, Horizon, and TrustPoint.
WorkspaceOne is a great idea- taking the EUC features of Airwatch, Identity Management, and Horizon and packaging them up in one platform. From the end-user point of view this looks slick, they download an app for their device from the regular appstore, sign in with their corporate email address, a profile is installed over the net and they are all set.
However I’m not sure it’s a fit for me from a Higher-Education IT point of view as the licensing is per-seat not based on concurrent users. Licensing this for tens of thousands of students a year could get expensive very quickly.
Horizon 7 went GA in March, and is “Hybrid-Cloud Ready”- offering the ability to manage cloud and on-prem VDI environments through a single pane of glass. Just-in-time Delivery means that 2000 desktops can be deployed in under 20 minutes, making the pre-loading of VMs for the start of a working day no longer necessary. This is helped along by February’s release of AppVolumes 3- apparently a preview of even better things to come later this year.
The Horizon Client has been improved, offering offloading of encoding from the CPU to an NVidia card and Blast extending to Linux to allow software encoding of Linux desktops. There’s also better integration with the OSX Keychain on the Mac client, and Aero window snapping, credential passthrough, and keyboard locale improvements in the Windows version.
The final piece of this EUC triumvirate is TrustPoint- The Image Service (Mirage) and the Security Platform (Tanium). The Image Service provides some useful abilities for those looking to do Win7 to Windows 10 migrations- including the ability to do an in-place upgrade from x86 to x64 platforms if you have any 32-bit installations that you want to lose. Amongst the benefits of the Security Platform is the ability to pick up and report on unmanaged endpoints appearing on the LAN within 15 seconds of them connecting.

SysTrack EUC Analytics
After lunch I chose the Lakeside Software session on using SysTrack in a Horizon Environment. In a bid to avoid PowerPoint poisoning they treated us to a short marketing video (see below) and then it was straight into a demo of the product in action.

I’ve used other tools in this space before, but it was interesting to see what SysTrack offers as an End-User analytics tool. There’s lots of information there and the ability to answer many questions- how many users are having a poor experience? (and is this number going down over time?) Are users in a particular department or location suffering? What resources do we need to migrate traditional desktops to VDI?
Data is collected by an agent that sits on the endpoint which works alongside browser plugins and VMware API calls for the monitoring of services hosted by vCenter. This agent data is pushed to a collector once a day to provide this trend analytics ability but there’s also realtime monitoring of the environment- spotting the apps causing a current peak in network traffic for example.

Lakeside Software Demo

Live Demo of SysTrack from Lakeside Software

Boldly Going Where No DC has gone before
The formal part of my day concluded with a session titled “Extreme VMware Datacentres”, a chance for some of the more seasoned VMUG members to discuss their best war stories. We heard about some of the “awesome” places that VMware datacentres have ended up – in everywhere from dusty warehouses to nuclear submarines (you put a datacenter at each end so you can make end-to-end failover jokes!)or dark sites where nothing goes in or out, especially not a USB stick with an installation script on. Each location provides it’s own set of challenges, and requires the IT Pro to come up with new and unique solutions. This was thoroughly entertaining content from some community speakers.

No VMUG post would be complete without a personal thank you to the sponsors- Simplivity, Tintri and Lakeside Software. Further thanks to 10-Zig, PernixData, and Nutanix for sponsoring the after-event vBeers. Finally, thanks again to the London VMUG committee for organising a great event.

The UK national VMUG User Conference, held in Birmingham, is in November and I hope to make it. Maybe I’ll see you there?

Quick Tip- Deleting iOS Apps

iOS OneDrive AppProblem: I’m trying to delete an iOS app from an iPad (9.3.2) , I hold down my finger on the icon on the homescreen till they all jiggle but no little x appears on the icon.
Solution: In the Settings App, under “General” then “Restrictions” ensure that “Deleting Apps” is allowed. Go back to the home screen and try again- the small x should now appear.
Enable Deleting Apps on iOS