I have written this blog post to outline the additional steps of deploying NSX to an environment that uses VMware Autodeploy / stateless servers; servers such as those that are part of a Hitachi UCP environment.
I have deployed NSX version 6.2 which was released on the 20th of August ’15. There are a number of changes from previous versions (such as support for cross vCenter NSX (so stretched vMotion between DCs on the same VXLAN – very cool), new tools such as Traceflow, a centralised CLI, improved health status monitoring of communication channels, etc…) Full list of changes is here:
Credit and thanks to Demian Wright for getting this working and sharing his work with us. There’s an official VMware doc for this outlined within this post somewhere. The parts about deploying NSX Manager and creating the controller clusters is light – there’s no different in the setup of a traditional NSX environment for these parts, but I have included them for completeness.
Deploy NSX Manager
Get the OVA from the Nicira download site.
Accept EULA, give VM a name, choose Resource Pool, storage, disk format, network mapping (your Management Network), complete properties:
Confirm details, click ‘Finish’:
After deployment - login to https://nsxmgr01/ with admin account and credentials specified as part of the deploy process:
Click ‘Manage vCenter Registration’:
Define lookup service:
and accept the certificate:
Deploy NSX Controllers
Create 3x NSX Controllers on the Management Cluster: Part of your management cluster should stipulate that you have three ESXi hosts and use DRS Anti-affinity rules to keep these three controller separate.
If you've not already done it, you'll need to define an IP Pool:
Prepare Autodeploy hosts for NSX
Get VIBs (required as we are using Autodeploy and need to wrap the VIBs into an ESXi Image rather than let NSX do the install). Browse to https://NSX_Manager_IPaddress/bin/vdn/nwfabric.properties
This address (/bin/vdn/nwfabric.properties) is specific for 6.2 and later versions – for 6.1 and earlier versions, browse to (/bin/vdn/vibs/5.5/vxlan.zip) and it’ll automatically begin download.
I had to do this through IE; not Chrome:
From here, you can pull the ‘/bin/vdn/vibs…’ file for your specific version of ESXi. Copy out and paste back into the browser – you’ll get a ‘vxlan.zip’ file begin to download:
This needs to be added to your stateless image – either the using the traditional Auto Deploy method http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2092871 or the UCP Director method – add a repository, clone the image and modify the cloned image to include the additional VIBs:
Confirm they are part of your image and deploy to your stateless blades:
N.B., in versions of NSX prior to 6.2, there were 3 VIBs (‘esx-visp’, ‘esx-dvfilter-switch-security’ and ‘esx-vxlan’). In version 6.2, these have been collapsed down into ‘esx-visp’ and ‘esx-vxlan’ as the ‘esx-vxlan’ VIB contains the ‘dvfilter-switch-security’.
Once deployed (through UCP or otherwise) confirm they are part of your host’s running image (for all hosts that are in a cluster you want to participate in NSX):
Update all host profiles, make sure there are no alarms, etc.
Now, through the NSX components of the vSphere Web Client (Networking & Security), install the components on the NSX hosts (first picture depicted below on the Management cluster as I forgot to take a screen shot before doing the necessary work):
Configure VXLAN, as above choosing VLAN, MTU size, etc. For this environment, I used the same IP pool as the controllers, although ideally you would use a different pool/range/VLAN.
After doing so, you should see an additional VMkernel on a new dvPG with an MTU of 1600 :
Update the host profile from reference host, then you’ll notice the additions relevant to NSX:
These need to be excluded from the Host Profile. To do so, right-click the profile, ‘Enable/Disable profile configuration’ and deselect the additions:
N.B., the folder structure may differ in the Web Client (which I should really be using I know). The UserVars. Are found under the folder ‘Advanced Configuration Options’.
At this stage, don’t try and apply the profile to a host that has already been configured for VXLAN. Reboot it, but applying the Host Profile will remove the VMkernel adapter and when trying to reconfigure VXLAN, you’ll see the below errors:
When you reboot, you may see the additional NSX components being started:
Also between the host coming online (i.e., disconnecting and reconnecting in vCenter) and the host profile applying, you will see the following errors from vCenter and NSX:
Eventually, you’ll see a successful ‘Add virtual NIC’ error. This is due to the vDS not being part of the host until the host profile has applied, so be patient.
After which, you're ready to create your transport zone, add clusters, create logical switches, etc... Have fun!