MyLab: Automated Instant-Clone Farm (VMware Horizon)

Introduction

This post will discuss the initial setup and configuration for an Automated Instant-Clone Farm for use with VMware Horizon. I am going to use this farm to publish applications and connect them into Workspace ONE Access (WS1 Access). Workspace ONE Access is not required for this technology, but can be leveraged. In this example, I am also going to leverage VMware App Volumes instead of installing the applications directly on the server.

Basically, after installing the Remote Desktop Services Role, we will restrict users to a single session through a local group policy, install the VMware Horizon Agent, install the VMware App Volumes Agent, and finally run the VMware Operating System Optimization Tool (OSOT). Once complete, the virtual machine can be shut down, a snapshot can be taken, and then a Farm can be established in the VMware Horizon Console.

Continue reading “MyLab: Automated Instant-Clone Farm (VMware Horizon)”

How to Remove Hidden Devices in Windows Device Manager

Quick Method: If running on a modern Windows 10, Windows 11, or Windows Server, check that PNPUTIL supports the /enum-devices and /remove-device command. I found that Windows Server 2019 did not support the /enum-devices and therefore did not have a /remove-device command.

If it does, however, the following one liner should work.

foreach ($dev in (Get-PnpDevice -Status UNKNOWN)) { &"pnputil" /remove-device $dev.InstanceId }

If you are on a system that lacks the listed switches in PNPUTIL or you just want the easy method, then you can try using DevManView by NirSoft. This utility allows you to select more than one device at a time as well which is convenitent if there are hundreds of “ghost” devices showing up. (https://www.nirsoft.net/utils/device_manager_view.html)

Have you ever set up VMware App Volumes and found that your virtualized applications didn’t appear to be attached to your virtual desktops? You start out with your basic troubleshooting, beginning with the lower layers of the OSI (Open Systems Interconnection) model. Are the servers hosting the App Volumes Managers online? Is the virtual machine that’s connected visible to App Volumes Manager? Is the service (svservice) running on the virtual machine? Was App Volumes Agent the very last agent installed as per VMware KB 2118048? Can I send and receive basic ICMP (Internet Control Message Protocol) requests from the virtual machine to the manager? Maybe there is a firewall blocking the dataflow? Huh, everything seems to be configured correctly, what could it possibly be?

Finding “Ghost” Devices with PowerShell

Get-PnpDevice -Status UNKNOWN

Look for Disconnected Virtual Disks

Suddenly, while frantically trying to figure out what is going on, you notice that there are virtual disks attached to the virtual machine. Well, now that’s interesting…let us check the Disk Management utility (diskmgmt.msc). Sure enough, there are virtual disks attached, but they are in an offline state. Huh, well that’s odd. We can’t just online them because these are Instant Clones, and that metadata will not persist.

Open Device Manager (devmgmt.msc) to look at the hardware, enabling the Show hidden devices option from the View menu. Let’s poke around a little bit by expanding each device type. Sure enough, there are disk drives, storage controllers, and other devices that appear to be disconnected. They appear as slightly transparent icons and are sometimes referred to as ghosted devices.

The Golden Image should be opened, and all the disconnected devices should be flushed out. Removing all disconnected devices ensures that your Golden Image will be clean and only contain hardware that exists. When the desktop pool is published and an end-user attaches a new device, Windows Plug and Play (PnP) will adapt to the hardware changes with minimal intervention. Refresh the desktop pool and this time when we log on, we have success! The virtualized applications are attached, visible, and function as expected. Open Device Manager and notice there are no disconnected devices. Open the Disk Management utility (diskmgmt.msc) and notice the disks are now online.

Automate The Solution

So now we have a root cause, the effect, and a working solution. However, the solution is very manual, can be painful, and may be error prone. Anytime a human gets in front of a computer, things are bound to go awry. Let’s automate a solution, after all, the computer should be working for us, not the other way around.

IF NOT EXIST C:\TempWork (MKDIR C:\TempWork)

First, let’s make a temporary directory to do our work.

PNPUTIL /enum-devices /disconnected > C:\TempWork\disconnected-devices.txt

Second, let’s get a list of the devices that are disconnected. This will print the disconnected devices into a text file. Unfortunately, there is too much information in this, so we will pare it down to what we do need.

FINDSTR /C:Instance "C:\TempWork\disconnected-devices.txt" > C:\TempWork\devices-to-delete.txt

Third, let us get just the lines that have the Instance ID that we need and store that in another file. Great! So now we have a file of zero, one, or more instance identifiers. Unfortunately, there is still more work to do. Don’t fret, though, the challenging work is almost done and certainly worth the reward, here. If you open this file, you will see each line begins with Instance ID:, a long space, and then the actual information we need to remove the device. The Windows Command Line utility, FOR, will treat each non-space word as a token. So, the first token is the word Instance, the second is ID:, and the third is what we are looking for.

FOR /F "tokens=3" %%G IN (C:\TempWork\devices-to-delete.txt) DO (PNPUTIL /remove-device %%G)

All we need to do from here is iterate through the file, acting upon each Instance ID and removing the device.

@echo off

IF NOT EXIST C:\TempWork (MKDIR C:\TempWork)

PNPUTIL /enum-devices /disconnected > C:\TempWork\disconnected-devices.txt
FINDSTR /C:Instance "C:\TempWork\disconnected-devices.txt" > C:\TempWork\devices-to-delete.txt
FOR /F "tokens=3" %%G IN (C:\TempWork\devices-to-delete.txt) DO (PNPUTIL /remove-device %%G)

PNPUTIL /enum-devices /disconnected

RD /S /Q C:\TempWork

We can put all these statements into a batch file and then all we need is to run it from an elevated command prompt. The work is done instantly, and we can be assured that we did not miss a device or accidentally remove a necessary connected device. If you desire to keep the files for a record of what was detected and removed, just comment out line 11 with REM.

When to run this Batch File

As a best practice, perform this task after removing unnecessary devices from the virtual machine, such as hard disks and CD/DVD devices, running the VMware OS Optimization tool, but before sealing the Golden Image. This way, you can be assured that the virtual machine’s hardware reflects what is actually installed and end-users will get the best experience possible.


Adding a Computer Account to MS SQL Server for a VMware App Volumes Manager Database

ref: https://www.enhansoft.com/updated-how-to-create-a-sql-server-computer-account-login/

This post will probably work for other use cases, but I am specifically needing it for VMware App Volumes Manager.

Open SQL Server Management Studio (SSMS)

Expand Security

Right-click Logins

Select New Login…

1. Do not use the Search… button! Type the Login name: as

DOMAIN\computer-name$

The dollar sign is necessary to signify the account as a computer and not the name of a user. (ref: https://social.technet.microsoft.com/Forums/en-US/eec574c0-5421-4d7a-a806-a3c5af3d29bf/why-in-samaccount-name-of-computer-account-in-active-directory?forum=winserverDS)

2. Choose the Windows authentication radio button.

3. Select the Default database for App Volumes Manager if it was already created. You can assign it later after creating the database if needed.

4. Select the Default language

Do not click OK!

On the Server Roles page, choose the sysadmin checkbox to grant the role to the user. Don’t click OK, yet.

On the User Mapping page, Choose the checkbox next to the database being mapped to the user (computer) account (assuming the database has already been created).

Click OK.

Verify the computer account is added to the list of logins.

This concludes this post.

Installing VMware App Volumes 4 Manager – Part 1

ref: https://docs.vmware.com/en/VMware-App-Volumes/4/com.vmware.appvolumes.install.doc/GUID-2E6F56D8-E657-4290-BAE7-E18E7556ADDC.html (VMware App Volumes Installation Guide)

ref: https://docs.vmware.com/en/VMware-App-Volumes/4/com.vmware.appvolumes.install.doc/GUID-25B53F4E-C22B-4DBD-A253-D7FA33D965BF.html (Installing App Volumes)

I will start out this post to mention that if you try to install App Volumes 4 and are stuck on the SQL database portion trying to get Windows Integrated Authentication (WIA) working, you are not alone. I have spent countless hours troubleshooting this and looking for documentation. Documentation seems to be almost non-existent regarding the database requirements, besides which version to use, but nothing specific on database settings or user/computer/other authentication settings. It is very frustrating.

I will also mention that I have installed MS SQL Server 2016 and have set up TLS certificates on the Windows guest as well as configured SQL Server to use the certificate. I have also added the App Volumes server computer account to the database permissions. Please see the following articles, Configuring Microsoft SQL Server for VMware Horizon View, MSSQL SSL/TLS Certificate Chain Fix, and Adding a Computer Account to MS SQL Server for a VMware App Volumes Manager Database, for more information. If you read and follow the three articles provided, you will not encounter an error when configuring the App Volumes Manager database authentication!

I am going to assume that you have some level of technical proficiency if you are this far along in your journey and will not add mundane details like how to mount an ISO or what buttons to click unless following a particular workflow and feel the details are absolutely necessary. Let’s get started…

After downloading App Volumes 4 (https://my.vmware.com/web/vmware/downloads/info/slug/desktop_end_user_computing/vmware_app_volumes/4_x), mount the ISO to your App Volumes server. Navigate to the Installation directory.

Double-click setup.exe.

Click Next.

Check the I accept the terms in the License Agreement checkbox (you did read all that, right?) and Click Next.

Choose the Install App Volumes Manager radio button since we are installing the App Volumes Manager. Click Next.

Not Shown! – You may be presented with a Windows User Account Control (UAC) prompt. Move past this accordingly. The installation up to this point has been a bootstrap of sorts. The next screens will do the actual Manager installation.

Déjà vu? Groundhog Day? Now we proceed with the actual installation of App Volumes Manager. Click Next.

Choose the Connect to an existing SQL Server Database radio button. Click Next.

There is a lot going on at this step. If you read the beginning of this post and followed the two additional articles provided, you should not have any trouble using Windows Integrated Authentication here. To quickly recap:

  • Your MS SQL server and App Volumes Manager servers can resolve DNS forward and reverse lookup records
  • Your MS SQL server has a TLS certificate in the local machine certificate store
  • The user running your MS SQL service has been provided access to the MS SQL server’s private key
  • The MS SQL server has been configured to use the machine’s TLS certificate
  • The MS SQL service or server has been restarted.
  • The App Volumes database has been created.
  • The App Volumes server account (the computer account) has a log in on the MS SQL server and is associated with the App Volumes database. Adding a Computer Account to MS SQL Server for a VMware App Volumes Manager Database
  1. Enter the FQDN for the MS SQL server.
  2. Choose the Windows Integrated Authentication radio button.
  3. Click the Browse… button. If your authentication works properly, this is a good test as it will either show you the databases or it will present you an error.

Choose the database specified for use with App Volumes and click Next.

4. Check the Enable SQL Server certificate validation checkbox.

Click Next.

Almost there, if you are to this point, the hardest part of the installation is over! Leave the default ports unless you like to complicate your life. Click Next.

Optional: Change the installation location if necessary.

Click Next.

Click Install. Take a break! The installer will need a few minutes to install the software.

Confirm that the Wizard completed successfully. See Troubleshooting section below if you did not have a successful installation.

Click Finish.

You have completed the install, but there are some prerequisite steps to take to make the configuration easier and work the first time. Check out the article here: Installing VMware App Volumes 4 Manager – Part 2.

Troubleshooting

Be sure to read the finished dialog! You may receive the following, reporting that the Wizard ended prematurely. You will have to open up the logs and investigate; navigate to your installation directory and look at the logs. The issue that generated the following screenshot for me was either a self-signed certificate issue or a SQL Server issue because I didn’t add the App Volumes Manager server login to the MS SQL server and assign it to the App Volumes database (see below for specific error message and how to fix). Delete the Cloud Volumes directory and try the installation again.

From the inst_ruby_script.log file, you see the following:

E, [2020-11-19T11:43:30.666302 #3196] ERROR -- : ERROR: 28000 (18456) [Microsoft][ODBC Driver 17 for SQL Server][SQL Server]Login failed for user 'AARONROMBAUT\APP-001V$'.
E, [2020-11-19T11:43:30.666571 #3196] ERROR -- : ["script/odbc_driver_validator.rb:145:in drvconnect'", "script/odbc_driver_validator.rb:145:inconnect_using_conn_string'", "script/odbc_driver_validator.rb:34:in odbc_connection'", "script/odbc_driver_validator.rb:76:invalidate_conn_and_version'", "script/odbc_driver_validator.rb:183:in <encoded>'", "script/odbc_driver_validator.rb:2:inRGLoader_load'", "script/odbc_driver_validator.rb:2:in `
'"]
Unable to connect to SQLServer. Exiting with status -1.

To resolve this, follow this article: Adding a Computer Account to MS SQL Server for a VMware App Volumes Manager Database.

MSSQL SSL/TLS Certificate Chain Fix

I was configuring VMware App Volumes and ran into an issue where the installer reported a MS SQL security alert. Since I am trying to get this to work in a production-like environment, I did not want to just “Trust server certificate” and move along. I wanted this to be installed appropriately. A couple minutes Googling and this post is a record of my findings for the future. Hopefully, it may help you as well if you stumbled here. Here is a screenshot of the security alert.

First of all, make sure you have a CA-signed certificate loaded in the Personal store on the server hosting your MS SQL Server. You can quickly check your machine certificates by clicking the Start button or opening Run and typing certlm.msc. If you don’t have that, stop here and go get one. I used Let’s Encrypt for my certificate.

The next thing you want to do is verify the service account that is running your MS SQL Server. In my case, I am using the default NT Service\MSSQLSERVER.

Right-click on your machine certificate and point to All Tasks, and choose Manage Private Keys…

The Permissions window opens up.

Add the account you verified as the Log On As user when checking the Services.msc management console. (You may need to change the location to the local server name from the Locations… button on the side of the Select Users or Groups window.) Ensure the user has Full Control on the certificate.

Go back to the Services.msc console and restart the SQL Server service and you should no longer have authentication or trust issues with connecting services.