VMMExpress - Deployment issue

[PnoT]

I'm running SCVMM 2022 RU2 with Windows Server 2022 standard edition and using a Sysprep 2022 vhd for deployment.

My deployment scenario:
standalone
gen2
highly available: $false
no logic switch deployed
only deploy NC_management network (no hnvpa, slb, or gw)

I've worked through many quirks and am at the point of deploying the NC service, but I'm running into a wall now.

The job of creating the service kicks off and bombs at 99% of step 1.4 Parallel execution step

https://imgur.com/LrItUpY

Here are my errors in the script log

`*****************************************************
*** 2023-12-22 07:16:41Z : New Deployment started ***

---

Using configuration from file [.\Fabricconfig.psd1]
Getting VMM server connection with VMM server [LLWVSCVMMP01]
Checking the Fabric Configuration Input Parameters
Successfully authenticated with domain lab
Logical Network and Logical Switch is not Pre-configured.
Starting to create Management Logical Network []
Getting the Host group with Name [System.Collections.Hashtable.NCHostGroupName]
Creating VLAN subnet for subnet [System.Collections.Hashtable.LogicalNetworkIPSubnet] and VLAN Id [System.Collections.Hashtable.LogicalNetworkVLAN]
Creating new Logical Network Definition
create a VMNetwork with the same name as Logical Network
Management Logical Network Deployment completed succssfully
Created Logical Network : NC_Management
Logical Network creation succeeded
Creating Logical Switch and Deploying to all Hosts in Host Group : [NC]
creating logical switch [NC_LogicalSwitch]
You have decided to use CA certificate. Hope you Placed the Cert in \Templates\NC\TrustedRootCertificate.cr folder
Trimmed VMName:[NCE-NCVM01.lab.local]
Recieved VMName : [NCE-NCVM01.lab.local]
VmName : [NCE-NCVM01.lab.local]
Mapping VHD to template package
Mapping NCSetup.cr to template package
Mapping ServerCertificate.cr to template package
Mapping TrustedRootCertificate.cr to template package
Starting Service Template Configuration
getting Management Network [NC_Management]
Getting the service setting
Creating Account
The deployment was not successful.
Reason: The specified change on the object NC Deployment service Template is not allowed. (Error ID: 21970)

Ensure that no service configuration or service is referencing the object.
`

The errors in the VMM job

`Error (22042)
The service NC was not successfully deployed. Review the event log to determine the cause and corrective actions.

Recommended Action
The deployment can be restarted by retrying the job.

Error (22753)
The script command with properties: Type (PreInstall), Deployment Order (3) and Parent Type (ApplicationProfile), failed to complete successfully. Refer to the errors list for more information.

Recommended Action
If the script command's job restart action is set to restart, then the script will be re-executed. Otherwise, the script command will be skipped when the job is restarted, in which case corrective action should be taken to mitigate the effects of the script command failure.

Error (22631)
The script command exit code 2148734208 matched the failure policy setting "Match any value other than zero." Standard output log data: "icrosoft.PowerShell.Core\Registry::HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSe
t\Services\Tcpip6\Parameters
PSParentPath : Microsoft.PowerShell.Core\Registry::HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSe
t\Services\Tcpip6
PSChildName : Parameters
PSDrive : HKLM
PSProvider : Microsoft.PowerShell.Core\Registry

Windows IP Configuration

Registration of the DNS resource records for all adapters of this computer has been initiated. Any errors will be reported in the Event Viewer in 15 minutes.
[2023-12-22T05:24:48.1097290-08:00]Installing NetworkController Role..
[2023-12-22T05:24:48.8412126-08:00]Caught an exception:
[2023-12-22T05:24:48.8412126-08:00] Exception Type: System.Exception
[2023-12-22T05:24:48.8412126-08:00] Exception Message: ArgumentNotValid: The role, role service, or feature name is not valid: 'NetworkController'. The name was not found.
[2023-12-22T05:24:48.8412126-08:00] Excepti"

Recommended Action
If the script command's job restart action is set to restart, then the script will be re-executed. Otherwise, the script command will be skipped when the job is restarted, in which case corrective action should be taken to mitigate the effects of the script command failure.

Error (20400)
1 parallel subtasks failed during execution.

Error (21952)
Application deployment failed for one or more tiers or application hosts in the service NC. Check job logs to get more information on the failed operation.

Recommended Action
Check error messages and retry the operation if needed.

`

Solution: My sysprep vhdx was Windows Server 2022 Standard Edition, which does not have the "Network Controller" role. I swapped it out with a Datacenter Edition vhdx.

[PnoT]

Issue:
[2023-12-22T11:36:19.9824919-08:00]Adding to local admin group..
[2023-12-22T11:36:22.3956947-08:00]Attempt 1 - Adding to Administrators group
[2023-12-22T11:36:22.4269519-08:00]Successfully added to Administrators group.
[2023-12-22T11:36:22.4269519-08:00]Adding trusted hosts
[2023-12-22T11:36:22.9113631-08:00]Finding certificate file in script directory. 'C:\NCInstall\certificate-ssl'
[2023-12-22T11:36:22.9426161-08:00]Error: Did not find an SSL certificate file deployed to VM.
[2023-12-22T11:36:22.9426161-08:00] Please create a valid certificate and include in the SSL certificate.cr "

Solution: Set the option to generate a self-signed cert to $true as I did not have a CA

After enabling the option, I ran into this problem:

Reason: CertEnroll::CX509Enrollment::_CreateRequest: Access is denied. 0x80070005 (WIN32: 5 ERROR_ACCESS_DENIED)

Solution: Run Virtual Machine Manager Command Shell with Administrator rights. I must have closed out of the window
and accidentally reopened the session as a non-admin so the cert generation would fail.

Issue:

VMName while onboarding NC : [NCE-NCVM01.lab.dmz]
COnnection String :
The deployment was not successful.
Reason: Execution of :: on the configuration provider failed. Detailed exception: Unable to connect to the network service. Check connection string and network connectivity. Execution of Microsoft.SystemCenter.NetworkService::OpenDeviceConnectionEx on the configuration provider 3e2875a7-5831-4fb2-b388-1672e1c20fee failed. Detailed exception: Microsoft.VirtualManager.NCRestApiWrappers.NCRestApiWrappersException:
Check the documentation for the configuration provider or contact the publisher support.
Unable to connect to the network service. Check connection string and network connectivity. (Error ID: 21426)

Solution: This issue has popped up a bunch from various users, and after troubleshooting, the credentials being used are not enough to connect correctly to the network controller. The function "OnBoardNetworkController" uses the credentials for "NC_MgmtAdminRAA" but if you manually go into SCVMM and try to add the controller with the runas account, you'll see that it does connect but shows "false" to all of the features and eventually will fail to add. Here is how the window is supposed to look https://www.tech-coffee.net/wp-content/uploads/2015/08/082115_1315_Deployandad13.png?w=620&ssl=1 You can see "True" for most of the properties. I'm trying to understand why this happens, but using my domain admin account resulted in it being added correctly. I want to get to the bottom of this because using my credentials is not ideal, and this is in my home lab for now.

These issues are the results of the credentials not working as well.
#492
#495

[DCoates-InnovateLtd]

what OS and what edition are you running for your network controllers? if the role isnt found its only a Datacenter roll. also if your using SCVMM check your template for the VM is setup correctly.