Operation return invalid status code 'Bad Request' : Microsoft.Powerplatform.Cds.Client -Version 0.2.16-Alpha

[bipinshan]

So, we got the following package which provide supports in .net core azure function
Microsoft.Powerplatform.Cds.Client -Version 0.2.16-Alpha

Let me include few details about our implementation:

1. We have used Dynamics connection string - ClientSecret Authentication types

2. Following things working fine with the above connection string.

• Retrieve records from Entities - Working fine.

• Insert/Update operation in Entity which doesn't contain EntityReference type (Lookup) - Working fine.

3. When we perform Insert/update operation in Entity which contains EntityReference type (Lookup), it gives error
   Operation return invalid status code 'Bad Request'

Let me attach some code which we implemented in console application:
Entity newIOTAlert = new Entity("msdyn_iotalert");
//Map Fault Name from Iconic to Description field in IOT Alert in Dynamics
newIOTAlert["msdyn_description"] = messageObject["FaultName"].ToString();
//Map Fault Json data from Iconic to Alert Data field in IOT Alert in Dynamics
newIOTAlert["msdyn_alertdata"] = faultData;
newIOTAlert["msdyn_CustomerAsset"] = new EntityReference("msdyn_customerasset", customerDeviceId);
Guid alertId = cdsClient.Create(newIOTAlert);

Exception has occurred: CLR/Microsoft.Rest.HttpOperationException
An unhandled exception of type 'Microsoft.Rest.HttpOperationException' occurred in Microsoft.PowerPlatform.Cds.Client.dll: 'Operation returned an invalid status code 'BadRequest''
at Microsoft.PowerPlatform.Cds.Client.CdsServiceClient.Create(Entity entity)

[MattB-msft]

@bipinshan Can you tell me what type "msdyn_CustomerAsset" is? also could you lowercase that attribute name and see if it clears this? We fixed an issue in .16 related to polymorphic entities and I wonder if this is actually a casing issue

[bipinshan]

@bipinshan Can you tell me what type "msdyn_CustomerAsset" is? also could you lowercase that attribute name and see if it clears this? We fixed an issue in .16 related to polymorphic entities and I wonder if this is actually a casing issue

I have tried with lowercase as well but no luck. We are using this code both in azure function and console app. Same behaviour in both app.
This is a single entity lookup field. This entity comes with field service module.

[MattB-msft]

any chance you can do a fiddler grab and report what is being sent on the wire?

[bipinshan]

Hi,

Please see below fiddler trace log.

Request

POST https://CRMORG.api.crm.dynamics.com/api/data/v9.1/msdyn_iotalerts HTTP/1.1
Host: msftindustryinn.api.crm.dynamics.com
Prefer: odata.include-annotations=*
Accept: application/json
OData-MaxVersion: 4.0
OData-Version: 4.0
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsIng1dCI6ImppYk5ia0ZTU2JteFBZck45Q0ZxUms0SzRndyIsImtpZCI6ImppYk5ia0ZTU2JteFBZck45Q0ZxUms0SzRndyJ9.eyJhdWQiOiJodHRwczovL21zZnRpbmR1c3RyeWlubi5jcm0uZHluYW1pY3MuY29tLyIsImlzcyI6Imh0dHBzOi8vc3RzLndpbmRvd3MubmV0LzcyZjk4OGJmLTg2ZjEtNDFhZi05MWFiLTJkN2NkMDExZGI0Ny8iLCJpYXQiOjE1OTc5ODM1MzgsIm5iZiI6MTU5Nzk4MzUzOCwiZXhwIjoxNTk4MDcwMjM4LCJhaW8iOiJFMkJnWUppdWRzQnlTZkRCeWpPeGkvM3J6b2txQXdBPSIsImFwcGlkIjoiY2FkNzE5MzgtYjk2MS00MmVmLWIzZGYtOTkyYjYwMzc3Zjk1IiwiYXBwaWRhY3IiOiIxIiwiaWRwIjoiaHR0cHM6Ly9zdHMud2luZG93cy5uZXQvNzJmOTg4YmYtODZmMS00MWFmLTkxYWItMmQ3Y2QwMTFkYjQ3LyIsIm9pZCI6IjBlNzYxNzFlLTZkYzEtNDJkZC04MjUyLWNmMjg3ZGRiMjQxNyIsInJoIjoiMC5BUm9BdjRqNWN2R0dyMEdScXkxODBCSGJSemdaMThwaHVlOUNzOS1aSzJBM2Y1VWFBQUEuIiwic3ViIjoiMGU3NjE3MWUtNmRjMS00MmRkLTgyNTItY2YyODdkZGIyNDE3IiwidGlkIjoiNzJmOTg4YmYtODZmMS00MWFmLTkxYWItMmQ3Y2QwMTFkYjQ3IiwidXRpIjoiTzl3MHZwX04tVU90Q1NSMC1nb0tBQSIsInZlciI6IjEuMCJ9.BX5dFPEoApGgszAz55t6t5xH6cm2lU4zgSxG4a7W_X5miXh_LiMbyeR1YCXms13u4WUBUpd45V2P81ir9fz260Fl0eZDLd-CXnohoK5Y4wQkjUMONXRVxho7Gy-Os1i4fTUVUzOsuvqaoC5BWTE6muQF6AAL_lkEFKALcaQN0p81sFGguijxg5jUFRiJ6hHBtP0b58HQ_F22SlAA2i4Np3JfWc2t4IiLqxWbBXt9HgZ1jvuXO2N7S6te32KU2TCELGVlNBYn2prQshRGAsgzVowjltmyMlROMmQv291cmN7mB9pqhBDmt8OH6iPbgmE1pu8Xi5NGMVS1XJErYVFPTw
x-ms-client-request-id: e8a37ec4-f64b-429e-9b75-2a974c30e04f
User-Agent: src
x-ms-client-session-id: 00000000-0000-0000-0000-000000000000
Connection: Keep-Alive
Content-Type: application/json
Content-Length: 1213

{"msdyn_description":"TestFaultName","msdyn_alertdata":"Test Data","msdyn_alerttype":"192350000","msdyn_alerttime":"2020-09-18T18:30:00.000Z","msdyn_customerasset@odata.bind":"/msdyn_customerassets(6eb05da9-fae2-ea11-a813-000d3a8ccaf5)"}

Response

HTTP/1.1 400 Bad Request
Cache-Control: no-cache
Allow: OPTIONS,GET,HEAD,POST
Content-Type: application/json; odata.metadata=minimal
Expires: -1
x-ms-service-request-id: 31961ec8-6c4d-4b15-8693-698ca800a15f
Set-Cookie: ARRAffinity=965a21324bf4da5d7fc7a956c03ec37036780e7411801ec44aa6fd36068a9f9f; domain=CRMORG.api.crm.dynamics.com; path=/; secure; HttpOnly
Set-Cookie: ReqClientId=5b394440-4803-4215-b8f6-d2dd0ba55b71; expires=Thu, 21-Aug-2070 04:24:22 GMT; path=/; secure; HttpOnly; samesite=none
Set-Cookie: orgId=3b986e21-22ad-4b03-adc1-3a89bd40dcaa; expires=Thu, 21-Aug-2070 04:24:22 GMT; path=/; secure; HttpOnly; samesite=none
x-ms-service-request-id: 01a32316-a037-44d9-93b5-083e2835e95b
Strict-Transport-Security: max-age=31536000; includeSubDomains
REQ_ID: 01a32316-a037-44d9-93b5-083e2835e95b
AuthActivityId: 96b704f6-52ef-4e9b-90bc-1e12456ddf41
REQ_ID: 01a32316-a037-44d9-93b5-083e2835e95b
x-ms-ratelimit-burst-remaining-xrm-requests: 5999
x-ms-ratelimit-time-remaining-xrm-requests: 1,199.99
OData-Version: 4.0
X-Source: 361211352413216126248184731939288105254213230144240191481210531521564858103184169246
Public: OPTIONS,GET,HEAD,POST
Timing-Allow-Origin: *
Set-Cookie: ARRAffinity=965a21324bf4da5d7fc7a956c03ec37036780e7411801ec44aa6fd36068a9f9f; domain=msftindustryinn.api.crm.dynamics.com; path=/; secure; HttpOnly
X-Source: 138211209483118215107188897080244118251204202182902442271712622897102158657919411174
Date: Fri, 21 Aug 2020 04:24:22 GMT
Content-Length: 6774

{"error":{"code":"0x0","message":"An error occurred while validating input parameters: Microsoft.OData.ODataException: An undeclared property 'msdyn_customerasset' which only has property annotations in the payload but no property value was found in the payload. In OData, only declared navigation properties and declared named streams can be represented as properties without values.\r\n at Microsoft.OData.JsonLight.ODataJsonLightResourceDeserializer.ReadUndeclaredProperty(IODataJsonLightReaderResourceState resourceState, String propertyName, Boolean propertyWithValue)\r\n at Microsoft.OData.JsonLight.ODataJsonLightResourceDeserializer.ReadPropertyWithoutValue(IODataJsonLightReaderResourceState resourceState, String propertyName)\r\n at Microsoft.OData.JsonLight.ODataJsonLightResourceDeserializer.<>c__DisplayClass9_0.b__0(PropertyParsingResult propertyParsingResult, String propertyName)\r\n at Microsoft.OData.JsonLight.ODataJsonLightDeserializer.ProcessProperty(PropertyAndAnnotationCollector propertyAndAnnotationCollector, Func2 readPropertyAnnotationValue, Action2 handleProperty)\r\n at Microsoft.OData.JsonLight.ODataJsonLightResourceDeserializer.ReadResourceContent(IODataJsonLightReaderResourceState resourceState)\r\n at Microsoft.OData.JsonLight.ODataJsonLightReader.StartReadingResource()\r\n at Microsoft.OData.JsonLight.ODataJsonLightReader.ReadResourceSetItemStart(PropertyAndAnnotationCollector propertyAndAnnotationCollector, SelectedPropertiesNode selectedProperties)\r\n at Microsoft.OData.JsonLight.ODataJsonLightReader.ReadAtStartImplementationSynchronously(PropertyAndAnnotationCollector propertyAndAnnotationCollector)\r\n at Microsoft.OData.ODataReaderCore.ReadImplementation()\r\n at Microsoft.OData.ODataReaderCore.InterceptException[T](Func1 action)\r\n at System.Web.OData.Formatter.Deserialization.ODataReaderExtensions.ReadResourceOrResourceSet(ODataReader reader)\r\n at System.Web.OData.Formatter.Deserialization.ODataResourceDeserializer.Read(ODataMessageReader messageReader, Type type, ODataDeserializerContext readContext)\r\n at System.Web.OData.Formatter.ODataMediaTypeFormatter.ReadFromStream(Type type, Stream readStream, HttpContent content, IFormatterLogger formatterLogger)","innererror":{"message":"An error occurred while validating input parameters: Microsoft.OData.ODataException: An undeclared property 'msdyn_customerasset' which only has property annotations in the payload but no property value was found in the payload. In OData, only declared navigation properties and declared named streams can be represented as properties without values.\r\n at Microsoft.OData.JsonLight.ODataJsonLightResourceDeserializer.ReadUndeclaredProperty(IODataJsonLightReaderResourceState resourceState, String propertyName, Boolean propertyWithValue)\r\n at Microsoft.OData.JsonLight.ODataJsonLightResourceDeserializer.ReadPropertyWithoutValue(IODataJsonLightReaderResourceState resourceState, String propertyName)\r\n at Microsoft.OData.JsonLight.ODataJsonLightResourceDeserializer.<>c__DisplayClass9_0.<ReadResourceContent>b__0(PropertyParsingResult propertyParsingResult, String propertyName)\r\n at Microsoft.OData.JsonLight.ODataJsonLightDeserializer.ProcessProperty(PropertyAndAnnotationCollector propertyAndAnnotationCollector, Func2 readPropertyAnnotationValue, Action2 handleProperty)\r\n at Microsoft.OData.JsonLight.ODataJsonLightResourceDeserializer.ReadResourceContent(IODataJsonLightReaderResourceState resourceState)\r\n at Microsoft.OData.JsonLight.ODataJsonLightReader.StartReadingResource()\r\n at Microsoft.OData.JsonLight.ODataJsonLightReader.ReadResourceSetItemStart(PropertyAndAnnotationCollector propertyAndAnnotationCollector, SelectedPropertiesNode selectedProperties)\r\n at Microsoft.OData.JsonLight.ODataJsonLightReader.ReadAtStartImplementationSynchronously(PropertyAndAnnotationCollector propertyAndAnnotationCollector)\r\n at Microsoft.OData.ODataReaderCore.ReadImplementation()\r\n at Microsoft.OData.ODataReaderCore.InterceptException[T](Func1 action)\r\n at System.Web.OData.Formatter.Deserialization.ODataReaderExtensions.ReadResourceOrResourceSet(ODataReader reader)\r\n at System.Web.OData.Formatter.Deserialization.ODataResourceDeserializer.Read(ODataMessageReader messageReader, Type type, ODataDeserializerContext readContext)\r\n at System.Web.OData.Formatter.ODataMediaTypeFormatter.ReadFromStream(Type type, Stream readStream, HttpContent content, IFormatterLogger formatterLogger)","type":"Microsoft.Crm.CrmHttpException","stacktrace":" at Microsoft.Crm.Extensibility.OData.CrmODataUtilities.ValidateInputParameters(ModelStateDictionary controllerModelState)\r\n at Microsoft.Crm.Extensibility.OData.EntityController.PostEntitySetImplementation(String& entitySetName, EdmEntityObject entityObject)\r\n at Microsoft.Crm.Extensibility.OData.CrmODataUtilities.<>c__DisplayClass10_02.<InvokeActionAndLogMetric>b__0()\r\n at Microsoft.PowerApps.CoreFramework.ActivityLoggerExtensions.Execute[TResult](ILogger logger, EventId eventId, ActivityType activityType, Func1 func, IEnumerable1 additionalCustomProperties)\r\n at Microsoft.Xrm.Telemetry.XrmTelemetryExtensions.Execute[TResult](ILogger logger, XrmTelemetryActivityType activityType, Func1 func)\r\n at lambda_method(Closure , Object , Object[] )\r\n at System.Web.Http.Controllers.ReflectedHttpActionDescriptor.ActionExecutor.<>c__DisplayClass10.b__9(Object instance, Object[] methodParameters)\r\n at System.Web.Http.Controllers.ReflectedHttpActionDescriptor.ExecuteAsync(HttpControllerContext controllerContext, IDictionary`2 arguments, CancellationToken cancellationToken)\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at System.Web.Http.Controllers.ApiControllerActionInvoker.d__0.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at System.Web.Http.Controllers.ActionFilterResult.d__2.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at System.Web.Http.Dispatcher.HttpControllerDispatcher.d__1.MoveNext()"}}}

[bipinshan]

If I use msdyn_CustomerAsset, Request is missing @odata.bind and because of this it is failing.

POST https://CRMORG.api.crm.dynamics.com/api/data/v9.1/msdyn_iotalerts HTTP/1.1
Host: CRMORG.api.crm.dynamics.com
Prefer: odata.include-annotations=*
Accept: application/json
OData-MaxVersion: 4.0
OData-Version: 4.0
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsIng1dCI6ImppYk5ia0ZTU2JteFBZck45Q0ZxUms0SzRndyIsImtpZCI6ImppYk5ia0ZTU2JteFBZck45Q0ZxUms0SzRndyJ9.eyJhdWQiOiJodHRwczovL21zZnRpbmR1c3RyeWlubi5jcm0uZHluYW1pY3MuY29tLyIsImlzcyI6Imh0dHBzOi8vc3RzLndpbmRvd3MubmV0LzcyZjk4OGJmLTg2ZjEtNDFhZi05MWFiLTJkN2NkMDExZGI0Ny8iLCJpYXQiOjE1OTc5ODQzNzcsIm5iZiI6MTU5Nzk4NDM3NywiZXhwIjoxNTk4MDcxMDc3LCJhaW8iOiJFMkJnWVBqajUvUFNLditXWGZpcXhWczRrcG5GQVE9PSIsImFwcGlkIjoiY2FkNzE5MzgtYjk2MS00MmVmLWIzZGYtOTkyYjYwMzc3Zjk1IiwiYXBwaWRhY3IiOiIxIiwiaWRwIjoiaHR0cHM6Ly9zdHMud2luZG93cy5uZXQvNzJmOTg4YmYtODZmMS00MWFmLTkxYWItMmQ3Y2QwMTFkYjQ3LyIsIm9pZCI6IjBlNzYxNzFlLTZkYzEtNDJkZC04MjUyLWNmMjg3ZGRiMjQxNyIsInJoIjoiMC5BUm9BdjRqNWN2R0dyMEdScXkxODBCSGJSemdaMThwaHVlOUNzOS1aSzJBM2Y1VWFBQUEuIiwic3ViIjoiMGU3NjE3MWUtNmRjMS00MmRkLTgyNTItY2YyODdkZGIyNDE3IiwidGlkIjoiNzJmOTg4YmYtODZmMS00MWFmLTkxYWItMmQ3Y2QwMTFkYjQ3IiwidXRpIjoiQTBwZ193RFdYVS1haUhOcFlsWUpBQSIsInZlciI6IjEuMCJ9.jX-fW4gn21R7_z8y6McJ9CbyGjYqyAr0e1N-DqpgjepWugWeZlVl50pxGnSiBj-5hDQA-6clXrC_5E2n0T47skHMwX6gsQRbXvsfqDuN5CYtCMi535pNVof51NdUOwYTTMnipLR8osgj8kK9h0y80Ro-p4DoEuAg-qb7MBz203kldP0ql7LH6NAZD7cVxELd33vBQ3vRXGApvQKvO1M7p0mvQkHBA-RJDUrMByveTlPe4bY63mK2EA4fk3ZLDzzkVkycz2NTk4dkiFMS3FeRGgaQbpTCKg54n8d_nJcZUT18CGfl5LO9-v9oT8O4ynCJe89Y0wWl0tiWpyJc1L3-DA
x-ms-client-request-id: 642f4395-f39c-4bd5-8531-6e0027a04dd3
User-Agent: src
x-ms-client-session-id: 00000000-0000-0000-0000-000000000000
Connection: Keep-Alive
Content-Type: application/json
Content-Length: 1202

{"msdyn_description":"TestFaultName","msdyn_alertdata":"Test Data"}","msdyn_alerttype":"192350000","msdyn_alerttime":"2020-09-18T18:30:00.000Z","msdyn_CustomerAsset":"/msdyn_customerassets(6eb05da9-fae2-ea11-a813-000d3a8ccaf5)"}

[MattB-msft]

Ok We have identified the issue here.
Unfortunately there is not an easy fix to it... so what we are going to do for now is allow you to send though a camel case property for the lookup and just accept what your telling us to do.
thus in your case,,
you can pass :
newIOTAlert["msdyn_CustomerAsset"] = new EntityReference("msdyn_customerasset", customerDeviceId);
and we will accept that and pass the camel case name on to the server.

We will work with the API team to see if we can get this class of issue sorted out.

[mathiasbl]

@MattB-msft Could you explain further what the problem is? Some conversion from logical name to entitysets?

[JimDaly]

@mathiasbl
I understand that this relates to the OData single-valued navigation property defined for the msdyn_iotalert entity.
This is visible in the $metadata service document.

<NavigationProperty Name="msdyn_CustomerAsset" Type="mscrm.msdyn_customerasset" Nullable="false" 
      Partner="msdyn_msdyn_customerasset_msdyn_iotalert_CustomerAsset">
     <ReferentialConstraint Property="_msdyn_customerasset_value" ReferencedProperty="msdyn_customerassetid" />
</NavigationProperty>

If the navigation property name was msdyn_customerasset rather than msdyn_CustomerAsset , perhaps it would work as expected? As Matt mentioned, if we didn't enforce case sensitivity it could work too.

For example, the account entity has a primarycontactid lookup that is represented in OData by this single-valued navigation property:

<NavigationProperty Name="primarycontactid" Type="mscrm.contact" Nullable="false" Partner="account_primary_contact">
                    <ReferentialConstraint Property="_primarycontactid_value" ReferencedProperty="contactid" />
</NavigationProperty>

In this case the navigation property name matches the lookup attribute logical name.

It is possible to update the navigation property names. But it is risky b/c any code written expecting the original name would be broken. Model-driven apps using UCI can adapt.

Does this help explain?

[MattB-msft]

Adding on to what Jim has discussed,
the CdsServiceClient relies on the metadata definition's returned by the server's metadata API to deal with translating from OrgAPI to WebAPI compatible tags and key's. in this case, the Navigation property's name is not reflected in the entity metadata system... thus if you have a non uniform key name, the CdsServiceClient cannot find it. This results in a badly formatted request to the server.

The fix we are applying to client will just accept what you tell it for attribute name vs trying to validate it with the server for now, while we are working though a proper fix for the server metadata.

this will increase the possibility of issues due to casing, but allow it to work when you have the attribute name casing correct for entity references.

MattB.

[MattB-msft]

Better update... we figured out a way to detect this and do the translation on the fly.
we are going to add that into this release.