Skip to content

Conversation

ayush3797
Copy link
Contributor

@ayush3797 ayush3797 commented Nov 28, 2023

Why make this change?

Currently, for a create mutation (or any graphql mutation essentially), we can only do insertion/mutation in one table and not in another table which is related to this table via some relationship (either defined in config or in the database). This PR aims to:

  1. Extend the functionality of the existing create mutations (which are basically point insertions) to allow insertions in related table.
  2. To generate additional createMultiple mutations which will allow multiple nested insertions starting from the top-level entity.

Note: All changes made in this PR are specific to MsSql. Nothing changes for the other database flavors.

Quick glossary:

  1. ObjectTypeDefinitionNode: Represents a table/view/stored-procedure. For a table, this contains all the column fields which belong to the table and the relationship fields which are defined in this table's entity configuration in the config file.
  2. InputObjectTypeDefinitionNode: Represents an input object type we generate for mutations.

What is this change?

  1. We no more ignore relationship fields while generating the input types for a create mutation: Till date, we were only considering the column fields in an object (of type ObjectTypeDefinitionNode) to generate the input type (of type InputObjectTypeDefinitionNode) for a table entity. But to support nested insertion, we would now also iterate over relationship fields to generate the input type for a create/createMultiple mutation.

  2. Addition of linking entities at the backend: To support nested insertions in tables which have a relationship with cardinality N:N, the user can provide a linking table with the source defined in linking.object field which we need the user to provide. We need to generate an object type (of type ObjectTypeDefinitionNode) for this linking entity. This object type is later be used to generate the input object type for the linked tables.
    Additional details about linking entities:

    -> A boolean property IsLinkingEntity is added with a default value of false to the Entity record. This ensures that we are backwards compatible, and all the entities provided in the config are not considered as linking entities. For all the linking entities, this boolean property will be set to true.
    -> For a linking entity, the GraphQL and REST endpoints are disabled (set to false) by default. This ensures that we don't accidentally expose the linking entity to the user.
    -> MsSqlMetadataProvider.EntityToDatabaseObject contains database objects for Entities in config + Linking Entities. After we get all the deserialized entities, we will create entities to represent linking tables. The database objects for all the entities in config will be generated by a call to SqlMetadataProvider.GenerateDatabaseObjectForEntities() method which in turn sequentially goes over all the entities in the config and call the method SqlMetadataProvider.PopulateDatabaseObjectForEntity() which actually populates the database object for an entity.
    -> The method SqlMetadataProvider.AddForeignKeysForRelationships() is renamed to SqlMetadataProvider.ProcessRelationships(). The method now in addition to adding FKs to metadata, also creates linking entities for M:N relationships. This is done via a call to method SqlMetadataProvider.PopulateMetadataForLinkingObject().
    -> The method PopulateMetadataForLinkingObject() is a virtual method which has an overridden implementation only for MsSql. Thus, linking entities are generated only for MsSql.

  3. ReferencingFieldDirectiveType: is a new directive type being added. The presence of this directive for a column field implies that the column is a referencing key to some column in another entity exposed in the config. With nested insertions support, the values for such fields can come via insertion in the referenced table. And hence while generating the input object for a create/createMultiple mutation, we mark such a field as not required.

  4. Name of the create multiple mutation: is generated via a call to the newly added helper method CreateMutationBuilder.GetInsertMultipleMutationName(singularName, pluralName) which takes in the singular/plural names of the entity. If singularName == pluralName, the createMultiple mutation will be generated with the name: create{singularName}_Multiple else, it will be generated with the name create{pluralName}. The pluralName for an entity is fetched using another newly added helper method GraphQLNaming.GetDefinedPluralName(entityName, configEntity)

  5. For create multiple mutations, the input argument name will be items which is stored in a constant string MutationBuilder.ARRAY_INPUT_ARGUMENT_NAME.

  6. Object types (of type ObjectTypeDefinitionNode) for linking entities will be generated using a method GraphQLSchemaCreator.GenerateObjectDefinitionsForLinkingEntities(linkingEntityNames, entities).

  7. sourceTargetLinkingNode: The object type for linking entity is later used to generate object type for a sourceTargetLinkingNode which will contain:
    -> Fields present in the linking table (but not used to relate source/target)
    -> All the fields from the target entity.
    This is done using a call to the newly added method GraphQLSchemaCreator.GenerateSourceTargetLinkingObjectDefinitions().

  8. Renamed method GraphQLSchemaCreator.FromDatabaseObject() to GraphQLSchemaCreator.GenerateObjectTypeDefinitionForDatabaseObject() to better convey what the method is doing.

  9. To make code more streamlined and bug-free the method SchemaConverter.FromDatabaseObject() is broken down into smaller chunks which handle smaller and easy to read/comprehend responsibilities. Passing of existing test cases confirm the refactor.

  10. Added a method GraphQLUtils.GenerateLinkingEntityName(sourceEntityName, targetEntityName) which concatenates the names of the source and target entities with a delimiter (ENTITY_NAME_DELIMITER = "$") between the names and prefixes the concatenated string with: LINKING_ENTITY_PREFIX = "LinkingEntity" string.

  11. Added a method GraphQLUtils.GetSourceAndTargetEntityNameFromLinkingEntityName(string linkingEntityName) which returns the the names of the source and target entities from the linking entity name.

  12. Split the CreateMutationBuilder.GenerateCreateInputType() method which was used to create input types for Relational (Pg/My/Ms/Dw Sql) and non-relational dbs (Cosmos_NoSql) into two new methods: CreateMutationBuilder.GenerateCreateInputTypeForRelationalDb() and CreateMutationBuilder.GenerateCreateInputTypeForNonRelationalDb().

  13. Split the CreateMutationBuilder.GetComplexInputType() method which was used to build input types for Relational (Pg/My/Ms/Dw Sql) and non-relational dbs (Cosmos_NoSql) into two new methods: CreateMutationBuilder.GenerateComplexInputTypeForRelationalDb() and CreateMutationBuilder.GenerateComplexInputTypeForNonRelationalDb().

Validations Required:

  1. Ensure non-conflicting names of fields in sourceTargetLinkingNode for N:N relationships: Object types are generated for sourceTargetLinkingNode for a source, target pair of tables (refer above for what all fields would be present in this object type). The name of the field (either a relationship field or a column field) in the target entity might conflict with the name of the column field in the linking table. Hence, we need a validation in place to ensure that if there is a conflict, we catch it during startup (currently this check is done during schema generation) and throw an appropriate exception with an actionable message.
  2. Ensure non-conflicting names of relationship fields in column fields in one entity: This is a validation which is required in the current architecture as well and is a bug (see here: [Known Issue] Relationship name and exposed field name of an entity must not overlap #1937). This is required to ensure that relationship field names don't conflict with the exposed names of column fields in a table.
  3. Ensure we have values for FK referencing fields to perform insertions: All the fields in a table which hold a foreign key reference to some other column in another entity are now nullable in the generate input objects for create mutation. This is because we assume that the value for such a field might come via insertion in the related entity/table. However, if the create mutation does not include a nested insert, the value of such a field still has to be specified by the user (unless the field is nullable/has default at the database level as well in which case we can give a null value/default value for the field- although this is highly unlikely since the field is a foreign key). If we don't get a value for such a FK referencing field:
    -> Either via nested insertion , or
    -> By the user
    we should throw an exception accordingly. This will maintain backwards compatibility. UNLESS THIS VALIDATION IS IN, WE ARE NOT BACKWARDS COMPATIBLE.
  4. Ensure there is only one source of value for FK referencing fields: If the user is providing a value for an FK referencing field, and there is a nested insertion involved which also returns a value for the same field, we should throw an exception as there are two conflicting sources of truth. In short, exactly one of the sources should provide a value.

How was this tested?

  • Unit Tests - Added to NestedMutationBuilderTests.cs class.

Sample Request(s)

image

@ayush3797 ayush3797 changed the base branch from main to dev/NestedMutations November 28, 2023 05:29
@ayush3797 ayush3797 self-assigned this Jan 2, 2024
@ayush3797 ayush3797 added the Multiple mutations Fixes/enhancements related to nested mutations. label Jan 2, 2024
@ayush3797 ayush3797 linked an issue Jan 2, 2024 that may be closed by this pull request
@ayush3797 ayush3797 added mssql an issue thats specific to mssql enhancement New feature or request labels Jan 2, 2024
@ayush3797 ayush3797 changed the title Schema generation for nested mutations Schema generation for nested insertions Jan 14, 2024
@Azure Azure deleted a comment from azure-pipelines bot Jan 15, 2024
@Azure Azure deleted a comment from azure-pipelines bot Jan 15, 2024
@Azure Azure deleted a comment from azure-pipelines bot Jan 15, 2024
Copy link
Collaborator

@Aniruddh25 Aniruddh25 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! Please get this in. No more changes other than any simple stylistic or obviously wrong assumptions.

@ayush3797
Copy link
Contributor Author

/azp run

Copy link

Azure Pipelines successfully started running 2 pipeline(s).

@Azure Azure deleted a comment from azure-pipelines bot Mar 13, 2024
@ayush3797 ayush3797 changed the title Schema generation for multiple create Multiple-create: Schema generation Mar 19, 2024
@ayush3797
Copy link
Contributor Author

/azp run

Copy link

Azure Pipelines successfully started running 2 pipeline(s).

@Azure Azure deleted a comment from azure-pipelines bot Mar 21, 2024
@ayush3797 ayush3797 merged commit e2f9418 into dev/NestedMutations Mar 21, 2024
@ayush3797 ayush3797 deleted the dev/agarwalayush/schemaGeneration branch March 21, 2024 14:26
severussundar added a commit that referenced this pull request Mar 26, 2024
…ue (#2116)

## Why make this change?

- Closes #1951
- PR #1983,
#2103 add CLI options to
enable or disable multiple mutation/multiple create operation through
CLI. With the changes introduced in the mentioned PRs, the configuration
properties successfully gets written to the config file. Also, during
deserialization, the properties are read and the
`MultipleMutationOptions`, `MultipleCreateOptions`,
`GraphQLRuntimeOptions` objects are created accordingly.
- The above-mentioned PRs do not introduce any change in DAB engine
behavior depending on the configuration property values.
- This PR introduces changes to read these fields and enable/disable
multiple create operation depending on whether the feature is
enabled/disabled through the config file. This is achieved by
introducing behavior changes in the schema generation.

## What is this change?
- This PR builds on top of a) Schema generation PR
#1902 b) Rename
nested-create to multiple create PR
#2103
- When multiple create operation is disabled, 
> i) Fields belonging to the related entities are not created in the
input object type are not created.
> ii) Many type multiple create mutation nodes (ex: `createbooks`,
`createpeople_multiple` ) are not created.
> iii) ReferencingField directive is not applied on relationship fields,
so they continue to remain required fields for the create mutation
operation.
> iv) Entities for linking objects are not created as they are relevant
only in the context of multiple create operations.

## How was this tested?

- [x] Unit Tests and Integration Tests 
- [x] Manual Tests 

**Note:** At the moment, multiple create operation is disabled in the
config file generated for integration tests. This is because of the plan
to merge in the Schema generation, AuthZ/N branches separately to the
main branch. With just these 2 PRs, a multiple create operation will
fail, hence, the disabling multiple create operation. At the moment,
tests that perform validations specific to multiple create feature
enable it by i) updating the runtime object (or) ii) creating a custom
config in which the operation is enabled.

## Sample Request(s)

### When Multiple Create operation is enabled - MsSQL

#### Related entity fields are created in the input object type

![image](https://github.com/Azure/data-api-builder/assets/11196553/7a3a8bbe-2742-43e0-98d7-9412ed05db33)

#### Multiple type create operation is created in addition to point
create operation

![image](https://github.com/Azure/data-api-builder/assets/11196553/c6513d9a-5b49-44cc-8fcc-1ed1f44f5f58)


#### Querying related entities continue to work successfully

![image](https://github.com/Azure/data-api-builder/assets/11196553/4c1a61b8-0cbb-4a1e-afaa-1849d710be27)



### When Multiple Create operation is disabled - MsSQL

#### Only fields belonging to the given entity are created in the input
object type

![image](https://github.com/Azure/data-api-builder/assets/11196553/a3b6beb2-7245-4345-ba13-29d8905d859e)

#### Multiple type create operation is not created


### When Multiple Create operation is enabled - Other relational
database types

#### Only fields belonging to the given entity are created in the input
object type

![image](https://github.com/Azure/data-api-builder/assets/11196553/b2a4f7f6-b121-410d-806d-8c5772253080)

#### Multiple type create operation is not created

---------

Co-authored-by: Ayush Agarwal <[email protected]>
@severussundar severussundar mentioned this pull request Mar 26, 2024
1 task
severussundar added a commit that referenced this pull request Mar 29, 2024
## Why make this change?

- All code changes for **Multiple Create** feature was being merged into
`dev/NestedMutations` branch.
- This PR attempts to merge all these changes to the `main` branch in
preparation for the `0.12.* rc1` release

## What is this change?

- Right now, `dev/NestedMutations` branch contains the code changes for
the following components of Multiple Create feature
- Schema Generation -
#1902
  - AuthZ - #1943
- Feature flag - CLI changes
#1983
- Feature flag - Re-naming changes
#2103
- Feature flag - Engine changes
#2116
- Each specified PR was reviewed before merging into
`dev/NestedMutations` branch.
- This PR aims to merge all the changes into `main` branch
  
## How was this tested?

- [x] Unit, Integration and Manual tests were performed on each PR
before merging into `dev/NestedMutations`

---------

Co-authored-by: Shyam Sundar J <[email protected]>
Co-authored-by: Sean Leonard <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request mssql an issue thats specific to mssql Multiple mutations Fixes/enhancements related to nested mutations.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Schema Generation - Nested Inserts
6 participants