Skip to content

Feat/serializer context decoupling #558

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 94 commits into from
Oct 10, 2019
Merged

Conversation

maurei
Copy link
Member

@maurei maurei commented Sep 27, 2019

Overview decoupling PR

This is an intermediate PR intended to provide insight into the structural decisions made so far. Please limit feedback to those elements. Commenting, variable naming and spacing consistency will be further improved along the way.

This PR contains rework of three main elements:

  • Deserializer
  • Serializer
  • Breaking up of RequestManager into more separated isolated services [WIP]
  • Breaking up ResourceGraph [WIP]

Serializers and Deserializers

see wiki here.

RequestManager

The RequestManager was becoming too big and it contained too many responsibilities. I have introduced the following approach.
The request manager is split up into services in one of two categories:

  • URL query param services: stuff related to anything behind the ? in URL, like inclusions, filters, etc.
  • request scope meta services: remaining meta data associated to scope of one request, like the main resource of the request (eg. Article when route like /articles**).

URL query param services

Each URL query operator has been moved to its own dedicated service. This is to prevent that if a service needs only a small subset (eg IncludedRelationshipBuilder needs only the inclusion chains), it does not exposed to all the other query parameters too.

  • IPageQueryService the former page manager
  • IFieldQueryService to access sparse field selections
  • IIncludedQueryService to access inclusion chains
  • IAttributeBehaviourQueryService to encapsulate the feature described here
  • Others are yet to be implemented: FilterQueryService, SortQueryService. I might be missing more.

Each service will also contain "business logic" related to that particular service. For example IIncludedQueryService

  • knows how to convert a chain ("articles.author.favorite-food") into corresponding RelationshipAttributes
  • throws errors when a non-existing relationship is targeted

Request scope meta services

Remaining metadata associated to a particular request. Also separated into multiple small services to prevent unnecessary exposure of data.

  • IUpdatedFields to keep track of which fields of a resource (AttrAttributes and RelationshipAttributes) are targeted in the scope of a request.
  • ICurrentRequest to access metadata that is not (yet) isolated to its own service. Like the main resource of a request.

Todo:

see the App vs. Request vs. Operations section in #253. Currently the RequestManager had both request-level and operational-level state information (updated attributes/relationships vs main request resource).
This particular example has been separated but we need to further categorize the above services such that it is compatible with this distinction.

We will currently not support bulk operations (dedicated issue + PR coming soon) for v4, but in order to re-support it in the future it is important to properly distinguish between operation-scoped services and request-scoped services (eg. IUpdatedFields is operation-level, ICurrentRequest request-level). All requests will then, for now, be associated to one operation, but can be associated with more.

ResourceGraph

Throughout JADNC, the resource graph is used for two purposed

  • getting ContextEntity given a exposed resource string, resource type, controller name
  • resolving relationships which require a lot of information from the graph, eg many-to-many relationships.

Because of these two pretty deviating responsibilities, I have introduced

  • IContextEntityProvider just to get a ContextEntity (various overloads)
  • still need to put the rest of resource graph behind a new interface, something like IRelationshipResolver, which can probably be merged with the already existing IInverseRelationshipResolver which was introduced in ResourceHooks.

Todo

Few things are left to be done in the domain of serialization:

Things left to be done in general

  • Wiring back up parsing of URL query parameters in QueryParser
  • Applying these queries in the data layer: JsonaApiContext is still in there, we need to create something like a QueryProcessorFactory service that instantiates eg SortQuerys or FilterQuerys using DI, and use that in the places were manual instantiation is being performed right now (see the DefaultEntityRepository class).
  • Removing the operations experimental feature. We might put it back later but for now it would delay v4 release by too much time. Also the current implementation deviates a lot from the extension that might be included in the v1.1 specs
  • Removing the EntityResource separation example and related code. See [RFC] Deprecate support for Resource Entity Separation  #553

Harro van der Kroft and others added 30 commits May 3, 2019 17:31
@maurei maurei merged commit 343912e into dev-v4 Oct 10, 2019
@maurei maurei mentioned this pull request Oct 23, 2019
14 tasks
maurei added a commit that referenced this pull request Oct 30, 2019
* Feat/context decoupling (#557)

* feat: started work on decoupling

* feat: add total record count

* feat: green tests, new managers

* feat: changed startup

* feat: most annoying commit of my life, rewriting dozens of controllers

* feat: rename: QueryManager -> RequestManager, deeper seperation of concerns, 12 tests running...

* feat: removed JsonApiContext dependency from controller, fixed namespaced tests

* feat: decoupled controllers

* chore: renamed resourcegraphbuilder,  took out some extensions, made benchmarks work again

* feat: json api context decoupling mroe and more

* chore: readded solutions

* feat: upgrading to 2.2, setting contextentity in middleware

* fix: removed outdated authorization test

* fix: requestmeta tests

* fix: some acceptance tests

* fix: pagination

* fix: total records in meta

* feat: introduced JsonApiActionFilter

* fix: more tests

* Feat/serializer context decoupling (#558)

* feat: started work on decoupling

* feat: add total record count

* feat: green tests, new managers

* feat: changed startup

* feat: most annoying commit of my life, rewriting dozens of controllers

* feat: rename: QueryManager -> RequestManager, deeper seperation of concerns, 12 tests running...

* feat: removed JsonApiContext dependency from controller, fixed namespaced tests

* feat: decoupled controllers

* chore: renamed resourcegraphbuilder,  took out some extensions, made benchmarks work again

* feat: json api context decoupling mroe and more

* chore: readded solutions

* feat: upgrading to 2.2, setting contextentity in middleware

* fix: removed outdated authorization test

* fix: requestmeta tests

* fix: some acceptance tests

* fix: pagination

* fix: total records in meta

* feat: introduced JsonApiActionFilter

* fix: more tests

* feat: decoupled deserializer and serializer

* chore: remove / cleanup old serializers

* chore: remove operation services

* tests: unit tests client/server (de)serializers

* chore: various edits to run tests

* fix: rm dasherized resolver

* chore: remove JsonApiContext and corresponding interface

* chore: reorganize namespaces

* chore: rm old tests

* chore: rm unused document builder

* chore: add comments to deserialization classes

* chore: add comments

* chore: rm IJsonApiContext from sort attr instantiation

* feat: (re)introduced omit null value behaviour (and now omit default value behaviour too) using a serializer options provider

* chore: remove more jsonapicontext references

* chore: minor renames, add wiki for serialization

* chore: improve wiki

* chore: wiki

* chore: wiki

* chore: add a bunch of comments, renamed a few services

* chore: wired up nested sparse field selection

* chore: wired up omit behaviour

* chore: prettied some comments

* chore: remove (almost) all references to jsonapicontext in unit tests project

* chore: removed last bits of jsonapicontext

* fix: tests serialization passing again

* chore: rename included query service to include query service

* chore: improve comment

* chore: improve comment

* chore: delete jsonapicontext tests

* chore: remove space

* test: email config

* chore: rename client / server (de)serializer to request/response (de)serializer

* chore: rename client / server (de)serializer to request/response (de)serializer

* feat: introduce IQueryParameter

* chore: move request/response (de)serializer to client/server namespace for easier future isolation

* chore: update namespaces

* chore: rm textfile

* chore: fixing failing unit tests

* chore: rename query services, adjustment namespace

* chore: reorganised test files for serialization, added relationship path serialization tests

* feat: request relationship in response serializer

* chore: wired up response serializer in formatter layer

* feat: IGetRelationshipService now returns TResource instead of object, which decouples it from serializers requirements

* fix: support for serving document of resource type different from request resource, as required by resourceservice.getrelationships()

* chore: simplified linkbuilders

* chore: several fixes for e2e tests

* fix: various e2e tests, decoupled service layer from serialization format

* fix: inclusion edgecase

* chore: fix error formatting tests

* chore: naming consistency sparsefield and include query services

* chore: various adjustments to make e2e test project build and pass again

* chore: fix build various e2e tests

* chore: fix build various e2e tests

* fix: e2e test Can_Include_Nested_Relationships

* fix: e2e test Can_Patch_Entity

* fix: e2e test Patch_Entity_With_HasMany_Does_Not_Include_Relationships

* fix: e2e test Can_Create_Entity_With_Client_Defined_Id_If_Configured

* chore: rename base document parser and builder

* fix: e2e test Can_Create_Guid_Identifiable_Entity_With_Client_Defined_Id_If_Configured

* fix: unit tests various

* feat: reorganisation inheritance serialization layer

* fix: unit tests after inheritance update

* fix: wire up correct resource object builder implementation in serializers

* fix: e2e remaining CreatingDataTests

* chore: refactor creatingdata tests

* fix: e2e test paging

* fix: e2e controller tests

* chore: wiring up new resource object builders to dependency graph

* chore: response resource object builder unit test, restored repo and resourcedef unit tests

* chore: finishing touches comments serialization

* Feat/serialization wiki (#561)

* docs: add decoupling architecture, wiki folder

* docs: add v4 wiki

* chore: restructure of intro

* chore: v4

* chore: remove example projects

* chore: remove bulk related code

* chore: remove bulk related tests

* chore: remove logic related to ER splitting

* chore: remove split test project

* chore: update other test projects to pass again

* chore: fix typo

* Added value types support to the filters isnull & isnotnull. (#549) (#573)

* fix: missing reference'

* chore: fix spacing

* feat: draft of reflectively retrieving all query parameters

* chore: scaffolded required query parameter service

* chore: migrate parsing logic sparsefields

* chore: reorganising test models in unit test project

* tests: add includeservicetests

* chore: migrate parsing logic of filter to filter service

* chore: prefix private variables underscore in editorconfig

* feat: common query parameter base service

* feat: interfaces of query parameter services

* feat: sort service and filter service

* feat: wired up filter and sort service with refactored data and repo layer

* feat: new omit default and null value services, wired them up in serialization

* feat: refactored remaining query services, introduced new queryparser for better extensibility

* feat: wired up new query parser in action filter, slimmed down current request

* chores: various test fixes, minor fixes

* chores: various test fixes, minor fixes

* chore: add more unit tests

* docs: query parameters wiki

* fix: typo

* chore: rename, update wiki

* chore: various minor fixes, PR self review

* feat: added new overload in query controllers (#580)

* Cleaning repository of remaining business logic (#579)

* feat: first draft of attribute diffing in repo layer

* chore: remove EF Core-specific DetachRelationshipPointers from repository interface

* chore: replace SingleOrDefaultAsync with FirstOrDefaultAsync for increased performance

* chore: decoupled ResourceDefinition from repository layer in Filter implementation

* chore: remove updatevaluehelper

* fix: Can_Patch_Entity test

* fix: EntityResourceTest, variable name updatedFields -> targetedFields

* chore: add comment referencing to github issue

* chore: rename pageManager -> pageService

* chore: remove deprecations

* fix: undid wrong replace all of BeforeUpdate -> pageService

* chore: remove unreferenced method in ResponseSerializer

* Allow for multiple naming conventions (camelCase vs kebab-case) (#581)

* feat: draft of DI-registered ApplicationModelConvention

* feat: injectable IResourceNameFormatter and IJsonApiRoutingCOnvention now working

* test: kebab vs camelcase configuration

* feat: kebab vs camel completed

* chore: spacing

* style: spacing

* refactor: share logic between resource name formatter implementations

* docs: resource formatter comments

* docs: comments improved

* chore: remove static reference to ResourceNameFormatter

* chore: reviewer fixes

* style: removed spacing

* style: removed spacing

* fix: #583

* feat: throw error on deeply nested selections

* docs: add extra comment

* chore: processed Wisepotatos review

* Separation of concerns ResourceGraph (#586)

* refactor: move HasManyThrough GetValue and SetValue logic to HasManyThroughAttribute

* feat: remove GetPublicAttributeName from ResourceGraph and remove dependency on ResourceGraph of controller layer

* style: spacing in BaseJsonApiController<,>

* feat: remove GetEntityFromControllerName from ResourceGraph

* feat: remove GetInverseRelationship from ResourceGraph

* feat: decouples UsesDbContext from ResourceGraph and introduces decoupled application instantation

* chore: remove redundant injections of IResourceGraph

* refactor: merge IContextEntityProvider and IFieldsExplorer into IResourceGraphExplorer

* style: cleanup of variable names

* chore: various fixes for PR review, mostly style and variable naming

* style: rename IResourceGraphExplorer back to IResourceGraph, consitent usage of IContextEntityProvider

* docs: comments for DefaultActionFilter and DefaultTypeFilter

* docs: comments JsonApiApplicationBuilder

* style: consistent variable naming|

* chore: wire up IControllerResourceMapping

* refactor: routing convention implements controller mapping interface

* Update variable / class naming conventions (#589)



style: DefaultResourceRepository and DefaultEntityService as consie 833da6a
style: rename old repository interfaces 							519513d
style: ContextEntity --> ResourceContext 							d038582
style: rename Entity to Resource 									f2e411c
style: TEntity --> TResource 										3af6f9b
style: DependentType, PrincipalType --> RightType, LeftType 		ce08b4f
style: replaced principal with left and dependent with right whenev b162519
chore: remove redundant bindings throughout entire codebase 		e509956
refactor: regroupe hook container and executors 					48495d3
style: IGenericProcessor --> IHasManyThroughUpdateHelper 			f19e606
chore: delete rogue file 											7d235ab
style: rename GenericProcessorFactory --> GenericServiceFactory. 	6a186a5
chore: fix NoEntityFrameworkExample project							354a1be

* chore: process feedback

* Fix DiscoveryTests and NoEntityFrameworkExample project/tests (#590)

* fix: DiscoveryTests project

* fix: NoEntityFrameworkExample project + tests

* Simplify DefaultResourceService constructor (#592)

* refactor: simplify resource service constructor with IEnumerable<IQueryParameterService>
* feat: FirstOrDefault<TQueryParameterService>() extension method
* style: remove whitespace

* fix: fix build.sh

* fix: appveyor build

* fix: hotfix constructor ServiceDiscoveryFacadeTests

* upgrade to .net core 2.1

* .NET core 2.1 -> 2.2 (#594)

* merge: merge into correct email:

* feat: upgrade to 2.2

* chore: spacing fix

* chore: revert silencing error

* chore: small spacing fix

* chore: rm whitespace

* fix: public setters to allow testing

* .NET Core 2.2 -> 3.0  (#597)

* merge: merge into correct email:

* feat: upgrade to 2.2

* chore: spacing fix

* chore: revert silencing error

* chore: small spacing fix

* chore: remove unneeeded dependencies

* chore: fix framework dependencies

* chore: updates

* chore: test fixes

* chore: start integration tests

* chore: cleanup name of dertest

* chore: start of getRepostory

* chore: upgrade testsdk version + xunit version

* feat: use template in version control of packages in integrationTests

* feat: unit test assemblyinfo add, port old test to integration (attribute updater)

* docs: add version compatibility

* chore: add asterisk for version number

* feat: port defaultentityrepositorytests to integration tests

* chore: should add logging, done.

* tests: bad response from tests

* chore: fixes for lang version and making ifs more noticable

* fix: fix routing issue with upgrade to enableendpointrouting

* chore: HostEnvironment -> IWebHostEnvironment

* chore: act -> Act, assert -> Assert, arrange -> Arrange

* chore: fix tests

* chore: re-add unit test for defaultentityrepository for test IAsyncQueryProvider that's actually a List

* refactor: make if statements more conscise

* chore: final stretch

* tests: revert for good tests

* Revert "tests: revert for good tests"

This reverts commit 98b1119.

* Revert "chore: final stretch"

This reverts commit d6f763f.

* fix: client generated id tests

* fix: create data tests with workaround for efcore 3.0 bug

* refactor: DetachRelationships usage of is operator

* fix: move MetaStartup to other assembly as .net core 3 bug workaround

* fix: UpdatingRelationshipTests locally evaluated queries error

* feat: reintroduced generic processor (RepositoryRelationshipUpdateHelper) to fix locally evaluated expressions issue

* feat: full sql support for UpdateRelationshipsAsync using expression trees

* fix: DeletingDataTest

* fix: paging test

* fix: paging unit tests

* chore: remove RE-split models, duplicate models to NoEntityFrameworkExample

* fix: NoEntityFrameworkExample tests fixed by workaround assembly problem

* fix: connection string e2e test projects

* chore: minor cleanup

* chore: minor cleanup

* chore: cleanup

* chore: update travis.yml

* chore: retry travis.yml

* chore: retry .travis.yml

* chore: appveyor.yml

* fix: CI builds

* chore: attempt to fix travis.yml

* chore: spacing

* chore: postgres version bump

* chore: fix readme + downgrade postgres

* chore: revert
@maurei maurei deleted the feat/serializer-context-decoupling branch November 11, 2019 11:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

2 participants