Feat/multi data model with validation #730
Open
+10,439
−4,445
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
… various tests and minor issues
ivarne
force-pushed
the
feat/multi-data-model-with-validation
branch
from
c2bfbae
to
f7b91e0
Compare
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
src/Altinn.App.Core/Features/Validation/Wrappers/DataElementValidatorWrapper.cs
Dismissed
Show dismissed
Hide dismissed
test/Altinn.App.Core.Tests/LayoutExpressions/TestUtilities/DynamicClassBuilderChatGPTTests.cs
Fixed
Show fixed
Hide fixed
test/Altinn.App.Core.Tests/LayoutExpressions/TestUtilities/DynamicClassBuilderChatGPTTests.cs
Dismissed
Show dismissed
Hide dismissed
test/Altinn.App.Core.Tests/LayoutExpressions/TestUtilities/DynamicClassBuilderTests.cs
Dismissed
Show dismissed
Hide dismissed
ivarne
force-pushed
the
feat/multi-data-model-with-validation
branch
from
1703585
to
9007d42
Compare
src/Altinn.App.Core/Internal/Expressions/LayoutEvaluatorStateInitializer.cs
Fixed
Show fixed
Hide fixed
…before CreateAppModelData call
Make expression engine async
* Unused usings * Mark members static Much easier to see theese issues before sonar cloud runs
Make it easier to get 0 sonar cloud warnings by making its issues warnings
ivarne
commented
Sep 10, 2024
src/Altinn.App.Core/Models/Validation/ValidationIssueWithSource.cs
Outdated
Show resolved
Hide resolved
| /// <summary> | ||
| /// Initialize LayoutEvaluatorState with given Instance, data object and layoutSetId | ||
| /// </summary> | ||
| public virtual Task<LayoutEvaluatorState> Init( | ||
| [Obsolete("Use the overload with ILayoutEvaluatorStateInitializer instead")] | ||
| public async Task<LayoutEvaluatorState> Init( |
test/Altinn.App.Core.Tests/Features/Validators/ValidationServiceTests.cs
Fixed
Show fixed
Hide fixed
src/Altinn.App.Core/Internal/Process/ProcessTasks/Common/ProcessTaskFinalizer.cs
Show resolved
Hide resolved
Some validators can neither run on every request for efficeincy, nor has a sensible way to implement HasRelevantChanges. Theese validators need to only run on process/next, or when explicitly requested in a call to /validate Also added NoIncrementalUpdates (with [JsonIgnore(WhenWritingDefault)]) to ValidationIssueWithSource, so that FE can distinguish what issues won't be fixed with incremental updates. /validate now supports a boolean to run either only incremental, only non incremnetal or all validators (default)
src/Altinn.App.Core/Features/Validation/Wrappers/FormDataValidatorWrapper.cs
Fixed
Show fixed
Hide fixed
…new dataProcessor
…n dataProcessWrite
* Use IInstanceDataMutator for actions
* Return List instead of dictionary for models and validation issues in MultipleDataPatch for improved swagger doc
* New IDataWriteProcessor interface with IInstanceDataMutator and change list to mimik IValidator
* IValidator gets ShouldRunForTask(string taskId) so that
* Required validators only runs for tasks with layout.
* Expression validators only runs for tasks with datatypes that has expressions validation files
* Revert DataElementIdentifier to a simple wrapper for Guid (that caches the string version)
Add IInstanceDataModifier and use it for a new dataProcessing interface and extend the UserActionContext to support modifying data elements.
Comment on lines
+80
to
+91
| foreach (var dataElement in formDataElementsForTask) | ||
| { | ||
| var issues = await ValidateFormData( | ||
| instance, | ||
| dataElement, | ||
| instanceDataAccessor, | ||
| validationConfig, | ||
| taskId, | ||
| language | ||
| ); | ||
| validationIssues.AddRange(issues); | ||
| } |
Check notice
Code scanning / CodeQL
Missed opportunity to use Select Note
src/Altinn.App.Core/Helpers/Serialization/ModelSerializationService.cs
Dismissed
Show dismissed
Hide dismissed
| "Error while running validator {validatorName} on task {taskId} in instance {instanceId}", | ||
| validator.GetType().Name, | ||
| taskId, | ||
| instance.Id |
Check failure
Code scanning / CodeQL
Log entries created from user input High
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix AI 40 minutes ago
To fix the problem, we need to sanitize the instance.Id before logging it. Since instance.Id is a GUID, we can ensure it does not contain any special characters or new lines by converting it to a string and replacing any new line characters. This approach will prevent log forging by ensuring that the logged value is safe.
Suggested changeset
1
src/Altinn.App.Core/Internal/Validation/ValidationService.cs
| @@ -147,6 +147,6 @@ | ||
| e, | ||
| "Error while running validator {validatorName} on task {taskId} in instance {instanceId}", | ||
| "Error while running validator {validatorName} on task {taskId} in instance {sanitizedInstanceId}", | ||
| validator.GetType().Name, | ||
| taskId, | ||
| instance.Id | ||
| instance.Id.Replace(Environment.NewLine, "") | ||
| ); |
Copilot is powered by AI and may make mistakes. Always verify output.
test/Altinn.App.Core.Tests/Infrastructure/Clients/Storage/TestData/ExampleModel.cs
Dismissed
Show dismissed
Hide dismissed
src/Altinn.App.Core/Infrastructure/Clients/Storage/DataClient.cs
Dismissed
Show dismissed
Hide dismissed
ivarne
force-pushed
the
feat/multi-data-model-with-validation
branch
from
3fb19e0
to
2ae6862
Compare
Open
19 tasks
also a few final touches and possible test fixes
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Work on supporting multiple data models and subforms in backend validations with expressions.
Related Issue(s)
Main changes
The main goal for this PR is to add
New single interface for validation
IValidatorWith changes in one data element affects the validity of another element, the current hierarchy of validators and only
IFormDataValidatorsupporting incremental validation break down. Adaptors for the old interfaces has been provided so no changes to current apps are required, but they might want to update in order to support incremental validation and read instance data with theIInstanceDataAccessor.New interface
IInstanceDataAccessorWhen validation has dependencies across data models, we need a cached way to access the content of data elements that does not require us to download the same data element multiple times for the same request. The implementation is not available from DI, but always passed as an argument in order to eliminate risk of it being reused cross requests because of Singleton services or other unintentionally shared global state.
New
DataElementIdstruct to ensure type safety for a wrapped Guid36 places in the updated code need to reference the Guid of a data element, and adding type safety is an important step to ensure correctness. An implicit cast is provided from
DataElementto simplify usageNew
ModelBindingstruct in expressions engineTo support the new syntax for referring to a non default data element in components and expressions
{ "dataModelBindings": {"field": "path.to.data", "dataType": "default"} "hidden": ["equals", null, ["dataModel", "path.to.other", "prefill"]] }When handling references to a specific fields (eg. for removal) we needed to also add
DataReferenceto combine a Field (path) with a specific DataElementIdNew PATCH api endpoint that allows frontend to update multiple data models at the same time
Saving and triggering incremental validation is expensive and some apps might want to batch multiple changes to reduce cost. The code in frontend would also be simplified if it could do a single request, instead of multiple.
ValidationIssueWithSourceclass for external apisPreviously we had a single class
ValidationIssuethat included aSourceproperty. Service owners sometimes tries to set this in their validators (because it shows up in intellisense), but as we override it based onIValidator.ValidationSourcein ourValidationServiceit would be good if it wasn't there to confuse. This PR creates a new typeValidationIssueWithSourcefor external apis (where the source is included) and marks it[Obsolete]in the type returned in the validator interfaces.Respect new app logic settings for data elements
DisallowUserCreateandDisallowUserDeleteThis means that users can not create or delete this types. Typically used for prefill, when you want only the org token to be able to modify the number of data elements on an instance.
Minor changes
Used IL code to generate real C# classes based on the json data models in shared tests. Previously different code was used to access properties of the test code as in production, which is not really good for tests.
Verification
Documentation