[9.x] Add ability to generate unique validation rules per nested array element

[stevebauman]

Description

This PR introduces the ability to generate unique validation rules per nested array element, granting you direct access to each value supplied the request, the attribute name, and the whole data set in case other attribute values are needed.

This will be done using a new Rule static method: Rule::nested($callback).

Purpose

Generation of unique validation rules in nested arrays is currently pretty difficult, especially if you use any sort of custom validation rules. There is no built in way to generate validation rules taking into account the current iteration's value, and you'll have to set the indexes and such manually inside of a $rules array prior to sending it to the validator.

This also becomes quite problematic because we need to trust the data structure that was sent to us in the request and then iterate over it to dynamically generate our rules prior to actual validation taking place.

Example

Before:

$rules = [
    'companies.*.id' => [Rule::exists(Company::class, 'id')],
];

foreach ($request->companies as $index => $company) {
    $rules["companies.{$index}.id"] = [new HasPermission('manage-company', $company['id'])];
}

Validator::validate($request->all(), $rules);

After:

$rules = [
    'companies.*.id' => Rule::nested(function ($attribute, $value) {
        return [
            Rule::exists(Company::class, 'id'),
            new HasPermission('manage-company', $value)
        ];
    }),
];

Validator::validate($request->all(), $rules);

---

Another example where this is extremely handy is the validation of complex nested relationships:

$rules = [
    'companies.*.id' => Rule::nested(function ($attribute, $value) {
        return [
            Rule::exists(Company::class, 'id'),
            new HasPermission('manage-company', $value)
        ];
    })
    'companies.*.locations.*.id' => Rule::nested(function ($attribute, $value, $data) {
        // $attribute = 'companies.0.locations.0.id'
        $company = Str::before($attribute, '.locations');

        return Rule::exists(Locations::class, 'id')->where('company_id', $data["$company.id"]);
    }),
];

You may also supply an array of rules as you would normally with nested instances, as well as supply multiple nested instances:

$rules = [
    'companies.*.id' => [
        Rule::exists(Company::class, 'id'),
        Rule::nested(function ($attribute, $value) {
            return [new HasPermission('manage-company', $value)];
        }),
        Rule::nested(function ($attribute, $value) {
            // ...
        }),
    ],
];

Validation of arrays is super tricky stuff. I think this greatly simplifies the developer experience, allowing us not to think too much about its complexities.

---

Please let me know if you have any questions/comments/concerns. Thanks so much for your time! ❤️

[georgebohnisch]

Nice work, this is going to make validation in my project much simpler.

[driesvints]

Changing the method signatures here could be a breaking change.

[driesvints]

This could also be a breaking change.

[ionutantohi]

This might be a nice addition.

A bit offtopic, but the same issue exists with the validator messages. You still need to loop through your array, and set the correct error message. Would be nice if both can be supported.

This is my current approach now. Without this, the validator will output something like The companies.0.name field is required and we can't present that to the user.

$rules = [];
$messages = [];

foreach ($request->input('companies', []) as $index => $company) {
    $rules = array_merge($rules, [
        "companies.{$index}.name" => ['required'],
    ]);

    $messages = array_merge($messages, [
        "companies.{$index}.name.required" => 'Company ' . ($index + 1) . ': The name field is required.',
    ]);
}

$validator = Validator::make($request->all(), $rules, $messages);

[siarheipashkevich]

@stevebauman thank you for the great PR. But what about attributes for this rule?

How we should declare attributes for the nested arrays?

[siarheipashkevich]

Could anybody explain how I can use it with distinct rule?

I need check that the each item does not have duplicated discounts.

For the example below items.0.discounts.* - not valid

$data = [
    'items' => [
        [
            'id' => 1,
            'discounts' => [
                ['id' => 1, 'discount' => 10], // duplicate
                ['id' => 1, 'discount' => 10], // duplicate
                ['id' => 2, 'discount' => 20],
            ],
        ],
         [
            'id' => 2,
            'discounts' => [
                ['id' => 1, 'discount' => 10],
                ['id' => 2, 'discount' => 10],
            ],
        ],
    ],
];

$rules = [];

foreach ($request->get('items', []) as $index) {
    $rules["items.{$index}.discounts.*.id"] => ['distinct'];
}

Validator::make($request->all(), $rules);

[siarheipashkevich]

$this->data is poor array

$this->data[$attribute] is it valid if the attribute will have complex key like data.test ?

[stevebauman]

Any data that is set with the attribute's key will be supplied into the Rule::nested() callback.

So if it's an array, it will be an array. This line you mention allows you to place a nested instance on any validation field:

$rules = [
    'name' => Rule::nested(function ($attribute, $value) {
        // (string) "name"
        var_dump($attribute);
        
        // (string) "Taylor Otwell"
        var_dump($value);
    }),

    'data.test' => Rule::nested(function ($attribute, $value) {
        // (string) "data.test"
        var_dump($attribute);
        
        // (string) "Foo"
        var_dump($value);
    }),
];

Validator::validate([
    'name' => 'Taylor Otwell',
    'data' => ['test' => 'Foo'],
], $rules);

The third argument inside of the Rule::nested() callback is only available when the attribute under validation is a wildcard array.

[amaelftah]

i liked this idea nice work steve 💪

[siarheipashkevich]

Could anybody explain how I can use it with distinct rule?

I need check that the each item does not have duplicated discounts.

For the example below items.0.discounts.* - not valid

$data = [
    'items' => [
        [
            'id' => 1,
            'discounts' => [
                ['id' => 1, 'discount' => 10], // duplicate
                ['id' => 1, 'discount' => 10], // duplicate
                ['id' => 2, 'discount' => 20],
            ],
        ],
         [
            'id' => 2,
            'discounts' => [
                ['id' => 1, 'discount' => 10],
                ['id' => 2, 'discount' => 10],
            ],
        ],
    ],
];

$rules = [];

foreach ($request->get('items', []) as $index) {
    $rules["items.{$index}.discounts.*.id"] => ['distinct'];
}

Validator::make($request->all(), $rules);

@stevebauman will it covered and how?

[stevebauman]

@ionutantohi

Yea that's true. I'm not sure we can get around this unless we implement some sort of unified system where rules, messages, and attributes can handle a closure/nested instance...

@siarheipashkevich

Currently for your posted example nested() won't help you, because we can't alter the rule array dynamically. Maybe we could have something like this API?

$rules = [
    'items.*' => Rule::nested(function ($attribute) {
        $index = Str::afterLast($attribute, '.');

        return ["{$index}.discounts.*.id" => 'distinct'];
    }),
];

// After rule parsing:
// [
//     'items.0.discounts.*.id' => ['distinct'],
//     'items.1.discounts.*.id' => ['distinct'],
// ]

The above is not supported in this PR, but I think supporting this use-case would grant us even finer control over array validation rules.

[siarheipashkevich]

@ionutantohi

Yea that's true. I'm not sure we can get around this unless we implement some sort of unified system where rules, messages, and attributes can handle a closure/nested instance...

@siarheipashkevich

Currently for your posted example nested() won't help you, because we can't alter the rule array dynamically. Maybe we could have something like this API?

$rules = [
    'items.*' => Rule::nested(function ($attribute) {
        $index = Str::afterLast($attribute, '.');

        return ["{$index}.discounts.*.id" => 'distinct'];
    }),
];

// After rule parsing:
// [
//     'items.0.discounts.*.id' => ['distinct'],
//     'items.1.discounts.*.id' => ['distinct'],
// ]

The above is not supported in this PR, but I think supporting this use-case would grant us even finer control over array validation rules.

Rule::nested can returns the array where item has key as name of the attribute and value is rules?

$rules = [
    'items.*' => Rule::nested(function ($attribute) {
        $index = Str::afterLast($attribute, '.');

        return ["{$index}.discounts.*.id" => 'distinct'];
    }),
];

Will it converted to?

$rules = [
     'items.0.discounts.*.id' => ['distinct'],
     'items.1.discounts.*.id' => ['distinct'],
];

[stevebauman]

@siarheipashkevich

Rule::nested can returns the array where item has key as name of the attribute and value is rules?

No, not currently. I'm saying that it may be possible to be able to.

I've converted this PR to a draft while I attempt to make this possible. As @driesvints points out -- this may need to be sent to the 9.x branch due to breaking changes in method signatures.

[bakerkretzmar]

Love this, had to do it manually on a client project recently and it was not pretty 😅

[siarheipashkevich]

Could you please add an example where I have additional columns in discounts array, for example (type: percent, absolute; and discount: numeric value), not one rule for id

[stevebauman]

Sure thing @siarheipashkevich! I've added another test so you can see this in action: a38ddf7

Here's the example:

$data = [
    'items' => [
        [
            'discounts' => [
                ['id' => 1, 'percent' => 30, 'discount' => 1400],
                ['id' => 1, 'percent' => -1, 'discount' => 12300],
                ['id' => 2, 'percent' => 120, 'discount' => 1200],
            ]
        ],
        [
            'discounts' => [
                ['id' => 1, 'percent' => 30, 'discount' => 'invalid'],
                ['id' => 2, 'percent' => 'invalid', 'discount' => 1250],
                ['id' => 3, 'percent' => 'invalid', 'discount' => 'invalid'],
            ],
        ],
    ],
];

$rules = [
    'items.*' => Rule::nested(function () {
        return [
            'discounts.*.id' => 'distinct',
            'discounts.*' => Rule::nested(function () {
                return [
                    'percent' => 'numeric|min:0|max:100',
                    'discount' => 'numeric',
                ];
            }),
        ];
    }),
];

[siarheipashkevich]

Thanks a lot @stevebauman !

One more question, how you validate discount value by discount type in our nested example above?

If the discount type equals percent then the discount value should be between 0 and 100.

If the discount type is equals absolute then the discount value should be any valid numeric value.

I will be grateful for your example.

[stevebauman]

No worries @siarheipashkevich! Thanks for the great examples to display 👍

I just tested this, here's how you could write it using Rule::nested():

$data = [
    'items' => [
        [
            'discounts' => [
                ['id' => 1, 'type' => 'percent', 'discount' => 120],
                ['id' => 1, 'type' => 'absolute', 'discount' => 100],
                ['id' => 2, 'type' => 'percent', 'discount' => 50],
            ],
        ],
        [
            'discounts' => [
                ['id' => 2, 'type' => 'percent', 'discount' => 'invalid'],
                ['id' => 3, 'type' => 'absolute', 'discount' => 2000]
            ],
        ],
    ],
];

$rules = [
    'items.*' => Rule::nested(function () {
        return [
            'discounts.*.id' => 'distinct',
            'discounts.*.type' => 'in:percent,absolute',
            'discounts.*' => Rule::nested(function ($attribute, $value) {
                return $value['type'] === 'percent' 
                    ? ['discount' => 'numeric|min:0|max:100']
                    : ['discount' => 'numeric'];
            }),
        ];
    }),
];

[taylorotwell]

@stevebauman can you target this at the 9.x branch?

[stevebauman]

Yup absolutely! Give me 10 minutes 👍

[stevebauman]

Ok @taylorotwell -- all good to go!

[taylorotwell]

A bit offtopic, but the same issue exists with the validator messages. You still need to loop through your array, and set the correct error message. Would be nice if both can be supported.

This is my current approach now. Without this, the validator will output something like The companies.0.name field is required and we can't present that to the user.

@ionutantohi personally I would just handle this by putting something in your validation.php language file. It's hard to let any validator just figure out what you want there.

[taylorotwell]

@stevebauman one thing that came to mind here was this fairly recent contribution: https://laravel.com/docs/8.x/validation#complex-conditional-array-validation

As you can see, it receives the entire array item for the current index. Is that useful at all in this scenario as well?

[stevebauman]

@taylorotwell It's certainly useful, but from what I understand I'm not able to segment rules so that distinct looks only in the parent array and not in the whole data array. For example:

$data = [
    'companies' => [
        ['locations' => [['id' => 1], ['id' => 2]]],
        ['locations' => [['id' => 1], ['id' => 2]]],
    ],
];

// Does not work -- sees duplicates across arrays:
$rules = [
    'companies.*.locations.*.id' => ['distinct', 'exists:locations,id'],
];

// Works:
$rules = [
    'companies.*' => Rule::nested(function () {
        return ['locations.*.id' => ['distinct', 'exists:locations,id']];
    }),
];

[stevebauman]

My use case in detail: I have a user creation form where a user can belong to multiple companies. Inside each company, they have to be associated to at least one location owned by that company. I could perform this pretty complex validation like so:

$data = [
    'companies' => [
        ['id' => 1, 'locations' => [['id' => 1], ['id' => 2]]],
        ['id' => 2, 'locations' => [['id' => 1], ['id' => 2]]],
    ],
];

$rules = [
    'companies' => ['required', 'array'],
    'companies.*.id' => ['required', 'distinct', Rule::exists(Company::class, 'id')],
    'companies.*' => Rule::nested(function ($attribute, $company) {
        return [
            'locations' => 'required',
            'locations.*' => ['required', 'array'],
            'locations.*.id' => [
                'required',
                'distinct',
                Rule::exists(Location::class, 'id')->where('company_id', $company['id']),
            ]
        ];
    }),
];

Maybe we could utilize Fluent for the callback arguments as well to follow suite with the complex conditional validation you've shared above? What are your thoughts?

[ionutantohi]

@ionutantohi personally I would just handle this by putting something in your validation.php language file. It's hard to let any validator just figure out what you want there.

@taylorotwell Thanks for your suggestion, but unfortunately i can't do this on my end, because that error message will be duplicated (and the same) for all items in the array

$input = [
    'companies' => [
        [
            'name' => 'Foo'
        ],
        [
            'name' => null
        ],
        [
            'name' => null
        ]
    ]
];

Using your suggestion the validator will produce the below, without any indication to which item it refers to

Company names must be valid, non-empty string.
Company names must be valid, non-empty string.

There was a pr #35754 which introduced a new placeholder which might help in this case.

That's it, I don't want to pollute this pr anymore

[stevebauman]

Sorry @taylorotwell -- I previously changed the PR base from 8.x to 9.x by simply selecting it in GitHub. I've just now rebased this PR branch to 9.x. Can you give it another shot?

[taylorotwell]

That messed it up further. Can you undo that? It was actually fine before.

[stevebauman]

Whoops -- okay I messed this PR up moving from branch to branch. Going to close this and re push this from scratch, my bad! 😅