Skip unknown settings with warnings.

[vitlibar]

I hereby agree to the terms of the CLA available at: https://yandex.ru/legal/cla/?lang=en

Changelog category:

• New feature

Changelog entry:
Allow skipping unknown settings with warnings instead of throwing exceptions.

[alexey-milovidov]

I prefer to simply pass all settings as strings in new format. This will make code easier.

[vitlibar]

I prefer to simply pass all settings as strings in new format. This will make code easier.

Done.

[nvartolomei]

Current behaviour caused us many headaches, but in my opinion this approach is far from ideal. I don't think the server should be the one deciding on silently (from client's perspective) ignore settings.

This might seem like a personal preference, but I'll try and prove my point by going through some examples for which this approach might be fatal.

totals_mode
distributed_product_mode
join_use_nulls
extremes
Suppose these (or similar settings which change query behaviour/result) are introduced in a new version which is not yet fully deployed on a cluster. A user might use them and get wrong results without realising it (in theory we can forbid adding new setting that changes query behaviour and instead add them at query syntax level).

Also, this PR doesn't seem complete: it should probably allow to ignore unknown settings values as well.

A better solution in my opinion would be for the server to expose list of supported settings (optionally, supported values), and let the client decide whether or not these settings are enough to execute the query.

[filimonov]

I think that changing way of serialization and have a possibility to ignore unknown settings is good .

At the same time the decision "to ignore / not ignore" particular (or all) settings should be controlled by user IMO (from both ends - client or server).

So client can send smth like 'ignore_unknown_settings=0' (by default), can be set to 1 during rolling updates in cluster, or even "ignore_unknown_settings=totals_mode,distributed_product_mode" (will be ignored only is server does not support them).

Or server admin can decide that he doesn't care if newer clients send something like 'mimic_old_version_behavior=true'.

[vitlibar]

A better solution in my opinion would be for the server to expose list of supported settings (optionally, supported values), and let the client decide whether or not these settings are enough to execute the query.

I'll try to implement a slightly different solution. The client will send can_ignore flag with every sent value and this flag will be calculated based on the setting and its value. If the server doesn't know this setting it will use this flag to decide what to do.

So client can send smth like 'ignore_unknown_settings=0' (by default), can be set to 1 during updates in cluster, or even "ignore_unknown_settings=totals_mode,distributed_product_mode" (will be ignored only is server does not support them).

It seems the client will be able to calculate that can_ignore flag without extra settings.

[filimonov]

It seems the client will be able to calculate that can_ignore flag without extra settings.

Does it mean that the decision allowing / forbidding ignorance will be hardcoded?
What if i prefer not to ignore that?

[vitlibar]

It seems the client will be able to calculate that can_ignore flag without extra settings.

Does it mean that the decision allowing / forbidding ignorance will be hardcoded?
What if i prefer not to ignore that?

The names and meanings of the settings are hardcoded, so it seems the ignorance flag should be hardcoded too. The developers of new settings will decide whether they can be ignored or not.