Add log_queries_min_type to filter which entries will be written to query_log

[azat]

Changelog category (leave one):

• New Feature

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):
Add log_queries_min_type to filter which entries will be written to query_log

Detailed description / Documentation draft:
Can be used to write into query_log only failed queries (i.e. on memory
exceeded error), by using:

  set log_queries_min_type='EXCEPTION_WHILE_PROCESSING'

[alexey-milovidov]

There is no assumed order in values of type.
It's only a coincidence that it looks useful.

[alexey-milovidov]

Current solution does not look practical.

[azat]

Current solution does not look practical.

Ok, how about making enum of the following possible values in settings?

• all (query + exception)
• query (query_start + query_finish)
• query_start
• query_finish
• exception (exception_before_start + exception_before_processing)
• exception_before_start
• exception_before_processing

[alexey-milovidov]

Ok, how about making enum of the following possible values in settings?

Almost Ok. We have to made setting values consistent with Enum values in system.query_log (QueryFinish instead of query_finish).
But this solution looks too heavy for this task and it will be over complicated. Need to think more...
And it's strange if I cannot simply list all values that I want to see.

If the user really wants to get this feature, they can use Null engine for system.query_log and create a meterialized view on top of it. Maybe it's enough?

[alexey-milovidov]

PS. Please don't worry about "spam" assigments by Ivan Blinkov. He has written a script that assigns pull request to random people. We decided that we will remove this script but Ivan has not stopped it yet.

[azat]

If the user really wants to get this feature, they can use Null engine for system.query_log and create a meterialized view on top of it. Maybe it's enough?

For some uses cases - yes, but it will not allow to change the type without recreating the table.

Why I found this useful is:

• you can limit amount of information that goes to query_log (excluding QueryStart or even enable Exception* on prod globally)
• you still want sometimes to debug the query completely (i.e. write all stages).
• this can be more-or-less hot path (for example if you set it to Exception* then non will be written unless you have some errors, but with views it will need to filter blocks always, maybe this will be insignificant, but anyway does not sounds like the thing that you should do on production globally to me).

PS. Please don't worry about "spam" assigments by Ivan Blinkov. He has written a script that assigns pull request to random people. We decided that we will remove this script but Ivan has not stopped it yet.

P.S. I knew it! (since it tags the PR as soon as it has been submitted).

[azat]

And it's strange if I cannot simply list all values that I want to see.

I can add parsing of a string, i.e. log_queries_type="QUERY_FINISH EXCEPTION_WHILE_PROCESSING", but this also does not looks good (at least to me), and making this at least more convenient to user I guess (but I'm open to any suggestions, maybe you know better ways to do this...)

P.S. at first I wanted to log only errors, but when I was writing the patch I decided to excluding say QUERY_START also can be useful

[alexey-milovidov]

Ok.

Let's make it as simple as possible.
Let's name the setting log_queries_min_type and it will take numeric values. We will not validate these values. And simply apply this setting by comparison.

Reason: this will make the implementation of this setting obvious (that it compares numeric values of type).