Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sessionctx/variable: avoid SysVar clone every time when visiting system variable #26308

Merged
merged 6 commits into from
Jul 19, 2021

Conversation

tiancaiamao
Copy link
Contributor

@tiancaiamao tiancaiamao commented Jul 16, 2021

What problem does this PR solve?

Problem Summary:

The current implementation for system variable is not good. To avoid DATA RACE in the unit test, it Clone() a new object every time.

When I handle #25573, I find it cause unnecessary allocation (every query needs to visit the session variable, and it's copy on read)

What is changed and how it works?

What's Changed:

Change SetSysVar implementation to removing the object from the map and then register a new object...

How it Works:

SysVar are almost immutable ... Here I say almost, I mean after initialized, they are not changed any more.

So we do not really need to Clone to avoid data race.

We can register this SysVar again, the map is protected under lock, and the SysVar is immutable, it do not need a lock or clone.

This change avoid the copy-on-read and is better for performance.

Check List

Tests

  • No code
cd session
go test -test.run TestXXX -test.bench BenchmarkPreparedPointGet -memprofile mem.out -benchmem -benchtime 30s

Before vs After:

BenchmarkPreparedPointGet-16    	 4316589	      8428 ns/op	    6510 B/op	      81 allocs/op
BenchmarkPreparedPointGet-16    	 4369659	      8196 ns/op	    6317 B/op	      80 allocs/op

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

  • No release note

@tiancaiamao tiancaiamao added type/enhancement The issue or PR belongs to an enhancement. high-performance labels Jul 16, 2021
@tiancaiamao tiancaiamao requested a review from morgo July 16, 2021 11:17
@ti-chi-bot ti-chi-bot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Jul 16, 2021
@tiancaiamao
Copy link
Contributor Author

/run-check_dev_2

1 similar comment
@tiancaiamao
Copy link
Contributor Author

/run-check_dev_2

@morgo
Copy link
Contributor

morgo commented Jul 16, 2021

@tiancaiamao LGTM, but there is another possible fix I wanted to run by you:

The only scenario where Set is required is in changing the value of None-scoped variables (this is a very small amount of the total scope). We could instead move the values of read-only sysvars to a map, and use copy-on-write here.

@tiancaiamao
Copy link
Contributor Author

This is the map:

// key -> value
tidb_transaction_isolation -> SysVar{Global | Session , Value: xxx}   <-- Value in SysVar actually means Default Value, 
tidb_enable_async_commit -> SysVar{Global | Session, Value: xxx}      <-- For global | session, SysVar & SysVar.Value never changes
tidb_tidb_slow_log_masking -> SysVar{}         <-- most of the system variables are initialized at init()
tidb_partition_prune_mode -> SysVar{}
tidb_server_version -> SysVar{ScopeNone, Value: xxx}       <-- None Scope, Immutable after initialize, some are initialized at SetSysVar()
...

The current code in this PR is using a lock to protect the map itself, and the SysVar objects in the map are immutable ...

We could instead move the values of read-only sysvars to a map, and use copy-on-write here.

I don't quite get your point. copy-on-write do you mean create a new map to replace the old one? @morgo
SetSysVar need to modify the map anyway, you can't copy a SysVar from the map and modify the copied SysVar value.

@ti-chi-bot ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label Jul 19, 2021
@tiancaiamao tiancaiamao requested a review from xhebox July 19, 2021 09:37
@ti-chi-bot
Copy link
Member

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • morgo
  • xhebox

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@ti-chi-bot ti-chi-bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Jul 19, 2021
@xhebox
Copy link
Contributor

xhebox commented Jul 19, 2021

/merge

@ti-chi-bot
Copy link
Member

This pull request has been accepted and is ready to merge.

Commit hash: a90c3c8

@ti-chi-bot ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Jul 19, 2021
@ti-chi-bot
Copy link
Member

@tiancaiamao: Your PR was out of date, I have automatically updated it for you.

At the same time I will also trigger all tests for you:

/run-all-tests

If the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@ti-chi-bot ti-chi-bot merged commit a542c58 into pingcap:master Jul 19, 2021
@tiancaiamao tiancaiamao deleted the optimize-sysvar branch July 19, 2021 10:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
size/M Denotes a PR that changes 30-99 lines, ignoring generated files. status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2. type/enhancement The issue or PR belongs to an enhancement.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants