Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mvcc: don't block on concurrent reads #7083

Closed
heyitsanthony opened this issue Dec 30, 2016 · 4 comments
Closed

mvcc: don't block on concurrent reads #7083

heyitsanthony opened this issue Dec 30, 2016 · 4 comments
Assignees
Milestone

Comments

@heyitsanthony
Copy link
Contributor

Right now mvcc must acquire a heavily contended mutex to make Range requests. For better scalability, protect it without taking the mutex.

@xiang90 xiang90 added this to the v3.2.0 milestone Dec 31, 2016
@heyitsanthony heyitsanthony self-assigned this Jan 3, 2017
@vimalk78
Copy link
Contributor

vimalk78 commented Jan 3, 2017

currently a Range request acquires

  • a read lock in mvcc.kvindex
  • a lock in mvcc.store
  • a lock in mvcc.backend.batchTx

are you planning to remove all three locks?

can you pls share your idea ?

@heyitsanthony
Copy link
Contributor Author

@vimalk78 mvcc.store and mvcc.backend.batchTx. There's already a RWMutex in mvcc.kvindex, so there's no read contention. There's going to be a separate transactional view for reads. This is related to getting the performance back for #6876.

@vimalk78
Copy link
Contributor

vimalk78 commented Jan 4, 2017

the mvcc.store.mu should be change to RWMutex, cannot completely remove it from Range path.

for reading the boltdb, do you plan to hold a single ReadOnly transaction, or create a new ReadOnly transaction for every Range request, or use DB.View method. perhaps using DB.View will be better as we dont have to deal with transactions directly and create/close will be done by boltdb.

@xiang90
Copy link
Contributor

xiang90 commented Jan 4, 2017

@vimalk78

xiang90@1fcc012

This is an example for what we should start with. But note that the readTx needs to capture the changes happened in batchTx before it. So we need to have a shared in-mem map in front of these txs.

@heyitsanthony might provide more thoughts on that.

heyitsanthony pushed a commit to heyitsanthony/etcd that referenced this issue Jan 5, 2017
Clean-up of the mvcc interfaces to use txn interfaces instead of an id.

Adds support for concurrent read-only mvcc transactions.

Fixes etcd-io#7083
heyitsanthony pushed a commit to heyitsanthony/etcd that referenced this issue Jan 5, 2017
Clean-up of the mvcc interfaces to use txn interfaces instead of an id.

Adds support for concurrent read-only mvcc transactions.

Fixes etcd-io#7083
heyitsanthony pushed a commit to heyitsanthony/etcd that referenced this issue Jan 7, 2017
Clean-up of the mvcc interfaces to use txn interfaces instead of an id.

Adds support for concurrent read-only mvcc transactions.

Fixes etcd-io#7083
heyitsanthony pushed a commit to heyitsanthony/etcd that referenced this issue Feb 13, 2017
Clean-up of the mvcc interfaces to use txn interfaces instead of an id.

Adds support for concurrent read-only mvcc transactions.

Fixes etcd-io#7083
heyitsanthony pushed a commit to heyitsanthony/etcd that referenced this issue Feb 17, 2017
Clean-up of the mvcc interfaces to use txn interfaces instead of an id.

Adds support for concurrent read-only mvcc transactions.

Fixes etcd-io#7083
heyitsanthony pushed a commit to heyitsanthony/etcd that referenced this issue Feb 18, 2017
Clean-up of the mvcc interfaces to use txn interfaces instead of an id.

Adds support for concurrent read-only mvcc transactions.

Fixes etcd-io#7083
heyitsanthony pushed a commit to heyitsanthony/etcd that referenced this issue Feb 22, 2017
Clean-up of the mvcc interfaces to use txn interfaces instead of an id.

Adds support for concurrent read-only mvcc transactions.

Fixes etcd-io#7083
heyitsanthony pushed a commit to heyitsanthony/etcd that referenced this issue Feb 22, 2017
Clean-up of the mvcc interfaces to use txn interfaces instead of an id.

Adds support for concurrent read-only mvcc transactions.

Fixes etcd-io#7083
heyitsanthony pushed a commit to heyitsanthony/etcd that referenced this issue Feb 23, 2017
Clean-up of the mvcc interfaces to use txn interfaces instead of an id.

Adds support for concurrent read-only mvcc transactions.

Fixes etcd-io#7083
heyitsanthony pushed a commit to heyitsanthony/etcd that referenced this issue Feb 24, 2017
Clean-up of the mvcc interfaces to use txn interfaces instead of an id.

Adds support for concurrent read-only mvcc transactions.

Fixes etcd-io#7083
heyitsanthony pushed a commit to heyitsanthony/etcd that referenced this issue Feb 24, 2017
Clean-up of the mvcc interfaces to use txn interfaces instead of an id.

Adds support for concurrent read-only mvcc transactions.

Fixes etcd-io#7083
heyitsanthony pushed a commit to heyitsanthony/etcd that referenced this issue Feb 25, 2017
Clean-up of the mvcc interfaces to use txn interfaces instead of an id.

Adds support for concurrent read-only mvcc transactions.

Fixes etcd-io#7083
heyitsanthony pushed a commit to heyitsanthony/etcd that referenced this issue Feb 27, 2017
Clean-up of the mvcc interfaces to use txn interfaces instead of an id.

Adds support for concurrent read-only mvcc transactions.

Fixes etcd-io#7083
heyitsanthony pushed a commit to heyitsanthony/etcd that referenced this issue Feb 27, 2017
Clean-up of the mvcc interfaces to use txn interfaces instead of an id.

Adds support for concurrent read-only mvcc transactions.

Fixes etcd-io#7083
heyitsanthony pushed a commit to heyitsanthony/etcd that referenced this issue Feb 27, 2017
Clean-up of the mvcc interfaces to use txn interfaces instead of an id.

Adds support for concurrent read-only mvcc transactions.

Fixes etcd-io#7083
heyitsanthony pushed a commit to heyitsanthony/etcd that referenced this issue Mar 1, 2017
Clean-up of the mvcc interfaces to use txn interfaces instead of an id.

Adds support for concurrent read-only mvcc transactions.

Fixes etcd-io#7083
heyitsanthony pushed a commit to heyitsanthony/etcd that referenced this issue Mar 1, 2017
Clean-up of the mvcc interfaces to use txn interfaces instead of an id.

Adds support for concurrent read-only mvcc transactions.

Fixes etcd-io#7083
heyitsanthony pushed a commit to heyitsanthony/etcd that referenced this issue Mar 1, 2017
Clean-up of the mvcc interfaces to use txn interfaces instead of an id.

Adds support for concurrent read-only mvcc transactions.

Fixes etcd-io#7083
heyitsanthony pushed a commit to heyitsanthony/etcd that referenced this issue Mar 1, 2017
Clean-up of the mvcc interfaces to use txn interfaces instead of an id.

Adds support for concurrent read-only mvcc transactions.

Fixes etcd-io#7083
heyitsanthony pushed a commit to heyitsanthony/etcd that referenced this issue Mar 3, 2017
Clean-up of the mvcc interfaces to use txn interfaces instead of an id.

Adds support for concurrent read-only mvcc transactions.

Fixes etcd-io#7083
heyitsanthony pushed a commit to heyitsanthony/etcd that referenced this issue Mar 4, 2017
Clean-up of the mvcc interfaces to use txn interfaces instead of an id.

Adds support for concurrent read-only mvcc transactions.

Fixes etcd-io#7083
heyitsanthony pushed a commit to heyitsanthony/etcd that referenced this issue Mar 4, 2017
Clean-up of the mvcc interfaces to use txn interfaces instead of an id.

Adds support for concurrent read-only mvcc transactions.

Fixes etcd-io#7083
heyitsanthony pushed a commit to heyitsanthony/etcd that referenced this issue Mar 6, 2017
Clean-up of the mvcc interfaces to use txn interfaces instead of an id.

Adds support for concurrent read-only mvcc transactions.

Fixes etcd-io#7083
heyitsanthony pushed a commit to heyitsanthony/etcd that referenced this issue Mar 8, 2017
Clean-up of the mvcc interfaces to use txn interfaces instead of an id.

Adds support for concurrent read-only mvcc transactions.

Fixes etcd-io#7083
heyitsanthony pushed a commit to heyitsanthony/etcd that referenced this issue Mar 8, 2017
Clean-up of the mvcc interfaces to use txn interfaces instead of an id.

Adds support for concurrent read-only mvcc transactions.

Fixes etcd-io#7083
heyitsanthony pushed a commit to heyitsanthony/etcd that referenced this issue Mar 9, 2017
Clean-up of the mvcc interfaces to use txn interfaces instead of an id.

Adds support for concurrent read-only mvcc transactions.

Fixes etcd-io#7083
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

No branches or pull requests

3 participants