Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[DocDB] Perf: Avoid unnecessary work being done in Tablet::PrepareTransactionWriteBatch #11409

Closed
rthallamko3 opened this issue Feb 9, 2022 · 0 comments
Closed

[DocDB] Perf: Avoid unnecessary work being done in Tablet::PrepareTransactionWriteBatch #11409

rthallamko3 opened this issue Feb 9, 2022 · 0 comments
Assignees
@spolitov
Labels
area/docdb YugabyteDB core features

Comments

@rthallamko3
Copy link
Contributor

Description

In Yugabyte, the data is not written to the RocksDbs WAL, however when it comes to writing data to the RocksDb layer, there are additional steps of serializing to the WAL format which is unnecessary and leads to wasted work. The time spent in Tablet::PrepareTransactionWriteBatch() can be considerably reduced by eliminating this additional serialization/deserialization step.
@rthallamko3 rthallamko3 added the area/docdb YugabyteDB core features label Feb 9, 2022
@rthallamko3 rthallamko3 self-assigned this Feb 9, 2022
@bmatican bmatican assigned spolitov and unassigned rthallamko3 Feb 14, 2022
spolitov added a commit that referenced this issue Feb 16, 2022
@spolitov
[#11409] DocDB: Skip creating write batch while writing to rocksdb
60b4711 
Summary:
RocksDB WAL is disabled in our system.
So we could avoid creating write batch while writing to RocksDB.
This diff introduces DirectWriter,that allows us to avoid serializing key and values before writing them to RocksDB.
DirectWriter is used when applying WriteOperation.

In a follow-up diff we could use DirectWriter to also apply transaction intents.

Test Plan: Jenkins

Reviewers: timur, mbautin

Reviewed By: timur, mbautin

Subscribers: mbautin, ybase, bogdan

Differential Revision: https://phabricator.dev.yugabyte.com/D15368
spolitov added a commit that referenced this issue Feb 21, 2022
@spolitov
[#11409] DocDB: Skip creating write batch while applying and removing…
6f9c6a4 
… intents

Summary:
Recently we added DirectWriter to write to RocksDB w/o creating WriteBatch.
I.e. write directly to MemTable.

Here we use DirectWriter to apply and remove intents.

Test Plan: Jenkins

Reviewers: mbautin, timur

Reviewed By: timur

Subscribers: kannan, ybase

Differential Revision: https://phabricator.dev.yugabyte.com/D15582
jayant07-yb pushed a commit to jayant07-yb/yugabyte-db that referenced this issue Mar 8, 2022
@spolitov
[yugabyte#11409] DocDB: Skip creating write batch while writing to ro…
da54a59 
…cksdb

Summary:
RocksDB WAL is disabled in our system.
So we could avoid creating write batch while writing to RocksDB.
This diff introduces DirectWriter,that allows us to avoid serializing key and values before writing them to RocksDB.
DirectWriter is used when applying WriteOperation.

In a follow-up diff we could use DirectWriter to also apply transaction intents.

Test Plan: Jenkins

Reviewers: timur, mbautin

Reviewed By: timur, mbautin

Subscribers: mbautin, ybase, bogdan

Differential Revision: https://phabricator.dev.yugabyte.com/D15368
jayant07-yb pushed a commit to jayant07-yb/yugabyte-db that referenced this issue Mar 8, 2022
@spolitov
[yugabyte#11409] DocDB: Skip creating write batch while applying and …
93fc86a 
…removing intents

Summary:
Recently we added DirectWriter to write to RocksDB w/o creating WriteBatch.
I.e. write directly to MemTable.

Here we use DirectWriter to apply and remove intents.

Test Plan: Jenkins

Reviewers: mbautin, timur

Reviewed By: timur

Subscribers: kannan, ybase

Differential Revision: https://phabricator.dev.yugabyte.com/D15582
@ymahajan ymahajan mentioned this issue Mar 15, 2022
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@spolitov spolitov

Labels
area/docdb YugabyteDB core features
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@spolitov @rthallamko3