A new stat for bytes read off the disk #117
Conversation
| // set the data offset | ||
| fdvIter.dvDataLoc = fieldDvLocStart | ||
|
|
||
| return fdvIter, nil | ||
| } | ||
|
|
||
| func (di *docValueReader) BytesRead() uint64 { |
There was a problem hiding this comment.
commentary indicating that these methods are an interface implementation and the reference to that interface here would be useful.
intDecoder.go
Outdated
| rv.dataStartOffset = offset + n | ||
| return rv | ||
| } | ||
|
|
||
| func (d *chunkedIntDecoder) bytesOffDisk() uint64 { |
There was a problem hiding this comment.
older func name (bytesOffDisk) lingering around?
| @@ -331,19 +346,35 @@ func (i *PostingsIterator) Size() int { | |||
| return sizeInBytes | |||
| } | |||
|
|
|||
| func (i *PostingsIterator) SetBytesRead(val uint64) { | |||
There was a problem hiding this comment.
commentary for the interface implementations..
intDecoder.go
Outdated
| rv.dataStartOffset = offset + n | ||
| return rv | ||
| } | ||
|
|
||
| func (d *chunkedIntDecoder) BytesRead() uint64 { |
There was a problem hiding this comment.
Similar commentary here like with docValueReader.
| // interface, as the intention is to retrieve the bytes | ||
| // read from the on-disk segment as part of the current | ||
| // query. | ||
| func (s *Segment) SetBytesRead(val uint64) { |
There was a problem hiding this comment.
Now that I'm looking at this - feels a bit distasteful to allow another library to just overwrite this parameter.
There was a problem hiding this comment.
I agree with you @abhinavdangeti,
@Thejas-bhat ,
Why do we need this API? This would be inherent task within the merge process.
Meaning the the merge process would take care of tracking the prev_bytes from segments and carrying it further to the newly formed segment.
Wouldn't that suffice the requirement?
There was a problem hiding this comment.
hm.. it may not possible without making disk format changes.
| @@ -96,6 +97,7 @@ func (s *SegmentBase) loadFieldDocValueReader(field string, | |||
| chunkOffsetsLen := binary.BigEndian.Uint64(s.mem[fieldDvLocEnd-16 : fieldDvLocEnd-8]) | |||
| // acquire position of chunk offsets | |||
| chunkOffsetsPosition = (fieldDvLocEnd - 16) - chunkOffsetsLen | |||
| s.bytesRead += uint64(16) | |||
There was a problem hiding this comment.
Some commentary here on why 16 here.
the persisted zap files
No description provided.