Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: store: BatchOpStore (GetMany/PutMany methods) #97

Draft
wants to merge 7 commits into
base: master
Choose a base branch
from

Conversation

i-norden
Copy link
Contributor

@i-norden i-norden commented Oct 17, 2023

This PR introduces a BatchOpIpldStore that exposes GetMany and PutMany methods. This is for use in parallel ForEach traversal methods for go-hamt-ipld and go-amt-ipld. Actually only the GetMany method is being used there, but a PutMany method was added to compliment it. This PR uses the GetMany blockstore method introduced in ipfs/boxo#492 which, in turn, relies on the TxnDatastore wrappers introduced in ipfs/go-datastore#210.

Parallel go-amt-ipld traversal work that uses this: filecoin-project/go-amt-ipld#84
Parallel go-hamt-ipld traversal work that uses this: filecoin-project/go-hamt-ipld#115

TODO:
Replace replace directives if/when dependencies are merged and released

@rvagg
Copy link
Member

rvagg commented Apr 19, 2024

@i-norden are you still working on this set of changes? sorry nobody's been able to review your changes, if these are still in the path of your work we can prioritise them .. just not this week for me sorry but I'll keep it in my queue, it looks quite useful.

@i-norden
Copy link
Contributor Author

Hey @rvagg sorry for abandoning this, we are no longer using this work but if there is still general interest in these changes and/or the linked PRs it is dependent on I can rebase and clean things up this week for review!

@gammazero gammazero marked this pull request as draft August 20, 2024 16:46
@gammazero
Copy link

Converted to draft for someone raises a production need for this.

@gammazero gammazero added the need/community-input Needs input from the wider community label Aug 20, 2024
@ribasushi
Copy link

@gammazero fwiw this is used in production by @aschmahmann here: https://github.com/aschmahmann/filexp/blob/main/go.mod#L5-L7

Having either filecoin-project/go-hamt-ipld#103 or filecoin-project/go-hamt-ipld#115 merge would be really great.

cc @rvagg

@gammazero gammazero added the need/triage Needs initial labeling and prioritization label Aug 26, 2024
@lidel lidel removed the need/triage Needs initial labeling and prioritization label Sep 3, 2024
@lidel lidel added the need/maintainer-input Needs input from the current maintainer(s) label Sep 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
need/community-input Needs input from the wider community need/maintainer-input Needs input from the current maintainer(s)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants