Skip to content
This repository has been archived by the owner on Apr 16, 2020. It is now read-only.

Add blockers file #21

Merged
merged 2 commits into from
Mar 25, 2019
Merged

Add blockers file #21

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
07930e4
Add Blockers file
andrew Mar 15, 2019
04b583e
Add link to blockers.md from readme
andrew Mar 15, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,12 @@
- [Package Management Categories](categories.md)
- [Package Manager list](package-managers)

## Blockers

Current issues with IPFS that are limiting package manager adoption today

- [Blockers](blockers.md)

## Team
- [@andrew](http://github.com/andrew)
- [@achingbrain](https://github.com/achingbrain)
Expand Down
24 changes: 24 additions & 0 deletions blockers.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# Blockers

Issues with IPFS that are limiting package manager adoption today

## File system based package managers

### Requires 2x disk space for mirroring

Filestore expects files to be immutable once added, so rsyncing updates to existing files [causes errors](https://github.com/protocol/package-managers/issues/18#issuecomment-471365124), one workaround is to not use the filestore but that requires copying the entire mirror directory into `.ipfs`.

### No easy way to directly add a directory to MFS with go-ipfs

Adding a directory of files to MFS means calling out to `ipfs files write` for every file, ideally there should be one command to write a directory of files to MFS.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Technically you can do

ipfs files cp /ipfs/$(ipfs add -rq dir) /some/mfs/path

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@magik6k good point, handy for the initial adding but I don't believe that doesn't help with some of the other restrictions of adding/updating some content later, although sounds like ipfs/kubo#6036 will help with that

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, updating is tricky, perhaps storing mod times with files would help with that, but it's not supported yet (https://github.com/ipfs/unixfs-v2 is what we need)

Copy link
Contributor

@momack2 momack2 Mar 22, 2019 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@djdv re needs for MFS from package managers, @warpfork re needs for unixfsv2

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For posterity, I'll mention that I'm currently working on a variant of go-ipfs that has writable access to the Files(MFS) and Name(IPNS) APIs, through the FUSE API.

While not fully viable today, I would like it to be capable of hosting a living/mutable package repo in the same way a traditional filesystem can (to the specifications our users expect).

Some issues I encountered while writing this have been resolved already, but there's a handful that are not simple fixes. Since this demand is pretty broad, I look forward to bugging collaborating with a bunch of us here to try and harmonize some of the next iterations of these APIs and formats to enable this kind of use case.

@aschmahmann's current work on IPNS will also likely have a bearing on this. I have an implementation to expose IPNS keys as writable files and directories through MFS which may be useful for package managers. It's basically an abstraction on the existing common pattern (modify a root via ipfs files -> stat changes -> name publish)


Alternative approach may be to mount MFS as a fuse filesystem (ala https://github.com/tableflip/ipfs-fuse)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is also a new mfs mounting feature brewing in go-ipfs - ipfs/kubo#6036


### Updating rolling changes requires rehashing all files

If there is a regular cron job downloading updates to a mirror with rsync, there's currently no easy way to only re-add the files that have been added/changed/removed without rehashing every file in the whole mirror directory.

Mounting MFS as a fuse filesystem (ala https://github.com/tableflip/ipfs-fuse) and rsyncing directly onto fuse may be one approach.

Alternatively there could be a ipfs rysnc command line tool that could talk directly with rsync server protocol.