Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ipfs migration without starting the daemon #7471

Closed
RubenKelevra opened this issue Jun 13, 2020 · 6 comments · Fixed by #8428
Closed

ipfs migration without starting the daemon #7471

RubenKelevra opened this issue Jun 13, 2020 · 6 comments · Fixed by #8428
Assignees
Labels
effort/days Estimated to take multiple days, but less than a week exp/intermediate Prior experience is likely helpful help wanted Seeking public contribution on this issue kind/feature A new feature P3 Low: Not priority right now status/blocked Unable to be worked further until needs are met

Comments

@RubenKelevra
Copy link
Contributor

Currently, it seems to be impossible to do a migration (if necessary) without starting the daemon.

I'm a package maintainer and like script the migration after installation of the new package instead of running the migration on the first run of the daemon, which - as discussed earlier in a PR - might hit the timeout of systemd for starting a service.

I like to have an ipfs migrate --assume-yes or something like this for unattended migration while updating a package with a package manager.

It should return 0 if everything is fine, regardless if a migration was necessary or not.

@RubenKelevra RubenKelevra added kind/feature A new feature need/triage Needs initial labeling and prioritization labels Jun 13, 2020
@RubenKelevra
Copy link
Contributor Author

This would solve #7269

@Stebalien
Copy link
Member

You can run the migration tool: https://dist.ipfs.io/#fs-repo-migrations.
Or you can use ipfs update to update, which will apply the migration when it installs the new version.

@RubenKelevra
Copy link
Contributor Author

RubenKelevra commented Jun 13, 2020

You can run the migration tool: https://dist.ipfs.io/#fs-repo-migrations.

This requires quite some boiler plating, which is already integrated into the ipfs-binary, to be used by ipfs daemon --migrate. Would be nicer if the same functionality would be available without starting the daemon.

I think I would need to parse repo/fsrepo/migrations/migrations.go to fetch the right version. 🤔

Or you can use ipfs update to update, which will apply the migration when it installs the new version.

This doesn't work, I build the binary from git. So it makes no sense for me to fetch an older version.

@Stebalien
Copy link
Member

Fair enough. I agree we should add an ipfs migrate command (probably ipfs repo migrate).

@Stebalien Stebalien added exp/intermediate Prior experience is likely helpful effort/days Estimated to take multiple days, but less than a week P3 Low: Not priority right now status/accepted This issue has been accepted and removed need/triage Needs initial labeling and prioritization labels Jun 15, 2020
@Stebalien
Copy link
Member

Command:

USAGE
  ipfs repo migrate - Apply any outstanding migrations to the repo.

SYNOPSIS
  ipfs repo migrate [--allow-downgrade]

OPTIONS
  --allow-downgrade bool - Allow downgrading to a lower repo version

This command should:

  1. Refuse to run when the daemon is online. Add a line to the cmdDetailsMap in cmd/ipfs/ipfs.go specifying cannotRunOnDaemon: true.
  2. Opens the repo (see how fsrepo.Open is used in cmd/ipfs/daemon.go).
  3. Checks to see if there's a "needs migration" error (again, look at cmd/ipfs/daemon.go).
  4. If so, runs migrate.RunMigration (see cmd/ipfs/daemon.go again).

@Stebalien Stebalien added help wanted Seeking public contribution on this issue status/ready Ready to be worked and removed status/accepted This issue has been accepted labels Jun 15, 2020
@Stebalien Stebalien mentioned this issue Jun 24, 2020
77 tasks
zaibon added a commit to zaibon/go-ipfs that referenced this issue Sep 7, 2020
this command allows to run the repo migration without starting the
daemon.

fixes ipfs#7471
zaibon added a commit to zaibon/go-ipfs that referenced this issue Sep 7, 2020
this command allows to run the repo migration without starting the
daemon.

resolves ipfs#7471
zaibon added a commit to zaibon/go-ipfs that referenced this issue Sep 7, 2020
this command allows to run the repo migration without starting the
daemon.

resolves ipfs#7471
zaibon added a commit to zaibon/go-ipfs that referenced this issue Sep 14, 2020
this command allows to run the repo migration without starting the
daemon.

resolves ipfs#7471
zaibon added a commit to zaibon/go-ipfs that referenced this issue Nov 13, 2020
this command allows to run the repo migration without starting the
daemon.

resolves ipfs#7471
@aschmahmann
Copy link
Contributor

Note: state of this issue is that this is blocked pending a refactor of fs-repo-migrations (i.e. ipfs/fs-repo-migrations#98) such that we're using many binaries instead of one. This will mean that calling ipfs repo migrate will get and run each of the migration binaries required and become the primary way to do manual migrations.

@aschmahmann aschmahmann added status/blocked Unable to be worked further until needs are met and removed status/ready Ready to be worked labels Dec 6, 2020
zaibon added a commit to zaibon/go-ipfs that referenced this issue Apr 3, 2021
this command allows to run the repo migration without starting the
daemon.

resolves ipfs#7471
christophedcpm pushed a commit to zaibon/go-ipfs that referenced this issue Apr 6, 2021
this command allows to run the repo migration without starting the
daemon.

resolves ipfs#7471
zaibon added a commit to zaibon/go-ipfs that referenced this issue Apr 16, 2021
this command allows to run the repo migration without starting the
daemon.

resolves ipfs#7471
christophedcpm pushed a commit to zaibon/go-ipfs that referenced this issue Aug 13, 2021
this command allows to run the repo migration without starting the
daemon.

resolves ipfs#7471
gammazero pushed a commit to zaibon/go-ipfs that referenced this issue Sep 3, 2021
this command allows to run the repo migration without starting the
daemon.

resolves ipfs#7471
gammazero pushed a commit to zaibon/go-ipfs that referenced this issue Sep 10, 2021
this command allows to run the repo migration without starting the
daemon.

resolves ipfs#7471
gammazero added a commit that referenced this issue Sep 10, 2021
This PR replaces #7658 that was originally contributed by zaibons, in order to move code into a branch and avoid some CI problem.

The command allows the user to run the repo migration without starting the daemon.

resolves #7471
@BigLep BigLep assigned gammazero and unassigned aschmahmann Mar 3, 2022
@BigLep BigLep added this to the Best Effort Track milestone Mar 3, 2022
guseggert pushed a commit that referenced this issue May 6, 2022
This PR replaces #7658 that was originally contributed by zaibons, in order to move code into a branch and avoid some CI problem.

The command allows the user to run the repo migration without starting the daemon.

resolves #7471
guseggert pushed a commit that referenced this issue May 6, 2022
This PR replaces #7658 that was originally contributed by zaibons, in order to move code into a branch and avoid some CI problem.

The command allows the user to run the repo migration without starting the daemon.

resolves #7471
guseggert pushed a commit that referenced this issue May 6, 2022
This PR replaces #7658 that was originally contributed by zaibons, in order to move code into a branch and avoid some CI problem.

The command allows the user to run the repo migration without starting the daemon.

resolves #7471
guseggert added a commit that referenced this issue May 6, 2022
* Add 'ipfs repo migrate' command

This PR replaces #7658 that was originally contributed by zaibons, in order to move code into a branch and avoid some CI problem.

The command allows the user to run the repo migration without starting the daemon.

resolves #7471

* return non-ErrNeedMigration errors from fsrepo.Open()

Co-authored-by: Gus Eggert <gus@gus.dev>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
effort/days Estimated to take multiple days, but less than a week exp/intermediate Prior experience is likely helpful help wanted Seeking public contribution on this issue kind/feature A new feature P3 Low: Not priority right now status/blocked Unable to be worked further until needs are met
Projects
No open projects
Archived in project
Development

Successfully merging a pull request may close this issue.

5 participants