migrating databases

[michellewehr]

Hi! Need help switching databases that we are using for our metaflow runs and service. In the metaflow metadata docker service, I have updated the credentials, as well as updated the metaflow params in our run to point to the new db. I have utilized aws glue to copy over tables (flows_v3, runs_v3, steps_v3, and tasks_v3) from the old db to the new. I am seeing a keyError when I attempt to run the flow pointing to new postgres database.

We are still on metaflow version 2.2.7 as we are trying to minimizes the number of changes at once.

Validating your flow...
 The graph looks good!
Bootstrapping conda environment...(this could take a few minutes)

    Internal error
Traceback (most recent call last):
  File "venv/lib/python3.8/site-packages/metaflow/cli.py", line 930, in main
    start(auto_envvar_prefix='METAFLOW', obj=state)
  File "venv/lib/python3.8/site-packages/click/core.py", line 1157, in __call__
    return self.main(
args, kwargs)
  File "venv/lib/python3.8/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "venv/lib/python3.8/site-packages/click/core.py", line 1688, in invoke
     return _process_result(sub_ctx.command.invoke(sub_ctx))
   File "venv/lib/python3.8/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, ctx.params)
   File "venv/lib/python3.8/site-packages/click/core.py", line 783, in invoke
   return __callback(args, kwargs)
   File "venv/lib/python3.8/site-packages/metaflow/cli.py", line 533, in wrapper
    return func(args, kwargs)
 File "venv/lib/python3.8/site-packages/click/decorators.py", line 45, in new_func
    return f(get_current_context().obj, args, kwargs)
   File "venv/lib/python3.8/site-packages/metaflow/cli.py", line 624, in run
    runtime = NativeRuntime(obj.flow,
  File "venv/lib/python3.8/site-packages/metaflow/runtime.py", line 64, in __init__
    self._run_id = metadata.new_run_id()
   File "venv/lib/python3.8/site-packages/metaflow/plugins/metadata/service.py", line 44, in new_run_id
    return self._new_run(tags=tags, sys_tags=sys_tags)
  File "venv/lib/python3.8/site-packages/metaflow/plugins/metadata/service.py", line 126, in _new_run
    return str(run['run_number'])
KeyError: 'run_number'

How do I best migrate databases so we can ultimately retire the old database we were using?

[savingoyal]

@michellewehr - any reason not to migrate database using snapshots?