use threading to run jobmanager loop

[jdries]

I propose to allow running the jobmanager cycle in a separate thread, allowing for a more clean option to interrupt it.

[soxofaan]

Overall I'm not sure we should do this with asyncio, as everything (HTTP requests, file IO) are just classic blocking calls currently, so there is not much to gain.

I think in this case working with Python threading will be a bit simpler

[soxofaan]

Using "thread" in naming and docs might be confusing and setting wrong expectations as asyncio is not about threading but coroutines

[jdries]

it's now converted to use an actual 'Thread' object, so the confusion is gone?

[jdries]

@soxofaan if this is fine now, we can merge and continue with the other PR's

[soxofaan]

shouldn't this normalize_df be handled automatically in the manager?

[jdries]

we lack a good mechanism to initialize a job db correctly, we'll have to come up with something

[jdries]

@soxofaan I introduced initialize_job_db to address this issue.

[soxofaan]

FYI: I pushed some tweaks (some automatic code style cleanups), but more importantly some tweaks to the sleep/wait logic in 95a4ec7. Feel free to revert

[soxofaan]

#614 (comment):

@soxofaan I introduced initialize_job_db to address this issue.

I see, but I'd think that this initialization is not something the user should be bothered with,
can't this just be done automatically like in run_jobs:

openeo-python-client/openeo/extra/job_management.py

Lines 358 to 363 in 0c2fdc1

	if job_db.exists():
	# Resume from existing db
	_log.info(f"Resuming `run_jobs` from existing {job_db}")
	elif df is not None:
	df = self._normalize_df(df)
	job_db.persist(df)

[jdries]

To initialize a job db, you need a dataframe (once), I try to avoid that, due to the confusion that Victor explained here:
#607 (comment)

[soxofaan]

remove the initialize_job_db concern from the scope of this PR and merged in ffa7be2