Skip to content

Latest commit

 

History

History
150 lines (85 loc) · 12.1 KB

README.md

File metadata and controls

150 lines (85 loc) · 12.1 KB

covid19-eu-data

covid19-eu-data is a dataset repository for COVID-19/SARS-CoV-2 cases in Europe. We pull data from official government websites regularly using the open-source scripts inside the repository.

If you would like to help or track the progress of this project

Changelog

On 2022-02-29, IE stopped updating the detailed covid infection data for weekends.

On 2021-11-01, we stopped collecting PL data as there are some data quality issues. But we update the original data here.

Breaking Change:

On 2021-01-03, we dropped the whole commit history and removed the cache files. This is done because the repo is growing into a behemoth.

On 2020-12-31, we stopped caching most of the webpages due to oversize of the repo.

On 2020-05-22, we removed documents/be and documents/dk. These two folders are bloating and our repo reached the GitHub storage hard limit (2GB). The files have been moved to covid19-eu-zh/covid19-eu-data-20200522 as a snapshot.

Full changelog: CHANGELOG.md

Update Status

Commit Status:

Workflow status by countries:

Country Status Data Source
AT CI Download AT Data
BE CI Download BE Data
CH CI Download CH Data
CZ CI Download CZ Data
DE CI Download DE SARS-COV-2 Cases from RKI
DK CI Download DK PDF
ES CI Download ES PDF Files
FR CI Download FR PDF
GR CI Download GR PDF
HU CI Download HU Data
IE CI Download IE Data and PDF
IT CI Download IT Data
NL CI Download NL SARS-COV-2 Cases from volksgezondheidenzorg
NO CI Download NO Data
PL CI Download PL Data
PT CI Download PT PDF
SE CI Download SE
FI CI Download SE
SI CI Download SI Data
UK CI Download UK
EU(ECDC) CI Download All EU from ECDC

Dataset

Tabular Data

The tabular data files are located in dataset folder. The folder dataset/daily holds the daily updates in each country.

The metadata for the tabular data is found in .dataherb/metadata.yml.

Other Data

Some of the countries publish more than simple tabular data. We cache the files in documents folder.

Scrapers

The scripts that are being used to update the data are located in scripts folder. Most of the scripts require the utils.py module to run. Create a new environment and run pip install -r requirements.txt to install the requirements.

Workflows

The workflows that update the dataset are defined in .github/workflows. The python scripts are scheduled to run on GitHub Actions.

Notes

AT

Caveats:

  1. We started tracking the recovered population and the deaths on 2020-03-13.

BE

  1. Only PDF files of the records are downloaded.

CH

  1. CH hospitalized indicates the current hospitalized patients.

DE

  1. For technical reasons, no data was transmitted from Hamburg on March 25th, 2020.

There is a repo cleaning up the raw data on ArcGis.

FR

  1. France stopped updating the case tables on the webpage on 2020-03-26. We went back to the PDF files.

NL

Caveats:

  1. NL doesn't publish the time of the data release. We use 00:00 of the day to denote the release time though it doesn't indicate the actual update time.

UK

We stopped tracking UK data.

  1. UK is already publishing data in an easy-to-use format. Click here for the full data
  2. There is already a very good github repo cleaning up the data. Click here for the repo.

Scotland

  1. Starting from 2020-04-08, Scotland doesn't report numbers less than 5. So missing value in Scotland dataset starting from 2020-04-08 indicates a number less than 5.

England

  1. In the first few days of reporting (before 2020-03-11), data of England is not always a number. To solve this problem, we added two columns, cases_lower and cases_upper, to reflect the range of the number of cases.
  2. England switched to ArcGIS later. We are downloading the CSV file directly.

Wales

  1. Wales stopped publishing detailed data on 2020-03-17.
  2. Wales switched to Tableau on 2020-04-08. https://public.tableau.com/profile/public.health.wales.health.protection#!/vizhome/RapidCOVID-19virology-Public/Headlinesummary

Northern Ireland

Northern Ireland does not publish detailed data.

IT

  1. The data source also provides the whole time-series data. Set the -f flag to true for scripts/download_it.py to redownload all dates.

Community

Bugs and requests: PRs are welcome.

Issues

Telegram Channel (in Chinese): 新冠肺炎欧洲中文臺

Chat