COVID-19 dataset clearinghouse: Difference between revisions

From Polymath Wiki
Jump to navigationJump to search
← Older edit
 
(76 intermediate revisions by 2 users not shown)
Line 1: Line 1:
== Data cleaning proposal ==
This is a repository for public data sets relating to the COVID-19 pandemic.  It was also initially envisioned as a clearinghouse for matching requests for data cleaning of such datasets with volunteers willing to perform this clearing, but the [https://discourse.data-against-covid.org/c/i-have-data/15 existing clearinghouse] at [http://united-against-covid.org/ United against COVID-19] is already up and running for this purpose, so we are redirecting such requests to that site in order not to fragment the pools of requests and volunteers.


* [https://terrytao.files.wordpress.com/2020/03/covid_19_polymath_project-1.pdf PDF format]
For discussion of this project, see [https://terrytao.wordpress.com/2020/03/25/polymath-proposal-clearinghouse-for-crowdsourcing-covid-19-data-and-data-cleaning-requests this blog post].
* [https://www.overleaf.com/project/5e7acd0e03821500012262bb Overleaf format]
* [https://terrytao.wordpress.com/2020/03/25/polymath-proposal-clearinghouse-for-crowdsourcing-covid-19-data-and-data-cleaning-requests Blog post discussing the proposal]


== Instructions for posting a request for a data set to be cleaned ==
== Data sets ==


Ideally, the submission should consist of a single plain text file which clearly delineates your request (specify what your “cleaned” data set should contain). This should specify the desired format in which the data should be saved (e.g. csv, npy, mat, json). This text file should also contain a link to a webpage where the raw data to be cleaned can easily be accessed and/or downloaded, and with specific instruction for how to locate the data set on said webpage.  
Further contributions are very welcome, and can be made either directly to this wiki page (after requesting an account), or placed in the comments to [https://terrytao.wordpress.com/2020/03/25/polymath-proposal-clearinghouse-for-crowdsourcing-covid-19-data-and-data-cleaning-requests/ this blog post], or by email to tao@math.ucla.edu.


We do not yet have a platform for these requests, so please post them for now at [https://terrytao.wordpress.com/2020/03/25/polymath-proposal-clearinghouse-for-crowdsourcing-covid-19-data-and-data-cleaning-requests the above blog post] or email tao@math.ucla.edu .
=== Epidemiology ===


== Data sets ==
* [https://www.kaggle.com/sudalairajkumar/novel-corona-virus-2019-dataset Novel Corona Virus 2019 Dataset - Day level information on covid-19 affected cases], Kaggle
 
* [https://www.kaggle.com/tags/covid19 COVID-19 data sets], Kaggle
** [https://www.kaggle.com/sudalairajkumar/novel-corona-virus-2019-dataset Novel Corona Virus 2019 Dataset - Day level information on covid-19 affected cases]
** [https://www.kaggle.com/paultimothymooney/coronavirus-genome-sequence Coronavirus Genome Sequence]
** [https://www.kaggle.com/paultimothymooney/repository-of-coronavirus-genomes Repository of Coronavirus Genomes]
* [https://docs.google.com/forms/d/e/1FAIpQLSc501xfAzEPADOwRmsdHmu-v8aN14jnKHBmEmdJJcTgRLddqw/viewform Aggregated foot traffic data], Safegraph
**  Needs non-commercial agreement to execute.
** [https://www.safegraph.com/dashboard/covid19-commerce-patterns?is=5e7a3815f20d617a17a33173 Sample visualization of Safegraph data]
* [https://ourworldindata.org/coronavirus Coronavirus Disease (COVID-19) – Statistics and Research], Our World in Data, by Max Roser, Hannah Ritchie and Esteban Ortiz-Ospina
* [https://ourworldindata.org/coronavirus Coronavirus Disease (COVID-19) – Statistics and Research], Our World in Data, by Max Roser, Hannah Ritchie and Esteban Ortiz-Ospina
* [https://github.com/CSSEGISandData/COVID-19 Novel Coronavirus (COVID-19) Cases], Johns Hopkins University Center for Systems Science and Engineering
** [https://ourworldindata.org/coronavirus?fbclid=IwAR0d1z_t-W9OQVDE08wMbR3-RX-XPCaogFM5hUmFMqL2QZDgY9x-R2CRErE Cases + mortality rate daily]
** [https://ourworldindata.org/covid-testing How many tests for COVID-19 are being performed around the world?]
* [https://github.com/CSSEGISandData/COVID-19 Novel Coronavirus (COVID-19) Cases], Johns Hopkins University Center for Systems Science and Engineering (JHU CSSE)
** [https://github.com/datasets/covid-19 Novel Coronavirus 2019 time series data on cases], sourced and cleaned from the above data set
** [https://github.com/datasets/covid-19 Novel Coronavirus 2019 time series data on cases], sourced and cleaned from the above data set
* [https://github.com/COVID19Tracking/covid-tracking-data COVID Tracking Data], from the [https://covidtracking.com/ COVID tracking project]
** [https://github.com/summyfeb12/COVID-19-JHU-Data-API JSON Wrapper API for the JHU CSSE data]
** A daily updated repository with CSV representations of data from the [https://github.com/COVID19Tracking/covid-tracking-api/blob/master/README.md Covid Tracking API].  Contains US data only
** [http://shiny.science.ku.dk/pbm/COVID19/ A visualization of the JHU CSSE data]
* [https://github.com/covid19-data/covid19-data 2019-nCoV Data Processing Pipelines and datasets]   
* [https://github.com/covid19-data/covid19-data 2019-nCoV Data Processing Pipelines and datasets]   
** Countries and state names are normalized with ISO 3166-1 code.
** Countries and state names are normalized with ISO 3166-1 code.
* [https://github.com/beoutbreakprepared/nCoV2019 Location for summaries and analysis of data related to n-CoV 2019, first reported in Wuhan, China], Outbreak and Pandemic Preparedness team at the Institute for Health Metrics and Evaluation, University of Washington
** A [https://www.healthmap.org/covid-19/ visualization of one of the data sets]
* [https://www.ecdc.europa.eu/en/publications-data/download-todays-data-geographic-distribution-covid-19-cases-worldwide Daily data on the geographic distribution of COVID-19 cases worldwide], European Centre for Disease Prevention and Control
* [https://docs.google.com/spreadsheets/d/1jS24DjSPVWa4iuxuD4OAXrE3QeI8c9BC1hSlqr-NMiU/edit#gid=1187587451 Google sheets from DXY.cn]
** Contains some patient information [age,gender,etc]
* [https://www.kaggle.com/imdevskp/corona-virus-report COVID-19 Complete Dataset (Updated every 24hrs)], Kaggle
* [https://datarepository.wolframcloud.com/resources/Epidemic-Data-for-Novel-Coronavirus-COVID-19 Epidemic Data for Novel Coronavirus COVID-19], Wolfram
* [https://coronavirus-disasterresponse.hub.arcgis.com/datasets/bbb2e4f589ba40d692fab712ae37b9ac Coronavirus COVID-19 Cases], ESRI
* [https://www.europeandataportal.eu/data/datasets/covid-19-coronavirus-data?locale=en COVID-19 Coronavirus data], European Union Open Data Portal
* [https://github.com/midas-network/COVID-19/tree/master/data/cases/who/novel_coronavirus_situation_reports/ Novel Coronavirus (2019 nCoV) situation reports], World Health Organization
* [https://github.com/midas-network/COVID-19/tree/master/data/cases/global/line_listings_oxford_github/ nCoV line listings from various sources and data processing], Oxford University
* [https://healthweather.us/?mode=Atypical US Health Weather Map], Kinsa Insights
** Cumulative amount of atypical illnesses observed since March 1
==== North America ====
* [https://github.com/COVID19Tracking/covid-tracking-data COVID Tracking Data], from the [https://covidtracking.com/ COVID tracking project]
** A daily updated repository with CSV representations of data from the [https://github.com/COVID19Tracking/covid-tracking-api/blob/master/README.md Covid Tracking API]. 
* [https://www.kaggle.com/sudalairajkumar/covid19-in-usa COVID-19 in USA], Kaggle
* [https://coronavirus-disasterresponse.hub.arcgis.com/datasets/628578697fb24d8ea4c32fa0c5ae1843_0?geometry=110.290%2C-19.609%2C-134.749%2C71.199 COVID-19 Cases US], ESRI
* [https://coronavirus.1point3acres.com/en COVID-19 in US and Canada]
* [https://coronavirus.1point3acres.com/en COVID-19 in US and Canada]
** [https://coronavirus.1point3acres.com/en/data Data request form]
** [https://coronavirus.1point3acres.com/en/data Data request form]
Line 33: Line 45:
** [https://docs.google.com/spreadsheets/u/2/d/e/2PACX-1vRwAqp96T9sYYq2-i7Tj0pvTf6XVHjDSMIKBdZHXiCGGdNC0ypEU9NbngS8mxea55JuCFuua1MUeOj5/pubhtml raw data]
** [https://docs.google.com/spreadsheets/u/2/d/e/2PACX-1vRwAqp96T9sYYq2-i7Tj0pvTf6XVHjDSMIKBdZHXiCGGdNC0ypEU9NbngS8mxea55JuCFuua1MUeOj5/pubhtml raw data]
** [https://covidtracking.com/api/ API]
** [https://covidtracking.com/api/ API]
* [https://www.covidcaremap.org COVID Care Map]
* [https://github.com/kgjenkins/covid-19-ny Covid-19 coronovirus cases in New York State]
** Open geospatial work to support health systems' capacity (providers, supplies, ventilators, beds, meds) to effectively care for rapidly growing COVID19 patient needs
* [https://covid19tracker.health.ny.gov/ COVID-19 Tracker], New York State Department of Health
** [https://www.covidcaremap.org/maps/us-healthcare-system-capacity/#6.07/40.085/-75.195 Open map data on US health system capacity to care for COVID-19 patients]
* [https://a816-hrt.nyc.gov/DataCatalog/Pages/DataView.cshtml?id=5ff1078d-8e34-43de-b339-7556dd09b5b2 Communicable Disease Surveillance Data], NYC Health Department Data Resources
* [https://github.com/beoutbreakprepared/nCoV2019 Location for summaries and analysis of data related to n-CoV 2019, first reported in Wuhan, China], Outbreak and Pandemic Preparedness team at the Institute for Health Metrics and Evaluation, University of Washington
* [https://www.nytimes.com/article/coronavirus-county-data-us.html Coronavirus Case Data for Every U.S. County], New York Times
** A [https://www.healthmap.org/covid-19/ visualization of one of the data sets]
** [https://github.com/nytimes/covid-19-data Github repository]
** [https://www.nytimes.com/interactive/2020/us/coronavirus-us-cases.html Interactive visualization]
* [https://drive.google.com/file/d/1Ec-PDaqthCOPjWNXqun1yopX45xeSc0k/view COVID-19 Coronavirus US Case Density over Time by State], using JHU CSSE data
* [http://coronavirusapi.com/ Coronavirus API Public Health Initiative]
** Record of official data from US government websites for the 50 states and DC
* [https://dph.georgia.gov/covid-19-daily-status-report COVID-19 Daily Status Report], Georgia Department of Public Health
* [https://www150.statcan.gc.ca/n1/en/type/data?text=COVID COVID statistics] (Canada), Statistics Canada
* [https://fr.flatten.ca/ FLATTEN] (Canada)
** Online screening tool to provide information on COVID-19
* [https://github.com/midas-network/COVID-19/tree/master/data/cases/canada/ontario_situation_updates/ The 2019 Novel Coronavirus (2019 nCoV), Status of cases in Ontario], Ontario Ministry of Health
* [https://github.com/reichlab/covid19-forecast-hub Projections of COVID-19, in standardized format], The Reich Lab at UMass-Amherst
 
==== Europe ====
 
* [http://www.influenzanet.info/ Influenzanet]
* [https://leoss.net/ Studying SARS-CoV-2 in European patients], Lean European Open Survey on SARS-CoV-2 Infected Patients (LEOSS)
* [https://github.com/pcm-dpc/COVID-19 COVID-19 Italia - Monitoraggio situazione]
* [https://www.epicentro.iss.it/coronavirus/sars-cov-2-sorveglianza-dati Sorveglianza integrata COVID-19: i principali dati nazionali] (Italy), Epicentro
* [https://www.kaggle.com/sudalairajkumar/covid19-in-italy COVID-19 in Italy], Kaggle
* [https://npgeo-corona-npgeo-de.hub.arcgis.com/search?groupIds=b28109b18022405bb965c602b13e1bbc RKI COVID19] (Germany), NPGEO Corona
* [https://coronavirus.digitaler-harz.de/ Coronavirus API Deutschland] (Germany), Digitaler Harz
* [https://www.kaggle.com/headsortails/covid19-tracking-germany COVID-19 Tracking Germany], Heads or Tails
* [https://www.bag.admin.ch/bag/en/home/krankheiten/ausbrueche-epidemien-pandemien/aktuelle-ausbrueche-epidemien/novel-cov/situation-schweiz-und-international.html New coronavirus: Current situation – Switzerland and international], Bundesamt für Gesundheit.
** [https://www.bag.admin.ch/dam/bag/de/dokumente/mt/k-und-i/aktuelle-ausbrueche-pandemien/2019-nCoV/covid-19-datengrundlage-lagebericht.xlsx.download.xlsx/200325_Datengrundlage_Grafiken_COVID-19-Bericht.xlsx data set]
* [https://www.mscbs.gob.es/profesionales/saludPublica/ccayes/alertasActual/nCov-China/situacionActual.htm Situacion Actual] (Spain), Ministerio de Sanidad, Consumo y Bienestar
* [https://atlo.team/koronamonitor/ Koronamonitor] (Hungary), atlatszo.hu, atlo.team
* [https://www.europeandataportal.eu/data/datasets/fr-sars-cov-2?locale=en Fr-SARS-CoV-2] (France), European Data Portal
* [https://github.com/opencovid19-fr/data Consolidation des données de sources officielles concernant l'épidémie de COVID19] (France)
* [https://www.europeandataportal.eu/data/datasets/coronavirus?locale=en Coronavirus] (Netherlands), Rijksinstituut voor Volksgezondheid en Milieu (RIVM)
* [https://www.europeandataportal.eu/data/datasets/covid-19-zarazheni?locale=en Covid 19 - заражени] (Serbia), European Data Portal
* [https://www.europeandataportal.eu/data/datasets/covid-19-samoizolatsija?locale=en Covid 19 - самоизолација] (Serbia), European Data Portal
* [https://www.europeandataportal.eu/data/datasets/covidcountystatisticshpscireland?locale=en CovidCountyStatisticsHPSCIreland] (Ireland), Health Surveillance Protection Centre
 
==== Asia ====
 
* [https://www.covid19india.org/ India COVID-19 tracker]
* [https://www.covid19india.org/ India COVID-19 tracker]
** [https://docs.google.com/spreadsheets/d/e/2PACX-1vSc_2y5N0I67wDU38DjDh35IZSIS30rQf7_NYZhtYYGU1jJYT6_kDx4YpF-qw0LSlGsBYP8pqM_a1Pd/pubhtml Patient database]
** [https://docs.google.com/spreadsheets/d/e/2PACX-1vSc_2y5N0I67wDU38DjDh35IZSIS30rQf7_NYZhtYYGU1jJYT6_kDx4YpF-qw0LSlGsBYP8pqM_a1Pd/pubhtml Patient database]
* [https://github.com/kgjenkins/covid-19-ny Covid-19 coronovirus cases in New York State]
* [https://www.kaggle.com/sudalairajkumar/covid19-in-india Dataset on Novel Corona Virus Disease 2019 in India], Kaggle
* [https://www.programmableweb.com/news/apis-to-track-coronavirus-covid-19/review/2020/03/18 Review of COVID-19 APIs], Wendell Santos
* [https://www.kaggle.com/imdevskp/covid19-corona-virus-india-dataset COVID-19 Corona Virus India Dataset], Kaggle
* [https://www.reddit.com/r/datasets/comments/exnzrd/coronavirus_datasets/ Reddit thread collecting coronavirus datasets]
** State/UT/NCR wise COVID-19 data
* [https://github.com/jihoo-kim/Data-Science-for-COVID-19-old Data Science for COVID-19 in South Korea]
** [https://www.kaggle.com/kimjihoo/coronavirusdataset The data set on Kaggle]
* [https://www.cdc.go.kr/board/board.es?mid=a30402000000&bid=0030 Press releases], Korea Centers for Disease Control and Prevention
* [https://github.com/midas-network/COVID-19/tree/master/data/cases/south%20korea/line_list_park_github/ COVID 19 South Korea], Sang Woo Park
* [https://github.com/ThisIsIsaac/Data-Science-for-COVID-19 COVID-19 Korea Dataset & Comprehensive Medical Dataset & visualizer], DS4C (Data Science for COVID-19) Project
* [https://hira-covid19.net/ #OpenData4Covid19], Ministry of Health and Welfare of Korea and Health Insurance Review and Assessment Service of Korea
** Medical history of COVID19 patients based on their insurance claims of the last five years.
* [https://github.com/midas-network/COVID-19/tree/master/data/cases/south%20korea/confirmed_cases_movement/ Confirmed patient movement route], Korean Centers for Disease Control
* [https://covid19ph.com/ COVID-19 Philippines], Negros Island
* [https://github.com/midas-network/COVID-19/tree/master/data/cases/china/death_count_imperial_college/ Hubei early deaths 2020 07 02], Imperial College
* [https://github.com/midas-network/COVID-19/tree/master/data/cases/china/daily_cases_chinacdc_EN/ Distribution of new coronavirus pneumonia], China CDC
* [https://github.com/midas-network/COVID-19/tree/master/data/cases/hong%20kong/doh_situation_updates/ Latest local situation of Severe Respiratory Disease associated with a Novel Infectious Agent] (Hong Kong), Hong Kong Center for Health Protection
* [https://github.com/midas-network/COVID-19/tree/master/data/cases/thailand/moh_situation_updates/ Novel Coronavirus 2019 Pneumonia Situation] (Thailand), Emergency Operation Center, Department of Disease Control
 
==== Other regional data ====
 
* [https://www.kaggle.com/unanimad/corona-virus-brazil Coronavirus (COVID-19) - Brazil Dataset], Kaggle
* [https://brasil.io/dataset/covid19/caso COVID-19] (Brazil)
** Boletins informativos e casos do coronavírus por município por dia
* [https://www.geopoll.com/blog/coronavirus-africa/ Coronavirus In Sub-Saharan Africa], Geopoll
* [https://www.health.nsw.gov.au/Infectious/diseases/Pages/covid-19-latest.aspx Latest updates on COVID-19], New South Wales
 
=== Genomics and homology ===
 
* [https://www.gisaid.org/ GISAID data] (Global Initiative on Sharing All Influenza Data)
* [https://www.gisaid.org/ GISAID data] (Global Initiative on Sharing All Influenza Data)
** [https://www.gisaid.org/registration/register/ Registration] is required.
** [https://www.gisaid.org/registration/register/ Registration] is required.
** [https://github.com/nextstrain/ncov Nextstrain build for novel coronavirus (nCoV)], based on GISAID data
** [https://github.com/nextstrain/ncov Nextstrain build for novel coronavirus (nCoV)], based on GISAID data
*** A [https://nextstrain.org/ncov Genomic epidemiology of novel coronavirus]
*** A [https://nextstrain.org/ncov Genomic epidemiology of novel coronavirus]  
* [https://docs.google.com/spreadsheets/d/1jS24DjSPVWa4iuxuD4OAXrE3QeI8c9BC1hSlqr-NMiU/edit#gid=1187587451 Google sheets from DXY.cn]  
* [https://www.kaggle.com/paultimothymooney/coronavirus-genome-sequence Coronavirus Genome Sequence], Kaggle
** Contains some patient information [age,gender,etc]
* [https://www.kaggle.com/paultimothymooney/repository-of-coronavirus-genomes Repository of Coronavirus Genomes], Kaggle
* [https://www.ecdc.europa.eu/en/publications-data/download-todays-data-geographic-distribution-covid-19-cases-worldwide Daily data on the geographic distribution of COVID-19 cases worldwide], European Centre for Disease Prevention and Control
* [https://www.kaggle.com/jamzing/sars-coronavirus-accession SARS coronavirus accession], Kaggle
** Exploration of mutations of the SARS corona virus with complete genome
* [https://datarepository.wolframcloud.com/resources/Genetic-Sequences-for-the-SARS-CoV-2-Coronavirus Genetic Sequences for the SARS-CoV-2 Coronavirus], Wolfram
** Nucleotide sequences of the SARS-CoV-2 virus (the virus associated with the COVID-19 disease, formerly known as 2019-nCoV) including location, collection time and similar supporting data.
* [https://3dprint.nih.gov/discover/3DPX-012867 Wuhan coronavirus 2019-nCoV protease homology model], National Institutes of Health
* [https://3dprint.nih.gov/discover/3DPX-012867 Wuhan coronavirus 2019-nCoV protease homology model], National Institutes of Health


== Data cleaning requests ==
=== Literature ===


We do not have a platform yet to handle queries or submissions to these cleaning requests, so for now please use the comment thread at [https://terrytao.wordpress.com/2020/03/25/polymath-proposal-clearinghouse-for-crowdsourcing-covid-19-data-and-data-cleaning-requests/ this blog post] for these.
* [https://www.ncbi.nlm.nih.gov/research/coronavirus/ LitCovid] - a curated literature hub for tracking up-to-date scientific information about the 2019 novel Coronavirus
* [https://connect.biorxiv.org/relate/content/181 COVID-19 SARS-CoV-2 preprints from medRxiv and bioRxiv]
* [http://biomed-sanity.com/ BioMed Sanity], karpathy
** Indexing bioRxiv papers on COVID-19
* [https://pages.semanticscholar.org/coronavirus-research COVID-19 Open Research Dataset (CORD-19)], Allen Institute for AI, Microsoft, NLM, CZI, Georgetown University
** Over 44,000 scholarly articles, including over 29,000 with full text, about COVID-19 and the coronavirus family of viruses for use by the global research community
** requested by the White House Office of Science and Technology Policy, and part of the [https://www.whitehouse.gov/briefings-statements/call-action-tech-community-new-machine-readable-covid-19-dataset/ Call to Action to the Tech Community on New Machine Readable COVID-19 Dataset]
* [http://eppi.ioe.ac.uk/cms/Projects/DepartmentofHealthandSocialCare/Publishedreviews/COVID-19Livingsystematicmapoftheevidence/tabid/3765/Default.aspx COVID-19: a living systematic map of the evidence], EPPI
 
=== Experts ===
 
* [https://www.science.org.au/covid19/experts COVID-19 Expert Database], Australian Academy of Science
** A mechanism for governments, the business sector, the research sector, and other decision-makers to easily access the expertise they need to inform their decision making.
 
=== Medical imagery and records ===
 
* [https://datarepository.wolframcloud.com/resources/Patient-Medical-Data-for-Novel-Coronavirus-COVID-19 Patient Medical Data for Novel Coronavirus COVID-19], Wolfram
* [https://www.kaggle.com/darshan1504/covid19-detection-xray-dataset COVID-19 Detection X-Ray Dataset], Kaggle
* [https://www.sirm.org/category/senza-categoria/covid-19/ COVID-19: casistica radiologica Italiana], Società Italiana di Radiologia Medica e Interventistica
* [https://www.covid19challenge.eu/ Fighting Covid-19 Challenge]
** A platform for open research on large Covid-19 imaging datasets
 
=== Healthcare, vaccine development and equipment ===
 
* [https://coronavirus-disasterresponse.hub.arcgis.com/datasets/definitivehc::definitive-healthcare-usa-hospital-beds Definitive Healthcare: USA Hospital Beds], ESRI
* [https://www.arcgis.com/home/webmap/viewer.html?webmap=6afcaeb7549f4390b07224a0be01b3a6 COVID-19 Provider Practice Locations], ArcGIS.
* [https://github.com/EdisonDesign/HealthOS Prototype of the HealthOS ventilation system], Edison Design
* [https://e-vent.mit.edu/ MIT Emergency Ventilator], MIT
* [https://www.covidcaremap.org COVID Care Map]
** Open geospatial work to support health systems' capacity (providers, supplies, ventilators, beds, meds) to effectively care for rapidly growing COVID19 patient needs
** [https://www.covidcaremap.org/maps/us-healthcare-system-capacity/#6.07/40.085/-75.195 Open map data on US health system capacity to care for COVID-19 patients]
* [https://milkeninstitute.org/covid-19-tracker COVID-19 Treatment and Vaccine tracker], Milken Institute
 
=== Social and traffic data ===
 
* [https://docs.google.com/forms/d/e/1FAIpQLSc501xfAzEPADOwRmsdHmu-v8aN14jnKHBmEmdJJcTgRLddqw/viewform Aggregated foot traffic data], Safegraph
**  Needs non-commercial agreement to execute.
** [https://www.safegraph.com/dashboard/covid19-commerce-patterns?is=5e7a3815f20d617a17a33173 Sample visualization of Safegraph data]
* [http://www.panacealab.org/covid19/ Covid-19 Twitter chatter dataset for scientific use], Panacea Lab, Georgia State University
* [https://covid19obs.fbk.eu/assets/files/last_info.csv Infodemics data from Twitter], CoMuNe lab
** [https://covid19obs.fbk.eu/ Infodemics Observatory]
* [https://app.powerbi.com/view?r=eyJrIjoiODZjNDhmYjAtZGQ3Zi00MDRlLTllNzctYTRjMmI4MTU5YWUyIiwidCI6IjZmZmEyMmY0LTQ1NjgtNDEwNS1hZDQzLTJlM2FkNDcyNjk1NyIsImMiOjN9 COVID-19 Online Survey], Harvard Humanitarian Initiative
** Analyzes social-behavioral aspects of outbreak control
* [https://covid19-civiclytics.citibeats.com/#/ ¿De qué está hablando la ciudadanía durante la pandemia COVID-19?], Grupo BID
** Twitter data relating to COVID in Latin America
* [https://dataforgood.fb.com/tools/disease-prevention-maps/ Disease prevention maps], Facebook
* [https://www.unacast.com/covid19 ‍Social Distancing Scoreboard], Unacast
 
=== Economic and Policy ===
 
* [https://docs.google.com/spreadsheets/d/19wJZekxpewDQmApULkvZRBpBwcnd5gZlZF2SEU2WQD8/htmlview?usp=sharing&pru=AAABcO6Xep8*i6rvRQXGMf3qhTlWDRbaSw# Colleges and universities closed/migrating online for COVID-19], Bryan Alexander
* [https://www.notion.so/Schools-affected-by-COVID-19-a28139cb40814869a2cd64cc9453d82c Schools affected by COVID-19]
* [https://www.marinetraffic.com/research/measuring-the-coronavirus-impact-on-trade/ Measuring the Coronavirus’ impact on trade], Marine Traffic Research
* [https://www.top10vpn.com/news/surveillance/covid-19-digital-rights-tracker/ COVID-19 Digital Rights Tracker], Top10VPN
* [https://rexdouglass.github.io/TIGR/TIGR_landing_page.nb.html Crowd-sourced COVID-19 Dataset Tracking Involuntary Government Restrictions (TIGR)], Rex W. Douglass
 
=== Data scrapers and aggregators ===
 
* [https://coronadatascraper.com/#home Corona Data Scraper]
* [https://github.com/jagsfan82/Covid19-WebScrape-Plus Covid19-WebScrape-Plus]
* [http://covid-19.seektable.com/ COVID-19], Seektable
 
=== Visualizations, projections, summaries ===
* [https://www.worldometers.info/coronavirus/ COVID-19 Coronavirus Pandemic], Worldometer
* [https://bnonews.com/index.php/2020/03/the-latest-coronavirus-cases/ Tracking coronavirus: Map, data and timeline], BNO News
* [https://gisanddata.maps.arcgis.com/apps/opsdashboard/index.html#/bda7594740fd40299423467b48e9ecf6 Coronavirus COVID-19 Global Cases], JHU CSSE
* [https://infection2020.com/ Infection2020]
* [https://covy.app/ covy.app]
* [https://www.coronatracker.com/ CoronaTracker]
* [https://ncov.dxy.cn/ncovh5/view/pneumonia?from=dxy&source=&link=&share= COVID-19 Global Pandemic Real-Time report], dxy.cn ([https://ncov.dxy.cn/ncovh5/view/en_pneumonia?from=dxy&source=&link=&share= English version])
* [https://www.ft.com/coronavirus-latest Coronavirus tracked: the latest figures as the pandemic spreads], Financial Times
* [https://www.mygov.in/covid-19/ COVID-19] - official Indian government site
* [https://www.kaggle.com/imdevskp/covid-19-analysis-visualization-comparisons/data COVID-19 - Analysis, Visualization & Comparisons], Kaggle
* [https://covidactnow.org/ COVID Act Now] - predictions of COVID cases in the US by state
** [https://covidactnow.org/model The model used]
* [https://covid19.healthdata.org/projections COVID-19 Projections], IHME
* [https://91-divoc.com/pages/covid-visualization/?fbclid=IwAR3vdyvNKRRvfw1t_xEXMwfEO4WMA-sOEoiSF_-w5lH8aDRJMR28vcOm2J8 An interactive visualization of the exponential spread of COVID-19]
* [http://nrg.cs.ucl.ac.uk/mjh/covid19/ CoVID 19 Worldwide Growth Rates], Mike Handley, UCL
* [https://corona.help/ Corona.help], Alex Dumitru
* [https://aatishb.com/covidtrends/ Covid trends], Aatish Bhatia, Minute Physics
* [https://hidden-fjord-23808.herokuapp.com/ COVID-19 Time Exploration]
 
=== Other lists, hubs, and groups ===
 
* [https://www.kaggle.com/datasets?search=covid-19 COVID-19 data sets], Kaggle
* [https://www.reddit.com/r/datasets/comments/exnzrd/coronavirus_datasets/ Reddit thread collecting coronavirus datasets]
* [https://www.programmableweb.com/news/apis-to-track-coronavirus-covid-19/review/2020/03/18 Review of COVID-19 APIs], Wendell Santos
* [https://npgeo-corona-npgeo-de.hub.arcgis.com/ NPGEO Corona Hub 2020], Nationale Plattform für geografische Daten (NPGEO)
* [https://datarepository.wolframcloud.com/search?i=COVID Data sets for COVID], Wolfram Data Repository
* [https://www.tableau.com/covid-19-coronavirus-data-resources COVID-19 Data Hub], Tableau
* [https://coronavirus-disasterresponse.hub.arcgis.com/ COVID-19 GIS Hub], ESRI
* [https://disasterresponse.maps.arcgis.com/home/group.html?id=e0140dbc514b48b5b90c351740c14639#overview Coronavirus (COVID-19) Data], ESRI
** This is a collection of data that is available from the Esri Living Atlas as well as data from authoritative sources.
* [http://www.firemountain.net/covid19.html COVID-19 information]
* [https://coronavirustechhandbook.com/data Coronavirus Tech Handbook]
* [https://www.europeandataportal.eu/en/highlights/covid-19 European Data Portal for COVID-19]
* [https://docs.google.com/document/d/1JWeD1AaIGKMPry_EN8GjIqwX4J4KLQIAqP09exZ-ENI/edit#heading=h.ozu9c5nu5x43 Call for Action: COVID-19 Data Collaboratives]
* [https://docs.google.com/spreadsheets/d/1w4czUYSQ1AfOxw-H3Zei5LZAvNkQrSAGeIu60aeXkZs/edit#gid=0 Possible Covid datasets]
* [https://docs.google.com/spreadsheets/d/1N_aVyjQWBzPT_MHiAWyoEqNgX4nKkoU7FznVZpvFTu0/edit#gid=1936591204 COVID-19 Data Providers], Amass Insights
* [https://alan-turing-institute.github.io/COVID-19_PSTC/ COVID-19 Pandemic Symptom Trackers], Alan Turing Institute
* [https://community.wuhan2020.org.cn/en-us/ Wuhan2020]
** a real-time and synchronous data service for hospitals, factories, procurement and other information
* [https://www.covid19-dataexchange.org/initiative COVID-19 Data Exchange], Dawex
* [https://www.endcoronavirus.org/ EndCoronavirus]
* [https://midasnetwork.us/covid-19/ Online Portal for COVID-19 Modeling Research], MIDAS
* [https://console.cloud.google.com/marketplace/details/bigquery-public-datasets/covid19-dataset-list?preview=bigquery-public-datasets COVID-19 Public Datasets], Google Cloud
 
== Data or Data cleaning requests ==
 
As mentioned at the top of this page, future requests for data or data cleaning should be directed to [http://united-against-covid.org/ this data discourse page] at [http://united-against-covid.org/ United Against COVID-19].  Below are the legacy requests of this project prior to this redirect.


=== From Chris Strohmeier (UCLA), Mar 25 ===
=== From Chris Strohmeier (UCLA), Mar 25 ===


The biorxiv_medrxiv file at https://www.kaggle.com/allen-institute-for-ai/CORD-19-research-challenge contains another folder titled biorxiv_medrxiv, which in turn contains hundreds of json files. Each file corresponds to a research article, at least tangentially related to COVID-19.
The biorxiv_medrxiv file at https://www.kaggle.com/allen-institute-for-ai/CORD-19-research-challenge contains another folder titled biorxiv_medrxiv, which in turn contains hundreds of json files. Each file corresponds to a research article, at least tangentially related to COVID-19.
Line 70: Line 251:


Contact: c.strohmeier@math.ucla.edu
Contact: c.strohmeier@math.ucla.edu
=== From Juan José Piñero de Armas (U. Católica de Murcia), Mar 27 ===
We request information (on a person basis) to perform survival analyses, regressions with random effects, etc.  Some data exists for instance at
https://www.kaggle.com/sudalairajkumar/novel-corona-virus-2019-dataset/data
https://www.kaggle.com/kimjihoo/coronavirusdataset
https://www.kaggle.com/imdevskp/covid-19-analysis-visualization-comparisons/data
https://www.sirm.org/category/senza-categoria/covid-19/
but we need much more detail (date when each person was diagnosed, date of infection for the same person, discharge date, date of death, gender, age, treatments, temperatures...) not just summaries or country-aggregated data.
Contact: jjpinero@ucam.edu


== Miscellaneous links ==
== Miscellaneous ==


* [https://www.ncbi.nlm.nih.gov/research/coronavirus/ LitCovid] - a curated literature hub for tracking up-to-date scientific information about the 2019 novel Coronavirus
* [https://epcced.github.io/ramp/ Rapid assistance in modelling the pandemic: RAMP]
* [https://connect.biorxiv.org/relate/content/181 COVID-19 SARS-CoV-2 preprints from medRxiv and bioRxiv]
**  A call for assistance, addressed to the scientific modelling community Coordinated by the Royal Society
* [https://www.mygov.in/covid-19/ COVID-19] - official Indian government site
* [https://www.nsf.gov/pubs/2020/nsf20052/nsf20052.jsp Letter on the Coronavirus Disease 2019 (COVID-19)], National Science Foundation
** A solicitation for RAPID funding requests relating to COVID-19
* [https://coronacheck.eurecom.fr/en CoronaCheck: Computational Fact Checking for Statistical Coronavirus Claims], Paolo Papotti (EURECOM), Immanuel Trummer (Cornell)
* [https://en.wikipedia.org/wiki/Wikipedia:WikiProject_COVID-19 COVID-19 Wikiproject]
* [https://helpwithcovid.com/ Help with COVID]
** New or established projects helping with the COVID-19 crisis that need help
* [https://airtable.com/shrPm5L5I76Djdu9B/tbl6pY6HtSZvSE6rJ/viwbIjyehBIoKYYt1?blocks=hide COVID-19 Solutions], Airtable

Latest revision as of 19:03, 26 April 2020

This is a repository for public data sets relating to the COVID-19 pandemic. It was also initially envisioned as a clearinghouse for matching requests for data cleaning of such datasets with volunteers willing to perform this clearing, but the existing clearinghouse at United against COVID-19 is already up and running for this purpose, so we are redirecting such requests to that site in order not to fragment the pools of requests and volunteers.

For discussion of this project, see this blog post.

Data sets

Further contributions are very welcome, and can be made either directly to this wiki page (after requesting an account), or placed in the comments to this blog post, or by email to tao@math.ucla.edu.

Epidemiology

North America

Europe

Asia

Other regional data

Genomics and homology

Literature

Experts

  • COVID-19 Expert Database, Australian Academy of Science
    • A mechanism for governments, the business sector, the research sector, and other decision-makers to easily access the expertise they need to inform their decision making.

Medical imagery and records

Healthcare, vaccine development and equipment

Social and traffic data

Economic and Policy

Data scrapers and aggregators

Visualizations, projections, summaries

Other lists, hubs, and groups

Data or Data cleaning requests

As mentioned at the top of this page, future requests for data or data cleaning should be directed to this data discourse page at United Against COVID-19. Below are the legacy requests of this project prior to this redirect.

From Chris Strohmeier (UCLA), Mar 25

The biorxiv_medrxiv file at https://www.kaggle.com/allen-institute-for-ai/CORD-19-research-challenge contains another folder titled biorxiv_medrxiv, which in turn contains hundreds of json files. Each file corresponds to a research article, at least tangentially related to COVID-19.

We are requesting:

  • A tf-idf matrix associated to the subset of the above collection which contain full-text articles (some appear to only have abstracts).
  • The rows should correspond to the (e.g. 5000) most commonly used words.
  • The columns should correspond to each individual json file.
  • The clean data should be stored as a npy or mat file (or both).
  • Finally, there should be a csv or text document (or both) explaining the meaning of the individual rows and columns of the matrix (what words do the rows correspond to? What file does each column correspond to).

Contact: c.strohmeier@math.ucla.edu

From Juan José Piñero de Armas (U. Católica de Murcia), Mar 27

We request information (on a person basis) to perform survival analyses, regressions with random effects, etc. Some data exists for instance at

https://www.kaggle.com/sudalairajkumar/novel-corona-virus-2019-dataset/data https://www.kaggle.com/kimjihoo/coronavirusdataset https://www.kaggle.com/imdevskp/covid-19-analysis-visualization-comparisons/data https://www.sirm.org/category/senza-categoria/covid-19/

but we need much more detail (date when each person was diagnosed, date of infection for the same person, discharge date, date of death, gender, age, treatments, temperatures...) not just summaries or country-aggregated data.

Contact: jjpinero@ucam.edu

Miscellaneous

Retrieved from "https://michaelnielsen.org/polymath/index.php?title=COVID-19_dataset_clearinghouse&oldid=11186"