approximate history of this project. total hours estimated: ~115h = 11h + 6h + 11h + 10h + 5h + 13h + 6h + 4h + 5h + 4h + 3h + 2h + 2h + 1h + 2h + 1h + 3h + 1h + 4h + 3h + 4h + 1h + 2h + 3h + 3h + 3h + 2h, plus: + 2h +2h +2h + 1h + 1h + 1h +3h + 1h + 4h + 3h + 5h + 4h + 4h + 3h + 2h + 5h + 5h + 3h +2h + 5h + 6h +2h 23/Jan/2022 17:30 github issues answered, redirect to jalsti fork; see the "So Long, and Thanks for All the Fish" issue answer --> 18:30 22/Jan/2022 10:00 found bug in risklayer data, Kreis 16056 Eisenach had errorenous nan values. But only one Kreis, so ... very ugly fix center=1. --> 12:30 21/Jan/2022 18:00 fix bug of incomplete return values in case NOTHING was wrong with CSV file --> 20:00 10/Jun/2021 05:00 yet another Risklayer input data flaw ... Last 4 days were suddenly not float anymore but int data ... Fixed by brutal casting :-) 02/May/2021 20:00 CSV file changed by Risklayer: column AGS renamed to ISO, mid April already, sorry. Also 'Haupt' sheet now not public anymore but authenticaed, switched download off completely, what a pity --> 21:15 05/Mar/2021 21:30 update failed thx Keyy for telling me, disk full & new ssl error, perhaps risklayer using https now? New Haupt sheet less source columns, fixed via QUELLEN_SPALTEN[v03]. Restarted everything. --> 01:00 ... some missing? 21/Oct/2020 20:30 began with trying out 2 of jalsti's pull requests. Modified his code, more talkative, timings. Long email. --> 22:45 19/Oct/2020 20:20 populationDensity.ipynb added (SQRT, SQRT) --> 21:30 15/Oct/2020 14:30 populationDensity.ipynb --> 18:30 14/Oct/2020 14:15 opendatasoft killed the table that is downloaded in districtDistances.py . Bad, sent them an email. Many recent emails, several pull requests by jalsti - hooray. VERY good. --> 15:45 19/Sep/2020 04:30 risklayer CSV had breaking changes: 'Population' = column 3, suddenly introduced, since Sept 11th already, ouch. Caught and fixed. Also email to them. --> 05:55 10/Sep/2020 15:45 new column 'total cases' so that table is sortable by that. Typed many ideas for @StefFun --> age structure over time --> 17:30 09/Sep/2020 09:05 districtDistances.py <-- make available to others, quickstart instructions, try it out. Tweeted --> 09:55 31/Aug/2020 16:30 choice.html = choose diagrams to combine into HTML table. v01 = fantastic work by @gibbsnich, extended by me in v02. https://github.com/covh/cov19de/issues/1#issuecomment-683846767 --> 17:30 04/Aug/2020 13:05 email risklayer 5 topics; CSV error "only 1 column", instead read as SSV, good error msgs; show app VERSION; push to heroku (pause 30 minutes) --> 16:50 28/Jul/2020 21:40 new hotspots MANY new hotspots in these 11 days --> 22:25 17/Jul/2020 07:30 heroku, new hotspots --> 9:00 12/Jul/2020 evening repeat.sh for execution after 2am: ~60 minutes incl testing & checking a few times 09/Jul/2020 03:50 new hotspots --> 04:15 01/Jul/2020 14:15 automate spreadsheet downloading? --> 16:25 and 16:55 --> 19:40 30/Jun/2020 19:15 new hotspots, additional category '14-days-incidence high' 27/Jun/2020 19:30 covhist spin-out --> 20:00. and 22:15 --> 23:45 ... 26/Jun/2020 14:55 scripted mass download of many 'Haupt' sheets, treat different time formats, run some tests, catch data typos, inspect CSVs. probably DONE. --> 16:11 and 17:33 --> 18:30. ... 25/Jun/2020 17:00 scripted mass download of many 'Haupt' sheets, add range and zeit column --> 17:40 24/Jun/2020 22:15 scripted mass download of many 'Haupt' sheets; half-ready --> 23:15 24/Jun/2020 13:20 dataFiles.py - COMMENT each function, refactor small bits, more consistent order of sections, etc. = make code more logical for third party (and me). Info header into ALL py files --> 15:50 24/Jun/2020 11:30 concept "INSTRUCTIONS - generate risklayer time series, from version history" --> 12:30 21/Jun/2020 10:25 neue Hotspots 20/Jun/2020 09:00 neuer hotspots, tweets an mich wg projekt, GT plus 100km because such a large outbreak, update tweet, heise: sponsor posting ... --> 12:30 18/Jun/2020 14:30 plot.legend - loc='upper left' nicht 'best' 16/Jun/2020 05:40 new hotspots, incl tweet --> 06:30 13/Jun/2020 02:00 new hotspots, incl tweet --> 2:30 12/Jun/2020 05:45 refactor early routines, to recycle them for readonly filesystem on heroku. Many small & large tweaks & tricks ... now it does run on heroku. Hooray. Useful for CSV checking online! --> 08:45 ... 11/Jun/2020 02:00 bottle server. heroku app. needs different approach to data, not in files but in RAM, because read-only-filesystem. almost there. --> 06:30 11/Jun/2020 01:30 'Haupt' sheet was changed, they replaced one column "Gestrige Quellen" (v01) --> "GESAMT Bestätigte ZAHL der Infektionsfälle (Crowdsourced) inc. Epidemiological" (v02). Updated googlesheet. --> 02:00 02/Jun/2020 17:45 new hotspots; tweeted --> 18:30 02/Jun/2020 17:00 correcting RiskLayer typo 'o'. In a more general way now, so future flaws are quicker to fix --> 19:00 02/Jun/2020 00:30 importing dates from 'Haupt' sheet needs to define dataformat, not to fall back to US scheme. Plus: another error in risklayer CSV: 'o' instead of '0' --> 02:00 29/May/2020 20:30 ideas how to identify hotspots. --> 21:15 29/May/2020 15:00 risklayer fixed their CSV. Generating new site. check feedback. --> 15:30 29/May/2020 00:30 source data broken CSV again. Noticed it because negative center day. Then ... 91 districts with falling cases for 2 days ago to yesterday. Sum is correct. Probably rows got mixed up. --> 02:00 27/May/2020 01:00 finish the fresh machine install; repair typos; data - assume unchanged, attribution text ; correct timezone; new hotspots --> 02:30 ... 26/May/2020 03:00 fresh machine install, improving instructions in README.md --> 04:30 25/May/2020 00:30 new error in risklayer CSV. Giving up for now. ... (giving up helps) Hooray, found it. --> 04:30 24/May/2020 16:00 automation script "scripts/downloadAndUpdate.sh" with full logging. heise forum answer emails. 24/7 install. Repo "initialize.sh". To sync via 2 machines was complex. But: done. --> 21:00 ... 23/May/2020 17:00 conditionally generated pages IFF new data CSV or manually wanted --> full automation possible soon. Tweeted summary --> 18:30 ... 22/May/2020 22:00 downloads: find out and print if downloaded data was new or already on disk --> 22:45 22/May/2020 15:00 new entries in hotspots.html, tweet --> 15:45 21/May/2020 17:00 ctd. Side effect of downloading "haupt": I now also have other columns, e.g. the WEB SOURCE URLs --> added below each district plot. --> 19:00 ... 21/May/2020 14:00 risklayer has not updated the CSV. How can I still get updated data? Trying to download CSV from their master sheet ... interactive4.ipynb --> 16:00 20/May/2020 14:45 new data, new hotspots, tweet germany --> NRW --> AGS 5382 --> neighbours --> 16:00 ... 18/Mai/2020 22:30 cool, mybinder.org allows to execute Jupyter notebooks in the browser, and including all dependencies because the repo gets turned into a docker container. Nice. See README.md and risklayer-data.html --> 00:30 17/Mai/2020 13:45 emailed the first two in https://www.heise.de/forum/p-36681505/ to point to my answer. BTC wallet had received first donation! --> 14:45 16/Mai/2020 17:15 forum summary post https://www.heise.de/forum/p-36681505/ --> 18:15 ... 16/Mai/2020 14:00 new hotspots overview tables. modify plots: red for last 2 weeks, triangle for "expectation day". rename 'center day' to 'expectation day' everywherein code and pages --> 15:30 15/Mai/2020 13:00 forum: 7 answers, added some todos --> 15:00 13/Mai/2020 18:00 table of neighboring districts, first version of hotspots page --> 21:30 12/Mai/2020 09:30 forum answers --> 11:15 12/Mai/2020 18:00 five answers to forum posts below heise.de article --> 19:30 12/Mai/2020 10:30 twitter DMs with a few ideas; added to todo.md, e.g. daily cases versus total cases; reordered todo.md; began some git scripts --> 13:00 11/May/2020 08:00 instructions how to import the risklayer data into a Python Jupyter Notebook. All German language tweets in one thread --> 11:00 10/May/2020 17:00 tweeted highest incidence districts as update to article. Accepted to Github Sponsors program, hooray, edited tiers and added description. --> 19:15 10/May/2020 14:00 expand table area or use scrollbar - because some users could initially not find the district they had been searching for. v02.8.1 --> 14:45 10/May/2020 11:30 read heise forum; search last week for "kreis" / "kreis OR kreissitz" on duckduckgo & google --> added to every kreis 'other sites', and announcement tweet v02.8 --> 14:00 09/May/2020 11:00 article forum - good idea: fixed table head, implemented via CSS. v02.7.2 --> 12:15 09/May/2020 06:00 article - watch forum discussions, add to todo.md. Reff_7_4 thoughts. Github sponsors. Thank publisher. --> 10:00 08/May/2020 20:30 Final version of heise article. Added infos to about.html: (A4) other sites, and disclaimer. Testing new and improved Reff_7_4(cumulative) --> 23:30 08/May/2020 14:20 todo.md; article publisher communication; TU-Dortmund linking; wikipedia, crawl data table ... now 294 Kreise have 2 wikipedia links! - --> 16:20. 17:20 --> 19:20. 08/May/2020 10:30 article v02, sent to publisher --> 11:45 08/May/2020 05:45 article v01 --> 09:00 07/May/2020 18:00 plots 500 / week / 1 mio population --> 19:00 07/May/2020 12:45 tweaked color scheme for tables; screenshots for German article --> 15:30 07/May/2020 10:00 python dependencies requirements.txt; longer README.md better getting started; added to todo.md --> 11:45 06/May/2020 14:00 trying out code analysis ...; error / confidence intervall for R_eff ??? --> 15:00 06/May/2020 03:25 show summaries (totals, diffs, etc) when downloading new data; generated the 6.5.2020 pics & pages; googlesheet copy; sorting desc first, link to maskUnity --> 06:00 05/May/2020 16:15 seven newspapers: Do you want to write an article about this? Or want me to write one? Teaser image cov19de-overview-v02.6-smaller.png --> 17:45 05/May/2020 11:45 mobile better: divs to limit width, with scrolling; some fonts smaller; reread everything; tweet layout improved --> 15:00 05/May/2020 01:30 updating data; dt Artikel Grobkonzept; repairing Reff to only take POSITIVE values as input. --> 03:30 04/May/2020 21:30 facebook testimonial, arbitrary regions idea - JavaScript, see issue 1 --> 23:15 03/May/2020 20:30 testimonials.html - screenshots, puzzled together in image, asked all 4 whether fine with them, made page, inserted as IFRAME into index.html; emails with heise.de; created 4.5.2020 pages and googlesheet copy. --> 01:30 03/May/2020 00:45 reproduction number estimate Reff_4_7; generated new pages, copied googlesheet; tweet. --> 05:15 01/May/2020 23:00 sort by anything ... new: 2-weeks incidence and absolute new cases; repaired sorting for when several sortable tables on one page; googlesheet copy & generated 2.5.2020 pages; new tweet --> 02:15 01/May/2020 19:00 sorting the tables when clicking a table header column --> 22:15 01/May/2020 00:30 new in Deutschland.html - 4x4 table integrated now, better navigation; communications follow up; googlesheet copy & generated 1.5.2020 pages; new tweet "linked projects" --> 04:00 30/April/2020 18:00 Other corona data people in Germany; Communications, heise, TU Dortmund, tweets. Googlesheet too often too broken, own copy now. --> 23:00 30/April/2020 08:00 tweets (also in German) --> 09:30 30/April/2020 05:00 ~ready; pushed new pages to server 30/April/2020 02:00 generate new plots, assumptions chapter extended; add 2x401 Kreis hyperlinks to the googlesheet prevalence / mortality tables; screenshots, link on Deutschland.html 29/April/2020 22:45 twitter feedback "plot by bundesland" --> Built 'bundeslaender_plots.html' as quick fix. --> 00:45 29/April/2020 17:30 add feedback to TODO; repair dirty data risklayer; twitter QaA... --> 19:45 29/April/2020 04:30 beta version ~ready ? 28/April/2020 17:00 presented to citizen, collected ideas; flags, font, column prevalence, total cases blue font, reduce images size to 66%, explain curve colors?, which day of the week are the peaks? Many small improvements. New Deutschland page now also contains huge kreise table. Much more. 28/April/2020 04:30 finish for today 28/April/2020 02:00 Bundesland & Deutschland pages, many improvements. about.html page 28/April/2020 01:00 bundesland page is ~working, early version 27/April/2020 22:00 continued with links and pages 27/April/2020 21:15 pairwise distances for 402 locations ~ ready 27/April/2020 16:45 continued; this history file 27/April/2020 05:35 file "interactive2.ipynb", went to bed 27/April/2020 05:23 newest Kreis plot 'Kreis_14524.png', one of 445 items in repo, so first version of generating all the plots (Kreis, BuLa, Germany) was ready. 27/April/2020 05:05 newest file "dataPlotting.cpython-37.pyc" so by then the refactoring was ~ready 26/April/2020 23:26 refactoring interactive.ipynb --> interactive.py 26/April/2020 09:52 file on disk "LICENSE" in the new repo 'covh/covviz', then went to bed 26/April/2020 09:38 file on disk "interactive.ipynb", the first version of the jupyter notebook 26/April/2020 07:32 file on disk "bundesland_Dummyland.png" so a first version of the Bundesland aggregation and plotting worked by then 26/April/2020 04:47 oldest "Kreis" file on my disk 'Kreis_09377.png' so by then the very first version of plotting a single "Kreis" must have worked 25/April/2020 22:45 oldest LibreOffice file on my disk 'GermanyValues_RiskLayer-20200425.csv.ods', so 1-2 hours before I must have started looking at the data 15/April/2020 got risklayer's time series data - but more importantly, this text needed attention first. Regardless how much I promoted it (and I did a lot), it never caught on, please explain why if you can. See medium: tiny.cc/pTools maths: tiny.cc/stoch swag: tiny.cc/maskUnity pdf: tiny.cc/pTpdf