Category: Covid

Covid Data–more enhancements to GUI–and Post Mortem–part 6

Once we have all data from the backend  set , it is a matter of time to have more display options. For example, I have added % of the current day vs previous day. Also a display into table . https://ignatandrei.github.io/WFH_Resources/covidDataTable .

 

Post Mortem

  • Is  better to grab data from the source , if possible
  • Is  better to have a CI / CD in place, with the source control
  • GitHub uses are tremendous –  hosting sites that do not need a backed, store data as database, running integrations, seeing how many users see your app
  • I do regret some choices that were imposed by the API and I pay now.  A month of programming can save 1 day of architecture
  • I am not so proud about how it looks the code. I just have written, without thinking at nothing but the GUI. So in some parts is cluttered
  • The project can go on and on , adding more display ( see https://github.com/ignatandrei/wfH_Resources/issues  if you want to participate)

Covid Data – making the application – part 4

Now I need an application. The first version was a simple html file with Data Tables ( https://datatables.net/ ). You can find at https://ignatandrei.github.io/WFH_Resources/output.html . It is a simple table with search / filter and sorting.

The next , with help from Bogdan we have  started an Angular application , deployed at https://ignatandrei.github.io/WFH_Resources within the docs folder : http://github.com/ignatandrei/wfH_Resources/docs

For Angular the only thing that gives me problems were the charts : https://ignatandrei.github.io/WFH_Resources/covidData/Italy-Romania  . I used the https://www.chartjs.org/ . It is very flexible and doing all that I have wanted. The only problem : it retains previous graphic. I have to manually clear the canvas before rendering again the chart.

if (this.lineChart1) {

this.lineChart1.destroy();

}

You can find the charts at https://ignatandrei.github.io/WFH_Resources/covidData/Italy-Romania

Covid Data–transforming – part 3

I figured out that from data ( .md files and .js files) I need to have a proper data to be processed by the application. Since the application will be a static web app, the easy way is to be a generated js file to be imported into the application.

1.   MD to js

So now I must figure a way to transforme .md files into .js files. I was having dreams about Stankins , but it is not yet ready. So Js to the rescue. You can find the .js file here: https://github.com/ignatandrei/WFH_Resources/blob/master/makeData/server.js

It reads  the folders files

var folders = [“Country”,”Entertainment”,”FreeSoftware”, “Kids”,”Learn”,”MapsAndData”,”NotParsed”];

//…

for (let folder of folders) {

const directoryPath = path.join(__dirname + “/..”, folder);

const list = await rra.list(directoryPath);

then parses a file( e.g. https://github.com/ignatandrei/WFH_Resources/blob/master/Learn/DigitalLibrary.md )  with a lexer

const tokens = marked.lexer(fileContents);

for(let iToken=0;iToken<tokens.length;iToken++){

and writes to a file

fs.writeFileSync(directoryPathWrite,content);

2. Importing  Country data  for Covid

Now I need every data for Covid. I figure out the the only one that have the data is https://github.com/CSSEGISandData/COVID-19/ . However , the data is not always in the same form.

Again, because t Stankins is not yet ready, I recurse to TypeScript. It is better that JS for me – it keeps tracking the errors that you may have . You can find the file at https://github.com/ignatandrei/WFH_Resources/blob/master/makeData/importData.ts

Some pain points :

or there are not always countries – see const NotCountry where  “Diamond Princess” and “MS Zaandam” are countries ( there are cruise ships)

  • I should verify if there is always data for this day, if it exists for the previous ( yes, I know, Covid will eventually disappear or not be registered otherwise than influenza –but, for the moment ,it is ok) :  see async function Verify in https://github.com/ignatandrei/WFH_Resources/blob/master/makeData/importData.ts
  • I have had problems finding a CSV parser to TypeScript – I have found papaparse

3. Importing other data

I need to have the country population for each country, to display the relative percentage of the people . For this, I have the data from https://en.wikipedia.org/wiki/List_of_sovereign_states_and_dependent_territories_by_mortality_rate – but how to import into already existing data https://github.com/ignatandrei/WFH_Resources/blob/master/makeData/countryList.ts ?

Simple: Import data into Excel. Make a case instruction with each country, iterate via countries, ( find discrepancies in names – use alternate names for this) and put the property as js , save the file. See https://github.com/ignatandrei/WFH_Resources/blob/master/makeData/test.ts

.

Covid Data- making a database–part 2

First, I need to understand how to store the data. For the resources , I was thinking to have a simple way for the people to edit data and to add resources ( yes, I know – I am counting too much on the kindness of people)

So I figure out the a .md file will be enough clear to do it. As an example, see : https://github.com/ignatandrei/WFH_Resources/blob/master/Learn/DigitalLibrary.md

or , in the raw form, https://raw.githubusercontent.com/ignatandrei/WFH_Resources/master/Learn/DigitalLibrary.md 

For the data about how many cases of covid are, I was thinking about JSON – see https://github.com/ignatandrei/WFH_Resources/blob/master/obj/allData.js

Now , about having  the data updated :

For the resources, it is enough to someone to edit the .md file and make a pull request.

For the data about Covid per country, it will be presented later in the CI / CD deployment

Covid Data–part 1

In this days of Covid I needed 2 things:

  1. Put somewhere the list of free resources
  2. Gather the data about Covid from different countries, in order to understand the evolution of the disease

Because I am a programmer the easy way to have a database is on GitHub  – and I figure that with the docs folder I can have a static site .

You can find the free resources here :  https://ignatandrei.github.io/WFH_Resources/resources

And you can find the situation here: https://ignatandrei.github.io/WFH_Resources/covidData/Italy-Romania

How I constructed the site-  in the next posts.

Andrei Ignat weekly software news(mostly .NET)

* indicates required

Please select all the ways you would like to hear from me:

You can unsubscribe at any time by clicking the link in the footer of our emails. For information about our privacy practices, please visit our website.

We use Mailchimp as our marketing platform. By clicking below to subscribe, you acknowledge that your information will be transferred to Mailchimp for processing. Learn more about Mailchimp's privacy practices here.