PowerBI Custom Visualizations – Part 1

Series:

http://msprogrammer.serviciipeweb.ro/2017/06/12/powerbi-visualizations-the-default-ones/

http://msprogrammer.serviciipeweb.ro/2017/06/19/powerbi-custom-visualizations-part-1/

http://msprogrammer.serviciipeweb.ro/2017/06/26/powerbi-custom-visualizationpart-2/

http://msprogrammer.serviciipeweb.ro/2017/07/03/powerbi-custom-visualizationwith-r-part-3/

 

Now I wanted to give a try of each new visualization.  There are 89 visualization on https://store.office.com/search.aspx?productgroup=powerBI .

I have played with some of the first 24 – and the .pbix file can be downloaded from https://1drv.ms/f/s!Aordxu1LWKDZgopsf3vS3IZx2UHOgQ  – you need just PowerBI Desktop.

The visualizations are

HistogramChart

TimeBrushSlicer

DualKPI

CountDown_Narrative_Scroller_Tile

JourneyGraph

RotatingChart

DrillDownDonut

Quadrant

ZoomCharts

PlayAxis

DrillDownChart

You can see the result here

PowerBI visualizations – the default ones

Series:

http://msprogrammer.serviciipeweb.ro/2017/06/12/powerbi-visualizations-the-default-ones/

http://msprogrammer.serviciipeweb.ro/2017/06/19/powerbi-custom-visualizations-part-1/

http://msprogrammer.serviciipeweb.ro/2017/06/26/powerbi-custom-visualizationpart-2/

http://msprogrammer.serviciipeweb.ro/2017/07/03/powerbi-custom-visualizationwith-r-part-3/

 

 

Because any program need visual representations, I decided to play a bit with PowerBI Desktop. I need some sample dataset to represent . There are thousands on the internet – I decided to go with Sales Transactions from here .

What I wanted to do is make one report for each visualization . The result can be found at https://1drv.ms/f/s!Aordxu1LWKDZgopsf3vS3IZx2UHOgQ  – you need just PowerBI Desktop.

 

The default visualizations that I made are :

100%StackedBarChart
Bubbles
Cards
Clustered bar chart
FilledMap
Funnel
GeoMap
LineClustered
LineStacked
PieDonut
Stacked Bar Chart
TableMatrix
TreeMap
Waterfall

 

If you do not have PowerBI , you can see images at
https://1drv.ms/f/s!Aordxu1LWKDZgopufBHcfMrTzJ_nQw

 

Again, you can download the file from https://1drv.ms/f/s!Aordxu1LWKDZgopsf3vS3IZx2UHOgQ – you need just PowerBI Desktop.

 

Next time I will do some custom visualizations from https://store.office.com/search.aspx?productgroup=PowerBI&qu=

Friday links 221

  1. Don’t tell me what my browser can’t do! | Christian Heilmann
  2. http://www.daedtech.com/avoid-code-reviews/
  3. Microsoft invests in new and expanded version of ‘Minecraft’ for the classroom, opens public preview of Learning Tools for OneNote – The Official Microsoft Blog
  4. OmarElabd/ObjectExporter: Object Exporter lets you export out an object while debugging in Visual Studio, the object can be serialized in either C#, JSON or XML. 
  5. http://www.businessinsider.com/weird-sentences-2014-1
  6. VeST Redux – Specification-driven development – SerialSeb
  7. How to customize your Search Profile in Google | Dean Hume
  8. Create a really, really simple offline page using Service Workers | Dean Hume
  9. Paging in ASP.NET Web API – Spencer Schneidenbach Spencer Schneidenbach
  10. Spencer Schneidenbach – Software Engineer for .NET and web. Problem solver. Business nerd. Spencer Schneidenbach
  11. https://www.quora.com/What-are-some-good-job-skills-that-are-easy-to-acquire/answer/John-L-Miller
  12. Time Zone Abbreviations – Worldwide List
  13. Optimize Your Email: 25 Apps to Perfect Your Subject, Body, Tone and More
  14. techcrunch.com/2016/01/24/living-off-hackathons-the-possible-rise-of-the-pro-hacker/
  15. www.codeproject.com/Articles/1073839/Create-SQL-Server-Management-Studio-Addin
  16. stackoverflow.com/questions/13045767/page-load-time-with-javascript
  17. Static vs. dynamic languages: a literature review
  18. Microsoft Garage
  19. Errata Security: How not to be a better programmer
  20. Demos of open web technologies | MDN
  21. Programming Sucks
  22. Export with Angular · Issue #19 · ignatandrei/Exporter
  23. The SQL Server 2016 Query Store: Accessing Query Store Information Using DMVs
  24. PostSharp extension
  25. export-for-trello/xlsx.js at master · llad/export-for-trello
  26. stephen-hardy/DOCX.js

Task and generic list

Could you spot the problem in transforming one code that is sequential into a Task ( threaded ) one?

First , this is the sequential one:

 

            var ret = new List<Indicator>();
            var jsonData = await data.JsonData();
            var jo = JArray.Parse(jsonData);
            var page = jo[0].ToObject<Pagination>();
            var array = jo[1].ToObject<Indicator[]>();
            ret.AddRange(array);
            var currentPage = 1;
            while (currentPage < page.pages)
            {
                currentPage++;
                jsonData = await data.JsonData(currentPage);
                jo = JArray.Parse(jsonData);
                array = jo[1].ToObject<Indicator[]>();
                ret.AddRange(array);

            }
            Debug.Assert(ret.Count == page.total, $"{nameof(ret.Count)} : {ret.Count} should be equal {nameof(page.total)} : {page.total}");
            return ret.ToArray();


 

This is the modified with task:

 

            var jsonData = await data.JsonData();
            var jo = JArray.Parse(jsonData);
            var page = jo[0].ToObject<Pagination>();
            var array = jo[1].ToObject<Indicator[]>();
            var ret = new List<Indicator>(array);

            var currentPage = 1;
            var downloads = new List<Task>();
            while (currentPage < page.pages)
            {
                currentPage++;
                var itemPage = currentPage;
                var task = data.JsonData(itemPage)
                    .ContinueWith(it =>
                    {
                        var data = JArray.Parse(it.Result);
                        var pageNr = data[0].ToObject<Pagination>();
                        var indicators = data[1].ToObject<Indicator[]>();
                       ret.AddRange(indicators);

                    }
                );

                downloads.Add(task);
            }
            await Task.WhenAll(downloads);
            Debug.Assert(ret.Count == page.total, $"{nameof(ret.Count)} : {ret.Count} should be equal {nameof(page.total)} : {page.total}");
            return ret.ToArray();

Ok. If you do not know , here is a helper:
Sometimes, at line

 ret.AddRange(indicators)

it gives the error:

System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. —> System.ArgumentException: Source array was not long enough. Check srcIndex and length, and the array’s lower bounds.
at System.Array.Copy(Array sourceArray, Int32 sourceIndex, Array destinationArray, Int32 destinationIndex, Int32 length, Boolean reliable)
at System.Collections.Generic.List`1.set_Capacity(Int32 value)
at System.Collections.Generic.List`1.InsertRange(Int32 index, IEnumerable`1 collection)

Ok. I let you think.
…..
…..
…..
…..
…..
…..
…..
…..
…..
…..
…..
…..
…..
…..
…..
…..
…..
…..
…..
…..
…..
…..
…..
…..
…..
…..
…..
…..

The answer is simple : List < T > is not Thread safe. If you have small quantities of data, does not matter. I was having 16174 records – and sometimes , when multiple threads have come to .AddRange , the internal re-dimensioning of array it does not fulfill the task ( imagine one thread redim to 100, then other 3 threads coming with request of 50+ data)

Solution 1.

Use ConcurrentBag< T > instead of List < T > . This is thread safe and does the trick

Solution 2.
Add to the array in the final of data ( not use ContinueWith) . The code is slightly low performant

           var jsonData = await data.JsonData();
            var jo = JArray.Parse(jsonData);
            var page = jo[0].ToObject<Pagination>();
            var array = jo[1].ToObject<Indicator[]>();
            var ret = new List<Indicator>(array);

            var currentPage = 1;
            var downloads = new List<Task<string>>();
            while (currentPage < page.pages)
            {
                currentPage++;
                var itemPage = currentPage;
                var task = data.JsonData(itemPage);
                downloads.Add(task);
            }
            await Task.WhenAll(downloads);

            foreach (var t in downloads)
            {
                var data = JArray.Parse(t.Result);
                //var pageNr = data[0].ToObject<Pagination>();
                var indicators = data[1].ToObject<Indicator[]>();
                //Console.WriteLine($"reading {pageNr.page} with {item.Length}");
                ret.AddRange(indicators);

            }
            //Console.WriteLine($"total records {ret.Count}");
            Debug.Assert(ret.Count == page.total, $"{nameof(ret.Count)} : {ret.Count} should be equal {nameof(page.total)} : {page.total}");
            return ret.ToArray();

You will find the code at IndicatorRepository in https://github.com/ignatandrei/WorldBankAPi

InternalsVisibleTo usage

I do not know if you know  InternalsVisibleToAttribute

https://msdn.microsoft.com/en-us/library/system.runtime.compilerservices.internalsvisibletoattribute%28v=vs.110%29.aspx?f=255&MSPPError=-2147217396  .

I have used on a project https://github.com/ignatandrei/WorldBankAPi , that deals with World Bank API to get information

Let’s take an example: Countries: http://api.worldbank.org/countries?format=json

I have had a class CountriesRepository, that deals with transforming countries Json from WorldBank to Country classes. How can I test it WITHOUT relying on reading data from the internet ?

Simple:

1. Make an interface to grab JSON , IJSonData

    public interface IJsonData
    {
        Task<string> JsonData(int page = 1);
    }

2. Make a class JSONCountry that reads JSON from HTTP and implements IJSonData

3.  Make the CountryRepository class with 2 constructors  – one with IJSonData and the default calling JSONCountry

[assembly: InternalsVisibleTo("WorldBank.Test")]
namespace WorldBank.Repository
{
    public class CountriesRepository
    {
        private IJsonData data;
        public CountriesRepository():this(new JsonCountries())
        {

        }
//this is internal - could not see by outside projects with exception of WorldBank.Test
//see InternalsVisibleTo above
        internal CountriesRepository(IJsonData data)
        {
            this.data = data;
        }
    }
}

Optional: when testing manually, save the json into same files on the hard disk – you will need for point 5

4. The code that will use CountryRepository in official way will use the default – will not see the one with IJSonData

var c = new CountriesRepository();
var data = c.GetCountries().Result;

5. The testing code will define another class JSONFromHard that implements IJSonData and reads data from Hard

class JsonFromHard : IJsonData
{
//code to read files from disk 
// you have saved those when testing manually 
//see point 2 the optional part 
}

6. Because I have defined InternalsVisibleToAttribute to the WorldBank.Test, the Test dll can see the constructor with IJSonData  and then it class with JSONFromHard   – and I have no dependecy on the Http

 

public void GetAndInterpretData()
        {
            //uses [assembly: InternalsVisibleTo("WorldBank.Test")]
            var c = new CountriesRepository(new JsonFromHard("Countries"));
            var data = c.GetCountries().Result;
            Assert.Equal(218, data.Length);
        }

You can download the project from https://github.com/ignatandrei/WorldBankAPi  to see working.