From unstructured data to application

Imagine that you have this kind of texts:

Alpert of Metz – 11th-century – France – Medieval writers
Aimoin of Fleury – c. 960 – c. 1010 – France – Medieval writers
Amulo Lugdunensis (Archbishop of Lyon) – 9th-century – France – Medieval writers
Amulo Lugdunensis (Archbishop of Lyon) – 9th-century – France / Carolingian Empire – Medieval writers
Andrew of Fleury – 11th-century – France – Medieval writers
Angelomus of Luxeuil – 9th-century – Francia / France – Medieval writers

Vitsentzos or Vikentios Kornaros or Vincenzo Cornaro – c. 1553 – c. 1614 – Kingdom of Candia / Crete / Greece – Renaissance

Adrianus of Tyre – c. 113 – c. 193 – Greece / Roman Empire – Ancient & Classical writers

How transform this unstrcutured data to make an application that search like in the picture below ( just the R from CRUD )?

Detailing  the tools:

1.  Transforming data: Excel + VBA – to parse from the text the name, country / countries , years period  movement and generate insert  /stored procedurees call

2. Storing+ retrieving data: Sql Server – creating tables and stored procedures to insert / retrieving data

3. Acessing data: .NET Core – to make WebAPI to export as WebAPI HTTP endpoints t

4. Display data:

5. Deploy:

  • RedfHat OpenShift to deploy site
  • Azure to have sql server on the web

Hours or work ~ 20 . And hey, it works!

.NET Core SignalR Hub+ Angular Observable

TL;DR: Deliver all the table content continuously paginated to the browser with .NET Core and Angular


Long Description:

I was thinking that delivering paginating data is kind of lame. Why should I , for seeing the whole data, should keep pressing next page / or scrolling  down ? I understand that on the server it is necessary to do this – but on the client ?

And I was thinking – what about Observables in Angular  , combined with SignalR ? Those observables could obtain data every time – and SignalR is pushing cotinously, until it arrives at the end of the data.

Done with talking –  let’s show some code:

We have an Employee classs

public class Employee
    {
        public string Name { get; set; }
        public int Id { get; set; }

    }

code

and we are feeding with data in a DALEmployee

public class DalEmployee
    {
        public int FindCountEmployees(string name)
        {
            return ((name ?? "").Length + 1) * 10;
        }

        public async Task<Employee[]> FindEmployee(int pageNumber, int pageSize, string name)
        {
            await Task.Delay((pageNumber + 1) * 1000);
            var ret = new Employee[pageSize];
            int startNumber = (pageNumber - 1) * pageSize;
            for (int i = 0; i < pageSize; i++)
            {
                var e =new Employee();
                e.Id = startNumber + i;
                e.Name = e.Id + "employee ";
                ret[i] = e;
            }

            return ret;
        }
    }

We have SignalR that delivers continously the data via a Hub that is connected to an URL

//do not forget  services.AddSignalR().AddJsonProtocol()  ;
            app.UseSignalR(routes =>
            {
                routes.MapHub<MyHub>("/employees");
            })

and delivers all data continuously

 public class MyHub: Hub
    {
       
        public override async Task OnConnectedAsync()
        {
            Console.WriteLine("!!!!connected");
            await Task.Delay(1000);
        }
        public ChannelReader<Employee[]> GetEmps(string name)
        {
            var channel = Channel.CreateUnbounded<Employee[]>();
            _ = WriteItems(channel.Writer, name); //it not awaits!

            return channel.Reader;
        }

        private async Task WriteItems(ChannelWriter<Employee[]> writer, string name)
        {
            var dal = new DalEmployee();
            var nr = dal.FindCountEmployees(name);
            int nrPages = 10;
            for (int i = 0; i < nr / nrPages; i++)
            {
                var data = await dal.FindEmployee(i+1, nrPages, name);
                await writer.WriteAsync(data);
            }

            writer.TryComplete();            
        }

And this is all that is required on the .NET Core side

For the Angular Part

We have a hubconnectionbuilder that connect to the hub

export class MydataService {

  h:HubConnection;
  constructor(private http: HttpClient) { 
    this.h= new HubConnectionBuilder()
      .withUrl(environment.urlAPI + "/employees")
      .withHubProtocol(new JsonHubProtocol())
      //.withHubProtocol(new MessagePackHubProtocol())
      .build();
      window.alert('start');
    this.h.start()
    .catch((t) => window.alert(1));
  }
  
  
}

and a observable that returns the values



  public getAllEmpsStream(name:string): Observable<Employee[]>{
    
    const subject = new Subject<Employee[]>();
    this.h.stream<Employee[]>("GetEmps",name).subscribe(subject);
    return subject.asObservable();

  }

The code on the component is aggregating the values into an array

findEmployees(){
    this.load=true;
    this.service
      
      .getAllEmpsStream(this.nameEmp)
      //.getEmps(this.nameEmp)
      .pipe(finalize(() => this.load=false))
      .subscribe(it  => {
        
        //window.alert(it.length);
        this.emp.push(...it);

      });

  }

And that is all. All the values are returned page after page without the need of human intervention…

Programmer Tools I use in 2017/2018

Development Languages / Frameworks

.NET Core

C# / F#

Javascript

Powershell

Angular

IDE

Visual Studio Code

Visual Studio Community Edition

SSMS

Powershell IDE

Visual Studio Addons

GhostDoc

Resharper

Visual Studio Code addons

Auto Close Tag

Auto Rename Tag

C#

Debugger For Chrome

TSLint

Visual Studio Keymap

Version Control

GitHub

Perforce

Mercurial

VSTS Online

Office

Excel

Word

Powerpoint

Reveal.js

Gliffy

Javascript Components

Jquery

Jquery UI

knockoutjs

Select2

jstree

Cordova

Electron

RxJS

Angular Components

Angular Material

angular-cli-ghpages

VMWare Clarity

.NET Components

SignalR

Roslyn

Benchmark.net

Servers

IIS

Jenkins

Nuget Gallery

Programmer Utilities

NuGet Package Explorer

Fiddler

Sysinternals ADSI

Postman

Selenium IDE

Reflector

WordPress

Windows Live Writer

Notepad ++

NuGetPackages

MediatR

structuremap

EntityFramework

nlog

Westwind.RazorHosting

iTextSharp-LGPL

ExporterWordExcelPDF

T4MVC

Swashbuckle.Core

NReco.VideoConverter

xunit

HtmlAgilityPack

Microsoft.AspNet.WebApi.Client

OneNoteOCR

NewtonsoftJSon

Database

Elastic search

Sql server

Mongodb

Reporting

Excel

PowerBI

KIbana

Other

Terminals – multiple Remote Desktop

VLC

AutoHotKey

TreeSizeFree

Camtasia

7zip

FreeCommander

Organizer

Todoist

Trello

Communications

Skype

Outlook

Slack

Yahoo

Gmail

Browsers

Chrome

Edge

InternetExplorer

Firefox

Sites Hosting

Azure

AppHarbor

Heroku

Chrome Addons

RemoveOverlay

TextMode on/off

FoxClocks

Silent Site Sound Blocker

Panic

Right Click New tab

Todoist

OneTab

Export for Trello

Angular , .NET Core, IIS Express and Windows Authentication

2 years ago I have written about .NET Core Windows Authentication :http://msprogrammer.serviciipeweb.ro/2016/09/26/asp-net-core-and-windows-authentication/ 

The idea was that for Windows authentication this is mandatory in Web.Config:

forwardWindowsAuthToken="true"

Now I want to show how to achieve the same in  IIS Express .NET Core as backend and Angular for front end

For .NET Core / IIS Express

1. Configure IISExpress : https://www.danesparza.net/2014/09/using-windows-authentication-with-iisexpress/ 

On short: find %userprofile%\documents\iisexpress\config\applicationhost.config  or %userprofile%\my documents\iisexpress\config\applicationhost.config  and put

<windowsAuthentication enabled="true">
    <providers>
        <add value="Negotiate" />
        <add value="NTLM" />
    </providers>
</windowsAuthentication>

2.  On your project: Put in launchSettings.json :

“windowsAuthentication”: true,
“anonymousAuthentication”: false,

3.  Optional: To verify add an Action that returns

(this.User?.Identity?.Name ?? "Not ad enabled")

For Angular

return this.http.get(url,{
withCredentials: true
});

4. ASP.NET Core 2.1
https://github.com/aspnet/Home/releases/tag/2.1.0

Windows authentication is not enabled after creating project with the Windows authentication option selected
When creating a template with Windows authentication, the settings to enable Windows authentication in launchSettings.json are not applied.
Workaround: Modify launchSettings.json as follows:

“windowsAuthentication”: true,
“anonymousAuthentication”: false,

.NET Core 2.0 WebAPI with .NET Framework 4.6( for COM interoperability)

The problem :

I want some .NET Core Web API in Kestrel mode that have access to some COM components ( such as Word.Application )

Solution:

( Theory : in order to have dll’s that works in .NET Framework application and in .NET Core, you should have .NET Standard 2.0 along .NET Framework 4.6.1 and .NET Core 2.0 ( see https://docs.microsoft.com/en-us/dotnet/standard/net-standard )

Step 1 :  create the .NET 4.6.1 dll that access the COM object

Step 2: create the .NET WEB API application. Edit the .csproj file and replace the target framework with

<PropertyGroup>
<TargetFrameworks>net461</TargetFrameworks>
</PropertyGroup>

Save the .csproj

Step 3 ; Remove the Microsoft.AspNetCore.All metapackage and install yours . For me it was enough to install those 3:

install-package microsoft.aspnetcore.mvc

install-package Microsoft.AspNetCore

install-package Microsoft.AspNetCore.Diagnostics

( you may want to add CORS and/or others)

Step 4: run

dotnet restore

and

dotnet build

Step 5: Reference the dll that you have created in the Step 1  and reference from a controller action. Build again.

Step 6; Check that you have now an exe file in build / debug / net461. You can run this exe / deploy in IIS /

Andrei Ignat weekly software news(mostly .NET)

* indicates required

Please select all the ways you would like to hear from me:

You can unsubscribe at any time by clicking the link in the footer of our emails. For information about our privacy practices, please visit our website.

We use Mailchimp as our marketing platform. By clicking below to subscribe, you acknowledge that your information will be transferred to Mailchimp for processing. Learn more about Mailchimp's privacy practices here.