New side project- WebAPI To CLI–part 1

This is a part of the series where about how I made the WebAPI2CLI - Execute ASP.NET Core WebAPI from Command Line
Source code on https://github.com/ignatandrei/webAPI2CLI/
1WebAPI2CLI - Description
2WebAPI2CLI- Organization
3WebAPI2CLI - implementing
4WebAPI2CLI - tests
5WebAPI2CLI - Devops and CI/CD
6WebAPI2CLI - documentation
7WebAPI2CLI - Conclusions

I have had an idea; What if the WebAPI application can be called from the console application itself  ?

This will permit just having the DI running  – and all the stuff that WebAPI can work with when calling a controller.

Will be good in a CI / CD workflow – testing the full API instead of relying to test just the dll ( Yes, I know about skinny controllers )

And will be good in Jenkins / any other tool / in order to execute directly the WebAPI instead of recurring to curl / powershell

 

So – seems to have a good piece of software to start – how we do this ?

First, how all the commands should be passed ? If we pass via command line, there is pretty much possible for someone to miss things.

Second , if the user does not pass via command line, the WebAPI should work as usual.

So , 5 days and 88 commits later , this is the result: https://github.com/ignatandrei/webAPI2CLI/ .

It has the documentation about how to install and use it. Enjoy !

 

GitHub Actions and docker for writing a book

I am trying to wrote a book  ( with Daniel Tila ) about what are the challenges to transfer an application from console to SAAS.

We have decided to put the code on GitHub – and , because of this, the text will be still on GitHub .Each chapter will be in the form of the readme.md file – usual for GitHub.

Now – how we can make a PDF / HTML out of this ?

So, first challenge: how to make the PDF ? In these days, all is node-js – and I have found https://www.npmjs.com/package/markdown-pdf .

So I have created a simple package.json file

{

“name”: “print”,

“version”: “1.0.0”,

“description”: “print on Docker”,

“author”: “Andrei Ignat”,

“scripts”: {

“start”: “node server.js”

},

“dependencies”: {

“markdown-pdf”: “10.0.0”

}

}

and a javascript file

‘use strict’;

console.log(`start print`);

var markdownpdf = require(“markdown-pdf”)

var mdDocs = [

“README.md”,”Chapter01/readme.md”, “Chapter02/readme.md”,”Chapter03/readme.md”,”Chapter04/readme.md”

]

, bookPath = “book.pdf”

markdownpdf().concat.from(mdDocs).to(bookPath, function () {

console.log(“Created”, bookPath)

})

And how I can execute ? Docker seems to be the answer ; run the javascript inside  docker ( in order to not install on my pc all kind of tools)

The docker file is simple

FROM node:10

WORKDIR /usr/src/

COPY print/package.json ./print/

RUN npm install ./print/

COPY . .

RUN ls -l print/*.*

RUN node print/server.js

CMD tail -f /dev/null

and to obtain the pdf book:

docker build -t ignatandrei/printsaas .. -f ./exportPDF.txt

docker run -d –name printsaas ignatandrei/printsaas

docker cp printsaas:/usr/src/book.pdf .

docker container kill printsaas

docker container rm printsaas

You can find all those files on https://github.com/ignatandrei/console_to_saas/tree/master/print .

Where should I put the PDF ? Turns out the GitHub has the docs folder – where you can put any kind of static data . Ad this is available for any project – by switching the name of the project from source code url https://github.com/ignatandrei/console_to_saas to https://ignatandrei.github.io/console_to_saas/

Now , the problem is how to run this every time ? Remember that GitHub ( like any other source control) has Actions – that can run every time something is pushed on the repository. So this is the yml file:

name: ‘CD for generate book and docs folder’

on:

push:

paths:

– ‘**.md’

jobs:

build:

runs-on: ubuntu-latest

steps:

– uses: actions/checkout@v2

– name: print pdf

run: |

        chmod +x ./print/getBook.bat

        cd print       

        ./getBook.bat

        cp ./book.pdf ../docs/ConsoleToSaas.pdf

– uses: actions/upload-artifact@v1

with:

name: ConsoleToSaas.pdf

path: ./print/book.pdf

– name: Commit files

run: |

        git config –local user.email “action@github.com”

        git config –local user.name “GitHub Action”

        git commit -m “generate pdf” -a –allow-empty

– name: Push changes

uses: ad-m/github-push-action@master

with:

github_token: ${{ secrets.GITHUB_TOKEN }}

( see https://github.com/ignatandrei/console_to_saas/blob/master/.github/workflows/main.yml )

So now I obtain the PDF automatically generated by docker, I put into docs folder( and into artifact ), and I commit the PDF back to the repository( and yes, GitHub is enough smart to not start the cycle again)

Now I wanted to have also HTML – or for any other format  . So I figure I need a general converte – and I have found PanDoc.

The code is more simple and just in the yml

convert_via_pandoc:

runs-on: ubuntu-18.04

needs: build

steps:

– uses: actions/checkout@v2

– run: |

          git pull

          mkdir output

– uses: docker://pandoc/latex:2.9

with: # needs a README in your repo root!

#args: “–standalone –output=output/README.html README.md”

args: “README.md Chapter01/readme.md Chapter02/readme.md  Chapter03/readme.md Chapter04/readme.md –standalone -f gfm -t html –number-sections –toc -o output/output.html”

– uses: actions/upload-artifact@master

with:

name: output

path: output

– run: |

          cp ./output/output.html ./docs/index.html

          rm -rf ./output

– name: Commit files

run: |

          git config –local user.email “action@github.com”

          git config –local user.name “GitHub Action”

          git commit -m “generate html” -a –allow-empty

– name: Push changes

uses: ad-m/github-push-action@master

with:

github_token: ${{ secrets.GITHUB_TOKEN }}

( see https://github.com/ignatandrei/console_to_saas/blob/master/.github/workflows/main.yml )- 

Some explanations:

  1. I need to serialize the 2 jobs – they cannot commit in parallel
  2. I need a git pull because the previous jobs commits  – and , somehow, the sources are just once read – not at run time
  3. I delete the output folder to not commit to the repository
  4. To commit , the file must exists and will be overwritten by the cp linux command ( that overwrites by default)

You can find the end result at https://ignatandrei.github.io/console_to_saas/ (make sure you see the download section)  and the source code at https://github.com/ignatandrei/console_to_saas

.NET Core global tool CD/CI with AzureDevops to Nuget

I want to can deploy automatically from GitHub  to Nuget a .NET Global ( or local ) tool.

It is enough simple;

1. Get an API key from GitHub

2. Goto to your AzureDevops , settings ( down the page ) and connect with a new Service Connection AzureDevops to GitHub ( name: nugetAndrei)

3. Make an yaml file similar with this:

# ASP.NET Core

# Build and test ASP.NET Core web applications targeting .NET Core.

# Add steps that run tests, create a NuGet package, deploy, and more:

# https://docs.microsoft.com/vsts/pipelines/languages/dotnet-core

#https://dev.azure.com/ignatandrei1970/AOPRoslyn/

pool:

vmImage: ‘VS2017-Win2016’

variables:

buildConfiguration: ‘Release’

deployNuget: ‘0’

steps:

– script: |

    cd AOPRoslyn

    dotnet tool restore

displayName: ‘restore tool’

– script: dotnet restore AOPRoslyn\AOPRoslyn.sln

displayName: ‘restore project’

– script: |

    cd AOPRoslyn

    dotnet tool run pwsh -f ./makenuget.ps1

displayName: ‘powershell to version and pack’

– script: dotnet build AOPRoslyn\AOPRoslyn.sln –configuration $(buildConfiguration)

displayName: ‘dotnet build $(buildConfiguration)’

– script: dotnet pack AOPRoslyn\aopCmd\aop.csproj  –no-build -o $(Build.ArtifactStagingDirectory) /p:Configuration=$(buildConfiguration) # –verbosity Detailed

displayName: ‘dotnet pack ‘

– task: PublishBuildArtifacts@1

inputs:

pathtoPublish: ‘$(Build.ArtifactStagingDirectory)’

artifactName: drop1

# – task: NuGetAuthenticate@0

#   inputs:

#     nuGetServiceConnections: ‘nugetAndrei’

– task: NuGetCommand@2

condition: and(succeeded(), eq(variables[‘deployNuget’], ‘1’))

inputs:

command: push

nuGetFeedType: external

packagesToPush: ‘$(Build.ArtifactStagingDirectory)/**/*.nupkg’

publishFeedCredentials: ‘nugetAndrei’

displayName: ‘dotnet nuget push’

# – task: DotNetCoreCLI@2

#   displayName: Push Nuget Package

#   inputs:

#     command: custom

#     custom: nuget

#     arguments: >

#       push $(Build.ArtifactStagingDirectory)\*.nupkg

#       -s https://api.nuget.org/v3/index.json

#       -k $(NuGetSourceServerApiKey)

You can find the source here:
https://github.com/ignatandrei/AOP_With_Roslyn/commit/e5b244f61e77cd0c5ec34c97e1a0122f9625de25

.NET Core Multiple Authentication–Windows AD, Azure AD, Database

I was having a project about how to do in .NET Core multiple authentication: Windows AD, Azure AD, Database – and integrate with Roles.

It was an interesting project – and I decide to not make everything – but to construct on an existing project.

The most promising sounds IdentityServer – with the Federation Gateway. : http://docs.identityserver.io/en/latest/topics/federation_gateway.html

So I decided to give a twist , and after many readings and searching , I have found https://github.com/damienbod/AspNetCoreWindowsAuth . It was pretty amazing  – however, some problems occurred.

 

External Integration – with local Active Directory

See in Startup.cs the following

    services.Configure<IISOptions>(iis =>
        {
            iis.AuthenticationDisplayName = "Windows";
            iis.AutomaticAuthentication = true;
        });

When the user clicks the Windows authenticatio the following code is called

public class AccountController : Controller
{
    //ommitted code
    [HttpGet]
    public async Task<IActionResult> ExternalLogin(string provider, string returnUrl)
    {
        if (AccountOptions.WindowsAuthenticationSchemeName == provider)
        {
            // windows authentication needs special handling
            return await ProcessWindowsLoginAsync(returnUrl);
        }
        //ommitted code
    }
    //ommitted code
    private async Task<IActionResult> ProcessWindowsLoginAsync(string returnUrl)
        {
            // see if windows auth has already been requested and succeeded
            var result = await HttpContext.AuthenticateAsync(AccountOptions.WindowsAuthenticationSchemeName);
            if (result?.Principal is WindowsPrincipal wp)
            {
                //ommitted code
            }
            else
            {
                // trigger windows auth
                // since windows auth don't support the redirect uri,
                // this URL is re-triggered when we call challenge
                return Challenge(AccountOptions.WindowsAuthenticationSchemeName);
            }
        }
}

External Integration – with Azure Active Directory

This was by far the most complicated
As a pre-requisites , we need to configure the Azure Active Directory and grab the client Id for the application.

The code in the startup.cs os

        services
        .AddAuthentication(IISDefaults.AuthenticationScheme)
        .AddOpenIdConnect("aad", "Sign-in with Azure AD", options =>
        {
            //options.Authority = $"https://login.microsoftonline.com/common/v2.0/";
            //options.Authority = $"https://ignatandreiyahoo.onmicrosoft.com";
            //options.Authority = $"https://login.windows.net/{tenantId}";
            options.Authority = "https://login.microsoftonline.com/common/v2.0/";
            options.ClientId = $"{clientId}";
            //options.RequireHttpsMetadata = true;
            options.RemoteAuthenticationTimeout = new System.TimeSpan(0,1,58);

            options.SignInScheme = IdentityServerConstants.ExternalCookieAuthenticationScheme;
            options.SignOutScheme = IdentityServerConstants.SignoutScheme;

            options.ResponseType = OpenIdConnectResponseType.IdToken; //"id_token";
            options.CallbackPath = "/signin-aad";
            options.SignedOutCallbackPath = "/signout-callback-aad";
            options.RemoteSignOutPath = "/signout-aad";

            options.TokenValidationParameters = new TokenValidationParameters
            {
                ValidateIssuer = false,
                //ValidAudience = "f59d5739-1ec9-46fc-961d-b01ef6fb3c51",

                NameClaimType = "name",
                RoleClaimType = "role"
            };
            options.Events.OnRemoteFailure = (context) =>
            {
                string s = context.ToString();
                return Task.CompletedTask;
            };
        })

        ;
        services.AddOidcStateDataFormatterCache("aad");
   

And the code that retrieves the user is:

public class AccountController : Controller
{

    [HttpGet]
    public async Task<IActionResult> ExternalLoginCallback()
    {
        //ommitted code
        // read external identity from the temporary cookie
        var result = await HttpContext.AuthenticateAsync(IdentityConstants.ExternalScheme);
        if (result?.Succeeded != true)
        {
            result = await HttpContext.AuthenticateAsync("aad");
            if (result?.Succeeded != true)
            throw new Exception("External authentication error");
        }

        // lookup our user and external provider info
        var (user, provider, providerUserId, claims) = await FindUserFromExternalProviderAsync(result);
        //ommitted code
    }
    //ommitted code
    private async Task<(ApplicationUser user, string provider, string providerUserId, IEnumerable<Claim> claims)> 
        FindUserFromExternalProviderAsync(AuthenticateResult result)
    {
        var externalUser = result.Principal;

        // try to determine the unique id of the external user (issued by the provider)
        // the most common claim type for that are the sub claim and the NameIdentifier
        // depending on the external provider, some other claim type might be used
        var userIdClaim = externalUser.FindFirst(JwtClaimTypes.Subject) ??
                          externalUser.FindFirst(ClaimTypes.NameIdentifier) ??
                          throw new Exception("Unknown userid");

        // remove the user id claim so we don't include it as an extra claim if/when we provision the user
        var claims = externalUser.Claims.ToList();
        claims.Remove(userIdClaim);

        var provider = result.Properties.Items["scheme"];
        var providerUserId = userIdClaim.Value;

        // find external user
        var user = await _userManager.FindByLoginAsync(provider, providerUserId);

        return (user, provider, providerUserId, claims);
    }
    

Configure Azure Active Directory

This implies to go to portal.azure.com.

First you create a new application in the Azure Active Directory

Please retain the applicationId in order to put to the code

    options.ClientId = $"{clientId}";
    

Do not forget about checking the token on authentication

because of this code

    options.ResponseType = OpenIdConnectResponseType.IdToken; //"id_token";
//ommitted code
    options.TokenValidationParameters = new TokenValidationParameters
    {
        ValidateIssuer = false,
        //ValidAudience = "f59d5739-1ec9-46fc-961d-b01ef6fb3c51",

        NameClaimType = "name",
        RoleClaimType = "role"
    };

Roles for Windows


In Computer Management, I define the user : testUser that belongs to MyGroup

In startup.cs I define the Policy for this:

services.AddAuthorization(options =>
            {
                options.AddPolicy("MyGroupPolicy", policy => policy.RequireClaim("role",@"MyGroup"));
            });
    

I also define an Controller that require this policy

public class TestController : Controller
{
	[Authorize(Policy="MyGroupPolicy")]
	public IActionResult Index()
	{
		return Content(" you can see this because you are authorized to see MyGroupPolicy");
	}
}
    

So now if you logon on Windows with testuser, it will require the policy to be satisfied

Now, to understand the code

Read

https://docs.microsoft.com/en-us/aspnet/core/security/authorization/claims?view=aspnetcore-3.1#multiple-policy-evaluation

and

https://docs.microsoft.com/en-us/aspnet/core/security/authorization/policies?view=aspnetcore-3.1


Now some more code:

Transforming Groups to role claims

string pcName = Environment.MachineName;
// add the groups as claims -- be careful if the number of groups is too large
if (AccountOptions.IncludeWindowsGroups)
{
	var wi = wp.Identity as WindowsIdentity;
	var groups = wi.Groups.Translate(typeof(NTAccount));
	
	var roles = groups
		.Select(it=>it.Value)
		.Select(it=> it.StartsWith(pcName +"\\",StringComparison.InvariantCultureIgnoreCase)?
					it.Substring(pcName.Length+1): it)                        
		.Select(x => new Claim(JwtClaimTypes.Role, x));
	id.AddClaims(roles);
}

And putting back to user

[HttpGet]
public async Task<IActionResult> ExternalLoginCallback()
{
//omitted code
	// lookup our user and external provider info
	var (user, provider, providerUserId, claims) = await FindUserFromExternalProviderAsync(result);
//omitted code
    var principal = await _signInManager.CreateUserPrincipalAsync(user);

	foreach (var claim in claims)
	{
		additionalLocalClaims.AddRange(claims);
	}
//ommitted code
	await HttpContext.SignInAsync(user.Id, name, provider, localSignInProps, additionalLocalClaims.ToArray());


You can see the code at https://github.com/ignatandrei/identintegra and more documentation at https://ignatandrei.github.io/IdentIntegra/

[ADCES] 11 febr 2020 , Docker for Developers & How to debug in production a memory leak in .NETCORE3.1

Today,

Docker for Developers & How to debug in production a memory leak in .NETCORE3.1

 

The meeting is at 19:00 and can be found here:

Docker for Developers & How to debug in production a memory leak in .NETCORE3.1

Tuesday, Feb 11, 2020, 7:00 PM

EY Romania
Bulevardul Ion Mihalache 15-17 București, RO

40 Members Attending

Presentation 1: Title: Docker for Developers Presenter: Andrei Ignat, http://msprogrammer.serviciipeweb.ro/ Description: The old mode of downloading and installing software ( SqlServer, Mongo, Rabbit, even frameworks as Angular and .NET Core) will be soon of the past memory. If you want to join the new wave , see how Docker can easy your work as a …

Check out this Meetup →

Andrei Ignat weekly software news(mostly .NET)

* indicates required

Please select all the ways you would like to hear from me:

You can unsubscribe at any time by clicking the link in the footer of our emails. For information about our privacy practices, please visit our website.

We use Mailchimp as our marketing platform. By clicking below to subscribe, you acknowledge that your information will be transferred to Mailchimp for processing. Learn more about Mailchimp's privacy practices here.