RecordVisitors–CI –part 4

It will be perfect for any project to have a CI to improve feedback. GitHub actions is helpful here  – just put a gitlab.yml.

After few iterations, mine looks like this

name: BuildAndTest

on:

push:

branches: [ main ]

  pull_request:

    branches: [ main ]

jobs:

  build:

    #runs-on: ubuntu-latest

    runs-on: windows-2019

    steps:

    – uses: actions/checkout@v2

    – name: Setup .NET

      uses: actions/setup-dotnet@v1

      with:

        dotnet-version: 5.0.x

    – name: Restore dependencies

      run: |

        cd src/RecordVisitors

        dotnet restore

    – name: Build

      run: |

        cd src/RecordVisitors

        dotnet build –no-restore

    – name: Test

      run: |

        cd src/RecordVisitors

        dotnet test –no-build –verbosity normal

    – name: code coverage

      run: |

        cd src/RecordVisitors

        dotnet tool restore

        dotnet coverlet AutomatedTestRecord\bin\Debug\net5.0\AutomatedTestRecord.dll –target “dotnet” –targetargs “test RecordVisitors.sln –no-build”  –format opencover –exclude “[SampleWeb*]*”  –exclude “[xunit*]*” –verbosity detailed

        #dotnet coverlet AutomatedTestRecord\bin\Debug\net5.0\AutomatedTestRecord.dll –target “dotnet” –targetargs “test RecordVisitors.sln”  –format opencover  –verbosity detailed

        dotnet reportgenerator “-reports:coverage.opencover.xml” “-targetdir:coveragereport” “-reporttypes:HTMLInline;HTMLSummary;Badges”

    – name: ‘Upload Artifact’

      uses: actions/upload-artifact@v2

      with:

        name: codeCoverage

        path: src/RecordVisitors/coveragereport

        retention-days: 1

    – name: verify code coverage

      run: |

        ls src/RecordVisitors/coverage.opencover.xml

    – uses: codecov/codecov-action@v1

      with:

        files: src/RecordVisitors/coverage.opencover.xml

        fail_ci_if_error: true # optional (default = false)

        #verbose: true # optional (default = false)

    – name: Pack

      run: |

        cd src/RecordVisitors/RecordVisitors

        dotnet pack –include-source –include-symbols

Looks pretty much self-explanatory to me. Build, test, run code coverage and uploads to https://app.codecov.io/gh/ignatandrei/RecordVisitors  ( I am proud : CC 94 % )

RecordVisitors–code coverage–part 3

Now I have to do some code coverage . The easy way is to test the web application – doing an integration test. Use this as a starting point: https://docs.microsoft.com/en-us/aspnet/core/test/integration-tests?view=aspnetcore-5.0

I have 2 kind of tests: HappyPath  -when all is working ok – and  TestErrors  – when it is not.

The code for all working ok is

public async void TestFakeUser()
{
    // Arrange
    var client = _factory.CreateClient();

    // Act
    var response = await client.GetStringAsync("/recordVisitors/AllVisitors5Min");

    // Assert
    var str = "Jean Irvine";
    Assert.True(response.Contains(str),$"{response} must contain {str}");
                
}

and for testing errors

[Fact]
public  void TestFakeUser()
{
    _factory.RemoveServices = true;
    _factory.RemoveFakeUser = false;
    var ex = Record.Exception(()=>_factory.CreateClient());

    Assert.IsType<ArgumentException>(ex);
            
}
[Fact]
public async void TestNoUser()
{
    _factory.RemoveServices = false;

    _factory.RemoveFakeUser= true;
    // Arrange
    var client = _factory.CreateClient();

    // Act
    var response = await client.GetStringAsync("/recordVisitors/AllVisitors5Min");

    // Assert
    var str = "Jean Irvine";
    Assert.True(response.Contains(str), $"{response} must contain {str}");

}

 

The code coverage has reached 94% after 3 iterations – it is ok. You can see the results at https://app.codecov.io/gh/ignatandrei/RecordVisitors

 

Please make sure that you read also https://andrewlock.net/converting-integration-tests-to-net-core-3/

RecordVisitors–code–part 2

Now , after the idea is complete, let’s see the code. ( Yes, I know that TDD will be better … )

For the implementation :

1. Problems with the manual test-  how to create a Fake User – see below the solution.

2. Problems of how to let user define his own data

You can find the original code at https://github.com/ignatandrei/RecordVisitors/releases/tag/just-Code

I do not want to bother with details about implementation – so some statistics ( obtained from VS with Analyze => Calculate Code Metrics)

Those are the classes  / Interfaces:

  1. Extensions – used to easy registration in Startup
  2. IRecordVisitorFunctions – to let the user define his function to get the name of the user
  3. IUsersRepository – to let the user define his function to save / retrieve from database
  4. RecordVisitorFunctions – implementation
  5. RecordVisitorsMiddleware
  6. UserRecorded – what to save in the default implementation
  7. UserRecordVisitors –  the default DBContext to save
  8. UsersRepository – default database implementation with EF in memory

Created also an ASP.NET Core WebAPI project to test . The following was added:

1. A MockAuthenticatedUser (  credits to https://visualstudiomagazine.com/Blogs/Tool-Tracker/2019/11/mocking-authenticated-users.aspx )

2.  To the startup:

services.AddAuthentication(“BasicAuthentication”)
     .AddScheme<AuthenticationSchemeOptions,
                   MockAuthenticatedUser>(“BasicAuthentication”, null);

app.UseAuthentication();
//put AFTER authentication
app.UseRecordVisitors();

app.UseEndpoints(endpoints =>
{
     endpoints.MapControllers();
     endpoints.UseVisitors();
});

The number of lines are aproximatively 200 ( the total number of lines written by VS is 1000 )

RecordVisitors-idea–part1

It will be interesting to have a middleware of .NET Core that can record the name of the visitors in order to see what are the latest visitors of your site .

What will be the uses ?

  1. See when the people has visiting your site
  2. See the latest visitors
  3. If you have a site for mobile and a site for web , prevent multiple logins
  4. Fun 😉

What characteristics should have ?

  1. Record visitors
  2. Make an endpoint – recordVisitors/AllVisitors5Min to see the latest visitors as JSON
  3. Let the programmer choose how to obtain  the name of the visitor  ( via Forms, , AD Identity, JWT,  Azure , OAUTH…)
    1. Choose a default to work fast
  4. Let the programmer choose his variant of persisting ( SqlServer, Mongo , Sqlite, CSV… ) where to be recorded
    1. Choose a default  to work fast
    2. Make as an event to intercept and  add additional details

Now let’s see what is for the programming!

Friday Links 426

  • How to Balance Hard Work and Pleasure for Happiness – The Atlantic
  • sourcegen.dev – Source Generator Playground – @davidwengier
  • Calling Bullshit – Case Studies
  • How to Be Productive, Feel Less Overwhelmed, and Get Things Done
  • Use YARP to host client and API server on a single origin to avoid CORS
  • What are Mixins, and how do you use them in TypeScript? | by Olusola Samuel | Jan, 2021 | JavaScript in Plain English
  • Using C# Source Generators to create an external DSL | .NET Blog
  • <��� The Top 100 Developer Tools of 2020 | StackShare
  • Introducing Dispatch. Netflix is pleased to announce the& | by Netflix Technology Blog | Netflix TechBlog
  • Humble Object at XUnitPatterns.com
  • 10 bad TypeScript habits to break this year
  • Stairway to Exploring Database Metadata SQLServerCentral
  • Making GitHub s new homepage fast and performant – The GitHub Blog
  • Using source generators to find all routable components in a Blazor WebAssembly app
  • Debugging a native deadlock in a .NET Linux application | by Kevin Gosse | Feb, 2021 | Medium
  • When DRY Doesn t Work, Go WET. It s okay if you repeat yourself | by Nick Bull | Better Programming
  • Clean code through Reactive programming in Angular with RxJS
  • ironcev/awesome-roslyn: Curated list of awesome Roslyn books, tutorials, open-source projects, analyzers, code fixes, refactorings, and source generators
  • amis92/csharp-source-generators: A list of C# Source Generators (not necessarily awesome) and associated resources: articles, talks, demos.
  • atifaziz/Transplator: Simple C# source generator for text templates
  • Dotnet diagnostics / open telemetry on CI in 6 simple steps

    Prerequisites : Having Jaeger / Zipkin for visualization . The easy way for installing Jaeger is to have Jaeger all in one ( either via Docker, either run on PC)

     

    Step 1: Modify the csproj to add this

    <ItemGroup>
      <PackageReference Include="OpenTelemetry" Version="1.1.0-beta1" />
        <PackageReference Include="OpenTelemetry.Instrumentation.AspNetCore" Version="1.0.0-rc3" />
        <PackageReference Include="OpenTelemetry.Exporter.Jaeger" Version="1.1.0-beta1" />
        <PackageReference Include="OpenTelemetry.Extensions.Hosting" Version="1.0.0-rc3" />
        <PackageReference Include="OpenTelemetry.Instrumentation.Http" Version="1.0.0-rc3" />
        <PackageReference Include="OpenTelemetry.Instrumentation.SqlClient" Version="1.0.0-rc3" />
      
        <AdditionalFiles Include="../AutoMethod.txt" />
        <PackageReference Include="AOPMethodsCommon" Version="2021.5.17.2230" />
        <PackageReference Include="AOPMethodsGenerator" Version="2021.5.17.2230" />
    
      </ItemGroup>
    
    
    

    Optional : To see the RSCG files, add also

      <PropertyGroup>
        <EmitCompilerGeneratedFiles>true</EmitCompilerGeneratedFiles>
        <CompilerGeneratedFilesOutputPath>$(BaseIntermediateOutputPath)Generated</CompilerGeneratedFilesOutputPath>
      </PropertyGroup>
    

    Step 2:

    2.1  If  ASP.NET Core  app,  add the following to Startup.cs

    services.AddOpenTelemetryTracing(b =>
    {
        string nameAssemblyEntry = Assembly.GetEntryAssembly().GetName().Name;
        var rb = ResourceBuilder.CreateDefault();
        rb.AddService(nameAssemblyEntry);
        var data = new Dictionary<string, object>() {
                    { "PC", Environment.MachineName } };
        data.Add("Exe", nameAssemblyEntry);
        rb.AddAttributes(data);
        _ = b
            .AddAspNetCoreInstrumentation(opt=>
            {
                opt.RecordException = true;
                            
            })
            .AddHttpClientInstrumentation(opt=>
            {
                opt.SetHttpFlavor = true;                        
            })
            .AddSqlClientInstrumentation(opt=>
            {
                opt.SetDbStatementForStoredProcedure = true;
                opt.SetDbStatementForText = true;
                opt.RecordException = true;
                opt.EnableConnectionLevelAttributes = true;
            })
            .AddSource("MySource")
            .SetResourceBuilder(rb)
            .AddJaegerExporter(c =>
            {
                //var s = Configuration.GetSection("Jaeger");
    
                //s.Bind(c);
                c.AgentHost = "localhost";//TODO: put the name of the PC with Jaeger
                            
            });
    
    
    })
        ;
    
    

    2.2 If .NET Core executable , add the following code to main.cs

    
    string nameAssemblyEntry = Assembly.GetEntryAssembly().GetName().Name;
    var rb = ResourceBuilder.CreateDefault();
    rb.AddService(nameAssemblyEntry);
    var data = new Dictionary<string, object>() {
    { "PC", Environment.MachineName } };
    data.Add("Exe", nameAssemblyEntry);
    rb.AddAttributes(data);
    
    openTelemetry = Sdk.CreateTracerProviderBuilder()
    .AddSource("MySource")
    .AddHttpClientInstrumentation()
    .AddSqlClientInstrumentation()
    .SetResourceBuilder(rb)
    .AddJaegerExporter(o =>
     {
       //var s = Configuration.GetSection("Jaeger");
    
       //s.Bind(c);
       c.AgentHost = "localhost";//TODO: put the name of the PC with Jaeger
     })
    .Build();
    
    

     

    Step 3: Create near the .sln the file AutoMetods.ps1 with the following contents

    function ReplaceWithAuto([string]$m )
    {
    $m=$m.Trim();
    $firstpart= $method
    $secondPart =""
    #Write-Host $method
    # do it fast now. Find first ( , name method is before that
    #( could be done better: find from the last : add 1when found ( , add -1 when found ), stop at 0 )
    # find first space before
    $nr = ([regex]::Matches($m, [regex]::Escape(")") )).count
    if($nr -gt 1){

    $characters = [char[]]$method;
    $nrOpen=0;
    $iterator= $characters.Length;
    do{
    $iterator--;
    if($characters[$iterator] -eq '('){
    $nrOpen++;
    }

    if($characters[$iterator] -eq ')'){
    $nrOpen--;
    }

    }while(($iterator -gt 0) -and ($nrOpen -ne 0))
    $firstpart = $m.Substring(0,$iterator).Trim()
    #return $firstpart
    $spaceArr= $firstpart.Split(' ');
    $firstpart =$spaceArr[$spaceArr.Length-1]
    #return $firstpart

    }
    $arr = $firstpart.Split("(");
    #Write-Host $arr[0]
    $splitSpace = $arr[0].Split(" ")
    $nameMethod = $splitSpace[$splitSpace.Length-1]
    #Write-Host $nameMethod
    [regex]$pattern = $nameMethod
    #$method= ($method -replace [regex]::Escape($nameMethod) ,("auto"+$nameMethod))
    $method= $pattern.Replace($method,("auto"+$nameMethod),1)
    #Write-Host $method
    return $method

    }

    function VerifyFolder()
    {
    Write-Host "starting"
    $fileName = Get-ChildItem "*.cs" -Recurse
    #$varItem = [regex]::Escape( "AOPMarkerMethod")
    $varItem = "AOPMarkerMethod"
    $filename | %{

    $fileContent = gc $_
    $x= ($fileContent -imatch "AOPMarkerMethod" )
    #Write-Host $x.Length
    if($x.Length -gt 0){
    Write-Host $_.fullname
    $LineNumbers =@( Select-String $_ -Pattern $varItem| Select-Object -ExpandProperty 'LineNumber')
    Write-Host "found " $LineNumbers
    Foreach ($LineNumber in $LineNumbers){
    $method = $fileContent[$LineNumber]

    Write-Host "Line where string is found: "$LineNumber
    Write-Host $method

    #Write-Host ReplaceWithAuto $method
    $fileContent[$LineNumber] = ReplaceWithAuto($method)

    Write-Host $fileContent[$LineNumber]
    }

    $fileContent |Set-Content $_.fullname

    }
    #(gc $_) -replace $varItem,"auto" |Set-Content $_.fullname

    }
    }

    VerifyFolder

    Step 4 : Create near the .sln Automethod.txt with the following content:

    //------------------------------------------------------------------------------
    // <auto-generated>
    // This code was generated by a tool.
    //
    // Changes to this file may cause incorrect behavior and will be lost if
    // the code is regenerated.
    // </auto-generated>
    //------------------------------------------------------------------------------
    using System;
    using System.Collections.Generic;
    using System.CodeDom.Compiler;
    using System.Runtime.CompilerServices;
    using System.Diagnostics;
    using System.Linq;
    using OpenTelemetry.Trace;
    namespace {{NamespaceName}} {

    [GeneratedCode("AOPMethods", "{{version}}")]
    [CompilerGenerated]
    public partial class {{ClassName}}{

    public static readonly ActivitySource MyActivitySource = new ActivitySource("MySource");

    {{~ for mi in Methods ~}}
    //{{mi.Name}}
    {{

    strAwait = ""
    strAsync =""
    if mi.CouldUseAsync == true
    strAwait = " await "
    strAsync = " async "
    end
    separator = ""
    if(mi.NrParameters > 0)
    separator = ","
    end
    returnString = ""
    if mi.CouldReturnVoidFromAsync == false
    returnString = " return "
    end
    }}
    public {{strAsync}} {{mi.ReturnType}} {{mi.NewName}} ({{mi.parametersDefinitionCSharp }}
    {{separator}}
    [CallerMemberName] string memberName = "",
    [CallerFilePath] string sourceFilePath = "",
    [CallerLineNumber] int sourceLineNumber = 0){
    var sw=Stopwatch.StartNew();
    using var span = MyActivitySource.StartActivity("{{mi.NewName}}", ActivityKind.Producer);

    span.Start();

    //parameters

    {{~ for miArg in mi.Parameters ~}}
    var val{{ miArg.Key }} = System.Text.Json.JsonSerializer.Serialize({{ miArg.Key }});
    span.SetTag("Argument_{{ miArg.Key }}",val{{ miArg.Key }});
    {{ end}}

    span.SetTag("method","{{mi.NewName}}");
    span.SetTag("called from ",memberName);
    span.SetTag("called from file",sourceFilePath);
    span.SetTag("called from line",sourceLineNumber);
    var utcTime =System.DateTime.UtcNow;
    span.SetTag("UTCDate",utcTime.ToString("yyyyMMddHHmmss"));
    var userEmail = ((System.Security.Claims.ClaimsPrincipal)System.Threading.Thread.CurrentPrincipal)?.Claims?.Where(c => c.Type == "email")?.SingleOrDefault()?.Value;
    userEmail ??= "NoUserConfigured";
    span.SetTag("user",userEmail);
    span.SetStatus(Status.Ok);
    try{
    Console.WriteLine("--{{mi.Name}} start ");
    {{returnString}} {{ strAwait }} {{mi.Name}}({{ mi.parametersCallCSharp }});
    }
    catch(Exception ex){
    Console.WriteLine("--{{mi.Name}} exception ");
    span?.RecordException(ex);
    span.SetStatus(Status.Error);
    throw;
    }
    finally{
    Console.WriteLine("--{{mi.Name}} finally ");
    span.SetTag("ElapsedMilliseconds",sw.Elapsed.TotalMilliseconds);
    span.Stop();
    //span.Duration= sw.Elapsed.TotalMilliseconds;

    }
    }//end {{mi.NewName}}

    {{ end}}
    }
    }

    Step 5: Decorate your method with [AOPMarkerMethod]  ( namespace AOPMethodsCommon )

    Something like this

    
    [AOPMarkerMethod]
    private NewCountry[] ArrangeCountriesSearch(NewCountry[] data)
    

    And in the class definition
    [AutoMethods(template = TemplateMethod.CustomTemplateFile,CustomTemplateFileName ="Automethod.txt", MethodSuffix = "bup")]

    Step 6: Modify your CI to run

    pwsh AutoMetods.ps1

    or

    powershell AutoMetods.ps1

     

     

    Enjoy, that will be all !

    Friday Links 425

  • Main demo | RevoGrid – Excel like data grid component
  • cloudcommunity/Free-Certifications: Curated list of free courses & certifications
  • Multi-Tenant Architecture. Software Architect s library | by Alex Gurbych | Feb, 2021 | Level Up Coding
  • A simple template for Onion Architecture with .NET 5 | by David Pereira | Feb, 2021 | Medium
  • Microservices Architecture. Software Architect s library | by Alex Gurbych | Jan, 2021 | Level Up Coding
  • Sprinkle a little ancient philosophy into your daily routines | Psyche Ideas
  • WePresent | Juggling a screen-based job with a hands-on hobby
  • How to display dates in your user’s time zone with the Intl API | Phil Nash
  • Free eBook: How to use Dapr for .NET Developers – Scott Hanselman’s Blog
  • Free Windows 10 development virtual machines for HyperV, Parallels, VirtualBox, and VMWare – Scott Hanselman’s Blog
  • Will you pay the consistency costs? – Ayende @ Rahien
  • Global distributed consistency, the easy way – Ayende @ Rahien
  • C# Coding Standards Updated | Jesse Liberty
  • What I wish I had known about single page applications – Stack Overflow Blog
  • Carbon | Create and share beautiful images of your source code
  • Create beautiful images of your code
  • Getting started with Dapr for .NET Developers – Laurent Kemp�
  • 10 Design Patterns every Software Architect and Software Engineer must know | by Ravindra Elicherla | Medium
  • 14 Useful Tools That I Use for Faster and Easier Web Development | by Josef Cruz | Mar, 2021 | JavaScript in Plain English
  • 11 Front End Developer Tools I Can t Live Without | by Josef Cruz | Feb, 2021 | JavaScript in Plain English
  • Open Tracing instrumentation for running process

    Open tracing allows you to trace calls between (micro)services . It has also calls for HTTP and Sql. For a ASP.NET Core application the code is at follows ( for exporting at Jaeger, for example):

    
    services.AddOpenTelemetryTracing(b =&gt;
    {
    string nameAssemblyEntry = Assembly.GetEntryAssembly().GetName().Name;
    var rb = ResourceBuilder.CreateDefault();
    rb.AddService(nameAssemblyEntry);
    var data = new Dictionary&lt;string, object&gt;() {
    { "PC", Environment.MachineName } };
    data.Add("Exe", nameAssemblyEntry);
    rb.AddAttributes(data);
    _ = b
    .AddAspNetCoreInstrumentation()
    .AddHttpClientInstrumentation()
    .AddSqlClientInstrumentation()
    .AddSource("MySource")
    .SetResourceBuilder(rb)
    .AddJaegerExporter(c =&gt;
    {
    var s = Configuration.GetSection("Jaeger");
    
    s.Bind(c);
    
    });
    
    
    })
    ;
    
    

    This code was made with

     <PackageReference Include=”OpenTelemetry” Version=”1.1.0-beta1″ />
    <PackageReference Include=”OpenTelemetry.Instrumentation.AspNetCore” Version=”1.0.0-rc3″ />
    <PackageReference Include=”OpenTelemetry.Exporter.Jaeger” Version=”1.1.0-beta1″ />
    <PackageReference Include=”OpenTelemetry.Extensions.Hosting” Version=”1.0.0-rc3″ />
    <PackageReference Include=”OpenTelemetry.Instrumentation.Http” Version=”1.0.0-rc3″ />
    <PackageReference Include=”OpenTelemetry.Instrumentation.SqlClient” Version=”1.0.0-rc3″ />

    For a .NET Core console, the code is :

    string nameAssemblyEntry = Assembly.GetEntryAssembly().GetName().Name;
    var rb = ResourceBuilder.CreateDefault();
    rb.AddService(nameAssemblyEntry);
    var data = new Dictionary<string, object>() {
    { "PC", Environment.MachineName } };
    data.Add("Exe", nameAssemblyEntry);
    rb.AddAttributes(data);
    
    openTelemetry = Sdk.CreateTracerProviderBuilder()
    .AddSource("MySource")
    .AddHttpClientInstrumentation()
    .AddSqlClientInstrumentation()
    .SetResourceBuilder(rb)
    .AddJaegerExporter(o =>
     {
     var s = config.GetSection("Jaeger");
     s.Bind(o);
                       })
    .Build();
    
    

    But how to transmit data for a Web that calls a process and not modify the command line ? Simple : via environment variable

    The code for running the process is :

    var pi = new ProcessStartInfo();
    pi.FileName = ...;
    pi.WorkingDirectory = ...;
    
    var act = Activity.Current;
    if (act != null)
    {
     pi.EnvironmentVariables.Add(nameof(act.TraceId), act.TraceId.ToHexString());
     pi.EnvironmentVariables.Add(nameof(act.SpanId), act.SpanId.ToHexString());
    }
    var p = Process.Start(pi);
    

    For the console, the code is:

     using (var act = MyActivitySource.StartActivity("StartProcess", ActivityKind.Producer))
                {
                    act.SetTag("proc", "main");
                   
                    var traceStr = (Environment.GetEnvironmentVariable(nameof(act.TraceId)));
                    if (!string.IsNullOrWhiteSpace(traceStr))
                    {
                        var trace = ActivityTraceId.CreateFromString(traceStr);
                        var span = ActivitySpanId.CreateFromString(Environment.GetEnvironmentVariable(nameof(act.SpanId)));
                        act.SetParentId(trace, span, ActivityTraceFlags.Recorded);
                    }
    
                    try
                    {
                        //executing code
                        act.SetStatus(Status.Ok);
                    }
                    catch (Exception ex)
                    {
                        act.SetStatus(Status.Error);
                        throw;
                    }
                    
                }  
    

    Andrei Ignat weekly software news(mostly .NET)

    * indicates required

    Please select all the ways you would like to hear from me:

    You can unsubscribe at any time by clicking the link in the footer of our emails. For information about our privacy practices, please visit our website.

    We use Mailchimp as our marketing platform. By clicking below to subscribe, you acknowledge that your information will be transferred to Mailchimp for processing. Learn more about Mailchimp's privacy practices here.