Category: .NET Core

SideCarCLI- finish interceptors

SideCarCLI

SideCar for CLI applications. Interceptors for Line, Finish, Timer

Code at https://github.com/ignatandrei/SideCarCLI
NoName + Link 
1Description
2Specifications
3Refactor specifications and code
4Create Release
5Finish interceptors
6Send part of command to interceptors
7Line Interceptors
8Wait amount of time to finish

The idea for FinishInterceptors is very simple – when the initial  process, StartApp , is finished –then run some other programs . And here comes some decisions:

  1. Pass the ExitCode of the StartApp ? ( YES)
  2. Pass the output lines of text of the StartApp ? ( NO  – could be enormous )
  3. What to do if the StartApp has not finished in time ? ( Do nothing – we need the exit code)
  4. Wait for FinishInterceptors to finish ? If yes, how much time ? ( yes )

 

Same problems for TimerInterceptors and for LineInterceptors.

But, I think, the best way is to draw a diagram for that.

In the meantime, the project is already WIP

SideCarCLI–refactor command line

SideCarCLI

SideCar for CLI applications. Interceptors for Line, Finish, Timer

Code at https://github.com/ignatandrei/SideCarCLI
NoName + Link 
1Description
2Specifications
3Refactor specifications and code
4Create Release
5Finish interceptors
6Send part of command to interceptors
7Line Interceptors
8Wait amount of time to finish

When starting to implement the project, suddenly realize that list interceptors command does not belong to  start app command . So the application should be refactored to new specifications

– StartApp command – contains the whole list of commands for starting the app , including timer, adding interceptor

– Interceptors command – should list the interceptors  (timer, line, finish)

The interceptors should be defined in a separate file , like this

{
“TimerInterceptors”: [],
“LineInterceptors”: [
{
“Name”: “echo”,
“FolderToExecute”: null
}
],
“FinishInterceptors”: []
}

So this will be how it looks the command line

 

————–
command path:
————–
SideCar for any other application

Usage:  [command] [options]

Options:
-h|–help          Show help information
-max|–maxSeconds  max seconds for the StartApp to run

Commands:
about
interceptors
listAllCommands
StartAPP

Run ‘ [command] -h|–help’ for more information about a command.

The most simplest form it is:
SideCarCLI startApp –name YourApp

————–
command path:–>about
————–
Usage:  about [options]

Options:
-h|–help  Show help information

————–
command path:–>StartAPP
————–
start the CLI application that you need to intercept

Usage:  StartAPP [options]

Options:
-n|–name <fullPathToApplication>             Path to the StartApp
-a|–arguments <arguments_to_the_app>         StartApp arguments
-f|–folder[:<folder_where_execute_the_app>]  folder where to execute the StartApp – default folder of the StartApp
-aLi|–addLineInterceptor                     Add Line Interceptor to execute
-aTi|–addTimerInterceptor                    Add Timer Interceptor to execute
-aFi|–addFinishInterceptor                   Add Finish Interceptor to execute
-h|–help                                     Show help information

————–
command path:–>interceptors
————–
Usage:  interceptors [options]

Options:
-lLi|–ListLineInterceptor    List line interceptor
-lLi|–ListTimerInterceptor   List timer interceptor
-lFi|–ListFinishInterceptor  List timer interceptor
-h|–help                     Show help information

————–
command path:–>listAllCommands
————–
List all commands for the app

Usage:  listAllCommands [options]

Options:
-h|–help  Show help information

SideCarCLI–Architecture and specs–part 2

SideCarCLI

SideCar for CLI applications. Interceptors for Line, Finish, Timer

Code at https://github.com/ignatandrei/SideCarCLI
NoName + Link 
1Description
2Specifications
3Refactor specifications and code
4Create Release
5Finish interceptors
6Send part of command to interceptors
7Line Interceptors
8Wait amount of time to finish

The SideCarCLI application should start another application and intercept in various ways .  What will execute when intercept is not known in advance – so should be read at runtime  . So the SideCarCLI will load the interceptors and make then available to the application. Let’s say that , for the moment, the SideCarCLI will take all the configuration from the command line.

So let’s analyze the command line . The most simplest form it is:

SideCarCLI startApp  –name YourCLIApplication

For intercepting the output or the error of the console we do not know in advance what we will use. So we will make start dynamically what applications will  intercept the output lines provided by StartApp.

Also this is available for intercepting the result of the application and for the thing that is running periodically.

Also I want to provide a way for the users to make on-the-fly code to intercept – so I will make also a plugin architecture to provide interceptors on the fly.

Let’s see how the application command line will look  –

 

 

————–
command path:
————–
SideCar for any other application

Usage:  [command] [options]

Options:
-h|–help          Show help information
-max|–maxSeconds  max seconds for the StartApp to run

Commands:
_about
_listAllCommands
startApp

Run ‘ [command] -h|–help’ for more information about a command.

The most simplest form it is:
SideCarCLI startApp –name YourApp

————–
command path:–>_about
————–
Usage:  _about [options]

Options:
-h|–help  Show help information

————–
command path:–>startApp
————–
start the CLI application that you need to intercept

Usage:  startApp [command] [options]

Options:
-n|–name <fullPathToApplication>             Path to the StartApp
-a|–arguments <arguments_to_the_app>         StartApp arguments
-f|–folder[:<folder_where_execute_the_app>]  folder where to execute the StartApp – default folder of the StartApp
-h|–help                                     Show help information

Commands:
finishInterceptors
lineInterceptors
plugins
timer

Run ‘startApp [command] -h|–help’ for more information about a command.

————–
command path:–>startApp–>lineInterceptors
————–
Specify application for start when StartApp has a new line output

Usage:  startApp lineInterceptors [options]

Options:
-l|–list    List interceptors for lines
-a|–add     Add interceptor to execute
-f|–folder  folder where to start the interceptor
-h|–help    Show help information

————–
command path:–>startApp–>timer
————–
Specify timer to start an application at repeating interval

Usage:  startApp timer [options]

Options:
-i|–intervalRepeatSeconds  Repeat interval in seconds
-l|–list                   List interceptors to execute periodically
-a|–add                    Add interceptor to execute
-f|–folder                 folder where to start the interceptor
-h|–help                   Show help information

————–
command path:–>startApp–>finishInterceptors
————–
Specify interceptors for start when finish the app

Usage:  startApp finishInterceptors [options]

Options:
-l|–list    List interceptors for finish application
-a|–add     Add interceptor to execute
-f|–folder  folder where to start the interceptor
-h|–help    Show help information

————–
command path:–>startApp–>plugins
————–
Load dynamically plugins

Usage:  startApp plugins [options]

Options:
-f|–folder  folder with plugins
-l|–list    List plugins
-a|–add     Add interceptor to execute
-h|–help    Show help information

————–
command path:–>_listAllCommands
————–
List all commands for the app

Usage:  _listAllCommands [options]

Options:
-h|–help  Show help information

SideCarCLI | Command Line – description –part 1

SideCarCLI

SideCar for CLI applications. Interceptors for Line, Finish, Timer

Code at https://github.com/ignatandrei/SideCarCLI
NoName + Link 
1Description
2Specifications
3Refactor specifications and code
4Create Release
5Finish interceptors
6Send part of command to interceptors
7Line Interceptors
8Wait amount of time to finish

There are those days a great deal of command line applications that performs a variety of things. The problem is that you cannot control them . The command line performs their duty well – but , from time to time, you need something else integrated with this.

The pattern for this is https://docs.microsoft.com/en-us/azure/architecture/patterns/sidecar – it is applied to microservices , but why not applied to command lines ?

So – let’s start this project  with specifications.

Let’s suppose that I want to run an command line application – let name it StartApp. But I want also:

  1. Intercept every line of the StartApp that is outputting to the console and run something with that with the line ( send to HTTP endpoint, log somewhere, and so on).
  2. Intercept the finish of the StartApp and run something with the output ( either all lines, either with the result of the application)
  3. Timers:
    • Do not let the StartApp to run more than an maximum amount of time
    • Run something periodically  with arguments  from the StartApp arguments

 

 

The solution is to make another application ( let’s name SideCarCLI ) that runs the  StartApplication.

What it is complicated her is how we pass the arguments of the SideCarCLI versus the arguments of the StartApp .

Deploy .NET Core +SqlServer application to Ubuntu

Tools used:

SSH –  Windows Native

DOS – Windows Native

SSMS – download https://docs.microsoft.com/en-us/sql/ssms/download-sql-server-management-studio-ssms?view=sql-server-ver15

FileZilla – download https://filezilla-project.org/download.php?platform=win64

How I did :

SSH : Install SqlServer: https://docs.microsoft.com/en-us/sql/linux/quickstart-install-connect-ubuntu?view=sql-server-ver15

SSMS: Copy database with SSMS export data

DOS : Windows Compile application for Linux: dotnet publish -r linux-x64

Filezilla: Transfer with FileZilla all files: manual

SSH -Change permission chmod +777 <application name>  – to can execute

SSH –execute:  sudo ./<application name>–urls http://<ip>:8090

Note: Pay attention to Environment Variables on Linux – it did not work . See  https://docs.microsoft.com/en-us/dotnet/api/system.environment.getenvironmentvariables?view=netcore-2.0 . It did not work for me with Export.

Maybe it is better to configure as Linux Service, as in https://irina.codes/net-api-as-a-linux-service/

Also, for editing in ssh , sudo nano .service

Also, to start at startup, sudo systemctl enable

ASP.NET Core WebAPI should/ (must?) have

I am trying to have an extensive list about what an ASP.NET Core WebAPI project should / must have . After al, can be easy integrated into a VS project – or a wizard. I have put into 3 categories:

  1. Development – you need at development time.
  2. Testing – needed at testing time
  3. Production – needed in production

Note: All three are needed to be developed  !

  1. Development – Visibility  – see definition of the API
  2. Development  – Authorization + Authentication
  3. Development-Problem Details –  flow the
    errors as JSON text , keep status code
  4. Testing – test your application in CI mode
    • Production – Versioning  API endpoints
      • Production – CORS
      • Production – WhiteBox Monitoring
      • Production –BlackBox Monitoring and Observability – read about RED and USE
      • Production –Rate limit – do not allow insane usage
      • Production –Caching data – to return latest values
      • Production  – Status of the system

      Nice to have:

      1. Graph with endpoints: https://andrewlock.net/adding-an-endpoint-graph-to-your-aspnetcore-application/ – read about https://en.wikipedia.org/wiki/DOT_(graph_description_language) 
      2. Versioning  end points- see what version is your program: http://msprogrammer.serviciipeweb.ro/2019/02/18/identify-version-for-application-and-components-for-backend-net-core-and-frontendangularpart-2-backend/
      3. Not yet discovered, but should have endpoints for environment, settings json and others.
      4. Formatters: BSON, CSV, YAML: https://github.com/WebApiContrib/WebAPIContrib.Core

      ASP.NET Core : Add controllers at runtime and detecting changes done by others

      Part 1 Adding controllers at runtime

      Adding controllers at runtime in ASP.NET Core involves the ApplicationPartManager and IActionDescriptorChangeProvider. Let’s say that we hardcode the creation of the controller

       

      private Assembly CreateController(string name)
               {
                   
                   string code = new StringBuilder()
                       .AppendLine("using System;")
                       .AppendLine("using Microsoft.AspNetCore.Mvc;")
                       .AppendLine("namespace TestBlocklyHtml.Controllers")
                       .AppendLine("{")
                       .AppendLine("[Route(\"api/[controller]\")]")
                       .AppendLine("[ApiController]")
                       .AppendLine(string.Format("public class {0} : ControllerBase", name))
                   
                       .AppendLine(" {")
                       .AppendLine("  public string Get()")
                       .AppendLine("  {")
                       .AppendLine(string.Format("return \"test - {0}\";", name))
                       .AppendLine("  }")
                       .AppendLine(" }")
                       .AppendLine("}")
                       .ToString();
      
                  var codeString = SourceText.From(code);
                   var options = CSharpParseOptions.Default.WithLanguageVersion(LanguageVersion.CSharp7_3);
      
                  var parsedSyntaxTree = SyntaxFactory.ParseSyntaxTree(codeString, options);
      
                  var references = new MetadataReference[]
                   {
                       MetadataReference.CreateFromFile(typeof(object).Assembly.Location),
                       MetadataReference.CreateFromFile(typeof(Console).Assembly.Location),
                       MetadataReference.CreateFromFile(typeof(System.Runtime.AssemblyTargetedPatchBandAttribute).Assembly.Location),
                       MetadataReference.CreateFromFile(typeof(Microsoft.CSharp.RuntimeBinder.CSharpArgumentInfo).Assembly.Location),
                       MetadataReference.CreateFromFile(typeof(RouteAttribute).Assembly.Location),
                       MetadataReference.CreateFromFile(typeof(ApiControllerAttribute).Assembly.Location),
                       MetadataReference.CreateFromFile(typeof(ControllerBase).Assembly.Location),
                   };
      
                  var codeRun = CSharpCompilation.Create("Hello.dll",
                       new[] { parsedSyntaxTree },
                       references: references,
                       options: new CSharpCompilationOptions(OutputKind.DynamicallyLinkedLibrary,
                           optimizationLevel: OptimizationLevel.Release,
                           assemblyIdentityComparer: DesktopAssemblyIdentityComparer.Default));
                   using (var peStream = new MemoryStream())
                   {
                       if (!codeRun.Emit(peStream).Success)
                       {
                           return null;
                       }
                       return Assembly.Load(peStream.ToArray());
                   }
      
      
      
              }
      
      
      

       

      Then we will load into ApplicationParts

      [HttpGet]
              public string AddRuntimeController([FromServices] ApplicationPartManager partManager, [FromServices]MyActionDescriptorChangeProvider provider)
              {
                  string name = "andrei" + DateTime.Now.ToString("yyyyMMddHHmmss");
                  var ass = CreateController(name);
                  
                  if (ass != null)
                  {
                      partManager.ApplicationParts.Add(new AssemblyPart(ass));
                      // Notify change
                      provider.HasChanged = true;
                      provider.TokenSource.Cancel();
                      return "api/"+ name;
                  }
                  throw new Exception("controller not generated");
              }
      
      

      and the code for is

      public class MyActionDescriptorChangeProvider : IActionDescriptorChangeProvider
          {
              public static MyActionDescriptorChangeProvider Instance { get; } = new MyActionDescriptorChangeProvider();
      
              public CancellationTokenSource TokenSource { get; private set; }
      
              public bool HasChanged { get; set; }
      
              public IChangeToken GetChangeToken()
              {
                  TokenSource = new CancellationTokenSource();
                  return new CancellationChangeToken(TokenSource.Token);
              }
          }
      

      and it is added to the DI services by

      services.AddSingleton<IActionDescriptorChangeProvider>(MyActionDescriptorChangeProvider.Instance);
      services.AddSingleton(MyActionDescriptorChangeProvider.Instance);
                  
      

      Usually , you do not need Part 2 – if you do not construct something like Blockly for .NET Core (https://netcoreblockly.herokuapp.com/), that needs to see controllers added  by the application

      Part 2  Detecting controller added at runtime by others

      You usually do not need this – Part 1 is enough.  I need because I do construct something like Blockly for .NET Core (https://netcoreblockly.herokuapp.com/), that needs to see controllers added  by the application.

      So we need ActionDescriptorCollectionProvider –  read the remarks at https://docs.microsoft.com/en-us/dotnet/api/microsoft.aspnetcore.mvc.infrastructure.actiondescriptorcollectionprovider?view=aspnetcore-3.1 . We can obtain by DI from IActionDescriptorCollectionProvider and convert to ActionDescriptorCollectionProvider

      The code is

      //in the class , obtained by DI from IActionDescriptorCollectionProvider 
      private readonly ActionDescriptorCollectionProvider cp;
      
      internal void registerCallback()
              {
                  cp.GetChangeToken().RegisterChangeCallback(a =>
                  {
                       //a is your class because of this parameter below
      
                      //do your work
                      s.registerCallback();
                  }, this);
      
              }
      
      

      ( All code is taken from https://github.com/ignatandrei/netcoreblockly)

      Integrating SPA ( Angular, React, Vue) with .NET Core for production

      If you have a .NET Core WebAPI application ( that only serves data ) and  a SPA application ( Angular, Vue,  React) you can have just one application following this 2 steps:

      1.  In .NET Core add in Startup.cs ( with using Microsoft.AspNetCore.Builder from Microsoft.AspNetCore.StaticFiles )

      app.UseDefaultFiles();
      app.UseStaticFiles();

      app.UseRouting();

      app.UseAuthorization();

      app.UseEndpoints(endpoints =>
                   {
                      

                  // If you know the endpoints, put it here

                 //endpoints.MapFallbackToFile(“dbodepartment/{**slug}”,”/index.html”);
                               
                       endpoints.MapControllers();
                       endpoints.MapFallbackToFile(“{**slug}”, “/index.html”);
      });

      2. Compile the SPA to index.html, deploy in wwwroot with the js and css

      ( yes, you should modify the endpoint for data when compiling to / –  for Angular see my previous post , http://msprogrammer.serviciipeweb.ro/2020/09/14/angular-base-href-for-web-desktop-mobile-and-paths-for-services/ )

      HCV- Postmortem–part 8

      As I look below , the code is a small part of what has been done – documentation, test, CI /CD

      The src folder has this number of lines / words ( obtained with Powershell

      Get-ChildItem -r “src” |?{!$_.PSIsContainer} | Select-Object { Get-Content $_.FullName  | Measure-Object -Line -Word  | Select-Object  -Property Lines,Words }

       

      @{Lines=6; Words=6}
      @{Lines=20; Words=112}
      @{Lines=36; Words=154}
      @{Lines=8; Words=39}
      @{Lines=36; Words=131}
      @{Lines=27; Words=53}
      @{Lines=35; Words=135}
      @{Lines=33; Words=86}
      @{Lines=24; Words=84}
      @{Lines=9; Words=14}
      @{Lines=10; Words=16}
      @{Lines=18; Words=78}
      @{Lines=16; Words=31}
      @{Lines=24; Words=45}
      @{Lines=63; Words=143}
      @{Lines=11; Words=41}
      @{Lines=35; Words=86}
      @{Lines=37; Words=65}
      @{Lines=50; Words=99}
      @{Lines=15; Words=30}

      ( And does not count the yaml for CI )

      That counts for 513 lines and 1448 words…

      So the code is a small part od the application …

       

      HealthCheckVersion

      Health Check for Version
       NameLink
      1Idea and Githubhttp://msprogrammer.serviciipeweb.ro/2020/07/20/healthcheckversion-idea-part-1/
      2Documentationhttp://msprogrammer.serviciipeweb.ro/2020/07/21/healthcheckversiondocumentation-part-2/
      3Testshttp://msprogrammer.serviciipeweb.ro/2020/07/22/hcv-adding-testspart-3/
      4Codehttp://msprogrammer.serviciipeweb.ro/2020/07/23/hcv-adding-codepart-4/
      5Test Applicationhttp://msprogrammer.serviciipeweb.ro/2020/07/27/hcvadding-test-applicationpart-5/
      6CIhttp://msprogrammer.serviciipeweb.ro/2020/07/28/hcvautomatic-cipart-6/
      7NuGethttp://msprogrammer.serviciipeweb.ro/2020/07/29/hcvpreparing-for-nugetpart-7/
      8PostMortemhttp://msprogrammer.serviciipeweb.ro/2020/07/30/hcv-postmortempart-8/

      HCV–preparing for NuGet–part 7

      Preparing a dll to be deployed to NuGet mneans to me:

      1. XML documentation

      2. Treat Warnings as errors

      3. Adding sources to the nuget package

      4. Versioning the package, adding description, licence and so on

      5. Generating CD in artifacts

      6. Deploying to NuGet

      7. Adding Badge with Nuget and Licence to the readme

       

      Where are the modifications:

      1. In the .csproj – adding description
      2. In the docker yaml – running dotnet pack
      3. In the batch file that run the docker – collecting pkg folder with nuget package
      4. In the CI yaml file for Github – collecting nuget package to the artifacts
      5. In the readme  – using badge from shields.io

       

      The modifications are at https://github.com/ignatandrei/HealthCheckVersion/tree/DeployNuGet

      HealthCheckVersion

      Health Check for Version
       NameLink
      1Idea and Githubhttp://msprogrammer.serviciipeweb.ro/2020/07/20/healthcheckversion-idea-part-1/
      2Documentationhttp://msprogrammer.serviciipeweb.ro/2020/07/21/healthcheckversiondocumentation-part-2/
      3Testshttp://msprogrammer.serviciipeweb.ro/2020/07/22/hcv-adding-testspart-3/
      4Codehttp://msprogrammer.serviciipeweb.ro/2020/07/23/hcv-adding-codepart-4/
      5Test Applicationhttp://msprogrammer.serviciipeweb.ro/2020/07/27/hcvadding-test-applicationpart-5/
      6CIhttp://msprogrammer.serviciipeweb.ro/2020/07/28/hcvautomatic-cipart-6/
      7NuGethttp://msprogrammer.serviciipeweb.ro/2020/07/29/hcvpreparing-for-nugetpart-7/
      8PostMortemhttp://msprogrammer.serviciipeweb.ro/2020/07/30/hcv-postmortempart-8/

      Andrei Ignat weekly software news(mostly .NET)

      * indicates required

      Please select all the ways you would like to hear from me:

      You can unsubscribe at any time by clicking the link in the footer of our emails. For information about our privacy practices, please visit our website.

      We use Mailchimp as our marketing platform. By clicking below to subscribe, you acknowledge that your information will be transferred to Mailchimp for processing. Learn more about Mailchimp's privacy practices here.