Category: sidecar

SideCarCLI- console to dot net tool

Summary links SideCarCLI

NoName + Link 
1Description
2Specifications
3Refactor specifications and code
4Create Release
5Finish interceptors
6Send part of command to interceptors
7Line Interceptors
8Finish process after some time
9Documetation Diagram
10Technical Summary
11Create dotnet tool
( Description : SideCar for CLI applications. Interceptors for Line, Finish, Timer . See Code )

To create a .NET Tool, please read first https://docs.microsoft.com/en-us/dotnet/core/tools/global-tools-how-to-create 

Of course , all the  nuget additional formats for csproj apply: read https://docs.microsoft.com/en-us/dotnet/core/tools/csproj . Of course, the easy way to in Visual Studio to right click the project, properties, Package.

This is the final xml from .csproj:

<PropertyGroup>
     <OutputType>Exe</OutputType>
     <TargetFramework>netcoreapp3.1</TargetFramework>
     <RuntimeIdentifiers>win-x64;linux-x64</RuntimeIdentifiers>
     <PackAsTool>true</PackAsTool>
     <ToolCommandName>sidecarcli</ToolCommandName>
     <Title>SideCarCLI</Title>
    
     <PackageOutputPath>./nupkg</PackageOutputPath>
     <PackageId>sidecarcli</PackageId>
     <Version>2020.111.104</Version>
     <Authors>Andrei Ignat</Authors>
     <Company>AOM</Company>
     <Product>sidecarcli</Product>
     <Description>A SideCar for Command Line Applications. Read http://msprogrammer.serviciipeweb.ro/category/sidecar/ . Code source at https://github.com/ignatandrei/sidecarcli</Description>
     <PackageLicenseExpression>MIT</PackageLicenseExpression>
     <PackageProjectUrl>http://msprogrammer.serviciipeweb.ro/category/sidecar/</PackageProjectUrl>
     <RepositoryUrl>https://github.com/ignatandrei/SideCarCLI/</RepositoryUrl>
     <RepositoryType>GIT</RepositoryType>
     <PackageTags>SideCarCLI;Side Car; Command Line</PackageTags>
     <PackageReleaseNotes>First version. Read http://msprogrammer.serviciipeweb.ro/category/sidecar/</PackageReleaseNotes>
   </PropertyGroup>

 

You can create the project with

dotnet pack

command

Now go to https://www.nuget.org/packages/manage/upload and upload the project.

You can find the final result at https://www.nuget.org/packages/sidecarcli/  and you can install it

And this concludes the work for https://github.com/ignatandrei/SideCarCLI/issues/10 .

SideCarCLI–looking at past and at future

Summary links SideCarCLI

NoName + Link 
1Description
2Specifications
3Refactor specifications and code
4Create Release
5Finish interceptors
6Send part of command to interceptors
7Line Interceptors
8Finish process after some time
9Documetation Diagram
10Technical Summary
11Create dotnet tool
( Description : SideCar for CLI applications. Interceptors for Line, Finish, Timer . See Code )

It was interesting to develop the project for command line same as for cloud design pattern, https://docs.microsoft.com/en-us/azure/architecture/patterns/sidecar .

The problem were more about

  1. architecture
  2. organizing features to be easy understandable
  3. , testing the application
  4. writing about

 

rather than  technical, about how to make the application.

Other , it was a pretty simple project , that can be useful in  CI / CD operations.

It remains :

Make a dot net tool –  https://github.com/ignatandrei/SideCarCLI/issues/10

Compiling as Windows and Linux https://github.com/ignatandrei/SideCarCLI/issues/11

Make documentation with use case – https://github.com/ignatandrei/SideCarCLI/issues/12

Make Tests  https://github.com/ignatandrei/SideCarCLI/issues/13

SideCarCLI- diagram for how it is working

Summary links SideCarCLI

NoName + Link 
1Description
2Specifications
3Refactor specifications and code
4Create Release
5Finish interceptors
6Send part of command to interceptors
7Line Interceptors
8Finish process after some time
9Documetation Diagram
10Technical Summary
11Create dotnet tool
( Description : SideCar for CLI applications. Interceptors for Line, Finish, Timer . See Code )

Usually , a picture is better than words . So , I can explain how SideCarCLI works – or I can draw an image.

The easiest way to draw an image is with  https://mermaidjs.github.io/ –  that has also a live editor at https://mermaid-js.github.io/mermaid-live-editor .

The code for generating the picture is

sequenceDiagram

participant S as SideCar

participant R as Real Application

participant L as Line Interceptor(s)

participant F as Finish Interceptor(s)

participant T as Timer Interceptor(s)

S->>R: Start Application

loop for each timer in json

S->>T: Start Timer application

end

loop for each line output from real application

R->>S: Send Output  Line to SideCae\r

S->>+L: Start Line Interceptor with line output in command line

L->>-S: execute and returns

end

R->>S: End Application

S->>+F: Send line output

F->>-S: execute and returns

and you can see here the picture

SideCARCLI–Finish process after some time

Summary links SideCarCLI

NoName + Link 
1Description
2Specifications
3Refactor specifications and code
4Create Release
5Finish interceptors
6Send part of command to interceptors
7Line Interceptors
8Finish process after some time
9Documetation Diagram
10Technical Summary
11Create dotnet tool
( Description : SideCar for CLI applications. Interceptors for Line, Finish, Timer . See Code )

One of the feature is to let the original process execute just a limited amount of time. For this, I have defined an option

var maxSeconds = app.Option(“-mS|–maxSeconds”, “max seconds for the StartApp to run”, CommandOptionType.SingleOrNoValue);

and the code for waiting is

Process p = new Process()
{
StartInfo = pi
};

//code

var res=p.WaitForExit(this.MaxSeconds);

//code

if (res)
{
exitCode = p.ExitCode;
RunFinishInterceptors(exitCode);
}
else
{
Console.WriteLine($”timeout {MaxSeconds} elapsed”);
exitCode = int.MinValue;
}

 

where the default value for this.MaxSeconds is –1 – wait undefinetely

SideCarCLI – Line interceptors

Summary links SideCarCLI

NoName + Link 
1Description
2Specifications
3Refactor specifications and code
4Create Release
5Finish interceptors
6Send part of command to interceptors
7Line Interceptors
8Finish process after some time
9Documetation Diagram
10Technical Summary
11Create dotnet tool
( Description : SideCar for CLI applications. Interceptors for Line, Finish, Timer . See Code )

For the SideCarCLI I have the concept of Line Interceptors. That means for each line from the original program another program will be start .

The json looks like this

“LineInterceptors”: [
{
“Name”: “WindowsStandardWindowsOutputInterceptor”,
“Arguments”: “/c echo \”{site} {line}\””,
“FullPath”: “cmd.exe”,
“FolderToExecute”: null,
“InterceptOutput”: true
},
{
“Name”: “NextInterceptor”,
“Arguments”: “/c echo {line}”,
“FullPath”: “cmd.exe”,
“FolderToExecute”: null,
“InterceptOutput”: true
}
]

and you can activate by command line

-aLi WindowsStandardWindowsOutputInterceptor

Pretty simple , right ? For the code , means intercepting each line

p.OutputDataReceived += P_OutputDataReceived;
p.ErrorDataReceived += P_ErrorDataReceived;

and then run the process

foreach(var item in runningInterceptors.LineInterceptors)
{
try
{

var local = item;
var dataToBeParsed=new Dictionary<string, string>();

if(this.argsRegex.Count>0)
{
foreach (var reg in argsRegex)
{
dataToBeParsed.Add(“{“+reg.Key+”}”, reg.Value);
}
}
dataToBeParsed[“{line}”]= e.Data;
allProcesses.TryAdd(local.Name, local.RunLineInterceptor(local.Name,dataToBeParsed));

}
catch (Exception ex)
{
Console.WriteLine($”LineInterceptor:{item?.Name} error:!!!” + ex.Message);
}
}

Every time it is interesting how a simple specification means multiple lines of code….

SideCarCLI-send arguments to interceptors

Summary links SideCarCLI

NoName + Link 
1Description
2Specifications
3Refactor specifications and code
4Create Release
5Finish interceptors
6Send part of command to interceptors
7Line Interceptors
8Finish process after some time
9Documetation Diagram
10Technical Summary
11Create dotnet tool
( Description : SideCar for CLI applications. Interceptors for Line, Finish, Timer . See Code )

For the SideCarCLI I want the interceptors ( line, finish , timer) to receive parts of the command line of the original application. But how to parse the command line and how to pass to the interceptors ?  RegEx to the rescue!

I will let the user define the command line of the starting application and parse the command line with his own regex.

For example , if he put as arguments

x=y z=t

a possible regex  will be

(?<FirstArg>\w+)=((\w+)) (?<LastArg>\w+)=((\w+))

Also, if the starting application is ping.exe and the command line is

-n 20 \”www.yahoo.com\”

then a possible regex to have the site is

(?<FirstArg>.+) (?<site>.+)

Then , in the interceptors , I will put

“LineInterceptors”: [
{
“Name”: “WindowsStandardWindowsOutputInterceptor”,
“Arguments”: “/c echo \”{site} {line}\””,
“FullPath”: “cmd.exe”,
“FolderToExecute”: null,
“InterceptOutput”: true
}

and  will replace the

  “Arguments”: “/c echo \”{site} {line}\””,

with the

site

parsed from regex.

The code is pretty simple . As inputs, we have this.regEx for the regex and this.Arguments for the command line

var regex = new Regex(this.regEx);
var names = regex.
GetGroupNames().
Where(it => !int.TryParse(it, out var _)).
ToArray();

var matches = regex.Matches(this.Arguments);
if (matches.Count == 0)
throw new ArgumentException($” the regex {regEx} has no matches for {Arguments}”);

var m = matches.FirstOrDefault();
foreach(var g in names)
{
if (m.Groups[g].Success)
{
argsRegex[g]=m.Groups[g].Value;
}
else
{
throw new ArgumentException($”cannot find {g}  in regex {regEx} when parsing {Arguments}”);
}
}

That’s all! You can find code at https://github.com/ignatandrei/SideCarCLI

 

 

.

SideCarCLI- finish interceptors

Summary links SideCarCLI

NoName + Link 
1Description
2Specifications
3Refactor specifications and code
4Create Release
5Finish interceptors
6Send part of command to interceptors
7Line Interceptors
8Finish process after some time
9Documetation Diagram
10Technical Summary
11Create dotnet tool
( Description : SideCar for CLI applications. Interceptors for Line, Finish, Timer . See Code )

The idea for FinishInterceptors is very simple – when the initial  process, StartApp , is finished –then run some other programs . And here comes some decisions:

  1. Pass the ExitCode of the StartApp ? ( YES)
  2. Pass the output lines of text of the StartApp ? ( NO  – could be enormous )
  3. What to do if the StartApp has not finished in time ? ( Do nothing – we need the exit code)
  4. Wait for FinishInterceptors to finish ? If yes, how much time ? ( yes )

 

Same problems for TimerInterceptors and for LineInterceptors.

But, I think, the best way is to draw a diagram for that.

In the meantime, the project is already WIP

SideCarCLI–create releases

Summary links SideCarCLI

NoName + Link 
1Description
2Specifications
3Refactor specifications and code
4Create Release
5Finish interceptors
6Send part of command to interceptors
7Line Interceptors
8Finish process after some time
9Documetation Diagram
10Technical Summary
11Create dotnet tool
( Description : SideCar for CLI applications. Interceptors for Line, Finish, Timer . See Code )

Because all code is in Github, the easy way is GitHub Actions , https://github.com/features/actions . Some modifications are required on CSPROJ file to build for Linux or Windows :

<PropertyGroup>

<OutputType>Exe</OutputType>

<TargetFramework>netcoreapp3.1</TargetFramework>

<RuntimeIdentifiers>win-x64;linux-x64</RuntimeIdentifiers>

</PropertyGroup>

 

And after 6 commits, this is the ( almost) final version:

name: .NET Core

on:

push:

branches: [ main ]

pull_request:

branches: [ main ]

jobs:

build:

runs-on: ubuntu-latest

steps:

– uses: actions/checkout@v2

– name: Setup .NET Core

uses: actions/setup-dotnet@v1

with:

dotnet-version: 3.1.301

– name: Build

run: |

cd src

cd SideCarCLI

dotnet restore

dotnet build –configuration Release –no-restore -r linux-x64 -o linuxx64

dotnet build –configuration Release –no-restore -r win-x64 -o winx64

– uses: actions/upload-artifact@v2

with:

name: winx64

path: src/SideCarCLI/winx64/

– uses: actions/upload-artifact@v2

with:

name: linuxx64

path: src/SideCarCLI/linuxx64/

#- name: Test

#  run: dotnet test –no-restore –verbosity normal

 

Your job , if you accept, is to modify to use

dotnet publish

with trimmed and self contained and publish single file. ( you can see latest version at https://github.com/ignatandrei/SideCarCLI/blob/main/.github/workflows/dotnet-core.yml )

SideCarCLI–refactor command line

Summary links SideCarCLI

NoName + Link 
1Description
2Specifications
3Refactor specifications and code
4Create Release
5Finish interceptors
6Send part of command to interceptors
7Line Interceptors
8Finish process after some time
9Documetation Diagram
10Technical Summary
11Create dotnet tool
( Description : SideCar for CLI applications. Interceptors for Line, Finish, Timer . See Code )

When starting to implement the project, suddenly realize that list interceptors command does not belong to  start app command . So the application should be refactored to new specifications

– StartApp command – contains the whole list of commands for starting the app , including timer, adding interceptor

– Interceptors command – should list the interceptors  (timer, line, finish)

The interceptors should be defined in a separate file , like this

{
“TimerInterceptors”: [],
“LineInterceptors”: [
{
“Name”: “echo”,
“FolderToExecute”: null
}
],
“FinishInterceptors”: []
}

So this will be how it looks the command line

 

————–
command path:
————–
SideCar for any other application

Usage:  [command] [options]

Options:
-h|–help          Show help information
-max|–maxSeconds  max seconds for the StartApp to run

Commands:
about
interceptors
listAllCommands
StartAPP

Run ‘ [command] -h|–help’ for more information about a command.

The most simplest form it is:
SideCarCLI startApp –name YourApp

————–
command path:–>about
————–
Usage:  about [options]

Options:
-h|–help  Show help information

————–
command path:–>StartAPP
————–
start the CLI application that you need to intercept

Usage:  StartAPP [options]

Options:
-n|–name <fullPathToApplication>             Path to the StartApp
-a|–arguments <arguments_to_the_app>         StartApp arguments
-f|–folder[:<folder_where_execute_the_app>]  folder where to execute the StartApp – default folder of the StartApp
-aLi|–addLineInterceptor                     Add Line Interceptor to execute
-aTi|–addTimerInterceptor                    Add Timer Interceptor to execute
-aFi|–addFinishInterceptor                   Add Finish Interceptor to execute
-h|–help                                     Show help information

————–
command path:–>interceptors
————–
Usage:  interceptors [options]

Options:
-lLi|–ListLineInterceptor    List line interceptor
-lLi|–ListTimerInterceptor   List timer interceptor
-lFi|–ListFinishInterceptor  List timer interceptor
-h|–help                     Show help information

————–
command path:–>listAllCommands
————–
List all commands for the app

Usage:  listAllCommands [options]

Options:
-h|–help  Show help information

SideCarCLI–Architecture and specs–part 2

Summary links SideCarCLI

NoName + Link 
1Description
2Specifications
3Refactor specifications and code
4Create Release
5Finish interceptors
6Send part of command to interceptors
7Line Interceptors
8Finish process after some time
9Documetation Diagram
10Technical Summary
11Create dotnet tool
( Description : SideCar for CLI applications. Interceptors for Line, Finish, Timer . See Code )

The SideCarCLI application should start another application and intercept in various ways .  What will execute when intercept is not known in advance – so should be read at runtime  . So the SideCarCLI will load the interceptors and make then available to the application. Let’s say that , for the moment, the SideCarCLI will take all the configuration from the command line.

So let’s analyze the command line . The most simplest form it is:

SideCarCLI startApp  –name YourCLIApplication

For intercepting the output or the error of the console we do not know in advance what we will use. So we will make start dynamically what applications will  intercept the output lines provided by StartApp.

Also this is available for intercepting the result of the application and for the thing that is running periodically.

Also I want to provide a way for the users to make on-the-fly code to intercept – so I will make also a plugin architecture to provide interceptors on the fly.

Let’s see how the application command line will look  –

 

 

————–
command path:
————–
SideCar for any other application

Usage:  [command] [options]

Options:
-h|–help          Show help information
-max|–maxSeconds  max seconds for the StartApp to run

Commands:
_about
_listAllCommands
startApp

Run ‘ [command] -h|–help’ for more information about a command.

The most simplest form it is:
SideCarCLI startApp –name YourApp

————–
command path:–>_about
————–
Usage:  _about [options]

Options:
-h|–help  Show help information

————–
command path:–>startApp
————–
start the CLI application that you need to intercept

Usage:  startApp [command] [options]

Options:
-n|–name <fullPathToApplication>             Path to the StartApp
-a|–arguments <arguments_to_the_app>         StartApp arguments
-f|–folder[:<folder_where_execute_the_app>]  folder where to execute the StartApp – default folder of the StartApp
-h|–help                                     Show help information

Commands:
finishInterceptors
lineInterceptors
plugins
timer

Run ‘startApp [command] -h|–help’ for more information about a command.

————–
command path:–>startApp–>lineInterceptors
————–
Specify application for start when StartApp has a new line output

Usage:  startApp lineInterceptors [options]

Options:
-l|–list    List interceptors for lines
-a|–add     Add interceptor to execute
-f|–folder  folder where to start the interceptor
-h|–help    Show help information

————–
command path:–>startApp–>timer
————–
Specify timer to start an application at repeating interval

Usage:  startApp timer [options]

Options:
-i|–intervalRepeatSeconds  Repeat interval in seconds
-l|–list                   List interceptors to execute periodically
-a|–add                    Add interceptor to execute
-f|–folder                 folder where to start the interceptor
-h|–help                   Show help information

————–
command path:–>startApp–>finishInterceptors
————–
Specify interceptors for start when finish the app

Usage:  startApp finishInterceptors [options]

Options:
-l|–list    List interceptors for finish application
-a|–add     Add interceptor to execute
-f|–folder  folder where to start the interceptor
-h|–help    Show help information

————–
command path:–>startApp–>plugins
————–
Load dynamically plugins

Usage:  startApp plugins [options]

Options:
-f|–folder  folder with plugins
-l|–list    List plugins
-a|–add     Add interceptor to execute
-h|–help    Show help information

————–
command path:–>_listAllCommands
————–
List all commands for the app

Usage:  _listAllCommands [options]

Options:
-h|–help  Show help information

Andrei Ignat weekly software news(mostly .NET)

* indicates required

Please select all the ways you would like to hear from me:

You can unsubscribe at any time by clicking the link in the footer of our emails. For information about our privacy practices, please visit our website.

We use Mailchimp as our marketing platform. By clicking below to subscribe, you acknowledge that your information will be transferred to Mailchimp for processing. Learn more about Mailchimp's privacy practices here.