Docker–third part–part 19
Trying different codes in the csproj to add dynamically content from a folder ( including the dll’s copied there ) and to retrieve in the output directory: ( reading from internet of all those stuff)
Version 1:
<!–<Target Name=”ContentsBeforeBuild” AfterTargets=”BeforeBuild”>
<ItemGroup>
<Content Include=”plugins\**” />
</ItemGroup>
</Target>—>
Version 2:
<!–<Content Update=”plugins\**\*.dll” CopyToOutputDirectory=”Always” LinkBase=”plugins”>
</Content>—>
Version 3:
<!–<Content Update=”$([System.IO.Directory]::GetFiles(‘$(ProjectDir)plugins’, ‘*.*’, SearchOption.AllDirectories))” CopyToOutputDirectory=”PreserveNewest”>–>
<!–</Content>—>
Version 4: – works!
<ItemGroup>
<None Remove=”plugins\” />
<Content Include=”plugins\**\*.dll” CopyToOutputDirectory=”Always”/>
</ItemGroup>
Now, I want to make this docker each time that I modify and show to the end users to test it.
However, this should be not in the same process as running the tests – the tests should be very fast in order to see problems – the making of release software could take minutes.
Going to https://dev.azure.com/ignatandrei0674/ and making a new project to deal with this.Creating new project, integrating with GitHub – seamless integration with .NET Core – even created a azure-pipelines.yml for me.
Extra bonus: GitHub actions runs on Ubuntu. I will put Azure to run on Windows. This way, I know that the build of the project works either way
Now, a different error in docker:
Access to the path ‘C:\app\InfoValutar\InfoValutarDOS\obj\InfoValutarDOS.csproj.nuget.dgspec.json’ is denied.
Oh, forgot to put a .dockerignore . First I put where the docker file was –big mistake. Should be put where the folder to be included is!
Now ,a different error in docker
Access to the path ‘C:\app\InfoValutar\InfoValutarWebAPI\InfoValutarWebAPI.xml’ is denied.
Oh, give me a break! Delete the .xml and add later to .gitignore and .dockerignore ( as full path, I do not want to ignore ALL xml files)
I works now!!!!
Infovalutar
And one hour passes...(This is the result of 1 hour per day auto-challenge as a full cycle developer for an exchange rates application)
( You can see the sources at https://github.com/ignatandrei/InfoValutar/ )
Docker–second part–part 18
Now , I must replace all with powershell files. Some problem – if I build twice, it says : resource already exists. So I need to add
–Force
parameter.
Now pushing to GitHub https://github.com/ignatandrei/InfoValutar/commit/2c487963da53f04f94a0a7b060999f8e27635179
and see if it works. No, error on .sln build.
ObjectNotFound: (/home/runner/work/I\u2026nfoValutar/plugins”
Apparently, it is a Unicode character ?
Try to save as ASCII file. Waiting for GitHub action to do it’s magic.
Same Error – now putting some messages with echo to see what is there
echo “args 0 ” $args[0]
echo “args 1 ” $args[1]
$a= Get-Location
echo $a
echo now copy
Now waiting for GitHub actions to display – it does NOT show the console from powershell, just the error!
Putting in After Build to copy the plugins.
Same error. Looking very, very attentive to the command , I see now. Can you see ?
pwsh $(ProjectDir)preBuild.ps1 “”$(RepoRoot)plugins\” “$(TargetDir)”
Apparently, pwsh from Windows is more forgetful than the Linux one… ( or it is not the same ? Maybe because I have powershell on my PC ?)
Now ( after deleting double double quotes) shows also the powershell messages!
Now I can make docker!
And the first problem arises. When building the .sln , it copies to the output folder the plugins. But not when publish the csproj.
Try with –no-build when publish and adding
<RuntimeIdentifiers>win10-x64;linux-x64</RuntimeIdentifiers>
to the .csproj
Now, dotnet publish does not copy the plugins – I should copy by hand ?
So – figure that I do not copy to the final output – but direct to the project.
1. I should copy there in the build event of the project – modify the arguments to have copy in the plugins folder
2. I should include in the .csproj any new plugins:
<Content Include=”plugins\**”>
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
</Content>
Error duplicate items – found it was
<Content Update=”plugins\**” CopyToOutputDirectory=”PreserveNewest”>
3. Do not include in GitHub – already done – plugins folder is not included
Now trying to execute in swagger to give list of plugins / banks => internal server error
Trying to publish without single file => give empty list /banks. Let’s see -but next time
Infovalutar
And one hour passes...(This is the result of 1 hour per day auto-challenge as a full cycle developer for an exchange rates application)
( You can see the sources at https://github.com/ignatandrei/InfoValutar/ )
Docker–first part–part 17
Trying to put the web into docker. In order to have easy builds for the developer , we copy the sln, then the csproj , then dotnet restore, then copy files. This way we get advantage of Docker caching.
Created dockerForWebAPI.txt
and run
docker build -f dockerForWebAPI.txt -t infovalutar ..
Docker has not recursive copy – https://github.com/moby/moby/issues/15858
Solution: copy the .csproj one by one. ( other solution: .tar file and copy the .csproj into tar, then use ADD command)
So copy the .sln with
COPY InfoValutar/*.sln ./
Then trying to copy files
Version 1
COPY InfoValutar/InfoValutarDOS/InfoValutarDOS.csproj ./
( not good, same dir)
Version 2
COPY InfoValutar/InfoValutarDOS/InfoValutarDOS.csproj ./app/InfoValutarDOS/InfoValutarDOS.csproj
(not good , discovered that creates another app folder – thanks for the docker run –rm -it infovalutar:latest /bin/bash )
Version 3
COPY InfoValutar/InfoValutarDOS/InfoValutarDOS.csproj ./InfoValutarDOS/InfoValutarDOS.csproj
Good. Now copy files and build solution
And now it is the moment when echo and xcopy for plugins will bite me – no xcopy on linux….
The only thing that I know to work in WIndows and Linux is … Powershell.
Any .NET Core 3 has this – run pwsh. It remains to install to my directory – see https://andrewlock.net/new-in-net-core-3-local-tools/
The version of powershell on my PC differs from the version on Windows Container from GitHub actions. Switch to Ubuntu.
Again – does not work . Try with pwsh –Command.
It does work on GitHub actions but NOT on my PC ( does not recognize command. ) . Thinking different pwsh version.
But on thing is the same – running file.
Create a powershell file
Copy-Item -Path $args[0] -Destination $args[1] –Recurse
and running this file with
pwsh $(ProjectDir)pb.ps1 “$(TargetDir)” “$(SolutionDir)plugins\$(ProjectName)”
( Do not forget to run first
dotnet tool restore
in the solution folder!)
Infovalutar
And one hour passes...(This is the result of 1 hour per day auto-challenge as a full cycle developer for an exchange rates application)
( You can see the sources at https://github.com/ignatandrei/InfoValutar/ )
Swagger/OpenAPI –part 16
In order to see the WebAPI, Swagger is a default technology. Reading https://docs.microsoft.com/en-us/aspnet/core/tutorials/web-api-help-pages-using-swagger?view=aspnetcore-3.0 , seems to have 2 options: Swashbuckle and NSwag. Looking at github repositories, both have > 2k stars, active development, tutorials. The only thing that I see more at NSwag are generators for Angular.
Integrated NSwag as for the tutorial from https://docs.microsoft.com/en-us/aspnet/core/tutorials/getting-started-with-nswag?view=aspnetcore-3.0&tabs=visual-studio . There is a small problem – when I try the swagger , I should put explicit the version. However, a not experienced user can not see this value.
Trying to put documentation : when I go to Project=>Properties=>Build and click the XML documentation, shows the full path to the XML. I do not need this! So I edit the .csproj directly and change from full path to file
Also, because I want to be sure to have documentation, I change “warnings to errors”.
So now I must document Program and main(removed public from startup.cs and createwebhost – to not document)
Total: 6 changes , https://github.com/ignatandrei/InfoValutar/commit/4fbff903c4510acbcb219d0263ae89736e9f2d8d
So now I can browse to swagger
However, I should indicate to people that swagger is enabled to invite them browse to this – and this is easy to do in Kestrel, right ? Just take the address of the url …
So, trying in Program.cs to find the adresses:
var build = CreateHostBuilder(args).Build(); var s = build.Services.GetService(typeof(KestrelServerOptions)); var s1 = build.Services.GetService(typeof(IServer)); var s2 = build.Services.GetService(typeof(IServerAddressesFeature)); build.Run();
Not a chance – not yet configured. So , next chance is in Startup.cs
public void Configure(IApplicationBuilder app, IWebHostEnvironment env) { //other code var sa = app.ServerFeatures.Get<IServerAddressesFeature>(); var urls =string.Join(",", sa.Addresses.Select(it => it + "swagger")); Console.WriteLine("please use " + urls);
( for this, you must not use IIS Express, but rather the project)
Infovalutar
And one hour passes...(This is the result of 1 hour per day auto-challenge as a full cycle developer for an exchange rates application)
( You can see the sources at https://github.com/ignatandrei/InfoValutar/ )
DOTNET DAYS -NEXT YEAR
Hi!
We are delighted to invite you to share our passion for technology at the second edition of the dotnetdays conference that will take place on 29 February 2020.
About dotnetdays
Dotnetdays is a .NET event driven by a great passion for technology and the .NET community.
The second edition will take place on 29 February 2020 at the International Hotel in Iasi, Romania. This year we’re excited to have some great speakers that are present at major conferences in Europe and the USA (most of them are for the first time in Romania), such as:
- Jon Galloway – Executive Director – .NET Foundation
- Alex Mang – Microsoft MVP & Regional Director, Microsoft Ignite Speaker
- Alex Thissen – Microsoft MVP & Cloud Architect
- Dennis Doomen – FluentAssertions author (more than 30 million NuGet downloads)
- Dennis van der Stelt – distributed system guru and part of NServiceBus team, will do 2 days workshop on SOA
- Jimmy Bogard – AutoMapper and MediatR author
- Martin Beeby – Principal Developer Evangelist @Amazon
And that’s not all. More speakers and workshops will be announced soon.
We will host 4 days of intensive technical talks and workshops. The schedule is yet to be finalized, but you get a peek at what’s about to come:
- 2 days of intensive workshops
- 1 full-day of free workshops delivered by wonderful people in the community
- 1 full-day conference
Until the 9th of January, you can also purchase tickets at Regular price. Get yours here.
Looking forward to your answer,
The dotnetdays team
Friday Links 346
Repairing the build and versioning–part 15
Now , after copy the plugins, the build is not working.
Thnking about the build events, I realize that to find same copy command on Linux and Windows is difficult – so Windows it is.
( and, BTW , In LInux the macro $(SolutionDir ) does not work use repoRoot
https://docs.microsoft.com/en-us/visualstudio/msbuild/customize-your-build?view=vs-2019
https://github.com/Microsoft/msbuild/issues/2397
)
Switch windows-latest on https://help.github.com/en/actions/automating-your-workflow-with-github-actions/workflow-syntax-for-github-actions#jobsjob_idruns-on
Now tests are failing – repairing the tests.
I realize that I cannot build just the csproj – the solution sln knows the order in which to build the projects- but the .csproj does not. So now building the sln and then publish
Now giving error in Github, but not on local. More, there was a not repeatable . I figured was because of the parallel builds so this is the final line
dotnet build –configuration Release InfoValutar/InfoValutar.sln /m:1
Now it works!
The only programming thing that I have done is to add versioning , https://github.com/microsoft/aspnet-api-versioning/wiki/How-to-Version-Your-Service . I do tend to say that the URL is a Unique Identifier – so I want the versioning be part of the URL.
As advised by https://github.com/microsoft/aspnet-api-versioning/wiki/Versioning-via-the-URL-Path
I added the pacakage Microsoft.AspNetCore.Mvc.Versioning and
services.AddApiVersioning();
and on the controller
[ApiVersion( “1.0” )]
[Route(“api/v{version:apiVersion}/[controller]/[action]”)]
and now I can access
api/v1.0/TodayRates/banks
Infovalutar
And one hour passes...(This is the result of 1 hour per day auto-challenge as a full cycle developer for an exchange rates application)
( You can see the sources at https://github.com/ignatandrei/InfoValutar/ )
Plugin copy and caching- part 14
I have had 2 problems to solve
- How to put the plugins for the console and web api ?
- How to cache the data ? ( plugins / exchange rates)
The solution for the first problem should be easy for the programmer PC, and for the deploying the application ( console/ web api/ etc )There are several solutions :
- Each plugin knows where to copy the dll ( console/web api)
- The application ( console / webapi) knows a special place from which to copy the plugin definition
I tend to agree with both – but mostly with the second. So … the idea is as follows:
The plugin builds and copy the output to a special folder, named “plugins” ( what a surprise, righyt?)
The console /webapi copies the folder plugins under his directory
The only problem here is that we should ensure that plugins are build first, even if the application does NOT have reference to them.
For this, there is a special menu in VS for solution, namely “build order” – you go to this –and after to the second tab, project dependencies. For the console and web I select as dependency the two loading projects – InfoValutarNBR and InfoValutarECB. The changes are reflected into the sln file
I could have done this on just the InfoValutarLoadingLibs project – but maybe I can have another way to load the libraries ( not so much for the moment )
So now I begin to modify what happens
after build events for InfoValutarNBR and InfoValutarECB
xcopy “$(TargetDir)*.*” “$(SolutionDir)plugins\$(ProjectName)” /E /F /I /Y /R
and pre build events for InfoValutarDOS and InfoValutarWebAPI.
Now, there are several things to do:
- Add to .gitignore the plugins folder to not put into the source control( ok)
- Write some documentation about the process
- Think about linux/docker compiling – there is a xcopy there ?
About the documentation – I just did. More, I create a InfoValutarPluginStarter , in order for others to copy this project and make a new plugin.
Realized also that the WebAPI could be simpler
[HttpGet(“{bank}”)]
public IAsyncEnumerable<ExchangeRates> Rates([FromRoute] string bank)
And also that I should throw another exception if Bank is null or empty:
if(string.IsNullOrWhiteSpace(bank))
throw new ArgumentNullException($”{nameof(bank)} cannot be empty”);
For docker compiling – next time
Infovalutar
And one hour passes...(This is the result of 1 hour per day auto-challenge as a full cycle developer for an exchange rates application)
( You can see the sources at https://github.com/ignatandrei/InfoValutar/ )
Creating the WebAPI–part 13
I am now tired of continuously improving the console application – so now some work that others can see really. Creating the WebAPI is simple as a project – and I was amazed to see a WeatherForecast controller.
I need just to be able to see the today exchange rates via API . So created a TodayRates , modify the launchSettings.json in order to start with my controller.
I need the plugin that loads to be a global dependency. So I add as a Singletion in the services
services.AddSingleton< LoadExchangeProviders>(new LoadExchangeProviders(“plugins”));
and then I inject into the controller constructor ( I can do in the method also, but many methods needs the same dependency )
public TodayRates(ILogger<TodayRates> logger, LoadExchangeProviders prov)
{
_logger = logger;
_prov = prov;
}
Now I need a method to see what are the banks / plugins that are loaded . Linq to the rescue :
[HttpGet]
public IEnumerable<string> Banks()
{
return _prov
.LoadExchange()
.Select(it=>it.Bank)
.ToArray();
}
Now we should load the data from the provider to show actual rates
public async IAsyncEnumerable<ExchangeRates> Rates(string bank)
{
var provBank = _prov
.LoadExchange()
.FirstOrDefault(it => string.Equals(bank, it.Bank, StringComparison.InvariantCultureIgnoreCase));switch (provBank)
{
case null:
yield return null;
break;
default:
{
await foreach (var data in provBank.GetActualRates())
{
yield return data;
}break;
}
}}
All good. There are 3 problems:
- How to put the plugins for the console and web api ?
- How to cache the data ? ( plugins / exchange rates)
- Should we throw an exception when a bank does not exists, instead of returning null ?
Thinking of this code, I realized that I begin to put logic into controllers. So – let’s move from the controller to the business.
For example, now I have this:
[HttpGet]
public IEnumerable<string> Banks()
{
return _prov.Banks();
}
And this
[HttpGet]
public IAsyncEnumerable<ExchangeRates> Rates(string bank)
{
return _prov.Rates(bank);
}
So now I can change in the Business Layer without problems.
Looking at the Console Application vs WebAPI, I realized that are different. The console is loading the whole data from the providers – the WebAPI is loading on demand. We could do for the console the same behavior – but the console is loading very fast all data.
For the answer at
Should we throw an exception when a bank does not exists, instead of returning null ?
– seems to be yes.
So instead of
case null:
yield return null;
I put
case null:
throw new ArgumentOutOfRangeException(nameof(bank),$”cannot find {bank}”);
For the 2 questions, the answer next time.
Infovalutar
And one hour passes...(This is the result of 1 hour per day auto-challenge as a full cycle developer for an exchange rates application)
( You can see the sources at https://github.com/ignatandrei/InfoValutar/ )