Benchmarking RSCG vs Reflection
I make a microservices Buffet . In this I consider having email as a service . When the DevOps wants email, he can choose between various plugins ( simple smtp email, gmail, exchange, others). Those plugins can have various properties – that must be edited by the primary administrator of the microservice. The properties can be discovered at runtime ( via Reflection ) o r at build time ( via Roslyn Source Code Generators – RSCG ).
But – we should see what is faster, right ? And the feeling is that RSCG is always faster – but it is , really ? Let’s see…
So = let’s make a test with https://github.com/dotnet/BenchmarkDotNet . You can have the source code by going to https://github.com/ignatandrei/AOP_With_Roslyn/tree/master/AOPMethods .
First , the class that is tested
public partial class EmailSmtpClientMS { public EmailSmtpClientMS() { Port = 25; } public string Name { get; set; } public string Type { get { return this.GetType().Name; } } public string Host { get; set; } public int Port { get; set; } public string Description { get { return $"{Type} {Host}:{Port}"; } } }
Second, with AOPMethods I generate the read properties values – properties that you can read – via a dictionary and via a switch. This can be achieved simply :
[AutoMethods(template = TemplateMethod.CustomTemplateFile, CustomTemplateFileName = "ClassToDictionary.txt")] public partial class EmailSmtpClientMS
And this will be generated by RSCG for the switch
protected object GetValueProperty(string propName) { switch (propName) { //true true case "Name": return this.Name; //true false case "Type": return this.Type; //true true case "Host": return this.Host; //true true case "Port": return this.Port; //true false case "Description": return this.Description; default: throw new ArgumentException("cannot find property " + propName); } }
and for the dictionary
private IDictionary<string, PropertyHelper> MyProperties() { var data = new Dictionary<string, PropertyHelper>(); PropertyHelper ph; ph = new PropertyHelper(); ph.Name = "Name"; ph.Type = "string"; ph.CanRead = !false; ph.CanWrite = !false; ph.Value = this.Name; data.Add("Name", ph); //Name string ph = new PropertyHelper(); ph.Name = "Type"; ph.Type = "string"; ph.CanRead = !false; ph.CanWrite = !true; ph.Value = this.Type; data.Add("Type", ph); //Type string ph = new PropertyHelper(); ph.Name = "Host"; ph.Type = "string"; ph.CanRead = !false; ph.CanWrite = !false; ph.Value = this.Host; data.Add("Host", ph); //Host string ph = new PropertyHelper(); ph.Name = "Port"; ph.Type = "int"; ph.CanRead = !false; ph.CanWrite = !false; ph.Value = this.Port; data.Add("Port", ph); //Port int ph = new PropertyHelper(); ph.Name = "Description"; ph.Type = "string"; ph.CanRead = !false; ph.CanWrite = !true; ph.Value = this.Description; data.Add("Description", ph); //Description string return data; }
The spec for benchmark are :
BenchmarkDotNet=v0.13.0, OS=Windows 10.0.19043.1052 (21H1/May2021Update)
Intel Core i7-6600U CPU 2.60GHz (Skylake), 1 CPU, 4 logical and 2 physical cores
.NET SDK=5.0.301
[Host] : .NET 5.0.7 (5.0.721.25508), X64 RyuJIT
DefaultJob : .NET 5.0.7 (5.0.721.25508), X64 RyuJIT
Third, I benchmark obtaining one single property – the Host – via the 3 methods:
public partial class EmailSmtpClientMSOneProperty: EmailSmtpClientMS { [Benchmark] public string GetHostReflection() { return this.GetType().GetProperty("Host").GetValue(this).ToString(); } [Benchmark] public string GetHostViaDictionary() { return this.ReadMyProperties()["Host"].ToString(); } [Benchmark] public string GetHostViaSwitch() { return this.GetValueProperty("Host").ToString(); } }
And in Program.cs
BenchmarkRunner.Run<EmailSmtpClientMSOneProperty>( ManualConfig .Create(DefaultConfig.Instance) .WithOption(ConfigOptions.DisableOptimizationsValidator, true) );
( of course, I have added on the class
//[SimpleJob(RuntimeMoniker.Net50)]
//[ShortRunJob(RuntimeMoniker.Net50)]
//[DryJob(RuntimeMoniker.Net50)]
[Orderer(SummaryOrderPolicy.FastestToSlowest, MethodOrderPolicy.Declared)]
//[SimpleJob(RuntimeMoniker.NetCoreApp31)]
[RPlotExporter]
[CsvMeasurementsExporter]
[MemoryDiagnoser]
[HtmlExporter]
[MarkdownExporterAttribute.GitHub]
)
The results are in ns –so, the less/smaller , that means better results.
The results are here in HTML form :
Method | Mean | Error | StdDev | Median | Gen 0 | Gen 1 | Gen 2 | Allocated |
---|---|---|---|---|---|---|---|---|
GetHostViaSwitch | 18.07 ns | 0.434 ns | 0.549 ns | 18.02 ns | – | – | – | – |
GetHostReflection | 144.13 ns | 2.582 ns | 5.501 ns | 142.13 ns | – | – | – | – |
GetHostViaDictionary | 451.59 ns | 12.363 ns | 33.635 ns | 441.72 ns | 0.3057 | – | – | 640 B |
The graphic may be more interesting:
Surprised ? The RSCG Switch Property is indeed the fastest one – but the Reflection is faster than RSCG Property Dictionary ( or , at least , for my implementation).
However, I realized that in real life , I will retrieve all properties in a Dictionary to be edited . So all implementations should occur the penalty of creating a Dictionary . Time for next benchmark . This time , the code is
[Benchmark] public IDictionary<string,object> GetHostReflection() { var props = this.GetType() .GetProperties() .Where(it=> it.CanWrite) .ToDictionary(it => it.Name, it=>it.GetValue(this)); ; return props; } [Benchmark] public IDictionary<string, object> GetHostViaDictionary() { var props = this.ReadMyProperties(); return props; } [Benchmark] public IDictionary<string, object> GetHostViaSwitch() { var props = ReadProperties .ToDictionary(it => it, it => GetValueProperty(it)); return props; }
and the results are:
Method | Mean | Error | StdDev | Median | Gen 0 | Gen 1 | Gen 2 | Allocated |
---|---|---|---|---|---|---|---|---|
GetHostViaDictionary | 462.1 ns | 14.70 ns | 40.97 ns | 453.6 ns | 0.3052 | – | – | 640 B |
GetHostViaSwitch | 479.5 ns | 7.34 ns | 7.54 ns | 479.9 ns | 0.2708 | – | – | 568 B |
GetHostReflection | 973.0 ns | 78.35 ns | 231.01 ns | 911.5 ns | 0.1984 | – | – | 416 B |
Now the graphic will help:
Interesting , right ?
Reflection = as normal – is the slowest one. But the difference between RSCG switch and RSCG Dictionary is not too much…
Conclusion 1: the feeling was right at the end. But – the first result was deceiving
Conclusion 2: Creating a dictionary is more time consuming than a simple reflection for one property retrieved
Conclusion 3: I do prefer RSCG Dictionary vs RSCG switch – less work for me as a programmer and similar time results.
Conclusion 4: do not over engineer if you do not feel the need . For just one property, Reflection is better….
Conclusion 5: This is not final. I should also write the values of the properties . Maybe next time a new benchmark….