Logging and instrumentation

Why

There are at least two reasons for an application to register what tfhe user do.

The first one and the most important is to see what is happening when a user has an error: what have been done, on which path did he go and what additional data(method parameters, usually) do you need for reproducing the bug.

The second one is to provide the application a way to record in-obtrusive messages for same actions that may seems not so right(check the event log for your computer for more examples 😉 )

Requirements for a logging system

The logging system should be:

1. Robust – if it can not write the message, it should not fail the entire app

2. Multi-write-data : it can write easily to multiple systems(such as databases, files on disk, send emails and so on)

3. Flexibility : the configuration of the logging system should be easy to modify. More, the modifications should be applied and “re-loaded” without re-compiling or re-starting the application

An implementation

For the implementation I will use log4net , http://logging.apache.org/log4net/ that it is satisfies all three conditions.

We will do a console application that sends a random error

using System;

using System.Collections.Generic;

using System.Linq;

using System.Text;

namespace ConsoleDemo1

{

class Program

{

static void Main(string[] args)

{

long ticks = DateTime.Now.Ticks;

FirstFunction(ticks);

}

static void FirstFunction(long i)

{

if (i % 2 == 0)

{

throw new ArgumentException("from first function");

SecondFunction(i);

}

static void SecondFunction(long i)

{

if (i % 2 == 1)

{

throw new ArgumentException("from second function");

}

}

}

}

As you see this application will have an error no matter if the ticks are even or odd. Download project from here : http://msprogrammer.serviciipeweb.ro/wp-content/uploads/loggingDemo.zip

Now we will see how the log4net writes data and error to a physical storage, in order to can be retrieved later.

Download log4net, add reference to it and make the following 4 steps

1. add a config file (I will name it log4net.config ) and in properties put “Copy to output Directory” to “Copy always” .

clip_image002[4]

Examples of log4net config files you will find everywhere on internet . If you look at the project you will see a bunch of “appenders” : ConsoleAppender, RollingLogFileAppender , SmtpAppender . The appender appends the log to the storage provided (Console, File, Email ). You will find a list of appenders at http://logging.apache.org/log4net/release/sdk/log4net.Appender.html and, if you do not found some appender you can wrote your own – just look at the source code.

For this application I want to use just ConsoleAppender and RollingFileAppender

<root>

<level value=”DEBUG” />

<appender-ref ref=”ConsoleAppender” />

<appender-ref ref=”RollingLogFileAppender” />

</root>

2. In order to see the file, put


log4net.Config.XmlConfigurator.ConfigureAndWatch(new System.IO.FileInfo("log4net.config"));

(for asp.net application ,put


log4net.Config.XmlConfigurator.ConfigureAndWatch(new System.IO.FileInfo(Server.MapPath("~/log4net.config")));

)

3. Add a variable to log the errors :


private static readonly log4net.ILog _logger = log4net.LogManager.GetLogger(System.Reflection.MethodBase.GetCurrentMethod().DeclaringType);

4. Add a try/catch to log errors :

try

{

FirstFunction(ticks);

}

catch (Exception ex)

{

if (_logger.IsErrorEnabled)

{

_logger.Error("see an error!", ex);

}

throw;

}

Now, if you run the ConsoleDemoLog4net project, then you will see how to exception is logged twice and you will find the currentlog.txt in the same folder as your application executable.

Automatically instrument an application(dll, exe, asp.net)

This was pretty good – but what about putting a log to every method to see where the application flow has been gone and with which arguments?

You can do this with postsharp / log4postsharp. Postsharp is … Log4Postsharp is …

Let’s make the application do this automatically.

We download Postsharp 1.0 (not 1.5!) from http://www.softpedia.com/progDownload/PostSharp-Download-114994.html (you can found a more good paid version at http://www.sharpcrafters.com/downloads/ )

Now we will do the following :

1. Add the log4net.config from the previous project

2. Add log4net , PostSharp.Public and PostSharp reference

3. Add to the main just:

log4net.Config.XmlConfigurator.ConfigureAndWatch(new System.IO.FileInfo("log4net.config"));

Do NOT add the log4net code to intercept!

4. Modify AssemblyInfo.cs and put

using Log4PostSharp;

[assembly: Log(AttributeTargetTypes = "*", EntryLevel = LogLevel. Error, EntryText = "postsharp :entering {signature} {paramvalues}", ExitLevel = LogLevel. Error, ExceptionLevel = LogLevel.Error, ExitText = "postsharp :exit {signature} {paramvalues} =&gt; {returnvalue}", ExceptionText = "postsharp : error in {signature} ")]

5. Add a text file named <yourprojectname>.psproj (in this case, ConsoleDemoPostSharp.psproj ) with the following content:

<?xml version=”1.0″ encoding=”utf-8″ ?>

<Project xmlns=”http://schemas.postsharp.org/1.0/configuration”>

<SearchPath Directory=”bin/{$Configuration}”/>

<SearchPath Directory=”{$SearchPath}” />

<SearchPath Directory=”lib” />

<Tasks>

<AutoDetect />

<Compile TargetFile=”{$Output}” IntermediateDirectory=”{$IntermediateDirectory}” CleanIntermediate=”false” />

</Tasks>

</Project>

6. Edit by hand the csproj file and add this :

<PropertyGroup>

<DontImportPostSharp>True</DontImportPostSharp>

<PostSharpDirectory> ..\libs\ postsharp\</PostSharpDirectory>

<PostSharpUseCommandLine>True</PostSharpUseCommandLine>

</PropertyGroup>

<Import Project="$(PostSharpDirectory)PostSharp.targets" Condition=" Exists('$(PostSharpDirectory)PostSharp.targets') " />

7. If there is a screen to ask you about a project modification, please tell “Load project normally”

So now , when it’s compiling , it says :

“D:\programe\youtube\loggingDemo\libs\postsharp\PostSharp.exe” “D:\programe\youtube\loggingDemo\ConsoleDemoPostSharp\ConsoleDemoPostSharp.psproj” “D:\programe\youtube\loggingDemo\ConsoleDemoPostSharp\obj\Release\Before-PostSharp\ConsoleDemoPostSharp.exe” “/P:Output=obj\Release\ConsoleDemoPostSharp.exe ” “/P:ReferenceDirectory=D:\programe\youtube\loggingDemo\ConsoleDemoPostSharp ” “/P:Configuration=Release ” “/P:Platform=AnyCPU ” “/P:SearchPath=bin\Release\,obj\Release\, ” “/P:IntermediateDirectory=obj\Release\PostSharp ” “/P:CleanIntermediate=False ” “/P:MSBuildProjectFullPath=D:\programe\youtube\loggingDemo\ConsoleDemoPostSharp\ConsoleDemoPostSharp.csproj ” “/P:SignAssembly=False ” “/P:PrivateKeyLocation= ”

PostSharp 1.0 [1.0.12.469] – Copyright (c) Gael Fraiteur, 2005-2008.

EXEC : warning PS0064: A new version of PostSharp 1.0 is available. You have currently 1.0.12.469 and you could download the version 1.0.13.630 from http://www.sharpcrafters.com/postsharp/download.

info PS0035: C:\Windows\Microsoft.NET\Framework\v2.0.50727\ilasm.exe “D:\programe\youtube\loggingDemo\ConsoleDemoPostSharp\obj\Release\PostSharp\ConsoleDemoPostSharp.il” /QUIET /EXE /PDB “/RESOURCE=D:\programe\youtube\loggingDemo\ConsoleDemoPostSharp\obj\Release\PostSharp\ConsoleDemoPostSharp.res” “/OUTPUT=D:\programe\youtube\loggingDemo\ConsoleDemoPostSharp\obj\Release\ConsoleDemoPostSharp.exe” /SUBSYSTEM=3 /FLAGS=1 /BASE=19595264 /STACK=1048576 /ALIGNMENT=512 /MDV=v2.0.50727

ConsoleDemoPostSharp -> D:\programe\youtube\loggingDemo\ConsoleDemoPostSharp\bin\Release\ConsoleDemoPostSharp.exe

Done building project “ConsoleDemoPostSharp.csproj”.

We then run the program and enjoy the power of PostSharp :

2010-04-17 23:29:12,705 [1] ERROR 2010-04-17 23:28:43,265 [1] ERROR D:\programe\youtube\loggingDemo\ConsoleDemoPostSharp\Program.cs ConsoleDemoPostSharp.Program [(null)] – postsharp :entering Void FirstFunction(Int64) “634071437232634478”

[more data]

So , as you see it gives you not only the trace , but also the value : 634071437232634478

You can download code from http://msprogrammer.serviciipeweb.ro/wp-content/uploads/loggingDemo.zip

Additional resources

Tracing : http://www.15seconds.com/Issue/020910.htm

Enterprise logging : http://msdn.microsoft.com/en-us/library/cc309257%28v=MSDN.10%29.aspx

VS2010 with Dotfuscator : http://msdn.microsoft.com/en-us/library/ff460257.aspx

CCI metadata : http://ccisamples.codeplex.com

Postsharp 2.0 improvements over 1.0 : one of many examples here : http://www.sharpcrafters.com/blog/post/introducing-postsharp-2-0-1-notifypropertychanged.aspx

Html Agility Pack

This is the ultimate reference of reading web pages.

IF you want to do it yourself , you can try with WebRequest , http://msdn.microsoft.com/en-us/library/system.net.webrequest.aspx . But not all HTML is an XML – so you must find a method to parse. But am I the only one ?No – so I found the  HTML Agility Pack , http://www.codeplex.com/htmlagilitypack , that knows how to transform HTML in XML. The code is easy :

HtmlWeb hw = new HtmlWeb();
hw.AutoDetectEncoding = true;
HtmlDocument  doc = hw.Load(Url);
HtmlNode NodeRoot = doc.DocumentNode;

And from NodeRoot you can start XPATH with SelectNodes . Try it – it is awesome!

Sql Server Management Studio free awesome addons

The two addons that I can not live without in Sql Server and that, more , are free :

  1. SSMS Tools Pack – now 1.7.5.1 , http://www.ssmstoolspack.com/Download
  2. SQL Search – now1.0 http://www.red-gate.com/products/SQL_Search/

 

The SSMS Tools Pack maintains a history of your commands . More, it saves on the C:\SSMSTools\SQLQueryActionLog

image

 

The Sql Search finds a text in the objects in the database – very useful if you decide to change a column name and find all stored procedures that have this reference

image

MVP

It’s the second year that I am MVP for C# for Romania – and I feel good about it!

image

 

Thursday, April 01, 2010
Re: Andrei Ignat, Most Valuable Professional, Visual C#
To whom it may concern,

It is with great pride we announce that Andrei Ignat has been awarded as a MicrosoftÂŽ Most Valuable Professional
(MVP) for 4/1/2010 – 4/1/2011. The Microsoft MVP Award is an annual award that recognizes exceptional
technology community leaders worldwide who actively share their high quality, real world expertise with users and
Microsoft. All of us at Microsoft recognize and appreciate Andrei’s extraordinary contributions and want to take this
opportunity to share our appreciation with you.
With fewer than 5,000 awardees worldwide, Microsoft MVPs represent a highly select group of experts. MVPs
share a deep commitment  to community and a willingness  to help others. They represent  the diversity of  today’s
technical communities. MVPs are present in over 90 countries, spanning more than 30 languages, and over 90
Microsoft  technologies. MVPs share a passion  for  technology, a willingness  to help others, and a commitment  to
community. These are the qualities that make MVPs exceptional community leaders. MVPs’ efforts enhance
people’s lives and contribute to our industry’s success in many ways. By sharing their knowledge and experiences,
and providing objective feedback, they help people solve problems and discover new capabilities every day. MVPs
are technology’s best and brightest, and we are honored to welcome Andrei as one of them.
To recognize the contributions they make, MVPs from around the world have the opportunity to meet Microsoft
executives, network with peers, and position themselves as technical community leaders. This is accomplished
through speaking engagements, one on one customer event participation and technical content development.
MVPs also receive early access to technology through a variety of programs offered by Microsoft, which keeps
them on the cutting edge of the software and hardware industry.
As a recipient of this year’s Microsoft MVP award, Andrei joins an exceptional group of individuals from around the
world who have demonstrated a willingness to reach out, share their technical expertise with others and help
individuals maximize their use of technology.
Sincerely,
Rich Kaplan
Corporate Vice President
Customer and Partner Advocacy
Microsoft Corporation

ASP.NET MVC , ORM and ViewModels

Since I am active on asp.net/mvc I find many peoples asking about

  1. One to many relationship .
  2. Dropdownlist
  3. How to use MVC with other ORM frameworks instead of Linq To Sql(L2S) ?
  4. What is a ViewModel and what is his purpose ?

For answering, I have made a small application with MVC2 in order to detail my answer.

First the problem :

You have Employees and Department.Each Employee belongs to an Department (let’s forget the fact that the employee can belong for a final date to the Department or any other life ideas)

The request is to create and view Employees .(forget delete,update – if I will have time and/or by general request , I will do this also this)

The database:

image

The beginning of the application :

First Layer , Data Access Layer
I use EF 3.5 (not 4.0 with POCO style – because is not ready yet) and make an edmx file and let EF put his classes.
You can use also ADO.NET with StoredProc, L2S, NHibernate or any other ORM you want.
Second Layer,Business Logic :
This is the layer where the logic should intervene (e.g., the name should be not null ).

Here is the list for Employee :

namespace BusinessLogic

{

    /// <summary>

    /// TODO : add IDataErroInfo in order to validate

    /// TODO : add the department

    /// </summary>

    public class Employee

    {

        public long ID;

        public string Name { get; set; }

        public Department Department{ get; set; }

        public void SaveNew()

        {

            //TODO : validate before save

            //TODO : automapper

            using (DAL_EF.testsEntities te = new DAL_EF.testsEntities("name=myconnection"))

            {

                DAL_EF.Employee e = new DAL_EF.Employee();

                e.Name = this.Name;

                //TODO : what if dept. deleted ?

                e.Department =te.Department.Where(d=>d.ID== this.Department.ID).FirstOrDefault();

                te.AddToEmployee(e);

                te.SaveChanges();

            }

        }

    }

    public class EmployeeList : List<Employee >

    {

        public void Load()

        {

            using (DAL_EF.testsEntities te = new DAL_EF.testsEntities("name=myconnection"))

            {

                foreach (var emp in te.Employee.Include("Department"))

                {

                    //TODO : use automapper

                    Employee e=new Employee() { ID = emp.ID, Name = emp.Name };

                    e.Department=new Department(){ ID=emp.Department.ID,Name=emp.Department.Name};

                    this.Add(e);

                }

            }

        }

    }

}

Please pay attention for Load procedure in EmployeeList : it uses the first layer, DAL, but it can use any ORM framework.

Third layer, ViewModel:

Because all the views for employees will need a select list for department ( either for sorting, creating or updating) I have put a base ViewModel class that retrieves the list of Departments:

namespace MVCAppEmp.Classes
{
    public class ViewModelEmployee
    {
        public static SelectList ListOfDepartments
        {
            get
            {
                //TODO : put this into cache to not load every time
                DepartmentList dl = new DepartmentList();
                dl.Load();
                return dl.SelectFromList(x => x.ID.ToString(), y => y.Name);
            }
        }
    }
    public class ViewModelEmployeeCreate : ViewModelEmployee
    {
        public Employee emp=new Employee();
    }
    public class ViewModelEmployeeList : ViewModelEmployee
    {
        public EmployeeList employees;
        public void Load()
        {
            employees = new EmployeeList();
            employees.Load();
        }
    }
}


Fourth Layer , MVC itself – view and controller :

<% =Html.ValidationSummary() %>
<% using (Html.BeginForm())
   { %>
    <h2>Create</h2>
    List of departments :
    <% =Html.DropDownList("DepartmentID", MVCAppEmp.Classes.ViewModelEmployee.ListOfDepartments)%>
    <%=Html.EditorFor(model => model.emp)%>
<input type="submit" value="create" />
<%} %>

and the controller

// POST: /Employee/Create
       // TODO: make a binding for employees
       [HttpPost]
       public ActionResult Create(Employee emp,long DepartmentID)
       {
           try
           {

               emp.Department = new Department() { ID = DepartmentID };
               emp.SaveNew();
               return RedirectToAction("Index");
           }
           catch(Exception ex)
           {
               ModelState.AddModelError("", ex.Message);
               return View(new ViewModelEmployeeCreate(){emp=emp});
           }
       }

So to answer the questions :

  1. One to many relationship – easy to do, most difficult to program
  2. Dropdownlist – easy – just transform a list into a select list, provided you have all data belongs
  3. How to use MVC with other ORM frameworks instead of Linq To Sql(L2S) ? Does not matter ! Just create your BusinessLogic and use that ORM!
  4. What is a ViewModel and what is his purpose ? To fill the data for GUI that BusinessLogic does not comply and transfer data to BusinessLogic. Reciprocally too!

You will find attached the application and the sql to create the database :

For MVC2  /MVC 3
http://msprogrammer.serviciipeweb.ro/wp-content/uploads/ASP.NETMVCORMandViewModels_135E/emp.zip

For MVC 4

http://msprogrammer.serviciipeweb.ro/wp-content/uploads/ASP.NETMVCORMandViewModels_135E/empMVC4.zip

What to do next, in order to familiarize yourself :
1. Make the update of the employees .Create view, save.
2. Filter the list of the employees on department. Search for all employees beginning with A that belong to IT department.
3. Search for TODO’s : there are part of other tutorials – but it will help you in MVC (and other!) development

This post have a continuation here:

http://msprogrammer.serviciipeweb.ro/2010/07/05/asp-net-mvc-and-dos-re-using-the-viewmodels/

ASP.NET MVC Extensions Methods

These are my extensions.

  1. Not display dropdownlist if the Select list is null 
    public static MvcHtmlString DropDownListNull(this HtmlHelper html, string Name, SelectList select, object htmlAttributes)
{

if (select == null)
return MvcHtmlString.Empty;
else
 return html.DropDownList(Name, select, htmlAttributes);

}
  2. Transform Enum to SelectList 
    #region enum to list
        private static List<KeyValuePair<long, string>> FromEnum(this Type e)
        {
            List<KeyValuePair<long, string>> kvp = new List<KeyValuePair<long, string>>();
            foreach (var s in Enum.GetValues(e))
            {
                kvp.Add(new KeyValuePair<long, string>((int)s, s.ToString()));
            }
            return kvp;
        }
        public static SelectList ToSelectList(this Type enumObj)
        {

            return new SelectList(enumObj.FromEnum(), "Key", "Value");

        }
        #endregion
  3. Transform Generic list into a SelectList 
                  #region Generic List to SelectItem
        public static SelectList SelectFromList<TItem>(this List<TItem> values,
            Expression<Func<TItem, string>> key, Expression<Func<TItem, string>> value)
        {
            var Key = key.Compile();
            var Value = value.Compile();
            List<KeyValuePair<string, string>> kvp = new List<KeyValuePair<string, string>>(values.Count);
            values.ForEach(item =>
                kvp.Add(new KeyValuePair<string, string>(Key.Invoke(item), Value.Invoke(item))));

            return new SelectList(kvp, "Key", "Value");

        }
        #endregion
  4. Display pager control 
    #region paged list control
        //after http://geekswithblogs.net/nuri/archive/2009/08/05/mvc-paged-list.aspx
        /// <summary>
        /// Shows a pager control - Creates a list of links that jump to each page
        /// </summary>
        /// <param name="page">The ViewPage instance this method executes on.</param>
        /// <param name="pagedList">A PagedList instance containing the data for the paged control</param>
        /// <param name="controllerName">Name of the controller.</param>
        /// <param name="actionName">Name of the action on the controller.</param>
        public static void ShowPagerControl(this ViewPage page, BasePagedListImplementation pagedList, string formatUrl)
        {
            HtmlTextWriter writer = new HtmlTextWriter(page.Response.Output);
            if (writer != null)
            {
                for (int pageNum = 1; pageNum <= pagedList.PageCount; pageNum++)
                {
                    bool samepage=(pageNum == pagedList.PageNumber);
                    if (!samepage)
                    {
                        writer.AddAttribute(HtmlTextWriterAttribute.Href,string.Format(formatUrl, pageNum));
                        writer.AddAttribute(HtmlTextWriterAttribute.Alt, "Page " + pageNum);
                        writer.RenderBeginTag(HtmlTextWriterTag.A);
                    }

                    writer.AddAttribute(HtmlTextWriterAttribute.Class,
                                        pageNum == pagedList.PageNumber ?
                                                            "pageLinkCurrent" :
                                                            "pageLink");

                    writer.RenderBeginTag(HtmlTextWriterTag.Span);
                    writer.Write(pageNum);
                    writer.RenderEndTag();

                    if (!samepage)
                    {
                        writer.RenderEndTag();
                    }
                    writer.Write("&nbsp;");
                }

                writer.Write("(");
                writer.Write(pagedList.TotalItemCount);
                writer.Write(" items in all)");
            }
        }

        #endregion

Other extensions available that are good :

  1. http://blog.wekeroad.com/blog/asp-net-mvc-list-helper-extension-method/
  2. http://www.hanselman.com/blog/ASPNETMVCSessionAtMix08TDDAndMvcMockHelpers.aspx
  3. http://helios.ca/2009/09/21/asp-net-mvc-extension-methods-of-urlhelper/
  4. http://blog.donnfelker.com/2010/02/25/asp-net-mvc-tempdata-extension-methods/
  5. http://inq.me/post/ASPNet-MVC-Extension-method-to-create-a-Security-Aware-HtmlActionLink.aspx

If you know more, please tell me

Andrei Ignat weekly software news(mostly .NET)

* indicates required

Please select all the ways you would like to hear from me:

You can unsubscribe at any time by clicking the link in the footer of our emails. For information about our privacy practices, please visit our website.

We use Mailchimp as our marketing platform. By clicking below to subscribe, you acknowledge that your information will be transferred to Mailchimp for processing. Learn more about Mailchimp's privacy practices here.