Tag: .NET 4.0

Backup application for Windows Azure

I have written( with Vunvulea Radu and Catalin Gheorghiu)  a backup application for Azure. It is made with MVC + Azure Tables.

It was made initially for a contest, but I intend to work further and progress .

You can play at http://backupapp.cloudapp.net/

The code source is on http://myazurebackup.codeplex.com/ – and you can add also your feedback at http://myazurebackup.codeplex.com/workitem/list/advanced

And , if you are a programmer, you can also improve it – just leave a comment on codeplex!

Thank you,

Optimizing EF4 and EDMGen2

Just reading (from Adrian Florea links ) how the EF4 could be optimized : http://www.codeproject.com/KB/database/PerfEntityFramework.aspx

First modification is to “Pre-generated Your View”. For this you must have .ssdl, .csdl, and .msl files – so you change the “Metadata Artifact Processing property to: Copy to Output Directory.”. Then you process the .ssdl, .csdl, and .msl with edmgen in order to can have the  views.

Until here , all ok.

But, in the next advice, is to keep “Metadata Artifact Processing property to: Embed in Output Assembly.”

One solution is to put “Metadata Artifact Processing property to: Copy to Output Directory.” , compile, put again “Metadata Artifact Processing property to: Embed in Output Assembly.” and compile again. But , if you change the edmx (add fields or tables ) you must redo the operation  – so you will have more things to do(if you remember)

A solution is to build them on the pre-build step .But how to generate the .ssdl, .csdl, and .msl files  ?

Edmgen2 , http://code.msdn.microsoft.com/EdmGen2 , to the rescue. Download, put into a lib folder under your solution folder and put a pre-build like this :

$(SolutionDir)lib\edmgen2\edmgen2 /FromEdmx $(ProjectDir)prod.edmx”

“%windir%\Microsoft.NET\Framework\v4.0.30319\EdmGen.exe” /mode:ViewGeneration /language:CSharp /nologo “/inssdl:prod.ssdl” “/incsdl:prod.csdl”   “/inmsl:prod.msl” “/outviews:$(ProjectDir)prod.Views.cs”

What you must change on your project?

1. IF you are on 64 bit, change Framework to Framework64

2. change the prod with your edmx name.

What is the performance?

Tested by loading a table with 301 rows by doing the steps :

1. open connection, load all table in objects(POCO), closing connection

2. open connection , find object with PK = 1, closing connection

3. open connection ,  loading 1 tables with 2 related (include ) , closing connection

The results are in milliseconds:

Without pre-compiled views

LoadTable LoadID LoadMultiple Total Time
579 800 172 1551
563 755 171 1489
559 754 169 1482
568 762 240 1570

With pre-compiled views:

LoadTable LoadID LoadMultiple Total Time
606 807 183 1596
509 706 177 1392
852 137 192 1181
530 733 221 1484
523 722 183 1428

The average / min / max results:

average max min
without 1523 1570 1482
with 1413.25 1596 1181

In the next picture the smaller the duration(milliseconds), is the better :

image

Conclusions:

1.  For the average and min the difference is 7%, respectively 20%. Please remember we are using only 3 queries.

For the max, it is vey curious : the with is more than without. The penalty is 1%, I think that is a measuring error ? – or maybe not. However , the penalty is small comparing with others.

2. Very curious, find after ID, with a table with 301 rows, took longer than loading the whole table.However, did not take into accound finding in the list the object( it is in memory also)

3. It may worth to add the pre-build step shown before to pre-compile views.

Links :

http://www.codeproject.com/KB/database/PerfEntityFramework.aspx

http://msdn.microsoft.com/en-us/library/bb896240.aspx

EF , automatic history of table and T4 files (TT files)

Usually the data of the tables should be tracking for who modified it.

Think about inserting/updating/deleting an employee :   you must know who did those actions and when. So you create another table, identically as structure, and you add another 3 fields , such as [ModifiedDate](when), [ModifiedBy](who), [ModifiedType] (what : insert, update, delete).

There are several methods to do it :

  1. from database :
  2. from programming code  – every time you modify an object, you remember to modify the history object with appropiate data.

The drawback with the database approach is that you can not retrieve who done the modifications ( usually the applications connect under a single account and have a roles table)

The drawback with the programming approach is that the programmer must REMEMBER doing so…If he does not(and does not wrote tests for history), you are stuck…

In the following I propose an automatically history – that maps convention over configuration in my template, but it is easy for you to modify.

The solution works with Entity Framework 4.0 and, for more easily spearation of concerns , with POCO generators.

Let’s say you have the following tables :

database diagram

As you see we have a Employee and a employee_history, an Department and Department_history

The conventions are:

the history table name = “object” table name  +  “_history” suffix

the history table fields = “object” table name  fields +[ModifiedDate], [ModifiedBy], [ModifiedType]

(if you change those conventions , please change the modelhistory.tt file)

If you want to see in action , please  download code history and do the following
1. create database tests
2. run history.sql
3. run project
4. if necessay, re-create the model1.edmx with the same name and replace the console application app.config with the new connection string

After works, please add any fields to department table  and to department_history table(same field names/type) .  Re-compile the application and modify the new field in department. You will see the modifications in the department_history table.

Ok,now how we do the magic :

We create two new tt file that points to the model.edmx .

The first one ModelHistory.tt , takes care of  creating the constructor for history entities by taking a parameter from the original entity :

public Department_History(Department original):this()
{
this.IDDepartment=original.IDDepartment;
this.Name=original.Name;
}

How it do this magic ? Simple : the ModelHistory.tt recognize the model and history in the name of tables:

</pre>
string inputFile = @"Model1.edmx";
string History = "_History";
<pre>

then it generate code for constructor :

	#>
		public <#=code.Escape(entity)#>():base()
		{
		}
		public <#=code.Escape(entity)#>(<#=NameEntityOriginal #> original):this()
		{
		<#
	foreach (EdmProperty edmProperty in entityOriginal.Properties.Where(p => p.TypeUsage.EdmType is PrimitiveType && p.DeclaringType == entityOriginal))
	{
		#>
				this.<#= code.Escape(edmProperty.Name) #>=original.<#= code.Escape(edmProperty.Name) #>;
		<#

	}
	#>
		}
	<#
</pre>

Ok, and then how to create the history entity ? I wish that the POCO template has had an event “Database saving” – but the only thing I can have is SaveChanges from the ObjectContext – so I create a new ObjectContext , derived from the default one that comes with the project, and creates a new history object :


public override int SaveChanges(SaveOptions options)
{
this.DetectChanges();
DateTime dtModified=DateTime.Now;
string UserModified=clsUser.UserName;
foreach (ObjectStateEntry ose in this.ObjectStateManager.GetObjectStateEntries(EntityState.Added | EntityState.Deleted | EntityState.Modified))
{

//could do this way too
//if (ose.Entity != null && ose.Entity.GetType() == typeof(...))
//{
//}
if (ose.Entity != null)
{
string NameType=ose.EntitySet.ElementType.Name;

switch(NameType)
{

case "Department":
var itemDepartment_History = new Department_History(ose.Entity as Department);
//if compile error here, that means you keep tracking
//of which modified with another properties
//please modify the tt accordingly
itemDepartment_History.ModifiedType= ose.State.ToString();
itemDepartment_History.ModifiedDate= dtModified;
itemDepartment_History.ModifiedBy= UserModified;
base.Department_History.AddObject(itemDepartment_History);
break;

case "Employee":
var itemEmployee_History = new Employee_History(ose.Entity as Employee);
//if compile error here, that means you keep tracking
//of which modified with another properties
//please modify the tt accordingly
itemEmployee_History.ModifiedType= ose.State.ToString();
itemEmployee_History.ModifiedDate= dtModified;
itemEmployee_History.ModifiedBy= UserModified;
base.Employee_History.AddObject(itemEmployee_History);
break;

}
}
}

return base.SaveChanges(options);
}

Now all is ready and I made a console application for testing manually (ok, should make a NUnit / MSTest / xUnit )

 using (var ctx = new testsEntitiesHistory())
            {
                var dep = new Department();
                dep.Name = "IT";
                ctx.Departments.AddObject(dep);
                ctx.SaveChanges();
                id = dep.IDDepartment;
            }
            using (var ctx = new testsEntitiesHistory())
            {
                var dep = ctx.Departments.Where(depart => depart.IDDepartment == id).FirstOrDefault();
                dep.Name = "Information tehnology";
                ctx.SaveChanges();
                //
            }
            using (var ctx = new testsEntitiesHistory())
            {
                var dep = ctx.Departments.Where(depart => depart.IDDepartment == id).FirstOrDefault();
                ctx.Departments.DeleteObject(dep);
                ctx.SaveChanges();

            }
            using (var ctx = new testsEntitiesHistory())
            {
                foreach (var dephist in ctx.Department_History)
                {
                    Console.WriteLine("Found {0} with state {1}", dephist.Name,dephist.ModifiedType);
                }
            }

And the output is :

history saving
automatically saving history department

Now you can add more tables to the edmx or change the fields – all is done automatically when compiling

If you want to see in action , please download code history

Update : for another way to do it( generating trigger and tables ) please see : http://msprogrammer.serviciipeweb.ro/2010/09/27/generating-history-trigger-with-ef-edmx-and-tt-files/

Caching in .NET

In every application you have some data that is more read-more, write-once or twice. For example you can have the list of Cities of a country, the list of Countries of the world or list of exchange currency. This data is modified rarely. Also, you can have data that is not very sensitive to be real-time , such as the list of invoices for the day.
In .NET 3.5 you have several options
1. ASP.NET caching – and implementing in other applications with HttpRuntime  ( even if MS says “The Cache class is not intended for use outside of ASP.NET applications”)

2. Enterprise caching block : hard to configure

3. memcached , velocity, sharedcache and other third party providers – that comes with their options and configuration

In .NET 4.0 you have a new kid : objectcache and , more important , an implementation : MemoryCache

http://msdn.microsoft.com/en-us/library/system.runtime.caching.memorycache%28v=VS.100%29.aspx

What is very good is that now you can cache in Memory what do you want – and apply easily to your Business Layer. More, the object is a singleton for the application – that is even better (see the test on the final of the post)

What it is missing is an easy implementation for List and an implementation to remove data after a defined time.

So I decided to do my implementation for that (ok, it is wrong to have both implementations in a single class – but you can separate easily )

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Runtime.Caching;

namespace CachingData{

    /// <summary>
    /// List<int> i = new List<int>() { 1, 10, 100 };
            //CacheData_List<List<int>, int> data = new CacheData_List<List<int>, int>(2);
            //data.Add(i);
            //Assert.AreEqual(3, data.Items().Count, "must be 3");
            //data = new CacheData_List<List<int>, int>();
            //Assert.AreEqual(3, data.Items().Count, "must be 3");
            //Assert.IsTrue(data.ContainsData, "must have data");
            //Thread.Sleep(1000 * 3);
            //Assert.IsFalse(data.ContainsData, "must not have data");
    /// </summary>
    /// <typeparam name="T">and generic ILIST </typeparam>
    /// <typeparam name="U"></typeparam>
    public class CacheData_List<T, U>
        where T:class,IList<U>, new()

    {
        /// <summary>
        /// use this for adding in cache
        /// </summary>
        public event EventHandler RemovedCacheItem;

        private System.Timers.Timer timerEvent;
        private MemoryCache buffer;
        private int TimeToLiveSeconds;

        private DateTimeOffset dto;
        private string Key;
        /// <summary>
        /// default constructor - cache 600 seconds = 10 minutes
        /// </summary>
        public CacheData_List()
            : this(600)
        {
        }
        /// <summary>
        /// constructor cache the mentioned TimeSeconds time
        /// </summary>
        /// <param name="TimeSeconds">value of time for cache</param>
        public CacheData_List(int TimeSeconds)
        {
            TimeToLiveSeconds = TimeSeconds;
            timerEvent=new System.Timers.Timer(TimeToLiveSeconds * 1000);
            timerEvent.AutoReset = true;
            timerEvent.Elapsed += new System.Timers.ElapsedEventHandler(timerEvent_Elapsed);
            dto = new DateTimeOffset(DateTime.UtcNow.AddSeconds(TimeToLiveSeconds));
            Key = typeof(T).ToString();
            buffer = MemoryCache.Default;
        }

        void timerEvent_Elapsed(object sender, System.Timers.ElapsedEventArgs e)
        {
            if (RemovedCacheItem != null)
            {
                RemovedCacheItem(this, EventArgs.Empty);
            }
        }
        /// <summary>
        /// remove item from cache
        /// </summary>
        public void Remove()
        {
            if (buffer.Contains(Key))
            {
                buffer.Remove(Key);

            }
            dto=new DateTimeOffset(DateTime.UtcNow.AddSeconds(TimeToLiveSeconds));
        }
        /// <summary>
        /// add multiple items to cache
        /// </summary>
        /// <param name="items">items to add to the cache</param>
        public void Add(T items)
        {

            if (buffer.Contains(Key))
            {
                T data = Items();
                foreach (var t in data)
                {
                    items.Add(t);
                }
                buffer.Remove(Key);
            }
            buffer.Add(Key, items, dto);
        }
        /// <summary>
        /// add a item to the IList of the cache
        /// </summary>
        /// <param name="item">an item to add</param>
        public void AddItem(U item)
        {

            T data=new T();
            if (buffer.Contains(Key))
            {
                data = buffer.Get(Key) as T;
                buffer.Remove(Key);
            }

            data.Add(item);
            buffer.Add(Key, data,dto);
        }
        /// <summary>
        /// usefull if you do not intercept the removed event
        /// </summary>
        public bool ContainsData
        {
            get
            {
                return buffer.Contains(Key);
            }

        }
        /// <summary>
        /// retrieve items
        /// </summary>
        /// <returns></returns>
        public T Items()
        {
            if (!buffer.Contains(Key))
                return null;

            return buffer.Get(Key) as T;
        }
    }
}

Please note that the test for usage is in the summary :

</pre>
List i = new List() { 1, 10, 100 };
CacheData_List <List<int>, int> data = new CacheData_List<List<int>, int>(2);
data.Add(i);
Assert.AreEqual(3, data.Items().Count, "must be 3");
data = new CacheData_List<List<int>, int>();
Assert.AreEqual(3, data.Items().Count, "must be 3");
Assert.IsTrue(data.ContainsData, "must have data");
Thread.Sleep(1000 * 3);
Assert.IsFalse(data.ContainsData, "must not have data");

You can download the file from

http://msprogrammer.serviciipeweb.ro/wp-content/uploads/2010/05/CachingData.zip

(Last Note : for synchonization maybe it was better to lock on readerwriterlockslim