Category: Entity Framework

Circular references on .NET , Entity Framework and WebAPI

Imagine having a class Department( ID, Name) and Employee ( ID , Name, IDDepartment) . You want to return in the WebAPI the Departments with the Employee . It is simple to wrote this code:


[HttpGet]
public IEnumerable<Department> Get([FromServices] MyTestDatabaseContext context)
{
var departments = context.Department.Include(iterator=>iterator.Employee).ToArray();
return departments;
}

But the problem is with circular references when serializing  : The Department has a list of  Employees  that have a Department that have a list of Employees that …

 

 

Solution 1 :  Delete/Comment from the EF generated code what you do not need ( in this case , the Department reference that the Employee has)


public partial class Employee
{
public int Idemployee { get; set; }
public string NameEmployee { get; set; }
public int Iddepartment { get; set; }

//public Department IddepartmentNavigation { get; set; }
}

Solution 2 : Nullify the reference


[HttpGet]
public IEnumerable<Department> Get([FromServices] MyTestDatabaseContext context)
{
var departments = context.Department.Include(iterator=>iterator.Employee).ToArray();
foreach (var dep in departments)
{
foreach (var emp in dep.Employee)
{
emp.IddepartmentNavigation = null;
}
}
return departments;
}

( maybe this should be add to the partial class of the employee ?)

 

 

Solution 3: Handle circular references in serialization


services
.AddMvc()
.AddJsonOptions(opt =>
{
opt.SerializerSettings.PreserveReferencesHandling = Newtonsoft.Json.PreserveReferencesHandling.Objects;
opt.SerializerSettings.ReferenceLoopHandling = Newtonsoft.Json.ReferenceLoopHandling.Ignore;
});

Solution 4: Make a read DDD design. Read https://siderite.dev/2018/08/client-models-vs-data-transfer-objects.html

Maintaining history of data

 

What is this about?

I tend to quote Fowler: “Usually when we see properties on a class, they represent questions we can ask of an object now. However there are times when we don’t want to just ask questions about a property of an object now, we also want to ask these questions about some point in the past when things may have changed. “

For example , we want to track every change that occurs at an Employee ( changing name, salary) and see the previous versions( what was the salary 2 years ago ?)

So what are the software solutions to do an audit log ? First, we should have a history table for each table that we want to maintain an audit log. E.g. Employee => EmployeeHstory

Solution 1 : Coding / Event Sourcing

That means code should maintain history himself – i.e. adding to the history table every modifications.

For Entity Framework this can be done easy, by overiding SaveChanges.

Demo at https://youtu.be/YZ9GDjFApns 

Solution 2 : Table Triggers

This can be done from database . There are triggers for insert , update, delete – and those triggers insert the data in the history table

Demo at https://youtu.be/9xsvCJ-1f7Q

Solution 3, Temporal Tables, Microsot  Sql Server 2016 :

Documentation at https://docs.microsoft.com/en-us/sql/relational-databases/tables/temporal-tables 

Limitations at https://docs.microsoft.com/en-us/sql/relational-databases/tables/temporal-table-considerations-and-limitations 

Demo at https://youtu.be/1Q3DmXJI5h8

EF Record and play – conclusions

 

Part 1 : What is EF record and play : http://msprogrammer.serviciipeweb.ro/2014/11/29/entity-framework-6-record-and-play-1-of-5/ 

Part 2: EF Record and play use: Testing : http://msprogrammer.serviciipeweb.ro/2014/12/08/entity-framework-6-record-and-play-use-unit-testing-part-2-of-5/

Part 3: EF Record and play use: Make demo: http://msprogrammer.serviciipeweb.ro/2014/12/14/entity-framework-6-record-and-play-use-making-demos-part-3-of-5/ 

Part 4: EF Record and play use: Record user Sql when a bug occurs: http://msprogrammer.serviciipeweb.ro/2014/12/26/ef-record-and-play-use-recording-user-sql-when-a-bug-occurred-part-4-of-5/

Part 5: EF record and play: conclusions: http://msprogrammer.serviciipeweb.ro/2015/01/05/ef-record-and-play-conclusions/

 

 

I have show to you how easy you can record and play again Entity Framework 6 actions. Also, some possible uses as testing, making demos and record user Sql.

One possible use that can be done is Profiling  – analyzing sql – for example, just like MiniProfiler or EFProfiler does. Read this article about profiling : http://msdn.microsoft.com/en-us/magazine/gg490349.aspx

I hope you enjoy this piece of code.

Source code is available at https://github.com/ignatandrei/EFRecordAndPlay/wiki/
There is also a NuGet package at https://www.nuget.org/packages/EFRecordAndPlay/

Video at https://www.youtube.com/playlist?list=PL4aSKgR4yk4Mi1eLKArsgoqQRluxXv2-Y

EF Record and play use: Recording user sql when a bug occurred (part 4 of 5)

 

Part 1 : What is EF record and play : http://msprogrammer.serviciipeweb.ro/2014/11/29/entity-framework-6-record-and-play-1-of-5/ 

Part 2: EF Record and play use: Testing : http://msprogrammer.serviciipeweb.ro/2014/12/08/entity-framework-6-record-and-play-use-unit-testing-part-2-of-5/

Part 3: EF Record and play use: Make demo: http://msprogrammer.serviciipeweb.ro/2014/12/14/entity-framework-6-record-and-play-use-making-demos-part-3-of-5/ 

Part 4: EF Record and play use: Record user Sql when a bug occurs: http://msprogrammer.serviciipeweb.ro/2014/12/26/ef-record-and-play-use-recording-user-sql-when-a-bug-occurred-part-4-of-5/

Part 5: EF record and play: conclusions: http://msprogrammer.serviciipeweb.ro/2015/01/05/ef-record-and-play-conclusions/

 

In this video tutorial I will show how to see the sql when a database bug occurs.

Let’s suppose that a department should have the name not null – and , by mistake, we have not prevented the registering of a null in the name of the department

We put those lines

 
 #region set record EF
            record = new InterceptionRecordOrPlay(@"DemoEmployeeAndDepartment.zip", ModeInterception.Record);

            DbInterception.Add(record);
            #endregion 

We register the null name department and  in the DemoEmployeeAndDepartment.zip we see the error

<?xml version="1.0"?>
<InterceptionData xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
  <CommandText>INSERT [Department]([NameDepartment])
VALUES (NULL)
; SELECT [Id]
FROM [Department]
WHERE [Id] = CAST (@@IDENTITY AS int)</CommandText>
  <NonQuery>0</NonQuery>
  <ThrownExc>
    <ExceptionType>System.Data.SqlServerCe.SqlCeException</ExceptionType>
    <ExceptionMessage>The column cannot contain null values. [ Column name = NameDepartment,Table name = Department ]</ExceptionMessage>
  </ThrownExc>
</InterceptionData>

Source code is available at https://github.com/ignatandrei/EFRecordAndPlay/wiki/
There is also a NuGet package at https://www.nuget.org/packages/EFRecordAndPlay/

Entity Framework 6 Record and play use : Making demos ( part 3 of 5)

 

Part 1 : What is EF record and play : http://msprogrammer.serviciipeweb.ro/2014/11/29/entity-framework-6-record-and-play-1-of-5/ 

Part 2: EF Record and play use: Testing : http://msprogrammer.serviciipeweb.ro/2014/12/08/entity-framework-6-record-and-play-use-unit-testing-part-2-of-5/

Part 3: EF Record and play use: Make demo: http://msprogrammer.serviciipeweb.ro/2014/12/14/entity-framework-6-record-and-play-use-making-demos-part-3-of-5/ 

Part 4: EF Record and play use: Record user Sql when a bug occurs: http://msprogrammer.serviciipeweb.ro/2014/12/26/ef-record-and-play-use-recording-user-sql-when-a-bug-occurred-part-4-of-5/

Part 5: EF record and play: conclusions: http://msprogrammer.serviciipeweb.ro/2015/01/05/ef-record-and-play-conclusions/

 

 

Let’s assume you want to register a demo for a application ( Desktop or Web).

You do arrange the database and you make some repetition before. However, the database must be re-arranged the next time you will make the demo.

For this case you can use EF 6 Record and play – it will make a .zip file with containing resultsets and you can make indefinitely the repetition of the demo – without the real database.

 

Let’s see in action here

 

 

The code is simple

 Database.SetInitializer<ContextForDatabase>(null);
            #region set record EF
            record = new InterceptionRecordOrPlay(@"DemoEmployeeAndDepartment.zip", ModeInterception.Play);

            DbInterception.Add(record);
            #endregion

Source code is available at  https://github.com/ignatandrei/EFRecordAndPlay/wiki/

There is also a NuGet package at https://www.nuget.org/packages/EFRecordAndPlay/

Entity Framework 6 Record and play use : Unit Testing ( part 2 of 5)

 

Part 1 : What is EF record and play : http://msprogrammer.serviciipeweb.ro/2014/11/29/entity-framework-6-record-and-play-1-of-5/ 

Part 2: EF Record and play use: Testing : http://msprogrammer.serviciipeweb.ro/2014/12/08/entity-framework-6-record-and-play-use-unit-testing-part-2-of-5/

Part 3: EF Record and play use: Make demo: http://msprogrammer.serviciipeweb.ro/2014/12/14/entity-framework-6-record-and-play-use-making-demos-part-3-of-5/ 

Part 4: EF Record and play use: Record user Sql when a bug occurs: http://msprogrammer.serviciipeweb.ro/2014/12/26/ef-record-and-play-use-recording-user-sql-when-a-bug-occurred-part-4-of-5/

Part 5: EF record and play: conclusions: http://msprogrammer.serviciipeweb.ro/2015/01/05/ef-record-and-play-conclusions/

 

Let’s suppose that we have a program that have Departments and Employees.

And we want to make sure that, when we add an employee, the department must exists.

We can ensure this from database ( by foreign key) but we can pro-actively search for the department and throw a more meaningful validation .

More, I like more validation than errors.

So, let’s suppose that in the Validation for the Employee we must check in the database for the IdDepartment to see if there is such a department.

How could we make a test for that runs without a database ?

With some trick:  we first Record with a database  – then we can Play the file – and we do not need anymore the database. The test is self contained. 

 

Let’s see in action here

 

Database.SetInitializer<ContextForDatabase>(null);
            #region set record EF
            var record = new InterceptionRecordOrPlay(@"VerifyIValidatableWorks.zip", ModeInterception.Play);

            DbInterception.Add(record);
            #endregion
            var e= new Employee();
            e.ValidateEmployee = true;
            e.IDDepartment = 60000;
            var err= e.Validate(null).ToArray();
            Assert.IsNotNull(err);
            Assert.AreEqual(1, err.Length);

Source code is available at  https://github.com/ignatandrei/EFRecordAndPlay/wiki/

 

There is also a NuGet package at https://www.nuget.org/packages/EFRecordAndPlay/

Entity Framework 6 Record and play – 1 of 5

 

Part 1 : What is EF record and play : http://msprogrammer.serviciipeweb.ro/2014/11/29/entity-framework-6-record-and-play-1-of-5/ 

Part 2: EF Record and play use: Testing : http://msprogrammer.serviciipeweb.ro/2014/12/08/entity-framework-6-record-and-play-use-unit-testing-part-2-of-5/

Part 3: EF Record and play use: Make demo: http://msprogrammer.serviciipeweb.ro/2014/12/14/entity-framework-6-record-and-play-use-making-demos-part-3-of-5/ 

Part 4: EF Record and play use: Record user Sql when a bug occurs: http://msprogrammer.serviciipeweb.ro/2014/12/26/ef-record-and-play-use-recording-user-sql-when-a-bug-occurred-part-4-of-5/

Part 5: EF record and play: conclusions: http://msprogrammer.serviciipeweb.ro/2015/01/05/ef-record-and-play-conclusions/

 

Entity Framework Record And Play

With this helper you can record and then play the actions in Entity Framework(>= 6).

For recording actions just reference the dll and use

DbInterception.Add(new InterceptionRecordOrPlay(@"a.zip", ModeInterception.Record)); 

(Note: For ASP.NET you will use Server.MapPath("~/a folder that supports write/namefile.zip")

For replay use

DbInterception.Add(new InterceptionRecordOrPlay(@"a.zip", ModeInterception.Play));

This can be use for

  1. Unit Testing
  2. Making demos
  3. Recording user actions when a bug occurred

 

Source code is available at  https://github.com/ignatandrei/EFRecordAndPlay/wiki/

and has also a test 😉

There is also a NuGet package at https://www.nuget.org/packages/EFRecordAndPlay/ 

 

image

Generating history trigger with EF , edmx and TT files

I have wrote in an older post ( http://msprogrammer.serviciipeweb.ro/2010/06/28/ef-automatic-history-of-table-and-t4-files-tt-files/ ) how to generate history code for tables . The easy solution was to create a tt file that track for the ObjectContext the SaveChanges for each table that has a “history” in name. the limitation is that , when you raise an sql command such as “update table ” you must load from database a lot of rows for this….

Now I want to show the same thing, but generating triggers in database for that ! I start also from edmx file and with a template stealed from http://forums.asp.net/p/1599616/4083198.aspx ( to have type of fields in the database ) and another stealed and modified from StackOverflow(to generate trigger for after insert , update, delete) I manage to have a solution to generate sql tables and trigger code.

When is that good ?At the beginning stages of a project when the table structure changes by adding a new parameter.

Sample code generated for table Notes(ID, TextNote, PersonID)

print 'Create table Notes_History ';
Create Table Notes_History(
		ID_Notes_History BIGINT IDENTITY NOT NULL
		,History_Action varchar(50)
		,History_From varchar(100) default HOST_NAME()
		,History_User varchar(50) default SYSTEM_USER
		,History_Date varchar(50) default  getdate()
	   ,Id  int   NOT NULL
	   ,Textnote  nvarchar (255)  NULL
	   ,PersonId  int   NULL
	)
print 'end Create table Notes_History ';
GO
print 'create trigger for Notes'
GO
CREATE TRIGGER dbo.TR_IUP_Notes
   ON  Notes
   AFTER INSERT, UPDATE, DELETE
AS
BEGIN
    SET NOCOUNT ON;
	DECLARE @Ins int

DECLARE @Del int

SELECT @Ins = Count(*) FROM inserted

SELECT @Del = Count(*) FROM deleted
if(@Ins  + @Del = 0)
	return;

declare @operation varchar(50)
	set @operation ='update';
	if(@ins < @del)
		set @operation ='delete';
	if(@ins > @del)
		set @operation ='insert';

	if(@ins <= @del)
	begin
		INSERT INTO Notes_History(History_Action ,Id,Textnote,PersonId)
			select @operation ,Id,Textnote,PersonId from deleted
	end
    else
	begin
    INSERT INTO Notes_History(History_Action ,Id,Textnote,PersonId)
			select @operation ,Id,Textnote,PersonId from inserted
	end
END

The drawback of the code : the user that executes is not always the logged sql server user…
Anyway, what you have to do to use this automatically generated history/audit for tables ?

download historysql and modify

string inputFile = @”Model1.edmx”;

from historysql.tt to your edmx name. Then take the generated code and execute in sql server.

Asp.NET MVC and DOS – re-using the ViewModels

(Please read first : http://msprogrammer.serviciipeweb.ro/2010/03/29/asp-net-mvc-orm-and-viewmodels/ )

One of the biggest challenges in programming was write once- GUI everywhere ( Ok, ORM impedance mismatch is another story)

I mean by that re-using the logic from an application in another application. ASP.NET MVC , with the commitment to strongly viewmodels, make me think that it will be now easier to transfer the viewmodels to an console application.

Let’s see the usual Employee-Department and creation.

First the Database :

Then the ViewModel for creating an employeeand for list of employees


public class ViewModelEmployee
 {
 public static DepartmentList ListOfDepartments
 {
 get
 {
 //TODO : put this into cache to not load every time
 DepartmentList dl = new DepartmentList();
 dl.Load();
 return dl;
 }
 }
 }
 public class ViewModelEmployeeCreate : ViewModelEmployee
 {
 public Employee emp = new Employee();

 public static void SaveNew(Employee emp)
 {
 emp.SaveNew();
 }
 }
 public class ViewModelEmployeeList : ViewModelEmployee
 {
 public EmployeeList employees;
 public void Load()
 {
 employees = new EmployeeList();
 employees.Load();
 }
 }

And now the magic :

ASP.NET MVC DOS

[HttpPost]
 public ActionResult Create(Employee emp,long DepartmentID)
 {
 try
 {

 emp.Department = new Department() { ID = DepartmentID };
 ViewModelEmployeeCreate.SaveNew(emp);

 return RedirectToAction("Index");
 }
 catch(Exception ex)
 {
 ModelState.AddModelError("", ex.Message);
 return View(new ViewModelEmployeeCreate(){emp=emp});
 }
 }


Console.WriteLine("choose department");
 var listdep=ViewModelEmployee.ListOfDepartments;
 foreach (var item in listdep)
 {
 Console.WriteLine(item.ID + ")" + item.Name);

 }
 string s=Console.ReadLine();
 long DepartmentID;
 if (!long.TryParse(s, out DepartmentID))
 {
 Console.WriteLine("exit : not a long :" + s);
 return;
 }
 if (!listdep.Exists(item => item.ID == DepartmentID))
 {
 Console.WriteLine("not a valid id:" + s);
 return;
 }
 Employee emp = new Employee();
 emp.Department = new Department() { ID = DepartmentID };
 Console.Write("employee name ?");
 emp.Name= Console.ReadLine();
 ViewModelEmployeeCreate.SaveNew(emp);

Now for listing employees:

ASP.NET MVC DOS
public ActionResult Index()
        {
            ViewModelEmployeeList vmel = new ViewModelEmployeeList();
            vmel.Load();
            return View(vmel);
        }

  ViewModelEmployeeList vmel = new ViewModelEmployeeList();
            vmel.Load();
            foreach (var item in vmel.employees)
            {
                Console.WriteLine(item.Name);
            }

As you can see , the codes are really similar ( although the console application is filled with first verification and the MVC is not )

Please download the application from testaddropdownlist

To install : run the emp.sql file, change in the app.config/web.config the connection string to point to the real database/sql server.

Summary : This simple application shows how to re-use ViewModels from an ASP.NET MVC and a DOS Console Application

Homework : Add  WindowsForms application and do the same.

EF , automatic history of table and T4 files (TT files)

Usually the data of the tables should be tracking for who modified it.

Think about inserting/updating/deleting an employee :   you must know who did those actions and when. So you create another table, identically as structure, and you add another 3 fields , such as [ModifiedDate](when), [ModifiedBy](who), [ModifiedType] (what : insert, update, delete).

There are several methods to do it :

  1. from database :
  2. from programming code  – every time you modify an object, you remember to modify the history object with appropiate data.

The drawback with the database approach is that you can not retrieve who done the modifications ( usually the applications connect under a single account and have a roles table)

The drawback with the programming approach is that the programmer must REMEMBER doing so…If he does not(and does not wrote tests for history), you are stuck…

In the following I propose an automatically history – that maps convention over configuration in my template, but it is easy for you to modify.

The solution works with Entity Framework 4.0 and, for more easily spearation of concerns , with POCO generators.

Let’s say you have the following tables :

database diagram

As you see we have a Employee and a employee_history, an Department and Department_history

The conventions are:

the history table name = “object” table name  +  “_history” suffix

the history table fields = “object” table name  fields +[ModifiedDate], [ModifiedBy], [ModifiedType]

(if you change those conventions , please change the modelhistory.tt file)

If you want to see in action , please  download code history and do the following
1. create database tests
2. run history.sql
3. run project
4. if necessay, re-create the model1.edmx with the same name and replace the console application app.config with the new connection string

After works, please add any fields to department table  and to department_history table(same field names/type) .  Re-compile the application and modify the new field in department. You will see the modifications in the department_history table.

Ok,now how we do the magic :

We create two new tt file that points to the model.edmx .

The first one ModelHistory.tt , takes care of  creating the constructor for history entities by taking a parameter from the original entity :

public Department_History(Department original):this()
{
this.IDDepartment=original.IDDepartment;
this.Name=original.Name;
}

How it do this magic ? Simple : the ModelHistory.tt recognize the model and history in the name of tables:

</pre>
string inputFile = @"Model1.edmx";
string History = "_History";
<pre>

then it generate code for constructor :

	#>
		public <#=code.Escape(entity)#>():base()
		{
		}
		public <#=code.Escape(entity)#>(<#=NameEntityOriginal #> original):this()
		{
		<#
	foreach (EdmProperty edmProperty in entityOriginal.Properties.Where(p => p.TypeUsage.EdmType is PrimitiveType && p.DeclaringType == entityOriginal))
	{
		#>
				this.<#= code.Escape(edmProperty.Name) #>=original.<#= code.Escape(edmProperty.Name) #>;
		<#

	}
	#>
		}
	<#
</pre>

Ok, and then how to create the history entity ? I wish that the POCO template has had an event “Database saving” – but the only thing I can have is SaveChanges from the ObjectContext – so I create a new ObjectContext , derived from the default one that comes with the project, and creates a new history object :


public override int SaveChanges(SaveOptions options)
{
this.DetectChanges();
DateTime dtModified=DateTime.Now;
string UserModified=clsUser.UserName;
foreach (ObjectStateEntry ose in this.ObjectStateManager.GetObjectStateEntries(EntityState.Added | EntityState.Deleted | EntityState.Modified))
{

//could do this way too
//if (ose.Entity != null && ose.Entity.GetType() == typeof(...))
//{
//}
if (ose.Entity != null)
{
string NameType=ose.EntitySet.ElementType.Name;

switch(NameType)
{

case "Department":
var itemDepartment_History = new Department_History(ose.Entity as Department);
//if compile error here, that means you keep tracking
//of which modified with another properties
//please modify the tt accordingly
itemDepartment_History.ModifiedType= ose.State.ToString();
itemDepartment_History.ModifiedDate= dtModified;
itemDepartment_History.ModifiedBy= UserModified;
base.Department_History.AddObject(itemDepartment_History);
break;

case "Employee":
var itemEmployee_History = new Employee_History(ose.Entity as Employee);
//if compile error here, that means you keep tracking
//of which modified with another properties
//please modify the tt accordingly
itemEmployee_History.ModifiedType= ose.State.ToString();
itemEmployee_History.ModifiedDate= dtModified;
itemEmployee_History.ModifiedBy= UserModified;
base.Employee_History.AddObject(itemEmployee_History);
break;

}
}
}

return base.SaveChanges(options);
}

Now all is ready and I made a console application for testing manually (ok, should make a NUnit / MSTest / xUnit )

 using (var ctx = new testsEntitiesHistory())
            {
                var dep = new Department();
                dep.Name = "IT";
                ctx.Departments.AddObject(dep);
                ctx.SaveChanges();
                id = dep.IDDepartment;
            }
            using (var ctx = new testsEntitiesHistory())
            {
                var dep = ctx.Departments.Where(depart => depart.IDDepartment == id).FirstOrDefault();
                dep.Name = "Information tehnology";
                ctx.SaveChanges();
                //
            }
            using (var ctx = new testsEntitiesHistory())
            {
                var dep = ctx.Departments.Where(depart => depart.IDDepartment == id).FirstOrDefault();
                ctx.Departments.DeleteObject(dep);
                ctx.SaveChanges();

            }
            using (var ctx = new testsEntitiesHistory())
            {
                foreach (var dephist in ctx.Department_History)
                {
                    Console.WriteLine("Found {0} with state {1}", dephist.Name,dephist.ModifiedType);
                }
            }

And the output is :

history saving
automatically saving history department

Now you can add more tables to the edmx or change the fields – all is done automatically when compiling

If you want to see in action , please download code history

Update : for another way to do it( generating trigger and tables ) please see : http://msprogrammer.serviciipeweb.ro/2010/09/27/generating-history-trigger-with-ef-edmx-and-tt-files/

Andrei Ignat weekly software news(mostly .NET)

* indicates required

Please select all the ways you would like to hear from me:

You can unsubscribe at any time by clicking the link in the footer of our emails. For information about our privacy practices, please visit our website.

We use Mailchimp as our marketing platform. By clicking below to subscribe, you acknowledge that your information will be transferred to Mailchimp for processing. Learn more about Mailchimp's privacy practices here.