NuGet Package Manager

Microsoft – Visual Studio Platform Team (Microsoft Corporation)

Productivity Power Tools

Team Foundation Server Power Tools December 2011

Web Standards Update for Microsoft Visual Studio 2010 SP1

PowerCommands for Visual Studio 2010

Web Essentials

SQL Server Compact Toolbox

EF 4.x POCO Entity Generator for C#

EF 4.x DbContext Generator for C#

Portable Library Tools 2 Beta

Visual Studio 2010 SP1 SDK

GhostDoc

Image Optimizer

Razor Generator

ReverseEquality

Windows Azure ASP.NET Providers Sample : http://code.msdn.microsoft.com/windowsazure/Windows-Azure-ASPNET-03d5dc14 – utils for fast membership and roles. Small problem on local.

More samples here: http://code.msdn.microsoft.com/windowsazure/

And that will be all , after reading the documentation and understanding the concepts ( for example,if you understand the session problem in azure, then you will find a Session provider in the samples and use it)

I have the opportunity to borrow a Kinect hardware  – to make an application( I will make a simple anti-theft application).

The steps for developing in Kinect are:

• Download the SDK from http://www.microsoft.com/en-us/kinectforwindows/develop/overview.aspx – it contains the drivers also
• Connect Kinect to USB , plug in the socket
• Run the Kinect Explorer or Kinect Shape Game from Kinect SDK Sample Browser installed at 1.  Ensure it works.
• Read the Kinect Explorer source –it is SO clear!
• Read documentation – finally, you should RTFM

Optional resources:

1. http://kinectcontrib.codeplex.com/ – Visual Studio template for Skeleton, Audio, Video. Simple example that works ( simpler than Kinect Explorer !)
2. http://c4fkinect.codeplex.com/ – added methods to Kinect.
3. http://kinecttoolbox.codeplex.com/ – detection of gesture.

Example 1: Integrating saving image in Kinect Explorer when a skeleton is detected

Add reference to optional resource 2( either download , either via Nuget)

Search for  KinectAllFramesReady in KinectSkeletonViewer.xaml.cs and put this code

						var takePic = this.skeletonData.Count(item => item.TrackingState == SkeletonTrackingState.Tracked) > 0;

						if (takePic)
						{
							DateTime imgDate = DateTime.Now;
							string imageName = "andrei" + imgDate.ToString("yyyyMMdd_HHmmss") + ".jpg";
							if (!File.Exists(imageName))
							{
								using (var image = e.OpenColorImageFrame())
								{
									if (image != null && this.skeletonData.Length > 0 && this.skeletonData.Count(item => item.TrackingState != SkeletonTrackingState.NotTracked) > 0)
										if (image != null && takePic)
										{
											var x = image.ToBitmapSource();
											var b = Save(x, ImageFormat.Jpeg);

											var t = Task.Factory.StartNew(

												(img) =>
												{

													imgDate = DateTime.Now;
													imageName = "andrei" + imgDate.ToString("yyyyMMdd_HHmmss") + ".jpg";
													if (File.Exists(imageName))
														return;
													byte[] i = img as byte[];
													if (i != null)
													{
														File.WriteAllBytes(imageName, i);
													};
												}, b);
											//t.Start();
										}
								}
							}
					}

Example 2 : Detecting circle by right hand in Kinect Explorer when a skeleton is detected

Add reference to optional resource 3( either download , either via Nuget)

Download circleKB.save file and put in your project. Ensure “Copy to output directory” is “copy always/copy if newer”

Add a variable named

TemplatedGestureDetector circleGestureRecognizer;

In KinectSkeletonViewer.xaml.cs in constructor put

			using (Stream recordStream = File.Open("circleKB.save", FileMode.Open))
			{
				circleGestureRecognizer = new TemplatedGestureDetector("Circle", recordStream);
				circleGestureRecognizer.OnGestureDetected += new Action<string>(circleGestureRecognizer_OnGestureDetected);
//TODO : implement circleGestureRecognizer_OnGestureDetected : void circleGestureRecognizer_OnGestureDetected(string obj)
			}

Search for KinectAllFramesReady in KinectSkeletonViewer.xaml.cs and put this code

foreach (Joint joint in skeleton.Joints)//existing code
							{
								Point mappedPoint = this.GetPosition2DLocation(depthImageFrame, joint.Position);//existing code
								jointMapping[joint.JointType] = new JointMapping//existing code
									{
										Joint = joint,
										MappedPoint = mappedPoint
									};
//new code from here:
									 if ((joint.TrackingState == JointTrackingState.Tracked) && (joint.JointType == JointType.HandRight))
									{
											circleGestureRecognizer.Add(joint.Position, Kinect);
									}

							}

That will be all…
If you have developed with Kinect, please share your sources as comments.

It is simple , once you have SqlLite Manager add-on for FF. ( add-on, source)

Just run the following query ( for the last 7 days bookmarks) by selecting “places.sqllite”:

SELECT
"<p><a target='_blank' href='" || moz_places.url  || "'>" || moz_bookmarks.title || "</a></p>"

FROM moz_bookmarks Inner JOIN moz_places WHERE moz_bookmarks.fk = moz_places.id AND moz_bookmarks.title != "null"
and last_visit_date BETWEEN  strftime('%s',date('now','-7 day'))*1000000 and strftime('%s',date('now'))*1000000
and dateAdded BETWEEN  strftime('%s',date('now','-7 day'))*1000000 and strftime('%s',date('now'))*1000000

(OK, I must know why I multiply by 1000000  – but,hey – it works ….)

I have re-installed the PC and I have been taken notice of what tools I have on the system now:

1. Magic Disc – can mount .iso files ( for  2 )
2. Visual Studio – must have for a easy developing path
3. Sql Server 2008 – primary database for me
4. 7-zip  - archiver
5. Firefox ( plus addons, see below)
6. Smtp4dev – to see messages
7. AspNetMVC3ToolsUpdateSetup
8. FreeCommander -  dual panels for windows explorer
9. Foxit reader – pdf viewer
10. Notepad ++ ( with hex addon)
11. Winmerge – files/ folders difference
12. Clean project – archive solution
13. SqlSearch from RedGate – fast search after names
14. SSMS tools – record every operation you do in sql server
15. EntityFramework41  – code first development
16. XUnit – automated testing + samples  to do BDD style
17. SSCERuntime_x86-ENU –SqlCompact provider. Works with EF4.1
18. SqlCe40Toolbox – SqlCompact viewer
19. NuGet.Tools.vsix – Nuget is awesome!
20. ImgBurn  – burn cd-s
21. consolas font  - see  1 and l ?
22. LogParser  – never know when you need to parse some files
23. Moq  – mocking tool
24. PreviewHandlerPack  -see c# code in preview window
25. Regulator and Regulazy – regular expression helpers
26. StringTemplate.NET  – templating generator. Maybe replaced by razor ?
27. NLog –  logging tool. Log4Net was pretty unreliable in .NET 4
28. FileHelpers – reading writing text data.
29. Tcmdwincearm – Total Commander for mobile. Free.
30. Jquery and Jquery UI.
31. DataTables – html tables supports sorting , filtering, others.
32. Windows Live Essential – blogging fast.
33. InsertFilePlugin – Live writer extension to insert files to upload.
34. OfficeOpenXMLPart4-MarkupLanguageReference  – markup for Office XML. Used with StringTemplate
35. SharpZipLib – knows how to zip multiple files.
36. AutoFixture – generating sample data
37. HtmlAgilityPack – parsing web pages
38. T4MVC – get rid of magic names for controllers, actions
39. Itextsharp – save as pdf
40. Ninject – DI provider
41. Hudson – continous integration
42. AutoMapper  – transferring data between DAL and BLL
43. Selenium – testing web interfaces
44. Svn  -source control
45. MVC Contrib – pages list and more
46. Msbuildtask from tigris – build make it easy
47. Psr – help made easy in Windows 7
48. Firefox addons here

https://addons.mozilla.org/en-US/firefox/collections/ignatandrei/ignatandrei/

For every tool search for it . The first link will give you all details.

You can have as pdf here:List of programmer tools 2011

What do you need
1. SharpzipLib from http://www.icsharpcode.net/opensource/sharpziplib/ ( or download via NuGet in VS)
2. obtain the file(s) that you want as a string or as a byte[] – let’s say you have a byte[] to store in a str variable
3. make in your action something like that:

 var fcr = new ZipResult("Export.xls", str);
            fcr.AddFile("readme.txt","this zip file contains ..");
            return fcr;

4. copy the following code in your asp.net mvc projects:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
using System.Web.Mvc;
using System.IO;
using ICSharpCode.SharpZipLib.Zip;
using System.Text;

namespace utilsResult
{
    public class ZipResult : FileResult
    {

        private Dictionary<string, byte[]> content = new Dictionary<string, byte[]>();
        public string FileNameZip;
        public ZipResult(string FileName, byte[] Contents)
            : base("application/octet-stream")
        {
            this.FileDownloadName = Path.GetFileNameWithoutExtension(FileName) + ".zip";
            AddFile(FileName, Contents);
        }
        public void AddFile(string FileName,  byte[] Contents)
        {
            content.Add(FileName, Contents);
        }
        public void AddFile(string FileName,string Contents, Encoding e = null)
        {
            if (e == null)
                e = ASCIIEncoding.ASCII;

            content.Add(FileName, e.GetBytes(Contents));
        }

        protected override void WriteFile(HttpResponseBase response)
        {

            using (ZipOutputStream zos = new ZipOutputStream(response.OutputStream))
            {
                zos.SetLevel(3);
                zos.UseZip64=UseZip64.Off;

                foreach (var item in content)
                {
                    ZipEntry ze = new ZipEntry(item.Key);
                    ze.DateTime = DateTime.Now;
                    zos.PutNextEntry(ze);
                    int count=item.Value.Length;
                    zos.Write(item.Value, 0, count);

                }
            }
        }
    }
}

5. future improvements:
Zip the file(s) in a dll project to made fully testable!

First modification is to “Pre-generated Your View”. For this you must have .ssdl, .csdl, and .msl files – so you change the “Metadata Artifact Processing property to: Copy to Output Directory.”. Then you process the .ssdl, .csdl, and .msl with edmgen in order to can have the  views.

Until here , all ok.

But, in the next advice, is to keep “Metadata Artifact Processing property to: Embed in Output Assembly.”

One solution is to put “Metadata Artifact Processing property to: Copy to Output Directory.” , compile, put again “Metadata Artifact Processing property to: Embed in Output Assembly.” and compile again. But , if you change the edmx (add fields or tables ) you must redo the operation  – so you will have more things to do(if you remember)

A solution is to build them on the pre-build step .But how to generate the .ssdl, .csdl, and .msl files  ?

Edmgen2 , http://code.msdn.microsoft.com/EdmGen2 , to the rescue. Download, put into a lib folder under your solution folder and put a pre-build like this :

$(SolutionDir)lib\edmgen2\edmgen2 /FromEdmx $(ProjectDir)prod.edmx”

“%windir%\Microsoft.NET\Framework\v4.0.30319\EdmGen.exe” /mode:ViewGeneration /language:CSharp /nologo “/inssdl:prod.ssdl” “/incsdl:prod.csdl”   “/inmsl:prod.msl” “/outviews:$(ProjectDir)prod.Views.cs”

What you must change on your project?

1. IF you are on 64 bit, change Framework to Framework64

2. change the prod with your edmx name.

What is the performance?

Tested by loading a table with 301 rows by doing the steps :

1. open connection, load all table in objects(POCO), closing connection

2. open connection , find object with PK = 1, closing connection

3. open connection ,  loading 1 tables with 2 related (include ) , closing connection

The results are in milliseconds:

Without pre-compiled views

With pre-compiled views:

The average / min / max results:

In the next picture the smaller the duration(milliseconds), is the better :

Conclusions:

1.  For the average and min the difference is 7%, respectively 20%. Please remember we are using only 3 queries.

For the max, it is vey curious : the with is more than without. The penalty is 1%, I think that is a measuring error ? – or maybe not. However , the penalty is small comparing with others.

2. Very curious, find after ID, with a table with 301 rows, took longer than loading the whole table.However, did not take into accound finding in the list the object( it is in memory also)

3. It may worth to add the pre-build step shown before to pre-compile views.

Links :

http://www.codeproject.com/KB/database/PerfEntityFramework.aspx

http://msdn.microsoft.com/en-us/library/bb896240.aspx

Long Description :

I have finished a application with Entity Framework and POCO – and custom generated (.tt) files.

Now it was time to investigate performance  – and it was a terrific problem.

How EF Prof helped me :

1.  Displaying number of ObjectContext created and disposed in a Unit of Work and / or web page.

2. Displaying problem with the ObjectCOntext : using same context from multiple threads, alerts about code like  here :

The red dot is about querying the database from the view. The gray one is about selecting a whole table without selecting top 1000 rows(OK, it was a dictionary table, like the list of countries)

3. When you want caching on your site , you can fast see this by seeing the difference between the number of queries and/or number of ObjectContext for the first time and the second one ( first time : 10, second time:2 or 0 )

4. See most expensive queries as time – usually this is a good option to put an index on the “where” columns.More , you can see also non-unique queries (that you perform more than 1 time)

5. Investigate easily all queries. Know what you want to optimize and where.

Improvements  :

1. Say where is the ObjectContext that is not closing.

Alternatives:

How to: View the Store Commands : http://msdn.microsoft.com/en-us/library/bb896348.aspx

Sql Profiler : http://msdn.microsoft.com/en-us/library/aa173918%28SQL.80%29.aspx

(free ones :

http://sites.google.com/site/sqlprofiler/

http://code.google.com/p/sqlexpressprofiler/

)

But the job will be done by yourself!

So logparser to help  – he knows already to read IIS logs with  -i:IISW3C

So I have come up with the syntax  :

C:\LogParser -e:10 -i:IISW3C “SELECT cs-uri-stem as url, DIV(SUM(time-taken),1000) as Seconds, Count(time-taken) as Requests, DIV(Seconds ,Requests) as TimeExecuting   INTO C:\newfile FROM   C:\Windows\System32\LogFiles\W3SVC1\ex100909.log GROUP BY cs-uri-stem Having SUM(time-taken)>0 and Seconds>0 order by Seconds   desc” -o:TPL -tpl:%2\iistime.tpl

Basically , this will do this report about statuses of URL requested  :

The problem is that C:\newfile and  C:\Windows\System32\LogFiles\W3SVC1\ex100909.log are hard-coded – we need to modify every time… So PowerShell to the rescue (Ok, I could do a C# Console program – but

1. it is more fun this way – fun meaning I want to learn something new

2. the script could be modified easily

)

So the same command is written this way with arguments , in order to can be executed each time :

%2\LogParser -e:10 -i:IISW3C "SELECT cs-uri-stem as url, DIV(SUM(time-taken),1000) as Seconds, Count(time-taken) as Requests, DIV(Seconds ,Requests) as TimeExecuting   INTO %2\%4 FROM   %5\*%1  GROUP BY cs-uri-stem Having SUM(time-taken)>0 and Seconds>0 order by Seconds   desc" -o:TPL -tpl:%2\iistime.tpl

But who will give arguments ( such as the system date ) ?Now powershell  to the rescue :

$namepc = (gc env:computername)
$a = get-date
$a = (get-date).AddDays(-1)
$allpath= Split-Path -Parent $MyInvocation.MyCommand.Path;
$logfolders = $env:WINDIR +"\system32\Logfiles\W3SVC*"
foreach($logfolder in  Get-ChildItem $logfolders)
{
$logfiles= $logfolder.FullName
Write-Host "parsing"  $logfiles
$log =  $a.ToString("yyMMdd") + ".log"
$process = [Diagnostics.Process]::Start($allpath + "\iis.bat" , $log + " "+ $allpath + " "+ $log + ".html" + " TIME" +$log + ".html" + " " + $logfiles)
$process.WaitForExit()
$content = "<h1>IIS REPORT " + $namepc  + "</H1>"
$content += (get-content ($allpath  + "\" + $log + ".html"))
$content += (get-content ($allpath  + "\TIME" + $log + ".html"))

$SmtpClient = new-object system.net.mail.smtpClient
$SmtpServer = "your server"
$SmtpClient.host = $SmtpServer

$mm = new-Object System.Net.Mail.MailMessage(“<a href="mailto:from@yourcompany.com">from@yourcompany.com</a>”,"<a href="mailto:to@yourcompany.com">to@yourcompany.com</a>")
$mm.Subject = "Report IIS " + $namepc
$mm.Body = $content
$mm.Body=$mm.Body.Replace("&lt;cmp&gt;",$namepc  )
$mm.IsBodyHtml = 1
$SmtpClient.Send($mm)
}

Explanation of code :
line 1: I take the computer name to put in the report
line 2 : take the date ( if you want the current date , just comment the line 3)
line 5 : I go to usual path to logfiles (did I say quick and dirty ?)
line 6 : get all W3SVC folders and iterate to send report
line 11 : launching the bat (that contains logparser command ) in order to parse  arguments
line 12 :waiting for the process to exit – in order to can send files.
line 13 to 15: get the output
line 17 to 27 : send the output by email

Homework :
1. Execute script on a system and modify if it does not work
2. Clean up the temporary files after sending email
3. Instead of sending an email, write into a database with the current date.

You can execute sc.bat at regular times ( such as 1:00 AM)

Here is the zip file with sources logiisparser

LogParser download : http://www.microsoft.com/downloads/en/details.aspx?FamilyID=890cd06b-abf8-4c25-91b2-f8d975cf8c07&displaylang=en

Powershell scripts : http://gallery.technet.microsoft.com/ScriptCenter/en-us/site/search?f[1].Type=SearchText&f[1].Value=internet&f[0].Value=applications&f[0].Type=RootCategory&f[0].Text=Applications&x=0&y=0

I have a bot from http://www.imified.com/ – and I log the messages with log4net in text files, with another messages.

An entry looks like that :

System.ArgumentException: ;channel=private;botkey=<guid>;userkey=<guid>;user=name@yhaoo.com;network=Yahoo;msg=hello;step=1;value0=hello;to=asdasd

And there are multiple log files that I want to parse and find the email adresses to collect feedback from those persons that use my bot.

LogParser to the rescue! Download from http://www.microsoft.com/downloads/details.aspx?FamilyID=890cd06b-abf8-4c25-91b2-f8d975cf8c07&displaylang=en and use this command line

LOGPARSER “Select Text into a.csv from current* where Text like ‘%@%’” -i:TEXTLINE

Explanation of command :

Select Text into a.csv from current* where Text like ‘%@%’ –means find in files that begin with current(current*) all text that contains emails ( ‘%@%’) and put in file a.csv  the results.

-i:TEXTLINE – means the format is text

What can be more simple ?

(Ok, for finding the user name I had to resort to excel, to remove duplicates … )

More I think it is fast enough : for parsing 114 files with 58.8 MB (PC with a 2GB RAM + 7200 RPM ) the results are :

Statistics:
———–
Elements processed: 487176
Elements output:    1044
Execution time:     7.69 seconds

Also logparser can be used for more than text files :

http://support.microsoft.com/kb/910447

http://www.stevebunting.org/udpd4n6/forensics/logparser.htm

More, it can be as a COM DLL in every .NET project, making it a usefull tool . See

http://www.codeproject.com/KB/recipes/SimpleLogParse.aspx

Next time I will show the using Powershell in combination with LogParser.