Hanalei, Hawaii Tuesday, February 09, 2010

Make Visual Studio Generate Your Repository

As many of you may know, I've been goofing around with Code Generation again with SubSonic's new MVC Addin. One of the things I really wanted to try and flex is Visual Studio 2008's code generation bits - T4.

As many of you may know, I've been goofing around with Code Generation again with SubSonic's new MVC Addin. One of the things I really wanted to try and flex is Visual Studio 2008's code generation bits - T4. Not many people know it even exists, and to me it's one of the Great Hidden Secrets of Visual Studio 2008.

What Is T4?
Quite literally, T4 stands for

Text Template Transformation Toolkit

Otherwise known as Code Generation. MSDN has a nice writeup on T4 as well. The easiest way to think of T4 is that it's just like coding ASP - except the script that you create is code that stays inside Visual Studio.

Every T4 script file (extension TT) is "executed" whenever a save happens inside Visual Studio, and the T4 file has been altered - sort of like a Build Provider. The nice thing here, however, is that there is some pretty nice debugging that you can do - and you can even step through the code that Visual Studio is generating.

It's insanely easy to work with T4 and there is a lot of information out there on the web. David Hayden put together (as usual) one of his stellar screencasts, which you can watch here. In addition, Oleg Sych is a T4 Ninja and has just about any resource and/or information you might need on his site.

A Look At The Template Code
Let's jump into the code! One of the things that you may want to do with T4 right off the bat is have it create your DB access code. David Hayden has a nice screencast for how he does it - I thought it would be fun to show you how you can wrap Linq To Sql into a nice, testable framework for use with something like... oh perhaps ASP.NET MVC :).

For these examples I'm going to use the "plumbing" code that I've worked up with SubSonic.Mvc (also known as Makai). I have some bits in there that grab a DBML file and tell me all kinds of nice schema information - I'll use this information to create my template.

First, this is your basic T4 template:

<#@ template language="C#" debug="True" hostspecific="True" #>
<#@ assembly name="System.Data" #>
<#@ assembly name="SubSonic.Mvc.dll" #>
<#@ assembly name="SubSonic.dll" #>
<#@ import namespace="System.Diagnostics" #>
<#@ import namespace="System.IO" #>
<#@ import namespace="System.Data" #>
<#@ import namespace="SubSonic" #>
<#@ import namespace="SubSonic.Mvc" #>

using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;


namespace Test{   
    
    class Program{
        
        static void Main(string[] args){
            <#for(int i=0;i<10;i++){#>
            Console.WriteLine("<#=i.ToString()#>");
            <#}#>
        }
    } 
   
}

The markup above with the "#" sign is a T4 command. The first few lines load up some assemblies and reference some namespaces. The rest of the code after that is "static" - stuff that will get "rendered" so to speak. If you look inside the Main() method - you can see I'm running a for-loop that outputs the value of the loop index.

The output of this code is:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;

namespace Test{   
    
    class Program{
        
        static void Main(string[] args){
            Console.WriteLine("0");
            Console.WriteLine("1");
            Console.WriteLine("2");
            Console.WriteLine("3");
            Console.WriteLine("4");
            Console.WriteLine("5");
            Console.WriteLine("6");
            Console.WriteLine("7");
            Console.WriteLine("8");
            Console.WriteLine("9");
        }
    } 
   
}

This is, of course, a very silly example. But hopefully you can see the potential here - and the point of it all :). It's very easy to extend and can be used in any type of project. In addition you can change the output extension to render ... well any type of output you like - HTML, vb, cs, etc.

Whenever you create a T4 template, a "shadow" file is created with it that contains your generated code:

t4

You can use T4 to get yourself started (and then jettison the template), or you can keep your template and modify as you need for your project. Personally I like to use code generation to get me to 80% or so - after that I end up tweaking the template more than my project :).

Wrapping Linq
As I mention above, I've created some templates here that wrap up Linq To Sql in a nice IQueryable Repository (what I've been doing with the storefront). I like the way these templates work - and it makes the repository system nice and simple to use for your project.

The use these, you need to do 3 things:

1) Pop SubSonic.Mvc somewhere on your drive - it doesn't need to be added as a reference to your application - but the templates will need to know where to find it.

2) Change the initial variables of the template to match your project. Each template has the following bits of info that it needs at the top of the template:

    //the path to your Linq To Sql DBML File
    string DBMLFilePath=@"C:\Projects\NorthwindDB.dbml";  
    //the Type name of your DataContext
    string dataContext="NorthwindDB.DB";
    //the namespace of your Linq To Sql Model classes
    string dataNamespace="NorthwindDB.Data";
    //if you've put your DataContext in a different namespace
    //set this to "using [DataContextNamespace]"
    string contextNamespaceDeclaration="";

The neat thing about these templates is you can jigger whatever you need to - you're not going to violate any rules here. This is just a text templating tool - so hard-code away! I've just set these here for clarity :).

3) Tweak as you need to. You will most likely have to tweak the template to get it to compile - that's just the nature of these things :). It uses your Linq To Sql DBML file for its schema reference, so if you want to change naming or whatnot - you should do it there.

Editing Your Own Templates
There's so much you can do with this stuff, it's really amazing. You owe it to yourself to spend a little time and see just how much time you can save by doing some light code-generation (caveat: as with all good things, there is a point of self-destruction brought on by over-indulgence).

One handy tool is Clarius's T4 template editor that brings all the syntax coloring and intellisense to life that you need.

Another nice read is what Damien Guard has done with the Linq To Sql templates that are generated by the designer. Damien works on the Linq To Sql team and has made some great changes to the code generation bits.

P.S. - Yes I will be working some of these up for SubSonic :)

You can grab my templates here.


robconery - October 14, 2008 - Hi Sean - nope, no acquisitions here :). They do work on the same principles, but CodeSmith works a bit differently.
robconery - October 14, 2008 - You too.
Tim Hardy - October 14, 2008 - I've been scouring the Net to find a way to programmatically execute T4 templates. Do you know of a way? I think their usefulness is mitigated by the fact that we can't suck their functionality into our own programs.



Codesmith just uses a syntax similar to aspx, so I don't think there's any rocket science to the syntax.
robconery - October 14, 2008 - Hi Tim - if you're looking for a scripting engine SubSonic can do that for you. Send me an email and I'll show you how to "hijack" our code generator :).



T4 works in the scope of the IDE but I'm sure there's a way to get it to execute... hmmm...
GarethJ - October 14, 2008 - Tim, I own T4 at Microsoft - T4 has a VS service API and also a deeper managed API, which is documented on MSDN in the VS SDK docs. However, both of these are restricted to creating tools, where some flavor of VS is on the machine. Drop me a mail at gareth.jones@microsoft.com with your requirements for where you'd use it in your own programs.
Adam - October 14, 2008 - Hey Rob,



Getting an file not found error for the subsonic.mvc.dll i think.

Where am i suppose to place it and how do i reference it??



Cheers
robconery - October 14, 2008 - Hi Adam - in the @assembly reference, just reference the DLL file - like "C:\files\SubSonic.Mvc.Dll". I think I forgot to mention that here.
Adam - October 14, 2008 - Sorry Rob, Still can't get it to work. Same error.

Anything else it could be?



more detailed error:



Error 2 Running transformation: System.IO.FileNotFoundException: Could not load file or assembly 'SubSonic, Version=2.1.1.0, Culture=neutral, PublicKeyToken=eadb47849839a332' or one of its dependencies. The system cannot find the file specified.

File name: 'SubSonic, Version=2.1.1.0, Culture=neutral, PublicKeyToken=eadb47849839a332'

at SubSonic.Mvc.LinqSchema.LoadFromDBML()

at SubSonic.Mvc.LinqSchema..ctor(String DBMLFilePath)

at SubSonic.Mvc.SchemaHelper.GetSchema()

at Microsoft.VisualStudio.TextTemplating85E4B1FD84AAF0AE5264C0C2A5D6F608.GeneratedTextTransformation.TransformText() in g:\Development\svn\RpAssist Project\RpAssist\App\IRepository.tt:line 25



WRN: Assembly binding logging is turned OFF.

To enable assembly bind failure logging, set the registry value [HKLM\Software\Microsoft\Fusion!EnableLog] (DWORD) to 1.

Note: There is some performance penalty associated with assembly bind failure logging.

To turn this feature off, remove the registry value [HKLM\Software\Microsoft\Fusion!EnableLog].

G:\Development\svn\RpAssist Project\RpAssist\App\IRepository.tt 1 1

robconery - October 14, 2008 - Hmm - looks like you need SubSonic too... I'll add to the zip.
Erik - October 17, 2008 - Adam how did you get to work. Just explain in plain english where you found 'SubSonic, Version=2.1.1.0,
Adam - October 14, 2008 - Got it working!

Brilliant
Mike - October 15, 2008 - I had no idea... thanks!
Igor - October 15, 2008 - Hi Rob,



Is it possible to generate VB code?



Realy cool stuf...



Thanks,



Igor
Adam - October 17, 2008 - @Erik



I copied the two dlls listed below in c:\

then added this to the top of all of the 4 tt files and wallah.



<#@ assembly name="c:\SubSonic.Mvc.dll" #>

<#@ assembly name="c:\SubSonic.dll" #>



Hope this helps.



Erik - October 17, 2008 - Thank you ! I functions after some code cleaning. The templates are also good to use if one wants to check that all tables have keys
robconery - October 15, 2008 - Why would you want to do that? :):) Of course - I don't know VB but others are welcome to do it.
Matthew M. - October 15, 2008 - Is this not included in the Standard version of Visual Studio ? I looked at the comparison page and could not find any reference to this. If it is included in the Std version (2008), how do you make it active ?



Thanks!
Matthew M. - October 15, 2008 - Nevermind!! I'm not smart. I don't know what I did wrong the first time.



*Hangs head in shame*
firefly - October 13, 2008 - Urgghh... I wish I knew about this about 6 month ago. Would have saved me quite sometimes... Thanks Rob :)
davidinbcn - October 13, 2008 - you're a cool guy rob
Ben Taylor - October 14, 2008 - Thanks for posting about T4 Rob. Keep the code gen, templating posts coming. I am a BIG fan of code generation and automation and I can already see several applications for T4. In the past I have used StringTemplate http://www.stringtemplate.org/ for similar(ish) tasks, but having the tools inside the IDE should cut the cost right down.



All I want now is to be able to extend the C# compiler! Do you know how to do that ;)

Mikael Kjellqvist - October 14, 2008 - Thanks for sharing this, i have built my own tool to do this. Would have saved me some time if i knew of this feature a couple of months ago. :)

Very nice.



daerid - October 14, 2008 - Wow.. I've been using MyGeneration for this kinda stuff, and now I can finally ditch that for an IDE integrated solutions. Thanks Rob!
Arch4ngel - October 14, 2008 - Wahhh... I didn't know that. I'll take a look at that to see if we can template our current solutions :)
Sean Taylor - October 14, 2008 - Looks good, looking forward to playing with this.



The syntax looks very similar to codesmith (http://www.codesmithtools.com/) Has Microsoft aquired Codesmith?
Steve - November 11, 2008 - Hi Rob. Thanks for posting this. I am unable to use the templates because I do not have SubSonic 2.1.1 and it is not included with the zip file. Is this version available for download anywhere? I am getting the same error that Adam posted but I still can not locate the correct dll.



Thanks.
ApasySarCap - April 16, 2009 - nice, really nice!
firefly - October 14, 2008 - Urgghh... I wish I knew about this about 6 month ago. Would have saved me quite sometimes... Thanks Rob :)
davidinbcn - October 14, 2008 - you're a cool guy rob
robconery - October 14, 2008 - You too.
Ben Taylor - October 14, 2008 - Thanks for posting about T4 Rob. Keep the code gen, templating posts coming. I am a BIG fan of code generation and automation and I can already see several applications for T4. In the past I have used StringTemplate http://www.stringtemplate.org/ for similar(ish) tasks, but having the tools inside the IDE should cut the cost right down.

All I want now is to be able to extend the C# compiler! Do you know how to do that ;)
Mikael Kjellqvist - October 14, 2008 - Thanks for sharing this, i have built my own tool to do this. Would have saved me some time if i knew of this feature a couple of months ago. :)
Very nice.
daerid - October 14, 2008 - Wow.. I've been using MyGeneration for this kinda stuff, and now I can finally ditch that for an IDE integrated solutions. Thanks Rob!
Arch4ngel - October 14, 2008 - Wahhh... I didn't know that. I'll take a look at that to see if we can template our current solutions :)
Sean Taylor - October 14, 2008 - Looks good, looking forward to playing with this.

The syntax looks very similar to codesmith (http://www.codesmithtools.com/) Has Microsoft aquired Codesmith?
robconery - October 14, 2008 - Hi Sean - nope, no acquisitions here :). They do work on the same principles, but CodeSmith works a bit differently.
Tim Hardy - October 14, 2008 - I've been scouring the Net to find a way to programmatically execute T4 templates. Do you know of a way? I think their usefulness is mitigated by the fact that we can't suck their functionality into our own programs.

Codesmith just uses a syntax similar to aspx, so I don't think there's any rocket science to the syntax.
robconery - October 14, 2008 - Hi Tim - if you're looking for a scripting engine SubSonic can do that for you. Send me an email and I'll show you how to "hijack" our code generator :).

T4 works in the scope of the IDE but I'm sure there's a way to get it to execute... hmmm...
GarethJ - October 14, 2008 - Tim, I own T4 at Microsoft - T4 has a VS service API and also a deeper managed API, which is documented on MSDN in the VS SDK docs. However, both of these are restricted to creating tools, where some flavor of VS is on the machine. Drop me a mail at gareth.jones@microsoft.com with your requirements for where you'd use it in your own programs.
Adam - October 14, 2008 - Hey Rob,

Getting an file not found error for the subsonic.mvc.dll i think.
Where am i suppose to place it and how do i reference it??

Cheers
robconery - October 14, 2008 - Hi Adam - in the @assembly reference, just reference the DLL file - like "C:filesSubSonic.Mvc.Dll". I think I forgot to mention that here.
Adam - October 15, 2008 - Sorry Rob, Still can't get it to work. Same error.
Anything else it could be?

more detailed error:

Error 2 Running transformation: System.IO.FileNotFoundException: Could not load file or assembly 'SubSonic, Version=2.1.1.0, Culture=neutral, PublicKeyToken=eadb47849839a332' or one of its dependencies. The system cannot find the file specified.
File name: 'SubSonic, Version=2.1.1.0, Culture=neutral, PublicKeyToken=eadb47849839a332'
at SubSonic.Mvc.LinqSchema.LoadFromDBML()
at SubSonic.Mvc.LinqSchema..ctor(String DBMLFilePath)
at SubSonic.Mvc.SchemaHelper.GetSchema()
at Microsoft.VisualStudio.TextTemplating85E4B1FD84AAF0AE5264C0C2A5D6F608.GeneratedTextTransformation.TransformText() in g:DevelopmentsvnRpAssist ProjectRpAssistAppIRepository.tt:line 25

WRN: Assembly binding logging is turned OFF.
To enable assembly bind failure logging, set the registry value [HKLMSoftwareMicrosoftFusion!EnableLog] (DWORD) to 1.
Note: There is some performance penalty associated with assembly bind failure logging.
To turn this feature off, remove the registry value [HKLMSoftwareMicrosoftFusion!EnableLog].
G:DevelopmentsvnRpAssist ProjectRpAssistAppIRepository.tt 1 1
robconery - October 15, 2008 - Hmm - looks like you need SubSonic too... I'll add to the zip.
Adam - October 15, 2008 - Got it working!
Brilliant
Mike - October 15, 2008 - I had no idea... thanks!
Igor - October 15, 2008 - Hi Rob,

Is it possible to generate VB code?

Realy cool stuf...

Thanks,

Igor
robconery - October 15, 2008 - Why would you want to do that? :):) Of course - I don't know VB but others are welcome to do it.
Matthew M. - October 16, 2008 - Is this not included in the Standard version of Visual Studio ? I looked at the comparison page and could not find any reference to this. If it is included in the Std version (2008), how do you make it active ?

Thanks!
Matthew M. - October 16, 2008 - Nevermind!! I'm not smart. I don't know what I did wrong the first time.

*Hangs head in shame*
Erik - October 17, 2008 - Adam how did you get to work. Just explain in plain english where you found 'SubSonic, Version=2.1.1.0,
Adam - October 18, 2008 - @Erik

I copied the two dlls listed below in c:
then added this to the top of all of the 4 tt files and wallah.

<#@ assembly name="c:SubSonic.Mvc.dll" #>
<#@ assembly name="c:SubSonic.dll" #>

Hope this helps.
Erik - October 18, 2008 - Thank you ! I functions after some code cleaning. The templates are also good to use if one wants to check that all tables have keys
Steve - November 11, 2008 - Hi Rob. Thanks for posting this. I am unable to use the templates because I do not have SubSonic 2.1.1 and it is not included with the zip file. Is this version available for download anywhere? I am getting the same error that Adam posted but I still can not locate the correct dll.

Thanks.
Gecko