Home MVC Storefront

SubSonic: Using Migrations

I've spent the passed few weeks polishing up some features for our 2.1 release (aka Pakala), and I've been paying particular attention to Migrations - something I promised would be ready to go a few months back. As of changeset 452 (made today, just now), Migrations are reasonably solid.

As Always, There's a Video
I like blogs with code, but singalongs are more fun. I made a quick video today (about 15 minutes) of how you can use Migrations in your project.

You can watch it here.

If you're more into the code and how they work - read on! All the code you will see below is viewable in the video.

Give Shawn Some Love
I laid the groundwork for Migrations a while ago, but didn't have the bandwidth to give it the love it deserved. Frankly I don't know if I was smart enough.  I asked Shawn Oster to help out and he took his massive swingin geek smarts and put together a pretty cool set of functionality.

So if you see Shawn at the local Denver Gamer Shop, setting up for the Monday Night D&D Showdown (LARP-fest), give him a wizardly high-five!

The Code>>>
The Migration system works by basically "sucking" out the Migration classes from your project, reading in the code, and then executing it in a virtual compiler. This may seem complex, but it's actually pretty simple and at it's core is what we do to generate the code files anyway.

To create a Migration, you have to follow our Conventional system and create a class file, something like

001_Initial.cs

... and place that class file into a folder called "Migrations" off the root of your project (you can override this directory convention - see below).

This class file must have one class (call it whatever you like), and it has to inherit from SubSonic.Migration:

using System;
using System.Collections.Generic;
using System.Text;
using SubSonic;

namespace MigrationSample.Migrations {

    public class Migration001:Migration {
    }
}

Each "Migration" consists of changing from one version to another, either up or down. The code, therefore, follows this logic and offers two methods for you to tell the DB what to look like.

For the first example (001_Init), this is version one. The Up() method therefore is transitioning the DB from version 0 to version 1, so we need to put in some code to tell it what we want it to do:

        public override void Up() {

            //Create the Records Table
            TableSchema.Table records = CreateTableWithKey("Records");
            records.AddColumn("RecordName");
            records.AddColumn("GroupID", System.Data.DbType.Int32);
            records.AddColumn("LabelID", System.Data.DbType.Int32);

            AddSubSonicStateColumns(records);

            //Create the Groups Table
            TableSchema.Table groups = CreateTableWithKey("Groups");
            groups.AddColumn("GroupName");
            AddSubSonicStateColumns(groups);

            //Link them
            CreateForeignKey(groups.GetColumn("id"), records.GetColumn("groupID"));


        }

For the Down() method, we need to reverse, exactly, everything we did with Up():

        public override void Down() {
            TableSchema.Table records = GetTable("records");
            TableSchema.Table groups = GetTable("groups");

            //drop the FK
            DropForeignKey(groups.GetColumn("id"), records.GetColumn("groupID"));
            
            DropTable("Records");
            DropTable("Groups");
        }

*Note: I'm working on the syntax to drop the FK constraint. I know it's heavy.

If you needed to alter a column, you use:

    AlterColumn("records", "RecordName", System.Data.DbType.String, 800);

You can change the column's length, type, name, and nullability if you need to. You can also remove a column from a table:

    RemoveColumn("records", "groupid");

Iterations
If your client says to you "hey great, you made a Records table - but you forgot Labels! You have labelID, where's labels!" - it's time to write another Migration, and migrate from version 1 to version 2:

Following our convention, add another class file to the Migrations folder:

002_AddLabels

Next up, add the code to Up/Down our migration:

 

namespace MigrationSample.Migrations {
    
    public class Migration002:Migration {
        public override void Up() {

            //add the labels table
            TableSchema.Table labels = CreateTableWithKey("Labels", "labelID");
            labels.AddColumn("LabelName");
            AddSubSonicStateColumns(labels);


            Execute("INSERT INTO Labels(labelname) VALUES('Capitol')");
            Execute("INSERT INTO Labels(labelname) VALUES('Arista')");
            Execute("INSERT INTO Labels(labelname) VALUES('Virgin')");

            TableSchema.Table records = GetTable("records");
            CreateForeignKey(labels.GetColumn("labelID"), records.GetColumn("id"));

        }

        public override void Down() {
            TableSchema.Table records = GetTable("records");
            TableSchema.Table labels = GetTable("labels");

            //drop the FK
            DropForeignKey(labels.GetColumn("labelID"), records.GetColumn("id"));
            DropTable("labels");
        }
    }
}

Yes! Migrations also allow you to add data! I know that inline script is probably not what you had in mind ;) - I'll have this worked out by the time we go final with 2.1 (allowing you to use our query tool) but for now - the ability is there.

Now, to execute...

sonic.exe /migrate

You can set this up like you do with other SubCommander commands - please see the video for how to make this happen!

Comparisons
There are other things out there (like DB projects with VS) and I go into some of this in the video (nudge nudge). This isn't a versioning tool per se - it's a development tool. If you like scripts, more power to ya. Migrations take a little getting used to, that's for sure.

Caveats
This stuff may still be a tad rough. We've tested it a lot, but whenever you talk about tweaking DB schema and code... well there's a reason there's not many Migration solutions in .NET land right now :). Please be patient and help us to get this up to par.

Zack Owens avatar
Zack Owens says:
Friday, June 06, 2008

Yuck to the inline SQL :)

Why don't you have a method that you CAN overide called UpData or something that will only fire if the transactions in Up all work? Just my 2 cents.

I did like the original migrator prototype that generated the migration classes for you, that was totally sweet!

Anyways, great job! Can't wait to try it out!!


John S. avatar
John S. says:
Friday, June 06, 2008

Very nice Rob!

I'd love to see automatic FK handing on DropTable. I can't imagine a scenario where I wouldn't want to delete the foreign key when deleting a table.

Will there be a utility for generating the first migration class from an existing DB?


Vladan Strigo avatar
Vladan Strigo says:
Friday, June 06, 2008

Rob,

Can you use your AR model in migrations?

I am thinking of making a simple migration lib for my framework and one of the goals would be that besides using the structure api (new Table()) that you can also execute the model of your project (e.g. to insert data).

How do you stand on that?

--

Vladan Strigo

http://vladan.strigo.net


Dietrich avatar
Dietrich says:
Friday, June 06, 2008

Not sure if you're aware of it but rails 2.1 no longer uses version numbers. 2.1 instead uses a date stamp. I haven't upgraded to 2.1 so I don't know if it's better but in theory it should be because if you're working with others or branching versions (and then merging) migration number conflicts can be a pain.

Instead of just storing just the latest version number in the migration tracking table, the table stores all run versions and then compares that against the migration directory. There's a railscasts.com on it.


Yitzchok avatar
Yitzchok says:
Friday, June 06, 2008

Great Stuff.

What about the ability to migrate from resource files in an assembly.


Troels Thomsen avatar
Troels Thomsen says:
Friday, June 06, 2008

I was playing around with the migrations and couldn't figure out why it wouldn't work. The output gave an useless "String cannot have zero length". I started looking through the CodeRunner class and found out it relies on the starting bracket to be on the same line as the namespace definition. I guess it has simply been missed as the class is parsed correctly even though the bracket is placed on the next line.


Willie Tilton avatar
Willie Tilton says:
Friday, June 06, 2008

Troels, I got a ticket and a patch in on codeplex. They had an issue with it not parsing the namespace correctly before, but now just blows chunks all over that section because regex returns a blank or something.

Also when I ran "Update" sql it hadn't already added the column so threw some errors there. I put them in different commands so hopefully that'll get integrated.

As far as doing it data migrations with the query tool, that'd be fine, but it's almost as cool that this could be used totally without generating any classes. For instance in this new project I have, there's already a ORM, and the app is in the final stages, so switching that up now wouldn't be the best idea.


Dan F avatar
Dan F says:
Saturday, June 07, 2008

I haven't watched the video, so apologies if this has been answered, but anywho...

How do you do sprocs/views/functions/etc? Inline SQL (probably embedded as a resource)? I'm digging the vibe for tables (looks awesome!), just wondering what the story is for the codeish bits of the database.


Troy Goode avatar
Troy Goode says:
Saturday, June 07, 2008

Rob, if you have a chance take a look at RikMigrations. Your project is very similar, but I think they've tightened the syntax a bit over what you currently have. It may be worth examining as you improve on this preview.

I especially like the flowing syntax of column creation (specifically foreign-key relationships). Example:

Table groups = db.AddTable("Groups");

groups.AddColumn<int>( "ID" ).PrimaryKey();

groups.AddColumn<string>( "GroupName" ).Unique();

Table records = db.AddTable("Records");

records.AddColumn<int>( "ID" ).PrimaryKey();

records.AddColumn<int>( "GroupID" ).References( "Groups", "ID" );

records.AddColumn<string>( "Name" ).NotNull();

www.rikware.com/.../RikMigrations-T


mike avatar
mike says:
Sunday, June 08, 2008

This is very cool. I will use it for sure.

I am wondering about something though. On each dev machine you have sonic.exe to run the migrations. Then at version 15 you deploy it live. Then at version 28 you want to update the live database. For both deployment scenario's, will sonic.exe be able to generate sql scripts, because I don't see how we can run the migrations on a hosting server.

Thanks, and keep up the good work!


Michael Hensen avatar
Michael Hensen says:
Sunday, June 08, 2008

I use subsonic a lot and again this release is a perfect example of "thinking about the datalayer". keep this up and going. Subsonic never has beer dead by me! Due to this new functionality I'm more and more eager to start contributing to this project! I got my idea's for functionalities and tools flowing, it's just time that is against.


Mike avatar
Mike says:
Monday, June 09, 2008

@Willie Tilton, do you have a link for that issue? I want to vote on it, it's a blocking bug, because in VB.NET there's no workaround (can't place a bracket in VB...), so you can't use Migrations in VB.NET :(


Robert avatar
Robert says:
Wednesday, June 11, 2008

When I try to add a DBType.Byte (TinyInt) column it seems to want to create a column with SQL type Image.



Search Me
Index Of MVC Screencasts

You can watch all of the MVC Screencasts up at ASP.NET, and even leave comments if you like.

Subscribe

Popular Posts
 
My Tweets
  • @mattberther do you really want to use the words "fair", "balanced" and "Hannity" in the same sentence? :):)
  • @ryanlanciaux McCain is becoming a national embarrasment like Bush before him. Cheapens our political process and makes us look like idiots
  • @ryanlanciaux The basics I spose - the ability to speak in full sentences, details about their plans, no overt smears and lies... like that
  • Time Magazine: "what a desperate empty embarrassment the McCain campaign has become". Have to agree. http://tinyurl.com/3kygnq
  • How does O'Reilly keep his job? http://tinyurl.com/3zw5r4
  About Me



Hi! My name is Rob Conery and I work at Microsoft. I am the Creator of SubSonic and was the Chief Architect of the Commerce Starter Kit (a free, Open Source eCommerce platform for .NET)

I live in Kauai, HI with my family, and when my clients aren't looking, I sometimes write things on my blog (giving away secrets of incalculable value).