Hanalei, Hawaii Tuesday, February 09, 2010

Source Control and Continuous Integration On The Cheap

There are many solutions out there for Source Control and Build Servers (with Continuous Integration) - most notably (and in no particular order) Team Fortress Foundation Server, Subversion, CruiseControl.NET, and CI Factory.

There are many solutions out there for Source Control and Build Servers (with Continuous Integration) - most notably (and in no particular order) Team Fortress Foundation Server, Subversion, CruiseControl.NET, and CI Factory. Each of these solutions works great - but sometimes you need something up, quick, with a minimal hit to the wallet. I'll show you how to do that today - it's very, very easy.

 

I'm In UR Fortress, Hold'n UR Codez
Two years ago I was hired on to a project to, in part, transition the dev team from Source Safe to Team Foundation Server. The manager of the team gave me a month and I was done in 3 days (after all the licenses), and then took another 2 days to get the CI server running. I was able to work the solution nicely, moving dependencies between branches and everyone was very happy - especially Mr. Manager since he could now enforce code policies and move tasks into TFS (out of Outlook). The teams productivity vaulted and everyone was happy.

TFS is a great solution for a corporate team (integrated workflow, task management, test integration, code policy, Sharepoint integration with reporting and metrics, etc), but if you don't have the means for a license (if you run an Open Source project for instance), you can still use a great solution set to get running in under 3 hours.

Yes, under 3 hours.

 

Using Subversion
Subversion is a perfect Source Control system for the independent developer /small shop /Open Source project. Not only can you store code in there - you can store just about any document or file. I use it for my code library, invoicing, and applets I don't want to lose.

 

Subversion Setup Made Simple - Visual SVN Server
Setting up Subversion isn't the easiest thing - especially if you want secure web access. VisualSVN Server makes this simple with a one-click installer that's free. Yes FREE. It comes bundled with Apache Tomcat and sets up your repository folder for you, then exposes it all on port 81 or 8080 (your choice - you can also set it up to use SSL).

VisualSVN gives you a GUI that works just like a DB manager, allowing you to manage all the properties of your repository, complete with security:

SVNServer

 

Using Tortoise For Client Access
Tortoise is a shell application that sits on top of Windows Explorer, allowing you to work with your repository right through Explorer. It's a very handy tool and so much has been written on it that I'm going to sidestep the description and recommend you read more here if you want.

 

Buy Visual SVN VS Client - It's Worth Every Penny
If you start using Subversion you'll quickly notice there's no direct tie-in to Visual Studio's source client API. The folks at Visual SVN have come up with a great solution to this: VisualSVN. It's $50 US and, in my mind, is quite the bargain.

It allows you to work pretty seamlessly in the Visual Studio environment with your Subversion server - or will create a repository for you on your hard drive if you like. I've even set one up on one of my flash drives :).

You can read more here.

 

Intermission
OK - check of the clock - you've probably spent maybe 45 minutes downloading and installing. Go grab a cold beer - we're almost done :).

All set? Ok...

 

Build Server Time
Usually when bringing up Continuous Integration and Build Server, people begin groaning as they imagine drowning in a sea of configuration, broad-sided by XML rogue waves.

The only time I've ever set up a CI server easily was using TFS - but that still involved moving target files around and all kinds of configuration - but it wasn't that hard and we were up and running in a day or so.

Harman tipped me off to TeamCity by JetBrains (the ReSharper guys), and I thought I'd give it a try since, well, it's FREE.

I didn't have a CI solution for SubSonic and needed one badly, so I downloaded it, clicked "run" - it asked me a few questions, and boom - it was up and running. Nice!

But that's the easy part. Configuring your project... that's another story. Isn't it?

Nope...

tcconfig

 

The setup for SubSonic took me, literally, 20 minutes. It goes like this:

  1. Give your project a name, description, etc
  2. Point TeamCity to your project's trunk in Subversion (doesn't have to be the same server - ours is up at http://subsonicproject.googlecode.com/svn/trunk/). You give it your repository login (if needed) so it can check out your project files, and that's it.
  3. Choose your build type. They make this insanely easy for Visual Studio users - just point to your .sln file (since that's the build file for your solution). You tell TeamCity where to drop the files on your hard drive when it pulls from Subversion, and then tell it the relative path to the .sln file, and you're done.
  4. Set the build trigger. By default this is set to poll your repository every minute - throttle as needed. If it detects a change go in, the source is updated and rebuilt.
  5. Dependency settings - tell TeamCity where to pull any dependency DLLs or artifacts when it's building.
  6. The rest are optional.

Did I mention this is free?

Add your users, setup your mailer settings, and you're off! TeamCity will run whenever you do a COMMIT to your source, and will pull the comments and user out (each user is tied to a Subversion account). There's so much that it does - all I can say is to go have a look at it. JetBrains did some amazing work here.

 

But Wait, There's More!
You can setup all kinds of tools and notification things - just like CC.NET. My favorite is the VS integration - it let's you see what's happening on the server at all times. There's also a tray tool to let you know what's happening and an RSS feed:

tctools

The really neat thing about the VS plugin is that it allows you to run PRIVATE BUILDS. This is very cool - so you can "pre-run" a build to see if your checkin will break anything. If it doesn't - it will commit the change for you and then run the test. Very cool indeed.

 

Time Check
All in all you should be approaching 3 hours by now. The best part is the work to set this all up takes about 2 hours - playing with TeamCity (which is fun) takes up the last hour :).

There are all kinds of plugins and test runners to set up - so do check it out. It's free for Professional Use and Open Source - which is very nice.

 

Summary
Getting yourself setup with this kind of system is very handy, and can save you a lot of time. If you don't have the option to go TFS, do check these tools out!


Steven Harman - January 27, 2008 - Glad I could be of help buddy! So is there a public URL that we can go to and check out the current build status for Subsonic?
Steve Trefethen - January 28, 2008 - Here is a video of CI setup from scratch with SVN, TortioseSVN and CCNet in 9 minutes and 30 seconds complete with an example of checking in a syntax error and watching the build fail then checking in the fix. http://www.stevetrefethen.com/videos/Continuous Integration.ashx An accompanying blog post is here: http://www.stevetrefethen.com/blog/VideoSettingUpAContinuousIntegrationEnvironment.aspx My point being that this stuff is simply too easy these days not to use, there's just no excuse anymore.
Shawn Oster - January 28, 2008 - Nice write up, hopefully it'll get people rocking on their own personal source control. Setting up a subversion server isn't actually hard, even with security. I even wrote up a nice little walk-thru on it, but a 5 minute Visual SVN Server install still beats my 10 minute manual setup.

Also I'd never heard of TeamCity before, pretty much all my CI work has been done around CC.NET, which of course after setting up a few times seems pretty simple to get going but it sounds like TeamCity has really streamlined the process. I'll have to give it a looky-lu, I'm a sucker for a good looking UI and the ReSharper guys are a pretty sharp bunch. I'm curious how well it interfaces with unit tests and fxcop.

I didn't see you mention how you got it linked up with your mbUnit tests. You *did* set it up to automatically run all your tests right? :) Getting it to build is really only half way there, getting that green light on all tests is the real safety net.

Great write up, I feel strongly that even solo developers should have source control and CI for all their projects and one myth I ofen hear is that it takes too long to get rolling, which as you nicely proved it definitly doesn't!
Simone - January 28, 2008 -
but sometimes you need something up, quick, with a minimal hit to the wallet
TFS costs money, but CC.NET and CI Factory are free, actually they are opensource. And setting up a CI server with CI Factory (as Steve Trefethen said) takes only a few minutes.
Dragan Panjkov - January 28, 2008 - What are main weaknesses of using VisualSourceSafe?
Danny - January 28, 2008 - Hi Rob, Thanks for the post. I actually posted on the same topic on my blog using Subversion, SpringLoops, and Basecamp (http://dannydouglass.com/post/2007/12/Development-Plan-Collaborate-Control.aspx). Like your similar solution, it is a very light hit on the wallet. It does not address CI, but does offer manual builds to dev, test, and production servers. Thanks!
Jay Flowers - January 28, 2008 - If it is a contest to see who can set up a subversion repo and CI system in the least amount of time I am sure that I can win! 1. Download and install: http://downloads.open.collab.net/collabnet-subversion.html (3mins) 2. Create the svn repo (5secs) 3. Download CI Factory: http://code.google.com/p/ci-factory/downloads/list (1min) 4. Configure CI Factory and execute project creator. (3mins) Done! :-P And this gives me all my build scripts too. So I have MbUnit/NUnit/MSTest working with NCover, NDepend, Simian, FxCop, etc. If it's not a contest I guess I am just an ass. :-)
Ray - January 28, 2008 - Interesting post, care to explain why you felt CI was necessary for SubSonic going forward? I like the idea of CI but it seems to be mainly of use to source repositories subject to many changes across many developers, where a build failure would be of high importance. (CI seems very flavor-of-the-month as well.)
josh - January 28, 2008 - Rob, nice write-up.. I've known about TeamCity but haven't tried it because I already had ccnet running nicely. I sometimes get some weird issues though when using repositories like googlecode. I might have to try TeamCity and see if that resolves it. Think you could mention a couple differences between ccnet and TeamCity? Obviously the point-n-click config in TeamCity is a nice thing.. since I don't mess with ccnet for long periods after I get it running, I find I have to go back to docs to remember what I need to do to make a change. -j
Rob Conery - January 28, 2008 - @Jay Flowers: The first CI server I tried was CI Factory (you persuaded me at DevConn). The issue I had with it was the setup - I know that, for you, it's probably 3 minutes. For me, who knows nothing, I gave it an hour and couldn't get it working. I'm sure I was doing something wrong and I don't mean to be negative - but I'd heard a lot about TeamCity so gave it a shot. No competition here - everyone wins and I'm sure your links will help people! @Dragan: SourceSafe is a disaster wrapped in an avalanche coated with explosive powder that sits on your professional tombstone. It's not good... @Ray: CI has been around for years and the reason i needed it for SubSonic is that we have 5 to 6 committers checking in at any given time. If one of them tweaks some code that breaks the build, we address the issue then and there instead of waiting to have build errors, then trying to figure out what happen. Moreover, if you have your CI server run your test suites then you know if a checkin broke your Unit Tests. These "flavors of the month" are good flavors to have :).
alberto - January 28, 2008 - I guess you won't need my help anymore. You didn't let me even get started. :D
As someone else said, now get those lights green!
@Ray Building is just a small part of what a CI server can give you automatically. Unit/integration testing, code coverage, searching for API changes, code similarities,... and whatever you get to plug into it.
Laurent Kempé - January 28, 2008 - I had the trial of Team City in my task list for a bit more than one month. I have CC.NET installed and configured, it took some time, it is extensible but then you really need to dig in the durt. So after reading your post Rob, I downloaded Team City and installed it in 10 minutes. Then it took me another 30 minutes and 4 build with error to have it runnning with NUnit tests. Great really easy.

No the question is how can we use other tools like NDepend...! This will need some investigation more.

But to start Team City is really good.
Stephen - January 28, 2008 - Subversion might be easy enough to setup, but I really cant stand using explorer for checking in the project.. for one, the project you see in the IDE isn't exactly whats in the folder.. VS for example, where files get related if they have different extensions.. tons of things..

IDE integration for most setups seems the much cleaner solution..

I'm just really glad I don't often end up working on projects that need source control like this, because I get really frustrated when things start to clutter.. which, ironically is what comes into my head when I think of source control and subversion..

Not saying this is a bad system, I just think its probably the one you would pick out of, getting your knackers squashed, having your fingers bent back, or hitting your head into a metal post..
Kevin Isom - January 28, 2008 - Rob, An alternative to Visual SVN Server if you don't want to have apache installed is http://svn1clicksetup.tigris.org/ sets up everything you need on a windows box, not as flash admin wise as Visual SVN but it get's the job done. Also for Visual Studio support for free try http://ankhsvn.tigris.org/ though it is nowhere as good as VisualSVN (worth every penny) cheers
josh - January 28, 2008 - i personally think you should always you source control, unless its just toy code. There are other places, but you can get a free account at www.assembla.com with subversion and trac and more. It's easier than setting up your own subversion repository.
Janus - January 28, 2008 - Can TeamCity be setup to create installers to hand-off, e.g. .MSI files?
Jay Flowers - January 28, 2008 - Bummer. :-( I would like to know more about your experience so that I can improve CI Factory.
Dave Savage - January 29, 2008 - Team "Fortress" Foundation Server HAHA!
I love it!
West - January 29, 2008 - Thanks for this!! I've been looking at the idea of having some source control locally and this article is exactly what I was looking for ... spot on! Have TFS at work, and it works well ... except a little slow when doing a big complicated merge between branches :-) Cheers for all you other hints, tips and insights. I've been reading your blog now for a while and I always try and find a few minutes to read each entry. Keep up the good work!! Cheers, W
Steve - February 2, 2008 - Excellent post :)
Dave Neeley - February 6, 2008 - Thanks for posting this! Now to get that information security group to let me run it...
Dave Neeley - February 13, 2008 - Did you switch from the internal teamcity database to mysql? How did that go? Also, what do run for an smtp server?
Rob Conery - February 13, 2008 - @Dave - i did switch over and it went flawlessly. I did have to find the instructions but once I did it wasn't too tough. RE SMTP I just pointed it to my local IIS. Make sure you use an IP and allow routing for it.
Gecko