Hanalei, Hawaii Tuesday, February 09, 2010

My Enemy, My Self

I’m building an application which I’ll talk more about later – but this one’s different from other applications I’ve built: this one’s for me. In the same way teaching your spouse/significant other how to do X is a major pain in the ass, building your own application makes you do dumb, silly things (I think Jeff Atwood might be able to back me up on this one…). This, friends, is a story of my pain to ease your Monday blues.

I’m building an application which I’ll talk more about later – but this one’s different from other applications I’ve built: this one’s for me. In the same way teaching your spouse/significant other how to do X is a major pain in the ass, building your own application makes you do dumb, silly things (I think Jeff Atwood might be able to back me up on this one…). This, friends, is a story of my pain to ease your Monday blues.

Starting Out Seemed So Simple…

I had an idea, and the idea was good. So I decided to build an application around this idea and I got together the various tools and things that I thought I would need. I played with the idea of using Rails (for a change – why not), but then a curious thing happened: my inner Type 1 Ass-kicker decided to wake up:

This aint no disco butthead. Build using something you know, stop dicking around!

He’s a real dick – that inner Type 1 Ass-kicker (we’ll call him Bad Rob) and if I don’t listen to him – or at least make him *think* I’m listening to him, well things will just get ugly.

I ditched my Rails musings and plowed ahead with ASP.NET MVC. I grabbed my favoriteTylerDurden testing tools, updated jQuery with my favorite plugins, and I was off to the races. With one exception – I snuck in a last-minute substitution while Bad Rob wasn’t looking: DB4O. Here are the tools I went with:

  1. ASP.NET MVC 1.0 
  2. Machine.Specifications (BDD testing stuff)
  3. XUnit (love u Brad!)
  4. jQuery UI – full suite, ajaxForm plugin, validation plugin
  5. DB4O – just because.

I’ve written about using DB4O before – I love the thing. Persistence made easy! I kicked up VS and got ready to roll…

Good Rob, Bad Rob

That’s when the fun started. See, Good Rob (the guy who likes to write tests *first*, the guy who tries to vet just WTF he’s going to do and why) would have sat down and cranked out some specs against some type of iteration and time plan. Then maybe tied everything to a nice build server (which is purring away at my webhost) and… yadda yadda yadda.

Whatever Nancy-boy, it’s not like you’re getting paid to dick around here and no one’s gonna see your code! This site needs to go LIVE so get ON IT. Yah testing is fun – test whatever you want – AFTER YOU’RE DONE.

And there it is: business vs. process. The best part is that this Fight Club dialog is taking place right inside my skull, and Bad Rob is playing on my fear of “getting something done” while Good Rob wants to do it right.

Yikes. Dueling success metrics (such a lovely corporate term): “results” vs. “quality”.

Digression

Yes, this kind of thing is common – in fact it’s expected – in every software project ever undertaken. It doesn’t matter the company and in fact it happened to me at Microsoft. I was working with a team right after I started and I was about a week into the work when we had a team meeting and I was asked to “give a demo”.

“I don’t really … umm… have anything to show… oh wait yah I do – take a look at my test results so far.”

And the room erupted into laughter. “Nice! We don’t have an app but we can show our users some great test results!”

Granted – they were teasing me. All the same I couldn’t help feel that the expectation was that I needed to Show Something. Next time I’m in that situation I’ll be smarter and bring some HTML mockups – either that or Balsamiq.

Good Rob Loses

As always, time is short when doing these things. Bad Rob was continuously reminding me that I needed to get this thing done and out and I gave in to the pressure. It really sucks to give in like this, but Bad Rob can be awfully, awfully convincing.

So I appeased him – I wrote the app quickly and cranked out the initial spec, and the decided I would back everything up with a set of tests. I really, really like the BDD approach so I started with that, and I gave myself 3 hours to write as many tests (errrr specs) as I could.

And then all hell broke loose.

I wish I could surround the following paragraph in glitter and flashing lights, because it’s something that I had forgotten by being “out of the mix” for a few years. I was surprised at myself – so much so that Bad Rob started to panic and got really pissed off at Good Rob for writing “such crappy code”. Here’s what happened:

  1. In my haste to write said application, I managed to goof up one of the main specs. I thought I was doing it right, but…
  2. When I started to write my tests I realized this error, and I started to fix it. This “fix” led to me refactoring a lot of “slop” that I had left until later – the “technical debt” if you will that I figured I’d refactor when I was testing.
  3. The slop started to build, and the refactoring started to spiral. I was fixing things that I knew I shouldn’t have done – things I would have dealt with properly had I slowed down. Next thing you know the entire app was in pieces on the floor…
  4. Including my tests. I had about 50 or so written and now I was commenting out my tests liberally – they were invalid and I felt really dumb for writing them in the first place without using them to validate the code I was going to write!

What was supposed to be a 3-hour fun session of writing tests turned into much more and ended with a shouting match between Good Rob and Bad Rob, with Good Rob threatening that he was “going to BLOG THIS” (looks like he went ahead and did that…).

Back On Track

I wasted hours and hours retro-fitting my little application and stuffing Bad Rob back into his id-box. As I say I knew better, but feel prey to the “we need it NOW” bomb that Bad Rob kept lighting off.

What I should have done was this:

  1. Use Tools you know. I like DB4O a lot – and it worked perfectly as a testing “Fake”. I ran into some problems with it (which I’ll talk about later) and what I should have done was go with something I know.
  2. Write out your specs. I should have written out the specs and behaviors of my app and formalized it into a testing suite. At least 20 or so to get going. Adjustments are easy and reading the behaviors crystallizes WTF you are doing in the  first place. If you’re not a BDD fan – then write your tests in “comment form” – just to get the ideas down.
  3. Have some damn discipline. I know I’ll need ViewModels to handle complex UI logic, I know I’ll need Service classes to handle specialized domain logic as well. I’ve built this application many, many times before – I know what’s out there. Yes – YAGNI and all but in terms of structure, there is arm-waving   and then there’s reality. These things I know.

What I ended up doing was:

  1. Moving to SubSonic. I know it like the back of my hand and the SimpleRepository fit the bill nicely. I was also able to add some cool features in (which I’ll talk about later). DB4O is great, but if I’m really in a hurry then I need to stick with what I can make sing.
  2. Stopping everything and specc’ing (how the hell do you spell that word?) out every last element of what I was building while thinking it through from end to end.
  3. Write code *only* when you have a mandate to do so. Yah I’m a smart guy and yes, I can juggle things in my head – I’ve been doing this for a long time. I was wrong, however, when I started to assume.
  4. Stopped worrying about domain names and CSS and didn’t look at the web app for the rest of the day.

The Score Card

For those of you who don’t think “qualitatively” – here’s the breakdown on lost time:

  1. I had a 40% loss on rewriting existing tests
  2. a 25% loss on refactoring things the way they should have been done in the first place (using Services, View Models, etc)
  3. a 40% loss on removing DB4O and going with SubSonic. Caveat: I didn’t *need* to do this but I had a gut feeling.
  4. a 30% loss on restructuring my repository to be less DB4O-specific (using Add/Update methods, removing Flush/Commit). This was dumb.

Overall I would have been 1/3 ahead had I “done things right” – or more appropriately I would have finished in 2/3 of the time. That’s a lot of time!

Summary

The only reason I got so derailed is because I have a major stake in this thing’s outcome. I can forgive myself – just this once – for getting carried away and I’m happy that the damage wasn’t too great.

Bad Rob is happy that the app works and it’s lighter and more nimble than before. Good Rob is happy because his assumptions are covered and he can see the light at the end of the tunnel.


Chance - August 31, 2009 - When I was running my own shop or now, when I am building my personal applications, I have this exact same debate *over* and *over* and *over* again. You would think that Bad Chance would learn, but the urgency of getting "that cool idea" out of your head and into a workable solution is just so demanding.
paul - August 31, 2009 - Fantastic article, Rob. We've all been there.
nightshade427 - August 31, 2009 - Great article! I try to keep reminding myself to slow down and be disciplined. But, I also fall into this trap lots. Keep up the great work.
Luke Sampson - August 31, 2009 - You've got my vote for the most committed Fight Club reference, Rob - actual dissociative identity disorder!

http://www.hanselman.com/blog/FirstRuleOfSoftwareDevelopment.aspx
http://blog.wekeroad.com/blog/youre-not-your-data-access/
http://www.codinghorror.com/blog/archives/000586.html

What's going on here? Love it!
German - August 31, 2009 - Too bad you had to remove db4o =(
Maybe next time!
German
Justin Rudd - August 31, 2009 - I'll throw my "me too" on the pile as well, but with a twist. I have an app that I'm working on that could make some money (but will never ever make me rick or even comfortable, more a labor of love), and I went back and forth for a couple of weeks - Ruby (Sinatra, DataMapper, PostgreSQL, mod_rack, etc.) or C# (ASP.NET MVC, NHibernate, MS SQL, IIS, etc.). In the end, I chose Ruby. Why? Cost. I can get a Linux VPS much cheaper than a Windows VPS. PostgreSQL is free. MSSQL isn't. SQL Express you can't ship on. Nginx/mod_rack or nginx/haproxy/thin are all free. IIS requires a Windows license.

Sure I could use a shared host, but I want a "real" host that I have root access on. I've got a few implementation ideas that wouldn't fly in a shared environment.

Just my $0.02 (which I can spend since my software is free)
joel - August 31, 2009 - You can always use mono, with mysql.. etc.
Rob Conery - August 31, 2009 - I was wondering if someone would catch that! I tend to "hit that" reference a lot. Fight Club touches on a lot of truths about self-deception, and it's hard not to draw parallels. Even harder to try and find a better one.
Chris Kolenko - August 31, 2009 - Umm.. Programmers time is worth more than hosting costs..

If i was to use free technologies i'm going to spend 1/3 longer developing the app.. That's about a years worth of IIS hosting etc..

Server costs are very small these days and should be considered but really don't leave a big hole in your budget..

If you just wanna mess around with different websites that aren't making any money (because it's a hobby) buy a box for home :D

just my $0.02 :D
Justin Rudd - August 31, 2009 - True. But if you know the technologies about the same (which I do), then programming cost is nil. It truly does come down to software licensing costs for me.
Justin Rudd - August 31, 2009 - Mono is nice. I'm actually using mono/C# fed from AMQP to do some data crunching. The problem is the web side of things. XSP2, mod_mono on Apache, etc. aren't as mature (in my dealings with them) as the core Mono libraries. Granted this was around the 2.2.x tree. I haven't redone my testing with the 2.4.x tree.
Khaja Minhajuddin - August 31, 2009 - Completely agree with what you say, I too sometimes think about just leaving the tests and banging out code. For some reason it's very very tempting. I'll keep this blog post at the back of my head, just to remind myself when things get hot. And I like the way you draw parallels with Fight Club, It's one of the my favorite movies.
A really kick ass post Rob! The more I read your blog, the more I look forward to your next posts.
Andrei Rinea - August 31, 2009 - Thank you Rob. We appreciate this useful post.

I've read in the last 3 years a lot of good blogs (including one) which all told us how to do things or gotcha's or...

But very, very few tell us how to **NOT** do things. We need counter-examples too.

Thanks again,

Andrei.
Paul Batum - August 31, 2009 - Thanks for the very enjoyable post Rob, the “going to BLOG THIS” line made me LOL.
Eyston - September 1, 2009 - "working jobs we hate so we can buy shit we don't need"
Eyston - September 1, 2009 - I write ASP.NET MVC at work all day but I'm playing with Ruby at home. I think it is healthy to get more points of view on how to do things -- it will only make me better at my job. Plus learning is enjoyable and satisfying.

The only bad thing is Ruby is making me start to question everything I thought was 'right' :)
Kelly - September 1, 2009 - Fortunately, I read your blog enough that this sort of thing doesn't happen to me a lot - Because I've seen you do it, and if you can't pull it off, I have an iceberg's chance in HELL! Thanks for this post, I'll paste it to the wall in my office as a reminder to do it RIGHT the first time!
Gabriel Florit - September 1, 2009 - Dude this was pretty funny. I can totally relate.
Oz - September 1, 2009 - I've been on that seesaw for a minute as I'm bout to start a lil something this week. Thanks for droppin those words of wisdom and keepin my foot on solid ground! It's always tempting to think you've become such a beast of a developer that planning can take a back seat to your godly logical invocations. But I heard someone say that pride comes before a fall, and in a parallel sense I totally dig your candid transparency. And thank you again for being such a freekin inspiration man! You freekin rock dude. The world's a better place because of you. Pax out.
Raoul Millais - September 1, 2009 - specc'ing --> specifying ;) It's not quite the same tone but you don't get the awkward apostrophe and single/double c dilemma! Thanks for a great post though, I can totally empathise with this.
Eric - September 2, 2009 - Are you having a mid-life crisis?
Tim - September 2, 2009 - Rob,

I recently went through the same decision process...have an idea, need an app. Go with what I know (.Net stack) or give Rails a shot? I started out with, "I want this done well, I'll stick with what I know." But, a day or so later of setting up the "right" things in .Net and I scrapped it in favor of Rails. My procrastination aside, I've been pretty successful with the Rails so far. And it feels lighter than when I was starting up nHibernate, Structure Map, etc, etc. (To be fair, I was struggling with some of my "business rules" that won't be any easier in Rails...I'm a horrible client.)

That said, I do like the main point of your article, in that doing it right will win out in the end. I was at scna (http://scna.softwarecraftsmanship.com/) last week in Chicago, and that was Uncle Bob's message: When the pressure is on, stick to your principles MORE.

I'm glad Good Rob took the time to blog about this.
jeffa - September 2, 2009 - BizSpark can help level the playing field for 3 years. Don't know if you've checked it out, but you can get all your MS software (including servers, SQL, etc) free for 3 years. If you wind up making money you pay for the software you use. If not you pay $100 and walk away.

Still cheaper to have free forever, of course, but if you are starting a biz (SEE how I played off the name!! sorry. Lame I am, yes) then it is an alternative.

My personal reason for sticking with MS is that C# and .Net give me a real breadth of areas to play in using one general set of tools. MVC, Silverlight, WPF, WinMo (well, not so much WinMo any mo...), XNA, etc.

Eyeston, I totally agree that learning different technologies is great, but the problem is time. .Net gives me a lot of bang for my buck (time). FYI, I wasn't always a .Net guy. I was a Java guy until around '05 (and various other things before that).
Chris - September 3, 2009 - Rob - did you run into the db4o object identity issue, or maybe its performance issues with string fields and large amounts of rows? It's nice to just save your objects without any inbetween layer but in my experience doesn't work well where you want detached objects (updating and closing the database).
Dan Finch - September 3, 2009 - I took your advice and tried DB4o a while ago, ran into issues, found NetDataContractSerializer to be easier to work with, and its the same idea.

http://msdn.microsoft.com/en-us/library/system.runtime.serialization.netdatacontractserializer.aspx
Dave - September 3, 2009 - Hey Justing, you mentioned you can’t ship your app on SQL Express. Can you tell where did you get this info from? As far as I know, you can register to redistribute SQL Express together with your app, for example a small scale LoB.
Of course, the price for the host system remains, but the rest can be managed with just around 2 cents also, using a complete MS environment with express editions.
Alex Strickland - September 7, 2009 - Nice, I have a a test project just looking at me, it's stopping me from starting "the real project", I know I should do the right thing, but ...

I know you know the guy, but I think "Bad Bob" has a nice ring to it?
James Coleman - September 14, 2009 - Sorry for a bit of a Necro post, but I have been crazy busy for the past 3 months and just catching up on your blog. Great post as usual and I think we all battle with this. My analogy that I use all the time when a client or my 'Bad Rob' tries to just get things done is: 'We don't want to get our shoes caught in our pants' - for I am sure most kids around 1st or 2nd grade have learned the lesson when coming home from school and they want to get outside and play as fast as possible. They think, I am going to leave my tennis shoes on, take off my pants and pull on some shorts - piece of cake... Until we are hopping on one foot, falling to the floor and trying to get our shoes out of our pants, we end up either going back to how we started or our foot comes out of the shoe - either way we had to take our shoes off anyway and we lost considerable time if we just did it right the first time.
Gecko