Kona: Continuous Integration and Better Unit Testing
This screencast went from a simple idea - talking to Brad Wilson about better Unit Testing - to some pretty broad topics such as source control, unit testing, and continuous integration. I seriously could have written a whole book based on the stuff flying through my mind.
Tuesday, April 07, 2009
This screencast went from a simple idea - talking to Brad Wilson about better Unit Testing - to some pretty broad topics such as source control, unit testing, and continuous integration. I seriously could have written a whole book based on the stuff flying through my mind. I reigned it in a bit and decided to focus on two things that I don't think get enough press: writing a good Unit Test coupled with Continuous Integration. It still came in at 50 minutes - oh well. I think it's all good.
What I decided to do was to answer the following question:
Why should I care about Unit Tests (aside from being made fun of)? Why should I care in a Business Way?When talking about Unit Testing (and TDD for that matter) most people will say "yah I know I should be writing more tests." in a "yah I know I shouldn't drink so much beer." sort of way. This isn't really a healthy thing - you shouldn't feel that you need to write tests to be a good person (though a few folks might say so). You're a smart person - and I'd like to respect that and see if I can appeal to your business mind. What if I told you that if you spent some time mastering Unit Tests (coupled with a Build Server) you could:
- Code in less time
- Prove you're finished (bug free)
- Come in under budget
- Sleep at night
Again excellent screencast.
I have small questions, in your code you have:
StructureMapControllerFactory.cs
KonaControllerFactory.cs
which are exaclty the same, what is the rule of avoiding duplication of code. Is there a tool out there that can link user stories to unit tests?
Thanks and keep up the good work
Yaz
But as soon as you introduce a team you obviously increase the chance of mistakes being made. Having a good set of tests to run, quickly gives you peace of mind (and a good nights sleep!). They become even more valuable when you have 2,3,4 other people working on the same project.... and even more so if you team is rapidly growing
not too sure.
Currently I'm using CC.Net, but TeamCity really looks promising.
Thanks for any explanation and a really good screencast BTW :-)
TDD is about:
Design
Executable documention
Continuous Integration.
Funny you mention Debug, why do you debug?
Because you do not know what is happening. This is a time when you needto write a Unit Test.
Most often if you cannot test something it is because the code is not well designed.
Cheers
Yaz
Unit Testing is vital when there are multiples people working on multiples project. It's not about proving oneself. It's all about quality control. Human are prone to mistakes; now if we take that to a team environment that error factor quickly become a factorial.
Speaking of business requirement then Unit Testing are one way to make sure that we are, at least, satisfying the basic requirement.
Now TDD is another story, I think it's a nice design tool for a small module but for a large project it could quickly turn into an architecture nightmare.
I ask is Unit Testing :) but I'm glad you found it useful!@
Thank you very much for the excellent presentation. It will help me a lot to convince my bosses :)
I was pleasantly surprised that you are using TeamCity - so am I for the last year (currently in the third big project). Before that I was using different continuous integration tools - CC.NET and CI-Factory.NET. They are OK, but TeamCity is a real improvement.
TDD is a hot topic in my team for quite some time - as you said many times, it does require a different mind set. It also not very easy to push into the project plan, etc. As someone mentioned here, we're paid for the "productive" LOC, not for the unit tests. Again, your presentation will help a lot!
In our current project (SubSonic 2.1 based, btw, we look forward for 3.0 please, please) we are using TestDriven + MbUnit + Galio but mostly as a regression insurance. It takes time and efforts to switch to TDrivenD...
Thanks Rob & Brad! You guys are great :)
experience. If you don't believe me, check this out:
http://code.google.com/p/subsonicproject/source/list
I rapid-fired about 40 bugs today and I feel great that the patches sent in
didn't break anything. I added tests for each feature and I moved through
this bug list at lightning speed. It really doesn't matter the size of the
app my friend :).
Test Driven Design is a development technique:
http://www.agiledata.org/essays/tdd.html
If you are so "smart" why you don't identify yourself? :-P
Brad Wilson's tip about naming methods was great too.
Your example about Subsonic is what I am getting at, your code/component is used for a much bigger picture so of course you would write tests for it. The people who use it would be pretty PO'd if you gave them buggy code. So the need for testing is almost a requirement.
How do you know that people don't already say "Dude, Will is so professional - you can't argue with these results and he only writes tests 25% of the time!". Just ask me people say it all the time ;)
it for yourself and see that it can work well for you (despite the weirdness
of it) - well then there's not much I can offer :). And that's OK - *as long
as you're still testing your code*
That's my ultimate point - not TDD but Unit Testing :)
I remember sitting in my local ASP.NET Developers meeting in NY City and watched a bug filled demonstration of app, point of which to demonstrate IntelliSense given by a purported member of the Dot Net Team. I wish I remembered her name. I just never thought I would ever recount the occasion. That room of about 150 in attendance, (if I remember correctly they were giving away a copy of Visual Studio to all in attendance) irrupted in applause for at least 5 minutes. (and I am not exaggerating). The domo needed no further explanation. It just maid sense I don't think anyone use Notepad for code development since then.
The point is, the resistance to this TDD is not because it is not a good idea. It's not because there isn't huge benefits to it's implementation. It's not because the developer is not serious about his work. The resistance is because it is cumbersome and tedious to implement. The developer has to implement it at a cost to his bottom line for no benefits for himself but for the satisfaction. Inside an organization where application are written for resale there as an assessed cost benefit structure for quality control and all the benefits mentioned in this blog. This implementation of TDD makes sense in these kinds of operation. In an operation where the developer is under the gun to meet deadline for new tools and keep the operation up and running and make a profit we need a solution that needs no explanation.
I know, with all my responses it is probably bordering on abuse. But I have seen this pitch all over the place by our elders and I think it is a little coordinated and a little preachy when there is a long history to show that the community responds in a positive way when the implementations need no explanation.
know who you are and that you're having some fun with me... could be... not
sure yet... getting there...
If you are serious - well it reminds me of about 100 different projects I've
picked up written by you: Rewrite :).
Inexplicable...a so nice guy works alone. No employee, no colleague, no team. A big loss for the world and the community!
XD
that's was awesome! I'm a big fan of CC.NET but TeamCity is already in my VPC too. Please make more and more and more and more screencasts!:)
Bests and really congratulations!
If your goal is to prove to someone quality of your code then by all means unit testing is the only way to do that. Most of developers are not hired to do that, rather they work to satisfy business requirement and support some business process. You can create set of tests that validate that but proving the quality of the code itself is not most of customers are willing to pay for.
Good points - the Return key is your friend also...
As far as I can tell you're more interested in being a dick and are trolling - tell me if I'm wrong.
Very good introduction in how to get started with TDD, forwarding it to our internal mailing list. I like the fact that it almost looks like you went out of your way to get non Microsoft products in there :) even the browser is Chrome.
-Mark
-Mark
WOOOH! talk about long winded
mind changed rather dramatically.
First, however, yah - I (used to) have 3 paypal.com domains as feathers in
my hat, my little pride and joy :). No, I didn't sleep - I was worried about
a lot of things. Yes, a complete set of tests would have made me feel better
about every push - so I do know something about Big Domains.
I won't lie to you - it takes time to get the hang of proper testing and to
get good at it. Once you do, you start to realize that tracing the
"co-incident" bugs you introduce is cut by an order of magnitude. So it's
not really up-front costs that you save, it's maintenance.
For instance, I'm adding lots of patches to SubSonic right now - every time
I do I run my tests. When I get all greens my level of confidence is pretty
high. I then added another test or two and make sure the patch fixes what it
said it would then I'm done - and I feel great about it.
Without tests - I wouldn't, and I'd spend a lot more time worrying about it.
The main thing here is that people like this stuff for a reason - and it's
not because we're greedy guys who want to charge more. If you've run a dev
shop then you're familiar with a developer saying "this works". Let's say
he's your ace developer - he rocks. How do you know he's done? Because you
trust him? Is that a good statement?
I trust every developer that's worked under me - it doesn't mean that I
shouldn't check their work. You need to be sure. If you see the results of a
set of tests and code coverage - you're sure.
So - it sounds to me like you're not a testing person - and that's your
choice. To me it's a maturity level that you reach in this profession when
you, quite literally, start taking your work seriously. You might bristle at
this - and you might think I'm being mean :). But consider, for a moment,
that we're talking about you proving the quality of your code to others.
This isn't a game of "I know Will - he's good people" it's a game of "Dude,
Will is so professional - you can't argue with these results!".
All I ask is that you try it and see. I promise you that if you give it some
time you'll have a change of heart.
With writing Unit/Integration tests isn't that like writing 1.5x the amount of code? Better yet I have found that some Unit/Integration tests have 3x more code than the actual code being tested. So I am not aure I believe in that one based on experience myself
"Prove you're finished (bug free)"
The code is bug free but in my experience it's not bug free until it is live and running without issues or do you run the Unit/Integration tests on the live environment too?
"Come in under budget"
Does that include the cost of the Unit tests? Try and explain charging 1.5x more than the next guy so you can write tests to make sure your code works, tests that the end user probably sees no value in no matter how many times you explain it. Shhhh that is a consultants dirty little secret...
"Sleep at night"
You have never maintained a site that gets 20k+ unique visitors a day and have rolled out updates every day for a week have you? No amount of Unit/Integration testing lets you sleep at night. Every environment is, well, just different and for some reason it usually never goes your way.
But seriously there are good and bad reasons for doing / not doing Unit/Integration testing, the bottom line is what kinds of code are you testing? Do you need to write tests to prove your database call is going to work or the file is going to be written? Probably not, however logic frameworks, business rules, CPU bound code should require this kind of testing because it is most often used in a bigger system. That should be tested automatically. The rest; meh.
(which I sort of was :). Given that i was up to 50 mins i had to cut it :)
As I said, you need those things because you either have clueless devs on project or just not so experienced ones.
I see incompetent and lazy developers hiding behind TDD, unit tests, patterns and refactoring... I have nothing against these techniques but everything has its place. You do not need 100% Unit Test coverage if you have competent people on project...
Peace brother!
Developers, give me some feedback as to how I can improve my practices: http://www.deepelement.com/blog/post/2009/04/08/Feature-Crew-Model-for-Small-Agile-Teams.aspx
Good luck with the next 20 years. If this approach works for COBOL why not other languages?
You also helped prove a point. Testing has more value with the more people you have on a project to ensure someone doesn't break the code.
really makes me want to listen closely to everything you say! Especially the
last part!
Is it possible to download the screencast and then listen to it on the train or plane without being connected to the internet
TIA
Yaz
I've looked at CI factory in the past but got too annoyed with all the setup and configuration. Team City seems to make this stuff much easier.
In the future are you going to explore having your full build deploy the application as well?
http://stackoverflow.com/questions/727181/asp-net-mvc-system-web-compilation-compilationlock
The hardest part for me is the start. Since my last project I've read Kent Beck's book on TDD. I feel like I need to do more diligence in defining exactly what I need. Previously I've kind of just picked a nugget of business logic without really thinking about where it is going to fit in which is I think why my tests haven't scaled well. I'm going to try to focus on the inputs and outputs a bit more -- what the user needs displayed, what I need from the user, what persistent information needs to be interacted with. I don't know if this is right or not, but I like the idea that tests as defining the program requirements and those types of things seem very requirement related. (I guess this isn't what Kent Beck did at all with his example on money/currency... he started with a simple piece of logic... maybe this is a bad idea!)
I guess I had hoped I could just read about this stuff (blogs / books) and be an expert but it is taking practice and experience (shocking!). Even if I haven't had a real successful implementation yet, I can definitely tell that it is impacting my code in beneficial ways and I'm improving.
Anyways, I guess that is a long winded way of saying thanks for sharing all of your experiences as well.
Developers are getting lazy. Instead of thinking about what they write and debugging it while they write it, they write unit tests. Instead of thinking of what are they trying to accomplish they constantly refactor.
If you are working on a team than you do need unit tests. The quality of the people on team vary and unit tests serve as protection from low quality developers. In Microsoft you certainly need it since very likely you will not be working on v2 of that code anyway and next developer working on it surely will not understand what he is doing...
As testimonial, I have code base of 300,000 LOC that is being improved constantly in last 10 Years by me alone. This is used by more than 10,000 people on daily basis. I have ZERO unit tests. Everything works very well...
most familiar with - haven't used other CI servers.
Let's say your client says \"I think I want to make sure that instead of having an email come at the end of an order - it should come at the beginning and say 'thanks we're putting it together'. Then, we'll take the Order offline and charge it when we can instead of synchronously - this way we can save time/money\".
OK - so you need to change your Order Process here big time. You sit down, think it through, then start refactoring. While you refactor you keep running your tests to make sure your changes don't crater the rest of your app - if they do you might want to look at why!
When you're done you run everything through your tests again (having killed some, changed others, added more) and guess what you have...
All in all, you'll come out ahead. I can't emphasize enough that everyone lives under these restrictions but still find a way :)
That said - if you use something like Linq to Sql or ActiveRecord (forthcoming in SubSonic) - this isn't a concern. It just depends on how strict you feel like being.
My question is... do you plan to change the way how you implemented data access logic in Kona?(currently in partial classes you have static methods) Or maybe you want to keep it in such way? why? do you plan to test it?
thanks in advance for your help!
Thanks!