TDD: The Road So Far
Friday, April 11, 2008 -
I've had some interesting experiences on TDD Island over the last month or so, and I thought it would be a good idea to share what I think so far. Evan Hoff asked me in a comment on my last post:
SOO..one of the big things with Agile is having a feedback loop. What's your feedback on TDD so far? Did you find the green lights slightly addicting?
I've been meaning to put fingers to keys on this for a few days now, if not just to look back on this post in a year and see the road I've travelled...
I Don't Need TDD
That's one thing I think I won't change my mind on. I've been doing quite well over the last few years, thank you, and the "Sudden Ray of Light" won't present itself in TDD I'm afraid.
I also don't need swim laps at the gym and take my stand-up paddle board out at sunset to blow off steam. I don't need to eat organic for that matter, and I certainly don't need to recycle anything and continue using that stinky mulch pile in my back yard.
The use of TDD doesn't resolve down to a need - it's a choice. Some may think it's the ONLY choice (more on that later) but like every other choice we have in life - there are pros and cons to it.
It comes down to what you want out of your geek life.
Pro #1: I Assume Way Too Much
One of the things that didn't make it into Part 3 was my major blunder with the TestCatalogRepository. I want to lie and say "oops, I forgot" but the truth is, I was embarrassed of this one :). Long story short - my routine for creating each product didn't gaurantee a unique ID - I had a nested loop and I mistakenly used the wrong loop variable to set the Product ID.
How did I catch this? Well, as I was sort of grumbling to myself writing what appeared to be a simple test for the products in the repository - making sure that the repository would only return one record if I asked it to - LO AND BEHOLD it failed. I felt very stupid.
I fixed the error, but when I did, 4 other tests failed. In my fix I managed to screw up the way the category was being set for each product. 10 minutes later and with a very sheepish feeling - I fixed everything and was now certain that I had things working in good order - thanks to the tests!
This is what I realized:
Over 90% of the bugs I introduce are stupid.
I say it every time - "damn that was a STUPID bug". The reason it happens? Haste. Assumptions.
The Pro here is actually a double-goody. One failed test found the bug - 4 failed tests told me I screwed up the fix. I like this a lot.
Pro #2: I Love YAGNI
This really should be #1, but the assumption thing... grrr. YAGNI (You Aint Gonna Need It) is wonderful - I am so very guilty of violating this principle! I stub out code that just sits there, unused, waiting like a puppy with a leash in its mouth that I have to feed and clean up its poop. I don't want a puppy.
How many times have you heard this:
"... So where is this method being used?"
I've heard it a lot.
The refactoring process of pruning out code is really quite fun (not sure why it's so fun to scrape out code). It's also fun being sure that everything you've written is needed, and you can see proof of it with a bunch of green icons.
Con #1: I'll Never Do This Right
TDD is a design process - not a coding process. Anything with the words "design" and "process" unavoidably are spawned from a theory, and invariably there are those who the theory works for who have the "Sudden Ray of Light" and think that you should have this too. And as with every theory there are the workers who kick some of the theory to the curb in favor (in their eyes) getting some work done.
I'll summarize my point by saying that TDD is discussed far more than it is applied. There's no problem in that in general - but what it points out is the uncertainty that everyone has as they search for the Right Way and their own Sudden Ray of Light. Although a lot has been written about the process.
In other words theory abounds on this topic, and in these seas There Be Dragons.
Summary
I'm happy to say I'm really digging TDD in general. The results are very positive, my code can leave the door open at night, feeling safe from my assumptions.
There's no greater Con to me then the way the community has divided a bit, with judgements going both ways (no I'm not singling out this week's discussions). It would be nice if this series allowed for positivity (I've already seen it)!
You are right in that we DON'T NEED tdd. But we do need it. It just helps making things easier. ;)
I think you summarized very well the points why I like TDD so much. Driving design and detecting both stupid mistakes and unexpected collateral bugs introduced when refactoring/mantaining code.
And I also think it's great you are not only spreading the word about TDD but doing it in such a public learning process.
*runs for cover*
There are a lot of heated arguments on both sides, but much of it comes from a misunderstanding of what TDD is. It's what you pointed out: design.
I've talked to plenty of folks that see the value of unit tests and write plenty of them, but don't see value in TDD as they see it as a vehicle for more unit tests. Any time you have a fundamental schism in the interpretation of the goals and meaning of an idea, you're bound to have some serious disagreements.
I'm right there with you, Rob:
blog.troyd.net/.../Test%2bSupporte
Rob - so, how far are you planning on taking the screencast series?
Rob-
I've been enjoying immensly the series you have been doing. Keep up the great work!
I completely agree with you about your take on TDD. Sometimes I wonder if some of the TDD zealots are having trouble with seeing the forest for all the test *wink*
@Jim: I like how you put that process. Unfortunatly I think you need a event handler for when the client changes their mind and adds or deletes requirements. *Big smile*
While people are fond of saying things like "There are a lot of heated arguments on both sides", I have to disagree. From my observation, the heat tends to come mostly from the pro-TDD crowd. I predict that you'll see this happen with this very post. Since the post is largely favorable towards TDD (with commenters already thanking you for this post's TDD evangelism potential), I'll bet you don't get many negative comments (if any). Indeed, if you look at the reactions to your previous MVC posts, you may have noticed that it's the pro-TDD side that has produced all the invective.
If we had an alternate universe handy where you inverted around the pro-TDD axis on this post, I'd lay odds that the response in your comments would be much more heated. Based on my own experiences as a public TDD skeptic, you'd be feeling crispy already. I'm not sure what it says about TDD that it inspires such strong emotion in its adherents, but that tendency isn't endearing to non-converts.
The biggest problem I have with TDD is that you have to know pretty much what you need to do before you do it. For a beginning developer, the kind where TDD would be most helpful, this is damn near impossible.
By the time you get to where you can see your architecture, understand the different parts of the design and know what methods you'll be using--down to the arguments and the return type--you'll find that setting up your tests before coding is a snap. But then, by that time, you will find the benefits to rigorous TDD have diminished somewhat.
I'm a fan of DDD. Well, what I call DDD, anyhow: Documentation driven development. I haven't read any papers on it; its just kind of how my style of coding has evolved.
Sit down and code against the requirements. Flesh out your code and see if your perception of what the architecture should be is valid. As you code, thoroughly document your methods. What are the inputs, outputs, exceptions? Are there prerequisites that must be met? What must be guaranteed about the current application state prior to the method running, and what is the state afterwards?
Documenting this thoroughly helps keep your code simple, keeps unintended consequences via side effects down, and makes sure you won't forget or assume something about code you wrote three months ago.
As you code, your documentation grows. As you document, your code changes (I don't want to do that/I forgot about that/that won't work). Use simple, throw away tests to verify your assumptions are correct.
When you get to the point (kind of midway) where you believe your objects and methods are relatively set, and your documentation is pretty well fleshed out, create tests based solely on the internal documentation. Everything your XML comments say must be true. Plug holes you find with more documentation and more code.
Tests, code, and documentation evolve together in my DDD and become much more closely aligned. It also reflects a more natural approach for development, particularly at the level of the majority of developers (~1-4 years of experience).
Anyhow, my opinion...
While I agree that you can still program without tdd, it just adds a certain amount of confidence/comfort knowing you're all green after you make that schema change. Its also helpful in finding & fixing bugs. You can write or run the test that cover code you are seeing problems with, and then find find-n-fix it. If you haven't tried testdriven.net, do so RIGHT NOW!!!! (just being dramatic) http://www.testdriven.net
@Will
>The biggest problem I have with TDD is that you have to know pretty much what you need to do before you do it.
Not true. R# creates what I think I need as I go. That's the "Design" part. Half the time I have no idea where I'm going, and I'm using TDD to figure it out.
The problem with documentation is that it grows stale _extremely_ quickly and is impossible to keep up-to-date. Documentation is not verifiable (or falsifiable) so I always assume the documentation is lying.
We start our junior developers day 1 on TDD and pair programming. Our results have been great, within about 6 weeks they're up to what we usually see in around 6 months of newbies at other places.
I've done documentation-driven development. The documents are valid for about 5 days at the start of development, then they slowly become invalid. Your DDD works, but it's pretty wasteful. Much of the documentation created has no use. We create some design, but only enough to get us started.
@Jacob
I think most of the heat comes around because the TDD crowd is pretty danged passionate on development. I haven't met anyone that's had an outright negative experience with correct TDD. The only time I've directly taken issue with certain folks (the Runtime people being some) is the spread of misinformation about what TDD is.
Rob pointed out that TDD is design. It's OK if he doesn't like to do it though.
Also, perhaps the heat comes from running into sooooo many legacy apps with zero unit tests. It gets old.
Personally I don't care one way or the other. I like to design using TDD, plus it gives me bonus unit tests. Some folks I know and respect do prototyping and TAD (test juuuust after development). That works for them.
TDD is a _practice_ that supports certain values and principles (Uncle Bob's SOLID, feedback, maintainability, etc.) If another practice supports your values and principles, great! My beef is with folks whose main value is "get it done as fast as I can with no regard to future maintainability".
I'm certainly no expert. But it seems one of the hardest thing about the "test" side of TDD is the maintenance of test code (over a long period) or perhaps the hardest thing is writing tests that's easy to maintain when your not really sure where you're going (the feedback part).
@Will: "For a beginning developer, the kind where TDD would be most helpful, this is damn near impossible."
TDD is for everyone. The hardest problem for beginning developers is how to design their software -- and the fail at it quite a bit. If you follow TDD you will end up with better software because it forces you to do better design.
@Jacob: The only TDD skeptics I've found are those who don't want to start. The journey is worth the effort.
Start off with what type of testing you are doing now. If it involves a debugger or your applications GUI then you are only testing the most common scenarios, and often just once. Plus, you are waisting time, lots of it, to get your application into a state to test. Usually involving setting up special data in the database, loading particular screens, etc. So you are only testing what you have to, then often not testing at all after that. Or minimally anyway.
Can you really sit back and say that the amount of testing that you are doing in that way is good enough?
There is a path that the software development community has taken (which is now littered with broken dreams and failed projects) that have led to these techniques. The sad fact is that we (programmers) make a lot of bad software.
TDD doesn't fix everything (and it doesn't replace anything), but it does help fix a number of problems.
I am an admirer of you guys, and new to TDD. Have some thoughts on the purist discussion, and would like to share.
I like TDD because it helps refactoring, and I don't find it to be that helpful in my design. In refactoring and debugging, I frequently have to change existing (and green) code. At that moment, it's very un-nature to Stop-->Find the Tests-->Think about Tests and Maybe change some of them-->come back to where you were.
In my experience, test and development take turns to drive one another. Whenever, I have a problem, I will let my thoughts flow, and solve the problem at hand. Later, when there is a red, I then stop (have to anyway) and look at all related code to figure out whether I need to change my code or the test. I am sure everyone changed test code sometimes, right?
It's probably more of a personal preference, I don't find my project has to be driven by test. I do try to give test higher priority in my mind, because my old habit is much deeply rooted.
Cheers,
Still on the TDD fence...
I can say, however, that if I tried it out and brought my code to show and tell and received the kind of criticism leveled at Rob recently, I would pack up my toys and go home. Call me sensitive. Rob is a bigger man than I am. Note to the TDD crowd: sometimes you make TDD as attractive as "advocacy" does for Linux on the desktop.
I didn't find TDD helpful until I started "spiking" everything first. My spiking is a little different in that I implement most of an object and its relatives (interfaces or actual objects) first. Then I start writing tests for that object, focusing on a specific feature. As I write the tests, I start refactoring that object (sometimes that specific feature, sometimes everything). In the end, it coalleces into a solid design and I now have a very good regression suite. I still don't know how you can use TDD effectively without extensive spiking first.
Maybe that's just my way of purging assumptions and preconceptions of the problem domain...
Rob, I'm very impressed with your civil and reasoned response to the criticisms you have received lately. You have done an excellent job of outlining why people choose to practice TDD. Keep up the great work.
++Alan
I've been trying to get on the TDD bandwagon. I definitely think it helps avoid writing spaghetti and there is happiness with good coverage and all green lights.
Lets face it though. Maintaining all those tests can be a pain in the ass.
I can't help but think that someday some clever person will figure out how to generate and execute useful tests dynamically and continuously... Something that executes in the background of VS that will automatically construct tests and execute them as we're coding... Something that shoots for complete coverage and can make some basic assumptions.
There's gotta be a better way than constantly maintaining all these tests by hand.
@Dietrich - I've found the same problem. In the end, I think it often comes from writing the tests after the code... I've blogged here about it.
www.bottleit.com.au/.../Loosely-couple-
One you can decouple that, the tests stand up to much greater change.
It would be interesting to understand better where in your examples of the PROs of TDD you were actually benefiting from TDD vs. just testing WHILE developing; it sounds to me like all your examples of the benefits are about having a coherent set of unit tests and it wouldn't have mattered if you wrote them 'first' or 'later' -- you would seem to still get the same benefits (e.g., your single bug-fix breaking several other tests showing that your bug-fix wasn't correct).
So much of the 'gestalt' of TDD seems to be focused on the (somewhat) arbitrary idea of red-green-refactor that I really feel it would be best to rechristen TDD as test-DURING-development instead of test-DRIVEN-development; the whole 'write a failing test first' thing strikes me as just so much intellectual masturbation in many cases except for the truly poorly-disciplined developer who cannot otherwise force themselves to write the tests that they really know they should be doing~!
I hope you continue the series with TDD. I'm not convinced yet and after watching your latest, I'm even less impressed with TDD. There is a cost to writing the test first: you end up doubling (tripling, or more) the work of refactoring the target product. The more tests you write up front, the harder it is to refactor. That must lead to some poor refactoring decisions.
I see that writing the tests first naturally leads to using the tests as a design process. A costly one with the real possibility of unintended consequences.
Will you document the amount of effort that goes into "correcting" the tests as you go along?
Rob,
Sorry, but I feel a need to chime in on something. I promise not to go too far.
I just feel a need to respond to something I saw here.
--------
I was *SHOCKED* to see my entire blogging community (theruntime.com or as stated here "the runtime people") bundled as a bunch of non-TDDers (considering that I believe at least one of our bloggers is quietly a TDDer).
That said I would echo Jacob's thoughts. When I was over on CodeBetter, I have put up posts to discuss my attempts at TDD. I found exactly what you have found here (TDD gives me quick feedback on broken code), but I felt like I was burning a lot of time to get little difference in coding than what I was already doing (without TDD). When I blogged this I was shouted down and told (as I have read here) that I must be doing it wrong or that I just didn't truly understand it (BTW, I have been studying the whole TDD space for about 4 years now, and still I supposedly don't get it). I was also told to quit analyzing it and just do it (I hear this a lot politically as well "don't think about it just believe what I tell you"). This approach by TDD evangelists end up souring me on the whole methodology and probably ultimately led me to move on from CodeBetter and start my own community.
What people don't realize about me is that my personal approach to development of products for clients is a combination of TJAD or TDD (depending on how much I'm pondering the problem or how much I'm in the coding zone... TDD seems to work as I'm contemplating a difficult problem... TJAD keeps me honest when I'm in a coding frenzy).
Anyway, I'm probably asking too many questions and I probably am just another charlatan who doesn't get it.
I've never used (nor even really understood) TDD (or Agile for that matter), but thanks to this series I think I'll be swimming in these waters very soon. I love the example of a simple bug "fix" breaking four different things. How true... how true...
I also agree that I find myself writing a lot of methods "just in case", especially overloaded signatures to constructors and methods. I need to start forcing myself to only write what I need when I need it. Maybe be Agile and TDD can help.
Thanks Rob, looking forward to more of the series.
Hey Rob
Keep up the good work - I am playing with the new MVC bits, and as it supports TDD - I have been writing a sample app following the same approach.
It is cool to see all the green lights - and then when you do the normal "web testing" it just works!!