Announcing AutoTest.NET 0.1

Wednesday, November 07 2007

Over the weekend I was at RubyConf and was introduced to an incredible little tool called Autotest. Autotest is a little command line utility that sits next to your editor and then runs all of your tests every time you save a change to your files. I downloaded and started using it, and it changes everything. It makes test driven development feel nature, and it kicks you around if you stray off the path. It's kind of like personal continuous integration.

So, I decided to write one for .NET. This first release (release early, release often!) basically watches an individual folder, when you change a .cs file, it builds your project, builds your test project, and then runs your tests. Right now its a console applications, but eventually I will do a system tray application and GUI for it. (although I like the console application myself)

Check out this screen cast for how it works.

If you want to try it out, grab it from google code and give it a try. This is very early stuff so it might blow up or stop working at any time. :)

-James

kick it on DotNetKicks.com

Comments

#1 Scott Bellware on 11.07.2007 at 6:09 AM

James,

This would be a great thing to use Snarl with.

http://www.fullphat.net/

#2 JAmes avery on 11.07.2007 at 6:29 AM

Yes! Definitely. I was going to use regular toast, but this looks even better.

#3 Mike Minutilllo on 11.07.2007 at 6:53 AM

Hey James,

This is very cool and nothing has ever made me want two monitors more. Just FYI I tried running it in VSCmdShell but it didn't work :(

Keep up the good work. I look forward to seeing the final (is there ever a final) product.

Mike

#4 kevin on 11.07.2007 at 3:05 PM

Makes so much sense not sure why it took so long for it to pop up on the .net side. Great stuff.

#5 JAmes avery on 11.07.2007 at 3:40 PM

Mike,
Looking into that right now. Thanks for the heads up.

-James

#6 Simone Busoli on 11.07.2007 at 7:05 PM

Nice project James, but as far as I've experienced a build server can be run locally as well, and it's not so difficult to setup with CruiseControl or CIFactory. Are you trying to compete with them? ;)

#7 JAmes avery on 11.07.2007 at 7:07 PM

My goal is to be much simpler than setting up a full builder server. Most build servers also watch source control, not the actual modification of your files.

#8 Simone Busoli on 11.08.2007 at 2:17 AM

Well, but you really want to build every time you save? I think I'd prefer going with a local source code repository and a build server with minimal configuration. Building at each save look like a too much continuous integration :)

#9 JAmes avery on 11.08.2007 at 2:54 AM

Yeah, thats exactly what I want. :) When doing test driven development you are either writing your test, which you shoudl run right away, or writing code, and you should run your tests right away. :)

#10 Mike Minutilllo on 11.08.2007 at 5:09 AM

I can see this being used to teach TDD to uni-students in a lab using the Free Visual Studio Express Editions where you don't have Source Control integration. It also could be used to enforce running tests on newer team-members for smaller projects where TFS policies aren't available.

@Scott Bellware - Snarl is awesome but the installer is missing the C# libraries :(

#11 服装加盟 on 11.09.2007 at 12:01 PM

good

#12 Steve Bohlen on 11.09.2007 at 4:23 PM

Hmmm...this is certainly kind of iteresting, but its not at all clear to me that one would want tests to auto-run as soon as a single .cs file is changed/saved.

If I'm making a change to a .cs file, its entirely possible that I need to change one or more others too before I'd want to bother running my tests.

Sort of the whole point of a CI server watching the SCC repository instead of a local file system (as a previous poster mentioned) is that in this way the developer is in charge of when they have made changes that are a cohesive whole and are in a state worth checking in and/or running tests against.

As I'm understanding the way this tool is described as working, it sounds to me like a recipe for like 100 failing test runs that are going to be largely meaningless to me since they are run against code-in-progress instead of code-ready-to-test. Wouldn't a user in such a situation spend more time trying to determine which test-run failures were meaningful and which weren't...?

Am I understanding this tool correctly? It seems to me its proceeding from a really false assumption that file-save = ready-to-test, no?

As its mentioned that this a .NET-targeting version of the same idea that's in RUBY, maybe someone can expand on how the RUBY community gets past this hurdle since its seems like they would experience the same troubles (I admit from the get-go that I know nothing about RUBY).

#13 JAmes avery on 11.09.2007 at 5:29 PM

Steve,
This tool is definitely not for everyone. People who like to make changes to large groups of files, then check those changes in, then run their tests would HATE this utility.

This tool is for people practicing TRUE test driven development. In real test driven development you write a test, run the test, write some code, run the test, repeat, repeat. For this mode of working this tool works great.

-James

#14 Steve Bohlen on 11.09.2007 at 5:56 PM

James:

Interesting point(s); I can see how this would be true in the pure TDD scenario you point out (avoid the need to keep firing off my unit test runner after ea test-code-retest cycle).

Although I do often use a somewhat TDD approach, I don't fall quite as firmly into that camp as I might need to in order to enable me to leverage your tool better.

I think I will check it out tho -- perhaps it has a place in my toolset arsenal even if its not always left running while I'm working.

Neat idea, BTW, and thx for the clarification.

#15 JAmes avery on 11.09.2007 at 6:05 PM

Steve,
What I noticed using the ruby autotest is that it actually helped me get better about TDD. I would sometimes go for awhile without running my tests, or get out of the practice on a particularly tough problem, but seeing the tests run right away helped pull me back.

-James

#16 Andrey Shchekin on 11.10.2007 at 12:04 PM

Cool, I have thought about it just several days ago. But. The solution is not compiling due to Class1.cs reference in AutoTest.csproj (I am to lazy to submit a patch for it).

By the way, could you give me a submit access to the codebase? I can not promise that I'll do something, but I promise not to do anything stupid. My google name is ashmind.

#17 Haacked on 11.10.2007 at 10:55 PM

It certainly creates an incentive for keeping your tests small and fast, both good TDD practices.

I like it. Use a stick to keep the TDDer in line. ;)

#18 Ryan Davis on 11.11.2007 at 1:46 AM

Why not just build on top of autotest?

#19 JAmes avery on 11.11.2007 at 2:07 AM

Ryan,
Thanks for leaving a comment, it was your hurting code talk that first introduced me to autotest. I thought about building on top of autotest, but right now ironruby isn't ready to handle something like this and I didn't want to make ruby a requirement for running autotest.net. I know that sounds silly, but most .NET developers would shy away from installing ruby to run a small utility like this.

(I will look into it again though)

#20 Si White on 1.07.2008 at 12:42 PM

Good Stuff

#21 Scott Hanselman on 3.12.2008 at 5:18 PM

Finally, something for my QuadProc to do. Good stuff!

Leave a Comment