Automated parametrized testing or testing without the repetition

The world of software testing is huge. Unit testing, integration testing, system testing, user acceptance testing, QA testing, load testing, DR testing and so forth. And good luck on finding two people who agree on the definitions of each testing category...

From a developers view, the water starts to get muddy on the fringes of unit tests. Basic unit tests are easy, and if well designed, more advanced tests with mock objects or stubs can fill out the unit test suite to the powerful and respected tool that many software development maven's proclaim it to be.

But then you find yourself in integration testing territory - where the options and pathways of what could happen tend to grow exponentially with each layer. And we developers love layers and interfaces. So sometimes integration testing to pull out error conditions and alternate pathways can be tricky, and perhaps redundant if our unit test suite has covered it.

But one thing I never really use integration tests for enough is automated "poke" testing - grabbing a whole bunch of data and firing it at the server to see if I can break it. Normally this requires live data and a little business understanding to do well, not often things developers might possess ! Tools like FitNesse are built around this principle. But as a developer I still find myself prodding software manually, rather than automated this "just bung a load of data in".

One of the historical reasons for this has been a lack of availability of parametrized testing frameworks or tools in the open source space to make this easier for me. But with the general acceptance point that JUnit 4.0 is "standard", I can make use of the parametrized test runner that provides this functionality. It's not too clean (the TestNG approach looks a little cleaner) but it seems to do the trick. Now when I get new test data, I can just throw it onto the stack and whenever I need to sanity check my code just inject all the test data through the relevant part of the system in one click.

In many ways, this approach is obvious. Child's play almost. But sometimes I can get so buried in the wide sea of information on Java and programming in general that I fail to stop, take a couple of minutes and reflect. It's only then that such things become highlighted as so obvious, and with that reflective knowledge I should be able to build up a powerful set of test suites. Just don't ask we which of them are integration tests and which are systems tests...