unit testing – Fredriks anteckningar

Tests – Silver Bullets?

A colleague wrote:

I have a client who needs to convince their old-school management that unit testing has value. I know. I know.

and he got some good material to work off. I myself recalled a very interesting session on CITCON in Budapest where a person playing the role as CTO asked us all to convince him that they needed pre-production tests (such as unit tests). Their (the company’s) experience was that they were doing absolutely fine without it, since they had close collaboration with the customer throughout the development process and lots of eye balls looking at hourly reports on how their web site(s) performed in terms of business.

In the rare cases where a bug went through and caused “damage”, the sales and support people had turned the unfortunate events (such as selling some Gucci or Versace type of bags for 1£ each, instead of 1000£ because of an unexpected comma (thousands delimiter) in the in-data file) to something positive and a marketing “event” kind of thing. These kinds of things, bugs/errors, didn’t happen very often at all and their customer base were actually mostly positive about when it did happen as they saw a company that in very short time responded to customer feedback, were open and honest about what they did to correct the situation etc. They had lots of in-production metrics/monitoring in place and people that closely monitored their systems. From a tech point of view, they seemed to have 0 issues. Business was happy with time-to-market etc.

The session was titled Do_we_need_tests_on_planet_retail? but no-one seems to have taken notes (or at least post them). Essentially, in this case we concluded that the only valid reasons to spend time writing pre-production tests were:

a) if the developers wanted to, then they should,

b) if the organization wanted to attract developers that wanted to write pre-production tests, then this could be encouraged by the organization as means of helping the recruitment of such individuals

c) if they started to get quality problems that they couldn’t sort out with whatever other means of quality improving measures they’d try (more code reviews, code analytics tools, …), then this is one route/option they could try out to see if it would help them fix that problem (but as of that day, they didn’t really have a quality problem)

We tried to pursue the argument that “tests/regression test suites” documents expected behavior of the system and helps people new to the code base not break features. Being strict with pair programming and not touching new areas without pairing with an “old timer” would ideally solve that too (and they didn’t have problems retaining their personnel). Risk obviously is a factor, “X people gets hit by a bus”-scenarios and such.

Teams of about 20 people in total doing development and ops. More people on community facing activities and sales. I understood they bought assets from companies that had gone bankrupt, and sold the stuff on-line. They’d been doing that for several years and were very profitable. Most of the work seemed to setup web sites and/or campaigns targeting certain audiences (high profile/expensive jewelery, football fans, 2013 Olympics Gadgets, …) and get statistics out of that to see how profitable the bought set of assets turned out to be. As such they monitored, from a businesses point of view, the system closely to see how the market responded to various on- and off-line activites and events.

I wouldn’t personally want to have my evenings with the kids depend on on that (systems with zero pre-production tests) but if/when what what you’re doing is mostly routine (similar things had been done many times) the value of writing the same tests over decreases as they become assertions of not typing the wrong thing. Good pair programming should save you from most of that.

TDD:ing in order to design the code, perhaps differently from previous iterations, is another subject. But in short – since the business was happy with what they were getting, and the developers lead happy lives too, it was hard to find reasons for them to have to change.

JsTestDriver

Most of my code writing time last week writing was spent in JavaScript land (or ECMAScript, wonder it the most commonly used name will ever change). I was to write a small templating package and had to write some new stuff and set out to use JsTestDriver and after a few minutes I was up and running in Eclipse.

The idea is OK I guess – run tests and production code in the same environment it will run in when deployed. There are a couple of cons to JsTestDriver (used through Eclipse) though:

if you’re configuration file doesn’t include the test file you’re trying to execute, there’s no clue to what you’ve done wrong – the panel in Eclipse will just show the green bar and 0/0 tests run.
if you have syntax (or construct like) errors in your unit under test, you’ll get the same behavior as above
sometimes you’re served a cached version of (some of) the JS source files – hence there’s a button to “refresh browsers” and it’s sad that it has to be there and that you actually have to use it
sometimes the engine gets into some strange state where it dryRuns the tests instead of actually executing them

Apart from that, its behavior is quite predictable. Minor issues:

running a test complete locks Eclipse’s UI thread
a Rerun quick command option would be great (Ctrl+3)

JsHamcrest

Another minor issue is that the “index” of JsHamcrest matchers is really hard to get an overview of. Whenever I wanted to use a certain type of matcher, it was really hard to quickly tell from the documentation if it was there or not. Go see for yourself and look for a matcher that mathes any object for instance.

JsHamcrest.Integration.JsTestDriver();

# get a list of matchers:
curl "http://jshamcrest.destaquenet.com/modules/matchers.html" | grep "dt id" |cut -d '"' -f 2 |cut -d '.' -f 3

JsMockito

JsMockito works really nice. The only thing that could’ve been better is the integration with JsTestDriver: when a verify step fails, you’ll get a good error message but you don’t get the typical feedback as from which line the verify/assert that failed was on. Not a big deal since you see which test that failed.

JsMockito.Integration.JsTestDriver();

assertThat(Hamcrest).looksNice();

I väntan på David Saff’s alternative assertThat(x).y() så nöjer jag mig med:

public class EmptyMatcher extends TypeSafeMatcher<Collection < ?>> {

    @Override
    public boolean matchesSafely(Collection< ?> c) {
        return c.isEmpty();
    }

    public void describeTo(Description desc) {
        desc.appendText("empty");
    }

    @Factory
    public static lt;T> Matcher< ? super Collection> isEmpty() {
        return new EmptyMatcher();
    }
}

som gör att man kan skriva trevlig enkelt läsbar kod, t ex:

@Test
public void filteringAnEmptyListReturnsAnEmptyList() {
    List anEmptyList = new ArrayList();
    List result = testee.filter(emptyList());
    assertThat(result, isEmpty());
    // men helst: assertThat(result).isEmpty();
}