Test driven joinery

I’ve been in the workshop a bit recently (after far too much time away).

My brand of joinery is very test driven.

Test M/T

Test M/T

Some folks measure, cut and hope, some folks measure, test, cut and relax. I’m in the latter group, if there was one thing I learnt at college (I learnt a lot, the teacher was superb) it was to try stuff out on a scrap before hacking into a big complex component.

I do that when I’m coding too, I write the simplest thing I can test, then test it. I know TDD purists would insist I write the test first, but I don’t, so what?

And I do the same with spreadsheet grid based stuff. I rarely leave new formulas before I have fed in enough test data to exercise all paths. That can be hard to do with big complex nested IFs, so I don’t use them.

If I write a SUMIF I always use a column of IFs with a SUM at the top to check, although I often delete big tests like this before release as they can cause as much hassle as they fix sometimes.

On some models I have built and maintained a separate ‘diagnostic’ workbook with a ton of tests in, but that can be a real burden, and is all but impossible for fluid models. I normally include a status sheet with a bunch of tests in, at least if they are in the same wb their refs will change on insert/delete. I have also done ‘CheckStatus’ code, but I prefer to use the face of a worksheet if I can.

How important (and early) is testing in your developments?

cheers

Simon

Advertisements

14 Responses to “Test driven joinery”

  1. Michael Foord Says:

    You’ll be very interested in the spreadsheet testing stuff we’ve been doing with Resolver One. Hopefully we get to do a release based on it sometime soon and I can blog about it…

  2. Biggus Dickus Says:

    I test everything I do immediately before moving on. It’s kinda “Code a little, test a little.” It breaks things down into small enough bites and allows me to confidently move onto the next step.

    Human nature makes people inclined to have too much faith in their work – I have none UNTIL IT’S TESTED….

    The only problem I have sometimes is when asked to change something (or if – perish the thought I find a BUG) I find I rush the fix and often miss the implications of the change throughout the app. That’s something I have to work on.

    Dick

  3. Al Gill Says:

    My FIRST REACTION was why can’t my users be like that?

    My SECOND REACTION was – hmm – wow – that’s a lot more virtuous than I am and I’m sure I don’t test every simple formula as I go along.

    My THIRD REACTION was – actually – maybe I do? When I’ve been sitting in front of a spreadsheet for more than ten seconds the first question I ask is normally ‘How have I ****ed up today?’ [Apologies to MS] I even test my SUMs when I put them in on the basis that I don’t believe I’m going to get them right. Is anybody else paranoid enough to check their checksums by hand? The only reason this didn’t occur to me during my second thought was that checking your simple SUMs is like breathing right – if you don’t do it, you die.

    FOURTH REACTION – I need to get out more.

    Biggus Dickus – I have to confess to being equally guilty (at least morally) on the bug fixing front. Yes if it’s a real spreadsheet project it’ll go in a bugs dB and go through the normal process but if somebody grabs me and says “what’s wrong with this?” and it’s a 30 second fix I’ll normally just fix it and say “Now you go and test it” – knowing full well they probably won’t.

    AJG

  4. Biggus Dickus Says:

    “Now you go and test it” – knowing full well they probably won’t.”

    Yes – and then they’ll blame YOU when the error is found in the middle of some important usage …. and rightly so :-)

    Dick

  5. Charlie Hall Says:

    For me the hardest part is testing a model that someone else developed and they are looking for enhancements beyond their capabilities. Since the models are usually complex, there is no way I will be able to figure out how to run it – too many industry specific inputs. And typically the models where not designed in such a way that the inputs are sufficiently separated from the outputs and unprotected such that one could capture the inputs and reapply them to newer versions of the model.

    There have been projects when it has worked out and I have built a testing tool to capture and reapply inputs, but it is rare that I get to use it, and it takes a long time to remember how it all worked.

    Usually I have to depend on the users to beta test the new version – and there is always that guilty feeling when they find a bug (usually after the beta testing period is over) and I wonder if I could have/should have caught it myself.

  6. Stephen Bullen Says:

    Charlie, and of course, by the time you’re called in, the model has been developed to the extent of that person’s self-perceived capabilities – which is usally quite a long time after they’ve reached the limit of their actual capabilities! So there’s lots of “And this bit here is something I tried but didn’t work, but I left it in in case it comes in useful later…”

  7. Simon Says:

    Charlie
    And then the client makes you feel thick for not understanding it all – ‘oh the other guy would only need a couple of hours to sort this’ except they are actually unable, well unable as Stephen says.

    No one seems to factor in the effort to unpick someone elses twisted ‘logic’, and that gets v twisted as they go far beyond their comfort zone.

    In my experience these things are so badly specified (ha ha – not ever right?) that its not possible for us to test our work. We can do reasonableness tests and comparisons with the existing, but the if the client knows what they want its only in their head, certainly not written down in a way that could drive a useful test plan.

    Stephen thats a good point about wading through the wreckage of previous fails, and you know if you do clear it up, there’ll be some hidden dependency somewhere. (Or you’ll need it!)

    Dick/Al I think everyone misses those hidden dependencies from time to time, (or always ;-)). I don’t think we are well served with tools to help this. I remember spending ages trying to understand a breaking change, looking at formulas & constants etc, only to eventually realise that there was a forms drop down that was firing in a ‘constant’ that kept changing.

  8. Bob Phillips Says:

    I am a crap tester, and I know it. My biggest problem is that I test based upon what I know. I coded it to check a,b and c, so I test for invalid a, b and c. But real users throw in a d, and I never even thought of that. For that reason, I always insist on a UAT on my deliverables, I will fix what the users find in that test, but anything after live is chargeable.

  9. jonpeltier Says:

    Bob –

    Lack of specific domain knowledge is part of my weakness in testing, just as you describe.

    Another problem that is uncovered by user testing is all the things that they wanted or expected, but hadn’t specified. For this reason I try to provide a working prototype that has much of the functionality but not all the polish. It can be tweaked or corrected with less hassle than a finished project, and the client gets a sense of progress.

  10. Bob Phillips Says:

    Implicit requirements … tell me about it!

    I also go the functional prototype route Jon, but even though you agree this approach in advance, don’t you find that the first thing they report as ‘bugs’ are things that are simply the polish?

  11. Simon Says:

    I think I’m ok at testing that what I wrote does what I expect it to given the inputs I expect. Sadly thats a world away from reality, so I too like to get the tattiest thing I can into the hands of the users ASAP so we can refine what the system should be.

    I only use 1’s or 0’s as test data now otherwise users tend to do a reasonableness check on the numbers rather than any indepth testing.

    I also promise to sort out formatting etc once we have agreed its doing what they want – they are too easily lulled into a false sense of security by decent formatting.

    Its a shame formatting commands are so front and foremost in E2007 I only use them in a small phase towards the end of development.

  12. jonpeltier Says:

    Bob – I actually left out a step. Generally as I build the dialogs, I set up the layout of the controls, then send a screen shot to the client. They munch on that while I code the functionality. I incorporate the major layout feedback into the prototype I send for functional testing.

    Simon – I think we can all determine that our code usually does what we expect. The trick is to expect the unexpected, trap invalid entries, and so forth. This is where experience comes into play.

  13. Bob Phillips Says:

    Jon – I used to do that, and at my last corporate employment it was the main requirements definition, but I have long thought that it gives the user a bit of a cop-out, it looks right so it must be doing a, b, c (and d!) implicitly. Do you actually get useful feedback from that step?

  14. jonpeltier Says:

    Bob –

    I do a lot of small projects, 1/2 day or 1 day, and this step helps to firm up the loose specs I usually start with. It also often results in wider specs (“how about just one more button…”), which are easier to implement here than after the “final” version.

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


%d bloggers like this: