Showing posts with label testing. Show all posts
Showing posts with label testing. Show all posts

Thursday, May 2, 2013

I broke a build... let's celebrate!

Some three years ago I wrote a small test for a very particular factory: it worked pretty well, even if I've never really liked the constraints imposed by the requirements about the exception that is expected to be thrown.

Yesterday, not quite unexpectedly, the build was broken.

Obviously this is not really a unit test, as it depends on real data, so I'd call it more an integration test. Anyway, the data changed and the test failed, thus breaking the build. Normally I wouldn't be so happy. Yet in this case, even if it might be an antipattern (it clearly is), I think I won't change the test to be independent from data, but change the test instead:

@Test(expected=IllegalArgumentException.class)
public void makeBaby() {
  Family family = moz.getFamily();
  assertEquals(3, family.getChildren().size());
  BabyFactory factory = family.getWife();
  Baby babyboy = factory.makeBabyBoy("Ethan");
  assertNotNull(babyboy);
  family.addChild(babyboy);
  assertEquals(4, family.getChildren().size());

  boss.askForRaise(moz);
}

Now I'm only waiting for the requirements to change so that I can remove the exception...

Saturday, January 21, 2012

Automated vs manual testing

Yesterday I had an interesting tweetversation (it actually looks like this word exists) with @lunivore which started with this sentence:
My #1 suggestion for legacy code: If it works, and you change it, check that it still works. If there aren't any tests, run the app already.

I replied something about writing tests, and the discussion went on with several interesting remarks that made me think. As on many others, here's one thing on which I agree with Liz: automated tests do not imply that your app is working as it should. I have seen it too many times to be so naive. It could be (almost) true if our code coverage was perfect, but let's face it, it is not. Please note that I'm not talking about having tests on getters. We should have unit tests, integration tests, end-to-end tests, there-is-no-try-yoda-tests and everything you can think of as long as it ends with "tests" for almost everything. And more often than not we don't.

And even if we did, tests are not users. Well they are, but they are not the users we're most interested in: those who receive value from our app.

Automated tests are very important, but in the end they only are our safety net (even if they also guarantee our customers), but in the end it all comes up to real users.

Automated tests never sign checks, users do. Or at least the bosses of users do. Then, if we belong to that overrated category of people that still insist on eating every day (possibly more than once), we shoud strive to have happy users more than happy tests.

Automated tests are one of our means, not our ends. Let's not forget it.

PS don't forget to follow Liz on Twitter and on her blog.

Monday, November 21, 2011

Startup without falldown: the slides

Here are the slides of my talk "Startup without falldown: strategic planning beyond wishful thinking", held at the 8th Italian Agile Day in Rome:
I'd like to thank everyone who has attended, I hope I haven't wasted their time. If you are one of the lucky (???) ones, please help me to improve and leave a feedback here.

If you want an exhaustive bibliography please register here and you'll receive it in a few days, with a bonus mind map of the talk.

Saturday, March 26, 2011

On "Working Effectively with Legacy Code"

In the last few weeks I have spent most of my time working on a legacy system which is going to be partly replaced by a new one which greatly broadens its scope. But what exactly is legacy code? Michael Feathers says that
To me, legacy code is simply code without tests.

It is as simple as that:
Code without tests is bad code. (...) With tests, we can change the behavior of our code quickly and verifiably. Without them, we really don't know if our code is getting better or worse.

Alas, too often the answer is not the one we'd like to hear.

We started out so well

Almost every system, if you have at least decent programmers working on it, starts out as a wonderful green field: defined architecture, code conventions, solid principles, patterns and practices.

After a while the gardeners get busy with other gardens, and a couple of weeds start to poke out. Then some more, then more. And more. Meanwhile seasons pass and rain falls, so our green field gets muddier and muddier. Actually it looks more like it has rained cats and dogs, day after day after day. Mud becomes quicksand, and nobody dares to even get close to it.

The (in)famous spaghetti incident

Too soon the project reaches the point in which whenever you pull a string you have an undesired and often undetected change in a completely different part of the system. When I want to sound important I refer to this as the butterfly effect, a metaphore used in chaos theory. Yes, I know that your projects are different and you have never seen it happen, so try to use your imagination and stay with me a little more, will you?

That was exactly where I was, all tangled up in obscure dependencies: the perfect spaghetti code.


As I tweeted, I love spaghetti when they are in my dish, not in my code, so I had to do something (even because deadlines are always waiting in ambush). To sharpen my tools I retrieved my copy of "Working Effectively with Legacy Code". The book covers many aspects of what we developers have to face daily: instead of the "write once, run anywhere" mantra, we have to deal with "write once, read it at least ten times", but too often our code is obscure. Yes, our code. And we don't have tests, or at least we don't have enough. Do you have a complete coverage? If you do, you have all my respect, otherwise welcome in the family.

How do we get out of this?

First of all, we have to get our code into a test harness, but it is not as easy as it sounds. The book contains a series of 24 different dependency breaking techniques, each of which presented in several almost-real situations, to minimize the impacts of changes and ensure that we are not breaking anything. It sounds like the classic catch-22 thing:

When we change code, we should have tests in place. To put tests in place, we often have to change code.


This is where the techniques come in handy. I was pleasantly surprised when I discovered that, after some years of TDD, almost every problem and solution described in the book sounded familiar, so I skimmed it more than actually reading it. I still remember when I first read it: man it was hard, not the writer's fault, but because of the reader.
One of the most important things is that you have to preserve the behavior of your system when you refactor it (and preserve the rest of the system when you introduce new features or change existing ones), so you should at least know what it does. What is better than some characterisation tests?

Cleaning it up

I love NetBeans. I always have, since it was Forte4Java. Now I love it even more because it has a heap of automated refactorings that make "Refactoring" needless (ehm... that's not true, NetBeans deals with the mechanics, but you should know what you're doing, so stop reading and go buy your copy if you still don't have it. Done? OK, let's get on).

Do you need to create a new test class? nothing simpler, just a simple SHIFT + CTRL + U and a template with all the skeleton methods is ready for you. Just punch in the starting conditions and the expected results and you're done. Feathers describes all refactorings steps by steps, but a few keystrokes are all NetBeans needs to extract superclasses, methods, interfaces, pull up or push down members and methods, delegate, and so on and so forth. Nevertheless, you should know what you're doing.

While working your code could get uglier in some places. It could be temporary, or temporary in the Italian way (which means definitive). Even in the latter case, at least you would have tests in place so that you know you're not breaking anything.

In several places Feathers suggests to reduce incapsulation to put code into a test harness. Somebody might find it strange, extremists might find it insane. Yet...

Encapsulation isn't an end in itself: it is a tool for understanding.


Remember when I wrote that you write code once and read the very same code over and over? What 's the point of an encapsulated design if I spend hours for the most trivial task? Mnd you, I'm not suggesting every field must be public and collections should be directly exposed, but nobody is going die if a private method becomes protected for the sake of testing.

Where have my spaghetti gone?

After a while, one after the other, your pull spaghetti out of your dish, slowly replacing them with neat and tidy sushi.

Before you realize it, you will look at your codebase with different eyes, and chances are that you will enjoy dealing with legacy code. Don't enjoy too much though, even if you have a lot of technical debt to pay: you don't have to pay it for the sake of doing it, but because it allows you to move faster when you need it (and you know you will need it).

Let me stress this again: if I didn't have some hundreds of tests already in place, these weeks would have been months. Baby steps, nip and tuck, red and green bar. But I only introduced tests and refactored legacy code when I needed it, otherwise I would spend too much time on activities that were not a priority. See something rotten? pen it down, fix what you're doing and if you still have time you can deal with it. Look for decisions that could change, but change your code just in time (the YAGNI rule rules).

Thursday, October 7, 2010

Don't forget the content type

In one of our legacy projects all web services were exposed through SAAJ servlets; having to expose a new one we opted for a newer JAX-WS style, also thanks to everything that NetBeans gives you right out of the box.

So we created a new Web Service (you can check this tutorial to see how it is done) and wrote all the tests and code we needed. Everything went fine, until we tried to test the whole project (which should be mandatory before you commit).

At this point some of our old tests failed, all reporting the same error:
SAAJ0532: Absent Content-Type
The offending code builds a message that we use to test the parser:
private SOAPMessage buildMessage(String filename) throws IOException,  SOAPException {
InputStream resourceAsStream = this.getClass().getClassLoader().getResourceAsStream(basepath + filename);
MessageFactory messageFactory = MessageFactory.newInstance();
SOAPMessage message = messageFactory.createMessage(new MimeHeaders(), resourceAsStream);
return message;
}
At first it caught me off balance because our additions and changes went nowhere near the code related to the failure. Then it must be something related to the environment, which is the fact that NetBeans added a dependency on the JAX-WS 2.1 library that probably has some conflicts with the SOAP jars we use.

As some manual tests seemed to confirm that the application is working normally, we deferred the investigations (not for very much longer!) and simply fixed the test:
private SOAPMessage buildMessage(String filename) throws IOException,  SOAPException {
InputStream resourceAsStream = this.getClass().getClassLoader().getResourceAsStream(basepath + filename);
MessageFactory messageFactory = MessageFactory.newInstance();
MimeHeaders headers = new MimeHeaders();
headers.addHeader("Content-Type", "text/xml");
SOAPMessage message = messageFactory.createMessage(headers, resourceAsStream);
return message;
}
And now an afterthought. The test failed, yet the application did not. Does this mean our test was wrong? Actually it only means that the previous library was more forgiving, because the test has proved useful for a long time (also spotting a regression). It is better to have a not-so-perfect test than none at all.

Tuesday, August 31, 2010

How to refactor and test a SAAJServlet

Mind you, not a simple SAAJServlet, but an old fashioned legacy transaction script without the scent of a test. Anyway... let's pick up the usual books (Working effectively with legacy code, Refactoring, Refactoring to patterns - just a starting point of course) to keep at hand as a reference and let's start refactoring.

First a word of caution: don't just copy from the books. Study them, try to apply the refactorings in a small and controlled environment, maybe in little katas, and get the basics beyond them, otherwise you'll do more harm than good.

Back to business. We basically have to test the onMessage method, that gets a SOAPMessage and returns another one. Without even looking at the code, one would expect an algorithm like the following one:
  • parse the message into a parameter
  • pass it to a service that executes whatever must be executed
  • build a new message based on the input parameter and on the outcome of the previous step
Obviously one should also consider errors management, but let's keep it simple. Let's take a look at the actual code:

@Override
public SOAPMessage onMessage(SOAPMessage message) {
SOAPMessage result = null;
Dialogue dialogue = null;
try {
SOAPBody body = message.getSOAPBody();

SOAPFactory sFactory = SOAPFactory.newInstance();
Name bodyName = sFactory.createName("myLocalName", "myPrefix", "myUri");
Iterator it = body.getChildElements(bodyName);
if (it.hasNext()) {
SOAPBodyElement be = (SOAPBodyElement) it.next();
dialogue = parseBodyElement(be);
Operation operation = operationFactory.getOperation(dialogue);
if (operation != null) {
try {
operation.execute(dialogue);
} catch (Throwable th) {
getServletContext().log(th.getMessage(), th);
dialogue.setCode(Dialogue.KO);
dialogue.setMessage("Service unavailable.");
}
}
} else {
dialogue.setCode(Dialogue.KO);
dialogue.setMessage("Malformed message.");
}
} catch (Throwable th) {
getServletContext().log(th.getMessage(), th);
dialogue.setCode(Dialogue.KO);
dialogue.setMessage("Service unavailable.");
} finally {
try {
MessageResponseGenerator mrg = new MessageResponseGenerator();
result = mrg.generateResponseMessage(dialogue);
} catch (Throwable th) {
result = null;
th.printStackTrace();
}
}
return result;
}
Now, though PMD does not complain too much about the structure of the method - just a few dataflow anomaly analysis, null assignments and catching throwables - I smell (more than) a rat.

Starting from the beginning, the first things that strucks me is the different level of abstractions of the code: you have both a low level implementation that extracts a SOAPBodyElement to be parsed and a call to an Operation that encapsulates the requested service (the Command pattern), and this is not good. There are also too many nested ifs and try-catch blocks to my liking.

There is a separated object that creates the response message, and this is good. Why on earth are we missing an object or at least a method that parses the message in the first place?

Why do we ask a factory for an Operation passing in a Dialogue and then we have to pass the same parameter back to the operation we get back from the factory? couln't the operation hold a reference to the dialogue? Would it be better? we shall see later.

And, obviously, all the references are hard-coded...

So far, we have just conducted a small code review. But what's the point in the first place? We have to introduce a new Operation object (maybe extending an existing one, and please note the alliteration). Nothing strange so far, but the resulting object for this operation should contain some more fields than the existing one. Still nothing terrible, but as you might have noticed currently the Dialogue object works as a collecting parameter through all the method, while we would need two different objects (or modify the initial parsing method in such a way to have an instance of the right Dialogue subclass, or introduce a composition in the Dialogue class, or explore a thousand other possibilities).

By the way, it is mandatory that we do not change the interface for the service, which means that the incoming and outcoming messages for the existing operations cannot change.

The fact is that before we start refactoring we have to guarantee that. Of course we could start refactoring right away, but, as you might have noticed if this is not the first post of mine you happen to read, I find it "just a little bit" risky.

So the answer to the first question, i.e. how to refactor a SAAJ servlet, is, at least in my mind, simple: write enough tests to describe the current behaviour before you do anything else. I know: easier said than done.

That's tricky: how can you do that? The class we have to test is a servlet, so we have four options:
  • use manual testing: we already have this in place, but each single change would require at least a dozen tests, making the whole process too slow (and not reliably repeatable, as it is based on people's good will)
  • mock all the objects provided by the servlet container needed by the servlet: quite a lot of work, and I'm not going down that way (even because I'm listening to Starway to Heaven and I don't wont to spoil the happy sensation)
  • deploy the application to a servlet container and use in-container testing, possibly with a framework like Cactus: this introduces an unwanted complexity and I'll avoid it too even if it would be tempting, as the servlet is deployed in a proprietary framework in which you can only access the database if you actually start the webapp (don't ask, you don't want to know). Why? because I think it would not force me to separate responsibilities as much as the next approach.
  • use a library such as HttpUnit, which contains ServletUnit. Being this a simulated container I am sure I would never have access to the database given the current situation, so I guess I'll have to work my way to a cleaner design.
Back to the servlet: there are cases in which the method returns a message without going all the way down to the database, so as a first step I'll exercise these and keep manual testing for end-to-end situations.

The very first test consists in producing a message with a wrong format: I should get a SOAPMessage with the code property set as KO and an error message of "Malformed message". Or should I?
@Test
public void testOnMessageWithMalformedMessage() throws IOException, SAXException {
System.out.println("testOnMessageWithMalformedMessage");

InputStream resourceAsStream = this.getClass().
getClassLoader().
getResourceAsStream(
"path/to/my/message.xml");

ServletRunner sr = new ServletRunner();
sr.registerServlet("myServlet", MyServlet.class.getName());

ServletUnitClient sc = sr.newClient();
WebRequest request = new PostMethodWebRequest(
"http://localhost:8084/myServlet",
resourceAsStream,
"text/xml");

WebResponse response = sc.getResponse(request);
assertNotNull("No response received", response);
assertEquals("OK", response.getResponseMessage());
assertTrue(response.getText().contains("KO"));
assertTrue(response.getText().contains("Malformed message."));
}
These steps are detailed in the ServletUnit documentation. Before we run the test, remember we're not going to change anything yet.

Too optimistic? actually I was. Running the tests gives me the following error:
Error on HTTP request: 500 javax.servlet.ServletException: SAAJ Post failed null.
Looks like the MessageResponseGenerator cannot handle null parameters very well. I admit I knew I had been too optimistic about the error message (how could I trust the generator had the same message the servlet used and that I was expecting?), but at least I hoped in a valid SOAPMessage... But after all isn't the NullPointerException the most widespread error in Java code?

It's a long way to go... remeber that before we change the current behaviour we have to describe it, so the way is even longer. Yet, the first step has been made.

Wednesday, August 25, 2010

The importance of unit tests II

Yesterday I had some fun spending some hours to introduce ajax in a legacy application (I call it legacy because it has a very small amount of automated tests). During the afternoon I was quite pleased with myself, partly due to the satisfaction associated with the work done, partly due to the "Steppenwolf" novel, partly due to the Beethoven Sonatas I was listening to, partly due to my full stomach. Being a very wise and intelllectual person, I am in favour of the latter.

All this abrupltly ended when I got an unexpected error from a web service called from a part of the codebase that I had modified in the morning, which seemed to go wrong when I submitted a foreign address.

First I checked the test that exercised the web service client, where I verified that all the parameters where correctly passed as expected. Then, I wrote some other tests trying to exercise the particular feature, narrowing the scope of my private investigations. Btw, this reminds me that I have not listened to the Dire Straits in a long time, which is a shame.

Everything seemed fine, so I spent some hours with the developers of the service trying to figure out what was going wrong with the service (if anything). After some head banging it emerged that he problem was that we were focusing on a particular parameter, while the error sneaked in another one. I hope they will not make me pay for the dents in the desk.

That would have been clear from the beginning if we checked all the parameters in the XML stream instead of focusing only on the ones that we thought were important for the specific call (the port in the service is only one, even if there should be many, and the behaviour is determined by which of the several zillions parameters are set and what their values are; I don't think this is very brilliant, but it cannot be changed, so complaining is useless... or at least it brings no business value).

So, everything went back to a wrong value, due to a "simple" setter method.

Now, you normally never write a test for a setter method. This is perfectly acceptable, as you should write tests for "everything that could possibily go wrong" and a normal setter is not included in the list.

Pity this setter was not a plain one, but contained an if:
public void setMyProperty(final String value) {
if (value != null || StringUtils.isEmpty(value.trim())) {
this.myProperty = "FIXED";
} else {
this.myProperty = value
}
}
This looks like a blunder... why that not-null check? is it to be able to trim the value or should it actually be a check the value is NOT null? If the former is true, why hasn't the author simply used the isBlank method instead? maybe because she was thinking of saving the trimmed property (which in the case she forgot to do) or because she didn't know the existence of the method?

Though strange this might sound, all this is not really important as I have access to the product owner and I could get all the answers I needed.

The point is: that method required a test, and it was nowhere to be found. You might argue that it's simple enough to avoid writing one, but all the time I wasted says something different. Still, you might add that it was my fault because I didn't check the setter in the first place, but this is only another arrow in my quiver: this is exactly the reason for which we need automated tests, as people do make mistakes and forget to check simple methods.

I'd like to think that if I were pair programming with the author of this code I'd never let her skip writing the failing tests first, at least not without fighting.

Note that I'm not pointing a blaming finger, I could have written that code myself (and sometimes I did, and I'm happy to say that I always regret it when I realize it).

I know I am fighting - and so far losing - a running battle, but tests are necessary, even if someone tells you they represent a cost. Actually they are not even a cost, they are an investment, they are one of the most important risk management tools software developers have.

It only took me some fifteen minutes to write four different tests that assessed the desired behaviour and rewrite the method from scratch. These fifteen minutes would spare three people some wasted hours. This also demonstrates how the cost of bug fixing dramatically increases with time.

Yet, for some unknown reasons and against all evidence, too many managers AND developers refuse to believe in practices like TDD or pair programming. Is it to mantain the illusion of control? is it fear of changes? is it lack of trust? is this evidence not so evident? let's try some math:
  • cost of writing the setter method: 5 minutes
  • perceived cost (usually coincides with the former): 5 minutes
  • cost of writing tests and the setter method: 15 minutes
  • perceived waste: cost of writing tests minus cost of writing the setter method = 10 minutes
  • cost of writing tests and the setter method, pair programming: 30 minutes
  • perceived as almost blasphemous waste: cost of writing tests pair programming minus cost of writing the setter method = 25 minutes
And this is where analysis normally end. On the other hand...
  • cost of finding the bug: 6 hours
  • actual cost: 6 hours plus 15 minutes PLUS 5 minutes = 6 hours and 20 minutes
  • actual waste: 6 hours PLUS 5 minutes
Also note that the time needed to write the code in the first place was completely wasted.

Still think that writing tests is too expensive? Well, the curious thing is that at this point everyone seem to agree on the reason behind the added cost: the code was sloppy. As a corollary, the blame is on the developer. Well, THIS IS NOT THE PROBLEM. The problem is that the way used to write the code was sloppy. And this does not depend entirely on the developer.

Think about it.

Tuesday, June 15, 2010

Our 1000th test

I am pleased to announce that today I updated and committed a test file that contains our 1000th test for one of the projects we're working on.


Thanks to everyone that made this possibile.

Friday, June 11, 2010

How to change the endpoint defined in the WSDL

Why should one do that? There might be several different reasons, I'll write about the one that moved me. We are developing a new integrated enterprise application for a customer that is located in another city: the core of the system is developed in RPG-LE and uses a cool UI that greatly extends the classic "green screen". The system is deployed on an iSeries and exposes Axis2 Web Services through an IBM Websphere Application Server; these services are accessed by a web application hosted in a protected network that is used by local offices all over Italy.

We periodically update their core system in a direct way, but for security reasons we do not have administrative access to their application server, so whenever we release an update we have to contact them and wait for them to republish the Web Services. As, despite of all my efforts to persuade the decisors, this still happens manually (nemo propheta in patria), every now and then we experience strange behaviours in remote clients.

As I still haven't managed to obtain dedicated services instances and databases for tests (see above) I must revert to plain old manual testing. More or less, as I definitely don't want to jump on the bandwagon of keyboard monkeys that seems so in fashion. Lazy as every good programmer should be (if nothing else I still got something of a good programmer) at least I have partly automated the process with a combination of Selenium, wsmonitor and Poster.

What I still missed was the possibility to easily override the endpoint defined in the WSDL I used when I generated the client in NetBeans for a scenario like this: we develop the core, expose the services, generate a client, use it in the remote application, run it and check that everything is working fine, deploy to the customer, ask them to republish the services, generate a client, use it in the remote application, run it and check that everything is working fine and it is not. And if you're wondering, yes there is some echo in this room.

As a matter of fact, every now and then we got as an answer a SOAP fault with a crystal clear message like

java.lang.NumberFormatException: High-order nibble of the byte at array offset 11 is not valid.  Byte value: 40

WTF??? At this point we normally check that the customer properly deployed the services diffing the WSDLs, then eventually contact them, but that's not always as easy as we'd wish. To get further informations (and to speedup the test process) I would like to use Poster to test that the service is working as expected, but for that I need the content to post. That's where wsmonitor would come useful, but it turns out that I cannot update the client through it:

Please enable REST support in WEB-INF/conf/axis2.xml

Before you ask, they wouldn't allow it. So I had to directly generate a client from their anthracite coal grey box (don't call it black, they'd be resented about it) but have it point to my local wsmonitor instead (I'm afraid either NetBeans still has to catch up a little with this or I have to find out how to sort it out) to see the outgoing and incoming messages.

Now the real magic you've been waiting for (well maybe you haven't, but I sure did): the client code generated by NetBeans looks like this snippet:

MyService service = new MyService();
MyServicePortType port =
service.getMyServiceSOAP11PortHttp();

At this point the port refers to the endpoint defined in the remote WSDL. To override it all you need to do is add this line:

((BindingProvider) port).getRequestContext().put(
BindingProvider.ENDPOINT_ADDRESS_PROPERTY,
"http://localhost:12345/path/to/MyService");

So what? as awkward as it might seem, the fastest solutions (obviously as far as I know, which is not much) turned out to generate the client from the original WSDL, debug the client application, start the wsmonitor, change the endpoint on the fly and apply the code changes class as needed, and... ta-daaah! here are the SOAP messages!

From this point on I can happily close everything else and switch to Poster: until a click on the POST button returns me a valid message I don't even waste a second on my IDE or on the client application.

Why did I choose 12345 for the port? Because

...that's the stupidest combination I've ever heard of in my life! That's the kinda thing an idiot would have on his luggage!

Now, if you're yelling

1, 2, 3, 4, 5? That's amazing! I've got the same combination on my luggage!

console yourself, you're not alone :-)

Monday, May 31, 2010

Are tests a burden?

Sometimes one might even think so (actually today for some moments I did). The story: one of our factories "left out" a property, meaning that we simply forgot it, both in tests and codebase. Do I hear someone say "then what are tests for? they are useless as they didn't trap your error"? well the someone I'm thinking about is on holiday so I'll be spared that, but she would be right on one thing: the test, these unit tests at least, did not trap MY error (assuming I wrote that part of the code).

So what? this oversight, when corrected (test first of course), caused some tests related to distant classes to fail, not because the correction was not right, but because the freshly failing tests needed to be updated.

Foolishly, I didn't run the complete suite of tests for the project before committing, so my CI server wrote me that I broke the build. This happened twice.

I must confess it was anoying: I had to add more code to correct the wrong assumptions of the test. This conceptually disturbed me, as I didn't have to fix the code (which was already correct) but the tests (it just so happened it was because I was using a mocking library and I had to set another couple of expectations).

This pushed me to a couple of considerations.

First, it is true that sometimes mocking libraries introduce a dependency you could happily live without.

Second, I would have avoided to trigger a remote build if I ran the whole suite of tests. The problem is that it would have take too long compared (in my opinion) to the small changes I made. Once more, I slammed my face on the wall, so I think I'll play ball by always applying the "Run Private Builds" pattern, that in the end would have saved me time.

Thursday, May 20, 2010

Create or update?

A couple of days ago a bunch of sectors of my hard disk decided to vanish into the blue, bringing with them some files of my local repository (no worry about that) and some files of my NetBeans installation, configuration files included (no worry about that too, but a little more work required). While my colleagues are providing me with a new hard disk, on which I asked for a fresh installation (before you ask: it is a company policy that we only use operating systems from Redmond on our personal computers) I am working on the laptop. Which, by chance, happens to be a fresh installation too as it is the replacement for my old one (who can nowadays use NetBeans AND Firefox AND SQLServer AND all the rest, antivirus included, on 1GB? Maybe 2GB is not that much, but at least it doubles my previous supply)

Ok, so after reinstalling and reconfiguring my stuff I checked out a group of three projects we're working on. Projects that our Hudson guarantees as healthy (see one below).


Open the group of projects, set as main, fix the configurations for my local databases (for each database we have one instance for manual and acceptance tests and one for the automated ones) configurations clean and build, test (just to be sure).

91% passed. What? That means fail! but... what's wrong? mmm... it seems there's a row in the User table that causes Hibernate to complain about constraint violations. That is strange, as - as far as I know, but I obviously might be wrong - all our automated persistence tests run with HibernateOpenViewSession, a simple Runner that before each test opens a Session and rollbacks everything after the invocation (nothing new under the sun, but quite handy):

@Override
protected void invokeTestMethod(Method method, RunNotifier notifier) {
try {
final Session session = HibernateUtil.getSessionFactory().getCurrentSession();
session.beginTransaction();

super.invokeTestMethod(method, notifier);

if (session.isOpen()) {
session.clear();
session.getTransaction().rollback();
}

if (session.isOpen()) {
session.close();
}
} catch (Throwable t) {
System.err.println("HibernateOpenViewSession RUNNER FAILURE");
notifier.fireTestFailure(new Failure(Description.EMPTY, t));
}
}

Yet, that row is still there. Well, maybe one of our tests doesn't use our favourite runner. So, why doesn't Hudson complain? After a little investigation it turned out that the Hibernate configuration for our CI server has a hibernate.hbm2ddl.auto=create property, where my local file had update. That fixed, all my local files ran smoothly and gave me a green bar.

Now the hunt for Red October will start. Ehm... I meant the hunt for the culprit test, of course... I'll keep my eyes on the ball.

By the way, we almost reached our 1000th test on the project :-) and it is only part of a group of three... it may sound a little to many out there, but for us it is a great success.

As a parting note, let me stress again the importance of a CI server to avoid the "it runs on my machine" syndrome - which in this case was "it doesn't work on my machine" :-)

Thursday, May 13, 2010

Write a failing test first

Checking a functionality of a Struts2 interceptor I realized that we forgot to check a condition. Adding the correction was trivial, but I resisted. I first added a failing test, even though the class had no tests at all (this was justified by the fact that most methods are actually pretty simple one-liners).

Yet, an error lurched its way from behind.

The answer is NOT the code fix. At least, not only the code fix. Because other errors are lurking, and I don't want to be in the balance: I want to avoid regression(s). There are no excuses, as testing with Struts2 is much easier than it was with Struts.

That said, I wrote two test cases to avoid regressions on what I knew was working. Green. Then I added another test case that checked the failing condition. Red, as supposed. Only at this point I could add the magical line of code I needed. Back to green. Now we're going somewhere. And I left my camp (slightly) better than how I found it.

Then it was time for some riskless refactoring (well maybe not riskless, but close). Following Steve Freeman suggestions, I changed the names of a couple of methods to make them more expressive. Of course I could have done this before, but having a test harness makes you feel less... vulnerable. Once you get the habit, it is hard to get rid of it. And I don't want to get rid of it :-) on the contrary, I want to keep the ball rolling.

Thursday, February 18, 2010

The importance of unit tests

Today I made a small change in a class in the application I'm working on. Being a diligent monkey, before adding the new feature I wrote a small dedicated test. The test failed, I added the feature, the test passed, I cleaned up a little bit (not too much, having just added three SLOCs) and finally committed the class in our SCM before hopping on to the next feature.

After a while our Continuous Integration Server pitilessly sent me a mail in which it informed me that the build was broken. No, better yet, that I broke the build, causing not one but 17 tests to fail. O shame and amazement, how was it possible? I had even written a test...

Despite all apparences, this is very good news. Why, you ask, grasshopper? Let me ask you a question: when do you want to know you have a problem? talking about software development, as soon as possible. Those 17 failures have saved me from delivering buggy code that would upset the user and set the stage for some annoying debugging sessions.

This has only been possible because of the test harness: all the time we spent writing tests has, once more, paid off well. So, without forgetting to add a test for the previously unchecked condition, I fixed the code in double-quick time.

I wish all managers understood this: writing tests is not a cost, but a tremendously effective cost saving tool.

Friday, November 20, 2009

Target 500 reached and passed

As our CI server is approaching to the 300th build of the project I'm working on, I am proud and happy to announce that we have reached and passed the 500th test, as we now have 528. Maybe #500 was mine, maybe it was lm's... or maybe it belonged to someone else. What really matters is that now it belongs to everyone, and it adds up to the status of good health of the codebase.


The number of tests has constantly grown, and the bar is almost always green - even if I should say that, using Hudson, the ball is blue (or the bar is blue, if you delve into details). I'll never get tired of saying it, CI server are really a wonderful tool - obviously if you provide tests for what you build, but that goes without saying.... or does it?

Wednesday, November 11, 2009

How to test a very particular factory

@Test(expected=IllegalArgumentException.class)
public void makeBaby() {
  Family family = moz.getFamily();
  assertNotNull(family);
  assertEquals(2, family.getChildren().size());
  BabyFactory factory = family.getWife();
  assertNotNull(factory);
  Baby babygirl = factory.makeBabyGirl("Mila");
  assertNotNull(babygirl);
  family.addChild(babygirl);
  assertEquals(3, family.getChildren().size());

  boss.askForRaise(moz);
}

Probably you should also set a timeout for the test, which should complete in about 40 weeks ;-)

Thursday, August 6, 2009

Checking beans properties with JMock

JMock is a great tool when you have to test how an object interacts with other objects for which you still haven't written a single line of code; all you have to do is to annotate the test class...


@RunWith(JMock.class)

...define a mockery...


private Mockery context = new JUnit4Mockery();

...and finally write a test method in which you define the mock objects you need, specify their behaviour then you execute your code and test the results:


@Test
public void testGetTable() {
System.out.println("getTable");
context.setImposteriser(ClassImposteriser.INSTANCE);
final LookupTableRepository repository = context.mock(LookupTableRepository.class);
final LookupTable table = context.mock(LookupTable.class);
final LookupTableFacadeResultFactory factory = context.mock(LookupTableFacadeResultFactory.class);
final LookupTableFacadeResult expResult = context.mock(LookupTableFacadeResult.class);

context.checking(new Expectations() {{
oneOf(repository).getTable(with(equal(tableName)));
will(returnValue(table));
oneOf(factory).make(with(equal(table)));
will(returnValue(expResult));
}});

LookupTableFacade instance = new LookupTableFacadeImpl(repository, factory);
LookupTableFacadeResult result = instance.getTable(tableName);
assertSame(expResult, result);
}

So far, so good. But what if you need to check that your object interacts with others passing as a parameter another object it has just built?

This is where custom matchers come in: you simply add an expectation that specifies that the parameter you're interested in has one property (or more) with a given value:


context.checking(new Expectations() {{
Matcher<LookupTable> tableNameMatcher = Matchers.hasProperty("tableName", equal(tableName));
oneOf(repository).save(with(tableNameMatcher));
will(returnValue(table));
oneOf(factory).make(with(equal(table)));
will(returnValue(expResult));
}});

In this example we expect our instance to ask the repository to save an object whose tableName property has the specified value.

Obviously, the matcher can be combined with others:


Matcher<LookupTableItem> codeMatcher = Matchers.hasProperty("code", equal(code));
Matcher<LookupTableItem> descriptionMatcher = Matchers.hasProperty("description", equal(description));
oneOf(repository).save(with(allOf(codeMatcher, descriptionMatcher)));
will(returnValue(table));

Tuesday, June 23, 2009

Testing an interface in JUnit4

How do you test an interface? Obviously, there is no way to instantiate it directly, so you have to write a test for each different implementation. I feel shivers down my spine even as I write it... that awfully smells like duplication, which - should you ask - is bad. Nevertheless, I'd like to test every possible implementation.

As JUnit Recipes reports, this is quite simple:
  1. Start from the test case for one of the implementation (I assume you have them, don't you?)
  2. Create a new abstract test case in which you will define the expected behavior
  3. Have your test case extend the abstract test case
  4. Move the code that instantiates the object under test into a separate method, storing the object as a reference to the interface and not the implementation.
  5. Adjust the rest of the code referring to the interface behavior accordingly
  6. Create an abstract creation method for each concrete creation method
  7. Move all tests for the interface to the abstract test case
At this point in the test cases for the implementations there should be only implementation specific behavior, not related to the interface.

I tried it all in JUnit 4... and it didn't work, as when I ran the tests for all the project I got a wonderful java.lang.InstantiationException (actually several of them) for the abstract test case.

After some personal tries and a bit of googling I found that all I needed to do to have everything run smoothly was to add the @Ignore assertion to the abstract test case. Now we have no excuses!

Monday, February 2, 2009

How to impose a timeout in JUnit tests

Ever had the need to check if a method is taking too long to complete? You can follow the instinctive path, get the System.currentTimeMillis() before and after the execution of the method and assert their difference is smaller than the acceptable value.

Or... you can save time and rely on JUnit to stop the test thread and fail the test (a TimeoutException is thrown), simply passing the desired timeout value to the @Test annotation:
@Test(timeout=1000)
public void testWithTimeout() {
...
}

Friday, January 23, 2009

The birthday greetings kata

On February the 14th the milano-xpug organized a kata whose subject was proposed by Matteo on his blog to introduce the exagonal architecture.

Unluckily I couldn't attend the meeting in the flesh, so I just decided to try the kata out here on my own. I first started from scratch, thinking it would prove more fun; then, as I was taking quite a different approach - you could also say that a whole different design was emerging - I decided to get back and start from the original code provided and refactor it to try and focus on the architectural issues which were the aim of the exercise.

Even if I think this exercise to be quite simple, I decided to post a trace anyway because someone might still find it useful. You can never say!

The heart of the application is the BirthdayService class, whose responsibilities consist in identifying employees whose birthday falls on a given date and sending them a greeting message.

This is the public method, called by the acceptance test:


I know it looks odd, but I've had enough of escaping characters and trying to format code to make it look nice, so I hope you will forgive me.

Anyway, the sendGreetings method parses an input file, creates employees, checks if they are eligible for a greeting message and delegate the actual sending of the message to a private method:


Seems a great bunch of things for a single class... let's try to sort it out. First I decoupled the service from the file with the employees' names introducing the interface EmployeeRepository, with the method getEmployees, then I wrote a FileSystemEmployeeRepository that implemented it. The repository was then injected into the BirthdayService via constructor by
the acceptance test. At this point the code looked like this:
...
List<Employee> employees = repository.getEmployees();
for (Employee employee : employees) {
if (employee.isBirthday) {
//compose message and call method that sends it
}
}
...
Still I thought that the check on the date of birth could be done by the repository itself, so I changed the interface adding the new findEmployeesBornOn(OurDate) method and changed the code so that it looked like this:
...
List<Employee> employees =
repository.findEmployeesBornOn(ourDate);
for (Employee employee : employees) {
//compose message and call method that sends it
}
...
Maybe not a very big change but I think it was worth it, as now the code is cleaner. Then there was the messaging part. First of all I decided to introduce a new BirthdayGreeting class, that produces a message body, a fixed subject and a recipient starting from an employee; at this point I extracted the interface SimpleMessage that exposes these three strings. The code had become
...
if (employee.isBirthday) {
SimpleMessage greeting = new BirthdayGreeting(employee);
sendMessage(smptHost, smptPort, "sender@here.com", greeting);
}
...
Still I had to resolve the dependency from the SMTP, so just like I did for the repository I introduced the MessagingService interface with the sendMessage(String sender, SimpleMessage message) method and injected it into the BirthdayService via constructor from the acceptance test. Some nip and tuck to move the body of the sendMessage method from BirthdayService into a new EmailService, implementer of MessagingService, and voila, the new BirthdayService code:
public void sendGreetings(int month, int day) {
List<Employee> employees =
repository.findEmployeesBornOn(month, day);
for (Employee employee : employees) {
BirthdayGreeting greeting = new BirthdayGreeting(employee);
messagingService.sendMessage("sender@here.com", greeting);
}
}
I also changed the method parameter from OurDate to month, day. Now we have a simple and decoupled service that can focus on its true responsibilities: ask for all people who should receive a greeting, create a new message and delegate its sending.

One might wonder whether the sender belongs where I put it, or if the service should directly instantiate a BirthdayGreeting with a new rather than getting it in some other way. But maybe that's going too far.

BTW I also decided not to use the Employee.isBirthday() method and I created a value object called IsBirthday, created with two parameters representing month and day, with the boolean method forEmployee(Employee). The code in the repository now looks like this:
public List findEmployeesBornOn(int month, int day) {
List<Employee> list = new ArrayList<Employee>();
IsBirthday isBirthday = new IsBirthday(month, day);

for (Employee employee : employees) {
if (isBirthday.forEmployee(employee)) {
list.add(employee);
}
}
return Collections.unmodifiableList(list);
}
That's all folks... maybe I'll manage to post the complete code here, where you can find some other (and more accredited) solutions produced by people who actually were there.

Tuesday, December 30, 2008

Retesting software modules

How do you test a piece of software or module that came for retesting, meaning you found some bugs, submitted them to developers that fixed them and sent the module back for retesting?

I think that, whether roles such as QA experts are more or less defined in the team, the person who found the bugs should accurately check the software, as she's the one with more experience on the history of the bugs. It would also be useful to hand the software to another tester, which could look at it with a different perspective and identify more and/or better tests.

The important thing is that tests should be repeatable, better yet automated where possible, otherwise it would be very difficult to positively assert that the bug has been fixed.