Showing posts with label refactoring. Show all posts
Showing posts with label refactoring. Show all posts

Saturday, January 21, 2012

Automated vs manual testing

Yesterday I had an interesting tweetversation (it actually looks like this word exists) with @lunivore which started with this sentence:
My #1 suggestion for legacy code: If it works, and you change it, check that it still works. If there aren't any tests, run the app already.

I replied something about writing tests, and the discussion went on with several interesting remarks that made me think. As on many others, here's one thing on which I agree with Liz: automated tests do not imply that your app is working as it should. I have seen it too many times to be so naive. It could be (almost) true if our code coverage was perfect, but let's face it, it is not. Please note that I'm not talking about having tests on getters. We should have unit tests, integration tests, end-to-end tests, there-is-no-try-yoda-tests and everything you can think of as long as it ends with "tests" for almost everything. And more often than not we don't.

And even if we did, tests are not users. Well they are, but they are not the users we're most interested in: those who receive value from our app.

Automated tests are very important, but in the end they only are our safety net (even if they also guarantee our customers), but in the end it all comes up to real users.

Automated tests never sign checks, users do. Or at least the bosses of users do. Then, if we belong to that overrated category of people that still insist on eating every day (possibly more than once), we shoud strive to have happy users more than happy tests.

Automated tests are one of our means, not our ends. Let's not forget it.

PS don't forget to follow Liz on Twitter and on her blog.

Saturday, March 26, 2011

On "Working Effectively with Legacy Code"

In the last few weeks I have spent most of my time working on a legacy system which is going to be partly replaced by a new one which greatly broadens its scope. But what exactly is legacy code? Michael Feathers says that
To me, legacy code is simply code without tests.

It is as simple as that:
Code without tests is bad code. (...) With tests, we can change the behavior of our code quickly and verifiably. Without them, we really don't know if our code is getting better or worse.

Alas, too often the answer is not the one we'd like to hear.

We started out so well

Almost every system, if you have at least decent programmers working on it, starts out as a wonderful green field: defined architecture, code conventions, solid principles, patterns and practices.

After a while the gardeners get busy with other gardens, and a couple of weeds start to poke out. Then some more, then more. And more. Meanwhile seasons pass and rain falls, so our green field gets muddier and muddier. Actually it looks more like it has rained cats and dogs, day after day after day. Mud becomes quicksand, and nobody dares to even get close to it.

The (in)famous spaghetti incident

Too soon the project reaches the point in which whenever you pull a string you have an undesired and often undetected change in a completely different part of the system. When I want to sound important I refer to this as the butterfly effect, a metaphore used in chaos theory. Yes, I know that your projects are different and you have never seen it happen, so try to use your imagination and stay with me a little more, will you?

That was exactly where I was, all tangled up in obscure dependencies: the perfect spaghetti code.


As I tweeted, I love spaghetti when they are in my dish, not in my code, so I had to do something (even because deadlines are always waiting in ambush). To sharpen my tools I retrieved my copy of "Working Effectively with Legacy Code". The book covers many aspects of what we developers have to face daily: instead of the "write once, run anywhere" mantra, we have to deal with "write once, read it at least ten times", but too often our code is obscure. Yes, our code. And we don't have tests, or at least we don't have enough. Do you have a complete coverage? If you do, you have all my respect, otherwise welcome in the family.

How do we get out of this?

First of all, we have to get our code into a test harness, but it is not as easy as it sounds. The book contains a series of 24 different dependency breaking techniques, each of which presented in several almost-real situations, to minimize the impacts of changes and ensure that we are not breaking anything. It sounds like the classic catch-22 thing:

When we change code, we should have tests in place. To put tests in place, we often have to change code.


This is where the techniques come in handy. I was pleasantly surprised when I discovered that, after some years of TDD, almost every problem and solution described in the book sounded familiar, so I skimmed it more than actually reading it. I still remember when I first read it: man it was hard, not the writer's fault, but because of the reader.
One of the most important things is that you have to preserve the behavior of your system when you refactor it (and preserve the rest of the system when you introduce new features or change existing ones), so you should at least know what it does. What is better than some characterisation tests?

Cleaning it up

I love NetBeans. I always have, since it was Forte4Java. Now I love it even more because it has a heap of automated refactorings that make "Refactoring" needless (ehm... that's not true, NetBeans deals with the mechanics, but you should know what you're doing, so stop reading and go buy your copy if you still don't have it. Done? OK, let's get on).

Do you need to create a new test class? nothing simpler, just a simple SHIFT + CTRL + U and a template with all the skeleton methods is ready for you. Just punch in the starting conditions and the expected results and you're done. Feathers describes all refactorings steps by steps, but a few keystrokes are all NetBeans needs to extract superclasses, methods, interfaces, pull up or push down members and methods, delegate, and so on and so forth. Nevertheless, you should know what you're doing.

While working your code could get uglier in some places. It could be temporary, or temporary in the Italian way (which means definitive). Even in the latter case, at least you would have tests in place so that you know you're not breaking anything.

In several places Feathers suggests to reduce incapsulation to put code into a test harness. Somebody might find it strange, extremists might find it insane. Yet...

Encapsulation isn't an end in itself: it is a tool for understanding.


Remember when I wrote that you write code once and read the very same code over and over? What 's the point of an encapsulated design if I spend hours for the most trivial task? Mnd you, I'm not suggesting every field must be public and collections should be directly exposed, but nobody is going die if a private method becomes protected for the sake of testing.

Where have my spaghetti gone?

After a while, one after the other, your pull spaghetti out of your dish, slowly replacing them with neat and tidy sushi.

Before you realize it, you will look at your codebase with different eyes, and chances are that you will enjoy dealing with legacy code. Don't enjoy too much though, even if you have a lot of technical debt to pay: you don't have to pay it for the sake of doing it, but because it allows you to move faster when you need it (and you know you will need it).

Let me stress this again: if I didn't have some hundreds of tests already in place, these weeks would have been months. Baby steps, nip and tuck, red and green bar. But I only introduced tests and refactored legacy code when I needed it, otherwise I would spend too much time on activities that were not a priority. See something rotten? pen it down, fix what you're doing and if you still have time you can deal with it. Look for decisions that could change, but change your code just in time (the YAGNI rule rules).

Tuesday, August 31, 2010

How to refactor and test a SAAJServlet

Mind you, not a simple SAAJServlet, but an old fashioned legacy transaction script without the scent of a test. Anyway... let's pick up the usual books (Working effectively with legacy code, Refactoring, Refactoring to patterns - just a starting point of course) to keep at hand as a reference and let's start refactoring.

First a word of caution: don't just copy from the books. Study them, try to apply the refactorings in a small and controlled environment, maybe in little katas, and get the basics beyond them, otherwise you'll do more harm than good.

Back to business. We basically have to test the onMessage method, that gets a SOAPMessage and returns another one. Without even looking at the code, one would expect an algorithm like the following one:
  • parse the message into a parameter
  • pass it to a service that executes whatever must be executed
  • build a new message based on the input parameter and on the outcome of the previous step
Obviously one should also consider errors management, but let's keep it simple. Let's take a look at the actual code:

@Override
public SOAPMessage onMessage(SOAPMessage message) {
SOAPMessage result = null;
Dialogue dialogue = null;
try {
SOAPBody body = message.getSOAPBody();

SOAPFactory sFactory = SOAPFactory.newInstance();
Name bodyName = sFactory.createName("myLocalName", "myPrefix", "myUri");
Iterator it = body.getChildElements(bodyName);
if (it.hasNext()) {
SOAPBodyElement be = (SOAPBodyElement) it.next();
dialogue = parseBodyElement(be);
Operation operation = operationFactory.getOperation(dialogue);
if (operation != null) {
try {
operation.execute(dialogue);
} catch (Throwable th) {
getServletContext().log(th.getMessage(), th);
dialogue.setCode(Dialogue.KO);
dialogue.setMessage("Service unavailable.");
}
}
} else {
dialogue.setCode(Dialogue.KO);
dialogue.setMessage("Malformed message.");
}
} catch (Throwable th) {
getServletContext().log(th.getMessage(), th);
dialogue.setCode(Dialogue.KO);
dialogue.setMessage("Service unavailable.");
} finally {
try {
MessageResponseGenerator mrg = new MessageResponseGenerator();
result = mrg.generateResponseMessage(dialogue);
} catch (Throwable th) {
result = null;
th.printStackTrace();
}
}
return result;
}
Now, though PMD does not complain too much about the structure of the method - just a few dataflow anomaly analysis, null assignments and catching throwables - I smell (more than) a rat.

Starting from the beginning, the first things that strucks me is the different level of abstractions of the code: you have both a low level implementation that extracts a SOAPBodyElement to be parsed and a call to an Operation that encapsulates the requested service (the Command pattern), and this is not good. There are also too many nested ifs and try-catch blocks to my liking.

There is a separated object that creates the response message, and this is good. Why on earth are we missing an object or at least a method that parses the message in the first place?

Why do we ask a factory for an Operation passing in a Dialogue and then we have to pass the same parameter back to the operation we get back from the factory? couln't the operation hold a reference to the dialogue? Would it be better? we shall see later.

And, obviously, all the references are hard-coded...

So far, we have just conducted a small code review. But what's the point in the first place? We have to introduce a new Operation object (maybe extending an existing one, and please note the alliteration). Nothing strange so far, but the resulting object for this operation should contain some more fields than the existing one. Still nothing terrible, but as you might have noticed currently the Dialogue object works as a collecting parameter through all the method, while we would need two different objects (or modify the initial parsing method in such a way to have an instance of the right Dialogue subclass, or introduce a composition in the Dialogue class, or explore a thousand other possibilities).

By the way, it is mandatory that we do not change the interface for the service, which means that the incoming and outcoming messages for the existing operations cannot change.

The fact is that before we start refactoring we have to guarantee that. Of course we could start refactoring right away, but, as you might have noticed if this is not the first post of mine you happen to read, I find it "just a little bit" risky.

So the answer to the first question, i.e. how to refactor a SAAJ servlet, is, at least in my mind, simple: write enough tests to describe the current behaviour before you do anything else. I know: easier said than done.

That's tricky: how can you do that? The class we have to test is a servlet, so we have four options:
  • use manual testing: we already have this in place, but each single change would require at least a dozen tests, making the whole process too slow (and not reliably repeatable, as it is based on people's good will)
  • mock all the objects provided by the servlet container needed by the servlet: quite a lot of work, and I'm not going down that way (even because I'm listening to Starway to Heaven and I don't wont to spoil the happy sensation)
  • deploy the application to a servlet container and use in-container testing, possibly with a framework like Cactus: this introduces an unwanted complexity and I'll avoid it too even if it would be tempting, as the servlet is deployed in a proprietary framework in which you can only access the database if you actually start the webapp (don't ask, you don't want to know). Why? because I think it would not force me to separate responsibilities as much as the next approach.
  • use a library such as HttpUnit, which contains ServletUnit. Being this a simulated container I am sure I would never have access to the database given the current situation, so I guess I'll have to work my way to a cleaner design.
Back to the servlet: there are cases in which the method returns a message without going all the way down to the database, so as a first step I'll exercise these and keep manual testing for end-to-end situations.

The very first test consists in producing a message with a wrong format: I should get a SOAPMessage with the code property set as KO and an error message of "Malformed message". Or should I?
@Test
public void testOnMessageWithMalformedMessage() throws IOException, SAXException {
System.out.println("testOnMessageWithMalformedMessage");

InputStream resourceAsStream = this.getClass().
getClassLoader().
getResourceAsStream(
"path/to/my/message.xml");

ServletRunner sr = new ServletRunner();
sr.registerServlet("myServlet", MyServlet.class.getName());

ServletUnitClient sc = sr.newClient();
WebRequest request = new PostMethodWebRequest(
"http://localhost:8084/myServlet",
resourceAsStream,
"text/xml");

WebResponse response = sc.getResponse(request);
assertNotNull("No response received", response);
assertEquals("OK", response.getResponseMessage());
assertTrue(response.getText().contains("KO"));
assertTrue(response.getText().contains("Malformed message."));
}
These steps are detailed in the ServletUnit documentation. Before we run the test, remember we're not going to change anything yet.

Too optimistic? actually I was. Running the tests gives me the following error:
Error on HTTP request: 500 javax.servlet.ServletException: SAAJ Post failed null.
Looks like the MessageResponseGenerator cannot handle null parameters very well. I admit I knew I had been too optimistic about the error message (how could I trust the generator had the same message the servlet used and that I was expecting?), but at least I hoped in a valid SOAPMessage... But after all isn't the NullPointerException the most widespread error in Java code?

It's a long way to go... remeber that before we change the current behaviour we have to describe it, so the way is even longer. Yet, the first step has been made.

Wednesday, February 18, 2009

Debt Metaphor

Ward Cunningham talks about the Debt Metaphor. As he is the one who started using it, it is supposed he knows what he's talking about...

Friday, January 30, 2009

Tangled and circular dependencies

Back to the cyclical dependencies problem I wrote about one year ago, the other day I finally tackled it (15 years spent playing rugby must have taught me something), as it was left hanging in the air for too long. We started out like this:

Basically project B uses the domain model and persistence services from project A, which in turns uses transport services and a the domain model from project B (no, it is not the same as above, because the model B is just based on model A but it is managed from a different perspective). Let's not forget to say that presentation, model and services are all tangled.

A disclaimer: this is not what I'd call a refactor, just a different way of organizing projects thanks to NetBeans features (no Maven complexities, thank you).

I first extracted the core domain classes of project A used in project B and moved them to a new project, let's call it A Core, then had both project A and B depending on this one. Sounds easy, but it wasn't... it was quite long and tiring, and it required a little tweaking each now and then, just like the subsequent passages. That lead me here:


Now that could look good, but even if it is better it was not enough because of some legacy installations in which Project A must be compiled using JDK 1.4 and run on a Tomcat 4, but project B has a JDK 1.5 source level (Tiger, or whatever it is called now) and runs on a Tomcat 6. This forced me to extract from Project B all classes on which Project A depended to move them on a new project, let's call it B core. Nip and tuck, here and there, to land that prince with the perfect hair, and here we are:


Now all cyclical dependencies are gone. Packages with the very same name exist in different projects, leading to a namespace which "doesn't talk" to the developer; on the other hand this can bring a lot of confusion, even if the compiler understands everything perfectly. That's where fun begins... I just began scraping the surface, now we have to dig in deep. The first step will be choosing the right names for packages, so to convey the rationale behind their repositioning. The next one will be moving related classes left untouched by the dependencies game but logically belonging to one of the "core" projects. Other steps are still up in the air...

Moral: always clean up your mess. BTW, it does not matter if you just inherited it. Now it is yours, and it is you the one who has to deal with it. Leave spaghetti to gravy and keep them out of your code. There must be a reason if they say "clean code that works" and not just "code that works"...

Friday, January 23, 2009

The birthday greetings kata

On February the 14th the milano-xpug organized a kata whose subject was proposed by Matteo on his blog to introduce the exagonal architecture.

Unluckily I couldn't attend the meeting in the flesh, so I just decided to try the kata out here on my own. I first started from scratch, thinking it would prove more fun; then, as I was taking quite a different approach - you could also say that a whole different design was emerging - I decided to get back and start from the original code provided and refactor it to try and focus on the architectural issues which were the aim of the exercise.

Even if I think this exercise to be quite simple, I decided to post a trace anyway because someone might still find it useful. You can never say!

The heart of the application is the BirthdayService class, whose responsibilities consist in identifying employees whose birthday falls on a given date and sending them a greeting message.

This is the public method, called by the acceptance test:


I know it looks odd, but I've had enough of escaping characters and trying to format code to make it look nice, so I hope you will forgive me.

Anyway, the sendGreetings method parses an input file, creates employees, checks if they are eligible for a greeting message and delegate the actual sending of the message to a private method:


Seems a great bunch of things for a single class... let's try to sort it out. First I decoupled the service from the file with the employees' names introducing the interface EmployeeRepository, with the method getEmployees, then I wrote a FileSystemEmployeeRepository that implemented it. The repository was then injected into the BirthdayService via constructor by
the acceptance test. At this point the code looked like this:
...
List<Employee> employees = repository.getEmployees();
for (Employee employee : employees) {
if (employee.isBirthday) {
//compose message and call method that sends it
}
}
...
Still I thought that the check on the date of birth could be done by the repository itself, so I changed the interface adding the new findEmployeesBornOn(OurDate) method and changed the code so that it looked like this:
...
List<Employee> employees =
repository.findEmployeesBornOn(ourDate);
for (Employee employee : employees) {
//compose message and call method that sends it
}
...
Maybe not a very big change but I think it was worth it, as now the code is cleaner. Then there was the messaging part. First of all I decided to introduce a new BirthdayGreeting class, that produces a message body, a fixed subject and a recipient starting from an employee; at this point I extracted the interface SimpleMessage that exposes these three strings. The code had become
...
if (employee.isBirthday) {
SimpleMessage greeting = new BirthdayGreeting(employee);
sendMessage(smptHost, smptPort, "sender@here.com", greeting);
}
...
Still I had to resolve the dependency from the SMTP, so just like I did for the repository I introduced the MessagingService interface with the sendMessage(String sender, SimpleMessage message) method and injected it into the BirthdayService via constructor from the acceptance test. Some nip and tuck to move the body of the sendMessage method from BirthdayService into a new EmailService, implementer of MessagingService, and voila, the new BirthdayService code:
public void sendGreetings(int month, int day) {
List<Employee> employees =
repository.findEmployeesBornOn(month, day);
for (Employee employee : employees) {
BirthdayGreeting greeting = new BirthdayGreeting(employee);
messagingService.sendMessage("sender@here.com", greeting);
}
}
I also changed the method parameter from OurDate to month, day. Now we have a simple and decoupled service that can focus on its true responsibilities: ask for all people who should receive a greeting, create a new message and delegate its sending.

One might wonder whether the sender belongs where I put it, or if the service should directly instantiate a BirthdayGreeting with a new rather than getting it in some other way. But maybe that's going too far.

BTW I also decided not to use the Employee.isBirthday() method and I created a value object called IsBirthday, created with two parameters representing month and day, with the boolean method forEmployee(Employee). The code in the repository now looks like this:
public List findEmployeesBornOn(int month, int day) {
List<Employee> list = new ArrayList<Employee>();
IsBirthday isBirthday = new IsBirthday(month, day);

for (Employee employee : employees) {
if (isBirthday.forEmployee(employee)) {
list.add(employee);
}
}
return Collections.unmodifiableList(list);
}
That's all folks... maybe I'll manage to post the complete code here, where you can find some other (and more accredited) solutions produced by people who actually were there.

Friday, August 1, 2008

Gain a better understanding of code

What motivates us to refactor? Joshua Kerievski asks (and answers) himself this very same question. One of the reasons he gives is to "gain a better understanding of code". To quote:

Sometimes we look at code and have no idea what it does or how it works. Even if someone could stand next to us and explain the code, the next person to look at it could also be totally confused. Is it best to write a comment for such code? No. If the code isn't clear, it's an odor that needs to be removed by refactoring, not by deodorizing the code with a comment.

I'd like to stress the second part of the quote. Is this the way we behave? I'd like to say yes, but too often I don't. The most recurring reason for which this happens is that the typical codebase on which you experience this problem has no test harness. I know, I know, I did not only buy the Feathers', I also read and studied it: everything looks neat and beautiful, but sometimes the processes described can be really exhausting, and you obviously come to fear changes (which is against XP values, for example).

Now, to everyone who thinks Star Wars to be just a useless bunch of movies, I'll show you how it can help you experiences professionals through. Powerful master Yoda said:

Fear is the path to the dark side.

The dark side is represented by hacks'n'comments. When you find them, you should follow Darth Sidious' advice:

Wipe them out. All of them.


Extreme as it might look, this is by far the best solution in the long run. And... May the force be with you.

Thursday, July 3, 2008

Is Refactoring a Necessary Waste?

In this article I found an interesting definition: necessary waste, which is qualified as "the best way we currently know how to do a job even if it does not have a customer value". Following this line Amr Elssamadisy says that refactoring - a necessary waste - is not valuable to the customer and that you should only "refactor code that no longer meets the requirements of the user".

I strongly disagree with Amr for a bunch of reasons.

The first and simplest of all is this one: refactoring eventually SAVES time, as once you'we written your software you AND many other people will have to read it. A complicated code thus implies more time needed to understand it, and probably much more time to safely modify it. And guess who is going to pay for it? Saved time IS valuable to the customer, as it translates into considerable money savings.

Leaving aside all other reasons and considerations you can find in literature, I'll have to add this one: I love clean and elegant code, just as I liked clean and elegant mathematics expressions since when I was at school; I strongly agreed with my regretted maths teacher, who used to value more students who presented elegant solutions. Unaesthetic code simply disturbs me. I'm not saying all my code is elegant and perfect (I'd like to, but it would be a huge lie), but that's what I aim to when I'm coding. This is one of the reasons for which I've joined the anti-if campaign (I hear an English version will soon be available so stay tuned on the site).

One caveat: if you're working on a piece of code and find something else that smells don't switch your tasks, even if the refactoring is (looks) trivial: just pen it down and come back later. I find it a much better strategy, or at least it is for me - and for mr. Kent Beck:

We'll make a to-do list to remind us what we need to do, to keep us focused and to tell us when we are finished. ... When we finish an item, we'll cross it off.


(Quoted from Test-Driven Development: By Example)