On repositories

I was writing code for a (actually not so) massive data export in a legacy application and I needed to retrieve all objects of a certain type modified after a given date, so I started writing a repository.

The simplest test that came into my mind was something like this:

@Test
public void queryingWithDateInTheFutureReturnsEmptyList(){
  DateTime tomorrow = new DateTime().plusDays(1);
  List<MyObject> result = instance.findModifiedAfter(tomorrow);
  assertTrue(result.isEmpty());
}

Some clickety-clack (SHIFT+CTRL+I, ALT+SHIFT+F, CTRL+s, CTRL+F6 for the most curious) and NetBeans gives me a red bar. That's good as there's no code yet :-)

The first implementation was really too easy to write it, but as a very few people believe I work this way and all the rest think I'm wasting time I'll publish it anyway, saving my opinion for another occasion:

public List<MyObject> findModifiedAfter(DateTime startDate){
  return Collections.emptyList();
}

So far, so good. Now, in the framework used in this project all persistent objects inherit from MyPersistenceObject (names have been changed to protect the innocents), which is a sort of Active Record with a hint of Row Data Gateway. Persistent object may also be managed by a MyPersistenceObjectController object, which is a sort of DAO that also manages transactions. Should you wonder, the class is not a controller at all, but the original developers thought that the name would fit. Before we go on, let me say that the framework works pretty well under many circumstances.

Not having a dedicated database that gets populated for each test end cleaned afterwards (did I mention the fact that this is a legacy application?) the next test tried to retrieve some data:

@Test
public void queryingSinceLastMonthReturnsAtLeast5kObjects(){
  DateTime lastMonth= new DateTime().minusMonths(1);
  List<MyObject> result = instance.findModifiedAfter(lastMonth);
  assertTrue(result.size() > 5000);
}

I know it's not great, and that it breaks if someone truncates the corresponding table, but we must start from somewhere, right? Reading the tests also suggests that the method should probably be named findModifiedSince, but that is hardly the point now - even if it shows another useful feature of writing tests.

A little lookup on the existing code easily gave me a first implementation based on the "glorious" copy-paste-fix pattern:

public List<MyObject> findModifiedSince(final DateTime date) {
  if (date == null || date.isAfterNow()) {
    return Collections.emptyList();
  }

  try {
    Object[] values = new Object[]{date.getMillis()};
    String[] orderBy = null;
    Criteria criteria = new Criteria("tms", "MyObjectImpl");
    SimpleCondition simple = new SimpleCondition("tms", SimpleCondition.GE);
    criteria.add(Criteria.NOP, simple);
    Vector objects = new MyObjectImpl().retrieveByAlternateKey(
            criteria,
            values,
            orderBy);
    List<MyObject> result = new ArrayList<MyObject>();
    result.addAll(objects);
    return result;
  } catch (Exception ex) {
    return Collections.emptyList();
  }
}

Clickety-clack, CTRL+F6...

...
...
(yawn)
...
...
(wtf?)
...
...

Green bar. After an insane amount of time. A little logging informed me that the test took 16 seconds to run, excluding the time needed to start and stop the persistence container.

A little profiling confirmed that also the memory usage grew abnormally. And all this for a little more than 6000 objects...

Now, to quote Eric Evans, a repository is

...an object that can provide the illusion of an in-memory collection of all objects of that type.

Well, this framework cannot provide that illusion, at least not whithout freezing everything else. I must admit that lately I was quite in clover, as using Hibernate I could simply write the very same method like this:

public List<MyObject> findModifiedSince(DateTime date) {
return HibernateUtil.getSession().
 createCriteria(MyObject.class).
 add(Restrictions.ge("tms", date)).
 setResultTransformer(Criteria.DISTINCT_ROOT_ENTITY).
 list();
}

and get my list, which is actually a list of proxies, in about no time. Now Evans says that repositories

...return fully instantiated objects or collections of objects whose attribute values meet the criteria, thereby encapsulating the actual storage and query technology.

Proxies are not exactly fully instantiated objects, but as they pretend to be I'm prepared to live with that :-)

At this point all I could do was to revert to the old give-me-the-ids-and-I'll-get-the-objects-myself :-(

Create or update?

A couple of days ago a bunch of sectors of my hard disk decided to vanish into the blue, bringing with them some files of my local repository (no worry about that) and some files of my NetBeans installation, configuration files included (no worry about that too, but a little more work required). While my colleagues are providing me with a new hard disk, on which I asked for a fresh installation (before you ask: it is a company policy that we only use operating systems from Redmond on our personal computers) I am working on the laptop. Which, by chance, happens to be a fresh installation too as it is the replacement for my old one (who can nowadays use NetBeans AND Firefox AND SQLServer AND all the rest, antivirus included, on 1GB? Maybe 2GB is not that much, but at least it doubles my previous supply)

Ok, so after reinstalling and reconfiguring my stuff I checked out a group of three projects we're working on. Projects that our Hudson guarantees as healthy (see one below).


Open the group of projects, set as main, fix the configurations for my local databases (for each database we have one instance for manual and acceptance tests and one for the automated ones) configurations clean and build, test (just to be sure).

91% passed. What? That means fail! but... what's wrong? mmm... it seems there's a row in the User table that causes Hibernate to complain about constraint violations. That is strange, as - as far as I know, but I obviously might be wrong - all our automated persistence tests run with HibernateOpenViewSession, a simple Runner that before each test opens a Session and rollbacks everything after the invocation (nothing new under the sun, but quite handy):

@Override
protected void invokeTestMethod(Method method, RunNotifier notifier) {
 try {
     final Session session = HibernateUtil.getSessionFactory().getCurrentSession();
     session.beginTransaction();

     super.invokeTestMethod(method, notifier);

     if (session.isOpen()) {
         session.clear();
         session.getTransaction().rollback();
     }

     if (session.isOpen()) {
         session.close();
     }
 } catch (Throwable t) {
     System.err.println("HibernateOpenViewSession RUNNER FAILURE");
     notifier.fireTestFailure(new Failure(Description.EMPTY, t));
 }
}

Yet, that row is still there. Well, maybe one of our tests doesn't use our favourite runner. So, why doesn't Hudson complain? After a little investigation it turned out that the Hibernate configuration for our CI server has a hibernate.hbm2ddl.auto=create property, where my local file had update. That fixed, all my local files ran smoothly and gave me a green bar.

Now the hunt for Red October will start. Ehm... I meant the hunt for the culprit test, of course... I'll keep my eyes on the ball.

By the way, we almost reached our 1000th test on the project :-) and it is only part of a group of three... it may sound a little to many out there, but for us it is a great success.

As a parting note, let me stress again the importance of a CI server to avoid the "it runs on my machine" syndrome - which in this case was "it doesn't work on my machine" :-)

A flexibility contest

The introduction in the Hibernate documentation reports that

A relational database is more flexible than a network programming language, so it doesn't need anything like a navigation direction - data can be viewed and retrieved in any possibile way.

As this is quite a statement, I stopped to consider it a while.

In a programming language you have to consider navigability, so if you have - as in the example - a Person class and an Event class and you want to get all events associated to a person you'll have to ask the person with something like person.getEvents(), but if you also want to get all persons associated to an event you'll have to ask the event with event.getPersons(), i.e. you have to provide accessors to both classes. And both classes hold a reference to a collection of their companion class. Moreover, when you add a person to an event you have to make sure that also the event gets modified, and vice versa.

In an RDBMS you can compose all the queries you like, starting from persons or events; the only thing you'll need is a well defined relation between the tables, which tipically translates into a bridge table. You don't even need to enforce referential integrity, though this is the path to the dark side (or is it?), as long as everybody knows how the tables should be connected. So when you add a person to an event you only have to insert a row in the bridge table.

In this sense databases are more flexible than programming languages. You can write

select *
from person p
left outer join person_event pe
on p.id = pe.person_id
left outer join events e
on pe.event_id = e.id

as well as

select *
from event e
left outer join person_event pe
on e.id = pe.event_id
left outer join person p
on pe.person_id = p.id

but this comes at a cost. You need keys. And you need indexes to manage them (even if this relates to the phisical aspect rather than to the logical one). And a bridge table to manage the many-to-many relationship, which doesn't belong in the domain at all but is a pure fabrication used to represent the model. This clutters things a bit, even if we're so used to it that we don't even notice.

The Java language gives a much clearer representation of the domain (at least, the signatures of the method do):

public class Person {
  // stuff
  public List getEvents() {...}
  public void addEvent(Event event) {...}
}

public class Event {
  //stuff
  public List getPersons() {...}
  public void addPerson(Person person) {...}
}

The language shields you from how the model is represented (that's where the rigidity is hidden), and it is more natural to use.

Databases can use views to simplify data retrieval, but they don't work for inserts and updates. That's where stored procedures come in handy, but... aren't they written in a programming language? Should the latter be considered part if the database or as an external means to get a higher abstraction over SQL (which in turn is a language of its own)? How far should we push this?

My conclusion? Everyone should get their act together to make sure that the quality of what they build is the highest that they can get, either when designing a good database schema or when implementing a bidirectional association. The problem is that "the highest they can get" can sometimes be a low standard, so everyone should always try to improve, no matter how high the standard. I think this rule applies not only to programmers in their own field of competence, but to men in the broader sense.