How to use Java to call an IBM i program

This post is a late update to a post I wrote a couple of years ago about exposing RPGLE programs so that we could call them from our webapps. That worked fine, but I didn't particularly like the workflow we used to produce the wrapper, so I tried to get rid of some muda.
The worst one was, in my opinion, the need to use two different IDEs, thus necessarily involving the collaboration of two persons (not every developer has IBM tools) that creates a bottleneck.
After some researches I found some official documentation (the problem with IBM is not scarcity but abundance), dug for our jt400.jar and started to experiment a little.

A small word of caution: the article is written in an introductory style, showing the TDD process that lead to the implementation of the functionality I needed. If you're only interested in the results you can easily skip to the end of the article (I will not know it, so I won't get offended).

The guinea pig for my experiment is a simple program that given a table name returns a serial number which is unique for that table. As I had have to use this program in an application that already uses a serial number generator, first I need an interface, so I use the NetBeans Extract Interface refactoring on my existing service:

public interface SerialsService {
    long getSerialNumber(String tableName);
}

That done I create an implementing class that gets the serial from the iSeries:

public class MyIseriesSerialsService implements SerialsService {
    public long getSerialNumber(String tableName){
        throw new UnsupportedOperationException();
    }
}

OK now we can start the real stuff. First thing first, we write a couple of simple tests in a new test class:

@Test(expected = "IllegalArgumentException.class"
public void testWithWrongTableNameShouldThrowException() {
    MyIseriesSerialsService instance = new MyIseriesSerialsService();
    instance.getSerialNumber("WrongName");
}

CTRL + F6 to start the test and NetBeans gives me a red bar. I change the type of exception thrown, CTRL + F6 again and here's a nice green bar. That was easy, wasn't it? now we have a service that behaves correctly when you don't invoke it correctly. This is good, but if your application is like mine I suppose you expect your service to behave correctly even when it is correctly invoked (yes, sometimes it happens).

To call an IBM i program you first have to connect to an iSeries, and for this you use the AS400 class, which IBM documentation actually calls AS400 IBM® Toolbox per Java™: I'll stick to that, so that is what I mean when I simply write AS400 - the same holds true for all other registered trademarks. If this is enough to keep lawyers at bay, as I hope, this ends the disclaimer.

The AS400 class, amongst other things, manages socket connections on behalf of a user, so at least we have to tell our service what is the iSeries we want to connect to, and which user we want to impersonate. This requires to change the constructor of the service (obviously it is not the only way, but I'd rather use an immutable object). We could pass in the name of the server (or the ip address if you don't want to rely on a DNS service), a username and a password, or we could create an AS400 object and pass it to the constructor of the service. This option separates responsibilities better (and can be tested more easily), so I choose it and write a simple test to check the connection to our system:

@Test
public void testConnection() throws Exception {
    AS400 system = new AS400(host, username, password);
    assertNotNull(system.getRelease());
    System.out.println(system.getRelease());
}

I am just experimenting, so as always I defer all the exception dealings. Green bar. I can now add a field to the constructor of the service...

public class MyIseriesSerialsService implements SerialsService {

    private AS400 system;

    public MyIseriesSerialsService(AS400 system) {
        this.system = system;
    }

    public long getSerialNumber(String tableName) {
        throw new IllegalArgumentException();
    }
}

...and modify the tests accordingly, also removing a small duplication:

@Before
public void setUp() {
    instance = new MyIseriesSerialsService(createValidSystem());
}

private AS400 createValidSystem() {
    return new AS400(host, username, password);
}

@Test(expected = IllegalArgumentException.class)
public void testWithWrongTableNameShouldThrowException() {
    instance.getSerialNumber("WrongName");
}

Skipping (for the sake of the article) all tests on invalid systems and other exceptions we can now start writing a more interesting test:

@Test
public void testGetSerialNumberReturnsPositiveNumber() throws Exception {
    long result = instance.getSerialNumber(tableName);
    assertTrue(result > 0);
}

CTRL + F6, red bar (as expected). This is where the rubber hits the road.

The MYPGM program is contained in the MYLIB library in the QSYS, and it also uses the MYOTHERLIB library (by the way, watch for the maximum length of the names of the libraries, which is 10). The wrapper class for a program call (the Command pattern in action) is ProgramCall (fantasy does not abound in IBM, but that's good as I easily found what I was looking for). It needs to know the system we're operating on, the path to the program and the parameter list. We have the system, so we need the path to the program, for which we use the QSYSObjectPathName class, and the parameter list, for which we use the ProgramParameter class.

The QSYSObjectPathName constructor takes as parameters the name of the library, the name of the program and it extension. Once we have a QSYSObjectPathName object we can ask it the path to the program, which is what precisely what we need:

QSYSObjectPathName pgmName = new QSYSObjectPathName(myLib, myPgm, pgmExtension);

The parameter list is actually an array of ProgramParameter objects, which cannot be null. Each parameter wraps an array of bytes, so we have to use the proper converter; luckily there are some converters ready for us.

Our program has an input/output parameter (the name of the table) and an output parameter (the serial number for the given table), so we have the following:

ProgramParameter[] paramList = new ProgramParameter[2];

The first parameter is the name of the table, which is a String. To deal with it we can use the AS400Text converter:

AS400Text textConverter = new AS400Text(10, system);
byte[] key = textConverter.toBytes(tableName);
paramList[0] = new ProgramParameter(key);

The first parameter for the constructor is the length of the IBM i text, the second one is the system we're operating on.

The output parameter is a long, so we'll have to use a different converter. Up to now let's just initialize it:

paramList[1] = new ProgramParameter(32);

Our command is now ready to be born:

ProgramCall pgm = new ProgramCall(system, pgmName.getPath(), paramList);

To execute it we simply call the run method, which returns a boolean. If the result is true we can extract data from the output parameter and convert it:

byte[] data = paramList[1].getOutputData();
AS400PackedDecimal pdconverter = new AS400PackedDecimal(12, 0);
long result = ((BigDecimal) pdconverter.toObject(data)).longValue();

And this is it. Let's see:

Testcase: testGetSerialNumberReturnsPositiveNumber(testserialias.MyIseriesSerialsServiceTest):
Caused an ERROR
6

Wow that's illuminating... Luckily we aldready know what's going on: if you check above you'll notice I wrote that the program needs another library, which is not loaded when we first connect to the system. To add the library we need to issue a command, thus we use the CommandCall class:

CommandCall cc = new CommandCall(system);
cc.setCommand("ADDLIBLE " + myotherLib);
cc.run();

Now we're going somewhere... As we trust but also want to control we add another simple test to check that the service returns bigger numbers on consecutive calls:

@Test
public void testGetSerialNumberReturnsBiggerNumbersOnFurtherCalls() {
    firstResult = instance.getSerialNumber(tableName);
    long secondResult = instance.getSerialNumber(tableName);
    assertTrue(firstResult < secondResult);
}

If we want to have more informations we can ask the program wrapper for an array of AS400Message objects:

private String buildMessage(final ProgramCall pgm) {
    AS400Message[] messageList = pgm.getMessageList();
    String message = "";
    for (int i = 0; i < messageList.length; i++) {
        AS400Message aS400Message = messageList[i];
        message += aS400Message.getText();
        message += "\r\n";
    }
    return message;
}

The RPGLE program already ensures that no serial can be duplicated, but I thought that adding a little bit of safety wouldn't be too much damage, so the main call is synchronized. The final draft:

import com.ibm.as400.access.AS400;
import com.ibm.as400.access.AS400Message;
import com.ibm.as400.access.AS400PackedDecimal;
import com.ibm.as400.access.AS400Text;
import com.ibm.as400.access.CommandCall;
import com.ibm.as400.access.ProgramCall;
import com.ibm.as400.access.ProgramParameter;
import com.ibm.as400.access.QSYSObjectPathName;
import java.math.BigDecimal;
import java.util.logging.Level;
import java.util.logging.Logger;

public class MyIseriesSerialsService implements SerialsService {

    private final String myLib = "MYLIB";
    private final String myOtherLib = "MYOTHERLIB";
    private final String myPgm = "MYPGM";
    private final String pgmExtension = "PGM";
    private final AS400 system;
    private QSYSObjectPathName pgmName;

    public MyIseriesSerialsService(final AS400 system) {
        this.system = system;
        addLibraries();
        createPathName();
    }

    public long getSerialNumber(final String tableName) {
        synchronized (this) {
            ProgramParameter[] paramList = createInputParameters(tableName);
            ProgramCall pgm = createProgramCall(paramList);

            try {
                pgm.run();
            } catch (Exception ex) {
                Logger.getLogger(MyIseriesSerialsService.class.getName()).
                        log(Level.SEVERE, ex.getMessage(), ex);
                String errorMessage = buildMessage(pgm);
                throw new RuntimeException(errorMessage, ex);
            }

            long result = extractResult(paramList);

            return result;
        }
    }

    private void addLibraries() {
        try {
            CommandCall cc = new CommandCall(system);
            cc.setCommand("ADDLIBLE " + myOtherLib);
            cc.run();
        } catch (Exception ex) {
            Logger.getLogger(MyIseriesSerialsService.class.getName()).
                    log(Level.SEVERE, ex.getMessage, ex);
            throw new RuntimeException("Unable to inizialize service", ex);
        }
    }

    private void createPathName() {
        pgmName = new QSYSObjectPathName(myLib, myPgm, pgmExtension);
    }

    private ProgramParameter[] createInputParameters(final String tableName) {
        ProgramParameter[] paramList = new ProgramParameter[2];
        paramList[0] = createInputParameter(tableName);
        paramList[1] = new ProgramParameter(32);
        return paramList;
    }

    private ProgramParameter createInputParameter(final String tableName) {
        AS400Text textConverter = new AS400Text(10, system);
        byte[] key = textConverter.toBytes(tableName);
        return new ProgramParameter(key);
    }

    private ProgramCall createProgramCall(final ProgramParameter[] paramList) {
        return new ProgramCall(system, pgmName.getPath(), paramList);
    }

    private long extractResult(final ProgramParameter[] paramList) {
        byte[] data = paramList[1].getOutputData();
        AS400PackedDecimal pdconverter = new AS400PackedDecimal(12, 0);
        return ((BigDecimal) pdconverter.toObject(data)).longValue();
    }

    private String buildMessage(final ProgramCall pgm) {
        AS400Message[] messageList = pgm.getMessageList();
        String message = "";
        for (int i = 0; i < messageList.length; i++) {
            AS400Message aS400Message = messageList[i];
            message += aS400Message.getText();
            message += "\r\n";
        }
        return message;
    }
}

And the test class which assisted me in all the small refactorings:

import com.ibm.as400.access.AS400;
import org.junit.After;
import org.junit.AfterClass;
import org.junit.Before;
import org.junit.BeforeClass;
import org.junit.Test;
import static org.junit.Assert.*;

public class MyIseriesSerialsServiceTest {

    private final String host = "myVeryExpensiveHost";
 private final String username = "myUsername";
    private final String password = "myPassword";
    private final String tableName = "myTable";
    //
    private MyIseriesSerialsService instance;

    @Before
    public void setUp() {
        instance = new MyIseriesSerialsService(createValidSystem());
    }
    private AS400 createValidSystem() {
        return new AS400(host, username, password);
    }

    @Test
    public void testConnection() throws Exception {
        AS400 system = new AS400(host, username, password);
        assertNotNull(system.getRelease());
    }

    @Test(expected = IllegalArgumentException.class)
    public void testWithWrongTableNameShouldThrowException() {
        instance.getSerialNumber("WrongName");
    }

    @Test
    public void testGetSerialNumberReturnsPositiveNumber() throws Exception {
        long result = instance.getSerialNumber(tableName);
        assertTrue(result > 0);
    }

    @Test
    public void testGetSerialNumberReturnsBiggerNumbersOnFurtherCalls() {
        long firstResult = instance.getSerialNumber(tableName);
        long secondResult = instance.getSerialNumber(tableName);
        assertTrue(firstResult < secondResult);
    }
}

As I said, this is but a draft, and could be improved in many ways, e.g. the tests are quite coarse and don't consider all the small things that could go wrong, some of which I discovered with a quick debugging while I was setting up the tests. Calling a system.disconnectAllServices() when you finish would not be bad either. Yet, it is easily readable and hopefully understandable, so I hope this will help everyone to get rid of the extra stack required by the application server when it is not needed (does that ring a bell?).