Server-side unit testing from IDE

Lately I had to extract something from my old codes as a code example and I found an interesting unit testing solution, that I applied in one of my projects. We had some problems with units testing, because almost each thing for testing required fully initialized runtime application context.

There are some solutions for testing application in such way, like Spring mock testing classes, but for this application we had as many technologies integrated, that maintaining second mock context with the main context, which was under intensive development, resulted in writing of large amount of unnecessary code and constant problems. Second thing is that we had to test some flows involving more users, and we had to implement it somehow that we were able to switch logged user. In this situation the only working and acceptable solution was doing test cases in real runtime, what additionally ensured 100% compatibility with production environment.

OK, it is nice to have some well written test suites, but running them using a web interface was very inconvenient. In such instance you cannot use a nice JUnit IDE integration in Eclipse. You have to provide your own way to define test suites, and to select test to execute etc. The most satisfactory solution would be to have tests run in server instance, but to be able to control and execute them from IDE. Then I had some simple idea to achieve it, which worked very nice. Below I show the general concept, because my concrete implementation used various libs from that project.

The concept is very simple. We have some classes in server context, and when we have hot deployment enabled, we can change the code when server instance is running, and the changes apply immediately in working server. From the other hand, the same classes can be invoked and executed locally from IDE, as executable classes with main() method, or as JUnit tests. Would be nice to run them locally as JUnit tests, and they would be executed in running application server.

As a result I wanted to have something like this, to be run locally and on the server:

public class ServerTestExample extends ServerTestCase {

  @Test
  public void testSomething() {
    doTest("admin", "admin", new Runnable() {
      assertTrue("False is true?", false);
    });
  }

}

I wanted to have locally executable JUnit test, which could execute some test code in runtime server environment, as a chosen user. The code to be run remotely should be located and maintained with main local JUnit test, so the easiest way is to have it in anonymous Runnable class. The solution I'm sketching should:

1. (locally) log in the user to the system (if given user is not already logged in)
2. (locally) call the server service with request of invoking given class and method (local test instance shouldn't execute any code from Runnable)
3. (remotely) some server service should receive request (let's name it ServerTestService), instantiate test class and invoke test method (being in the server instance the Runnable code should be executed)
4. (remotely) service should catch any uncaught exception from remote test call, serialize it and pass it to local test call
5. (locally) in case of exception the local test instance should re-throw this exception to present it in local test execution stacktrace

The final code is easy to implement. I used org.apache.commons.httpclient.HttpClient for requesting server instance. To execute test we need working server instance somewhere. The base class outline for all server test cases could look as below:

/* Some response wrapper to have all response pieces together */
public class Response {
  private final InputStream body;
  private final Map headers;
  private final int status;

  public Response(int status, InputStream body, Map headers) {
    super();
    this.body = body;
    this.headers = headers;
    this.status = status;
  }

  public int getStatus() {
    return status;
  }

  public InputStream getBody() {
    return body;
  }

  public Map getHeaders() {
    return headers;
  }
}

/* Helper enum request method factory */
public enum HttpMethodType {
  GET {
    @Override
    public HttpMethod buildMethod() {
      return new GetMethod();
    }
  },

  POST {
    @Override
    public HttpMethod buildMethod() {
      return new PostMethod();
    }
  };

  public HttpMethod buildMethod() {
    return null;
  }
};

/* The main server case base class */
public class ServerTestCase {

  public static final String LOGIN_URL = "...";   // user login url
  public static final String LOGOUT_URL = "...";  // user logout url
  public static final String TEST_URL = "...";    // test case invocation url

  public static final String EXCEPTION_HEADER = "Exception"; // exception HTTP header

  protected HttpClient httpClient;    // client to make requests
  protected String loggedUser = "";   // currently logged user name
  protected boolean serverMode = false; // are we in server mode? by default we aren't

  public ServerTestCase() {
    httpClient = new HttpClient();
    // configure httpClient here: host configuration, cookies management, etc.
  }

  protected Response doRequest(HttpMethodType type, String url, NameValuePair... params) {
    HttpMethod method = type.buildMethod()
    method.setPath(url);
    method.setQueryString(params);
    try {
      int status = getHttpClient().executeMethod(method);
      HashMap headers = new HashMap();
      for (Header h : method.getResponseHeaders())
        headers.put(h.getName(), h.getValue());
      return new Response(status, method.getResponseBodyAsStream(), headers);
    } catch (Exception e) {
      throw new RuntimeException(e);
    }
  }

  public void doLogin(String user, String password) {
    if (isServerMode()) // someone calls it in server mode?
      return;

    if (!loggedUser.equals(user)) { // don't log in the same user again
      if (!("".equals(loggedUser)))
        doLogout();
      Response r = doRequest(HttpMethodType.POST, LOGIN_URL, 
        new NameValuePair("user", user),
        new NameValuePair("password", password)
      );
      if (/* check if user is logged in properly */) 
        loggedUser = user;
      else
        throw new RuntimeException(); // or something
    }
  }

  public void doLogout() {
    if (isServerMode()) // someone calls it in server mode?
      return;

    if (!("".equals(loggedUser))) { // some has to be logged
      Response r = doRequest(HttpMethodType.GET, LOGOUT_URL);
      if (/* check if user is logged out properly */) 
        loggedUser = "";
      else
        throw new RuntimeException(); // or something
    }
  }

  /* Main testing helper method */
  public void doTest(String user, String password, Runnable runnable) {
    if (isServerMode())
      runnable.run(); // in server mode just execute runnable
    else {
      // a trick to detect from stack trace from which method we are calling the test
      StackTraceElement[] elements = new Throwable().getStackTrace();
      assertTrue("Cannot find testcase to track", elements.length > 1);
      try {
        assertTrue("Last stacktrace element is not in ServerTestCase",
            ServerTestCase.class.isAssignableFrom(
              Class.forName(elements[1].getClassName())));
      } catch (ClassNotFoundException e) {
        throw new RuntimeException(e);
      }

      // ok, we know now the class name and method to call at the server side
      String testClass = this.getClass().getName();
      String testMethod = elements[1].getMethodName();

      // and we can perform request to server test service on working server
      doLogin(user, password);
      Response r = doRequest(HttpMethodType.GET, TEST_URL,
          new NameValuePair("testClass", testClass),
          new NameValuePair("testMethod", testMethod));

      // if we've had an exception, server test service will notify us by the header
      if (r.getHeaders().containsKey(EXCEPTION_HEADER)) {
        // if the header is present, it means that body contains serialized exception 
        // thrown in server mode
        Throwable thrown = null;
        try {
          ObjectInputStream in = new ObjectInputStream(r.getBody());
          thrown = (Throwable) in.readObject();
        } catch (Exception e) {
          throw new RuntimeException(e);
        }

        if (thrown instanceof Error) // if test fails, junit throws AssertionError
          throw (Error) thrown; 
        else { // otherwise something other could go wrong
          String failedInfo = String.format(
            "Failed execution of %s.%s in server mode (see JUnit trace)", 
            testClass, testMethod);
          throw new RuntimeException(failedInfo, thrown);
        }
      }
    }
  }

  public boolean isServerMode() {
    return serverMode;
  }

  public void setServerMode(boolean serverMode) {
    this.serverMode = serverMode;
  }

}

Now we have working test case base and we need to implements some test service listening calls on our TEST_URL. The test service should get test class name and method to call, instantiate it and invoke test method. When the exception is thrown, it should serialize it and pass to local service, additionally indicating exceptional situation by header. The service you can implement using any framework you need, so let's present only a simple template:

@BindURL(ServerTestCase.TEST_URL)
public class ServerTestSerive extends MyServiceBase {

  protected String testClass;
  protected String testMethod;

  public Resolution execute() {
    // I assume that framework has already performed the parameters binding
    Throwable thrown = null;
    ServerTestCase test = null;

    try {
      Class c = Class.forName(testClass);
      test = (ServerTestCase) c.newInstance();
      test.setServerMode(true); // now test will work in server mode
      test.getClass().getMethod(testMethod).invoke(test); // invoke test in server mode
    } catch (Exception e) {
      thrown = e;
    }

    // we can have exception and we need to forward it to local environment
    if (thrown != null) { 
      // prepare the exception stream
      if (thrown.getCause() != null) // Remove InvocationTargetException from stack trace
        thrown = thrown.getCause();
      ByteArrayOutputStream out = new ByteArrayOutputStream();
      try {
        ObjectOutputStream objout = new ObjectOutputStream(out);
        objout.writeObject(thrown);
      } catch (IOException e) {
        e.printStackTrace();
      }

      // pass everything back with appropriate header
      return new Resolution("application/java-serialized-object", 
        new ByteArrayInputStream(out.toByteArray())).
        addHeader(ServerTestCase.EXCEPTION_HEADER, "true");
    } else {
      // we return nothing in this example, when everything goes fine
      return new Resolution("text/plain", ""); 
    }
  }

  public String getTestClass() {
    return testClass;
  }

  public String getTestMethod() {
    return testMethod;
  }

  public void setTestClass(String testClass) {
    this.testClass = testClass;
  }

  public void setTestMethod(String testMethod) {
    this.testMethod = testMethod;
  }

}

I used here some Resolution concept as service result, to avoid this whole crap with handling with http request, response and streams. I hope it is clearly understood what I mean. Both local and remote classes makes up our little testing framework.

This solution was very good for us and we used it constantly in the project. You can control everything from IDE and you can use all JUnit goods locally. You can also easily automate tests on various deployment environments.

Have nice testing!