Monday, November 22, 2010

Using Eclipse with multi level Maven project structure

Yesterday I've been struggling with setting up Eclipse to work with some project having multi level Maven project structure. Eclipse uses its own flat project layout and isn't able to handle this properly. I've tried multiple approaches to achieve this project working, being built and debug in Eclipse. Finally I've found some workaround.

By multi level Maven project I mean following layout:

|- project
   |- pom.xml    <-- parent pom.xml
   |- project1   <-- subproject 1
      |- src
      |- pom.xml <-- subproject pom.xml using parent pom.xml
   |- project2   <-- subproject 2
      |- src
      |- pom.xml <-- subproject pom.xml using parent pom.xml

In such layout we cannot simply import subprojects into workspace separately and use maven 2 eclipse plugin, because they use upper level pom.xml. On the other hand - we cannot have subproject in Eclipse project subfolder. It seems to be no easy way to have it working.

There are also some other problems, for example I wanted to have everything in single subversion node to be transparently checked in and updated. My subprojects used some resources and other code from main project, and from each other. The additional fun was the requirement to have all projects run in single Tomcat instance working in IDE in debug mode. But after longer time I've found a soultion.

To do the job you need M2eclipse only. First start with checking out the repo into a single project. After this, you have the project in workspace with no sources, not even Java project. And this won't be your really used project, but will be useful to operate on svn.

Now you have to download somewhere appropriate maven version and setup in Window | Preferences | Maven | User Settings path to maven conf/settings.xml file. I don't know if it is really necessary because I haven't played without it.

We will use standard Dynamic Web Project with linked sources from our main project to configure whole env in following steps:
  1. Create Dynamic Web Project.
  2. Uncheck Use default location and select project1 dir from workspace in field below. Here I've had some NullPointerException with setting Dynamic Web Module facet on this project. If you have that too, after creation you need to enable Dynamic Web Module facet manually in project properties, Project Facets section. After these steps you should have Dynamic Web Project with Maven layout, linked to project1 dir. You probably have there single source folder.
  3. Click on project and select Maven | Enable dependency management from context menu. Maven should correct source paths and configure libs - you should have now few source dirs there. Now the Maven is enabled and after cleaning and building project you should have the project properly built. You can experience only some "missing artifact" problems with pom, but they're harmless. If your project is not built properly, check Java Build Path entries in project properties and fix the problems. Previously I run command line maven compilation to download all necessary libraries into repository, so if Eclipse can't found libs maybe you have to do it.
  4. When you can build project properly, the last thing is to setup server deployment policy. You can do it in Deployment Assembly section of project properties. On the beginning it's completely invalid and configured to work with standard eclipse Dynamic Web Project. Remove everything there, and add:
    • /src/config/YOURCONFIG deployment to /WEB-INF/classes (using Add | Folder)
    • /src/main/java -> /WEB-INF/classes
    • /src/main/resources -> /WEB-INF/classes
    • /src/main/webapp -> /
    • Maven Dependencies -> /WEB-INF/lib (using Add | Java build path entries)
  5. You can now remove unnecessary build and WebContent directories from your project.
Now, add your project as Server Runtime Module and try to rebuild everything, publish and run in the server. You should now be able to work with code from eclipse with the maven layout. You can now create other projects in workspace, linked with different subprojects in your super-project, and configure them as web modules too, to be working together. The last problem I encountered has been a fact, that Eclipse deploys all Maven Dependencies into /WEB-INF/lib directory, event those marked for "provided" scope. This can results with problems, when these libs conflict with those from server runtime. These problems can appear in various ways (I've had some JSP exception, since the jsp-api.jar has been overridden). Unfortunately there is no simple solution neither in Eclipse nor M2eclipse to exclude these files (at least I haven't been able to find something working). I've overcome this problem with another workaround - I've built the project manually using package maven goal, and the I joined libs from generated webapp to Deployment Assembly in project properties, instead of whole Maven Dependencies.

Friday, November 19, 2010

Server-side unit testing from IDE

Lately I had to extract something from my old codes as a code example and I found an interesting unit testing solution, that I applied in one of my projects. We had some problems with units testing, because almost each thing for testing required fully initialized runtime application context.

There are some solutions for testing application in such way, like Spring mock testing classes, but for this application we had as many technologies integrated, that maintaining second mock context with the main context, which was under intensive development, resulted in writing of large amount of unnecessary code and constant problems. Second thing is that we had to test some flows involving more users, and we had to implement it somehow that we were able to switch logged user. In this situation the only working and acceptable solution was doing test cases in real runtime, what additionally ensured 100% compatibility with production environment.

OK, it is nice to have some well written test suites, but running them using a web interface was very inconvenient. In such instance you cannot use a nice JUnit IDE integration in Eclipse. You have to provide your own way to define test suites, and to select test to execute etc. The most satisfactory solution would be to have tests run in server instance, but to be able to control and execute them from IDE. Then I had some simple idea to achieve it, which worked very nice. Below I show the general concept, because my concrete implementation used various libs from that project.

The concept is very simple. We have some classes in server context, and when we have hot deployment enabled, we can change the code when server instance is running, and the changes apply immediately in working server. From the other hand, the same classes can be invoked and executed locally from IDE, as executable classes with main() method, or as JUnit tests. Would be nice to run them locally as JUnit tests, and they would be executed in running application server.

As a result I wanted to have something like this, to be run locally and on the server:

public class ServerTestExample extends ServerTestCase {

  @Test
  public void testSomething() {
    doTest("admin", "admin", new Runnable() {
      assertTrue("False is true?", false);
    });
  }

}

I wanted to have locally executable JUnit test, which could execute some test code in runtime server environment, as a chosen user. The code to be run remotely should be located and maintained with main local JUnit test, so the easiest way is to have it in anonymous Runnable class. The solution I'm sketching should:
  1. (locally) log in the user to the system (if given user is not already logged in)
  2. (locally) call the server service with request of invoking given class and method (local test instance shouldn't execute any code from Runnable)
  3. (remotely) some server service should receive request (let's name it ServerTestService), instantiate test class and invoke test method (being in the server instance the Runnable code should be executed)
  4. (remotely) service should catch any uncaught exception from remote test call, serialize it and pass it to local test call
  5. (locally) in case of exception the local test instance should re-throw this exception to present it in local test execution stacktrace
The final code is easy to implement. I used org.apache.commons.httpclient.HttpClient for requesting server instance. To execute test we need working server instance somewhere. The base class outline for all server test cases could look as below:

/* Some response wrapper to have all response pieces together */
public class Response {
  private final InputStream body;
  private final Map headers;
  private final int status;

  public Response(int status, InputStream body, Map headers) {
    super();
    this.body = body;
    this.headers = headers;
    this.status = status;
  }

  public int getStatus() {
    return status;
  }

  public InputStream getBody() {
    return body;
  }

  public Map getHeaders() {
    return headers;
  }
}

/* Helper enum request method factory */
public enum HttpMethodType {
  GET {
    @Override
    public HttpMethod buildMethod() {
      return new GetMethod();
    }
  },

  POST {
    @Override
    public HttpMethod buildMethod() {
      return new PostMethod();
    }
  };

  public HttpMethod buildMethod() {
    return null;
  }
};

/* The main server case base class */
public class ServerTestCase {

  public static final String LOGIN_URL = "...";   // user login url
  public static final String LOGOUT_URL = "...";  // user logout url
  public static final String TEST_URL = "...";    // test case invocation url

  public static final String EXCEPTION_HEADER = "Exception"; // exception HTTP header

  protected HttpClient httpClient;    // client to make requests
  protected String loggedUser = "";   // currently logged user name
  protected boolean serverMode = false; // are we in server mode? by default we aren't

  public ServerTestCase() {
    httpClient = new HttpClient();
    // configure httpClient here: host configuration, cookies management, etc.
  }

  protected Response doRequest(HttpMethodType type, String url, NameValuePair... params) {
    HttpMethod method = type.buildMethod()
    method.setPath(url);
    method.setQueryString(params);
    try {
      int status = getHttpClient().executeMethod(method);
      HashMap headers = new HashMap();
      for (Header h : method.getResponseHeaders())
        headers.put(h.getName(), h.getValue());
      return new Response(status, method.getResponseBodyAsStream(), headers);
    } catch (Exception e) {
      throw new RuntimeException(e);
    }
  }

  public void doLogin(String user, String password) {
    if (isServerMode()) // someone calls it in server mode?
      return;

    if (!loggedUser.equals(user)) { // don't log in the same user again
      if (!("".equals(loggedUser)))
        doLogout();
      Response r = doRequest(HttpMethodType.POST, LOGIN_URL, 
        new NameValuePair("user", user),
        new NameValuePair("password", password)
      );
      if (/* check if user is logged in properly */) 
        loggedUser = user;
      else
        throw new RuntimeException(); // or something
    }
  }

  public void doLogout() {
    if (isServerMode()) // someone calls it in server mode?
      return;

    if (!("".equals(loggedUser))) { // some has to be logged
      Response r = doRequest(HttpMethodType.GET, LOGOUT_URL);
      if (/* check if user is logged out properly */) 
        loggedUser = "";
      else
        throw new RuntimeException(); // or something
    }
  }

  /* Main testing helper method */
  public void doTest(String user, String password, Runnable runnable) {
    if (isServerMode())
      runnable.run(); // in server mode just execute runnable
    else {
      // a trick to detect from stack trace from which method we are calling the test
      StackTraceElement[] elements = new Throwable().getStackTrace();
      assertTrue("Cannot find testcase to track", elements.length > 1);
      try {
        assertTrue("Last stacktrace element is not in ServerTestCase",
            ServerTestCase.class.isAssignableFrom(
              Class.forName(elements[1].getClassName())));
      } catch (ClassNotFoundException e) {
        throw new RuntimeException(e);
      }

      // ok, we know now the class name and method to call at the server side
      String testClass = this.getClass().getName();
      String testMethod = elements[1].getMethodName();

      // and we can perform request to server test service on working server
      doLogin(user, password);
      Response r = doRequest(HttpMethodType.GET, TEST_URL,
          new NameValuePair("testClass", testClass),
          new NameValuePair("testMethod", testMethod));

      // if we've had an exception, server test service will notify us by the header
      if (r.getHeaders().containsKey(EXCEPTION_HEADER)) {
        // if the header is present, it means that body contains serialized exception 
        // thrown in server mode
        Throwable thrown = null;
        try {
          ObjectInputStream in = new ObjectInputStream(r.getBody());
          thrown = (Throwable) in.readObject();
        } catch (Exception e) {
          throw new RuntimeException(e);
        }

        if (thrown instanceof Error) // if test fails, junit throws AssertionError
          throw (Error) thrown; 
        else { // otherwise something other could go wrong
          String failedInfo = String.format(
            "Failed execution of %s.%s in server mode (see JUnit trace)", 
            testClass, testMethod);
          throw new RuntimeException(failedInfo, thrown);
        }
      }
    }
  }

  public boolean isServerMode() {
    return serverMode;
  }

  public void setServerMode(boolean serverMode) {
    this.serverMode = serverMode;
  }

}

Now we have working test case base and we need to implements some test service listening calls on our TEST_URL. The test service should get test class name and method to call, instantiate it and invoke test method. When the exception is thrown, it should serialize it and pass to local service, additionally indicating exceptional situation by header. The service you can implement using any framework you need, so let's present only a simple template:

@BindURL(ServerTestCase.TEST_URL)
public class ServerTestSerive extends MyServiceBase {

  protected String testClass;
  protected String testMethod;

  public Resolution execute() {
    // I assume that framework has already performed the parameters binding
    Throwable thrown = null;
    ServerTestCase test = null;

    try {
      Class c = Class.forName(testClass);
      test = (ServerTestCase) c.newInstance();
      test.setServerMode(true); // now test will work in server mode
      test.getClass().getMethod(testMethod).invoke(test); // invoke test in server mode
    } catch (Exception e) {
      thrown = e;
    }

    // we can have exception and we need to forward it to local environment
    if (thrown != null) { 
      // prepare the exception stream
      if (thrown.getCause() != null) // Remove InvocationTargetException from stack trace
        thrown = thrown.getCause();
      ByteArrayOutputStream out = new ByteArrayOutputStream();
      try {
        ObjectOutputStream objout = new ObjectOutputStream(out);
        objout.writeObject(thrown);
      } catch (IOException e) {
        e.printStackTrace();
      }

      // pass everything back with appropriate header
      return new Resolution("application/java-serialized-object", 
        new ByteArrayInputStream(out.toByteArray())).
        addHeader(ServerTestCase.EXCEPTION_HEADER, "true");
    } else {
      // we return nothing in this example, when everything goes fine
      return new Resolution("text/plain", ""); 
    }
  }

  public String getTestClass() {
    return testClass;
  }

  public String getTestMethod() {
    return testMethod;
  }

  public void setTestClass(String testClass) {
    this.testClass = testClass;
  }

  public void setTestMethod(String testMethod) {
    this.testMethod = testMethod;
  }

}

I used here some Resolution concept as service result, to avoid this whole crap with handling with http request, response and streams. I hope it is clearly understood what I mean. Both local and remote classes makes up our little testing framework.

This solution was very good for us and we used it constantly in the project. You can control everything from IDE and you can use all JUnit goods locally. You can also easily automate tests on various deployment environments.

Have nice testing!