Sunday, July 27, 2014

Tomcat, Atmosphere and Spring Security

Here I'd like to describe another interesting case I've been struggling with for recent few days. This involves the following use case: enable asynchronous events support for Tomcat/Spring multi-tenancy SaaS application, that can be pushed to listening client groups. To be specific, the event should be channeled to following groups: to specific user, to all users of specific tenant and to all users.

Atmosphere + Tomcat

The fancy new technology for async processing in Java world is Atmosphere, and I use it in this example. Unfortunately I started with horribly preconfigured Atmosphere which apparently locked Tomcat after making some number requests, and it wasn't able to serve more requests. It turned out that the working Tomcat + Atmosphere config is something not so obvious, so let's quickly describe all these problems to move on.

I started with the following maven dependency:

<dependency>
<groupId>org.atmosphere</groupId>
<artifactId>atmosphere-runtime</artifactId>
<version>2.1.7</version>
</dependency>

After a lot of struggling I came to conclusion that there's no way to properly run Tomcat with Atmosphere using this library (at least in 2.1.7 version). I started with standard Atmosphere configuration, which uses native Tomcat async implementation (Comet support). In this scenario there's a bug in Atmosphere which results in using Tomcat BIO support (blocking IO) instead of NIO (non-blocking IO). Finally, you have a thread created for each async request, which is then suspended and moved to waiting pool. When you reach the tomcat thread pool capacity (default is 200) you end up with completely frozen application.

Afterward I changed the implementation from native Tomcat async support to Servlet 3 specification, using following flags:

<init-param>
    <param-name>org.atmosphere.useNative</param-name>
    <param-value>false</param-value>
</init-param>
<init-param>
    <param-name>org.atmosphere.useWebSocketAndServlet3</param-name>
    <param-value>true</param-value>
</init-param>

Using this config, it started to work through Tomcat NIO, but the odd things started to happen as well. For example random freezes on standard request processing, and a lot of java.lang.IllegalStateException: Cannot forward after response has been committed exceptions. Something similar to this guy situation.

After a lot of debugging what is really happening in the Atmosphere and Tomcat threads I gave up and I found the solution with so called "native" atmosphere implementation, what apparently is the same lib with only one class changed with fixed native Tomcat support for Atmosphere (scenario 1), which is called:

<dependency>
<groupId>org.atmosphere</groupId>
<artifactId>atmosphere-runtime-native</artifactId>
<version>2.1.7</version>
</dependency>

And is describe here. It finally works well using Tomcat Comet support and/or native Tomcat websockets support. Additionally it requires /META-INF/context.xml with following content:

<Context>
<Loader delegate="true"/>
</Context>

Atmosphere + Spring

Now something which is simple and can be found in many examples on the net. How to configure Atmosphere so that it can route requests to Spring DispatcherServlet.  To skip unnecessary words, I'll make it quick:


Things that might be explained a little more are following:
  1. org.atmosphere.useNativeorg.atmosphere.useWebSocketAndServlet3 make it finally clear that we want to go using Tomcat native async support.
  2. org.atmosphere.cpr.broadcaster.maxProcessingThreads - this is the limitation to 10 for Atmosphere threads. Atmosphere spawns some threads sweeping suspended requests (eg. by flushing their response buffers).
  3. org.atmosphere.cpr.broadcasterLifeCyclePolicy=EMPTY_DESTROY is the lifecycle policy for Atmosphere Broadcaster objects. Usually Broadcaster has assigned some AtmosphereResource-s, representing opened async connections. When all connections for particular Broadcaster are closed, the Broadcaster object may still be held in memory and reused. For SaaS application, that may handle hundreds of tenants and thousands of users concurrently I consider it a bad pattern. EMPTY_DESTROY tells Atmosphere to relase all Broadcaster objects if they don't have assigned any resources, and remove them from memory.
  4. org.atmosphere.cpr.AtmosphereInterceptor is the important one here, because after Atmosphere invokes broadcasting operation, the response buffers are flushed periodically with all data written, so they could contain more than a single message at one flush operation. In such instance your client would receive two or more messages in one event listener notification, what is usually unwanted. This can be overcome by using TrackMessageSizeInterceptor on the server side, and trackMessageLength parameter in Atmosphere client.
  5. AtmosphereSpringControllerResolver enables direct AtmosphereResource injection to Spring controller.

Atmosphere + Spring Security

Now what we'd like to have is the Spring Security context injected to Atmosphere requests, in order to extract user from the SecurityContextHolder and to apply broadcasting operations on suspended requests. The answer on the question how to do it is simple: you can't.

There are two problems I came across with this subject. First the Spring Security filters aren't applied to MeteorServlet, because it's not a reguler servlet, but CometProcessor, supporting async requests. For such type of servlets only CometFilter can be applied, not a reguler Filter, which is implemented by Spring Security DelegatingFilterProxy. You can overcome this problem, though, by either wrapping the Spring Security filters with your own CometFilter-s, or by overriding the default FilterChain by your own implementation. Anyway, it doesn't work as well.

This is because the SecurityContextHolder default storage strategy is ThreadLocalSecurityContextHolderStrategy, which holds the SecurityContext in ThreadLocal (this is the only production implementation and one cannot imagine different working strategy for this problem). It works well for standard requests, processed in separate threads, but for suspended Atmosphere requests there's a problem. When the resources are swept and buffers are flushed, all this process happens in internal Atmosphere thread pool, and one thread supports many AtmosphereResource-s in single execution, so the SecurityContext can't be bound to the thread, because you end up with an exception, or much worse, with different user authorized than it should be.

So what I do, and I'll show in the further example, is how to extract user directly for HTTP session to be used with AtmosphereResource to create appropriate broadcasters.

There's another remark about this overall architecture. If you can run DispatcherServlet through Atmosphere, you might tend to run your whole application through MeteorServlet wrapper. But, when you consider above facts, that you can't apply normal filters to this servlet, and moreover you can't apply security filters on it, the conclusion is simple: just don't do it. Define your "async" Atmosphere servlet context separately from another regular "sync" DispatcherServlet, and everything will be fine.

Broadcast events to user groups

Before final implementation, we need to understand how Atmosphere and Atmosphere Broadcaster-s work internally. 

In regular request processing, when the request comes, Tomcat takes the free thread from the thread pool (or queues the job in the thread pool queue, if all threads from pool are busy), and delegates the request processing to this thread. The Spring Security filters extract the user from HTTP session and put him to the SecurityContext held in current request thread ThreadLocal. Then the work is delegated to your servlet, response buffers are filled, everything is cleaned out and thread is released back to the pool. The response buffer is then written to the client.

In async request processing Atmosphere waits for incoming requests with its own thread pool. When the request comes, one of these threads receives it (or, like in above situation, the job is queued waiting to release at least one thread from the pool), suspends it, and returns thread to the pool. The suspension figures on releasing the processing thread, while the TCP connection is still opened. All these suspended requests are stored in an internal storage, and can be accessed in any moment in application. For all of them the TCP connections are opened, and one can write to the opened Response object to send async events to the client.

But, how to tell apart one suspended request from another? For our example - how to find all requests sent from all logged users of specific tenant? For such use cases Atmosphere introduces the Broadcaster concept. With a single suspended request (AtmosphereResource) you can associate one or more broadcasters, and use these broadcasters to send events to choosen clients. With each AtmosphereResource there's a single Broadcaster created with random UUID. 

Using this idea and knowing this UUID you may send the async event to each suspended request separately, by choosing appropriate broadcaster. Another Atmosphere concept is MetaBroadcaster. It can be used to send event using all broadcasters fitting to the expression. For example:
  1. User A connects to async service, the broadcaster with ID="/UUID-1" is created.
  2. User B connects to async service, the broadcaster with ID="/UUID-2" is created.
  3. Using MetaBroadcaster you may send data to either first or second user by broadcastTo("/UUID-1", event) or broadcastTo("/UUID-2", event).
  4. Or you can send event to all users by broadcastTo("/*", event).
This well concept can be adapted to our use case. Let's assume we have a TENANT_ID and USER_ID, defining our tenant and its user. We need to assign only one broadcaster to each async request to achieve our goals:
  1. User connects to async service, the broadcaster with ID="/TENANT_ID/USER_ID" is created.
  2. To send event to this particular user, use broadcastTo("/TENANT_ID/USER_ID", event).
  3. To send event to all logged users of specific tenant, use broadcastTo("/TENANT_ID/*", event).
  4. To send event to all logged users, use broadcastTo("/*", event).
And here comes the implementation with all described above:


Finally, in AsyncDispatcher controller we just need to suspend request using AsyncService.suspend() method, to make it all working together.

Friday, May 9, 2014

Global state in AngularJS controllers

Suprisingly for me the controller state in AngularJS is not preserved between the controller invocations. I at least expected an option to switch it on and off on demand. For the classic application it was difficult to achieve that we may restore the state for a given view (eg. to be on the same page as we were leaving the view), what sounds great for me from the application usability point of view.

But, it's right there, so it can be implemented. However, I don't like using the $rootScope for this, what can be found in many examples on the net, in the same way I don't use globals, because they produce a mess. Here I'd like to propose an elegant solution for this.

I have the service that holds all controller states, and provide the initialization function that may be used when the state hasn't been cached yet. The code snippet:

Thursday, October 24, 2013

Java Cryptography Architecture and common encryption usages

JCA can be really tricky to perform simple, common tasks. After latest usage of java cryptography I'd like to present below the simplest usage of cryptography methods, involving mainly symmetric (AES) and asymmetric (RSA/DSA) encryption plus some helper methods.

I don't use specific JCA provider, but let it choose appropriate one. If you want to select specific provider, you must provide additional parameters, usually to getInstance() methods of various algorithm elements. In examples I use RSA1024 (you might change it to DSA) and AES256.

Note about AES256 - to enable this for your VM you need Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files. With default policy you can only use AES128.

The amount of "things can go wrong" is pretty big, so if you write your own cryptography util, there's a good thing to wrap JCA exception in something that can be easy caught in the user code.

public class EncryptionException extends Exception {
 
 public EncryptionException() {
 }
 
 public EncryptionException(String message) {
  super(message);
 }
 
 public EncryptionException(String message, Throwable cause) {
  super(message, cause);
 }
 
 public EncryptionException(Throwable cause) {
  super(cause);
 }
 
}

Firstly a simple thing, BASE64 encoding wrappers (to be able to switch implementation, what of course will be never used):

public static String encodeBase64(byte[] data) {
 return new BASE64Encoder().encode(data);
}
 
public static byte[] decodeBase64(String data) throws EncryptionException {
 try {
  return new BASE64Decoder().decodeBuffer(data);
 } catch (IOException e) {
  throw new EncryptionException("Error decoding base64", e);
 }
}

How to generate RSA 1024 keys with SecureRandom:

/**
 * Generates new keypair
 */
public static KeyPair generateKeys() throws EncryptionException {
 try {
  KeyPairGenerator keyGen = KeyPairGenerator.getInstance("RSA");
  SecureRandom random = SecureRandom.getInstance("SHA1PRNG");
  keyGen.initialize(1024, random);
  return keyGen.generateKeyPair();
 } catch (Exception e) {
  throw new EncryptionException("Error generating keypair", e);
 }
}

How to encrypt with asymmetric key (public or private):

/**
 * Encrypts data with key
 */
public static byte[] encryptAsymmetric(Key key, byte[] data)
throws EncryptionException {
 try {
  Cipher cipher = Cipher.getInstance("RSA");
  cipher.init(Cipher.ENCRYPT_MODE, key);
  return cipher.doFinal(data);
 } catch (Exception e) {
  throw new EncryptionException("Error key encrypting", e);
 }
}

For above, how to decrypt with asymmetric key (public or private, the different one than used for encryption):

/**
 * Decrypts data with key
 */
public static byte[] decryptAsymmetric(Key key, byte[] data)
throws EncryptionException {
 try {
  Cipher cipher = Cipher.getInstance("RSA");
  cipher.init(Cipher.DECRYPT_MODE, key);
  return cipher.doFinal(data);
 } catch (Exception e) {
  throw new EncryptionException("Error key decrypting", e);
 }
}

How to build symmetric key for AES256 encryption:

/**
 * Builds a random secret key for symmetric algorithm
 */
public static Key buildSymmetricKey() throws EncryptionException {
 try {
  KeyGenerator keyGen = KeyGenerator.getInstance("AES");
  keyGen.init(256, SecureRandom.getInstance("SHA1PRNG"));
  return keyGen.generateKey();
 } catch (Exception e) {
  throw new EncryptionException("Error generating secret key", e);
 }
}

The building of the AES key based on user password is a little tricky. You need to have a salt (N-bytes array) that is used to generate a proper AES key (with a proper length). If you need to recover this key in the future, using just the same user password, you need to use exactly the same salt, so it's probably best to hardcode it somewhere and use for further keys generation (random 8 bytes):

private static byte[] SALT = new byte[]{
 (byte) 0xa1, (byte) 0x22, (byte) 0x33, (byte) 0xa4,
 (byte) 0x11, (byte) 0x22, (byte) 0x12, (byte) 0x22};
 
/**
 * Builds a secret key for symmetric algorithm recoverable by password
 */
public static Key buildSymmetricKey(String password)
throws EncryptionException {
 try {
  SecretKeyFactory factory =
   SecretKeyFactory.getInstance("PBKDF2WithHmacSHA1");
  KeySpec spec = new PBEKeySpec(password.toCharArray(), SALT, 256,
   256);
  SecretKey tmp = factory.generateSecret(spec);
  return new SecretKeySpec(tmp.getEncoded(), "AES");
 } catch (Exception e) {
  throw new EncryptionException("Error encoding secret key", e);
 }
}

Now, how to encrypt the arbitrary length data block with AES:

/**
 * Encrypts data with symmetric algorithm and password
 */
public static byte[] encryptSymmetric(Key key, byte[] data)
throws EncryptionException {
 try {
  Cipher cipher = Cipher.getInstance("AES/ECB/PKCS5Padding");
  cipher.init(Cipher.ENCRYPT_MODE, key);
  return cipher.doFinal(data);
 } catch (Exception e) {
  throw new EncryptionException("Error symmetric encrypting", e);
 }
}

Note, that I use ECB as block cipher mode of operation. This is not the safest one, better would be CBC (refer wikipedia). But if you need to recover your data only by AES key, you can't do this. To recover from CBC you need to store your CBC initialization vector together with the password. So, I use ECB for this simple example, to make all it working only with keys.

The decryption for above symmetric encryption is similar:

/**
 * Decrypts data with symmetric algorithm and password
 */
public static byte[] decryptSymmetric(Key key, byte[] data)
throws EncryptionException {
 try {
  Cipher cipher = Cipher.getInstance("AES/ECB/PKCS5Padding");
  cipher.init(Cipher.DECRYPT_MODE, key);
  return cipher.doFinal(data);
 } catch (Exception e) {
  throw new EncryptionException("Error symmetric descrypting", e);
 }
}

This is all about working encryption/decryption. Now about storing the keys in db. If you'd like to use BASE64 encoding or even to hold everything in BLOB-s, methods below will be useful.

Converting the key to BASE64 is easy:

/**
 * Converts key to base64 encoded string
 */
public static String keyToString(Key key) {
 return encodeBase64(key.getEncoded());
}

But recovering the key from BASE64 or byte[] is another tricky part:

/**
 * Converts base64 encoded string to assymetric key
 *
 * @param publicKey if true returns public key, private key otherwise
 */
public static Key asymmetricKeyFromString(String s, boolean publicKey)
throws EncryptionException {
 return asymmetricKeyFromBytes(decodeBase64(s), publicKey);
}
 
/**
 * Converts bytes to assymetric key
 *
 * @param publicKey if true returns public key, private key otherwise
 */
public static Key asymmetricKeyFromBytes(byte[] bytes, boolean publicKey)
throws EncryptionException {
 try {
  if (publicKey) {
   return KeyFactory.getInstance("RSA").generatePublic(
    new X509EncodedKeySpec(bytes));
  } else {
   return KeyFactory.getInstance("RSA").generatePrivate(
    new PKCS8EncodedKeySpec(bytes));
  }
 } catch (Exception e) {
  throw new EncryptionException("Can't decode assymetric key", e);
 }
}

The same for symmetric key looks much easier:

/**
 * Converts base64 encoded string to symmetric key
 */
public static Key symmetricKeyFromString(String s) throws
EncryptionException {
 return symmetricKeyFromBytes(decodeBase64(s));
}
 
/**
 * Converts bytes to symmetric key
 */
public static Key symmetricKeyFromBytes(byte[] bytes)
throws EncryptionException {
 return new SecretKeySpec(bytes, "AES");
}

Sometimes you also need to convert your private/public keys to PEM format for exporting. Without bouncycastle you need your own method for this:

public static String getPem(Key key) {
 StringBuilder sb = new StringBuilder();
 if (key instanceof PrivateKey || key instanceof PublicKey)
  sb.append(String.format("-----BEGIN %s %s KEY-----\n", "RSA",
   key instanceof PublicKey ? "PUBLIC" : "PRIVATE"));
 else
  sb.append("-----BEGIN KEY-----");
 sb.append(encodeBase64(key.getEncoded()));
 if (key instanceof PrivateKey || key instanceof PublicKey)
  sb.append(String.format("\n-----END %s %s KEY-----", "RSA",
   key instanceof PublicKey ? "PUBLIC" : "PRIVATE"));
 else
  sb.append("\n-----END KEY-----");
 return sb.toString();
}

And this was just last example of simple Java cryptography API.