Thursday, February 28, 2013

Lucene permissions and Hibernate Search / Compass

In our application we have some permissions system, restricting access to various entities for particular users. The permissions system consists of:
  1. Users - entity contains a list of users that may access it.
  2. Groups - entity contains a list of groups that may access it. Each group contains lists of users belonging to the group and can have subgroups.
  3. Roles - entity contains a list of roles that may access it. Each roles contains list of users having a role, and may have parent roles.
To make it simple and efficient, the User, Group and Role shares one Actor table and have unique ID. The base Actor entity represents Hibernate relation to permitted actors:

public class SecuredEntity {
  public Set<Actor> getAllPermittedActors() {
    return ...;
  }
}

How the permissions filtering works internally for DB queries is not a subject of this article, but one can imagine it easily. Additionally we calculate all Id-s for given Actor, and use them in filters:

public class Actor {
  public Collection<Long> getAllActorIds() {
    return ...;
  }
}

Now, all important entities are indexed by Lucene using Hibernate Search. It is crucial from security point of view to not to return in search result objects, that the current actor can't access, but Lucene doesn't provide any security mechanism. Hovewer it can be done easily in following way.

First we introduce indexed property, containing all permitted Id-s for given object. Let's define the appropriate interface, returning Id-s for Lucene:

public interface ISearchPerms {

  public static final String FIELD_SEARCHPERMS = "searchPerms";

  public static class Helper {

    public static Set<Long> getDefaultSearchPerms() {
      Set<Long> set = new HashSet<Long>();
      set.add(0l);
      return set;
    }

    public static Set<Long> getSearchPermsForActors(Collection<Actor> actors) {
      Set<Long> set = new HashSet<Long>();
      for (Actor a: actors)
        set.add(a.getId());
      return set;
    }

  }

  Set<Long> getSearchPerms();

}
If you have different Id-s in your Actor classes, you may return strings representing Users, Groups and Roles: eg. "U1", "U2", "G1", "G2", "R1", "R2".

Now this interface needs to be implemented in your @Indexed entities, with @Field definition for Hibernate Search (or @SearchableProperty for Compass).

@Indexed
public class SecuredEntity implements ISearchPerms {

  @Override
  @Field(store = Store.YES, name = ISearchPerms.FIELD_SEARCHPERMS, 
    index = Index.UN_TOKENIZED)
  @FieldBridge(impl = LongCollectionFieldBridge.class)
  public Set<Long> getSearchPerms() {
    return ISearchPerms.Helper.getSearchPermsForActors(getAllPermittedActors());
  }

}

For non-secured objects we need the same, but we use common "non-secured-id", ie. 0, what means that the object is not secured, but should be always returned in the search results for all:

@Indexed
public class NonSecuredEntity implements ISearchPerms {

  @Override
  @Field(store = Store.YES, name = ISearchPerms.FIELD_SEARCHPERMS, 
    index = Index.UN_TOKENIZED)
  @FieldBridge(impl = LongCollectionFieldBridge.class)
  public Set<Long> getSearchPerms() {
    return ISearchPerms.Helper.getDefaultSearchPerms();
  }

}

As you can see, we need some field bridge (LongCollectionFieldBridge) to implement. This is due to Hibernate Search can't map the Set<Long> using default bridges (while Compass has it out-of-the-box; it's unsupported for more than two years from now, though). We map particular Long from collection to the separate lucene field, with the same name:

public class LongCollectionFieldBridge implements FieldBridge, StringBridge {
        
  @Override
  public void set(String name, Object value, Document document, 
      LuceneOptions luceneOptions) {
    Collection<Long> collection = (Collection<Long>) value;
    if (collection == null && luceneOptions.indexNullAs() != null)
      luceneOptions.addFieldToDocument(name, luceneOptions.indexNullAs(), 
        document);
    else if (collection != null)
      for (Long l: collection)
        luceneOptions.addFieldToDocument(name, l+"", document);
  }

  @Override
  public String objectToString(Object object) {
    if (object instanceof Long)
      return ((Long) object).toString();
    return null;
  }

}

And this is how we have all required to perform Lucene query with permissions filtering (below the example using Hibernate Search QueryBuilder):

// query builder
QueryBuilder qb = fullTextSession.getSearchFactory().buildQueryBuilder().
  forEntity(SecuredEntity.class).get();
BooleanJunction bool = qb.bool();

// add query
bool = bool.must(qb.keyword().onField(SomeMyField).
  matching("my keywords to search").createQuery());

// filter perms
BooleanJunction permBool = qb.bool().should( // default perms = no perms, always
  qb.keyword().onField(ISearchPerms.FIELD_SEARCHPERMS).matching(0l).createQuery());
for (Long id: currentUser.getAllActorIds()) // current user perms
  permBool = permBool.should(
    qb.keyword().onField(ISearchPerms.FIELD_SEARCHPERMS).matching(id).
      createQuery());
bool = bool.must(permBool.createQuery());

return bool.createQuery();

Tuesday, September 11, 2012

SonOfObsidian color scheme mod for IDEA 11

After latest Eclipse Juno installation, and few weeks of working in this, experiencing big performance downgrade and seeing the direction of current product development (like new fancy UI with animations) I realized that this great product has become an overloaded crap (with installation file growth from 90 MB for Helios to 220 MB for Juno - what can acknowledge this), and after almost 10 years of my Eclipse adventure I decided to look for another IDE. I recently tried to test Intellij IDEA 11 (commercial), because I've bought already their other product PhpStorm (really great IDE for PHP).

Few remarks about this IDE after a week of work:
  • very fast
  • great refactoring / code browsing / autocomplete capabilites
  • highly configurable from A to Z
  • it has everything I use besides java (html/css, groovy, javascript, properties editor, svn/git integration, tomcat, hibernate/spring suport, trac integration etc.)
And some worse things:
  • very hard to switch from Eclipse because of completely different keymap - but I accomplished fair comfort of work after this week
  • you need to spend at least a day to configure it for yourself, mainly because it has a big amount of code inspection hints displayed in editor, what makes it looking as the christmas tree (if everything is shown as w warning/hint, you cannot see anything worthful there)
  • I'd say that product is still wanting in some details, and reported few bugs/feature requests to them (but they are responsible)
But generally I find it a very useful and great IDE.

Unfortunately this IDE doesn't come with color schemes for editors, and only the default color scheme is available. I don't like to burn out my eyes with bright white and prefer some dark schemes. I've found one appropriate on the net - SonOfObsidian - translated from VisualStudio by some guy Roger (thanks, Roger).

Unfortunately after loading it into IDE, it turned out that only the Java editor is done well (in my opinion). Multiple editors have dark font on dark background. On the other hand other editors have a big amount of markers with light background, what completely kills the concept of "dark scheme". During this week of work I was trying to fix the most annoying parts, and make some standards between editors (eg. to have keywords, strings, number in similar colors in various editors), and I changed the font size as well to not to exhaust my eyes so much. Here are some examples of this work (before - after):

















Here are the sources for those who share my preferences. They need to be put into ~/.IntelliJIdea11/config/colors.

[UPDATE] Here is the update for Idea 13. 

Wednesday, August 22, 2012

javax.mail and plain, STARTTLS, SSL/TSL connection for POP3 and SMTP without keystore

Today my target was to implement various connections for POP3/SMTP email accounts. incuding:
  • plain connection with no encryption
  • STARTTLS
  • SSL/TSL
The key requirement here was to have it without java keystore file configured for storing and accepting server certificates. This should work transparently for the user.

Having dug through tons of not working examples on the net, and after trying figure out some solutions by trial and error, I decided to take my Plain Old Eclipse Debugger and dive into the javax.mail classes. Here are quick results of HOWTO.

The environment you need is Java Mail API 1.4.5 (1.4 doesn't work properly, due to inability to set custom socket factory for STARTTLS). On the other hand, I don't use plain SMTP connection, but Spring JavaMailSender, but it has exactly the same config (with properties).

AlwaysTrustSSLContextFactory

First thing to have for encrypted connection is to have SSLContextFactory, that trusts all certificates, to skip the verification stage (this is not required for my application). Here is the code:

package com.blogspot.lifeinide;

import java.io.IOException;
import java.net.InetAddress;
import java.net.Socket;
import java.net.UnknownHostException;
import java.security.KeyManagementException;
import java.security.NoSuchAlgorithmException;
import java.security.SecureRandom;
import java.security.cert.CertificateException;
import java.security.cert.X509Certificate;

import javax.net.SocketFactory;
import javax.net.ssl.KeyManager;
import javax.net.ssl.SSLContext;
import javax.net.ssl.SSLSocketFactory;
import javax.net.ssl.TrustManager;
import javax.net.ssl.X509TrustManager;

/**
 * SSL socket factory accepting any certificate.
 */
public class AlwaysTrustSSLContextFactory extends SSLSocketFactory {

    private SSLSocketFactory factory;
        
    public static class DefaultTrustManager implements X509TrustManager {

        @Override
        public void checkClientTrusted(X509Certificate[] arg0, String arg1) 
        throws CertificateException {}

        @Override
        public void checkServerTrusted(X509Certificate[] arg0, String arg1)
        throws CertificateException {}

        @Override
        public X509Certificate[] getAcceptedIssuers() {
            return new X509Certificate[0];
        }
    }

    public AlwaysTrustSSLContextFactory() 
    throws NoSuchAlgorithmException, KeyManagementException {
        super();

        SSLContext ctx = SSLContext.getInstance("TLS");
        ctx.init(new KeyManager[0], new TrustManager[] {
            new DefaultTrustManager()}, new SecureRandom());
            factory = (SSLSocketFactory) ctx.getSocketFactory();
    }

    public static SocketFactory getDefault() {
        try {
            return new AlwaysTrustSSLContextFactory();
        } catch (Exception e) {
            throw new RuntimeException("Cannot instantiate default AlwaysTrustSSLContextFactory", e);
        }
    }

    public Socket createSocket() throws IOException {
        return factory.createSocket();
    }

    public Socket createSocket(InetAddress address, int port, 
    InetAddress localAddress, int localPort)
    throws IOException {
        return factory.createSocket(address, port, localAddress, localPort);
    }

    public Socket createSocket(InetAddress host, int port) throws IOException {
        return factory.createSocket(host, port);
    }

    public Socket createSocket(Socket s, String host, int port, 
    boolean autoClose) throws IOException {
        return factory.createSocket(s, host, port, autoClose);
    }

    public Socket createSocket(String host, int port, InetAddress localHost, 
    int localPort) throws IOException, UnknownHostException {
        return factory.createSocket(host, port, localHost, localPort);
    }

    public Socket createSocket(String host, int port) throws IOException, 
    UnknownHostException {
        return factory.createSocket(host, port);
    }

    public boolean equals(Object obj) {
        return factory.equals(obj);
    }

    public String[] getDefaultCipherSuites() {
        return factory.getDefaultCipherSuites();
    }

    public String[] getSupportedCipherSuites() {
        return factory.getSupportedCipherSuites();
    }

}

The funny part is that the getDefault() static method is crucial here, because javax.mail classes get the SSLContextFactory from this static method, and not by creating the object using constructor. This is a big trap here that can consume a lot of time to conceive. This also can reveal how the rest of sources are written, what is pretty bad for me (strange initializations by static methods and poorly described string properties, with which you can build a lot of combinations to try make it work).

Now, without a lot of complaining more, the revealed solutions. The thing we want to achieve is to get the connected javax.mail.Store from javax.mail.Session to work with POP3, and configured JavaMailSender (with javax.mail properties) for SMTP.

Plain connection

The easiest one. Implementation for POP3:

Session session = Session.getDefaultInstance(new Properties(), null);
Store store = session.getStore("pop3");
store.connect(host, port, username, password);

And SMTP:

JavaMailSenderImpl sender = new JavaMailSenderImpl(); 
sender.setHost(host);
sender.setUsername(username);
sender.setPassword(password);
sender.setPort(port);
                
Properties props = new Properties();
props.setProperty("mail.smtp.auth", "true");

sender.setJavaMailProperties(props);

SSL/TLS

Now the version working from the beginning through secure channel. POP3:

Properties props = new Properties();

props.setProperty("mail.pop3s.socketFactory.class", "com.blogspot.lifeinide.AlwaysTrustSSLContextFactory");
props.setProperty("mail.pop3s.socketFactory.port", port);
props.setProperty("mail.pop3.ssl.enable", "true"); 

URLName url = new URLName("pop3s", host, port, "", username, password);
Session session = Session.getInstance(props, null);
Store store = session.getStore(url);
store.connect();

And the SMTP likewise:

JavaMailSenderImpl sender = new JavaMailSenderImpl(); 
sender.setHost(host);
sender.setUsername(username);
sender.setPassword(password);
sender.setPort(port);

Properties props = new Properties();
sender.setProtocol("smtps");
props.setProperty("mail.smtps.auth", "true");
props.setProperty("mail.smtps.socketFactory.class", "com.blogspot.lifeinide.AlwaysTrustSSLContextFactory");
props.setProperty("mail.smtps.socketFactory.port", port);
props.setProperty("mail.smtp.ssl.enable", "true"); 

sender.setJavaMailProperties(props);

STARTTLS

The last one. POP3 version:

Properties props = new Properties();

props.setProperty("mail.pop3.ssl.socketFactory.class", "com.blogspot.lifeinide.AlwaysTrustSSLContextFactory");
props.setProperty("mail.pop3.ssl.socketFactory.port", port);
props.setProperty("mail.pop3.starttls.enable", "true");

URLName url = new URLName("pop3", host, port, "", username, password);
Session session = Session.getInstance(props, null);
Store store = session.getStore(url);
store.connect();

And SMTP:

JavaMailSenderImpl sender = new JavaMailSenderImpl(); 
sender.setHost(host);
sender.setUsername(username);
sender.setPassword(password);
sender.setPort(port);
                
Properties props = new Properties();
props.setProperty("mail.smtp.auth", "true");
props.setProperty("mail.smtp.ssl.socketFactory.class", "com.blogspot.lifeinide.AlwaysTrustSSLContextFactory");
props.setProperty("mail.smtp.ssl.socketFactory.port", port);
props.setProperty("mail.smtp.starttls.enable", "true");

sender.setJavaMailProperties(props);

Looking for amount of questions and not working examples on the net, and amount of time I needed to solve it, the guys implemented it this way should get the prize for one of the most obscure solutions in java I was working with.