Increase Trac productivity with plugins

A substantial time amount passed since last post in this blog caused only because I'm recently very busy, although the list of interested subjects to describe in my Tomboy notes are extending monthly. Unfortunately I don't know when the appropriate time comes to make it...

Anyway today I had some more free day and I needed to setup some Trac environments for the team and the customers. Casually I reviewed some Trac plugins, that seemed to me sensible, and should increase productivity with Trac. Bellow is the short list of those I've found useful.

NeverNotifyUpdaterPlugin

First and foremost I always install this plugin. This is very annoying in Trac for me, that it always sends a notification to the person who updates the ticket. This is completely unnecessary feature, because I believe that if someone is updating the ticket, he knows about that, and doesn't require additional email notification. The installation of this plugin simply disables this feature.

AnnouncerPlugin

This is new for me, and I've installed this to fully verify the usefulness of the FullBlogPlugin, described below. This plugin basically adds a few great notification capabilities to Trac.

First is the "watch" capability for the tickets and wiki pages. While the "watch" feature for tickets is available by default using CC field, I've found this more friendly using this plugin. For each ticket/wiki page you can just click "Watch/Unwatch" links on the top of the ticket/wiki page view, and you are then notified about all changes in the ticket/wiki page. This is especially great for wiki pages describing some important facilities in our application. By subscribing changes all team can be informed about updates in these mechanisms.

Instead of subscribing all wiki pages by whole team (by clicking "Watch" on important pages), there's an option in Settings, that enables entering the names (also with using wildcards) of all wiki pages that should be by default watched by all team members.

Another function that may be considered useful is the possibility of watching concrete users, and their changes in Trac (new tickets, updates etc...). Currently I see few usages of this - eg. if you need to manage some subset of Trac users, or you have Trac opened to the customers and you'd like to trace their updates.

FullBlogPlugin

This plugin adds a blog feature to the Trac, which involves adding new blog posts, browsing and commenting them. This is the great feature, which I believe that can support scrum meetings, and even introduce scrum-meeting-like functionality for remote teams, working in different time zones.

In recent project, to notify about some important changes in the project, new features, and deployment events we were using one big ticket with all team set on CC. This may be now changed to the blog, when this plugin is installed. If each developer is told to write blog entries about recent work, new mechanisms and important changes in existing ones, the whole team can keep up-to-date information about the project as totality (which is one of responses of the scrum meetings).

The FullBlogPlugin lacks by default if the notification capabilities, but when you install the current trunk version of AnnouncerPlugin (described above), it introduces these lacking features. You can for example "watch" the blog entries, or follow some chosen bloggers. But for applying mentioned function of the blog feature in the team, the best are automatic options for subscribing blogs, which are following:

• always notify me when any blog is modified, changed, deleted or commented on
• always notify me when any blog post is created
• always notify me when any blog that I posted is modified or commented on
• always notify me when a blog that I'm watching changes
• always notify me when any blogger that I follow has a blog update

TracHoursPlugin

This is maybe less about productivity, but more about saving the time, of tracking the hours spent on the work. But first a little introduction...

There's a much bigger Trac plugin for time tracking and estimation: TimingAndEstimationPlugin. I was working with this once. This plugin allows you to estimate tickets time, add hours spent on ticket by particular developers, make some estimation and budget calculation using these information, and even connect another plugins making burn down charts, etc. This might be a better choice for someone, depending on what he wants to achieve, although in my personal opinion this plugin was not really good, and we used it just once for one project. AFAIR the information calculated/managed was not clear to us, it introduced some more complexity to Trac (and the power of this system is kept, when it's kept simple). Moreover, for estimation, requirements and user stories management, and generally managing project in scrum with burn down charts etc. I can much more recommend Agilo, which is complete tool for scrum project management based on Trac, than the TimingAndEstimationPlugin.

Now, returning to TracHoursPlugin, this is very simple one. It adds the "Add hours" link on the ticket view page, and the Hours view in menu, to calculate hours added to tickets with some query filters (like default query filters for Trac reports). So you can for example check the hours spent in some time range, spent by some developers, or spent on some ticket types or component.

This plugin is really great when you make some project, and you base on hourly rate, but with no time commitment, so for example one week you can work 10 hrs, while another 30 hrs, etc. This plugin allows you to really easy track the time and to make the financial settlements with the customer.

ChildTicketsPlugin

For a longer time I've been looking for better way to organize Trac tickets. This is the biggest problem in bigger projects managed with Trac, that it quickly becomes a mess with the plain "ticket list" views. only Unfortunately I didn't find anything that 100% satisfies me here, and if I manage project with Trac, I usually support myself with the mind mapping software, like Freemind (on the other hand this is best project management tool I've been using until now ;) ).

The ChildTicketsPlugin is an approach to introduce parent - child relations in the Trac tickets, and I 've currently chosen this plugin for test if it really helps with tickets organization in my new project. Maybe for some time I'll write a little more about this. For now I can just write that it looks simple. It allows you to define parent - child relations between tickets, and you can define what type of parent can have what type of children. Depending on how you like to organize your project, it can be eg. Requirement - User Story - Task, or Feature - Task + Feature - Bug etc. Whatever structure you find useful, you can configure here.

Now, the rest is easy, adding new ticket you can enter the parent ID, and you create the parent - child relation between these tickets. However, this is not very usable workflow: add ticket - enter parent ID, because you'd need to know the parent ID first, but the plugin provides another way of adding tickets - you can first find the appropriate parent ticket, and add any of the allowed child ticket types using a button.

This simple feature can be supported by additional reports, allowing us to view some kind of root ticket types, and "drill down" the project. We can also use empty/not empty parent ID filter, to determine orphaned tickets. Moreover there's a ChildTicketTreeMacro macro available, which can display the tickets tree structure in various places (like wiki pages, or ticket descriptions).

There's an alternative plugin for Trac, which is called MasterTicketsPlugin, what might be useful for some situations, but looking on the description it does far to much than I need, and brings more complexity for Trac. However, I didn't tested it today because of lack of time.

And more...

There are a lot more of Trac plugins that might be considered useful from the productivity point of view, which I didn't test. But currently these mentioned here appeases all my needs and fills the lacking functionality in Trac. However, I hope I'll extend this post in future.

Transparent dynamic forms with LazyList

Some time ago I wrote few words about some dynamic forms implementation methods using LazyList or AutoPopulatingList with items removing. The problem that interested me today was related to this subject, but concerned something else: how to implement dynamic forms in the way, so that it can be used transparently for any object (even a Hibernate entity) without creating specialized form beans for that.

If you write a Hibernate application there are at least two schools about implementing forms backend. First tells you to create a object for each form, representing given form data, second lets you use directly Hibernate entities in forms. The first one is maybe cleaner than the second one from the "patterns compliance" point of view, but produces a lot of unnecessary code. There's s lot of stuff on the net to read about this. Some people do this, some that. I'm not eager to generalize things and give the best solution always, but from my point of view it just depends. It depends mainly on the project size and overall application requirements, but generally I believe that if you don't make a big financial system with 20+ developers, and if you can apply session-per-request (or session-per-conversation) model, (ie. you don't have eg. many databases and session factories, distributed transactions management etc.) working together with OSIV, you may consider using directly you business objects across the whole application.

I can evaluate that 50-80% forms in the usual web application are just the domain object editors. In mentioned application, if you like to edit all these entities with form beans, you need to maintain large (1:1 with business model) class hierarchy, looking exactly the same like the domain object hierarchy. Moreover, you need to maintain a lot of conversion code, making one objects from another. Is this really needed? I consider not, as well as for example these guys. I'm the follower of using domain objects throughout whole application, in as many places as you can, and write just a feature code, instead of maintaining every day tons of code that will never bring any benefits.

Today I had a time to verify these assumptions in pure Spring environment. I previously made other projects with such approach, but never really used pure Spring in a web layer. I consider this as a very good (and lightweight in proportion to pure JEE) middleware, but about Spring MVC/WebFlow I had a different opinion, in which these are not well-rounded and very crude. For those people looking for better solutions I can recommend Stripes Frameworks, what is the best web framework I used until now in Java world. Spring guys could learn a lot from Stripes guys about how to create a greatly flexible web (layer) framework. But, let's don't gripe, I use also Spring in Stripes-based applications as a base framework (IOC, scheduling, webservice, you know ...) and it's great. The only caveat I can have to MVC/WebFlow part.

Returning to the subject: how to implement transparent dynamic forms for any class used as form bean, I need to tell that this all is about the Binder you use. By the Binder I mean the code that converts your plain strings from HTTP request into an object property values. For example StripesFramework has its own binder that is very flexible and extendable, so I've never had problems with writing anything with this. Regarding Spring code it looks little worse, but this is for a while. Anyway, if you'd like to have dynamic forms handling transparent, you  need to have appropriately configured binder, that is able to manage with these concerns. If you don't have a flexible Binder, you need to implement all stuff with your form beans with eg. LazyList wrappers, and also all this stuff behind.

In the Spring the main role in binding plays BeanWrapperImpl class. If you take a glance at implementation, you will see some code in getPropertyValue() method taking elements from the target bean collections. This is base implementation assuming that you have these elements already initialized, otherwise it throws exceptions (default Spring implementation).

OK, this is the code that should be extended:  the BeanWrapperImpl should be switched to a different implementation, considering dynamic lists for whatever objects playing "form bean" role. But, unfortunately, not for Spring. The BeanWrapperImpl class simply cannot be extended, because of its horrible design, like a lot of private methods and dependencies to other package-visible classes. In spring MVC, this (the most) important part of the request processing is completely monolith and not extendable. From the good framework I expect that if the behavior is undesirable, I can change this, but Spring doesn't give that.

But, what if I want to have this in a Spring or whatever else badly designed binder? I just need to make it myself in the independent code. The working solution I applied is below. First thing is to get to a binding code. In your framework you will probably have somewhere some code looking like this:

public class Binder {

  public void bind(Object root, RawValues values) {
    doBind(..);
  }     
        
}

 

This is the base binder call that requires a root object (form bean) and the request parameter values in some form. For example if you use WebFlow in FormAction the binder (WebDataBinder) is being created in createBinder() method. You can switch the implementation there to you binder class and override some behavior (the DataBinder in Spring can be easily inherited, but it internally uses BeanWrapperImpl that does all job, and it cannot). This is how does look my data binder implementation for Spring:

public class DynamicDataBinder extends WebDataBinder {
        
  @Override
  protected void doBind(final MutablePropertyValues mpvs) {
    new DynamicBindingWrapper(getTarget(), new Runnable() {
      @Override
      public void run() {
        _doBind(mpvs);
      }
    });
  }
        
  protected void _doBind(MutablePropertyValues mpvs) {
    super.doBind(mpvs);
  }
        
}

This is the Spring example, but of course you can use this for any framework you use, because binding needs always to look similar in any framework (requires an object to be bound and the parameters). What I do here clearly is to wrap the binding process (done in super.toBind() - I needed to put it into separate method because of anonymous Runnable class utilization), that can be done in its default way, with some code applying the dynamic forms behavior to form bean.

The thing that enclosing code need to do is to:

• make all required collections appropriate for dynamic form binding
• allow to execute the binding in the way that binder code in unaware of non-existing collection items
• restore collections to previous state, to be further (after binding) processes in a regular way

The last point is very important, because for Hibernate entities that are already persistent, we cannot switch a collection (eg. to HashSet)  and disjoin the Hibernate persistent (eg. PersistentSet) implementation. For persistent entities we have to ensure that after binding all entity collections are in the same implementations as they were before binding.

To mark the collections that during the binding should behave "dynamically" I  use additional annotation, because only for some special collection we need to apply this. For example if we have an Invoice having InvoiceItem collection that we need to edit in a dynamic way on a single form. The annotation I use is very simple:

@Target({ElementType.FIELD})
@Retention(RetentionPolicy.RUNTIME)
public @interface DynamicList {
  Class value();
}

In the value() attribute we will point the exact collection generic type, so that the list could produce new items. This also can be done by reflection, but here for simplify we will name exact class name. So, our exemplary Invoice for dynamic item handling should look like that:

private class Invoice {

  @DynamicList(InvoiceItem.class)
  protected List<InvoiceItem> items = new ArrayList<InvoiceItem>();

  public List<InvoiceItem> getItems() {
    return items;
  }

  public void setItems(List<InvoiceItem> items) {
    this.items = items;
  }
  
}

I've intentionally omitted Hibernate annotations, that should be put on this class and its fields (if you use annotation Hibernate configuration). We are currently interested only in dynamic forms.

Now, I have to say something about how I assume this works. The dynamic collection for me is completely (also considering underlying objects graph) reflected in the HTML form. This may be not appropriate in some cases, so if it's not appropriate for you, you need to apply different "merging" mechanism. For my (usual) cases it's easy, because if we have all stuff held in the form (these already persisted items, together with a new ones), we may always produce the entity collection from the beginning: by clearing the existing collection and producing new one (the existing items will be deleted and re-created). This is the simplest approach and it's appropriate in most cases.

Having all these assumptions, the final algorithm is following:

• we look for @DynamicList collection fields in target form bean, these ones  we've found we wrap with ShrinkableLazyList to be "dynamic form aware",
• when we produce new items by LazyList factory, we need also to wrap these entities in the same fashion, because we may have much longer objects graph requiring the same dynamic forms behavior,
• on the end of process we need to "return" previous collection instances to the original form beans (if it's persistent Hibernate entity, it would very much like to have the same PersistentCollection instance that it assumes to be present in particular entity property).

The implementation looks following:

public class DynamicBindingWrapper {

  /** 

   * A SkrinkableLazyList implementation for collections wrapping.

   * By this implementation we assert that this wrapping collection

   * is compatible with both Set and List.

   **/
  protected class DynamicBindingCollection extends ShrinkableLazyList
      implements Set {

    protected Object target;
    protected Field field;
    protected Collection wrapped;

    protected Class itemClass;

    public DynamicBindingCollection(Object target, Field field,
        final Class itemClass) throws IllegalArgumentException,
        IllegalAccessException {
      // initialize on empty wrapped collection + init items factory       

      super(new ArrayList(), new Factory() { 

 
        @Override
        public Object create() {
          try {
            Object o = itemClass.newInstance();
            begin(o); // all new instances need to be wrapped to assert  

                      // dynamic binding on nested paths 

            return o;
          } catch (Exception e) {
            e.printStackTrace();
            return null;
          }
        }

      }, true);

      // init fields       

      this.target = target;
      this.field = field;
      this.itemClass = itemClass;

      // wrap the original object 
      field.setAccessible(true);
      Object value = field.get(target);

      if (value instanceof Set || value instanceof List) {
        this.wrapped = (Collection) value;
        field.set(target, this); // wrap the target bean collection
        register(this); // register for unwrapping       

      } else
        throw new IllegalArgumentException(
            "Only Set and List instances can be wrapped for DynamicList");
    }

    public void unwrap() throws IllegalArgumentException,
        IllegalAccessException {
      // clear wrapped collection to be populated again from the form 
      wrapped.clear();

      // get real values (indirectly calls ShrinkableLazyList.shrink())       

      for (Object o : this)
        wrapped.add(o); 

      // restore original collection 
      field.setAccessible(true);
      field.set(target, wrapped);
    }

  }

  protected List<DynamicBindingCollection> collections = 
    new ArrayList<DynamicBindingWrapper.DynamicBindingCollection>();

  public DynamicBindingWrapper(Object target, Runnable task) {
    begin(target);
    task.run();
    done();
  }

  /** Main wrapping method **/

  protected void begin(Object target) {
    try {
      Class clazz = target.getClass();
      while (clazz != null) {

        for (Field field : clazz.getDeclaredFields()) {
          DynamicList annot;
          if ((annot = field.getAnnotation(DynamicList.class)) != null)

            // wrap the collection with lazy list            new DynamicBindingCollection(target, field, annot.value());
        }

        clazz = clazz.getSuperclass();
      }
    } catch (Exception e) {
      throw new RuntimeException("Cannot get properties from bean", e);
    }
  }

  /** Ends binding session and unwraps instances */
  protected void done() {
    try {
      for (DynamicBindingCollection collection : collections)
        collection.unwrap();
    } catch (Exception e) {
      throw new RuntimeException("Cannot unwrap dynamic collection", e);
    }
  }

  protected void register(DynamicBindingCollection collection) {
    collections.add(collection);
  }

}

Today tests showed that this works nice both for transient and persistent Hibernate entities used as "form beans" (and of course for non-Hibernate form beans as well).

Edit conflicts in eclipse using submerge

Everyone using Eclipse + SVN (or other non-locking version control system) has gone through the hell of editing conflicts with default Eclipse tool. I don't know, maybe there are people liking this, but for me is very unhandy, messy and produces more work than doing the merge without any tool. I just cannot see what is currently merged, how it's done and what is to be done already in this editor window:

Furthermore every important thing needs here to be done by manual copy & paste from right to left.

I remember when once (and enoughs ;) ) I worked in company using Windows, where I was using TortoiseSVN integrated with Windows Explorer. The comfort between their "edit conflict" tool and this one just cannot be compared.

For longer time I was looking for alternatives in Linux. I passed through all known tools like kdiff3 or meld but in my opinion they are great for diff, but not for edit conflicts. However recently I've finally found appropriate editor and I'm very satisfied from using it. This is a submerge tool from subcommander.

In "edit conflicts" mode it provides three panels with "mine" and "theirs" version (at the top) and initial merge result (at the bottom). It's very convenient for usage - you can navigate change by change using buttons on toolbar. By default always the blue change is applied into a merged file. If you like to change this you just double click on the version you want to apply in "mine" or "their" file. All conflicting changes are highlighted with red color, and you can choose appropriate change also with double click. The exemplary conflict edition looks following:

To install submerge just download and install latests subcommander from the website or from your package repository (it's available in Debian / Ubuntu repositories as subcommander). After installation you should have submerge command available from command line.

Last thing is to integrate submerge into Eclipse. This can be done using Windows | Preferences | Team | SVN | Diff Viewer. Add here new file association for all files ('*'):

The arguments for both commands to copy&paste are following:

• diff ${base} ${mine}
• merge ${base} ${mine} ${theirs} ${merged}

You can then enjoy with submerge in "Compare with..." and "Edit conflicts" functions.