Monday, January 23, 2012

Modular Java Applications - A Microkernel Approach

Software Engineering is all about reuse. We programmers therefore love to split applications up into smaller components so that each of them can be reused or extended in an independent manner.

A keyword here is "loose coupling". Slightly simplified, this means, each component should have as few dependencies to other components as possible. Most important, if I have a component B which relies on component A, I don't want that A needs to know about B. The component A should just provide a clean interface which could be used and extended by B.

In Java there are many frameworks which provide this exact functionality: JavaEE, Spring, OSGI. However, each of those frameworks come with their own way to do things and provide lots and lots of additional functionality - whether you want it or not!

Since we here at scireum love modularity (we build 4 products out of a set of about 10 independet modules) we built our own little framework. I factored out the most important parts and now have a single class with less than 250 lines of code+comments!

I call this a microkernel approach, since it nicely compares to the situation we have with operating systems: There are monolithic kernels like the one of Linux with about 11,430,712 lines of code. And there is a concept called a microkernel, like to one of Minix with about 6,000 lines of executable kernel code. There is still an ongoing discussion which of the two solituons is better. A monolithic kernel is faster, a microkernel has way less critical code (critical code means: a bug there will crash the complete system. If you haven't already, you should read more about mikrokernels on Wikipedia.

However one might think about operating systems - when it comes to Java  I prefer less dependencies and if possible no black magic I don't understand. Especially if this magic involves complex ClassLoader structures. Therefore, here comes Nucleus...

How does this work?
The framework (Nucleus) solves two problems of modular applications:
  • I want to provide a service to other components - but I only want to show an interface and they should be provided with my implementation at runtime without knowning(referencing) it.
  • I want to provide a service or callback for other components. I provide an interface, and I want to know all classes implementig it, so I can invoke them.
Ok, we probably need examples for this. Say we want to implement a simple timer service. It provides an interface:

public interface EveryMinute {
    void runTimer() throws Exception;
}

All classes implementing this interface should be invoked every minute. Additionally we provide some infos - namely, when was the timer executed last.

public interface TimerInfo {
    String getLastOneMinuteExecution();
}

Ok, next we need a client for our services:


@Register(classes = EveryMinute.class)
public class ExampleNucleus implements EveryMinute {

    private static Part<TimerInfo> timerInfo = 
                                     Part.of(TimerInfo.class);

    public static void main(String[] args) throws Exception {
        Nucleus.init();

        while (true) {
            Thread.sleep(10000);
            System.out.println("Last invocation: "
                    + timerInfo.get().getLastOneMinuteExecution());
        }

    }

    @Override
    public void runTimer() throws Exception {
        System.out.println("The time is: "
                + DateFormat.getTimeInstance().format(new Date()));
    }
}

The static field "Part<TimerInfo> timerInfo" is a simple helper class which fetches the registered instance from Nucleus on the first call and loads it into a private field. So accessing this part has almost no overhead to a normal field access - yet we only reference an interface, not an implementation.

The main method first initializes Nucleus (this performs the classpath scan etc.) and then simply goes into an infinite loop, printing the last execution of our timer every ten seconds.

Since our class wears a @Register annotation, it will be discovered by a special ClassLoadAction (not by Nucleus itself) instantiated and registered for the EveryMinute interface. Its method runTimer will then be invoced by our timer service every minute.

Ok, but how would our TimerService look like?

@Register(classes = { TimerInfo.class })
public class TimerService implements TimerInfo {

    @InjectList(EveryMinute.class)
    private List<EveryMinute> everyMinute;
    private long lastOneMinuteExecution = 0;

    private Timer timer;

    public TimerService() {
        start();
    }

    public void start() {
          timer = new Timer(true);
          // Schedule the task to wait 60 seconds and then invoke
          // every 60 seconds.
          timer.schedule(new InnerTimerTask(), 
                         1000 * 60, 
                         1000 * 60);
    }
    private class InnerTimerTask extends TimerTask {

        @Override
        public void run() {
            // Iterate over all instances registered for
            // EveryMinute and invoke its runTimer method.
            for (EveryMinute task : everyMinute) {
                    task.runTimer();
            }
                             // Update lastOneMinuteExecution
            lastOneMinuteExecution = System.currentTimeMillis();
        }

    }

    @Override
    public String getLastOneMinuteExecution() {
        if (lastOneMinuteExecution == 0) {
            return "-";
        }
        return DateFormat.getDateTimeInstance().format(
                new Date(lastOneMinuteExecution));
    }
}




This class also wears a @Register annotation so that it will also be loaded by the ClassLoadAction named above (the ServiceLoadAction actually). As above it will be instantiated and put into Nucleus (as implementation of TimerInfo). Additionally it wears an @InjectList annotation on the everyMinute field. This will be processed by another class named Factory which performs simple dependency injection. Since its constructur starts a Java Timer for the InnerTimerTask, from that point on all instances registered for EveryMinute will be invoced by this timer - as the name says - every minute.

How is it implemented?
The good thing about Nucleus is, that it is powerful on the one hand, but very simple and small on the other hand. As you could see, there is no inner part for special or privileged services. Everything is built around the kernel - the class Nuclues. Here is what it does:

  • It scans the classpath and looks for files called "component.properties". Those need to be in the root folder of a JAR or in the /src folder of each Eclipse project respectively. 
  • For each identified JAR / project / classpath element, it then collects all contained class files and loads them using Class.forName.
  • For each class, it checks if it implements ClassLoadAction, if yes, it is put into a special list.
  • Each ClassLoadAction is instanciated and each previously seen class is sent to it using: void handle(Class<?> clazz)
  • Finally each ClassLoadAction is notified, that nucleus is complete so that final steps (like annotation based dependency injection) could be performed.
That's it. The only other thing Nucleus provides is a registry which can be used to register and retrieve objects for a class. (An in-depth description of the process above, can be found here: http://andreas.haufler.info/2012/01/iterating-over-all-classes-with.html).

Now to make this framework useable as shown above, there is a set of classes around Nucleus. Most important is the class ServiceLoadAction, which will instantiate each class which wears a @Register annoation, runs Factory.inject (our mini DI tool) on it, and throws it into Nucleus for the listed classes. Whats important: The ServiceLoadActions has no specific rights or privileges, you can easily write your implementation which does smarter stuff.

Next to some annotations, there are three other handy classes when it comes to retrieving instances from Nucleus: Factory, Part and Parts. As noted above, the Factory is a simple dependency injector. Currently only the ServiceLoadAction autmatically uses the Factory, as all classes wearing the @Register annotation are scanned for required injections. You can however use this factory to run injections on your own classes or other ClassLoadActions to do the same as ServiceLoadAction. If you can't or don't want to rely in annotation based dependency magic, you can use the two helper classes Part and Parts. Those are used like normal fields (see ExampleNucleus.timerInfo above) and fetch the appropriate object or list of objects automatically. Since the result is cached, repeated invocations have almost no overhead compared to a normal field.

Nucleus and the example shown above is open source (MIT-License) and available here:
https://github.com/andyHa/scireumOpen/blob/master/src/examples/ExampleNucleus.java
https://github.com/andyHa/scireumOpen/tree/master/src/com/scireum/open/nucleus


If you're interested in using Nucleus, I could put the relevant souces into a separater repository and also provide a release jar - just write a comment below an let me know.

Update:  I moved nucleus into a repository on its own: https://github.com/andyHa/nucleus - It even includes a distribution jar.


This post is the fourth part of the my series "Enterprisy Java" - We share our hints and tricks how to overcome the obstacles when trying to build several multi tenant web applications out of a set of common modules.

11 comments:

  1. It looks great .it will be great if you share the code as well.

    ReplyDelete
    Replies
    1. The code is available via GitHub: https://github.com/andyHa/scireumOpen/tree/master/src/com/scireum/open/nucleus

      If anyone plans to use it, le me know. I'll put it in a separate repository and provide a single jar.

      cheers Andy

      Delete
    2. Could you provide single jar I want to test it?

      Delete
    3. There you go: https://github.com/andyHa/nucleus/blob/master/dist/nucleus.jar - Let me know if it works for you.

      Delete
  2. In several projects I've used an even simpler approach: A class that had a hashmap where individual services could be registered using their interface as key. An static instance of this class can be easily called from anywhere in the application, in a simmilar way as you are doing in the ExampleNucleus class. It was the most straightforward implementation of the Service Locator pattern I could think of.

    Agreed, I had to write code to register the services instead of relying on the classes being detected and injected by the classloader, but on the other hand it also gave me detailed control about which components were actually added.

    Nevertheless, it's certainly interesting the flexibility that this system provides by mixing the DI and SL patterns.

    ReplyDelete
    Replies
    1. Enrique, thanks for your feedback. I originally had the same approach. But, in my eyes you always end up with a chicken-egg-problem: In order to register all services, you need to discover all components - so that these register their services provided. Therefore you can't get around a discovery mechanism - whichever you choose. In my next post, I'll talk about JPA vs. modular applications. That'll provide more background on my solution.

      However, I'm glad to hear that I'm not the only one who "reinvented" the wheel ;-)

      regards Andy

      Delete
  3. Yes, I have to admit that it felt a bit awkward to do almost the same other people was doing (eg. Spring, Guice, ...) without actually using them.

    In our case it made sense to create the instances manually and not using a discovery service, though. We had something like 10-15 "services" which we share between several projects, easy enough to be created manually. Most of them are pretty general and reusable and are stored for convenience in our framework library, but in some cases we need to be able to use a subclass of the general one with some extra functionality specific for a given project. So in this case both classes must be in our classpath, but we actually only want the subclass to be instantiated, otherwise functionality would be duplicated in some cases and errors would appear in others.

    I guess that in this case it would be possible to write an implementation of ServiceLoadAction which inserted the classes in the Factory statically instead of relying just on the annotations, but I don't know if it is possible to disable the standard one to prevent the superclass from being registered also.

    Regards,
    Enrique

    ReplyDelete
  4. Thanks Andreas for this application. We are planning to use it for its simplicity in our project and just to point out one of the things that can be a headache is if the main path has spaces The class loader does not find any classes.

    Regards,
    Bwire.

    ReplyDelete
  5. Thanks Andy.
    The problem with the spaces in the file name is caused by the URl class which URL-encodes the path name given to the File object constructor:
    File file = new File(url.getPath());
    Changing this to
    File file = new File(url.getURI());
    will solve this tiny issue.

    ReplyDelete
    Replies
    1. Glad you find our approach useful and thanks for improving on it.

      I just updated the repository using your fix - thanks again.

      regards Andy

      Delete