The Watchmaker Framework for Evolutionary Computation has reached version 0.7.1.  This is an incremental release that refines a couple of the changes made in the substantial 0.7.0 update.  It also adds support for Evolution Strategies to complement the existing support for generational and steady-state evolutionary/genetic algorithms.

In addition, the (still incomplete) user manual has a new section on the different selection strategies that are available.

• Downloads
• Full Changelog
• API Documentation
• User Manual

The Watchmaker Framework for Evolutionary Computation has reached version 0.7.0. If you’re new here, Watchmaker is a Java library for implementing evolutionary/genetic algorithms. This release is the most substantial update for some time.

Backwards-Incompatibilities

Firstly, I’ve refactored the evolution engine so that it is not as tightly tied to the standard generational model of evolution. What this means in practice is that the class formerly known as ConcurrentEvolutionEngine is now called GenerationalEvolutionEngine. For most users this should be the only backwards-incompatibility in this release. The constructor arguments are the same and all the old methods are still there so it should just be a case of updating your code to use the new name. If you were using SequentialEvolutionEngine, and I think this mostly affects the Mahout guys, this class no longer exists. Concurrency is no longer managed via different sub-classes. Instead you should also use GenerationalEvolutionEngine and call the new setSingleThreaded method to force all work to be done synchronously on the request thread.

If you use Watchmaker to evolve BitString objects, this release improves the performance of the provided mutation and cross-over operators. One side-effect of this is that the semantics of the BitStringMutation class have been modified, so you may need to adjust your parameters.

Steady-State Evolution

OK, on to the good stuff. The reason for the evolution engine changes was to make it easier for the framework to support different types of evolutionary algorithm. The benefits of these changes will become more apparent in subsequent releases but there is one example in this release.  The framework now has first-class support for steady-state evolution via the new SteadyStateEvolutionEngine.  In steady-state evolution the population is evolved one member at a time rather than all members in parallel. This was possible in previous versions of the Watchmaker Framework but only via a nasty hack.

Island Model Evolution

The major new feature in 0.7.0 is support for island model evolution. Instead of evolving a single population, you evolve multiple populations in parallel.  The idea is that the different populations will randomly proceed in different directions making it less likely that the evolution will get stuck at a local optimum. What makes this different from simply evolving a single population multiple times is that there is periodic migration of individuals between islands.  The motivation behind the migration is that combining independently evolved individuals with different traits will hopefully result in even fitter offspring. There’s a lot more detail that I’m glossing over that I intend to cover in a later post, but if you want to get started right away the IslandEvolution class is the place to start.

Sigma-Scaling Selection

0.7.0 also includes a new selection strategy. SigmaScaling uses the population’s fitness standard deviation to regulate the selection pressure, making premature convergence less likely.

More Information

• Full Changelog
• API Documentation
• (Incomplete) User Manual
• Download

Demo Applets

• Sudoku Solver
• Evolution of the Mona Lisa
• Travelling Salesman
• Biomorphs

Tomek Kaczanowski has written a blog post showing how to use ReportNG with Gradle (an alternative build system for Java/Groovy/Scala). I’ve never used Gradle but it seems to have two things in its favour.  Firstly, it uses Groovy syntax rather than the cumbersome XML employed by Ant. Secondly, it’s not Maven. Maybe I should try it out some day?

I recently signed-up for GitHub. Compared to Java.net or Sourceforge, it provides a much lower barrier of entry for code hosting.  There’s no need to wait an indeterminate period of time for somebody to approve your project, you just upload it. And because it’s a DVCS, it’s easy for other people to fork your projects and submit patches. Open Source project hosting has become so straightforward, thanks to sites like GitHub and the Bazaar-based Launchpad, that it encourages developers to open up code that they might otherwise have kept to themselves. After all, why bother with local repositories and back-ups when you can get somebody else to do it for you and get free web-hosting and issue-tracking too?

I have a number of trivial and incomplete projects hosted in local Subversion repositories. I am slowly adding to GitHub those that have any worthwhile substance to them. I’m making no promises about the quality of this code, and I don’t intend to spend much time supporting it, but I’m putting it out there in case somebody might have a use for it.

First up is Zeitgeist. This is a small Java library/application for identifying common topics among a set of news articles downloaded from RSS feeds. It’s sort of like what Google News does. There is a basic HTML publisher included that generates a web page for displaying the current top news stories, including relevant pictures.

You give the program a list of RSS feeds that cover a certain topic (maybe world news, or music news, or a particular sport) and it uses non-negative matrix factorisation to detect similarities in the article contents and to group the articles by topic. The original idea comes from Programming Collective Intelligence.

The default HTML output looks a bit like the image below, but you could customise it with CSS or by hacking the default templates to modify what information is included (for example, you could add an excerpt instead of just displaying headlines).

The algorithm is not infallible and how well it works depends a lot on the feeds that you select. It’s also non-deterministic, so if you run it multiple times with the same input you will get variations in the output.  Perhaps Zeitgeist is not that useful in it’s current form but it could be used for adding on-topic news headlines to a website or as the basis for something more advanced.

When I released version 0.9.9 of ReportNG last week, I didn’t call it version 1.0 because I thought I’d probably have to fix some problem caused by the changes I’d made. I was right. It turns out that I broke the one tiny piece of JavaScript that the HTML reports use. Well that’s fixed now and, because none of the three people who downloaded 0.9.9 have complained about anything else, and because I’m not keen on double-figure components in version numbers (as in 0.9.10), I’ve decided that this can be version 1.0.  You can get it here (or from the Java.net Maven repository if that’s how you roll).

I’ve just released version 0.9.9 of ReportNG, the HTML/XML reporting plug-in for TestNG. This version incorporates several minor changes, mostly improvements to the HTML/CSS.

If you want to see what ReportNG does, it generates HTML test reports that look something like this. You can customise the appearance with your own stylesheets. ReportNG also generates XML reports for compatibility with tools that support JUnit XML files.

For those of you afflicted by Maven, you should be able to get ReportNG from the Java.net Maven 2 repository. Not being a Maven user, I haven’t actually tested that this works, but it should be less wrong than it was last time.

Big news from JetBrains.  Their IntelliJ IDEA Java IDE will be offered in a free Open Source edition from version 9.0.  The free version will include all the JavaSE features, version control integrations for CVS, Git and Subversion, support for unit testing with JUnit and TestNG, and language support for Groovy and Scala.  The full paid-for version will add the enterprise Java tools, UML diagrams, more version control integrations, and language support for SQL, PHP, JavaScript and more.

IntelliJ IDEA is, in my opinion, faster and more powerful than NetBeans and slicker and more intuitive than Eclipse.

As well as opening up the Java IDE, JetBrains are open-sourcing the IntelliJ platform, which is the foundation of all of their IDEs.  As a bonus, they’ve chosen to use the permissive Apache Licence rather than less commercial-friendly GPL.

At the time of writing, the JetBrains web server appears to have crumbled under the weight of the traffic that this announcement has prompted.

In the days before the Watchmaker Framework the two most popular Java evolutionary computation libraries were probably ECJ (Evolutionary Computation [in/for] Java) and JGAP (Java Genetic Algorithms Package). Since the advent of Watchmaker the two most popular Java evolutionary computation libraries are probably ECJ and JGAP. So that worked out well then, but at least the Watchmaker Framework is now being mentioned alongside these established projects.

A recent series of posts over at the Hidden Clause blog compares the pros and cons of implementing the same genetic programming (GP) examples using each of the three frameworks. The article about the Watchmaker Framework is here. In the final analysis Watchmaker is ranked second, ahead of JGAP but behind ECJ, with mostly positive comments (particularly in respect to Watchmaker being the only one of the three frameworks that takes advantage of Java 5′s generics).

Ultimately it seems that the lack of specialist GP support is the main black mark against Watchmaker in this review. Watchmaker is, in its current incarnation, a more general-purpose evolutionary computation framework rather than a specialist system for genetic programming. There is proof-of-concept GP code included but the classes are not part of the core framework. Eventually I would like to include proper GP support, either as part of the core library or as an official add-on library.

I understand that there will be further articles on GP in Java (using ECJ, JGAP and Watchmaker) at Hidden Clause, so it might be worth subscribing if you are interested in this kind of stuff.

This is a bug fix release that addresses a couple of issues with thread management. In version 0.6.1, if you were creating and discarding multiple ConcurrentEvolutionEngines, the threads from the discarded engines would not be cleared up properly.  This could eventually lead to OutOfMemoryErrors if you created a large number of evolution engines.

In version 0.6.2, all ConcurrentEvolutionEngines share a common thread pool so there is no need to create and destroy additional threads.

0.6.2 also reverts the switch to non-daemon threads in version 0.6.1.  If you were having problems with the JVM not exiting when your program completed, this should fix it.

• Download
• Full Changelog
• API Documentation
• User Manual

The Java Language changes accepted for inclusion in JDK7 have been announced by Joseph Darcy. We already knew that closures were off the menu.  So too, unfortunately, is language support for arbitrary-precision arithmetic. The final list is pretty non-controversial and includes a number of changes that will reduce the verbosity of Java programs (one of the main criticisms of Java from proponents of other languages). Java will never be as terse as Perl or Haskell, but that’s no bad thing. One of the strengths of Java is its readability. There are however some areas where the language is needlessly verbose and that’s what these changes are addressing.

Simplified Generics

The last major revision of the Java language was Java 5.0, which introduced generics, auto-boxing, enums, varargs and annotations. Despite the compromises of type erasure, generics have been a major improvement to the language.  They have also contributed to the verbosity of Java code. The necessity to specify, in full, both the reference type and value type of a field or variable has led to some very long declarations:

Map<String, List<BigDecimal>> numberMap = new TreeMap<String, List<BigDecimal>>();

JDK7′s proposed diamond notation allows the programmer to omit the generic parameters on the righthand side if they are the same as the left:

Map<String, List<BigDecimal>> numberMap = new TreeMap<>();

Collection Literals

The long overdue addition of collection literals will help to reduce the size of Java code and make it more readable. Lists, sets and maps can be created and populated without the need for cumbersome instance initialisers:

List<Integer> powersOf2 = {1, 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024};

Map<String, Integer> ages = {"John" : 35, "Mary" : 28, "Steve" : 42};

Automatic Resource Management

Josh Bloch’s proposal for automatic resource management gives Java an alternative to C++’s RAII and C#’s using. It eliminates much of the boiler-plate exception handling that surrounds the proper creation and disposal of resources, such as IO streams, in Java code. The proposal introduces a new interface, Disposable, that resources will implement. The syntax of try/catch/finally is extended to allow resources to be specified at the start. These resources are then automatically disposed upon completion. Here’s an example of the new syntax in action (taken from the proposal):

static String readFirstLineFromFile2(String path) throws IOException
{
    try (BufferedReader reader = new BufferedReader(new FileReader(path))
    {
        return reader.readLine();
    }
}

Other Changes

As well as the above changes to tackle verbosity, JDK7 adds binary integer literals and the ability use String literals in switch statements.  JDK7 will also fix the problem of mixing varargs parameters with generic types.