Play Our Online Football Game and Help Raise Money for Charities

Posted in Python, The Real World by Dan on August 8th, 2014

If you’re not in the UK or you’re not interested in the Beautiful Game and helping charities then this post isn’t for you. I’ve generally tried to keep the articles on this site at least tangentially related to software development but on this occasion the link is a bit more tenuous than usual.

Super10 promo image

I’d like to ask you to consider playing our online charity football game, Super10. You can win a share of several hundred pounds in cash prizes and, more importantly, you can help to raise money for charities. The charity aspect is two-pronged. Firstly you can win money for one of the four organisations that the game is supporting this season (Demelza Hospice Care for Children, Kent, Surrey and Sussex Air Ambulance, Pilgrims Hospices, and homelessness charity Porchlight) and secondly all additional proceeds go to Donations With A Difference (DWAD), the newly-registered charity, of which I am now a trustee, that is running the competition. DWAD will use this money to make grants to individuals and organisations to support the improvement of physical health, mental health and education in the UK.

Super10 is a little bit like fantasy football except that instead of picking players, you pick ten clubs – seven from the English game and three from the top European leagues. These teams score points for you throughout the season. Compared to traditional fantasy football it’s a low-maintenance game as there are no substitutions to deal with each week and transfers only occur during the January transfer window. You can make your selections and mostly forget about them. That said, it does have a way of making you care deeply about the results of teams you previously had no attachment to. Never been to Peterborough? Doesn’t matter, you’ll still feel the despair as they concede a late equaliser to Port Vale.

So what’s the software angle? Well Super10 has been around for a few seasons, certainly much longer than I’ve been involved, but this year we’re trying to take it beyond its previously limited scope. That has meant getting it fully online including being able to take payments online so that we can extend the game to a much wider audience and therefore raise more money. To enable this I’ve built the responsive Super10 website using Python, Django and Bootstrap. Much of the website was in use for most of last season but now we’ve extended it to open up Super10 to the whole country for the first time.

We’re restricting Super10 to the UK only for now because we are not familiar enough with the laws governing this kind of competition in other countries. The deadline for entries is noon (BST) on Saturday 16th August. Give it a go and get your friends to play too.



A Sceptic’s View of Google Glass

Posted in Android, Hardware by Dan on July 18th, 2014

Back in 1999, I was one of a group of Computer Science students invited to visit the research labs of a large consumer electronics company. I don’t remember a great deal about the prototype products we were shown but I do clearly remember being told that wearable computing was the next big thing. 15 years later wearable computers are still the next big thing.

This time around the devices have made it beyond the lab with a slew of underwhelming “smart” watches already on sale. In addition, Apple has been rumoured to be preparing its own for years now. Google on the other hand has taken a different approach by creating a new category of product with Google Glass.

It’s been over two years since Glass sky-dived into public view but it remains subject to a pretty exclusive public beta that you have to pay a hefty premium to join ($1500/£1000). Until a few weeks ago Glass headsets could only be purchased in the US. They are now also available in the UK. On Wednesday evening Google held its first European Glass developer meet-up at Skills Matter in London.

Since first hearing about Glass I’ve been deeply sceptical about it. It’s clearly capable of doing a few neat things that could be useful in a few niche areas but it just seems so inessential, it looks thoroughly ridiculous, and the current price tag is not destined to appeal to sensible individuals. However, I’ve often been too dismissive of new technologies in the past, so I was prepared to at least give it a go.

There was a full house at the Skills Matter Exchange including several Glass-adorned Googlers and a surprising number of other people who had presumably been parted from a grand of their own money. Interestingly, almost everybody had opted for the version with the plastic lenses. When the alternative is the bizarre lens-less titanium forehead band with nose perch it is entirely understandable, whether you need vision correction or not. The spectacle facade makes Google Glass look a lot less conspicuously weird. It’s still not a good look though, even without the beyond-parody third-party add-ons.

Google developer advocates Hoi Lam and Timothy Jordan delivered a couple of presentations suggesting how you should approach building apps for Glass (or Glassware as Google likes to call them). One major drawback for those who might be interested in building these apps is that at present there is no direct way for developers to make money from developing Glassware. Presumably that has to change at some point but for now apps can only be distributed free-of-charge (subject to Google’s approval), and in-app advertising is, mercifully, banned.

Following the presentations, those of us who hadn’t experienced Glass firsthand were given the opportunity to try out the headsets. Due to time constraints and the number of people who wanted to have a go, we didn’t get long enough to be able to get a feel for what it would be like to have this thing on your face all day but here are a few things I noted that might be of interest to those who haven’t tried one of the devices yet.

  1. The early promo shots of Glass tended to avoid showing the battery pack that rests behind your right ear and those pictures that did show it made it look awkwardly bulky. In reality this battery is quite thin and the headset is not as heavy as it looks.
  2. As a POV camera, Glass works well. It’s very easy to take snapshots or video (although I didn’t get an opportunity to check the quality of the results on a bigger screen). Unfortunately, according to Scoble, the battery is only good for 45 minutes of video and it will cook your face in the process. If all you care about is POV photography there are probably much better/cheaper options available.
  3. In a room with a lot of background conversation, the voice recognition worked well. The microphone clearly does a good job of isolating the voice of the wearer.
  4. The “screen” was underwhelming, albeit in a poorly-lit environment. The resolution wasn’t great and the focus didn’t feel entirely comfortable although that may have been partly due to my imperfect contact lens prescription. It may well have been possible to adjust the focus but I didn’t have the time to find out.
  5. The user interface doesn’t feel like it would scale well to having a large number of apps installed. At the moment there is a lot of swiping through cards in a linear fashion.
  6. Unsurprisingly, Glass appears to be tightly integrated with Google+.

I was never going to be a person who would consider paying £1000 for one of these devices but having tried it very briefly I’m now certain that I wouldn’t buy one at a lower price either, even if I ignored the way it looks. There is, as yet, no compelling use case for the average person. One of the main features is that you can get your e-mails, SMS messages and other Android notifications beamed directly on to your retina without having to remove your phone from your pocket. I really don’t have any need for that kind of urgency. I’d rather ignore interruptions until I choose to deal with them. That’s not to say that there aren’t people who could find a use for Glass given better battery life and a more attractive price tag, but for most of us it’s a clever solution in search of a problem – the hardware equivalent of Google Wave.


Deploying an APK to Multiple Devices/Emulators Simultaneously Using Ant

Posted in Android, Ant by Dan on January 4th, 2014

Following my detour to Djangoland, the last week or so I have been back in the world of mobile app development. One of the things I’ve been working on is updating an Android app that has two versions built from the same codebase. Part of this update has been to re-skin both versions of the app and to make sure that everything looks reasonable on various devices running versions of Android from 2.2 up to the latest 4.4. When I build a new APK I want to check it on each of the three devices connected to my machine. If I had more spare USB ports I would conceivably be testing with even more devices. In some scenarios I might also have one or more emulators running to test configurations not represented among my devices. Installing the new APK on each of these from the command line is cumbersome. The adb tool will only install on one device/emulator at a time and, if you have more than one connected, you have to specify a long unique device ID to tell it which one to use. You could write a bash script to retrieve the IDs of connected devices and emulators and to run the adb install command for each, but since I’m using Ant I have written a custom task to do the job. For bonus points I’ve made it issue the adb commands in parallel so that the total install time is determined by the slowest device and not the sum of all the devices.

import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
import java.util.concurrent.TimeUnit;
 * Custom task that uses the ADB tool to install a specified APK on all
 * connected devices and emulators.
 * @author Daniel Dyer
public class InstallAPK extends Task
    private File apkFile;
    public void setAPK(File apkFile)
        this.apkFile = apkFile;
    public void execute() throws BuildException
        if (apkFile == null)
            throw new BuildException("APK file must be specified");
            List<String> devices = getDeviceIdentifiers();
            System.out.printf("Installing %s on %d device(s)...%n", apkFile, devices.size());
            ExecutorService executor = Executors.newFixedThreadPool(devices.size());
            List<Future<Void>> futures = new ArrayList<Future<Void>>(devices.size());
            for (final String device : devices)
                futures.add(executor.submit(new Callable<Void>()
                    public Void call() throws IOException, InterruptedException
                        return null;
            for (Future<Void> future : futures)
            executor.awaitTermination(60, TimeUnit.SECONDS);
        catch (Exception ex)
            throw new BuildException(ex);
    private void installOnDevice(String device) throws IOException, InterruptedException
        String[] command = new String[]{"adb", "-s", device, "install", "-r", apkFile.toString()}
        Process process = Runtime.getRuntime().exec(command);
        consumeStream(process.getInputStream(), System.out, device);
        if (process.waitFor() != 0)
            consumeStream(process.getErrorStream(), System.err, device);
            throw new BuildException(String.format("Installing APK on %s failed.", device));
    private void consumeStream(InputStream in, PrintStream out, String tag) throws IOException
        BufferedReader reader = new BufferedReader(new InputStreamReader(in));
            for (String line = reader.readLine(); line != null; line = reader.readLine())
                out.println(tag != null ? String.format("[%s] %s", tag, line.trim()) : line);
    private List<String> getDeviceIdentifiers() throws IOException, InterruptedException
        Process process = Runtime.getRuntime().exec("adb devices");
        List devices = new ArrayList<String>(10);
        BufferedReader reader = new BufferedReader(new InputStreamReader(process.getInputStream()));
            for (String line = reader.readLine(); line != null; line = reader.readLine())
                if (line.endsWith("device"))
            if (process.waitFor() != 0)
                consumeStream(process.getErrorStream(), System.err, null);
                throw new BuildException("Failed getting list of connected devices/emulators.");
        return devices;

Once compiled and on your classpath, using it is simple:

<taskdef name="installapk" classname="yourpackage.InstallAPK" />
<installapk apk="path/to/yourapp.apk" />

Django ModelChoiceField and HTML <optgroup>

Posted in Python by Dan on December 21st, 2013

I’ve been dabbling with Django over the last couple of months. I’ve played around with it a couple of times previously but this time I’ve actually built something reasonably substantial, which has meant that I’ve had to delve a bit deeper. One of the minor problems I solved today was how to group items in the HTML <select> element generated by a form’s ModelChoiceField. HTML has the <optgroup> tag for this purpose.

It’s not immediately obvious how you can get ModelChoiceField to use optgroups without over-riding the render method and re-implementing the HTML generation yourself, or bypassing ModelChoiceField completely and building something based on the basic ChoiceField (which does support optgroups). The only potential solutions I found from searching took the former approach (here and here). The reason I’m writing this post is because I think I’ve found a better, more concise solution that might be of use to future searchers.

ChoiceField accepts a choices parameter to its constructor. In the simple case this is just a list of items (each item is value/label pair). However, it can also accept a list of groups where each group is a tuple consisting of the group label and a list of items. The problem is that ModelChoiceField is different in that it has a queryset parameter instead, so there is no way to pass in the group information.

However, a comment in the source code says that we can set a property called choices after constructing the ModelChoiceField instance and the queryset will be ignored. The HTML <select> will instead be populated from this data structure with <optgroup> elements as required.

Assuming we want to group items by a field on the model, we can build the list of tuples from the queryset by sub-classing ModelChoiceField and over-riding the constructor. In this example I’m assuming that the field is a list of countries grouped by continent, where the continent is just a text field on the country model.

from django.forms import ModelChoiceField
from itertools import groupby
from operator import attrgetter
class CountryChoiceField(ModelChoiceField):
    def __init__(self, *args, **kwargs):
        super(CountryChoiceField, self).__init__(*args, **kwargs)
        groups = groupby(kwargs['queryset'], attrgetter('continent'))
        self.choices = [(continent, [(, self.label_from_instance(c)) for c in countries])
                        for continent, countries in groups]

In order for this to work, the queryset must be sorted by the field that it is to be grouped by so that items in the same group are adjacent. You can do this before you pass it to the constructor or you can change the code above to call order_by on the queryset.

ReportNG 1.1.4

Posted in Java by Dan on June 11th, 2013

It’s been a couple of years since the last release of ReportNG, my cobbled together alternative reporting plug-in for TestNG. In the time since I’ve done nothing but a few users have submitted useful improvements. So, after some prompting, I’ve released version 1.1.4 (download zip, tgz).

Thanks to Kayla Nimis, criccio, Arcadie, and Nalin Makar the new version has the following improvements:

  • The report now shows the reason for skipping a test.
  • You can specify the org.uncommons.reportng.failures-only property to generate a more minimal report.
  • ReportNG will now create any missing parent directories of the report directory rather than failing if they are absent.

Google Skynet

Posted in The Internet by Dan on May 16th, 2013

Google policy is to get right up to the creepy line and not cross it.

These are the words of former Google CEO Eric Schmidt, speaking to The Atlantic in October 2010.

We don’t need you to type at all. We know where you are. We know where you’ve been. We can more or less know what you’re thinking about.

This was two and a half years ago. Today Google is so far over the creepy line it can’t even see the line any more. The problem is not so much individual Google products but the way in which the vast data from all these disparate services is combined to build a very detailed profile of you. Not just what you do on Google sites but also any other site you visit that includes Google’s +1, Adsense or Analytics JavaScript (including this one). Google can read every e-mail you send and receive, it knows everything you search for online and knows pretty much every website you ever visit. It knows where you are, where you’ve been and probably with whom. Google knows more about you than your mother does and with Google+ it’s all neatly connected to your real identity.

I’ve so far avoided signing up for Google+ but it’s increasingly difficult to ignore, particularly as an Android developer, as it’s becoming more tightly integrated into everything that Google does. It’s the keystone of Google’s anti-privacy agenda. When the time comes for the last remnants of the human race to send an Austrian-accented cyborg back in time to thwart the rise of the machines, its first targets will be the architects of Google+.

The privacy implications are not the only issue. Most users perceive Google’s search results to represent some sort of objective truth, with links ranked only by their relevance to the search query, but in fact each user is served personalised results. If you and I both search for information on some contentious political topic, Google won’t necessarily give us the same response; it will show us each what it thinks we want to see based on its profiling of us.

In his Google I/O Q&A yesterday, Google CEO Larry Page dismissed concerns over the implications of this profiling:

[Audience member]: Most of my opinion, I can trace back to a Google search. As search becomes more and more personalized, and predictive, I worry that it informs my world view and rules out the possibility of some other serendipitous discovery. Any comment on that?

[Page]: People have a lot of concern about that – I’m totally not worried about that at all.

Personally, I don’t like Google’s all-encompassing vision. It’s not the only company that employs such methods but I can easily ignore the likes of Facebook and Bing as I don’t use them. Google is everywhere.

Spurred more by the closure of Google Reader than anything else, a couple of months ago I began to consider alternatives to relying on the benevolence of one omniscient company for the services I use every day.

The first thing to go was GMail. E-mail is important enough to be worth paying for so I signed-up with FastMail (now owned by Opera), which offers an ad-free service from $4.95 per year. It has a refined web interface, IMAP access, and a simple way to import your existing messages from GMail.

I eventually replaced Google Reader with The Old Reader and for search I’m now using DuckDuckGo, which promises not to track its users or filter search results. DuckDuckGo doesn’t quite match Google in terms of the freshness or depth of its results but it’s fast and has some useful features. On DuckDuckGo’s recommendation I also installed Ghostery for Opera to block Google and others’ attempts to track me on third-party sites. I’ve removed the Google +1 buttons from this blog and will be looking for more privacy-friendly alternatives to Google Analytics and Adsense (both of which would be useless anyway if everybody is driven to use the likes of Ghostery).

I’m not giving up on Google entirely, I just prefer to keep it at arm’s length. I’ll still be developing for Android (for which there were many very welcome developer announcements at I/O yesterday) and no doubt I’ll continue to use some of its other services.

Android ListView – Fixing Missing/Blank Dividers

Posted in Android, Java by Dan on January 24th, 2013

A post with code in it. Because it’s been a long time since the last one.

If you work with ListViews in Android, as all Android developers will do at some point, you may notice that if you set your list items to be non-selectable the dividers that are drawn between each cell disappear. In the Holo Light theme you would normally get a thin light grey dividing line between cells. Depending on how you’ve implemented your adapter, you may find that these dividers become white/transparent when the cells are not selectable. According to Android framework engineer Romain Guy, this is the intended behaviour.

As Haythem Souissi points out in this Stack Overflow answer, you can work around this by ensuring that the areAllItemsEnabled method returns true, even though all items are not enabled (maybe none of them are). The isEnabled method will take care of actually disabling the cells and the dividers will be drawn between each of them.

All very straightforward so far but it all goes wrong again when you try to add a non-selectable header view to the list. The way that ListView deals with headers and footers is that it creates its own adapter to wrap/decorate yours and insert the header/footer views in the appropriate places. This is fine but it’s not delegating the areAllItemsEnabled method, so our above fix no longer works.

Fortunately, and unusually for Android, everything we need to resolve this issue is part of the public API. The adapter decorator class is android.widget.HeaderViewListAdapter. We just need to create our own instance, wrapping our own adapter, and override areAllItemsEnabled as above. There’s a slight complication in that we have to wrap the header view in an instance of ListView.FixedViewInfo and this is a non-static inner class of ListView, but we can reach into our bag of obscure Java tricks to create an instance from outside the enclosing class.

// I assume you know how get a reference to the ListView and create your own adapter.
ListView listView = (ListView) view.findViewById(;
CustomAdapter adapter = new CustomAdapter(context, listOfItems);
// You can create any view you like for the header.
TextView listHeader = (TextView) inflater.inflate(R.layout.list_header, null);
listHeader.setText("My Header Text");
// This is how you create an instance of the non-static nested class, providing
// it with a reference to an instance of the containing class (ListView).
ListView.FixedViewInfo headerInfo = FixedViewInfo();
headerInfo.view = listHeader;
headerInfo.isSelectable = false;
// HeaderViewListAdapter insists on concrete ArrayLists.
ArrayList headers = new ArrayList(1);
ArrayList footers = new ArrayList(0);
HeaderViewListAdapter wrapper = new HeaderViewListAdapter(headers, footers, adapter)
    public boolean areAllItemsEnabled()
        return true;

Google’s Lag Problem – Or Why Android App Development is 22 Months Behind iOS

Posted in Android, iOS by Dan on January 20th, 2013

On 12th October 2011 Apple unveiled iOS 5.0 to the world. 7 days later Google released Android 4.0 (Ice Cream Sandwich).

Apple rarely announces how many devices are running each version of its mobile operating system but, according to third-party stats[1], today only a few percent of iPhone/iPad/iPod Touch owners are still running an earlier version of iOS.

In contrast, Google is much more up-front about how many people are running each version but it doesn’t compare favourably. Just 39.3% of users have access to the 15-month-old Android OS or one of its later revisions.

Of course, there are some mitigating factors. Android is a much more open ecosystem with dozens of manufacturers needing to test and approve updates for hundreds or even thousands of devices. Apple on the other hand retains a strict control over its (smaller) mobile empire and is able to push out updates directly to end users. But whatever the reasons, and despite Google’s recent efforts to rein in manufacturers and network operators, the fact remains that, unlike their iOS counterparts, Android app developers cannot target a recent version of the operating system and hope to have their app runnable by the majority of device owners.

Google has tried to address this issue with its support library, which brings some of the more recent API additions to earlier Android versions but this is only a partial solution that adds complexity and does nothing to resolve the vast visual differences between Android 4.x and its predecessors.

Today if you want to reach over 90% of iOS users you must support iOS 5.1, which was released in March 2012, and later versions. To reach the same proportion of Android users you would have to target Android 2.2 (Froyo), which dates back to May 2010.

The many comparisons of the relative merits of the latest iOS versus the latest Android version are largely irrelevant to Android app developers who are working from a baseline that is almost two years older than that of their iOS counterparts. An iOS developer can build an app that requires the absolute latest version of the operating system confident that the user base will soon be there (iOS 6 has reached 78.5% penetration in four months). Android developers will always need to be more conservative.

1. These stats may not be fully representative of all iOS users but they provide useful ballpark figures.

Uncommons Maths 1.2.3

Posted in Java by Dan on November 22nd, 2012

It’s five years ago this week that I first released Uncommons Maths as a standalone project spun-off from an early version of the Watchmaker Framework. Uncommons Maths is a Java library that provides, among other things, various classes for working with random numbers, probability distributions and combinatorics. There hasn’t been a release in over two years so today I’ve published a maintenance release, version 1.2.3, on GitHub.

There are two main changes in this version. Firstly, thanks to a contribution from Dave LeBlanc, the Uncommons Maths JAR file is now a valid OSGi bundle. Secondly, I found and fixed a pretty fundamental bug in the convertBytesToLong method of BinaryUtils. The fact this bug went undetected for so long suggests that nobody is actually using this method. It also reinforces that full test coverage is worthless if you don’t select your test inputs carefully.

Finally, the documentation has been updated to make it clear that none of the RNG implementations support the seeding mechanism inherited from the java.util.Random base class. The reason for this is that the setSeed method only takes a single long argument. This means it’s not possible to provide more than 64 bits of entropy, which is insufficient for most of the Uncommons Maths RNGs. In practice calling this method on an RNG instance has no effect. Ideally the method would be over-ridden to throw UnsupportedOperationException but to do so is not possible because setSeed is invoked internally by the java.util.Random constructors. If it threw an exception it would not be possible to create an instance of an object that inherits from java.util.Random.

Thoughts on Droidcon London 2012

Posted in Android by Dan on October 27th, 2012

Twelve months on from my first visit to the UK edition of Droidcon, I was back in Islington this week for this year’s version. The format was the same: barcamp on day 1, conference proper on day 2. I’m not sure the distinction between the two days is all that worthwhile. After the initial hour of pitching and organising, the first day proceeds along similar lines to the second, with four tracks of presentations. I’d vote for just having two normal conference days with the programme for both days published in advance, which appears to be how the Dutch Droidcon works. If there is a need to democratise the programme, this could be done online beforehand.

Anyway, while it’s all still fresh in the memory, here are some of the things I thought worth noting. These may or may not be of interest to other Android developers. If you want to watch some of the talks, keep an eye on the Skills Matter website as I suspect the videos will start appearing on there shortly.

Proguard author Eric Lafortune talked about obfuscation and optimisation of Android apps. I’ve been using his Java obfuscator for many years, since well before it became an integral part of the Android tool set, so it was good to speak to him and finally put a face to the name. Eric’s start-up has another Proguard-based obfuscator/optimiser, called DexGuard, with extra Android-specific features, which may be of interest to app developers who need stronger protection and are willing to pay for it.

Erik Hellman made using OpenCV for computer vision on Android look reasonably painless. I’d previously discarded a good app idea that required this kind of functionality as too complicated and too processor-intensive. Might need to revisit that.

David Teitelbaum of Apkudo showed step-by-step how to crack an APK and inject code. He was able to cheat at one of Zynga’s word games by reverse-engineering the obfuscated code. It was an eye-opening demo and may well have caused a few app developers to seriously consider DexGuard as an extra line of defence against this kind of attack.

Jenkins contributors Christopher Orr and Jørgen Tjernø spoke about the plugins that are available for continuous integration on Android projects. If you have the server resources to dedicate to it, you can use Jenkins to automatically build and test your app with an emulator for every combination of Android version, screen size and configuration that you are interested in. A long time ago I wrote why you should use Jenkins for general Java projects, or Hudson as it was then (technically Hudson still exists as a separate Oracle-backed project but the community has moved en masse to the Jenkins fork). Christopher and Jørgen made the case for using it to improve the quality of your Android apps.

James Hugman gave an updated overview of Kirin, a hybrid approach to cross-platform app development that uses native-specific UIs and Javascript for logic. This was one of the more interesting topics from last year but I never got around to actually trying it out. It was interesting to see how the project has progressed. It now seems to be based around Node.js.

Given my interest in artificial intelligence, the session I was most interested in attending was the one called “Integrating AI into your app”. Unfortunately this was a complete waste of time and served only to prevent me from taking an earlier train home. It was nothing more than an advert for some tool for building expert systems. Neither the tool itself nor the generated systems had anything to do with Android. You expect some degree of being advertised at from the fee-paying keynote speakers who are subsidising the event but at least they have the decency not to stray too far from the main theme of the conference.

That’s the type of session I’d like to see fewer of but I’d like to see more like the preceding session by Anders Ericsson. He delivered a live-coding demo of a fairly clean approach to applying animation to custom views. The results were impressive (the approach is also outlined in this blog post) and the presentation followed on nicely from Chiu-Ki Chan‘s earlier talk on developing custom views. A couple of people pointed out that the new animation features in Android 3.0 and later can be used to achieve similar effects but this approach also works on the large number of Android devices that are still running Android 2.x. This session and David Teitelbaum’s APK cracking were probably the two best talks and are the kind of things I’d like to see if I attend next year.

« Older Posts