Deploying an APK to Multiple Devices/Emulators Simultaneously Using Ant

Posted in Android, Ant by Dan on January 4th, 2014

Following my detour to Djangoland, the last week or so I have been back in the world of mobile app development. One of the things I’ve been working on is updating an Android app that has two versions built from the same codebase. Part of this update has been to re-skin both versions of the app and to make sure that everything looks reasonable on various devices running versions of Android from 2.2 up to the latest 4.4. When I build a new APK I want to check it on each of the three devices connected to my machine. If I had more spare USB ports I would conceivably be testing with even more devices. In some scenarios I might also have one or more emulators running to test configurations not represented among my devices. Installing the new APK on each of these from the command line is cumbersome. The adb tool will only install on one device/emulator at a time and, if you have more than one connected, you have to specify a long unique device ID to tell it which one to use. You could write a bash script to retrieve the IDs of connected devices and emulators and to run the adb install command for each, but since I’m using Ant I have written a custom task to do the job. For bonus points I’ve made it issue the adb commands in parallel so that the total install time is determined by the slowest device and not the sum of all the devices.

import java.io.BufferedReader;
import java.io.File;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.PrintStream;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
import java.util.concurrent.TimeUnit;
import org.apache.tools.ant.BuildException;
import org.apache.tools.ant.Task;
 
/**
 * Custom task that uses the ADB tool to install a specified APK on all
 * connected devices and emulators.
 * @author Daniel Dyer
 */
public class InstallAPK extends Task
{
    private File apkFile;
 
    public void setAPK(File apkFile)
    {
        this.apkFile = apkFile;
    }
 
    @Override
    public void execute() throws BuildException
    {
        if (apkFile == null)
        {
            throw new BuildException("APK file must be specified");
        }
        try
        {
            List<String> devices = getDeviceIdentifiers();
            System.out.printf("Installing %s on %d device(s)...%n", apkFile, devices.size());
            ExecutorService executor = Executors.newFixedThreadPool(devices.size());
            List<Future<Void>> futures = new ArrayList<Future<Void>>(devices.size());
            for (final String device : devices)
            {
                futures.add(executor.submit(new Callable<Void>()
                {
                    public Void call() throws IOException, InterruptedException
                    {
                        installOnDevice(device);
                        return null;
                    }
                }));
            }
            for (Future<Void> future : futures)
            {
                future.get();
            }
            executor.shutdown();
            executor.awaitTermination(60, TimeUnit.SECONDS);
        }
        catch (Exception ex)
        {
            throw new BuildException(ex);
        }
    }
 
    private void installOnDevice(String device) throws IOException, InterruptedException
    {
        String[] command = new String[]{"adb", "-s", device, "install", "-r", apkFile.toString()}
        Process process = Runtime.getRuntime().exec(command);
        consumeStream(process.getInputStream(), System.out, device);
        if (process.waitFor() != 0)
        {
            consumeStream(process.getErrorStream(), System.err, device);
            throw new BuildException(String.format("Installing APK on %s failed.", device));
        }
    }
 
    private void consumeStream(InputStream in, PrintStream out, String tag) throws IOException
    {
        BufferedReader reader = new BufferedReader(new InputStreamReader(in));
        try
        {
            for (String line = reader.readLine(); line != null; line = reader.readLine())
            {
                out.println(tag != null ? String.format("[%s] %s", tag, line.trim()) : line);
            }
        }
        finally
        {
            reader.close();
        }
    }
 
    private List<String> getDeviceIdentifiers() throws IOException, InterruptedException
    {
        Process process = Runtime.getRuntime().exec("adb devices");
        List devices = new ArrayList<String>(10);
        BufferedReader reader = new BufferedReader(new InputStreamReader(process.getInputStream()));
        try
        {
            for (String line = reader.readLine(); line != null; line = reader.readLine())
            {
                if (line.endsWith("device"))
                {
                    devices.add(line.split("s")[0]);
                }
            }
            if (process.waitFor() != 0)
            {
                consumeStream(process.getErrorStream(), System.err, null);
                throw new BuildException("Failed getting list of connected devices/emulators.");
            }
        }
        finally
        {
            reader.close();
        }
        return devices;
    }
}

Once compiled and on your classpath, using it is simple:

<taskdef name="installapk" classname="yourpackage.InstallAPK" />
<installapk apk="path/to/yourapp.apk" />

Django ModelChoiceField and HTML <optgroup>

Posted in Python by Dan on December 21st, 2013

I’ve been dabbling with Django over the last couple of months. I’ve played around with it a couple of times previously but this time I’ve actually built something reasonably substantial, which has meant that I’ve had to delve a bit deeper. One of the minor problems I solved today was how to group items in the HTML <select> element generated by a form’s ModelChoiceField. HTML has the <optgroup> tag for this purpose.

It’s not immediately obvious how you can get ModelChoiceField to use optgroups without over-riding the render method and re-implementing the HTML generation yourself, or bypassing ModelChoiceField completely and building something based on the basic ChoiceField (which does support optgroups). The only potential solutions I found from searching took the former approach (here and here). The reason I’m writing this post is because I think I’ve found a better, more concise solution that might be of use to future searchers.

ChoiceField accepts a choices parameter to its constructor. In the simple case this is just a list of items (each item is value/label pair). However, it can also accept a list of groups where each group is a tuple consisting of the group label and a list of items. The problem is that ModelChoiceField is different in that it has a queryset parameter instead, so there is no way to pass in the group information.

However, a comment in the source code says that we can set a property called choices after constructing the ModelChoiceField instance and the queryset will be ignored. The HTML <select> will instead be populated from this data structure with <optgroup> elements as required.

Assuming we want to group items by a field on the model, we can build the list of tuples from the queryset by sub-classing ModelChoiceField and over-riding the constructor. In this example I’m assuming that the field is a list of countries grouped by continent, where the continent is just a text field on the country model.

from django.forms import ModelChoiceField
from itertools import groupby
from operator import attrgetter
 
class CountryChoiceField(ModelChoiceField):
    def __init__(self, *args, **kwargs):
        super(CountryChoiceField, self).__init__(*args, **kwargs)
        groups = groupby(kwargs['queryset'], attrgetter('continent'))
        self.choices = [(continent, [(c.id, self.label_from_instance(c)) for c in countries])
                        for continent, countries in groups]

In order for this to work, the queryset must be sorted by the field that it is to be grouped by so that items in the same group are adjacent. You can do this before you pass it to the constructor or you can change the code above to call order_by on the queryset.

ReportNG 1.1.4

Posted in Java by Dan on June 11th, 2013

It’s been a couple of years since the last release of ReportNG, my cobbled together alternative reporting plug-in for TestNG. In the time since I’ve done nothing but a few users have submitted useful improvements. So, after some prompting, I’ve released version 1.1.4 (download zip, tgz).

Thanks to Kayla Nimis, criccio, Arcadie, and Nalin Makar the new version has the following improvements:

  • The report now shows the reason for skipping a test.
  • You can specify the org.uncommons.reportng.failures-only property to generate a more minimal report.
  • ReportNG will now create any missing parent directories of the report directory rather than failing if they are absent.

Google Skynet

Posted in The Internet by Dan on May 16th, 2013

Google policy is to get right up to the creepy line and not cross it.

These are the words of former Google CEO Eric Schmidt, speaking to The Atlantic in October 2010.

We don’t need you to type at all. We know where you are. We know where you’ve been. We can more or less know what you’re thinking about.

This was two and a half years ago. Today Google is so far over the creepy line it can’t even see the line any more. The problem is not so much individual Google products but the way in which the vast data from all these disparate services is combined to build a very detailed profile of you. Not just what you do on Google sites but also any other site you visit that includes Google’s +1, Adsense or Analytics JavaScript (including this one). Google can read every e-mail you send and receive, it knows everything you search for online and knows pretty much every website you ever visit. It knows where you are, where you’ve been and probably with whom. Google knows more about you than your mother does and with Google+ it’s all neatly connected to your real identity.

I’ve so far avoided signing up for Google+ but it’s increasingly difficult to ignore, particularly as an Android developer, as it’s becoming more tightly integrated into everything that Google does. It’s the keystone of Google’s anti-privacy agenda. When the time comes for the last remnants of the human race to send an Austrian-accented cyborg back in time to thwart the rise of the machines, its first targets will be the architects of Google+.

The privacy implications are not the only issue. Most users perceive Google’s search results to represent some sort of objective truth, with links ranked only by their relevance to the search query, but in fact each user is served personalised results. If you and I both search for information on some contentious political topic, Google won’t necessarily give us the same response; it will show us each what it thinks we want to see based on its profiling of us.

In his Google I/O Q&A yesterday, Google CEO Larry Page dismissed concerns over the implications of this profiling:

[Audience member]: Most of my opinion, I can trace back to a Google search. As search becomes more and more personalized, and predictive, I worry that it informs my world view and rules out the possibility of some other serendipitous discovery. Any comment on that?

[Page]: People have a lot of concern about that – I’m totally not worried about that at all.

Personally, I don’t like Google’s all-encompassing vision. It’s not the only company that employs such methods but I can easily ignore the likes of Facebook and Bing as I don’t use them. Google is everywhere.

Spurred more by the closure of Google Reader than anything else, a couple of months ago I began to consider alternatives to relying on the benevolence of one omniscient company for the services I use every day.

The first thing to go was GMail. E-mail is important enough to be worth paying for so I signed-up with FastMail (now owned by Opera), which offers an ad-free service from $4.95 per year. It has a refined web interface, IMAP access, and a simple way to import your existing messages from GMail.

I eventually replaced Google Reader with The Old Reader and for search I’m now using DuckDuckGo, which promises not to track its users or filter search results. DuckDuckGo doesn’t quite match Google in terms of the freshness or depth of its results but it’s fast and has some useful features. On DuckDuckGo’s recommendation I also installed Ghostery for Opera to block Google and others’ attempts to track me on third-party sites. I’ve removed the Google +1 buttons from this blog and will be looking for more privacy-friendly alternatives to Google Analytics and Adsense (both of which would be useless anyway if everybody is driven to use the likes of Ghostery).

I’m not giving up on Google entirely, I just prefer to keep it at arm’s length. I’ll still be developing for Android (for which there were many very welcome developer announcements at I/O yesterday) and no doubt I’ll continue to use some of its other services.

Android ListView – Fixing Missing/Blank Dividers

Posted in Android, Java by Dan on January 24th, 2013

A post with code in it. Because it’s been a long time since the last one.

If you work with ListViews in Android, as all Android developers will do at some point, you may notice that if you set your list items to be non-selectable the dividers that are drawn between each cell disappear. In the Holo Light theme you would normally get a thin light grey dividing line between cells. Depending on how you’ve implemented your adapter, you may find that these dividers become white/transparent when the cells are not selectable. According to Android framework engineer Romain Guy, this is the intended behaviour.

As Haythem Souissi points out in this Stack Overflow answer, you can work around this by ensuring that the areAllItemsEnabled method returns true, even though all items are not enabled (maybe none of them are). The isEnabled method will take care of actually disabling the cells and the dividers will be drawn between each of them.

All very straightforward so far but it all goes wrong again when you try to add a non-selectable header view to the list. The way that ListView deals with headers and footers is that it creates its own adapter to wrap/decorate yours and insert the header/footer views in the appropriate places. This is fine but it’s not delegating the areAllItemsEnabled method, so our above fix no longer works.

Fortunately, and unusually for Android, everything we need to resolve this issue is part of the public API. The adapter decorator class is android.widget.HeaderViewListAdapter. We just need to create our own instance, wrapping our own adapter, and override areAllItemsEnabled as above. There’s a slight complication in that we have to wrap the header view in an instance of ListView.FixedViewInfo and this is a non-static inner class of ListView, but we can reach into our bag of obscure Java tricks to create an instance from outside the enclosing class.

// I assume you know how get a reference to the ListView and create your own adapter.
ListView listView = (ListView) view.findViewById(android.R.id.list);
CustomAdapter adapter = new CustomAdapter(context, listOfItems);
 
// You can create any view you like for the header.
TextView listHeader = (TextView) inflater.inflate(R.layout.list_header, null);
listHeader.setText("My Header Text");
 
// This is how you create an instance of the non-static nested class, providing
// it with a reference to an instance of the containing class (ListView).
ListView.FixedViewInfo headerInfo = listView.new FixedViewInfo();
 
headerInfo.view = listHeader;
headerInfo.isSelectable = false;
 
// HeaderViewListAdapter insists on concrete ArrayLists.
ArrayList headers = new ArrayList(1);
headerInfoList.add(headerInfo);
ArrayList footers = new ArrayList(0);
 
HeaderViewListAdapter wrapper = new HeaderViewListAdapter(headers, footers, adapter)
{
    @Override
    public boolean areAllItemsEnabled()
    {
        return true;
    }
};
listView.setAdapter(wrapper);

Google’s Lag Problem – Or Why Android App Development is 22 Months Behind iOS

Posted in Android, iOS by Dan on January 20th, 2013

On 12th October 2011 Apple unveiled iOS 5.0 to the world. 7 days later Google released Android 4.0 (Ice Cream Sandwich).

Apple rarely announces how many devices are running each version of its mobile operating system but, according to third-party stats[1], today only a few percent of iPhone/iPad/iPod Touch owners are still running an earlier version of iOS.

In contrast, Google is much more up-front about how many people are running each version but it doesn’t compare favourably. Just 39.3% of users have access to the 15-month-old Android OS or one of its later revisions.

Of course, there are some mitigating factors. Android is a much more open ecosystem with dozens of manufacturers needing to test and approve updates for hundreds or even thousands of devices. Apple on the other hand retains a strict control over its (smaller) mobile empire and is able to push out updates directly to end users. But whatever the reasons, and despite Google’s recent efforts to rein in manufacturers and network operators, the fact remains that, unlike their iOS counterparts, Android app developers cannot target a recent version of the operating system and hope to have their app runnable by the majority of device owners.

Google has tried to address this issue with its support library, which brings some of the more recent API additions to earlier Android versions but this is only a partial solution that adds complexity and does nothing to resolve the vast visual differences between Android 4.x and its predecessors.

Today if you want to reach over 90% of iOS users you must support iOS 5.1, which was released in March 2012, and later versions. To reach the same proportion of Android users you would have to target Android 2.2 (Froyo), which dates back to May 2010.

The many comparisons of the relative merits of the latest iOS versus the latest Android version are largely irrelevant to Android app developers who are working from a baseline that is almost two years older than that of their iOS counterparts. An iOS developer can build an app that requires the absolute latest version of the operating system confident that the user base will soon be there (iOS 6 has reached 78.5% penetration in four months). Android developers will always need to be more conservative.

1. These stats may not be fully representative of all iOS users but they provide useful ballpark figures.

Uncommons Maths 1.2.3

Posted in Java by Dan on November 22nd, 2012

It’s five years ago this week that I first released Uncommons Maths as a standalone project spun-off from an early version of the Watchmaker Framework. Uncommons Maths is a Java library that provides, among other things, various classes for working with random numbers, probability distributions and combinatorics. There hasn’t been a release in over two years so today I’ve published a maintenance release, version 1.2.3, on GitHub.

There are two main changes in this version. Firstly, thanks to a contribution from Dave LeBlanc, the Uncommons Maths JAR file is now a valid OSGi bundle. Secondly, I found and fixed a pretty fundamental bug in the convertBytesToLong method of BinaryUtils. The fact this bug went undetected for so long suggests that nobody is actually using this method. It also reinforces that full test coverage is worthless if you don’t select your test inputs carefully.

Finally, the documentation has been updated to make it clear that none of the RNG implementations support the seeding mechanism inherited from the java.util.Random base class. The reason for this is that the setSeed method only takes a single long argument. This means it’s not possible to provide more than 64 bits of entropy, which is insufficient for most of the Uncommons Maths RNGs. In practice calling this method on an RNG instance has no effect. Ideally the method would be over-ridden to throw UnsupportedOperationException but to do so is not possible because setSeed is invoked internally by the java.util.Random constructors. If it threw an exception it would not be possible to create an instance of an object that inherits from java.util.Random.

Thoughts on Droidcon London 2012

Posted in Android by Dan on October 27th, 2012

Twelve months on from my first visit to the UK edition of Droidcon, I was back in Islington this week for this year’s version. The format was the same: barcamp on day 1, conference proper on day 2. I’m not sure the distinction between the two days is all that worthwhile. After the initial hour of pitching and organising, the first day proceeds along similar lines to the second, with four tracks of presentations. I’d vote for just having two normal conference days with the programme for both days published in advance, which appears to be how the Dutch Droidcon works. If there is a need to democratise the programme, this could be done online beforehand.

Anyway, while it’s all still fresh in the memory, here are some of the things I thought worth noting. These may or may not be of interest to other Android developers. If you want to watch some of the talks, keep an eye on the Skills Matter website as I suspect the videos will start appearing on there shortly.

Proguard author Eric Lafortune talked about obfuscation and optimisation of Android apps. I’ve been using his Java obfuscator for many years, since well before it became an integral part of the Android tool set, so it was good to speak to him and finally put a face to the name. Eric’s start-up has another Proguard-based obfuscator/optimiser, called DexGuard, with extra Android-specific features, which may be of interest to app developers who need stronger protection and are willing to pay for it.

Erik Hellman made using OpenCV for computer vision on Android look reasonably painless. I’d previously discarded a good app idea that required this kind of functionality as too complicated and too processor-intensive. Might need to revisit that.

David Teitelbaum of Apkudo showed step-by-step how to crack an APK and inject code. He was able to cheat at one of Zynga’s word games by reverse-engineering the obfuscated code. It was an eye-opening demo and may well have caused a few app developers to seriously consider DexGuard as an extra line of defence against this kind of attack.

Jenkins contributors Christopher Orr and Jørgen Tjernø spoke about the plugins that are available for continuous integration on Android projects. If you have the server resources to dedicate to it, you can use Jenkins to automatically build and test your app with an emulator for every combination of Android version, screen size and configuration that you are interested in. A long time ago I wrote why you should use Jenkins for general Java projects, or Hudson as it was then (technically Hudson still exists as a separate Oracle-backed project but the community has moved en masse to the Jenkins fork). Christopher and Jørgen made the case for using it to improve the quality of your Android apps.

James Hugman gave an updated overview of Kirin, a hybrid approach to cross-platform app development that uses native-specific UIs and Javascript for logic. This was one of the more interesting topics from last year but I never got around to actually trying it out. It was interesting to see how the project has progressed. It now seems to be based around Node.js.

Given my interest in artificial intelligence, the session I was most interested in attending was the one called “Integrating AI into your app”. Unfortunately this was a complete waste of time and served only to prevent me from taking an earlier train home. It was nothing more than an advert for some tool for building expert systems. Neither the tool itself nor the generated systems had anything to do with Android. You expect some degree of being advertised at from the fee-paying keynote speakers who are subsidising the event but at least they have the decency not to stray too far from the main theme of the conference.

That’s the type of session I’d like to see fewer of but I’d like to see more like the preceding session by Anders Ericsson. He delivered a live-coding demo of a fairly clean approach to applying animation to custom views. The results were impressive (the approach is also outlined in this blog post) and the presentation followed on nicely from Chiu-Ki Chan‘s earlier talk on developing custom views. A couple of people pointed out that the new animation features in Android 3.0 and later can be used to achieve similar effects but this approach also works on the large number of Android devices that are still running Android 2.x. This session and David Teitelbaum’s APK cracking were probably the two best talks and are the kind of things I’d like to see if I attend next year.

Using an O2 Mobile Broadband Dongle SIM in an Android Tablet

Posted in Android by Dan on October 24th, 2012

O2 mobile broadband APN settingsA couple of months ago I purchased an O2 Mobile Broadband USB dongle so that I could continue working during an ADSL outage. Once the ADSL was back on I had no further use for it. Seeing as I also had a 3G-enabled Samsung Galaxy Tab 10.1 without a SIM, I thought it would make sense to use the remaining data allowance on that. However I was initially unable to find the right settings to make the tablet connect to O2′s network.

Having given it another shot before the unused data expires, I have eventually stumbled upon the correct settings, which I’ll document here for anyone who might need to achieve the same. Previously I was trying the device’s default O2 access point configurations and the APN settings I found on the web. The problem with these is that they are for pay-monthly and pay-as-you-go SIMs sold specifically for use in phones and tablets. The settings for the mobile broadband SIMs are different. You need to use the m-bb.o2.co.uk APN rather than payandgo.o2.co.uk or mobile.o2.co.uk. The full working settings from my Galaxy Tab are shown to the right (the obscured password is, predictably, ‘password’).

Update (30/1/2013): After the remaining data had expired, I didn’t use the 3G functionality of the tablet for a while, during which time I updated the tablet to Android 4.0.4. For whatever reason, the settings shown here no longer worked after adding more credit to the SIM. Eventually I discovered that the “APN type” field had to be set to “default” rather than “internet”. I also switched to using the “o2bb” username that the OS X connection manager software uses with the dongle.

Carmack on Static Analysis and Functional Purity

Posted in Haskell, Software Development by Dan on October 16th, 2012

In the absence of anything worthwhile of my own to write here at the moment, I thought I’d instead highlight a couple of interesting blog posts that I’ve discovered recently by id Software’s John Carmack.

I read Carmack’s thoughts on static analysis a few weeks ago and his reports on the effectiveness of tools for analysis of C++ code chimed with my own experiences with Java tools such as FindBugs and IntelliJ IDEA.

The first step is fully admitting that the code you write is riddled with errors. That is a bitter pill to swallow for a lot of people, but without it, most suggestions for change will be viewed with irritation or outright hostility. You have to want criticism of your code.

Automation is necessary. It is common to take a sort of smug satisfaction in reports of colossal failures of automatic systems, but for every failure of automation, the failures of humans are legion. Exhortations to “write better code” plans for more code reviews, pair programming, and so on just don’t cut it, especially in an environment with dozens of programmers under a lot of time pressure. The value in catching even the small subset of errors that are tractable to static analysis every single time is huge.

In the other article, which I only read today, Carmack espouses the virtues of pure (i.e. side-effect-free) functions. His is a pragmatic approach concerned with how to exploit purity in mainstream languages, where it is entirely optional, as opposed to advocating jumping ship to Haskell. Even when 100% purity is impractical there are still benefits in minimising impurity.

A large fraction of the flaws in software development are due to programmers not fully understanding all the possible states their code may execute in. In a multithreaded environment, the lack of understanding and the resulting problems are greatly amplified, almost to the point of panic if you are paying attention. Programming in a functional style makes the state presented to your code explicit, which makes it much easier to reason about, and, in a completely pure system, makes thread race conditions impossible.

My experiences with Haskell have informed how I approach coding in other languages. In certain cases I’ve ended up taking the functional approach to such extremes that I’ve been left questioning my original choice of implementation language.

No matter what language you work in, programming in a functional style provides benefits. You should do it whenever it is convenient, and you should think hard about the decision when it isn’t convenient.

« Older Posts