Functional testing an Android application

By Tim Lavers

(Test. Crash? Code.)+
That's my preferred workflow as a programmer. So when I started writing a simple Android application my first task was to get this productive cycle happening. In particular, I wanted to get multi-screen functional tests running, one after the other, in the Android emulator. This article explains how such tests can be written. We will also look at the "ui proxy" pattern that allows functional tests to be written in a very fluid programming style. Finally, we will discuss the Robotium toolkit, which is great for simulating user actions with test code.

The IndoFlash App

The application that I've been writing is a flashcard application for learning Indonesian. It presents words to the user, who tries to remember their meaning. If the user has trouble with a word they add it to their favourites list. There are buttons for showing words and their definitions, adding words to the favourites list, shuffling the list, and for showing word pairs with either Indonesian or English first:
There are three screens, or Activities: a WordListDisplay for showing the words, a WordListSelecter for choosing a list of words to work on, and a ChapterSelecter for choosing a collection of word lists. The underlying Android application is an instance of a class called IndoFlash. This class manages the state of the system, such as which list is selected and what is in the list of favourites.

Fluent Functional Tests

To test IndoFlash I wanted to write tests that operate the application as a user would: pressing buttons, scrolling lists, and moving between different screens. I also wanted to test that navigational information and user choices were preserved between application invocations. For example, if a user chooses to view the lists as shuffled, then if the application is closed and re-started, the lists should be presented as shuffled.
The Android platform supplies an API for reading the state of screens and for injecting user events such as button presses and scroll gestures. This library is quite low level and one of my aims was to convert this into a much more fluid style of programming in which method names very clearly state what the method does. This has worked well and the tests are largely self-documenting. For example, here is a test that checks that a word can be added to the list of favourites:
ui.checkThatCurrentWordIs("you");//Sanity check. ui.addCurrentWordAsFavourite(); ui.showDefinitionOfCurrentWord(); ui.checkThatCurrentTranslationIs("anda"); ui.showNextWord(); ui.checkThatCurrentWordIs("what"); openFavouritesList(); ui.checkThatCurrentWordIs("you");//It was previously showing "what". ui.checkThatCurrentTranslationIsEmpty();//It was previously showing "anda". ui.showDefinitionOfCurrentWord(); ui.checkThatCurrentTranslationIs("anda");//Check that the definition is carried along with a word.

UI Proxy Classes

In the code shown above, the object ui is a "test proxy" for the WordListDisplay, which is the main application Activity. The class of ui is UIWordListDisplay. Every user operation of the WordListDisplay is matched by a method in this class.
The call openFavouritesList(); midway through the test is an example of a multi-screen operation. It is implemented as follows:
protected void openFavouritesList() { UIWordListSelecter uiWordListSelecter = ui.showWordLists(); ui = uiWordListSelecter.selectFavourites(); }
We see here that there is also a test proxy class for WordListSelecter, which is the Activity that is used to switch between different word lists. There is also a ui proxy, UIChapterSelecter, for the chapter selection activity.
It is the use of ui proxy classes that allows the fluent programming style in writing tests. This pattern can be used for pretty much any user interface platform. In the book Swing Extreme Testing it is used to test desktop applications.
The user actions that are performed in the function tests, such as clicking a button or scrolling, make use of the TouchUtils class of the android.test package. For example, to select the favourites list, we need to do some scrolling:
public UIWordListDisplay selectFavourites() { TouchUtils.scrollToBottom(testCase(), activity(), listView()); return selectList(listView().getChildCount() - 1); }

Android Functional Tests

Android tests are applications that exercise the main application within an emulated Android device. The use of the emulator provides a lot of advantages. For example, we can see our app being exercised by the tests and we can easily try out our app on a variety of different devices. If the tests pass then we have very high degree of certainty that the app will work on a physical device. The downside of testing in an emulator is that the tests are very slow to run. For example, it takes about seven minutes for me to build the software, install it on the emulator and run a suite of 20 or so tests.
Android function tests extend android.test.ActivityInstrumentationTestCase2. This class provides access to an Instrumentation instance, which manages the interactions between the test harness and the app that is being tested. These interactions are controlled by the use of Instrumentation.ActivityMonitor and Intent objects. I found this part of the Android testing API hard to use because if methods are called in the wrong order or in the wrong context, they can simply hang. The documentation doesn't give much information about how things work, and the sheer slowness of running tests on an emulator makes it hard to discover more about the API through experimentation.
However, by trial and error and Google, I came across this sequence of steps to launch an Activity:
//Add a Monitor for the Activity to the Instrumentation instance. //Create an Intent to start the Activity. //Call Instrumentation.startActivity with the Intent. //Wait for the Activity to be launched, using the Monitor.
Within the tests for IndoFlash this is implemented as the launch() method of UIWordListDisplay:
public void launch() { Instrumentation.ActivityMonitor monitor = instrumentation().addMonitor(WordListDisplay.class.getName(), null, false); setMonitor(monitor); Intent intent = new Intent(Intent.ACTION_MAIN); intent.setFlags(Intent.FLAG_ACTIVITY_NEW_TASK); intent.setClassName(instrumentation().getTargetContext(), WordListDisplay.class.getName()); instrumentation().startActivitySync(intent); Activity activity = instrumentation().waitForMonitorWithTimeout(monitor(), 5); setActivity(activity); }
Each of the function tests used in the development of IndoFlash extends from FunctionTest, which provides this basic test structure:
//Launch the main activity, which is WordListDisplay //Run the specific test, which is the doIt() method of subclasses. //Get the application back to a known state.
Specifically, this is implemented as follows:
public void test() { //Launch the main activity, which is WordListDisplay. ui = new UIWordListDisplay(this); ui.launch(); //Run the actual test. doIt(); //Get the application back to a known state. IndoFlash application = (IndoFlash) ui.activity().getApplication(); ChapterSpec chapter1Spec = application.applicationSpec().chapterSpecs().get(0); application.setCurrentChapter(chapter1Spec); application.setWordList(chapter1Spec.wordLists().get(0)); ui.finish(); application.clearFavourites(); if (application.shuffle()) { application.toggleShuffle(); } if (application.showIndonesianFirst()) { application.toggleShowIndonesianFirst(); } //Cleanup. pause(5); }
Most of the code is in getting the application, which is an instance of IndoFlash, back to its initial state: showing the first word list in the first chapter, with the English translation first and the word list unshuffled. The pause at the very end of the test is annoying, but without it the tests are flaky.
So far there are 16 function tests and the entire build-test cycle takes almost 7 minutes on a fairly fast laptop. Of course, 80 seconds of that 7 minutes is caused by the 5 second pause at the end of each test - an example of how evil these pauses are.

Robotium

In a code snippet earlier we saw how TouchUtils can be used to perform simple actions such as pressing buttons and scrolling through lists. To write tests that involve more complex actions it is best to make use of the Robotium tool. Robotium has a rich API for simulating user actions and also has very useful methods for investigating the state of the application.
Let's see an example of a Robotium test. One of the requirements of IndoFlash is that the action bar always shows a help button and an information button. When these are pressed, message windows should show that display the appropriate information. For example:
UIActivity has a solo() method that returns a Robotium Solo object, which is the workhorse of the Robotium API. This can be used to check that the help button shows the correct information:
ui.solo().clickOnActionBarItem(R.id.action_help); ui.solo().waitForDialogToOpen(); Assert.assertTrue(checkThatAlertDialogShows("How to use IndoFlash", "Press the Show button to see")); ui.solo().clickOnButton("OK"); ui.solo().waitForDialogToClose();
The trickiest part of this test is the method checkThatAlertDialogShows. This is why. The IndoFlash code that creates this dialog uses the AlertDialog.Builder class:
Our difficulty is that in the resulting View object, we don't know the id of either of the important TextView instances. However, with the help of Robotium, we can track them down:
boolean checkThatAlertDialogShows(@NotNull CharSequence title, @NotNull CharSequence messageStart) { boolean titleFound = false; boolean messageFound = false; ArrayList<View> currentViews = ui.solo().getCurrentViews(); for (View view : currentViews) { if (view instanceof TextView) { CharSequence text = ((TextView) view).getText(); if (title.equals(text)) { titleFound = true; } else if (isPrefixFor(messageStart, text)) { messageFound = true; } } } return titleFound && messageFound; } private boolean isPrefixFor(CharSequence pref, CharSequence text) { if (pref.length() > text.length()) return false; CharSequence sub = text.subSequence(0, pref.length()); return sub.equals(pref); }
In a more complex application, in which there were more tests involving ActionDialog, it might be worth refactoring this kind of code into a UIActionDialog class.

Summary

This article has presented code for running suites multi-screen functional tests of an Android application. The tests are written in a very fluid style, thanks to the use of the ui proxy design pattern. By harnessing the power of Robotium, it is possible to write tests that are even more expressive and powerful. The source code for IndoFlash and its tests are an excellent starting point for anyone implementing multi-screen functional tests for their Android applications.

Resources

The code for IndoFlash and its tests can be downloaded from BitBucket.
Robotium is available from Google code.
The book Android in Practice contains a good overview of Android testing.
This excellent article contained some code snippets that helped me get started with multi-screen tests.