Another Good Review for Swing Extreme Testing

Our book Swing Extreme Testing has received another good review, this time from Alex Ruiz. Although the review is positive overall ("The book is well-written, interesting and useful."), there are a few criticisms of the book that we think are unwarranted, which we respond to here. Also, the review does not make any mention of a number of important topics covered in our book, that are not covered in any other books, and that will be of great value to Java developers.

What He's Not Sure About

"Although the book points out the importance of accessing UI components in the EDT, I'm not sure their implementation, which involves calling the method invokeAndWait (in javax.swing.SwingUtilities,) is completely correct. We tried a similar approach in FEST, and we ended up with horrible OutOfMemoryErrors."
We routinely run thousands of tests with no such issues arising. Perhaps the FEST approach differs in some subtle but problematic way.

The Test-First Approach

"The authors suggest that is impossible to create software using a test-first approach...."
We are totally in favour of this approach, as far as it can be practically applied. For example, on page 48 we write "... we prefer to write the test for a method before the method itself...". However, we also acknowledge that if we follow the 'test first' dictum slavishly, then at some point we have a test that does not compile, and certainly cannot be run.

The Arrangement of Source and Test Code

"The authors propose to have test classes in the same root folder as the source code, with the same package name with ".test" added at the end..."
This is the way we have arranged our source and test code, but in the book we acknowledge that a lot of people will prefer the 'parallel hierarchies' approach. This doesn't seem like a hugely important issue to us.

Testing Package-Protected Code

"Because of the point mentioned above, the authors suggest using reflection to test package-protected classes."
On the contrary, our policy is not to have explicit tests for package-private and private classes and methods. Our policy is to test all public and protected classes and all of their public and protected methods and constructors. This is already a huge amount of testing and it is certainly sufficient to ensure complete code coverage, as we demonstrate in the book. On the rare occasions we feel we need to test a package-private class, we either use reflection or take the pragmatic step of making it more accessible.
We know that exposing code to make it more testable is deeply offensive to some, and it is always a last resort for us. But, as we said, we almost never need to do it. Of course, the real problem here is that the visibility options available in Java are very coarsely defined, which JSR 294 (Improved Modularity Support in the JavaTM Programming Language) may alleviate.

We Use Our Own Testing Tool

"Instead of using one of the well-known testing frameworks (TestNG or JUnit,) the authors created their own testing framework: Grand Test Auto. I don't understand why."
There are several advantages of GTA.
First, it runs unit, function and load tests, that is the three levels of testing we use in our day-to-day development of LabWizard. At the unit test level, it ensures test coverage, as explained elsewhere on this site and in the book. Our book is not just about unit testing, and we need a tool that can cope well higher levels of tests.
Second, it is extremely flexible and with just a few keystrokes we can run either all of our tests, or just those for some packages, or just those for some classes within a package, or some methods within a class. We have so many tests and run them so frequently that this kind of flexibility is really essential.
Third, with GTA we can distribute the tests across a number of machines.
Fourth, GTA is less intrusive than other tools that we've seen.
Finally, why should we always use the same old tools? Who is sorry that people got sick of make and developed ant?
We do agree that the lack of IDE support for GTA is unfortunate, but this does not really bother us, as we run our tests from the command line or using CruiseControl.
Of the twenty chapters in the book, only one is directly concerned with GTA. It seems odd to criticise the use of this tool and not mention so many of the other chapters (see below).

Design By Contract

"The authors promote 'design by contract' (DbC) using Javadoc comments, because, according to them, using JDK 1.4 assert requires discipline to "check that comments accurately reflect the checks that are being done." I completely disagree. Javadoc comments for DbC do not have IDE support and are not type safe, therefore they can potentially be a maintenance nightmare. In addition, I haven't found a DbC tool that allows me to have the checks and the source code in the same file while keeping type safety."
Assert statements are useful, but are in no way a replacement for a full-powered Design-by-Contract tool. Anyone who thinks otherwise is kidding themselves. In the early days of our project we made extensive use of a tool called iContract, that allowed us to insert preconditions, postconditions and class invariants as executable JavaDoc comments. (We give an example in the book of the kind of really powerful contracts we were able to make using iContract.) This tool really was valuable, but is no longer supported. In the best of all possible worlds, we would still be using something like iContract.
In the absence of true Design-by-Contract support in Java, we agree that the assert keyword is better than nothing.
Actually, since our book was published, we've started making a lot of use of the @NotNull and @Nullable annotations provided by JetBrains. These annotations show up in code-completion and parameter help dialogs displayed by IntelliJ, which makes them even more visible than JavaDoc comments. Maybe these annotations will develop further into a proper DBC tool. We live in hope.

Database management

"What I disagree with is the authors' vision of a unified way to access data, regardless of the underlying storage technology. I have found that the design of the data access layer depends on many factors, such as the database itself and how it is being accessed."
Our LabWizard software can be run using either a relational database (MS Access) or a BTree database (InfinityDB) to persist the KnowledgeBases. All database code is confined to a single package that only has a few really big classes in it. There is no reference to databases in any of the other hundred-odd packages comprising the software. We think that this is an outstanding example of the underlying storage technology being hidden from the rest of the application.

What is not mentioned

The criticisms we've responded to above are in fact pretty slight. Is the arrangement of test and source code really worth getting excited about? No. Our main source of disappointment with the review is that it did not mention that our book deals with:


Overall, we're grateful that Alex Ruiz has a lot of positive things to say about our approach to the testing of Swing user interfaces. However, we're disappointed that he spends so much of his review in what seem to us to be petty criticisms (the layout of test code, not using JUnit, not adhering too slavishly to a test-first approach etc.) and doesn't make mention of a lot of the other material in the book that we think a lot of readers would be very interested in.