TestabilityExplorer.org records the testability scores for many open source and commercial Java libraries.
The compiled bytecode for the library is analyzed and metrics are for the testability of individual classes. Those classes fall into one of three categories - 'excellent', 'good' and 'needs work'. Generally speaking, injectability, mockabiliy and composition are good, and static state is bad. Figures are recursively calculated, but only inside the jar in question.
The metrics are a calculation of the skill of the development team in making their classes testable. You cannot use these metrics to say that Tomcat is better than Jetty or vice versa, as the features of each are not taken into account. These metrics will also not tell you whether a particular library will be easy to use or not. It just tells you how dedicated the development team was to making testable software. As we track the changing figures overtime, we can see whether the team in question was dedicated to improvement or not.
Project Metrics at a Glance
The graph below represents all of the projects indexed by testability explorer web site. The "x" contains the size of the project (in JAR bytes / number of classes) and the "y" axis contains the testability-cost. The size of the bubble is proportional to the size of the project. All bubbles of the same color are different versions of the same project. (The testable projects are low on the "y" axis) If you want to compare two projects use the filter field. (For example to compare the open source databases use: "mckoi; hsql")
Jetty: a case study
Jetty is a highly regarded servlet container. In September of 2006, the team released 6.0.1. In April of 2008 they released 6.1.9. In between there were some ten numbered releases. The 6.1.9 release is more testable than the 6.0.1 release. Versions in between trended towards that higher level. From this we can suggest that the development team worked on testability as well as adding features and fixing bugs.
It is not all roses though. One individual class 'org.mortbay.xml.XmlConfiguration' became slightly harder to test over the same period. The general trend was towards better though.
Check back often; we will add more projects soon! In the meantime we invite you to join us for a discussion at or email us: .
Post Comment
Questions & Comments: Contact
Metrics generated using
Subscribe to hear about updates: or email
