Code Coverage

Owned by Former user (Deleted)
Last updated: Jul 04, 2006Version comment
2 min read

Prototype - Code Coverage

Introduction

Code coverage allows you to see with X ray precision which parts of the code are is not being used during a given run. The technique is helpful for answering a number of questions associated with the primary question: Which parts of my code are not being exercised? For example:

  • Which extra Functional Tests are needed to exercise the majority of the code base?
  • Which code needs to be removed as Junk DNA?
  • Roughly which code is called when a given bug occurs?

The Open Source tool EMMA can enable code coverage functionality for SAKAI without modifying code. Emma achieves this via looking at the byte code of the Classes and adding its own intercepts. This slows the performance of the system by about 15% and causes an extra bloat of 10 MB to the Sakai-Demo. Thus code coverage is easily possible in a test environment.

The first link below points to a test page. Saka-Demo has been started and then stopped. You can clearly see that the static member logger is instanced and the init and destroy functions have been called. The second link is to an index of all things measured during the start and stop event.

NOTE: For the majority of the links the Java source cannot be found. This is caused by my rush to get this out to the community and not a bug in the tool.

Show us the money
Show us the money

Report making Recipe

You will need to download the following two archives and expand them to locations of convenience:

This archive contains class information as generated by EMMA
http://mercator.ic.uva.nl/sakai_review/coverage/example/output.tar.gz
An Emma enabled version of the Sakai-Demo 2.006
http://mercator.ic.uva.nl/sakai_review/coverage/example/sakai-demo.tar.gz
Further you will need the emma.jar file
The emma jar file

Enabling Emma and generating a report.

(1) Add the lib/emma.jar file to the extension directory of the Java that you are running on your test system. E.G: copy to $JAVA_HOME/jre/lib/ext/emma.jar
(2) Run your Sakai-Demo and run your tests.
(3) Stop the Sakai-Demo. After a few seconds in the Sakai root you should see the file coverage.ec. You now have the raw data to generate reports.
(4) Run emma in report mode. For example:

Assuming that:
html report directory is /home/alan/temp/output/x/tool
Location of source code for your subproject is /home/alan/src/sakai_2-2-006/portal/portal-tool/tool/src/java
Location of your Sakai demo is /home/alan/sakai-demo/
Location of the tool.em from the unpacked output.tar.gz file is /home/alan/temp/output/

mkdir /home/alan/temp/output/x/tool
java emma report -r html -sp /home/alan/src/sakai_2-2-006/portal/portal-tool/tool/src/java -in /home/alan/temp/output/tool.em -in /home/alan/sakai-demo/coverage.ec -Dreport.html.out.file=/home/alan/temp/output/006/tool/tool.html

How to emma enable your code base

Assuming that the directory that you wish to place you symbols in is /home/alan/temp/output_007
and the demo directory for your class files is /home/alan/sakai-demo/webapps/sakai-gradebook-tool/WEB-INF

then the java command next will instrument the .class files ready for report generation later.
For each subproject the instrumentation process takes a few hundred milliseconds.

echo Instrumenting sakai-gradebook-tool
cd /home/alan/sakai-demo/webapps/sakai-gradebook-tool/WEB-INF
java emma instr -ix - -ip ./classes -m overwrite -out=/home/alan/temp/output_007/sakai-gradebook-tool.em

NOTE: I have written a rough Perl script that generates about 100 of the above mentioned statements to cover the whole of the Sakai (or whatever) code bases.

Limitations

  • Only covers WEB-INF/classes files
  • Only dumps information when Tomcat is shutdown.

Note: The tool is only at a prototype level and I have not given myself the time to find solutions. Just want to get this out to the community on time.