Ant and multiple JUnit test cases

In this Ant tutorial installment, we'll use the JUnit task to execute multiple test classes. Our starting point will be the trivial JUnit task presented previously. This example was developed using version 1.5.1 of Ant and version 3.8.1 of JUnit.
For simplicity, we'll add the additional JUnit test class shown below. This class uses the assertEquals method with arguments that will always cause an AssertionFailedError. Please place this file in a sub-directory named test as before.

import junit.framework.*;
public class Test extends TestCase {
    public void test()
    {
        assertEquals( "Equality Test", 0, 1 );
    }
}

And here is the build.xml file that we'll be using. There are just two changes from our previous example (shown in bold). First, we set property TALK to false in order to decrease the amount of output. And second, we replace the single test sub-element of the JUnit task with a batchtest that uses a file system regular expression to identify the test classes to run.

<project default="all">
  <property name="tst-dir" location="test" />
  <property name="TALK" value="false" />

  <path id="classpath.base">
  </path>

  <path id="classpath.test">
    <pathelement location="/Projects/Java/Lib/junit.jar" />
    <pathelement location="${tst-dir}" />
    <path refid="classpath.base" />
  </path>

  <target name="compile-test">
    <javac srcdir="${tst-dir}"
           verbose="${TALK}"
           >
      <classpath refid="classpath.test"/>
    </javac>
  </target>
  <target name="clean-compile-test">
    <delete verbose="${TALK}">
      <fileset dir="${tst-dir}" includes="**/*.class" />
    </delete>
  </target>

  <target name="test" depends="compile-test">
    <junit>
      <classpath refid="classpath.test" />

      <formatter type="brief" usefile="false" />
      <batchtest>
        <fileset dir="${tst-dir}" includes="**/Test*.class" />
      </batchtest>
    </junit>
  </target>

  <target name="all" depends="test" />

  <target name="clean" depends="clean-compile-test" />
</project>

We can now execute our test.

$ ant clean
Buildfile: build.xml

clean-compile-test:
   [delete] Deleting 2 files from /Tutorial/Ant/JUnit2/test

clean:


BUILD SUCCESSFUL
Total time: 4 seconds
$ ant test
Buildfile: build.xml

compile-test:
    [javac] Compiling 2 source files

test:
    [junit] Testsuite: Test
    [junit] Tests run: 1, Failures: 1, Errors: 0, Time elapsed: 0.012 sec

    [junit] Testcase: test(Test): FAILED
    [junit] Equality Test expected:<0> but was:<1>
    [junit] junit.framework.AssertionFailedError: Equality Test expected:<0> but was:<1>
    [junit]  at Test.test(Unknown Source)


    [junit] TEST Test FAILED
    [junit] Testsuite: TestExample
    [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.001 sec


BUILD SUCCESSFUL
Total time: 5 seconds

Note that ant exits with BUILD SUCCESSFUL despite a junit failure. This was an intentional choice by the designers to allow the completion of a test suite. A revised target that includes the recommended JUnit task structure is shown below. In this structure, a property is set when the JUnit task encounters a failure (test failure or JUnit error). After the JUnit task completes, we fail the target if the failureProperty was set.

  <target name="test" depends="compile-test">
    <junit failureProperty="test.failure" >
      <classpath refid="classpath.test" />
      <formatter type="brief" usefile="false" />
      <batchtest>
        <fileset dir="${tst-dir}" includes="**/Test*.class" />
      </batchtest>
    </junit>
    
    <fail message="test failed" if="test.failure" />
  </target>

We'll also modify the compile-test target by setting debug="true" in the javac task as shown. This will add line number information to the errors reported from JUnit.

  <target name="compile-test">
    <javac srcdir="${tst-dir}"
           verbose="${TALK}"
           debug="true"
           >
      <classpath refid="classpath.test"/>
    </javac>
  </target>

Now we execute ant:

$ ant clean
Buildfile: build.xml

clean-compile-test:
   [delete] Deleting 2 files from /Tutorial/Ant/JUnit2/test

clean:

BUILD SUCCESSFUL
Total time: 3 seconds
$ ant test
Buildfile: build.xml

compile-test:
    [javac] Compiling 2 source files

test:
    [junit] Testsuite: Test
    [junit] Tests run: 1, Failures: 1, Errors: 0, Time elapsed: 0.009 sec

    [junit] Testcase: test(Test): FAILED
    [junit] Equality Test expected:<0> but was:<1>
    [junit] junit.framework.AssertionFailedError: Equality Test expected:<0> but was:<1>
    [junit]  at Test.test(Test.java:7)


    [junit] TEST Test FAILED
    [junit] Testsuite: TestExample
    [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0 sec


BUILD FAILED
file:/Tutorial/Ant/JUnit2/build.xml:36: test failed

Total time: 4 seconds

Much better. Here's the full build.xml file:

<project default="all">
  <property name="tst-dir" value="test" />
  <property name="TALK" value="false" />

  <path id="classpath.base">
  </path>

  <path id="classpath.test">
    <pathelement location="/Projects/Java/Lib/junit.jar" />
    <pathelement location="${tst-dir}" />
    <path refid="classpath.base" />
  </path>

  <target name="compile-test">
    <javac srcdir="${tst-dir}"
           verbose="${TALK}"
           debug="true"
           >
      <classpath refid="classpath.test"/>
    </javac>
  </target>
  <target name="clean-compile-test">
    <delete verbose="${TALK}">
      <fileset dir="${tst-dir}" includes="**/*.class" />
    </delete>
  </target>

  <target name="test" depends="compile-test">
    <junit failureProperty="test.failure">
      <classpath refid="classpath.test" />

      <formatter type="brief" usefile="false" />
      <batchtest>
        <fileset dir="${tst-dir}" includes="**/Test*.class" />
      </batchtest>
    </junit>

    <fail message="test failed" if="test.failure" />
  </target>

  <target name="all" depends="test" />
  <target name="clean" depends="clean-compile-test" />
</project>