Using Maven with TeamCity

https://www.safaribooksonline.com/library/view/learning-continuous-integration/9781849699518/ch04s02.html

Apache Maven (http://maven.apache.org/) is a build, deployment, and dependency management tool for Java-based projects. Maven really emphasizes convention over configuration. What this means is that it is very simple to start managing the build and deployment steps of our project with Maven by following simple conventions. We don't have to write a lot of custom tasks to get started, and therefore we can focus on the software itself, rather than spending a lot of time on how the software is built.

This can be contrasted with the Ant build file that we used in the previous section on Ant, even though that itself is a simple build file that doesn't do a lot of things. In the Ant build file, we had to use properties to specify the location of the source code, the directory where we wanted to generate the build output, and also the directory where we wanted to put the distribution. We then used these properties in various tasks, such as compiling and generating the JAR files. We also had to specify the classpath explicitly so that the junit task can find the necessary libraries for it to run.

In Maven, these locations are assumed to be located in certain directories. Source code, for example, is assumed to be located in <root>/src/main/java. The distributable is assumed to be generated at <root>/target, and so on. By following such conventions, Maven makes it very easy to get started with the builds. Also, it makes it very easy for people switching between projects, and also working on multiple projects, to know exactly what does what and what gets generated where.

Note

Convention over configuration means that there are some defaults that are assumed of the project. But project requirements change from team to team, and tools like Maven allow such customizations/configurations to be done as well.

Assuming such defaults and having an opinionated framework like Maven works well for build and deployment purposes. These are tasks that vary little between projects. The usual activities of compile, running tests, and packaging, are almost the same, and they save minor differences. Yet, each team would end up writing their own build scripts that did virtually the same thing. Maven helps us in removing such duplication of work.

Enough with talking about Maven, let's start using it!

Installing Maven

Maven can be downloaded from http://maven.apache.org/download.cgi. The installation steps are pretty similar to what we saw for Ant:

cd /usr/local
wget http://mirror.symnds.com/software/Apache/maven/maven-3/3.1.1/binaries/apache-maven-3.1.1-bin.tar.gz
tar xvfz apache-maven-3.1.1-bin.tar.gz
export M2_HOME=/usr/local/apache-maven-3.1.1
export M2=$M2_HOME/bin
export PATH=$M2:$PATH
mvn –version

Tip

Maven can also be installed using a package manager on your OS of choice. For example, on OS X, if you are using brew, Maven can be installed using a command as simple as the following:

brew install maven.

We change to the /usr/local folder and download the binary distribution from a mirror using wget. We extract the distribution using the tar command and set the necessary environment variables to get Maven working. We set the M2_HOME environment variable pointing to the directory we just extracted. The M2 environment variable is set pointing to the bin directory within M2_HOME. To add the Maven command to path, we also add M2 to PATH. Finally, we run mvn --version to verify that Maven is available and working fine.

Tip

The installation procedures will vary between different operating systems, especially around setting the environment variables, including PATH. The export commands here affect these environment variables only in the current session.

Creating a Maven project

Maven comes with the archetype plugin, which can be used to generate our project following the standard Maven directory structure.

We can generate our project using the following command:

mvn archetype:generate -DgroupId=com.stacktoheap.maven_ci_example -DartifactId=maven_ci_example -DarchetypeArtifactId=maven-archetype-quickstart -DinteractiveMode=false

We run the Maven goal, provided by the archetype plugin, to generate a project. We use the maven-archetype-quickstart archetype to generate our project. There are various other archetypes available for us to choose from. The first time this command is run, it will take a while for Maven to complete as Maven will download all the dependencies and artifacts it needs.

When the command finishes, we will see a directory with the name maven_ci_example, which is the name we gave for artifactId. This directory has the structure followed by Maven projects. The source code is present at maven_ci_example/src/main/java, and maven_ci_example/src/test/java has the tests.

The maven_ci_example folder also has the pom.xml file.

Introducing the Project Object Model (POM)

The pom.xml file, named after Project Object Model, is the main configuration file used by Maven to build our project. The pom.xml file generated by the archetypeplugin is as follows:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>com.stacktoheap.maven_ci_example</groupId>
  <artifactId>maven_ci_example</artifactId>
  <packaging>jar</packaging>
  <version>1.0-SNAPSHOT</version>
  <name>maven_ci_example</name>
  <url>http://maven.apache.org</url>
  <dependencies>
    <dependency>
      <groupId>junit</groupId>
      <artifactId>junit</artifactId>
      <version>3.8.1</version>
      <scope>test</scope>
    </dependency>
  </dependencies>
</project>

This defines a Maven project by the name maven_ci_example. The<packaging>jar</packaging> tag states that the project generates jar as the build output (a webapp would generate war, for example). The pom.xml also adds junit as a dependency, as a sample test class was also generated by the archetype plugin, under maven_ci_example/test/java/com/stacktoheap/maven_ci_example/AppTest.java.

Note

We will not be going into the details of Maven in this book as that in itself is worthy of a book or two. We will touch on the necessary aspects of Maven needed to set up CI for a project using Maven in TeamCity. To understand more about Maven, I recommend the Maven By Examplebook by Tim O'Brien, John Casey, Brian Fox, Jason Van Zyl, Juven Xu, Thomas Locher, Dan Fabulich, Eric Redmond, and Bruce Snyder, found online at http://books.sonatype.com/mvnex-book/reference/public-book.html. If you are not using Maven already and are looking at starting to use Maven with TeamCity, I recommend a read of this book before proceeding.

Building the project

We can build our project using the mvn install command. This command will compile our project, run the junit tests, package it into JAR, and install it in the local Maven repository.

Note

The install parameter to mvn can be seen as being similar to how we pass target names to the Ant command. These are called goals in Maven. When we used the archetype:generate command previously to generate our project, the archetype:generate was the goal. To be more specific, we are identifying the plugin,archetype, and the goal within that plugin, generate, to be executed.

Maven utilizes various plugins, such as maven-jar-plugin to generate the JAR file, and maven-surefire-plugin to run the unit tests. A section of the log, as generated by the previous command, is shown in the following screenshot:

The previous screenshot shows how the junit tests are run by the Surefire plugin and then proceeds to generate the JAR file using maven-jar-plugin.

Note

Understanding the Maven build lifecycle is essential when working with Maven. As we build a project, the build process moves through various lifecycles and also phases within these lifecycles. Based on the plugins we use, and the configurations in our pom.xml, different goals get associated with different lifecycles and phases, and it is this association that gives the goals and the order in which they are executed.

The install goal used in the mvn install command is tied to the install lifecycle phase. All the previous lifecycles phases are executed for a given goal/lifecycle, and this is the reason why the goals in previous phases, such as compile, test, and package, were executed as well.

The install goal installs the generated JAR file into an appropriate location in the local Maven repository. In our case, it was installed to ~ /.m2/repository/com/stacktoheap/maven_ci_example/maven_ci_example/1.0-SNAPSHOT/maven_ci_example-1.0-SNAPSHOT.jar.

We have achieved a lot with a simple pom.xml file and didn't have to worry about adding junit to path, setting where the JAR file is to be generated, and so on. All of these were achieved by following the conventions set by Maven, and our project was following these conventions already because we used the archetype plugin to generate our project.

Using Maven in a build configuration

I have created a repository on GitHub for our project created in the previous section, which can be found at https://github.com/manojlds/maven_ci_example.

We will create a new build configuration with the name maven_build in our Java CI with TeamCity project.

We will then create and attach a new VCS root pointing to the repository on GitHub.

These steps are similar to the ones we covered in the previous chapter.

When it comes to adding a build step, we will choose Maven as the build runner. A section of the settings page for the build runner is given in the following screenshot:

The Goals option is similar to the Targets options that we set for the Ant build runner. Here, we specify install as the goal that we want to run for our build configuration.

Tip

It is recommended to specify clean install as the goals so that we perform a cleanup before starting our builds. The clean goal will remove any directories and other output generated by previous builds. Clean and Install are different lifecycles, and therefore mvn install doesn't call the clean goal by default, and it needs to be called as mvn clean install.

The path to the POM file can be left at the default pom.xml value in our case but may be changed if we are using a POM file not located at the root.

Additional Maven command line parameters can be used to specify other parameters that we may want TeamCity to pass to the mvn command. One such parameter could be the–P flag used to set profiles.

Note

Maven build profiles provide the ability to specify environment-related information to a build. For instance, we can specify a profile dev for development purposes, and another one namedci to run on CI. They are usually specified in the pom.xml file or in a settings file.

The Working directory setting is used to define the working directory to execute themvn command for the build.

The Maven Home section is used to define the location where Maven is installed on the agents running the build. The various options are explained as follows:

• The <Default> option will make TeamCity find a Maven installation pointed to byM2_HOME. If we want to use our own installation of Maven in TeamCity, we need to follow the previous installation instructions as the user under which the agent is running.
• The <Custom> option allows us to enter a path in the settings to the location where Maven is installed and removes dependence on the M2_HOME environment variable.
• The Bundled Maven 2 and Bundled Maven 3 options use the Maven versions (for 2.x and 3.x respectively) bundled with TeamCity. This is similar to using the bundled Ant.

We can choose the bundled Maven 3 in our case for simplicity, but it is recommended to maintain and use our own version of TeamCity so that developers can use the latest (or a specific version) Maven that is needed, rather than depending on the version that is bundled with TeamCity.

The User Settings section is used to specify the location of the Maven settings (settings.xml) file. This is usually located at ~/.m2/settings.xml. Using<Default> will use this path on the agent. By using <Custom>, we can specify a custom path to this file. We will leave it on <Default> in our case, as we don't depend on any settings from the file.

Note

Maven settings files are used to define servers, repositories, authentication, profiles, and other details.

Java parameters are similar to what we saw for Ant, used to specify the JDK path and also to provide any additional JVM parameters.

The Use own local artifact repository option can be checked to isolate the local repository of this particular build configuration from that of others.

The Enable incremental building option can be enabled to allow TeamCity to build only modules that are affected by the commits being used in the current build. TeamCity also has enough smarts to run only the tests that are affected. This option can be selected to reduce the build times. The ability to build incrementally is on top of such features provided by Maven itself.

We will not enable Code Coverage for our build configuration at this point in time.

Let's Save the build configuration and run it manually to see the fruits of our labor.

We should see our build configuration pass and also show the test information in the status message, as seen in the following screenshot:

We can also see from the screenshot that there is a link to the Maven Build Infotab, which gets added to build configurations using the Maven build runner. This tab provides information, such as the Maven projects in the build, the plugins used, and so on. The information provided is similar to the effective POM settings displayed by the mvn help:effective-pom command.

Setting version number

In our pom.xml, we have defined the version as:

<version>1.0-SNAPSHOT</version>

When we are building the project as part of the CI, we want to use a proper release version (which corresponds to the build number/version on TeamCity).

We can set a version on the pom.xml from the command line by running the following line of code:

mvn versions:set versions:commit -DnewVersion="1.0.1"

This will set the version to 1.0.1 during the build, and our mvn install will finally install the specified version in our repository.

We will perform this as a new build step for the maven_build configuration. Let's add a build step with Maven runner from the build configuration's settings page. The goals settings for this build step will be versions:set versions:commit. Save the build step to have it added to the build configuration. In the Build Steps page, we can click on Reorder build steps to move the new build step before the existing build step, which performs mvn install.

So how do we pass in –DnewVersion="build number" to the build step? If you thought system properties, you guessed it right!

We can head to the Build Parameters section and add a new System property. We provide the %build.number% parameter as the value for the newVersion system property. As we have seen previously, the %build.number% parameter has the build number of the current build.

This newVersion system property is automatically passed to Maven asDnewVersion="build number" while running the builds. Note that this is applied to both the build steps in the build configuration, but that is fine.

We can see the build number being appended to the names of the JAR files installed in our local repository when we perform mvn install.

Setting up code coverage for our build

We will be setting up code coverage for our build using the JaCoCo coverage runner along with its Maven plugin. We will begin by adding the following section to ourpom.xml file:

<build>
  <plugins>
    <plugin>
      <groupId>org.jacoco</groupId>
      <artifactId>jacoco-maven-plugin</artifactId>
      <version>0.6.2.201302030002</version>
      <executions>
        <execution>
          <goals>
            <goal>prepare-agent</goal>
          </goals>
        </execution>
        <execution>
          <id>report</id>
          <phase>prepare-package</phase>
          <goals>
        <goal>report</goal>
          </goals>
        </execution>
      </executions>
    </plugin>
  </plugins>
</build>

We add the JaCoCo Maven plugin and hook it up in different execution phases so that it can run before the tests start running and also to ensure that JaCoCo coverage reports are generated in the end.

When we run mvn install, we can see that the JaCoCo reports are generated atmaven_ci_example/target/site/jacoco/index.html.

Let's set up the reports as an artifact in our maven_build build configuration. Head to the configuration page for our maven_build build configuration. In General settings, under Artifacts, we can enter the following to package the reports located in the path into a coverage.zip file and expose it as an artifact on TeamCity:

target/site/jacoco/**/*.* => coverage.zip

We can trigger the build to see what happens. After the build passes, we can see that a shortcut link to Code Coverage is automatically added, as seen in the following screenshot:

Clicking on this link takes us to the Code Coverage tab (added automatically), which shows the index.html page generated by JaCoCo.

How did this happen? TeamCity automatically recognizes artifacts named coverage.zip(and having an index.html file in the root) as coverage reports and sets up the Code Coverage tab for them. If we didn't follow this name, or just uploaded the artifact as a directory, we would have had to set up our own report tab pointing to theindex.html file within the artifact.

Tip

The ability of TeamCity to identify artifacts based on their names and automatically create tabs for them is configurable from the Administration Page | Report Tabs (under Integrations in the left-hand side bar.) In this page, we can see how the Code Coverage tab is created automatically on detecting coverage.zip as an artifact. We can edit this behavior or add new ones from this page.

The way we have set up coverage, using the JaCoCo plugin for Maven, does not provide statistics information regarding coverage. But even these are possible using a TeamCity feature called service messages(http://confluence.jetbrains.com/display/TCD8/Build+Script+Interaction+with+TeamCity), which we will explore in a future chapter. As of TeamCity 8.1, the JaCoCo coverage tool support is included out of the box. Using this, we get much tighter and easier integration with JaCoCo for our builds.

Maven on TeamCity, beyond the build runner

Support for Maven in TeamCity is not just limited to providing a build runner for it. In this section, we will be exploring some of the other Maven-related features that TeamCity provides, and which makes it a great CI tool for use with Maven.

Creating a Maven build configuration

So far, we have created build configurations for our Maven-based builds the usual way. You might have noticed from the settings page of a project that there is also aCreate Maven build configuration button right next to the Create build configuration button. Clicking on this button takes us to the create settings page as seen in the following screenshot:

In this page, we can either enter the URL to the POM file for our Maven build or directly upload the POM file.

The pom.xml file should include scm settings for this to work. TeamCity needs this information to populate the VCS for the build configuration. The scm configuration for our sample project looks like the following snippet:

<scm>
<connection>scm:git:git://github.com/manojlds/maven_ci_example.git</connection>
<developerConnection>scm:git:git://github.com/manojlds/maven_ci_example.git</developerConnection>
<tag>HEAD</tag>
<url>https://github.com/manojlds/maven_ci_example</url>
</scm>

We can use https://raw.github.com/manojlds/maven_ci_example/master/pom.xml in our case.

The Username and Password provide the authentication settings needed to fetch thepom.xml file. Goals, as expected, configures the goals to run as part of the build.Triggering can be checked to trigger the build when a dependency of the project (as specified in the POM file) changes.

Clicking on Create creates the build configuration. TeamCity will use the settings from the specified pom.xml file to configure the build appropriately. This is a quick and easy way of creating our Maven build configurations.

Tip

I recommend using the normal way of creating Maven build configurations. We can tweak all the important settings, such as the build options, and others, when we create our first build configuration and can then copy this build configuration to create new ones.

Global Maven settings file

From Administration page | Maven settings (found under Integrations in the left-hand side bar), we can upload a global settings file for use in the Maven builds.

As of TeamCity 8.1, it is now possible to specify the Maven settings for a project from the project's settings page. TeamCity uses these settings to get information about the repositories to trigger builds that use Maven-based triggers.

Setting up Maven-based triggers

TeamCity also comes with support to trigger dependent builds when their dependencies change in the repository. This support is over the artifact-based triggering mechanism.

There are two Maven-based triggers:

• Maven Snapshot Dependency Trigger: This triggers the build when any of thesnapshot dependencies of the project change. The dependency information is obtained from the pom.xml file.
• Maven Artifact Dependency Trigger: This triggers the build when the dependenciesof the project change. This includes both SNAPSHOT and version-based dependencies, unlike the Maven Snapshot Dependency Trigger, which only looks at SNAPSHOTS.

Note

Do not confuse snapshot here with the Snapshot dependencies in TeamCity. The Snapshots concept mentioned here are Maven- specific.

We will look at the Maven Artifact Dependency Trigger in detail. I have created another sample project at https://github.com/manojlds/maven_ci_dependant_example. This project uses the maven_ci_example project as a dependency. The pom.xml file for both the projects are updated with the repository information to store and retrieve the artifacts from a remote repository. The addition to pom.xml formaven_ci_example is as follows:

<distributionManagement>
  <repository>
    <uniqueVersion>false</uniqueVersion>
    <id>maven_ci_repo</id>
    <name>Maven CI repo</name>
    <url>file:///Users/Admin/.m2/repository2</url>
    <layout>default</layout>
  </repository>
</distributionManagement>

When we run mvn deploy, Maven will deploy the JAR file and pom.xml of our project to the configured remote repository (which, for simplicity, is still a local filesystem location).

The maven_ci_dependant_example project has the maven_ci_example project as a dependency. It also configures the repository from which this dependency has to be fetched. The relevant sections of the pom.xml for this project are as follows:

<dependencies>
  <dependency>
    <groupId>com.stacktoheap.maven_ci_example</groupId>
    <artifactId>maven_ci_example</artifactId>
    <version>${maven_ci_example.version}</version>
  </dependency>

</dependencies>
<repositories>
  <repository>
    <id>maven_ci_repo</id>
    <url>file:///Users/Admin/.m2/repository2</url>
    <snapshots>
      <enabled>true</enabled>
    </snapshots>
    <releases>
      <enabled>true</enabled>
    </releases>
  </repository>
</repositories>

With the pom.xml files set up, we can set up the triggers. We will add a new build configuration for the maven_ci_dependant_example project. We can do this by simply copying the maven_build build configuration and then changing the VCS root and other settings as needed. We can call this maven_dependant_build. We will also update the build step of the maven_build to do mvn deploy, rather than mvn install, so that we can deploy the project to the configured remote repository.

Heading to the Build triggers section of the maven_dependant_build build configuration, we can add the Maven Artifact Dependency Trigger. The necessary configuration is seen in the following screenshot:

Group ID and Artifact ID are the corresponding values that we set in pom.xml.Version range specifies the versions of the artifact that can be used, in this case between 1.0 and 2.0, with 1.0 inclusive. Type specifies the packaging, which is jarfor our project. Finally, we configure Maven repository URL for TeamCity to detect changes that trigger the builds.

Clicking on Save adds the build trigger. Now, as the maven_build build configuration runs and deploys the distributables to the repository, TeamCity will detect the changes and trigger the maven_dependant_build build configuration.