# Using gradle in docker with Jenkins

Over the last few days I migrated the whole CI/CD pipeline from plain jenkins to docker with jenkins. What sounds like an easy task took way longer that it should have due to various pitfalls when using docker with jenkins. This also applies to builds running in openshift / kubernetes.

In this post I want to share the experience, so you don't have to go through it all again.

# Build process

My primary build tool is gradle. The Project structure I'm using is quite complex but here is a high level overview:

Overview

Most of the project can be build using a regular gradle and jdk installation. Only the last modules in the tree require a specific SDK or platform to be build:

# Android

In order to build the android apps the android-sdk is required.

# iOS / OSX

For the iOS / OSX app XCode is required. Since OSX doesn't natively support docker there is no way around using a jenkins agent on a mac.

# Windows

The windows version requires windows for creating the installer. Also docker can't be used here since I'm still using a windows 7 VM for running some backward compatibility tests. Once updated to windows 10 this could be moved into docker as well.

# Target architecture

After analyzing the project it gets clear that we need three different docker containers:

Regular gradle with jdk
Gradle with jdk and xcfb for integration UI tests
Gradle with jdk and android SDK

Overview

In order run some steps inside a docker container the jenkins pipeline provides a simple api:

docker.image('gradle:6.3-jdk14'){
   sh 'gradle ...'
}

at least in theory...

# Caching in $HOME

Gradle uses a .gradle caching directory which is usually stored in $HOME (or the java property user.home)

But this causes issues since jenkins uses its own UID/GID when running inside the container. In other container solution such as openshift a random UID will be used so this issue applies there as well.

Since the container doesn't know about this UID/GID gradle tries to create its cache dir in /.gradle which fails due to missing permissions.

we could use --tmpfs and mount it to /.gradle but this won't work on windows which we might need later on. So we just create our own docker image which includes a cache dir with the appropriate permissions.

You could also choose any other path by adjusting $HOME/user.home accordingly.

# JDK + desktop

The second issue was that the gradle image comes with a smaller JDK installation which does not include some packages, including java.desktop which is required for the desktop build. Also the linux dependencies for jlink and jpackage are missing as well.

# Gradle dockerfile

Here is my complete jenkins compatible dockerfile with JDK-14 and gradle 6.3. You can adjust the javaUrl argument or GRADLE_VERSION ENV if required.


FROM ubuntu:groovy
ARG javaUrl https://download.java.net/java/GA/jdk14.0.1/664493ef4a6946b186ff29eb326336a2/7/GPL/openjdk-14.0.1_linux-x64_bin.tar.gz
USER root
ENV LANG=en_US.UTF-8 LANGUAGE=en_US:en LC_ALL=en_US.UTF-8
ENV JAVA_HOME /opt/java/openjdk
ENV JAVA_URL $javaUrl
ENV GRADLE_VERSION=6.3
ENV GRADLE_USER_HOME /.gradle
ENV GRADLE_HOME=/opt/gradle
ENV PATH=${JAVA_HOME}/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
RUN apt-get update \
	&& apt-get install -y --no-install-recommends curl ca-certificates fontconfig locales unzip \
	&& echo "en_US.UTF-8 UTF-8" >> /etc/locale.gen \
	&& locale-gen en_US.UTF-8 \
	&& rm -rf /var/lib/apt/lists/*
RUN echo Downloading java from ${JAVA_URL} \
	&& curl -Lo jdk.tar.gz ${JAVA_URL} \
	&& tar xzf jdk.tar.gz \
	&& rm jdk.tar.gz \
	&& mkdir -p /opt/java \
	&& mv jdk-* ${JAVA_HOME}
RUN set -o errexit -o nounset \
	&& echo "Downloading Gradle" \
	&& curl -Lo gradle.zip https://services.gradle.org/distributions/gradle-${GRADLE_VERSION}-bin.zip \
	&& echo "Installing Gradle" \
	&& unzip gradle.zip \
	&& rm gradle.zip \
	&& mv "gradle-${GRADLE_VERSION}" "${GRADLE_HOME}/" \
	&& ln --symbolic "${GRADLE_HOME}/bin/gradle" /usr/bin/gradle \
	&& echo "Testing Gradle installation" \
	&& gradle --version
# .gradle and .android are a cache folders
RUN mkdir -p ${GRADLE_USER_HOME}/caches /.android \
	&& chmod -R 777 ${GRADLE_USER_HOME} \
	&& chmod 777 /.android
# Required for jlink and jpackage
RUN apt-get update \
	&& apt-get -y install binutils fakeroot

# Android image

Creating the android image is now pretty easy. All we need to do is to download the sdk-manager, fake the "license accepted" part and download the required packages.

As baseImage you can use the previously build gradle image.

ARG baseImage
FROM $baseImage
USER root
ENV SDK_URL https://dl.google.com/android/repository/commandlinetools-linux-6200805_latest.zip
ENV ANDROID_HOME /usr/local/android-sdk
ENV ANDROID_SDK_ROOT /usr/local/android-sdk
ENV ANDROID_VERSION 28
ENV ANDROID_BUILD_TOOLS_VERSION 30.0.0-rc2
# .android is a cache folder
RUN mkdir -p $ANDROID_HOME/licenses
RUN cd $ANDROID_HOME \
	&& curl -o sdk.zip $SDK_URL \
	&& unzip sdk.zip \
	&& rm sdk.zip
# Create the license files so the sdkmanager doesn't ask for any confirmation
RUN echo "24333f8a63b6825ea9c5514f83c2829b004d1fee" > "$ANDROID_HOME/licenses/android-sdk-license"
RUN echo "84831b9409646a918e30573bab4c9c91346d8abd" > "$ANDROID_HOME/licenses/android-sdk-preview-license"
# Now download the android SDK stuff
RUN $ANDROID_HOME/tools/bin/sdkmanager --sdk_root=${ANDROID_HOME} --update
RUN $ANDROID_HOME/tools/bin/sdkmanager --sdk_root=${ANDROID_HOME} "build-tools;${ANDROID_BUILD_TOOLS_VERSION}" \
    "platforms;android-${ANDROID_VERSION}" \
    "platform-tools"

# Making it run in jenkins

So now we got all the images ready and can start building the jenkinsfile.

# Shared dependency cache

To reduce build times gradle allows sharing of the dependency cache. For this to work you need to copy the .gradle/caches/modulees-2 folder of an existing gradle installation (which has all dependencies already cached) and set the GRADLE_RO_DEP_CACHE environment variable to the path of the copy. In my case the folder structure is:

/opt/jenkins/.gradle/caches/modules-2
/opt/jenkins/.gradle-ro-cache/modules-2

Final docker command in the jenkinsfile. I explicitly set user.home to / since the JDK might otherwise return ? to the application causing for example the android sdk to give some warnings.

docker.image('dev-core:gradle-java-14').inside('--mount type=bind,src=/opt/jenkins/.gradle-ro-cache,dst=/.gradle-ro-cache') {
    withEnv(['GRADLE_RO_DEP_CACHE=/.gradle-ro-cache', 'JAVA_TOOL_OPTIONS=-Duser.home=/']) {
        sh 'gradle..'
    }
}

If you don't want to use a shared cache you can simply remove the GRADLE_RO_DEP_CACHE variable, and the volume mount.

Hope this post could help you. Let me know in the comments if you have any questions.