Continuous Integration

Continuous Integration

Create automated, clean, and consistent test environments with Turbo while keeping the Continuous Integration (CI) server free from conflict causing dependencies.

Existing CI servers may have many varying libraries, runtimes, custom settings, and applications, possibly in different languages and versions, that are required for the various automated builds. Sometimes these dependencies collide to create inconsistent test environments and headaches when configuring automated builds.

With Turbo no installed dependencies are required on the CI server. All dependencies are built into containers providing consistent environments and a dependency-free CI server. This eliminates the possibility of dependency collision and makes automated build configuration very simple.

The basic steps for integrating Turbo into a CI server are to create a TurboScript, integrate it into an automated build on a CI server, and finally run and test the container.

Create TurboScript

The TurboScript contains all the steps necessary to build the container.

# Creates a new container from the specified images
layer nodejs/nodejs git/git

# Clone a project in the container
cmd mkdir c:\root
cmd git clone https://github.com/project/repo c:\root

# Install Node.js depencies 
cmd cd c:\root\server & npm install

Save your script as a .me file. See the TurboScript reference for more information on TurboScript script instructions.

Integrate into the CI server

The next step is to configure an automated build on the CI server that will execute the TurboScript and create an image. You'll need to configure the necessary triggers, schedules, notifications, etc., which will vary between CI servers.

Now add the follow commands to your automated CI build script:

# Log in if images from a hub repository are specified
turbo login <username> <password>

# Execute the TurboScript and build a new image
turbo build -n=<name> C:\path\to\turbo.me

# Export the image to a location on the host system
turbo export <name> c:\root\image.svm

Rather than exporting the image to the host system, you can also turbo push the image to the hub where other users could pull it down and test.

See the Command Line Interface page for more information on turbo commands.

Run the container

# First import the image to your local registry
> turbo import -n=<name> C:\root\image.svm

# Run the image
> turbo run <name> <command>

If you alternately pushed to the hub then use these commands to pull and run:

> turbo pull <account name>/<name>
> turbo run <account name>/<name> <command>

Example use case: running tests inside a container on the CI server

We would like to run the tests of a project inside a turbo container, all within a CI server job.

Let's assume we have an ready-built image, pushed to the hub, containing the project source code and all the development dependencies: jdk,maven,git.

layer jdk,maven,git

cmd mkdir c:\root
cmd git clone https://github.com/JodaOrg/joda-time.git C:\root

The CI server job that runs the tests inside a turbo container would look like this:

turbo login %SPOON_USER% %SPOON_PWD%

rem pull latest changes and run the tests
turbo run turbo/sample -n=ci-example --attach -w="C:\root\joda-time" /c git pull ^&^& mvm clean test
set TESTS_EXITCODE=%errorlevel%

rem capture the test results from the container, make them available to the CI server
mkdir surefire-reports
turbo cp ci-example:C:\root\joda-time\target\surefire-reports surefire-reports

rem discarding container
turbo rm ci-example

rem exit using the testrun's exit code
exit /b %TESTS_EXITCODE%

The CI server stores the Turbo credentials as environment variables SPOON_USER and SPOON_PWD

When running the image, it first pulls latest code changes and then launches the build and runs the tests.

After the tests complete, there are two things that need to be done, before it finally exits:

  • test results need to be made available outside of the container, for tasks like test results parsers, which display rich information about test failures.

  • the temporary container needs to be discarded.

Once these things completed, the script exits using the original tests run exit code. This to ensure that if the test run failed, the CI server will detect this by the exit code returned.

Using Turbo Studio Snapshot

While many application can be installed directly into Turbo containers through a normal install process or be built using a TurboScript, some applications require the use of snapshots to create a working image.

Snapshotting is a technique for creating images of virtual applications by detecting file system and registry modifications made during installation.

Here we will discuss how to create an automated pipeline for creating snapshot-based application images using Turbo Studio, Vagrant and Jenkins. Jenkins is an automation server used to manage automated processes. Vagrant is a tool used to manage virtual machine instances (in our case, VirtualBox).

The Basics

Creating a virtual application image using snapshots is a multi-step process. It includes taking a snapshot of a clean operating system image before installing an application, performing the installation, and taking the second snapshot after the setup completes. Extra time is required for managing a virtual machine or downloading the application installer. Overall, building application images via snapshot manually is time consuming, so the process benefits dramatically from scaling via automation.

The TurboScript plugin for Jenkins provides a build step to automate creating snapshot-based application images. With a small amount of configuration Jenkins can perform all tasks necessary to create an application image via snapshot:

  • Boot a clean virtual machine
  • Take the before snapshot
  • Install an application
  • Take the after snapshot
  • Build an application image
  • Import an application image to the local repository on the host machine
  • Delete the virtual machine

Features available off the shelf in Jenkins help to further customize the workflow by triggering a build by an external event, downloading an application installer or publishing an image to the Turbo.net Hub.

As an example, we will show how to create a Jenkins build project for FL Studio, a popular music production system.

Initial Setup

The configuration presented below was tested using Jenkins 2.134, Vagrant 1.7.4, and VirtualBox 4.3 installed on Windows 8.1 and Windows Server 2012 R2.

First, install Jenkins for Windows, Vagrant, VirtualBox, and Turbo Studio.

The TurboScript plugin for Jenkins uses PowerShell to call external tools like Vagrant and Turbo Studio. To enable execution of PowerShell scripts on the Jenkins host machine, run the command below in an elevated command prompt:

> powershell -Command Set-ExecutionPolicy RemoteSigned

Configure Jenkins

Open the Jenkins Plugin Manager and install the latest version of TurboScript Plugin.

Go to the Jenkins configuration page and scroll down to the Studio Snapshot section.

Specify the file paths to XStudio.exe in the Turbo Studio install directory and a license file on Jenkins host machine. If you installed Turbo Studio using default settings, XStudio.exe can be found in the C:\Program Files (x86)\Turbo.net\Turbo Studio 18 folder.

Optionally, you may want to change the default virtual machine used to take snapshots. Boxes are downloaded from Vagrant's Service. The list of boxes is available here.

Create the Build Project

Go to the Jenkins dashboard and create a new Turbo Project. Name it FL Studio Snapshot and click OK.

Create a PowerShell script file which downloads the latest FL Studio installer. You can copy the code snippet below:

(New-Object System.Net.WebClient).DownloadFile(
    "http://support.image-line.com/redirect/FLStudio_Installer",
    "install.exe")
    
 $versionInfo = (Get-Item -Path "install.exe").VersionInfo
 
 $tag = ("{0}.{1}.{2}.{3}" -f $versionInfo.FileMajorPart,
     $versionInfo.FileMinorPart,
     $versionInfo.FileBuildPart,
     $versionInfo.FilePrivatePart)
 
 "flstudio:$tag" | Set-Content "image.txt"

Optionally, the script may create an image.txt file in the current working directory. The file should contain the name of the output image. It is considered a best practice to include a product version. In the sample code above the product version is extracted from file attributes.

Save the script file on the Jenkins host machine and execute it in a build step.

To add a build step, use the Execute Windows batch command option.

Specify the following command in the build step configuration:

powershell.exe -File "path_to_the_script"

Next, add a build step Take Studio snapshot and setup it in the following way:

Select the Generate using template option for installation script.

Specify /S to force a silent install.

Select the Ignore exit code checkbox.

Use a Fixed startup file set to @PROGRAMFILESX86@\Image-Line\FL Studio 12\FL.exe. For most applications, the default startup file selection made by Turbo Studio is sufficient.

Optionally, select the Overwrite checkbox if you want Jenkins to continue the build in case an image with the same version is already available in the Turbo.net Hub. Otherwise, the build will be aborted.

Save the build project and go back to the main Jenkins dashboard.

Trigger the Build

Open the context menu next to project name and click Build Now.

The build should finish in approximately half an hour. The FL Studio image will be saved in a local repository. If you want to publish it to the Turbo Hub, simply add the build steps Login to Turbo Hub and Push Turbo image.

Questions? Projects? Talk to us.