Icon

You are viewing the documentation of TeamCity 2018.x, which is not the most recently released version of TeamCity.
View this page in the latest documentation or refer to the listing to choose the documentation corresponding to your TeamCity version.

 

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Besides Besides storing settings in version control in  in XML format, TeamCity also allows storing the settings in the DSL (based on the Kotlin language).

Using the version control-stored DSL enables you to define settings programmatically, without the need to use XML files. Since Kotlin is statically typed, you automatically receive the auto-completion feature in IntelliJ IDEA IDE which makes the discovery of available API options a much simpler task.

Note that once you enable storing of the settings in DSL, you no longer can use TeamCity UI to edit affected project's settings.
Also, all the security implications apply.

Check also our Check also our blog post series on  on Kotlin DSL.

On this page:

Table of Contents

 

Advantages of Kotlin DSL vs XML

The Kotlin settings format provides higher level of abstraction, and better code reuse.

  • Build configurations often share settings and need to be different in few aspects only. For example, when you need to test your software against different platforms, let's take an ORM framework to be tested against different databases, then the builds running tests are mostly the same, the only difference is the part configuring a database connection.With Kotlin, you can extract the common part and reuse it across all builds which ensures that the common part does not change and makes maintaining builds easier.
  • Kotlin also allows you to describe your setup in terms of your domain. For example, TeamCity provides generic requirements for build agents and you can define requirements specific to your installation:

    Code Block
    languagegroovy
    val linux = Requirements() {
       contains("os.name", "linux")
    }
    
    val oracle = Requirements() {
       equals("db.name", "oracle")
    }
    
    val java6 = Requirements() {
       contains("env.JAVA_HOME", "1.6")
    }
    

    then you can use them in a build configuration:

    Code Block
    languagegroovy
    buildType {
     ...
     requirements(linux + oracle + java6)
     ...
    }

Getting Started with Kotlin DSL

Learning Kotlin

The

...

 Kotlin tutorial

...

 helps you learn most Kotlin features in a few hours.

Downloading Settings in Kotlin

To You can start working with Kotlin DSL , go to the project settings page, click the Actions menu and select Download settings in kotlin format. This will generate a .zip archive with project settings which can be used in your sandbox project.

When you have familiarized yourself with Kotlin, you can use the Kotlin-based DSL for project settings in production.

Using Settings in Kotlin in Production

...

by creating an empty sandbox project on your server and follow the steps below: 

  1. Enable versioned settings for your project.
  2. Select

    kotlin

    Kotlin as the format.

    Commit Make sure the Generate portable DSL scripts option is enabled.

  3. Click Apply and TeamCity will commit the settings to your version control.

  4. Once you apply the versioned settings to your project, TeamCity will generate necessary Kotlin files for this project and check them in to the specified repository under the .teamcity directory. 

Tip

The easiest way to experiment with DSL in TeamCity 2017.2+ is to edit settings in UI and check the UI diff script generated by TeamCity.

DSL Documentation

After Kotlin DSL is enabled for a project, the TeamCity server provides html documentation for the core DSL API as well as the API provided by installed TeamCity plugins. The online documentation is available on your local server, accessible via the link on the Versioned Settings project tab in the UI or by running the mvn -U dependency:sources command in the IDE. See an example.

Working with Kotlin DSL

The .teamcity folder contains a maven project consisting of pom.xml file and  the <Project Id> directory with the settings.kts and Project.kt files, as well as buildTypes, pluginData, and vcsRoots subdirectories. If the project contains subprojects, they will be placed in the directories with corresponding names.

Opening Project in IDE

Open the pom.xml in your favourite IDE and check that the URL of the teamcity-server repository points to your TeamCity.

If the server url is correct and your TeamCity is up and running, all necessary dependencies will be resolved automatically and you can start working with Kotlin DSL right away.

 

Note

Since 2017.2 the TeamCity server uses the pom.xml file to fetch required dependencies from the Maven repository; the file is also used by the IDE to download dependencies and to provide code completion as well as detect compilation errors.

Debugging Maven ‘generate’ Task

The pom.xml file provided for a Kotlin project has the 'generate' task, which can be used to generate TeamCity XML files locally from the Kotlin DSL files. Since TeamCity 2017.2 this task supports debugging. If you’re using IntelliJ IDEA, you can easily start debugging of a Maven task, see screenshot:

 Image Removed

Ability to Use External Libraries

Since TeamCity 2017.2 You can now use external libraries in your Kotlin DSL code, which allows sharing code between different Kotlin DSL-based projects. To use an external library in your Kotlin DSL code, add a dependency on this library into the .teamcity/pom.xml file in the settings repository and commit this change so that TeamCity detects it. Then, before starting the generation process, the TeamCity server will fetch the necessary dependencies from the Maven repository, compile code with them, and then start  the settings generator. 

Essential Files

settings.kts

The settings.kts script is an entry point for defining a single TeamCity project. TeamCity looks for the settings.kts file in a project directory and runs it if it is found, so the script name must not be changed and its package must be the same as the project's id.

The script should contain a single call to the project() function with a Project instance or the init function as an argument, you can also specify both of them to refine the specified Project in the init function.

Project.kt

This file contains the project definition.

...

  1. repository.

Note

After enabling Kotlin for project settings, editing of the project in the web UI may not be available right after the switch. TeamCity needs to detect its own commit in the repository, and only after that editing will be enabled. Usually it takes a minute or two.



Anchor
portableDSL
portableDSL

Project Settings Structure

After the commit to the repository, you will get the the .teamcity settings directory with the following files:

  • pom.xml
  • settings.kts

settings.kts is the main file containing all the project configuration, that's the file we will use to change project settings.
pom.xml is only required to open the project in an IDE to get the auto-completion feature as well as ability to compile code, write unit tests for it, etc.

Opening Project in IntelliJ IDEA

To open the Kotlin DSL project in IntelliJ IDEA, open the .teamcity/pom.xml file as a project. All necessary dependencies will be resolved automatically right away. If all dependencies have been resolved, no errors in red will be visible in the settings.kts.
If you already have an IntelliJ IDEA project and want to add Kotlin DSL module to it, see this section.

Editing Kotlin DSL

If you created an empty project, that’s what you’ll see in your IDE when you open settings.kts:

Code Block
import jetbrains.buildServer.configs.kotlin.v2018_1.*
/* some comment text  */ 
version = "2018.1"

project {
}

project { } represents the current project whose settings we'll define in the DSL. This is the same project where we enabled versioned settings on the previous step. This project ID and name can be accessed via a special DslContext object but cannot be changed via the DSL. 

You can create different entities in this project by calling vcsRoot(), buildType(),  and template(), or subProject() methods respectively.

Subprojects can be defined either in their own settings.kts or by calling the subProjects() method in this project.

...

languagegroovy

...

For instance, to add a build configuration with a command line script, we can do the following:

Code Block
import jetbrains.buildServer.configs.kotlin.v2018_1.*
import jetbrains.buildServer.configs.kotlin.v2018_1.buildSteps.script 

version = "2018.1"

project {
  buildType {
    id("HelloWorld")
    name = "Hello world"
      val vcsRoot = GitVcsRoot(steps {
        uuidscript = "my_vcs_root_id"{
         id = "ExampleOfDSL_VcsRoot"         name scriptContent = "Exampleecho of DSL VCS Root'Hello world!'"
        url}
= "<url to my git}
repository>"  }
  })
    buildType {
        uuid = "my_build_type_id"
        id = "ExampleOfDSL_Build"
        name = "Build"
        buildNumberPattern = "%build.counter%"
}

After that you can submit this change to the repository, TeamCity will detect it and apply. If there are no errors during the script execution, then we should see a build configuration named "Hello world" in our project.

Note

To get familiar with Kotlin API, see the online documentation on your local server, accessible via the link on the Versioned Settings project tab in the UI or by running the mvn -U dependency:sources command in the IDE.  See the documentation on the public TeamCity server as an example.

The documentation is generated in a separate Java process and might take several minutes to build after the server restart

You can also use the Download settings in Kotlin format option from the project Actions menu. For instance, you can find a project that defines some settings that you want to use in your Kotlin DSL project and use this "download" action to see what the DSL generated by TeamCity looks like.

Anchor
EditingKotlinsettingsinUI
EditingKotlinsettingsinUI

Patches

TeamCity allows editing of a project via the web interface, even though the project settings are stored in Kotlin DSL. For every change made in the project via the web interface, TeamCity will generate a patch in the Kotlin format, which will be checked in under the project patches directory with subdirectories for different TeamCity entities. For example, if you change a build configuration, TeamCity will submit the .teamcity/patches/buildTypes/<id>.kt script to the repository with necessary changes.

For instance, the following patch adds the Build files cleaner (Swabra) build feature to the build configuration with the ID SampleProject_Build:

Code Block
changeBuildType(RelativeId("SampleProject_Build")) { // this part finds the build configuration where the change has to be done  
    features {
       vcs add {
            root(vcsRoot) // thanksthis tois Kotlin,the herepart wewhich canshould havebe staticcopied reference to projecta VCScorresponding rootplace of the settings.kts file
    }        swabra steps {
            maven {   filesCleanup = Swabra.FilesCleanup.DISABLED
           goals =}
"clean test"       }
    }
}

It is implied that you move the changes from the patch file to you settings.kts and delete the patch file. Patches generation allows smooth transition from editing settings via UI to Kotlin DSL.

Sharing Kotlin DSL Scripts

Besides simplicity, one of the advantages of the portable DSL script is that the script can be used by more than one project or more than one server (hence the name: portable).

If you have a repository with .teamcity  containing settings in portable format, then you can easily create another project based on these settings. The Create Project From URL feature can be used for this.

Simply point TeamCity to your repository and it will detect the .teamcity directory and offer to import settings from there:

Image Added

Note

The settings.kts script can always access a DslContext object which contains the id and some other settings of the current project.

Based on the context, the DSL script can generate slightly different settings, for instance:

Code Block
var deployTarget = if (DslContext.projectId == AbsoluteId("Trunk")) {
 }   "staging"
} else {
  requirements { "production"
}

Advanced Topics

Non-Portable DSL

Versions before 2018.1 used a different format for Kotlin DSL settings. This format can still be enabled by turning off  the Generate portable DSL scripts checkbox on versioned settings page.

When TeamCity generates non-portable DSL, the project structure in the .teamcity directory looks as follows:

  • pom.xml
  • <project id>/settings.kts
  • <project id>/Project.kt
  • <project id>/buildTypes/<build conf id>.kt
  • <project id>/vcsRoots/<vcs root id>.kt

where <project id> is the ID of the project where versioned settings are enabled. The Kotlin DSL files producing build configurations and VCS roots are placed under the corresponding subdirectories.

settings.kts

In the non-portable format each project has the following settings.kts file:

Code Block
package MyProject
import jetbrains.buildServer.configs.kotlin.v2018_1.*
/* ... */
version = "2018.1"

containsproject("teamcity.agent.jvm.os.name", "Linux")
        }
    }
})  

As this is a Kotlin code, you are free to  add conditions, loops, methods, classes, etc.

Dedicated DSL for plugins

 Since TeamCity 2017.2 dedicated DSL code is provided for new plugins bundled with TeamCity (Docker, .NET CLI) and coverage plugins. Here is typical DSL code for the Docker-Compose build step:

 

Code Block
steps {
  dockerCompose {
    file = "docker-compose.yml"
  }
}

Required Identifiers

All TeamCity entities - project, build configuration, VCS root and template have a so-called uuid. The uuid is an identifier which can be used to uniquely distinguish this specific entity from others. If at some point the entity receives a new uuid, it is considered a new entity by TeamCity. For instance, if the build configuration uuid changes, the builds history of this build configuration will become empty (if you change it back before cleanup, the history will be restored). The same rule applies to a project - if its uuid changes, the project will lose its investigations and muted tests. We suggest selecting some naming scheme for uuids in your project and never changing them. 

There is also the id field (extId prior to TeamCity 2017.2) which is mandatory. The id is used to represent the project ID, build configuration (or template) ID / VCS root ID in the web UI; it is also used in the DSL to reference other entities. Besides, it is used in various links (such as build configuration links) and REST API.  

Making Changes to Project Settings

Once a project is switched to the Kotlin DSL format, we recommend the following approach for changing the project settings:

  1. Set up unit tests.

  2. Create a dedicated project on your TeamCity server to run these unit tests.

  3. Create a separate branch/repository and run tests on your changes there. 

    Note

    Avoid pushing changes right into the branch monitored by TeamCity without running any tests, as it can affect your project and all of the users currently working with this project  

  4. If the tests are green, merge the changes into the default settings branch. 

    Note

    To merge the branch automatically if the build is green, consider using the TeamCity automatic merge build feature.

Creating New Project

You can create a new project via DSL inside a parent project which already has uses settings in the Kotlin format. A new project can be added either in its own directory with settings.kts or using the subProject() method in the parent project. 

Apply your changes as described in the section below.

Applying Changes to TeamCity

 When you have made changes to your Kotlin DSL code, сommit your code into the .teamcity directory containing your project settings. 

 

Note

Prior to TeamCity 2017.2, no external resources are supported and since pom.xml is not used on the server, any libraries that you might want to add will be ignored during DSL processing. 

Since TeamCity 2017.2, it is possible to use external libraries in your Kotlin DSL code. See details below.

TeamCity will apply the changes to  configuration as follows: 

  1. TeamCity will detect the commit, execute Kotlin DSL files and produce new configuration files. Since internally TeamCity operates with XML, TeamCity will execute DSL files in a secure sandbox to produce XML configuration files. These XML files then will be applied to the existing project.
  2. TeamCity will apply the new configuration files to the project effectively changing its configuration. In case of any problems (compilation failures, runtime errors, etc.), new changes will not be applied, and the current project settings will be preserved on the server and the error message will appear under the Versioned Setting tab in project settings.
  3. The builds will run with the new settings.

 Note that:

  • The Kotlin script is executed on the server, and since this is a potentially dangerous operation, the script is executed in the sandbox. It cannot modify the file system except the place where it is executed, it cannot run other programs, cannot use reflection, and so on.
  • The Kotlin DSL is experimental, and the provided API may change in future versions.

...

Since TeamCity 2017.2, when you switch a project to the Kotlin DSL, project and build configuration settings can be editied in the web UI. All changes made in a project via the UI will be presented as a patch in the Kotlin format which will be checked in under the project patches directory. Moreover, if after switching to the Kotlin DSL you did not change the generated DSL files, the UI changes will be applied in place, no patch will be generated at all. 

Note
TeamCity does not enable editing of the project right after the switch to Kotlin format. TeamCity needs to detect its own commit in the repository, and only after that editing will be enabled. Usually it takes a minute or two.

The existing projects in Kotlin need to switch to the new v2017_2  DSL API version. 

Notes on patches

When settings are edited in the UI, TeamCity generates a patch script in the dedicated package ‘<project id>.patches.(buildTypes|templates|vcsRoots|projects)’; the script name is <uuid of the entity>.kts. The patch script is executed after regular dsl scripts, it applies the UI changes to the generated settings if the settings are in the expected state. For example, if you change the build configuration name, the patch script checks that the name produced by the regular script isn’t changed and then updates the name. Once the patch script is committed to the settings repository, you can apply its changes to your settings. After that, the patch script should be removed. 

Example of a patch changing name of a build configuration: 

Code Block
/*
This patch script was generated by TeamCity on settings change in UI.
To apply it, change the buildType with uuid '5dc8e147-11dc-4cb6-83aa-cfdb2595797d'
accordingly and delete the patch script.
*/
changeBuildType("5dc8e147-11dc-4cb6-83aa-cfdb2595797d") {
    check(name == "Old Name") {
        "Unexpected name: '$name'"
    }
    name = "New Name"
}

 

Limitations 

Some of the UI actions will still be disabled for projects in Kotlin: 

  • Changing the id (for all entities - project, build configuration, template or VCS root)

  • move actions

  • editing of the versioned settings VCS root

  • versioned Settings configuration change in a project

...

Since TeamCity 2018.1, there is an option to generate portable DSL scripts. When the option is enabled, TeamCity generates a single .teamcity/settings.kts script defining a project hierarchy. All ids in portable scripts are relative to the project where versioned settings were enabled, they are expanded to the full id during DSL execution. Portable DSL has no restriction on a directory name for project settings. All settings can be moved to a single file.
Since portable scripts do not contain absolute ids, they can be used several times on the same or different TeamCity servers.
Since there is no restriction on directory name for project settings, branching and merging configs becomes easier.
Portable DSL for a simple project looks like this:

 

Code Block
languagegroovy
import jetbrains.buildServer.configs.kotlin.v2018_1.*
import jetbrains.buildServer.configs.kotlin.v2018_1.buildSteps.script
import jetbrains.buildServer.configs.kotlin.v2018_1.vcs.GitVcsRoot

project {
    buildType(Build)
    vcsRoot(Sources)
}


object Build : BuildType({
    name = "Build"
    vcs {
        root(Sources)
    }
    steps {
        script {
            scriptContent = """
                #!/bin/bash
                echo 'Hello World'
            """.trimIndent()
        }
    }
})


object Sources : GitVcsRoot({
    url = "https://github.com/me/project.git"
})

 

Common Problems

New URL of Settings VCS Root 

...

MyProjectId.Project)

This is the entry point for project settings generation. Basically, it represents a Project instance which generates project settings.

Project.kt

The Project.kt file looks as follows: 

Code Block
package MyPackage
import jetbrains.buildServer.configs.kotlin.v2018_1.*
import jetbrains.buildServer.configs.kotlin.v2018_1.Project

object Project : Project({
   uuid = "05acd964-b90f-4493-aa09-c2229f8c76c0"
   id("MyProjectId")
   parentId("MyParent")
   name = "My project"
   ...
})

where:

  • id is the absolute id of the project, the same id we'll see in browser address bar if we navigate to this project
  • parentId is the absolute id of a parent project where this project is attached
  • uuid is some unique sequence of characters.
    The uuid is a unique identifier which associates a project, build configuration or VCS root with its data. If the uuid is changed, then the data is lost. The only way to restore the data is to revert the uuid to the original value. On the other hand, the id of an entity can be changed freely, if the uuid remains the same. This is the main difference of the non-portable DSL format from portable. The portable format does not require specifying the uuid, but if it happened so that a build configuration lost its history, one has to reattach it again via the web interface. 

    Note

    If the build history is important, it should be restored as soon as possible: after the deletion, there is a configurable timeout (5 days by default) before the builds of the deleted configuration are removed during the build history clean-up.

In case of non-portable DSL, patches are stored under the project patches directory of .teamcity:

  • pom.xml
  • <project id>/settings.kts
  • <project id>/Project.kt
  • <project id>/patches/<uuid>.kt

Working with patches is the same as in portable DSL: you need to move the actual settings from the patch to your script and remove the patch.

Debugging Maven ‘generate’ Task

The pom.xml file provided for a Kotlin project has the generate task, which can be used to generate TeamCity XML files locally from the Kotlin DSL files. This task supports debugging. If you’re using IntelliJ IDEA, you can easily start debugging of a Maven task:

  1. Navigate to View | Tool Windows | Maven Projects. The Maven Projects tool window is displayed.
  2. Locate the task node: Plugins | teamcity-configs | teamcity-configs:generate, the Debug option is available in the context menu for the task:

    Image Added

Ability to Use External Libraries

You can use external libraries in your Kotlin DSL code, which allows sharing code between different Kotlin DSL-based projects.

To use an external library in your Kotlin DSL code, add a dependency on this library to the .teamcity/pom.xml file in the settings repository and commit this change so that TeamCity detects it. Then, before starting the generation process, the TeamCity server will fetch the necessary dependencies from the Maven repository, compile code with them, and then start the settings generator. 

View DSL in UI 

There is an option in the UI to help those using Kotlin-based DSL. When viewing your build configuration settings in the UI, you can click View DSL in the sidebar: the DSL representation of the current configuration will be displayed and the setting being viewed (e.g. a build step, a trigger, dependencies) will be highlighted. To go back, click Edit in UI.

FAQ and Common Problems

Anchor
nonUniformIDs
nonUniformIDs

Why portable DSL requires the same prefix for all IDs?

In TeamCity projects, templates, build configurations and VCS roots all have unique IDs. These IDs usually look like:

<parent project id>_<entity id>

Since these IDs must be unique, there cannot be two different entities in the system with the same ID. 

But one of the reasons why portable DSL is called portable is because the same settings.kts script can be used to generate settings for two different projects even on the same server.

In order to achieve this and overcome IDs uniqueness, TeamCity operates with relative IDs in portable DSL scripts. These relative IDs do not have parent project ID prefix in them. So when TeamCity generates portable Kotlin DSL scripts, it has to remove the parent project ID prefix from the IDs of all of the generated entities.

But this will not work if not all of the project entities have this prefix in their IDs. In this case the following error can be shown:

Build configuration id '<some id>' should have a prefix corresponding to its parent project id: '<parent project id>_'

To fix this problem, one should change the ID of the affected entity. If there are many such IDs, Bulk edit IDs project action can be used to change all of them at once.

How to Add .teamcity as a New Module to a Project?

Question: How to add the .teamcity settings directory as a new module to an existing project in IntelliJ IDEA?

Solution: In your existing project in  IntelliJ IDEA:

  • Go to File | Project Structure, or press Ctrl+Shift+Alt+S.
  • Select Modules under the Project Settings section.
  • Click the plus sign, select Import module and choose the folder containing your project settings. Click Ok and follow the wizard.
  • Click Apply. The new module is added to your project structure.

New URL of Settings VCS Root (Non portable format)

Problem: I changed the URL of the VCS root where settings are stored in Kotlin and now TeamCity cannot find any settings in the repository at the new location.

Solution:    

  • Fix the URL in the Kotlin DSL in the version control and push the fix.

...

  • Disable versioned settings to enable the UI.

...

  • Fix the URL in the VCS root in the UI.

...

...

  •  with same VCS root and the Kotlin format again, TeamCity will detect that the repository contains the .teamcity

...

  •  directory and ask you if you want to import settings.

...

...

  • Choose to import settings.

How to Read Files in Kotlin DSL

Problem: I want to generate a TeamCity build configuration based on the data in some file residing in the VCS inside the .teamcity directory

Solution: TeamCity executes DSL with the .teamcity as the current directory, so files can be read using the paths relative to the .teamcity directory e.g. File("data/setup.xml"). Files outside the .teamcity directory are not accessible to Kotlin DSL.

Kotlin DSL API documentation is not initialized yet

 Problem: 

  • app/dsl-documentation/index.html on our Teamcity server displays "Kotlin DSL API documentation is not initialized yet"
  •  OutOfMemoryError during TeamCity startup with `org.jetbrains.dokka` in stack trace

 Solution: set the internal property teamcity.kotlinConfigsDsl.docsGenerationXmx=768m 

Passwords-Related Questions

Prior to TeamCity 2017.1

Problem: I  I do not want the passwords passwords to be committed to the VCS, even in a scrambled form.

SolutionYou can move the passwords to the parent project whose settings are not committed to a VCS.

 

Problem: I want to change passwords after the settings have been generated.

SolutionThe passwords will have to be scrambled manually using the following command in command in the  maven maven project with settings:

 

Code Block
mvn -Dtext=<text to scramble> org.jetbrains.teamcity:teamcity-configs-maven-plugin:scramble

 

Since TeamCity 2017.1

Solution: Use  Use tokens instead of passwords. Please refer to the the related section.

How to read files in Kotlin DSL

Problem: I want to generate a TeamCity build configuration based on the data in some file residing in the VCS inside the .teamcity directory

Solution: TeamCity executes DSL with the .teamcity as a current directory, so files can be read using the paths relative to the .teamcity directory e.g. File("data/setup.xml"). Files outside the .teamcity directory are not accessible to Kotlin DSL.

 

See also:

Panel
bgColor#FFFFFF
borderStyledashed

Administrator's Guide: Storing Project Settings in Version Control