Skip to end of metadata
Go to start of metadata

Besides storing settings in version control in XML format, TeamCity also allows storing the settings in the DSL (based on the Kotlin language). Using the version control-stored DSL enables you to define settings programmatically, without the need to use XML files. Since Kotlin is statically typed, you automatically receive the auto-completion feature in IntelliJ IDEA which makes discovery of available API options a much simpler task.

Note that once you enable storing of the settings in DSL, you no longer can use TeamCity UI to edit affected project's settings.
Also, all the security implications apply.

Check also our blog post series on Kotlin DSL.

On this page:

 

Advantages of Kotlin DSL vs XML

The Kotlin settings format provides higher level of abstraction, and better code reuse.

  • Build configurations often share settings and need to be different in few aspects only. For example, when you need to test your software against different platforms, let's take an ORM framework to be tested against different databases, then the builds running tests are mostly the same, the only difference is the part configuring a database connection.With Kotlin, you can extract the common part and reuse it across all builds which ensures that the common part does not change and makes maintaining builds easier.
  • Kotlin also allows you to describe your setup in terms of your domain. For example, TeamCity provides generic requirements for build agents and you can define requirements specific to your installation:

    then you can use them in a build configuration:

Getting Started with Kotlin DSL

Learning Kotlin

The Kotlin tutorial helps you learn most Kotlin features in a few hours.

Downloading Settings in Kotlin

To start working with Kotlin DSL, go to the project settings page, click the Actions menu and select Download settings in kotlin format. This will generate a .zip archive with project settings which can be used in your sandbox project.

When you have familiarized yourself with Kotlin, you can use the Kotlin-based DSL for project settings in production.

Using Settings in Kotlin in Production

  1. Enable versioned settings for your project.
  2. Select kotlin as the format.
  3. Commit the settings to your version control.  

    Icon

    Prior to TeamCity 2017.2, as soon as a project is switched to Kotlin, editing of its settings is disabled in the UI (except a few pages: Versioned settings, Maven Settings, SSH keys and Meta-runners).  If you need to enable editing, switch versioned settings to XML format or disable versioned settings.

    Since TeamCity 2017.2, project and build configuration settings will be available for editing from the web UI. See details below.

  4. Once you apply the versioned settings to your project, TeamCity will generate necessary Kotlin files for this project and check them in to the specified repository under the .teamcity directory. If this repository already contains project settings in the XML format, they will be preserved, but TeamCity will no longer use them:  once you get familiar with Kotlin files, these xml files can be removed.

DSL Documentation

Since TeamCity 2017.2 after Kotlin DSL is enabled for a project, the TeamCity server provides html documentation for the core DSL API as well as the API provided by installed TeamCity plugins. The documentation is accessible via the link on the 'Versioned Settings' project tab in the UI or by running the mvn -U dependency:sources command in the IDE. 

Working with Kotlin DSL

The .teamcity folder contains a maven project consisting of pom.xml file and  the <Project Id> directory with the settings.kts and Project.kt files, as well as buildTypes, pluginData, and vcsRoots subdirectories. If the project contains subprojects, they will be placed in the directories with corresponding names.

Opening Project in IDE

Open the pom.xml in your favourite IDE and check that the URL of the teamcity-server repository points to your TeamCity.

If the server url is correct and your TeamCity is up and running, all necessary dependencies will be resolved automatically and you can start working with Kotlin DSL right away.

 

Icon

The TeamCity server does not use the pom.xml file; it is used by the IDE to download required dependencies and to provide code completion as well as detect compilation errors, so any changes you will make to this POM file won’t have any effect during the DSL settings application to the server. You can also delete this file.

Debugging Maven ‘generate’ Task

The pom.xml file provided for a Kotlin project has the 'generate' task, which can be used to generate TeamCity XML files locally from the Kotlin DSL files. Since TeamCity 2017.2 this task supports debugging. If you’re using IntelliJ IDEA, you can easily start debugging of a Maven task, see screenshot:

 

Ability to Use External Libraries

Since TeamCity 2017.2 You can now use external libraries in your Kotlin DSL code, which allows sharing code between different Kotlin DSL-based projects. To use an external library in your Kotlin DSL code, add a dependency on this library into the .teamcity/pom.xml file in the settings repository and commit this change so that TeamCity detects it. Then, before starting the generation process, the TeamCity server will fetch the necessary dependencies from the Maven repository, compile code with them, and then start  the settings generator. 

Essential Files

settings.kts

The settings.kts script is an entry point for defining a single TeamCity project. TeamCity looks for the settings.kts file in a project directory and runs it if it is found, so the script name must not be changed and its package must be the same as the project's id.

The script should contain a single call to the project() function with a Project instance or the init function as an argument, you can also specify both of them to refine the specified Project in the init function.

Project.kt

This file contains the project definition.

VCS roots, Build types, and Templates of this project must be registered inside the project using the vcsRoot(), buildType(), and template() methods respectively.

Subprojects can be defined either in their own settings.kts or by calling the subProjects() method in this project.

As this is a Kotlin code, you are free to  add conditions, loops, methods, classes, etc.

Dedicated DSL for plugins

 Since TeamCity 2017.2 dedicated DSL code is provided for new plugins bundled with TeamCity (Docker, .NET CLI) and coverage plugins. Here is typical DSL code for the Docker-Compose build step:

 

Required Identifiers

All TeamCity entities - project, build configuration, VCS root and template have a so-called uuid. The uuid is an identifier which can be used to uniquely distinguish this specific entity from others. If at some point the entity receives a new uuid, it is considered a new entity by TeamCity. For instance, if the build configuration uuid changes, the builds history of this build configuration will become empty (if you change it back before cleanup, the history will be restored). The same rule applies to a project - if its uuid changes, the project will lose its investigations and muted tests. We suggest selecting some naming scheme for uuids in your project and never changing them. 

There is also the id field (extId prior to TeamCity 2017.2) which is mandatory. The id is used to represent the project ID, build configuration (or template) ID / VCS root ID in the web UI; it is also used in the DSL to reference other entities. Besides, it is used in various links (such as build configuration links) and REST API.  

Making Changes to Project Settings

Once a project is switched to the Kotlin DSL format, we recommend the following approach for changing the project settings:

  1. Set up unit tests.

  2. Create a dedicated project on your TeamCity server to run these unit tests.

  3. Create a separate branch/repository and run tests on your changes there. 

    Icon

    Avoid pushing changes right into the branch monitored by TeamCity without running any tests, as it can affect your project and all of the users currently working with this project  

  4. If the tests are green, merge the changes into the default settings branch. 

    Icon

    To merge the branch automatically if the build is green, consider using the TeamCity automatic merge build feature.

Creating New Project

You can create a new project via DSL inside a parent project which already has uses settings in the Kotlin format. A new project can be added either in its own directory with settings.kts or using the subProject() method in the parent project. 

Apply your changes as described in the section below.

Applying Changes to TeamCity

 When you have made changes to your Kotlin DSL code, сommit your code into the .teamcity directory containing your project settings. 

 

Icon

Prior to TeamCity 2017.2, no external resources are supported and since pom.xml is not used on the server, any libraries that you might want to add will be ignored during DSL processing. 

Since TeamCity 2017.2, it is possible to use external libraries in your Kotlin DSL code. See details below.

TeamCity will apply the changes to  configuration as follows: 

  1. TeamCity will detect the commit, execute Kotlin DSL files and produce new configuration files. Since internally TeamCity operates with XML, TeamCity will execute DSL files in a secure sandbox to produce XML configuration files. These XML files then will be applied to the existing project.
  2. TeamCity will apply the new configuration files to the project effectively changing its configuration. In case of any problems (compilation failures, runtime errors, etc.), new changes will not be applied, and the current project settings will be preserved on the server and the error message will appear under the Versioned Setting tab in project settings.
  3. The builds will run with the new settings.

 Note that:

  • The Kotlin script is executed on the server, and since this is a potentially dangerous operation, the script is executed in the sandbox. It cannot modify the file system except the place where it is executed, it cannot run other programs, cannot use reflection, and so on.
  • The Kotlin DSL is experimental, and the provided API may change in future versions.

Editing Kotlin Settings in UI

Since TeamCity 2017.2, when you switch a project to the Kotlin DSL, project and build configuration settings can be editied in the web UI. All changes made in a project via the UI will be presented as a patch in the Kotlin format which will be checked in under the project patches directory. Moreover, if after switching to the Kotlin DSL you did not change the generated DSL files, the UI changes will be applied in place, no patch will be generated at all. 

Icon
TeamCity does not enable editing of the project right after the switch to Kotlin format. TeamCity needs to detect its own commit in the repository, and only after that editing will be enabled. Usually it takes a minute or two.

The existing projects in Kotlin need to switch to the new v2017_2  DSL API version. 

Notes on patches

When settings are edited in the UI, TeamCity generates a patch script in the dedicated package ‘<project id>.patches.(buildTypes|templates|vcsRoots|projects)’; the script name is <uuid of the entity>.kts. The patch script is executed after regular dsl scripts, it applies the UI changes to the generated settings if the settings are in the expected state. For example, if you change the build configuration name, the patch script checks that the name produced by the regular script isn’t changed and then updates the name. Once the patch script is committed to the settings repository, you can apply its changes to your settings. After that, the patch script should be removed. 

Example of a patch changing name of a build configuration: 

 

Limitations 

Some of the UI actions will still be disabled for projects in Kotlin: 

  • Changing the id (for all entities - project, build configuration, template or VCS root)

  • move actions

  • editing of the versioned settings VCS root

  • versioned Settings configuration change in a project

Common Problems

New URL of Settings VCS Root 

 Problem: I changed the URL of the VCS root where settings are stored in Kotlin and now TeamCity cannot find any settings in the repository at the new location.

Solution:  

  1. Fix the URL in the Kotlin DSL in the version control and push the fix.
  2.  Disable versioned settings to enable the UI.
  3.  Fix the URL in the VCS root in the UI.
  4.  Enable versioned settings with same VCS root and the Kotlin format again, TeamCity will detect that the repository contains the .teamcity directory and ask if you want to import settings. 
  5. Choose to import settings.

Passwords-Related Questions

Prior to TeamCity 2017.1

Problem: I do not want the passwords to be committed to the VCS, even in a scrambled form.
Solution: You can move the passwords to the parent project whose settings are not committed to a VCS.

Problem: I want to change passwords after the settings have been generated.
Solution:  The passwords will have to be scrambled manually using the following command in the  maven project with settings:

 

Since TeamCity 2017.1 

Solution: Use tokens instead of passwords
For projects with settings in kotlin-based DSL with the option to store security data outside the VCS enabled, it is possible to use tokens instead of passwords.  Tokens can be generated via the "Generate Token for password" action available in the Project -> Actions menu. This action can be useful when project settings are generated via DSL (because the UI is disabled in this case) or if someone wants to change the password not via the user interface, but via a commit into VCS.

At this time passwords are not inheritable by projects hierarchy. If a setting in a project  (a VCS root, OAuth connection, cloud profile) requires a password, the token generated for this password can be used in this project only. For instance, it is not possible to take a generated token and use it in a similar setting in a subproject. A new token should be generated in this case.
If you need to use a secure value in the nested projects, consider adding a password parameter with the secure value and using a reference to the parameter in the nested projects.

 

How to read files in Kotlin DSL

Problem: I want to generate a TeamCity build configuration based on the data in some file residing in the VCS inside the .teamcity directory

Solution: TeamCity executes DSL with the .teamcity as a current directory, so files can be read using the paths relative to the .teamcity directory e.g. File("data/setup.xml"). Files outside the .teamcity directory are not accessible to Kotlin DSL.

 

See also:










 

 

 

 

 

 


 

 

 

 

 

 

 

 

 

 

  • No labels

4 Comments

  1. How do I apply Kotlin DSL where the project hasn't been created previously? I want to create new projects via DSL, since the whole point of us attempting to use the DSL was to quickly define projects programmatically and create them WITHOUT having to interact with the UI. Having to create a project via the REST API and then push changes via XML or the API in order to add a repository to pull the Kotlin DSL seems to be defeating the purpose of using the DSL. We might as well just build our own interface to the API.

    1. You can create a new project via DSL inside a parent project which already uses settings in Kotlin format. Just define a new project and commit it into repository, TeamCity will pick it up. A new project can be added either in its own directory with settings.kts or using the subProject() method in the parent project. Let me know if it helps.

  2. Changing the password after doesn't work as documented. It always produces the same string.

    The value that needs to be changed is in the pom.xml but that is checked in and that only takes one value at a time which is ok but extremely cumbersome.

    Is there any other alternative?

    1. Thanks for reporting it, I filed the issue https://youtrack.jetbrains.com/issue/TW-48732 . It will be fixed in TeamCity 10.0.5. As a workaround you can temporarily change TeamCity version in pom.xml to 10.0.5-SNAPSHOT, with that scrambling should work.