Icon

You are viewing the documentation of TeamCity 10.x and 2017.x, which is not the most recently released version of TeamCity.
View this page in the latest documentation or refer to the listing to choose the documentation corresponding to your TeamCity version.

 

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Database Note:
When using the server extensively, the database performance starts to play a greater role.
For reliability and performance reasons you should use external database.
Please see the notes on choosing external database.
The database size requirements naturally vary based on the amount of data stored (number of builds, number of tests, etc.) | The active server database usage can be estimated at several gigabytes of data per year.

...

A general example of hardware configuration capable to handle up to 100 concurrently running builds and running only TeamCity server can be:
Server-suitable modern multicore multi-core CPU, 8Gb of memory, fast network connection, fast and reliable HDD, fast external database access

...

Anchor
maxAgentsNumber
maxAgentsNumber
The latest TeamCity version is known to work well with up to 300 with 500+ build agents (300 500 concurrently running builds actively logging build run-time data). In synthetic tests the server was functioning OK with as many as 500 1000 concurrent builds (the server with 8 cores, 32Gb of total memory running under Linux, and MySQL server running on a separate comparable machine). The load on the server produced by each build depends on the amount of data the build logs produces (build log, tests number and failure details, inspections/duplicates issues number, etc.). Keeping the amount of data reasonably constrained (publishing large outputs as build artifacts, not printing those into standard output; tweaking inspection profiles to report limited set of the most important inspection hits, etc.) will help scale the server to handle more concurrent builds.
If you need much more agents/parallel builds, it is recommended to use several nodes setup. If a substantially large amount of agents is required, it is recommended to consider using several separate TeamCity instances and distributing the projects between them. We constantly work on TeamCity performance improvements and are willing to work closely with organizations running large TeamCity installations to study any performance issues and improve TeamCity to handle larger loads. See also a related post on the maximum number of agents which TeamCity can handle

See also a related post: description of a substantial TeamCity setup.

Configuring TeamCity Server for Performance

Here are some recommendations to tweak TeamCity server setup for better performance. The list for production server use is a prerequisite:

  • Regularly review reported Server Health reports (including hidden ones)
  • Use a separate reverse proxy server (e.g. NGINX) to handle HTTPS
  • Use a separate server for the external database and monitor the database performance
  • Monitor the server's CPU and IO performance, increase hardware resources as necessary (see also hardware notes)
  • Make sure clean-up is configured for all the projects with a due retention policy, make sure clean-up completely finishes regularly (check Administration / Clean-Up page)
  • Consider ensuring good IO performance for the <TeamCity Data Directory>/system/caches directory, e.g. by moving it to a separate local drive (or storing on a local drive you choose to store the TeamCity Data Directory on a network storage)
  • Regularly archive obsolete projects
  • Regularly review the installed not bundled plugins and remove those not essential for the server functioning
  • Consider using agent-side checkout whenever possible
  • Make sure the build logs are not huge (tens megabytes at most, better less than 10 Mb)
  • If lots VCS roots are configured on the server, consider configuring repository commit hooks instead of using polling for changes or at least increase VCS polling interval to 300 seconds or more)
  • If the server is often used by large number of users (e.g. more than 1000), consider increasing UI refresh intervals
  • When regularly exceeding 500 concurrently running builds which log a lot of data, consider using Several Nodes Setup

Retrieve Administrator Password

...

Allocating 1 GB for the redo log (see the table below) and undo files is sufficient in most cases.

...

MS SQL Server: it is NOT recommended to use the JTDS jTDS driver: it does not work with nchar/nvarchar, and to preserve unicode streams it may cause queries to take a long time and consume a lot of IO. Also see the information on redo log in the Microsoft Knowledge base. If you use jTDS, please migrate.

MySQL: the query optimizer might be inefficient: some queries may get a wrong execution plan causing them to take a long time and consume huge IO.

...

We've seen patterns of having an agent per each 20 build configurations (types of builds). Or a build agent per 1-2 developers.

See also notes on maximum supported number of agents.

...

TeamCity does not provide specific support for full replication/redundancy/high availability (HA) or clustering solutions. The only related ability is to set up separate node for processing running builds.
However, to address fast disaster recovery scenarios, it supports active - failover (hot cold standby) approach: the data that the TeamCity server uses can be replicated and a solution put in place to start a new server using the same data if the currently active server malfunctions.

...

TeamCity agents farm can be reused between the main and the failover servers. Agents will automatically connect to the new server if you make the failover server to be resolved via the old server DNS name and agents connect to the server using the DNS name. See also information on switching from one server to another.
If appropriate, the agents can be replicated just as the server. However, there is no need to replicate any TeamCity-specific data on the agents except for the conf\buildAgent.properties file as all the rest of the data can typically be renewed from the server. In case of replicated agents farm, the replica agents just need to be connected to the failover server.

In case of two servers installations for redundancy purposes, they can use the same set of licenses as only one of them is running at any given moment.

TeamCity Security Notes

The following notes are provided for your reference only and are not meant to be complete or accurate in their entirety.

TeamCity is developed with security concerns in mind, and reasonable efforts are made to make the system invulnerable to different types of attacks. We work with third-parties on assessing TeamCity security using security scanners and penetration tests.
We aim to promptly address newly discovered security issues in the nearest bug-fix releases for the most recent TeamCity major version.
However, the general assumption and recommended setup is to deploy TeamCity in a trusted environment with no possibility for it to be accessed by malicious users. 

Along with these guidelines, please review notes on configuring the TeamCity server for production use. For the list of disclosed security-related issues, see our public issue tracker and the "Security" section in the release notes.
It is recommended to upgrade to newly released TeamCity versions as soon as they become available as they can contain security-related fixes.

Note that since TeamCity 2017.2 the TeamCity Windows installer modifies permisisons of the TeamCity installation directory not to use inheritable permissions and explicitly grants access to the directory to the Administrators user group and the accrount under which the service is configured to run.
It is strongly recommended to restrict permissions to the TeamCity Data Directory in the same way.

Additional security-realted settings

Consider adding the "teamcity.installation.completed=true" line into the <TeamCity Home Directory>\conf\teamcity-startup.properties file - this will prevent the server from creating an administrator user if no such user is found

TeamCity has no built-in protection against DoS attack: high rate of requests can overload the server and make it not responsive. If your TeamCity instance is deployed in the environment which allows such service abuse, implement the protection on the reverse proxy level.

Version-specific settings

Some of the security-related checks and protections are implemented but not turned on by default to prevent compatibility issues. Here is the list of the related settings to enhance security of the server, provided you run 2017.2.3+ version:

  • update the IDE plugins for all users to the latest version and then add the "teamcity.uploadPersonalPatch.requireAuthorization=trueinternal property;
  • add "teamcity.artifacts.restrictRequestsWithArtifactReferer=trueinternal property.

hidden-data
Project administrator can run arbitrary code on the server: https://youtrack.jetbrains.com/issue/TW-50054
To workaround one can add the following properties:
teamcity.hg.customClonePathEnabled=false
teamcity.hg.customHgPathEnabled=false
teamcity.hg.customConfigEnabled=false
teamcity.git.customClonePathEnabled=false
teamcity.server.git.executable.path=git  //???
teamcity.perforce.customP4Path=p4
teamcity.hubPlugin.export.options.enabled=true //???

Short checklist (see below for full notes)

  • You are running the latest released TeamCity version and are ready to upgrade to the newly released versions within weeks
  • Access to the TeamCity web interface is secured using HTTPS (e.g. with the help a proxying server like NGINX). Best practices for securing web application are employed for the TeamCity web interface
  • The TeamCity server machine does not run agents (at least under the user permitted to read the TeamCity server's home directory and TeamCity Data Directory)
  • TeamCity server and agents processes are run under limited users with minimal required permissions. Installation directories are readable and writable only by a limited set of OS users. The conf\buildAgent.properties file and server logs as well as the Data Directory are only readable by OS users who represent administrators of the services, because reading those locations may allow taking over the agent or server respectively.
  • Guest user and user registration is disabled or roles are reviewed for guest and the All Users group
  • TeamCity users with administrative permissions have non-trivial passwords
  • If you have external authentication configured (such as LDAP), the built-in authentication module is disabled
  • Passwords are not printed into the build log, not stored in build artifacts, nor are they stored in non-password parameters

The following notes are provided only for your reference and are not meant to be complete or accurate in their entirety.

TeamCity is developed with security concerns in mind, and reasonable efforts are made to make the system not vulnerable to different types of attacks. We work with third-parties on assessing TeamCity security using security scanners and penetration tests.
We aim to promptly address newly discovered security issues in the nearest bug-fix releases for the most recent TeamCity major version.
However, the general assumption and recommended setup is to deploy TeamCity in a trusted environment with no possibility for it to be accessed by malicious users.

Anchor
security-rolesRelated
security-rolesRelated
Security-related Risks Evaluation

Here are some notes on different security-related aspects:

  • man-in-the middle concerns
    • between the TeamCity server and the user's web browser: It is advised to use HTTPS for the TeamCity server. During login, TeamCity transmits the user login password in an encrypted form with a moderate encryption level.
    • between a TeamCity agent and the TeamCity server: see the section.
    • between the TeamCity server and other external servers (version control, issue tracker, etc.): the general rules apply as for a client (the TeamCity server in the case) connecting to the external server, see the guidelines for the server in question.
  • users that have access to the TeamCity web UI: the specific information accessible to the user is defined via TeamCity user roles.
  • users who can change the code that is used in the builds run by TeamCity (including committers in any branches/pull requests if they are built on TeamCity): the users have the same permissions as the
    • can do everything what can do the system user under whom the TeamCity agent is running
    . Have
    • , have access to OS resources and other applications installed on the
    same machine. Can
    • agent machines where their builds can run.
    • can access and change source code of other projects built on the same agent, modify the TeamCity agent code, publish any files as artifacts for the builds run on the agent (which means the files can be then displayed in the TeamCity web UI and expose web vulnerabilities or can be used in other builds), etc.
    Can
    • can impersonate a TeamCity agent (run a new agent looking the same to the TeamCity server).
    Also, the users
    • can do everything that users with the "View build configuration settings" permission for all the projects on the server can do (see below).
    It
    • can retrieve settings of the build configurations where the builds are run, including the values of the password fields.
    • can download artifacts from any build on the server.
      Hense, it is advised to run TeamCity agents under users with only necessary set of permissions and use the agent pools feature to ensure that projects requiring a different set of access are not built on the same agents.
  • users with the "View build configuration settings" permission (the "Project developer" TeamCity role by default) can view all the projects on the server, but since TeamCity 9.0 there is a way to restrict this, see details in the corresponding issue TW-24904.

  • users with the "Edit project" permission (the "Project Administrator" TeamCity role by default) in one project, by changing settings can retrieve artifacts and trigger builds from any build configuration they have only the view permission for (TW-39209). The users might also be able to make the TeamCity server run any executable located on the server.
  • users with the "Change server settings" permission (the "System Administrator" TeamCity role by default): It is assumed that the users also have access to the computer on which the TeamCity server is running under the user account used to run the server process. Thus, some operations like server file system browsing can be accessible by the usersthe users can get full access to the machine under that OS user account: browse file system, change files, run arbitrary commands, etc.
  • TeamCity server computer administrators: have full access to TeamCity stored data and can affect TeamCity executed processes. Passwords that are necessary to authenticate in external systems (like VCS, issue trackers, etc.) are stored in a scrambled form in TeamCity Data Directory and can also be stored in the database. However, the values are only scrambled, which means they can be retrieved by the users who have access to the server file system or database.
  • Users who have read access for TeamCity server logs (TeamCity server home directory) can escalate their access to TeamCity server administrator
  • Users who have read access to TeamCity Data Directory can access all the settings on the server, including configured passwords
  • Users who have read access to the build artifacts in TeamCity Data Directory (<TeamCity Data Directory>\system\artifacts) get the same permissions as users with the "View build runtime parameters and data" permission (in particular, with access to the values of all the password parameters used in the build)
  • TeamCity agent computer administrators: same as "users who can change code that is used in the builds run by TeamCity".
  • TeamCity users with administrative permissions have non-trivial passwords
  • When storing settings in VCS is enabled:
    • any user who can access the settings repository (including users with "View file content" permission for the build configurations using the same VCS root) can see the settings and retrieve the actual passwords based on their stored scrambled form
    • any user who can modify settings in VCS for a single build configuration built on the server, via changing settings can retrieve artifacts and trigger builds from any build configuration they have only view permission for (TW-39192).
    • users who can customize build configuration settings on a per-build basis (e.g. one who can run personal builds when versioned settings are set to "use settings from VCS") via changing settings in a build can retrieve artifacts and trigger builds from any build configuration they have only view permission for (TW-46065).
  • Other:
    • TeamCity web application vulnerabilities: the TeamCity development team makes a reasonable effort to fix any significant vulnerabilities (like cross-site scripting possibilities) once they are uncovered. Please note that any user that can affect build files ("users who can change code that is used in the builds run by TeamCity" or "TeamCity agent computer administrators") can make a malicious file available as a build artifact that will then exploit cross-site scripting vulnerability. (TW-27206)
    • TeamCity agent is fully controlled by the TeamCity server: since TeamCity agents support automatic updates download from the server, agents should only connect to a trusted server. An administrator of the server computer can force execution of arbitrary code on a connected agent.
    • Binaries of the agent plugins installed on the server are available to anyone who can access the server URL

Wiki Markup
{hidden-data}Security concerns of using cloud integration with a publicly accessible server{hidden-data}

...

TeamCity tries not to pass password values in the web UI (from a browser to the server) in clear text and uses RSA with 1024-bit key to encrypt them. However, it is recommended to use the TeamCity web UI only via HTTPS so this precaution should not be relevant.
TeamCity stores passwords in the settings (where the original password value is necessary to perform authentication in other systems) in a scrambled form. The scrambling is done using 3DES with a fixed key.

hidden-data
TeamCity stores user passwords when built-in authentication is used in a salted hashed (MD5) form.
Hidden because of https://youtrack.jetbrains.com/issue/TW-41757

...

This section covers the recommended setup of reverse-proxy servers installed in front of the TeamCity server web UI. Configuring HTTPS on the proxy level is recommended, but is out of the scope of these instructions - refer to the documentation of the proxy server for that.

Consider the example:
TeamCity server is installed at URL (local URL): http://teamcity.local:8111/tc
It is visible to the outside world as URL (public URL): http://teamcity.public:400/tc

You need to set up a reverse proxy (see Proxy Server Setup below) and also configure TeamCity's bundled Tomcat server (see TeamCity Tomcat Configuration below) to make sure TeamCity "knows" the actual absolute URL used by the client to access the resources. These URLs are then used to generate absolute URLs in client redirects and other responses.

Note: An internal TeamCity server should work under the same context (i.e. part of the URL after the host name) as it is visible from outside by an external address. See also TeamCity server context changing instructions. If you still need to run the server under a different context, note that context changing proxy should conceal this fact from the TeamCity: e.g. it should map server redirect URLs as well as cookies setting paths to the original (external) context.

Proxy Server Setup

The proxy should be configured with generic web security in mind. e.g. headers like Referer and Origin should usually be passed to the TeamCity web application in the unmodified form.
Also, unknown HTTP request headers should be passed to the TeamCity web application unmodified. For example TeamCity relies on "X-TC-CSRF-Token" header added by the clients.

Apache

Versions 2.4.5+ are recommended. Earlier versions do not support the WebSocket protocol, so use the settings noted in the previous documentation version.

When using Apache, make sure to use the Dedicated "Connector" node approach for configuring TeamCity server.

...

For the NGINX configuration below, use the "RemoteIpValve" Approach for configuring TeamCity server.

...

Code Block
http {
    # ... default settings here
    proxy_read_timeout     1200;
    proxy_connect_timeout  240;
    client_max_body_size   0;    # maximum size of an HTTP request. 0 allows uploading large artifacts to TeamCity
 
	map $http_upgrade $connection_upgrade { # WebSocket support
        default upgrade;
        '' '';
    }
    
    server {
        listen       400; # public server port
        server_name  teamcity.public; # public server host name
    
        location /tc { # public context (should be the same as internal context)
            proxy_pass          http://teamcity.local:8111/tc; # full internal address
            proxy_http_version  1.1;
            proxy_set_header    Host $server_name:$server_port;
            proxy_set_header    X-Forwarded-Host $http_host;    # necessary for proper absolute redirects and TeamCity CSRF check
            proxy_set_header    X-Forwarded-Proto $scheme;
            proxy_set_header    X-Forwarded-For $remote_addr;
            proxy_set_header    Upgrade $http_upgrade; # WebSocket support
            proxy_set_header    Connection $connection_upgrade; # WebSocket support
        }
    }
}

Wiki Markup
{hidden-data}
some more NGINX params which might also relate:

client_header_timeout  10m;
client_body_timeout    10m;
send_timeout           10m;
proxy_max_temp_file_size 10240m;
{hidden-data}
Where client_max_body_size controls the maximum size of an HTTP request. It is set to 0 to allow uploading large artifacts to TeamCity.


Anchor
OtherServers
OtherServers
Other servers

Make sure to use a performant proxy with due (high) limits on request (upload) and response (download) size and timeouts (idle timeout should be hours to support server-side checkout). Since Teamcity 2017.1.3, setting a long idle timeout is not required, as TeamCity behind a proxy server keeps the connection to the agent alive while the patch is being created on the server.

It is recommended to use a proxy capable of working with the WebSocket protocol.

...

  • use a dedicated "Connector" node in the server configuration with hard-coded public URL details and make sure that the port configured in the connector is used only by the requests to the public URL configured.
  • configure proxy to pass due request headers: "Host" (original request host), "X-Forwarded-Proto" (original request protocol), "X-Forwarded-Port" (original request port) and configure "RemoteIpValve" for the TeamCity Tomcat configuration. 
Anchor
Proxy-Tomcat-Connector
Proxy-Tomcat-Connector
Dedicated "Connector" Node Approach

...

This approach can be used when the proxy server sets "Host", "X-Forwarded-Proto", "X-Forwarded-Port" request headers to the values of the original URL. Since TeamCity 2017.1 the "X-Forwarded-Host" header can be used to pass the original host name instead of preserving the value of the "Host" header. Also since TeamCity 2017.1 for the majority of functionality to work, it is not required to set up RemoteIpValve if only a single proxy is used (i.e. the mentioned headers store only a single value).

Also, while not critical for the most setups, this approach can be used to make sure the original client IP is passed correctly to the TeamCity server correctly. This is important for agents using bidirectional communication.

...

TeamCity can use proxy server for certain outgoing HTTP connections made by the TeamCity server to other services like issues trackers, etc. To

To point TeamCity at to your proxy server, you need to pass additional JVM options to :
Since TeamCity 2017.1.5 the following server internal properties are available (see the section below for previous version):

Code Block
languagetext
# For HTTP protocol
## The domain name or the IP address of the proxy host and the port: 
teamcity.http.proxyHost=proxy.domain.com
teamcity.http.proxyPort=8080
 
## The hosts that should be accessed without going through the proxy, usually internal hosts. Provide a list of hosts, separated by the '|' character. The wildcard '*' can be used:
teamcity.http.nonProxyHosts=localhost|*.mydomain.com)
 
## For an authenticated proxy add the following properties:
### Authentication type. "basic" and "ntlm" values are supported. The default is basic.
teamcity.http.proxyAuthenticationType=basic
### Login and Password for the proxy:
teamcity.http.proxyLogin=username
teamcity.http.proxyPassword=password
 
# For HTTPS protocol
## The domain name or the IP address of the proxy host and the port: 
teamcity.https.proxyHost=proxy.domain.com
teamcity.https.proxyPort=8080

## The hosts that should be accessed without going through the proxy, usually internal hosts. Provide a list of hosts, separated by the '|' character. The wildcard '*' can be used:
teamcity.https.nonProxyHosts=localhost|*.mydomain.com

## For an authenticated proxy add the following properties:
### Authentication type. "basic" and "ntlm" values are supported. The default is basic.
teamcity.https.proxyAuthenticationType=basic
### Login and Password for the proxy:
teamcity.https.proxyLogin=login
teamcity.https.proxyPassword=password

hidden-data
Seems like there is no reason to document these:### Alternatively, instead of proxyLogin&proxyPassword use:
teamcity.http.proxyAuthentication=formatted authentication string: the "username:password' for basic authentication; Windows NT credentials for NTLM authentication### Alternatively, instead of proxyLogin&proxyPassword use:
teamcity.https.proxyAuthentication=formatted authentication string: the "username:password' for basic authentication; Windows NT credentials for NTLM authentication
Anchor
before2017_1_5
before2017_1_5
 The alternative approach, which will work for any TeamCity version, is to pass additional space-delimited additional JVM options to the TeamCity server on the start up:

JVM arguments:

These JVM options should be spacedelimited when passed to the TeamCity server process
Code Block
title
-
-Dproxyset=true
-Dhttp.proxyHost=proxy.domain.com
-Dhttp.proxyPort=8080
-Dhttp.nonProxyHosts=domain.com
-Dhttps.proxyHost=proxy.domain.com
-Dhttps.proxyPort=8080
-Dhttps.nonProxyHosts=domain.com

 

Configure TeamCity Agent to Use Proxy To Connect to TeamCity Server

This section covers the configuration of a proxy server for TeamCity agent-to-server connections (since TeamCity 2017.1)

Excerpt
Note

Note that the forward proxy has to be configured not to cache any TeamCity requests; instead all requests must be proxied to the TeamCity Server, e.g. if you use Squid, add the following line to the squid.conf file:

Code Block
cache deny all

On the TeamCity agent side, specify the proxy can be specified to connect to TeamCity server using the following properties in the buildAgent.properties file:

Code Block
## The URLdomain name or the IP address of the proxy host and the port
teamcity.http.proxyHost=123.45.678.9
 
## The optional proxy port, leave blank to use the default port
teamcity.http.proxyPort=12348080


## If the proxy requires authentication, specify the login and password
teamcity.http.proxyLogin=login
 
## If the proxy requires authentication, specify the password
teamcity.http.proxyPassword=password
 

Note that the proxy has to be configured not to cache any TeamCity server responses; e.g. if you use Squid, add "cache deny all" line to the squid.conf file.

Install Multiple Agents on the Same Machine

See the corresponding section under agent installation documentation.

...

It's advised to try a new TeamCity version before upgrading your production server. The usual procedure is to create a copy of your production TeamCity installation, then upgrade it, try the things out, and, when everything is checked, drop the test server and upgrade the main one.
When you start the test server, remember to change the Server URL, disable Email and Jabber notifiers as well as other features on the new server.

Anchor
copy_server
copy_server

Create a Copy of TeamCity Server with All Data

One of the ways to In case you want to preserve the original server as well as the copy, make sure to check the lincensing considerations.

Create a Server Copy

You can create a copy of the server is to create a backup, then install using TeamCity backup functionality or manually.

Use TeamCity Backup
  1. Create a backup.
  2. Install a new TeamCity server of the same version that you

...

  1. are already runnning. Ensure that:
    - the appropriate environment is configured (see the notes below)

...

  1.  
    - the server uses its own TeamCity Data Directory and its own database

...

  1.  (check config/database.properties in the Data Directory)
  2. Restore the backup.

...

  1. Perform the necessary environment transfer.
Note

The new server won't get build artifacts and some other

...

data. If you need them, you will need to copy appropriate directories (e.g. the entire "artifacts" directory)

...

from .BuildServer/system

...

 from the original to the copied server. Make sure to finish copying over the artifacts before starting the new server.

Also consider transferring other settings to the new installation:

  • settings in <TeamCity home directory>\conf\teamcity-startup.properties file
  • any custom settings in <TeamCity home directory>\conf\server.xml
  • the OS user under whom the server is running
  • any user/machine settings like default ssh keys, cached VCS access credentials
  • any special settings or exceptions related to the machine in the network configuration, etc.
Copy Manually

If you do not want to use bundled backup functionality or need manual control over the process, here is a description of the general steps one would need to perform to manually create copy of the server:

  1. create Create a backup so that you can restore it if anything goes wrong,
  2. ensure Ensure the server is not running,
  3. either Either perform clean installation or copy the TeamCity binaries (TeamCity Home Directory) into a new place (the temp and work subdirectories can be omitted during copying). (warning) Use exactly the same TeamCity version. If you plan to upgrade after copying, perform the upgrade only after you have the existing version up and running.
  4. transfer relevant environment if it was specially modified for existing TeamCity installation. This might include:
    • if you run TeamCity with OS startup (e.g. Windows service), make sure all the same configuration is performed on the new machine
    • use the same TeamCity process launching options, specifically check/copy environment variables starting with TEAMCITY_.
    • use appropriate OS user account for running TeamCity server process with appropriately configured settings, global and file system permissions
    • transfer OS security settings if required
    • ensure any files/settings that were configured in TeamCity web UI are accessible; put necessary libraries/files inside TeamCity installation if they were put there earlier)
  5. copy TeamCity Copy TeamCity Data Directory. If you do not need the full copy, refer to the items below for optional itemsoptions.
    • .BuildServer/config to preserve projects and build configurations settings
    • .BuildServer/lib and .BuildServer/plugins if you have them
    • files from the root of .BuildServer/system if you use internal database and you do not want to perform database move.
    • .BuildServer/system/artifacts (optional) if you want build artifacts and build logs (including tests failure details) preserved on the new server
    • .BuildServer/system/changes (optional) if you want personal changes preserved on the new server
    • .BuildServer/system/pluginData (optional) if you want to preserve state of various plugins, build triggers and settings audit diff
    • .BuildServer/system/caches and .BuildServer/system/caches (optional) are not necessary to copy to the new server, they will be recreated on startup, but can take some time to be rebuilt (expect some slow down).
  6. Artifacts directory is usually large and if you need to minimize the downtime of the server in case of the server move, you can use the generic approach for copying the data: use rsync, robocopy or alike tool to copy the data while the original server is running. Repeat the sync run several times until the amount of data synced reduces. Run the final sync after the original server shut down. Alternative solution for the server move is to make the old data artifacts directory accessible to the new server and configure it as second location of artifacts. Then copy the files over from this second location to the main one while the server is running, restart the server after copying completion.
  7. create Create copy of the database that your TeamCity installation is using in new schema or new database server. This can be done with database-specific tools or with the bundled maintainDB tool by backing up database data and then restoring it.
  8. configure Configure new TeamCity installation to use proper TeamCity Data Directory and database (.BuildServer/config/database.properties points to a copy of the database)

...

  1. Perform the necessary environment transfer.
Info

If you want to do a quick check and do not want to preserve builds history on the new server you can skip step 6 (cloning database) and all items of the step 5 marked as optional.

Anchor
environment_transfer
environment_transfer
Environment transferring

Consider transferring relevant environment if it was specially modified for an existing TeamCity installation. This might include:

  • use the appropriate OS user account for running the TeamCity server process with properly configured settings, global and file system permissions
  • use the same TeamCity process launching options, specifically check/copy environment variables starting with TEAMCITY_
  • ensure any files/settings that were configured in the TeamCity web UI with absolute paths are accessible
  • if relying on the OS-level user/machine settings like default ssh keys, cached VCS access credentials, transfer them as well
  • consider replicating any special settings or exceptions related to the machine in the network configuration, etc.
  • If the TeamCity installation was patched in any way (GrrovyPlug plugin, native driver for MS SQL Server integrated security authentication), apply the same modifications to the installation copy
  • if you run TeamCity with the OS startup (e.g. Windows service), make sure the same configuration is performed on the new machine
  • review and tranfer settings in the <TeamCity home directory>\conf\teamcity-startup.properties file
  • consider any custom settings in <TeamCity home directory>\conf\server.xml

Anchor
copy_server_license
copy_server_license
Licensing issues

A single TeamCity license cannot be used on two running servers at the same time.

  • A copy of the server created for redundancy/backup purposes can use the same license as only one of the servers will be running at a time.
  • A copy of the server created for testing purposes requires an additional license. You can get the time-limited TeamCity evaluation license once from the official TeamCity download page
    If you need an extension of the license or you have already evaluated the same TeamCity version, please contact our sales department.
  • A copy of the server intended to run at the same time as the main one regularly/for production purposes requires a separate license.

Anchor
copy_server_checklist
copy_server_checklist

...

  • ensure the new server is configured to use another data directory and another database than the original server; check also "Artifact directories" setting on server's Global Settings;
  • change server unique id by removing "uuid" attribute from XML of <TeamCity Data Directory>\config\main-config.xml file before the first start;
  • ensure the same license keys are not used on several servers (more on licensing);
  • update Server URL on Administration | Global Settings page to the actual URL of the server;
  • check that you can successfully authenticate on the new server, use super user access if necessary;
  • check that VCS servers, issue tracker servers, email and Jabber server and other server-accessed systems are accessible;
  • check that any systems configured to push events to TeamCity server (like VCS hooks, automated build triggering, monitors, etc.) are updated to know about the new server;
  • review the list of installed plugins to determine if their settings need changes;
  • install new agents (or select some from the existing ones) and configure them to connect to the new server (using the new server URL);
  • check that clients reading from the server (downloading artifact, using server's REST API, NuGet feed, etc.) are reconfigured, if necessary.

...

  • disable Email, Jabber (in "Administration > " Notifier" sections) and possibly also custom notifiers or change their settings to prevent the new server from sending out notifications;
  • disable email verification (in "Administration > Authentication" section);
  • be sure not to run any builds which change (e.g. deploy to) production environments. This also typically includes Maven builds deploying to non-local repositories. You can prevent any builds from starting by pausing the build queue;
  • disable any plugins which push data into other non-copied systems based on TeamCity events (like commit status publishing);

    hidden-data
    https://youtrack.jetbrains.com/issue/TW-47362
    

  • disable cloud integration (so that it does not interfere with the main server);
  • disable functionality to store project settings in VCS: set teamcity.versionedSettings.enabled=false internal property;

    Wiki Markup
    {hidden-data}related issue: [https://youtrack.jetbrains.com/issue/TW-38304]{hidden-data}

  • consider significantly increasing VCS checking for changes interval (server-wide default and overridden in the VCS roots) or changing settings of the VCS roots to prevent them from contacting production servers. Since TeamCity 10.0.3 see also TW-47324.

See also the notes section below on moving the server from one machine to another. Anchorcopy_server_licensecopy_server_license Licensing issues
You cannot use a single TeamCity license on two running servers at the same time, so to run a copy of TeamCity server you will need another license. Copies of the server created for redundancy/backup purposes can use the same licenses as they only should be running one at a time.
If you are only going to run the server for testing purposes, you can get time-limited TeamCity evaluation license once from the official TeamCity download page. If you need an extension of the license or you have already evaluated the same TeamCity version, please contact our sales department.
If you plan to run the second server at the same time as the main one regularly/for production purposes, you need to purchase separate licenses for the second server.

Move TeamCity Projects from One Server to Another

If you need to move data to a fresh server without existing data, it is recommended to move the server or copy it and then delete the data which is not necessary on the new server.

...

If you need to move the existing TeamCity installation to a new hardware or clean OS, it is recommended to follow the instructions on copying the server from one machine to another and then switch from the old server to a new one. If you plan to use the same database instance on the new server, just skip the items on copying the database. If you are sure you do not need to preserve the old server, you can perform move operations instead of copying in those instructions.

You can use the existing license keys when you move the server from one machine to another (as long as there are no two servers running at the same time). As license keys are stored under <TeamCity Data Directory>, you transfer the license keys with all the other TeamCity settings data.

It is usually advisec advised not to combine TeamCity update with any other actions, like environment or hardware changes, and perform the changes one at a time so that, if something goes wrong, the cause can be easily tracked.

Anchor
switch_server_to_another_address
switch_server_to_another_address

Switching from one server to another
Please note that <TeamCity Data Directory > and database should be used by a single TeamCity instance at any given moment. If you configured a new TeamCity instance to use the same data, please ensure you shutdown and disable the old TeamCity instance before starting the new one.

...

  1. Write a build script that will perform the deployment task for the binary files available on the disk. (e.g. use Ant or MSBuild for this. For FTP/SSH tasks check the Deployer plugintypical deployment transports use Deployer runners). See also #Integrate with Build and Reporting Tools. You can use Meta-Runner to reuse a script with convenient UI.
  2. Create a build configuration in TeamCity that will execute the build script and perform the actual deployment. If the deployment is to be visible or startable only by the limited set of users, place the build configuration in a separate TeamCity project and make sure the users have appropriate permissions in the project.
  3. In this build configuration configure artifact dependency on a build configuration that produces binaries that need to be deployed.
  4. Configure one of the available triggers in the deploying build configuration if you need the deployment to be triggered automatically (e.g. to deploy last successful of last pinned build), or use "Promote" action in the build that produced the binaries to be deployed.
  5. Consider using snapshot dependencies in addition to artifact ones and check Build Chains tab to get the overview of the builds. In this case artifact dependency should use "Build from the same chain" option.
  6. If you need to parametrize the deployment (e.g. specify different target machines in different runs), pass parameters to the build script using custom build run dialog. Consider using Typed Parameters to make the custom run dialog easier to use or handle passwords.
  7. If the deploying build is triggered manually consider also adding commands in the build script to pin and tag the build being deployed (via sending a REST API request).
    You can also use a build number from the build that generated the artifact.

...

Data collection
The easiest way for a start is to modify your build scripts to make use of the selected tool and collect all the required data.
If you can run the tool from a command line console, then you can run it in TeamCity with a command line runner. This will give you detection of the messages printed into standard error output. The build can be marked as failed is the exit code is not zero or there is output to standard error via build failure condition.
If the tool has launchers for any of the supported build scripting engines like Ant, Maven or MSBuild, then you can use corresponding runner in TeamCity to start the tool.
See also #Use an External Tool that My Build Relies on for the recommendations on how to run an external tool.

...

If the tool reports code-attributing information like Inspections or Duplicates, TeamCity-bundled report can be used to display the results. A custom plugin will be necessary to process the tool-specific report into TeamCity-specific data model. Example of this can be found in XML Test Reporting plugin and FXCop plugin (see a link on Open-source Bundled Plugins).

See also #Import coverage results in TeamCity.

For advanced integration, a custom plugin will be necessary to store and present the data as required. See Developing TeamCity Plugins for more information on plugin development.

Restore Just Deleted Project

TeamCity moves settings files of deleted projects under settings directories (which are named after the project id) to TeamCity Data Directory/config/_trash directory adding ".<internal ID>" suffix to the directories.
To restore a project you should , find the project directory on in the server _trash directory and move it into regular projects settings directory: <TeamCity Data Directory>/config/projects . Also you should remove suffix \.projectN while removing the ".projectN" suffix from the directory name.
You can do this while server is running, it should pick up the restored project automatically.

Please note that TeamCity preserves builds history and other data stored in the database for deleted projects/build configurations for 24 hours since after the deletion time. All the associated data (builds and test history, changes, etc.) is removed during the next cleanup after the 24 hours -hour timeout elapses.

The config/_trash directory is not cleaned automatically and can be emptied manually if you are sure you do not need the deleted projects. No server restart is required.

...

  • publish coverage HTML report as TeamCity artifact: most of the tools produce coverage report in HTML format, you can publish it as artifact and configure report tab to show it in TeamCity. If artifact is published in the root artifact directory and its name is coverage.zip and there is index.html file in it, report tab will be shown automatically. As to running an external tool, check #Integrate with Build and Reporting Tools.
  • extract coverage statistics from coverage report and publish statistics values to TeamCity with help of service message: if you do so, you'll see coverage chart on build configuration Statistics tab and also you'll be able to fail a build with the help of a build failure condition on a metric change (for example, you can fail build if the coverage drops).

...

CVE-2015-0235 vulnerability is found in glibc library which is not directly used by TeamCity code. It is used by the Java/JRE used by TeamCity under *nix platforms. As Java is not bundled with TeamCity distributions, you should apply the security measures recommended by the vendor of the Java you use. At this time there are no related Java-specific security advisories released, so updating the OS should be enough to eliminate the risk of the vulnerability exploitation.

hidden-data
h4. Apache FileUpload
CVE-2016-3092 moderate level vulnerability was found in FileUpload library and can cause remote DOS attach via high CPU usage. TeamCity versions starting from 10.0.4 are not affected by the vulnerability even though bundled Tomcat version can be reported as affected: TeamCity uses own library copy and not Tomcat's one. The library was updated to a version without the issue.

FREAK

CVE-2015-0204 vulnerability is found in OpenSSL implementation and TeamCity does not bundle any parts of OpenSSL product and so is not vulnerable. You might still need to review the environment in which TeamCity server and agents are installed as well as tools installed in addition to TeamCity for possible vulnerability mitigation steps necessary.

...

CVE-2017-5638 affects Jakarta Multipart parser in Apache Struts. Since no versions of TeamCity use struts, TeamCity is not affected.CVE-2016-1181 also affects multipart requests processing in some older versions of Struts.

TeamCity bundles IntelliJ IDEA which contains jars from both: Apache Struts 1.x and Apache Struts 2.x. These jars are only used by IntelliJ IDEA Struts plugin when IntelliJ IDEA collects inspections for a project on a TeamCity agent.

But under no circumstances these versions of Apache Struts are used to handle any HTTP requests. Thus neither TeamCity server, not TeamCity agent are affected by these vulnerabilities.

 

Tomcat Under Windows

Based on the wording of the description of CVE-2017-12615, CVE-2017-12616 and CVE-2017-12617 TeamCity server installed under Windows is a potential subject for the attack. However, our analysis of the vulnerabilities indicates that these potential vulnerabilities cannot be exploited in the default TeamCity installation as the related configuration of Tomcat is inactive in all the TeamCity versions.
If necessary, Tomcat bundled with TeamCity can be upgraded to the version 7.0.82 which also removes the vulnerability form the Tomcat code.

hidden-data
related issues:
https://youtrack.jetbrains.com/issue/TW-51625
https://youtrack.jetbrains.com/issue/TW-51905

Watch Several TeamCity Servers with Windows Tray Notifier

...

"major" release below means any release with a change in the first or second version number (e.g. X in X.X.Z)
"bugfix" release means releases with a change in the third version number (e.g. Z in X.X.Z)

Release stages that we generally have are:
Available under EAP (Early Access Program) - usually available only for major releases, starts several months after previous major release and usually months before the next major release. Typically, new EAP releases are published on a monthly or bi-monthly basis.
General Availability - : as a rule, there is a major release each 8 months. There are multiple bugfix releases following the major release. Bugfix The bugfix releases and support patches for critical issues (if applicable) are provided until "End of Sale" of the release.
End of Sale - occurs  occurs with the release of a new major version. After this time, no bugfix updates or patches are usually provided (Exceptions are critical issues without a workaround which at the same time allow for relatively simple fix and inability for the customer to upgrade for an important reason). Only limited support is provided for these versions.
End of Support - occurs with the release of  occurs when two newer major versions have been released. At this point we stop providing regular technical support for the release.

Dates for the previous releases can be seen at The dates of previous releases and the sequence of TeamCity versions are listed on the Previous Releases Downloads page.

 

 

 

 

 

 

 

 

 

 

Wiki Markup
{hidden-data}
empty lines to allow proper anchor positioning
 
 
 
Setting up TeamCity for demo mode
- when setting up a build configuration from scratch, consider disabling favorite builds on your user profile if you are not demoing those
- reduce global server checking for changes interval
- reduce build trigger/starting periods (TBD, internal properties)
{hidden-data}