TeamCity 2018.x Documentation

[Documentation for Previous Versions]

Icon

You are viewing the documentation of TeamCity 2018.x, which is not the most recently released version of TeamCity.
View this page in the latest documentation or refer to the [listing] to choose the documentation corresponding to your TeamCity version.

 

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  • CPU: TeamCity utilizes multiple cores of the CPU, so increasing number of cores makes sense. For non-trivial TeamCity usage at least 4 CPU cores are recommended.
  • Memory: See a note on memory usage. Consider also that required memory may depend on the JVM used (32 bit or 64 bit). used by the TeamCity server main process and child processes (used for Maven integration, version control integration, Kotlin DSL execution). See the notes on the main process memory usage. Generally, you will probably not need to dedicate more than 4G of memory to TeamCity server if you do not plan to run more then 100 concurrent builds (agents) and , support more then 200 online users or work with large repositories.
  • HDD/disk usage: This sums up mainly from the temp directory usage (<TeamCity Home>/temp and OS temp directory) and .BuildServer/system usage. Performance of the TeamCity server highly depends on the disk system performance. As TeamCity stores large amounts of data under .BuildServer/system (most notably, VCS caches and build results) it is important that the access to the disk is fast (in particular reading/writing files in multiple threads, listing files with attributes). Ensuring disk has good performance is especially important if you plan to store the data directory on a network drive.
  • Network: This mainly sums up from the traffic from VCS servers, to clients (web browsers, IDE, etc.) and to/from build agents (send sources, receive build results, logs and artifacts).

...

  • number of build configurations;
  • number of builds in the history;
  • number of the builds running daily;
  • amount of data consumed and produced by the builds (size of the used sources and artifacts, size of the build log, number and output size of unit tests, number of inspections and duplicates hits, size and number of produced artifacts, etc.);
  • cleanup rules configured
  • number of agents and their utilization percentage;
  • number of users having TeamCity web pages open;
  • number of users logged in from IDE plugin;
  • number and type of VCS roots as well as the configured checking for changes interval for the VCS roots. VCS checkout mode is relevant too: server checkout mode generates greater server load. Specific types of VCS also affect server load, but they can be roughly estimated based on native VCS client performance;
  • number of changes detected by TeamCity per day in all the VCS roots;
  • the size of the repositories TeamCity works with;
  • total size of the sources checked out by TeamCity daily.

...

  • 60 projects and 300 build configurations (with one forth being active and running regularly);
  • more than 300 builds a day;
  • about 2Mb log per build;
  • 50 build agents;
  • 50 web users and 30 IDE users;
  • 100 VCS roots (mainly Perforce and Subversion using server checkout), average checking for changes interval is 120 seconds;
  • more than 150 changes per day;
  • Kotlin DSL is not used;
  • the database (MySQL) is running on the same machine;
  • TeamCity server process has -Xmx1100m JVM setting.

...

  • Regularly review reported Server Health reports (including hidden ones)
  • Use a separate reverse proxy server (e.g. NGINX) to handle HTTPS
  • Use a separate server for the external database and monitor the database performance
  • Monitor the server's CPU and IO performance, increase hardware resources as necessary (see also hardware notes)
  • Make sure clean-up is configured for all the projects with a due retention policy, make sure clean-up completely finishes regularly (check Administration / Clean-Up page)
  • Consider ensuring good IO performance for the <TeamCity Data Directory>/system/caches directory, e.g. by moving it to a separate local drive (or storing on a local drive you choose to store the TeamCity Data Directory on a network storage)
  • Regularly archive obsolete projects
  • Regularly review the installed not bundled plugins and remove those not essential for the server functioning
  • Consider using agent-side checkout whenever possible
  • Make sure the build logs are not huge (tens megabytes at most, better less than 10 Mb)
  • If lots VCS roots are configured on the server, consider configuring repository commit hooks instead of using polling for changes or at least increase VCS polling interval to 300 seconds or more)
  • If the server is often used by large number of users (e.g. more than 1000), consider reducing the frequency of background UI requests their by increasing UI refresh intervals
  • When regularly exceeding 500 concurrently running builds which log a lot of data, consider using Several Nodes Setup

...

Allocating 1 GB for the redo log (see the table below) and undo files is sufficient in most cases.

...

We've seen patterns of having an agent per each 20 build configurations (types of builds). Or a build agent per 1-2 developers.

See also notes on maximum supported number of agents.

Setup TeamCity in Replication/Clustering Environment

TeamCity does not provide specific support for full replication/redundancy/high availability (HA) or clustering solutions. The only related ability is to set up separate node for processing running builds.
However, to only supports a single instance of the main server, but it is possible to add secondary node (to provide view-only UI over the current data) and running builds node to collect the running builds data form the agents. All three nodes need to be connected to the same TeamCity Data Directory and the database.

To address fast disaster recovery scenarios, it TeamCity supports active - failover (cold standby) approach: the data that the TeamCity server uses can be replicated and a solution put in place to start a new server using the same data if the currently active server malfunctions.

...

TeamCity agents farm can be reused between the main and the failover servers. Agents will automatically connect to the new server if you make the failover server to be resolved via the old server DNS name and agents connect to the server using the DNS name. See also information on switching from one server to another.
If appropriate, the agents can be replicated just as the server. However, there is no need to replicate any TeamCity-specific data on the agents except for the conf\buildAgent.properties file as all the rest of the data can typically be renewed from the server. In case of replicated agents farm, the replica agents just need to be connected to the failover server.

...

Note that since TeamCity 2017.2 the TeamCity Windows installer modifies permisisons of the TeamCity installation directory not to use inheritable permissions and explicitly grants access to the directory to the Administrators user group and the accrount under which the service is configured to run.
It is strongly recommended to restrict permissions to the TeamCity Data Directory in the same way.

Additional security-realted related settings

Consider adding the "teamcity.installation.completed=true" line into the <TeamCity Home Directory>\conf\teamcity-startup.properties file - this will prevent the server started with the empty database from granting access to the machine for the first coming user. 

...

  • You are running the latest released TeamCity version and are ready to upgrade to the newly released versions within weeks
  • Access to the TeamCity web interface is secured using HTTPS (e.g. with the help a proxying server like NGINX). Best practices for securing web

    application

    applications are employed for the TeamCity web interface. e.g.: It is not possible to access the server using HTTP protocol. Reverse proxy does not strip Referer request header

    hidden-data
    otherwise Referer header can be not present and e.g. build artifacts protection will not work

  • The TeamCity server machine does not run agents (at least under the user permitted to read the TeamCity server's home directory and TeamCity Data Directory)
  • TeamCity server and agents processes are run under limited users with minimal required permissions. Installation directories are readable and writable only by a limited set of OS users. The conf\buildAgent.properties file and server logs as well as the Data Directory are only readable by OS users who represent administrators of the services, because reading those locations may allow taking over the agent or server respectively.
  • Guest user and user registration is disabled or roles are reviewed for guest and the All Users group
  • TeamCity users with administrative permissions have non-trivial passwords
  • If you have external authentication configured (such as LDAP), the built-in authentication module is disabled
  • Passwords are not printed into the build log, not stored in build artifacts, nor are they stored in non-password parameters

...

You need to set up a reverse proxy (see Proxy Server Setup below) and also configure TeamCity's bundled Tomcat server (see TeamCity Tomcat Configuration below) to make sure TeamCity "knows" the actual absolute URL used by the client to access the resources. These URLs are then used to generate absolute URLs in client redirects and other responses.

...

When using Apache, make sure to use the Dedicated "Connector" node approach for configuring TeamCity server.

...

For the NGINX configuration below, use the "RemoteIpValve" Approach for configuring TeamCity server.

...

Code Block
http {
    # ... default settings here
    proxy_read_timeout     1200;
    proxy_connect_timeout  240;
    client_max_body_size   0;    # maximum size of an HTTP request. 0 allows uploading large artifacts to TeamCity
 
	map $http_upgrade $connection_upgrade { # WebSocket support
        default upgrade;
        '' '';
    }
    
    server {
        listen       400; # public server port
        server_name  teamcity.public; # public server host name
    
        location /tc { # public context (should be the same as internal context)
            proxy_pass          http://teamcity.local:8111/tc; # full internal address
            proxy_http_version  1.1;
            proxy_set_header    Host $server_name:$server_port;
            proxy_set_header    X-Forwarded-Host $http_host;    # necessary for proper absolute redirects and TeamCity CSRF check
            proxy_set_header    X-Forwarded-Proto $scheme;
            proxy_set_header    X-Forwarded-For $remote_addr;
            proxy_set_header    Upgrade $http_upgrade; # WebSocket support
            proxy_set_header    Connection $connection_upgrade; # WebSocket support
        }
    }
}

Wiki Markup
{hidden-data}
Some NGINX configuration resource: https://nginxconfig.io/
some more NGINX params which might also relate:

client_header_timeout  10m;
client_body_timeout    10m;
send_timeout           10m;
proxy_max_temp_file_size 10240m;
{hidden-data}

Common misconfigurations

Check that your reverse proxy (or a similar tool) conforms to the following requirements:

  • URLs with paths starting with a dot (the "." symbol) are supported (path to hidden artifacts start contain the ".teamcity" directory)
  • URLs with a colon (":" symbol) are supported (many TeamCity resources use the colon). Related IIS setting. Symptom: build has no artifacts with "No user-defined artifacts in this build." text even if there are artifacts.
  • maximum response length / time are not too restrictive (since TeamCity can serve large files to slow clients, the responses can be of Gb in size and hours in time)
  • gzip Content-Encoding is fully supported. e.g. certain IIS configurations can result in the "Loading data..." in UI and 500 HTTP responses (see the related issue)
Anchor
OtherServers
OtherServers
Other servers

Make sure to use a performant proxy with due (high) limits on request (upload) and response (download) size and timeouts (idle timeout should be hours to support server-side checkout)Since Teamcity 2017.1.3, setting a long idle timeout is not required, as TeamCity behind a proxy server keeps the connection to the agent alive while the patch is being created on the serverat least tens of minutes and gigabyte, according to the sizes of the codebase and artifacts).

It is recommended to use a proxy capable of working with the WebSocket protocol as that helps UI to refresh sooner.

Generally, you need to configure the TeamCity server so that it "knows" about the original URL used by the client and it can generate correct absolute URLs accessible for the client. Alternatively,  the proxy should rewrite local TeamCity server absolute URLs in responses to public URLs, but since there are many locations which can use the URLs, this is usually not feasible at full scalePreferred way to achieve that is to pass original "Host" header to TeamCity. Alternative is to set "X-Forwarded-Host" header to the original "Host" header value.

Note that whenever the value of the "Host" header is changed by the proxy (while it is recommended to preserve original Host header value) and no "X-Forwarded-Host" header with the original Host value is provided, the values of the Origin and Referer headers should be mapped correspondingly if they contain the original Host header value (if they do not, they should not be set in order not to circumvent TeamCity CSRF protection).

...

  • use a dedicated "Connector" node in the server configuration with hard-coded public URL details and make sure that the port configured in the connector is used only by the requests to the public URL configured.
  • configure proxy to pass due request headers: "Host" or "X-Forwarded-Host" (original request host), "X-Forwarded-Proto" (original request protocol), "X-Forwarded-Port" (original request port) and configure "RemoteIpValve" for the TeamCity Tomcat configuration. 
Anchor
Proxy-Tomcat-Connector
Proxy-Tomcat-Connector
Dedicated "Connector" Node Approach

...

This approach can be used when the proxy server sets  "Host", "X-Forwarded-Proto", "X-Forwarded-Port" request headers to the values of the original URL.

hidden-data
Since TeamCity 2017.
1 the "X-Forwarded-Host" header can be used to pass the original host name instead of preserving the value of the "Host" header. Also since TeamCity 2017.1 for the majority of functionality to work,
 it
 it is not required to set
up RemoteIpValve
 up RemoteIpValve if only a single proxy is used (i.e. the mentioned headers store only a single value).
Also, while not critical for the most setups, this approach can be used to make sure the original client IP is passed to the TeamCity server correctly. This is important for legacy agents using bidirectional communication.

Add the following into the Tomcat main <Host> node of the conf\server.xml file (see also Tomcat doc):

Code Block
xml
xml
<Valve
   className="org.apache.catalina.valves.RemoteIpValve"
   remoteIpHeader="x-forwarded-for"
   protocolHeader="x-forwarded-proto"
   portHeader="x-forwarded-port"
   />

It is also recommended to specify internalProxies attribute with the regular expression matching only IP address of the proxy server. e.g. internalProxies="192\.168\.0\.1"

Wiki Markup
{hidden-data}see also comment: [http://youtrack.jetbrains.com/issue/TW-11166#comment=27-331468]{hidden-data}

Configure HTTPS for TeamCity Web UI

For small servers, you can set up HTTPS via the internal Tomcat means, but this is not recommended as it may significantly increase the CPU load.

For configuring clients to access TeamCity server via HTTPS while using self-signed certificate, check the related instructions.

Configure TeamCity to Use Proxy Server for Outgoing Connections

...

To point TeamCity to your proxy server:
Since TeamCity 2017.1.5 the following server internal properties are available (see the section below for previous version):

Code Block
languagetext
# For HTTP protocol
## The domain name or the IP address of the proxy host and the port: 
teamcity.http.proxyHost=proxy.domain.com
teamcity.http.proxyPort=8080
 
## The hosts that should be accessed without going through the proxy, usually internal hosts. Provide a list of hosts, separated by the '|' character. The wildcard '*' can be used:
teamcity.http.nonProxyHosts=localhost|*.mydomain.com)
 
## For an authenticated proxy add the following properties:
### Authentication type. "basic" and "ntlm" values are supported. The default is basic.
teamcity.http.proxyAuthenticationType=basic
### Login and Password for the proxy:
teamcity.http.proxyLogin=username
teamcity.http.proxyPassword=password
 
# For HTTPS protocol
## The domain name or the IP address of the proxy host and the port: 
teamcity.https.proxyHost=proxy.domain.com
teamcity.https.proxyPort=8080

## The hosts that should be accessed without going through the proxy, usually internal hosts. Provide a list of hosts, separated by the '|' character. The wildcard '*' can be used:
teamcity.https.nonProxyHosts=localhost|*.mydomain.com

## For an authenticated proxy add the following properties:
### Authentication type. "basic" and "ntlm" values are supported. The default is basic.
teamcity.https.proxyAuthenticationType=basic
### Login and Password for the proxy:
teamcity.https.proxyLogin=login
teamcity.https.proxyPassword=password

...

It's advised to try a new TeamCity version before upgrading your production server. The usual procedure is to create a copy of your production TeamCity installation, then upgrade it, try the things out, and, when everything is checked, drop the test server and upgrade the main one.
When you start the test server, remember to change the Server URL, disable Email and Jabber notifiers as well as other features on the new server.

Anchor
copy_server
copy_server

...

In case you want to preserve the original server as well as the copy, make sure to check the lincensing licensing considerations.

Create a Server Copy

You can create a copy of the server using TeamCity backup functionality or manually. Manual approach is recommended for the complete server move.

Use TeamCity Backup
  1. Create a backup including everything. (You can skip the option to backup build logs is you are moving the artifacts, see below).
  2. Install a new TeamCity server of the same version that you are already runnningrunning. Ensure that:
    - the appropriate environment is configured (see the notes below) 
    - the server uses its own TeamCity Data Directory and its own database (check config/database.properties in the Data Directory)
  3. Restore the backup.
  4. Move the artifacts (these are not included into the backup) by moving the content of  <TeamCity Data Directory>/system/artifacts directory from the old location to the new one. Since the artifacts can be large in the size, this can take a lot of time, so plan accordingly.
  5. Perform the necessary environment transfer.
Note

The new server won't get build artifacts and some other data. If you need them, you will need to copy appropriate directories (e.g. the entire "artifacts" directory) from .BuildServer/system from the In order to ensure complete server transfer, it is recommended to copy the entire content of the TeamCity Data Directory from the original to the copied server. Make sure to finish copying over the artifacts It is recommended to complete the copying before starting the new server.

...

  1. Create a backup so that you can restore it if anything goes wrong,
  2. Ensure the server is not running,
  3. Either perform clean installation or copy the TeamCity binaries (TeamCity Home Directory) into a new place (the temp and work subdirectories can be omitted during copying). (warning) Use exactly the same TeamCity version. If you plan to upgrade after copying, perform the upgrade only after you have the existing version up and running.
  4. Copy TeamCity Data Directory. If you do not need the full copy, refer to the items below for options.
    • .BuildServer/config to preserve projects and build configurations settings
    • .BuildServer/lib and .BuildServer/plugins if you have them
    • files from the root of .BuildServer/system if you use internal database and you do not want to perform database move.
    • .BuildServer/system/artifacts (optional) if you want build artifacts and build logs (including tests failure details) preserved on the new server
    • .BuildServer/system/changes (optional) if you want personal changes preserved on the new server
    • .BuildServer/system/pluginData (optional) if you want to preserve state of various plugins, build triggers and settings audit diff
    • .BuildServer/system/caches and .BuildServer/system/caches (optional) are not necessary to copy to the new server, they will be recreated on startup, but can take some time to be rebuilt (expect some slow down).
  5. Artifacts directory is usually large and if you need to minimize the downtime of the server in case of the server move, you can use the generic approach for copying the data: use rsync, robocopy or alike tool to copy the data while the original server is running. Repeat the sync run several times until the amount of data synced reduces. Run the final sync after the original server shut down. Alternative solution for the server move is to make the old data artifacts directory accessible to the new server and configure it as second location of artifacts. Then copy the files over from this second location to the main one while the server is running, restart the server after copying completion.
  6. Create copy of the database that your TeamCity installation is using in new schema or new database server. This can be done with database-specific tools or with the bundled maintainDB tool by backing up database data and then restoring it.
  7. Configure new TeamCity installation to use proper TeamCity Data Directory and database (.BuildServer/config/database.properties points to a copy of the database)
  8. Perform the necessary environment transfer.
Info

If you want to do a quick check and do not want to preserve builds history on the new server you can skip step 6 (cloning database) and all items of the step 5 marked as optional.

...

  • ensure the new server is configured to use another data directory and another database than the original server; check also "Artifact directories" setting on server's Global Settings;
  • change server unique id by removing "uuid" attribute from XML of <TeamCity Data Directory>\config\main-config.xml file before the first start;
  • ensure the same license keys are not used on several servers (more on licensing);
  • update Server URL on Administration | Global Settings page to the actual URL of the server;
  • check that you can successfully authenticate on the new server, use super user access if necessary;
  • check that VCS servers, issue tracker servers, email and Jabber server and other server-accessed systems are accessible;
  • check that any systems configured to push events to TeamCity server (like VCS hooks, automated build triggering, monitors, etc.) are updated to know about the new server;
  • review the list of installed plugins to determine if their settings need changes;
  • install new agents (or select some from the existing ones) and configure them to connect to the new server (using the new server URL);
  • check that clients reading from the server (downloading artifact, using server's REST API, NuGet feed, etc.) are reconfigured, if necessary.

...

  • disable Email, Jabber (in the "Administration > Notifier" sections) and possibly also custom notifiers or change their settings to prevent the new server from sending out notifications;
  • disable email verification (in the  "Administration > Authentication" section);
  • be sure not to run any builds which change (e.g. deploy to) production environments. This also typically includes Maven builds deploying to non-local repositories. You can prevent any builds from starting by pausing the build queue;
  • disable cloud integration (so that it does not interfere with the main server);
  • disable external artifact storage (as otherwise running/deleting builds and server cleanup will affect the storage which might be used by the production server);
  • disable Git registry cleanup (or just disable cleanup on the server);
  • disable Commit Status Publishing;
  • disable any plugins which push data into other non-copied systems based on the TeamCity events (like commit status publishing);

    hidden-data
    https://youtrack.jetbrains.com/issue/TW-47362
    

  • disable

    cloud integration (so that it does not interfere with the main server);

    disable functionality to store project settings in VCS: set teamcity.versionedSettings.enabled=false internal property;

    Wiki Markup
    {hidden-data}related issue: [https://youtrack.jetbrains.com/issue/TW-38304]{hidden-data}

  • consider significantly increasing VCS checking for changes interval (server-wide default and overridden in the VCS roots) or changing settings of the VCS roots to prevent them from contacting production servers. Since TeamCity 10.0.3, see also TW-47324.

See also the section below on moving the server from one machine to another.

...

If you need to move data to a fresh server without existing data, it is recommended to move the server or copy it and then delete the data which is not necessary on the new server.

If you need to join the data with already existing setdata on the new server, there is a dedicated feature to move projects with most of the associated data from one server to another: Projects Import.

...

Move TeamCity Installation to a New Machine

If you need to move the an existing TeamCity installation to a new hardware or a clean OS, it is recommended to you can install the same TeamCity version on the new machine, stop the old server, connect the new server to the same TeamCity Data Directory and make sure the server uses the same environment.
Alternatively, you can follow the instructions on copying the server from one machine to another and then switch from the old server to a new one. If you plan to use the same database instance on the new server, just skip the items on copying the database. If you are sure you do not need to preserve the old server, you can perform move operations instead of copying in those instructions.

After the move, make the clients use the new server address, if changed.

You can use the existing license keys when you move the server from one machine to another (as long as there are no two servers running at the same time). As license keys are stored under <TeamCity Data Directory>, you transfer the license keys with all the other TeamCity settings data.

...

  • Switch agents to the new URL (requires updating the serverUrl property in buildAgent.properties on each agent). If you want to install agents anew but preserve agent's name and authentication status, you can install a new agent and copy the conf\ buildAgent.properties file from an old agent (checking that any paths in it are updated accordingly).
  • Upon the new server startup, remember to update the Server URL on Administration | Global Settings page.
  • Notify all TeamCity users on the new address
  • Consider updating settings of external services if they depend on the request originating address

Move TeamCity Agent to a New Machine

Apart from the binaries, TeamCity agent installation stores its configuration and data left from the builds it run. Usually the data from the previous builds makes preparation for the future builds a bit faster, but it can be deleted if necessary.
The configuration is stored under conf and launcher\conf directories.
The data collected by previous build is stored under work and system directories.

...

  • stop existing agent
  • install a new agent on the new machine
  • copy conf/buildAgent.properties from the old installation to a new one
  • start the new agent.

...

  1. Write a build script that will perform the deployment task for the binary files available on the disk. (e.g. use Ant or MSBuild for this. For typical deployment transports use Deployer runners). See also #Integrate with Build and Reporting Tools. You can use Meta-Runner to reuse a script with convenient UI.
  2. Create a build configuration in TeamCity that will execute the build script and perform the actual deployment. If the deployment is to be visible or startable only by the limited set of users, place the build configuration in a separate TeamCity project and make sure the users have appropriate permissions in the project.
  3. In this build configuration configure artifact dependency on a build configuration that produces binaries that need to be deployed.
  4. Configure one of the available triggers in the deploying build configuration if you need the deployment to be triggered automatically (e.g. to deploy last successful of last pinned build), or use "Promote" action in the build that produced the binaries to be deployed.
  5. Consider using snapshot dependencies in addition to artifact ones and check Build Chains tab to get the overview of the builds. In this case artifact dependency should use "Build from the same chain" option.
  6. If you need to parametrize the deployment (e.g. specify different target machines in different runs), pass parameters to the build script using custom build run dialog. Consider using Typed Parameters to make the custom run dialog easier to use or handle passwords.
  7. If the deploying build is triggered manually consider also adding commands in the build script to pin and tag the build being deployed (via sending a REST API request).
    You can also use a build number from the build that generated the artifact.

...

If you have a build tool or a tool that generates some report/provides code metrics which is not yet supported by TeamCity or any of the plugins, most probably you can use it in TeamCity even without dedicated integration.

...

Data collection
The easiest way for a start is to modify your build scripts to make use of the selected tool and collect all the required data.
If you can run the tool from a command line console, then you can run it in TeamCity with a command line runner. This will give you detection of the messages printed into standard error output. The build can be marked as failed is the exit code is not zero or there is output to standard error via build failure condition.
If the tool has launchers for any of the supported build scripting engines like Ant, Maven or MSBuild, then you can use corresponding runner in TeamCity to start the tool.
See also #Use an External Tool that My Build Relies on for the recommendations on how to run an external tool.

...

If the tool reports code-attributing information like Inspections or Duplicates, TeamCity-bundled report can be used to display the results. A custom plugin will be necessary to process the tool-specific report into TeamCity-specific data model. Example of this can be found in XML Test Reporting plugin and FXCop plugin (see a link on Open-source Bundled Plugins).

See also #Import coverage results in TeamCity.

For advanced integration, a custom plugin will be necessary to store and present the data as required. See Developing TeamCity Plugins for more information on plugin development.

...

TeamCity moves deleted projects settings directories (which are named after the project id) to TeamCity Data Directory/config/_trash directory adding ".<internal ID>" suffix to the directories.
To restore a project, find the project directory in the _trash directory and move it into regular projects settings directory: <TeamCity Data Directory>/config/projects while removing the ".projectN" suffix from the directory name.
You can do this while server is running, it should pick up the restored project automatically.

Please note Note that TeamCity preserves builds history and other data stored in the database for deleted projects/build configurations for 24 hours 5 days after the deletion time. All the associated data (builds and test history, changes, etc.) is removed during the next cleanup clean-up after the 24-hour configurable (5 days by default) timeout elapses. 

The config/_trash directory is not cleaned automatically and can be emptied manually if you are sure you do not need the deleted projects. No server restart is required.

...

  • publish coverage HTML report as TeamCity artifact: most of the tools produce coverage report in HTML format, you can publish it as artifact and configure report tab to show it in TeamCity. If artifact is published in the root artifact directory and its name is coverage.zip and there is index.html file in it, report tab will be shown automatically. As to running an external tool, check #Integrate with Build and Reporting Tools.
  • extract coverage statistics from coverage report and publish statistics values to TeamCity with help of service message: if you do so, you'll see coverage chart on build configuration Statistics tab and also you'll be able to fail a build with the help of a build failure condition on a metric change (for example, you can fail build if the coverage drops).

...

This section describes effect and necessary protection steps related to recently the announced security vulnerabilities.

...

CVE-2015-0204 vulnerability is found in the OpenSSL implementation and . TeamCity does not bundle any parts of OpenSSL product and so is not vulnerable. You might still need to review the environment in which the TeamCity server and agents are installed set up, as well as the tools installed in addition to TeamCity for possible vulnerability mitigation steps necessary.

Apache Struts

CVE-2017-5638 affects Jakarta Multipart parser in Apache Struts. CVE-2016-1181 also affects multipart requests processing in some older versions of Apache Struts.

...

But under no circumstances these versions of Apache Struts are used to handle any HTTP requests. Thus neither TeamCity server, not TeamCity agent are affected by these vulnerabilities.

 

Tomcat Under Windows

Based on the wording of the description of CVE-2017-12615, CVE-2017-12616 and CVE-2017-12617 TeamCity server installed under Windows is a potential subject for the attack. However, our analysis of the vulnerabilities indicates that these potential vulnerabilities cannot be exploited in the default TeamCity installation as the related configuration of Tomcat is inactive in all the TeamCity versions.
If necessary, Tomcat bundled with TeamCity can be upgraded to the version 7.0.82 which also removes the vulnerability form the Tomcat code.

hidden-data
related issues:
https://youtrack.jetbrains.com/issue/TW-51625
https://youtrack.jetbrains.com/issue/TW-51905

Tomcat CVE-2018-8037

TeamCity version 2018.1 is not vulnerable to the issue as it was identified and addressed in the TeamCity codebase before the official Tomcat announcement (actually, the issue was found in a TeamCity installation and we worked with the Tomcat team on fixing it). Earlier TeamCity versions are vulnerable, so upgrading to TeamCity 2018.1+ is necessary.

Watch Several TeamCity Servers with Windows Tray Notifier

...

In relation to the TeamCity product, JetBrains does not collect any personal data of the final on-premises TeamCity installation users. The related documents governing relationship of users customers with JetBrains are available on the official web site: privacy policy, terms of purchase, TeamCity license agreement.

...

  • full name and username - stored in the database and shown on the user's profile and whenever the user is referenced. When a user triggers a build, these are also stored in the build's parameters and passed into the build
  • user's email - stored in the database and shown on the user's profile, used to send out notifications
  • IP adderss address of the clients accessing the server - can appear in the internal logs

TeamCity internal logs can also record some unstructured user-related information (e.g. submitted by the user or sent by the browser with the HTTP requests, retrieved according to the configured settings from the users source like LDAP)

Deleting the User Data

When you want to delete personal data of a specific user, the best way to do it is to delete the user in TeamCity. This way all the references to the user will continue to store the numeric user id, while all the other user information will not be stored anymore. Note that Audit records will mention internal numeric user id after the user deletion.

...

If you want the users to accept a special ageement agreement before using your TeamCity instance, you can install a dedicated plugin developed by JetBrains for this purpose. Refer to the pluginsplugin's documentation for more details.

...

If you want to encrypt the data used by TeamCity, it is recommedned recommended to use generic, non-TeamCity-specific tools for this as TeamCity does not provide dedicated funcaionalityfunctionality.
TeamCity stores the data in the SQL database and on the file system.
You can configure the database to store the data in encrypted form and use secure JDBC-backed connection to the database (configured in the database.properties).
Also, you can configure encryption on the disk storage on the OS level.

...

If you want to ensure that you do not store the internal TeamCity logs for more than a limited amount of days, you can configure internal logging to rotate the log files each day and limit the number of files to keep. TeamCity agents generally do not operate with any user-related data in a structural way, but if you need to ensure the logs are regularly rotated on the agent, you will need to configure agent logging the same way.
Per-day rotation can be configured by adding <param name="rotateOnDayChange" value="true"/line within all <appender > line within all required <appender name="ROLL..." class="jetbrains.buildServer.util.TCRollingFileAppender"appenders> appenders. This change should be done to the default conf\teamcity-server-log4j.xml and also logging presets stored under <TeamCity Data Directory>\config\_logging.

hidden-data
https://youtrack.jetbrains.com/issue/TW-52220
TeamCity can also store diagnostics data like thread dumps which can record user-related data in unstructured way. It is recommended to review the content of the <TeamCity Home>\logs directory regularly and ensure that no old files are preserved in there. Also, the extra logs should be deleted after logging customization sessions like collecting debug logs, etc.
hidden-data
Related internal properties:
Do not add teamcity.build.triggeredBy properties to the newly started build (server restart is required) (since 2017.2.4): teamcity.build.parameters.triggeredBy.enabled=false
https://youtrack.jetbrains.com/issue/TW-54969
 
Delete diagnostics thread dumps and traces after the number of days (since 2017.2.4):
teamcity.diagnostics.threadDumpsCleaner.maxDaysToKeep=30
teamcity.diagnostics.threadStatCleaner.maxDaysToKeep=30
https://youtrack.jetbrains.com/issue/TW-54874
There is a known issue that logs\catalina.out file if not rotated automatically at all. It is recommended to establish an automatic procedure to rotate the file regularly.

...