If you experience problems running TeamCity and believe they are related to the software, please contact us with a detailed description of the issue.

To fix a problem, we may need a wide range of information about your system as well as various logs. The section below explains how to collect such information for different issues.

In this section:

Slowness, Hangings and Low Performance

If TeamCity is running slower than you would expect, please use the notes below to locate the slow process and send us all the relevant details if the process is a TeamCity one.

Determine Which Process Is Slow

If you experience a slow TeamCity web UI response, checking for changes process, server-side sources checkout or other slow server activity, your target should be the machine where the TeamCity server is installed.
If the issue is related only to a single build, you will need to also investigate the TeamCity agent machine which is running the build.
Investigate the system resources (CPU, memory, IO) load. If there is a high load, determine the process to causing it. If it is not a TeamCity-related process, that might need addressing outside of the TeamCity scope.

If it is the TeamCity server which is loading the CPU/IO or there is no substantial CPU/IO load and everything runs just fine except for TeamCity, then this is to be investigated further.

If you have a substantial TeamCity installation, please check you have appropriate memory settings as the first step.

Collect Data

During the slow operation, take several thread dumps of the slow process (see below for thread dump taking approaches) with 5-10 seconds interval. If the slowness continues, please take several more thread dumps (e.g. 3-5 within several minutes) and then repeat after some time (e.g. 10 minutes) while the process is still being slow.

Then send us a detailed description of the issue accompanied with the thread dumps and full server (or agent) logs covering the issue. Unless it is undesirable for some reason, the preferred way is to file an issue into our issue tracker . Please include all the relevant details of investigation, including the CPU/IO load information, what specifically is slow and what is not, etc.

Server Thread Dump

It is recommended that you take a thread dump of the TeamCity server from the Web UI (if the hanging is local and you can still open the TeamCity Administration pages): go to the Administration | Server Administration | Diagnostics page and click the View server thread dump link to open the thread dump in a new browser window or Save Thread Dump button to save it to the <TeamCity home>/logs directory (where you can later download the files from "Server Logs").
If the web UI is not responsive, try the direct URL using the actual URL of your TeamCity server.

If the UI is not accessible, you can take a server thread dump manually using the approaches described below.

Collecting CPU Profiling Data on Server

If you experience degraded server performance and the TeamCity server process is producing a large CPU load, take a CPU profiling snapshot and send it to us accompanied with the detailed description of what you were doing and what your system setup is.
You can take CPU profiling and memory snapshots by installing the server profiling plugin and following the instructions on the plugin page.

Here are some hints to get the best results from CPU profiling:

Agent Thread Dump

It is recommended that you take an agent thread dump from the Web UI: go to the Agent page, Agent Summary tab, and use the Dump threads on agent action.

If the UI is not accessible, you can take the dump thread manually using the approaches described below. Note that the TeamCity agent consists of two java processes: the launcher and agent itself. The agent is triggered by the launcher. You will usually be interested in the agent (nested) process and not the launcher one.

Taking Thread Dump

These can help if you are unable to take a thread dump from the TeamCity web UI.

To take a thread dump:

Under Windows

You have several options:

Under Linux

See also Server Performance section below.

Database-related Slowdowns

When the server is slow, check if the problem is caused by database operations.
It is recommended to use database-specific tools.

You can also use the debug-sql server logging preset. Upon enabling, all the queries which take longer 1 second will be logged into the teamcity-sql.log file. The time can be changed by setting the teamcity.sqlLog.slowQuery.threshold internal property. The value should be set in milliseconds and is 1000 by default.


Along with the server thread dump, please attach the output of the "show processlist;" SQL command executed in MySQL console. Like with thread dumps, it makes sense to execute the command several times if slowness occurred and send us the output.
Also, MySQL can be set up to keep a log of long queries executed with the changes in my.ini:


The log can also be sent to us for analysis.

Back to top

OutOfMemory Problems

If you experience problems with TeamCity "eating" too much memory or OutOfMemoryError in the server log, please do the following:

See how to change JVM options for the server and for agents.

Back to top

"Too many open files" Error

  1. Determine what computer it occurs on
  2. Determine the process locking the files (on Linux use lsof, on Windows you can use handle or TCPView for listing sockets) and the files list
  3. If the number is not large, check the OS and process limits on the file handles (on Linux use ulimit -n) and increase them if necessary. Please note that default Linux 1024 handles per process is way too small for a server application like TeamCity. Please increase the number to at least 16000. Please check the actual process limits after the change as there are different settings in the OS for settings global and per-session limits.

If the number of files is large and is looking suspicions and the locking process is a TeamCity one (TeamCity agent or server with no other web applications running), grab the list of open handles several times with several minutes interval and send the result to us for investigation together with relevant details.

Please note that you will most probably need to reboot the machine with the error after the investigation to restore normal functioning of the applications.

Logging events

TeamCity server and agent create logs that can be used to investigate issues.

Before reproducing the problem it makes sense to enable 'DEBUG' log level for TeamCity classes.
On the server side, go to the Administration | Server Administration | Diagnostics page and select debug logging level ('debug-all', 'debug-vcs', etc).

After that, DEBUG messages will go to teamcity-*.log files (read more).

For detailed information, please refer to the corresponding sections:

TeamCity Server Logs
Viewing Build Agent Logs

Back to top

Version Control debug logging

To enable VCS logging on the server side, switch logging preset to "debug-vcs" in administration web UI and then retrieve logs/teamcity-vcs.log log file.

Most VCS operations occur on the TeamCity server, but if you're using agent-side checkout, VCS checkout occurs on the build agents.

Forthe agent and the server, you can change the Log4j configuration manually in <TeamCity home>/conf/teamcity-server-log4j.xml or <BuildAgent home>/conf/teamcity-agent-log4j.xml files to have fragment:

<category name="jetbrains.buildServer.VCS" additivity="false">
    <appender-ref ref="ROLL.VCS"/>
    <appender-ref ref="CONSOLE-ERROR"/>
    <priority value="DEBUG"/>

<category name="jetbrains.buildServer.buildTriggers.vcs" additivity="false">
    <appender-ref ref="ROLL.VCS"/>
    <appender-ref ref="CONSOLE-ERROR"/>
    <priority value="DEBUG"/>

Please also update <appender name="ROLL.VCS" node to increase number of the files to store:

    <param name="maxBackupIndex" value="30"/>

If there are separate logging options for specific version controls, they are described below.

Subversion debug logging

To enable SVN logging on the server side, switch logging preset to "debug-SVN" in the administration web UI and then retrieve logs/teamcity-vcs.log and logs/teamcity-svn.log log files.

An alternative manual approach is also necessary for agent-side logging.

First, please enable generic VCS debug logging, as described above.

Uncomment SVN-related parts (SVN.LOG appender and javasvn.output category) of the Log4j configuration file on the server and on the agent (if agent-side checkout is used). The log will be saved to the logs/teamcity-svn.log file. Generic VCS log should be also taken from logs/teamcity-vcs.log


Uncomment Clearcase-related lines in the <TeamCity home>/conf/teamcity-server-log4j.xml file.
The log will be saved to logs/teamcity-clearcase.log directory.

Patch Application Problems

In case the server-side checkout is used, the "patch" that is passed from server to the agent can be retrieved by:

the build log and the agent log will contain the line "Patch is saved to file ${file.name}"
Get the file and provide it with the problem description.

Back to top

Remote Run Problems

The changes that are sent form the IDE to the server on a remote run can be retrieved from server's .BuildServer/system/changes directory. Locate the <change_number>.changes file that corresponds to your change (you can pick the latest number available or deduce the from the URL of the change form the web UI).
The file contains the patch in the binary form. Please provide it with the problem description.

Back to top

Logging in IntelliJ IDEA/Platform-based IDEs

To enable debug logging for IntelliJ Platform based IDE plugin, you can manually change the Log4j configuration in <IDE home>/bin/log.xml file to have the fragment:

<appender name="TC-FILE" class="org.apache.log4j.RollingFileAppender">
  <param name="MaxFileSize" value="10Mb"/>
  <param name="MaxBackupIndex" value="10"/>
  <param name="file" value="$LOG_DIR$/idea-teamcity.log"/>
  <layout class="org.apache.log4j.PatternLayout">
    <param name="ConversionPattern" value="%d [%7r] %6p - %30.30c - %m \n"/>

<appender name="TC-XMLRPC-FILE" class="org.apache.log4j.RollingFileAppender">
  <param name="MaxFileSize" value="10Mb"/>
  <param name="MaxBackupIndex" value="10"/>
  <param name="file" value="$LOG_DIR$/idea-teamcity-xmlrpc.log"/>
  <layout class="org.apache.log4j.PatternLayout">
    <param name="ConversionPattern" value="%d [%7r] %6p - %30.30c - %m \n"/>

<category name="jetbrains.buildServer.XMLRPC" additivity="false">
  <priority value="DEBUG"/>
  <appender-ref ref="TC-XMLRPC-FILE"/>

<category name="jetbrains.buildServer" additivity="false">
  <priority value="DEBUG"/>
  <appender-ref ref="TC-FILE"/>

After changing this file please restart the IDE. TeamCity plugin debug logs will appear in <IDE data directory>/system/log directory (idea-teamcity* files).

Open in IDE Functionality Logging

(Applicable to IntelliJ IDEA and Eclipse)
Add the following JVM option before starting IDE:
Logging related to open in IDE functionality will appear in the IDE console.

No Suitable Build Configurations Found for Remote Run

First of all, check that your VCS settings in IDEA correspond to the VCS settings in TeamCity.
If they do not, change them and it should fix the problem.

If the settings are the same but the problem is there, do the following:

Back to top

Logging in TeamCity Eclipse plugin

To enable tracing for the plugin, run Eclipse IDE with the -debug <filename> command line parameter. The <filename> portion of the argument should be a properties file containing key-value pairs. The name of each property corresponds to the plugin module and the value is either 'true' (to enable debug) or 'false'. Here is an example of enabling most common tracing options:

jetbrains.teamcity.core/debug = true
jetbrains.teamcity.core/debug/communications = false
jetbrains.teamcity.core/debug/ui = true
jetbrains.teamcity.core/debug/vcs = true
jetbrains.teamcity.core/debug/vcs/detail = true
jetbrains.teamcity.core/debug/parser = true
jetbrains.teamcity.core/debug/platform = true
jetbrains.teamcity.core/debug/teamcity = true
jetbrains.teamcity.core/perfomance/vcs = true
jetbrains.teamcity.core/perfomance/teamcity = true

Read more about Eclipse Debug mode Gathering Information About Your Plug-in and built-in Eclipse help.

Back to top

TeamCity Visual Studio Addin issues

TeamCity Addin logging

To capture logs from TeamCity Visual Studio Addin please run Microsoft Visual Studio executable (devenv.exe) with /TeamCity.LogFile <PATH_TO_FILE> switch specified.

Visual Studio logging

To troubleshoot common Visual Studio problems please run Visual Studio executable file with /Log command Line switch and send us resulting log file.

Back to top

JVM Crashes

On a rare occasion of TeamCity server or agent process terminating unexpectedly with no reason, it can happen that this is caused by Java runtime crash.
If this happens, Oracle JVM creates a file named hs_err_pid*.log in the working directory of the process. Working directory can be <TeamCity server or agent home>/bin or other like C:\Windows\SysWOW64. You can search the disk for the recent files with "hs_err_pid" in the name.
See also related description.

Please send this file to us for investigation and consider updating JVM for the server (or for agents) to the latest version available.

Build Log Issues

While investigating issues related to build log we might need raw binary build log as stored by TeamCity.
It can be downloaded via Web UI from the Build Log build's tab: select "Verbose" log detail and use "raw messages file" link at the top-right.

Sending Information to the Developers

Files under 10Mb in size can be attached right into the tracker issue (if you do not want the attachments to be publicly accessible, limit the attachment visibility to "teamcity-developers" user group only).
You can also send small files (up to 2 Mb) via email: teamcity-feedback@jetbrains.com Please do not forget to mention your TeamCity version and environment and archive the files before attaching.

If the file is over 10 Mb, you can upload the archived files to ftp://ftp.intellij.net/.uploads and let us know the exact file name. If you receive the permission denied error on an upload attempt, please rename the file. It's OK that you do not see the file listing on the FTP.
The FTP accepts standard anonymous credentials: username: "anonymous", password: "<your e-mail>".
In addition to usual, unencrypted connections, TLS ones are also supported.

In case of access issues, time-out errors, etc. please try using passive FTP mode.

Back to top