Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Migrated to Confluence 4.0

...

Then send the detailed description of the issue to us accompanied with the thread dumps and full server (or agent) logs covering the issue.

Server thread dump

...

If the server UI is not usable please use approaches described below.
Please note that if you can take a thread dump without using TeamCity UI, that can be preferable since the thread dump taken from UI can lack some diagnostics information.

...

To take a thread dump:
Under Windows
You have several options:

  • if you need server thread dump and the server is run from console, press Ctrl+Break in the console window (this will not work for agent, since it's console belongs to launcher process).
  • if JDK 1.6 is used, use jstack <pid_of_java_process> command (jstack is located in the "bin" directory of JDK installation). If it does not work, ensure you use the tools from the same version of JDK as is used to run the process you take the dump from (same x86/x64 and same version).
  • you can also use TeamCity-bundled thread dump tool (can be found in agent's plugins). Run the command:
    Code Block
    
    <TeamCity agent>\plugins\stacktracesPlugin\bin\x86\JetBrains.TeamCity.Injector.exe <pid_of_java_process>
    

If the hanging process is run as a service, taking the thread dump is more complicated and under some environments does not have a reliable approach. Recommended approach is to run the agent or server process from a console and use Ctrl+Break approach above.

...

Anchor
linux_thread_dump
linux_thread_dump

Under Linux

  • run jstack <pid_of_java_process> or kill -3 <pid_of_java_process>. In the latter case output will appear in <TeamCity server home>/logs/catalina.out or <TeamCity agent home>/logs/error.log.

You can also use third-party GUI tool: AdaptJ StackTrace Utility.
It supports Windows, Linux or Mac OS X. Choose "Launch" on the page (if you have Java installed). When the application is started, select the process id via "Process > Select", then dump its thread dump with "Process > Thread Dump".

See also Server Performance section below.

Database-related slowdowns

...

The log can also be sent to us for analysis.

Back to top

Anchor
oom
oom

OutOfMemory Problems

...

  • Determine what process encounters the error (the actual building process, the TeamCity server, or the TeamCity agent)
  • If server is to blame, please check you have increased memory settings from default ones for using the server in production (see the section).
  • If you use x64 JVM, please consider using 32 bit JVM (instructions for server).
  • Try to increase the memory for the process via '-Xmx' JVM option, like -Xmx1200m. If the error message is "java.lang.OutOfMemoryError: PermGen space", increase the value in -XX:MaxPermSize=270m JVM option.
    The option needs to be passed to the process with problems:
    • if it is the building process itself, use "JVM Command Line Parameters" settings in the build runner. e.g. Inspections builds may specifically need to increase the parameter;
    • refer to the corresponding documentation for TeamCity server or agent. See also Installing and Configuring the TeamCity server;
  • If increasing memory size does not help, please get the memory dump, archive it and send it to us for further analysis. Please reduce the Xmx setting to normal before getting the dump (smaller snapshots are easier to analyze and easier to upload):
    • to get a memory dump (hprof file) automatically when an OutOfMemory error occurs, add the following JVM option (works for JDK 1.5.0_07+): -XX:+HeapDumpOnOutOfMemoryError. When OOM error occurs next time, java_xxx.hprof file will be created in the process startup directory (<TeamCity home>/bin or <TeamCity Agent home>/bin);
    • you can also take memory dump manually when the memory usage is at its peak. Go to the Administration | Server Configuration | Diagnostics tab for your TeamCity web UI and click Dump Memory Snapshot.
    • another approach to take the memory dump manually is to run TeamCity server with JDK 1.6+ and use jmap standard JVM util. e.g. jmap -dump:file=<file_on_disk_to_save_dump_into>.hprof <pid_of_your_TeamCity_server_process>

See how to change JVM options for the server and for agents.

Back to top

Logging Events
Anchor
logging
logging

...

TeamCity Server Logs
Viewing Build Agent Logs

Back to top

Version Control Debug Logging
Anchor
vcs
vcs

...

First, please enable generic VCS debug logging, as described above.

Uncomment SVN-related parts (SVN.LOG appender and javasvn.output category) of Log4j configuration file on server and on agent (if agent-side checkout is used). The log will be saved to the logs/teamcity-svn.log file. Generic VCS log should be also taken from logs/teamcity-vcs.log

...

Build log and agent log will contain the line "Patch is saved to file ${file.name}"
Get the file and provide it with the problem description.

Back to top

Anchor
remote
remote

Remote Run Problems

The changes that are sent form the IDE to the server on a remote run can be retrieved from server's .BuildServer\system\changes directory. Locate the <change_number>.changes file that corresponds to your change (you can pick the latest number available or deduce the from the URL of the change form the web UI).
The file contains the patch in the binary form. Please provide it with the problem description.

Back to top

Anchor
serverperformance
serverperformance

...

Here are some hints to get the best results from CPU profiling:

  • after server startup wait for some time to allow it to "warm up". This can take from 5 to 20 minutes depending on the data volume that TeamCity stores.
  • when CPU usage increase is found on the server, please try to indicate what actions cause the load
  • start CPU profiling and repeat the action several times (5 - 10)
  • capture the snapshot
  • archive the snapshot and send it to us including description of the actions that cause CPU load

Back to top

Logging in IntelliJ IDEA/Platform-based IDEs

...

(Applicable to IntelliJ IDEA and Eclipse)
Add the following JVM option before starting IDE:
-Dteamcity.activation.debug=true
Logging related to open in IDE functionality will appear in the IDE console.

Back to top

Logging in TeamCity Eclipse plugin

...

Read more about Eclipse Debug mode Gathering Information About Your Plug-in and built-in Eclipse help.

Back to top

Logging in TeamCity Visual Studio plugin

...

  1. Close all instances of Microsoft Visual Studio.
  2. Open <username-profile-folder>\Local Settings\Temp\JetLogs folder.
  3. Delete all of the files in this folder.
  4. Restart Microsoft Visual Studio.
  5. Open a solution.
  6. Try to log in to TeamCity.
  7. Close Microsoft Visual Studio.
  8. Navigate back to the <username-profile-folder>\Local Settings\Temp\JetLogs folder. All of the files that were created are logs.

Back to top

Anchor
sendingLargeFiles
sendingLargeFiles

...

If the file is over 5 Mb, you can upload the archived files to ftp://ftp.intellij.net/.uploads and let us know the exact file name. If you receive permission denied error on upload attempt, please rename the file. It's OK that you do not see the file listing on the FTP.
The FTP accepts standard anonymous credentials: username: "anonymous", password: "<your e-mail>".

Back to top