Here are some hints to get the best results from CPU profiling:
- after starting the server, wait for some time to allow it to "warm up". This can take from 5 to 20 minutes depending on the data volume that TeamCity stores.
- when a CPU usage increase is found on the server, please try to indicate what actions cause the load.
- start CPU profiling and repeat the action several times (5 - 10).
- capture a snapshot.
- archive the snapshot and send it to us including the description of the actions that cause the CPU load.
Agent Thread Dump
It is recommended that you take an agent thread dump from the Web UI: go to the Agent page, Agent Summary tab, and use the Dump threads on agent action.
You have several options:
- To take a server thread dump if the server is run from the console, press Ctrl+Break in the console window (this will not work for an agent, since its console belongs to the launcher process).
- Alternatively, run
jstack <pid_of_java_process>in the
bindirectory of the JVM used to by the application.
- To take an agent thread dump or if your server is running as a service, use the TeamCity-bundled thread dump tool (can be found in the agent's plugins). Run the command:
Note that if the hanging process is run as a service, the server or the agent process must be run from a console with elevated permissions (using Run as Administrator). If the service is run under System account, you might also need to launch the thread dumping tools via "PsExec.exe -s <path to the tool>\<tool> <options>"
<TeamCity agent>\plugins\stacktracesPlugin\bin\x86\JetBrains.TeamCity.Injector.exe <pid_of_java_process>
jstack <pid_of_java_process>(using jstack from the same JVM as used by the application) or
kill -3 <pid_of_java_process>. In the latter case output will appear in
<TeamCity server home>/logs/catalina.outor
<TeamCity agent home>/logs/error.log.
See also Server Performance section below.
To collect additional logs generated by JetBrains dotCover, add the
teamcity.agent.dotCover.log configuration parameter to the build configuration with a path to an empty directory on the agent.
All dotCover log files will be placed there and TeamCity will publish zipped logs as hidden build artifact
On a rare occasion of the TeamCity server or agent process terminating unexpectedly with no apparent reason, it can happen that this is caused by a Java runtime crash.
If this happens, the Oracle JVM creates a file named
hs_err_pid*.log in the working directory of the process. The working directory can be
<TeamCity server or agent home>/bin or other like
C:\Windows\SysWOW64. You can also search the disk for the recent files with "hs_err_pid" in the name.
See also Oracle documentation, the Fatal Error Log section.