- Project Settings Configuration Changes
- Two-node server configuration
- Team Foundation Work Items Tracking
- Cloud Support
- Email Verification
- Exclude Patterns for Artifact Paths
- Agent-Server Communication
- Project-based Agent Management Permissions
- Flaky Test Detection
- New Create project / Create build configuration buttons
- REST API Enhancements
- Bundled Tools Updates
- Other Improvements
Project Settings Configuration Changes
Starting from this EAP, configuration of issue trackers, versioned settings, custom charts, shared resources and third-party report tabs was moved from
<TeamCity Data Directory>/config/projects/<ProjectID>/pluginData/plugin-settings.xml to
<TeamCity Data Directory>/config/projects/<ProjectID>/project-config.xml file. The file now has the
<project-extensions> element which contains all of the above-mentioned project features.
Two-node server configuration
Starting with this EAP TeamCity server can operate in two modes: main TeamCity server (default mode), build messages processor.
If TeamCity server is started in build messages processor mode its functionality is reduced to only one feature - process real time data coming from the running builds.
The primary purpose of this mode is to provide scalability option for really large installations, running several hundreds of agents on a single server now and planning on expanding their installation in the future.
Several important notes about two-node configuration:
- at the moment build messages processor node handles all of the data produced by running builds (build logs, artifacts, statistic values), pre-processes it and stores in database
- in two-node installation, both main TeamCity server and build messages processor require access to the same data directory (which should be shared somehow if nodes are installed on separate machines) and to the same database
- URL where build messages processor operates should be accessible by the agents and main TeamCity server (occasionally main TeamCity server also communicates with build messages processor by HTTP)
- main TeamCity server handles all other tasks: user interface, VCS related activity, management of agents, etc
First of all ensure that both machines where TeamCity software will be installed share the same TeamCity data directory. For large installations we recommend using NAS. It is possible to share directory using NFS, or windows share, although performance of such installation can be worse.
Once you have two machines, proceed with installing TeamCity software as usual: download distribution package, unpack it or follow installation wizard.
Configure TEAMCITY_DATA_PATH environment variable on both machines, make sure it points to shared data directory.
To start main TeamCity server, follow our usual instructions.
Before starting server in build messages processor node, add additional arguments to TEAMCITY_SERVER_OPTS environment variable, for example:
export TEAMCITY_SERVER_OPTS=-Dteamcity.server.mode=build-messages-processor -Dteamcity.server.rootURL=<processor url> <your regular options if you have them>
Where <processor url> is the URL where build messages processor will operate. This URL should be accessible by both agents and main server. If you do not have HTTP proxy installed in front of TeamCity servlet container and you did not change port in servlet container during the installation then by default this URL will be: http://<your host name>:8111/
To start build messages processor use our regular scripts: teamcity-server.bat or teamcity-server.sh
Build messages processor uses the same approach to logging as the main server. You can check how startup is going in teamcity-server.log file. You can also open <processor URL> in your browser, there you should see regular TeamCity startup screens.
Enabling build messages processor
If both - main server and build messages processor are up and running, you should see "Nodes configuration" page in Administration area on main TeamCity server:
By default build messages processor is disabled. And all traffic produced by running builds is still handled by the main server. Once you enable build messages processor, all newly started builds will be routed to this node. Existing running builds will continue their execution on the main server.
And vice versa, if you decide to disable messages processor, only newly started builds will be switched to the main server, builds that were already running on the messages processor will continue running there.
At any point of time you can see how many builds are currently assigned to each node. If everything is configured correctly (node is accessible by agents) and there are no problems with processing builds on the messages node, eventually all of the running builds should be switched to processor node once you enable it.
Restarting servers while builds are running
Build messages processor as well as main TeamCity server can be stopped or restarted while builds are running there. If agents can't connect to build messages processor for some time, they will re-route their data to main server. If main server is also unavailable, agents will keep their data and re-send it once servers re-appear.
Upgrade sequence should be the following:
- start upgrade on the main TeamCity server as usual
- at some point you will be warned that there is build messages processor running and using the same database; you'll need to shutdown it first
- proceed with upgrade
- make sure everything is ok, agents are connecting etc (since messages processor is not available anymore, agents will re-route date to the main server)
- upgrade software on the messages processor machine to the same version
- start messages processor and check that it is connected via Nodes configuration page on the main server
At the moment build messages processor node does not support plugins.
Team Foundation Work Items Tracking
Since TeamCity 10, Team Foundation Work Items tracking is integrated with TeamCity. Supported versions are Microsoft Visual Studio Team Foundation Server 2010-2015, and Visual Studio Team Services.
TFS work items support can be configured on the Issue trackers page for a project. If a project has a TFVC root configured, TeamCity will suggest configuring the issue tracker as well.
By default, the integration works the same way as the other issue tracker integrations: you need to mention the work item ID in the comment message, so the work items can be linked to builds and the links will be displayed in various places in the TeamCity Web UI. Additionally, if your changeset has related work items,TeamCity can retrieve information about them even if no comment is added to the changeset. Besides, custom states for resolved work items are supported by TeamCity.
In addition, resolved states in TeamCity can be customized by using the
teamcity.tfs.workItems.resolvedStates internal property set to
"Closed?|Done|Fixed|Resolved?|Removed?" by default.
- When configuring a cloud image, you can now select an agent pool for the newly created cloud agents. Previously all of them were placed in the default pool.
- Custom names for agent images are supported now. The names of virtual machines in VMware must be unique. When using the same image in different cloud profiles, to avoid possible conflicts, use the custom agent image name when configuring a cloud profile in TeamCity. This feature can be also useful with naming patterns for agents. When a custom agent image name is specified, the names of cloud agent instances cloned from the image will be based on this name.
- EBS optimization is turned on by default for all instance types supporting it.
TeamCity administrators can enable / disable email verification (off by default) on the Administration | Authentication page.
If email verification is enabled on the TeamCity server, the Email address field in the user account registration form becomes mandatory. When an address is added / modified, the users will be asked to verify their it. If the email address is not verified, TeamCity will display a notification on the General tab of the user account settings. Verified email addresses will be marked with a green check on the Administration | Users page.
When project import scope if configured, users with the same username and email are compared based on their email verification. TeamCity will display the conflicts information and the administrator can choose whether to merge the users found.
Exclude Patterns for Artifact Paths
It is now possible to specify newline- or comma-separated paths in the form of
-:source [ => target] to exclude files or directories from publishing as build artifacts.
Rules are grouped by the right part and are applied in the order of appearance, e.g.
will tell TeamCity to publish all files except for
folder1 into the
Starting from this EAP TeamCity uses unidirectional agent-to-server connection via the polling protocol by default. If for some reason the polling protocol cannot be used, TeamCity switches to the fallback bidirectional communication via xml-rpc.
Project-based Agent Management Permissions
Starting from this EAP, TeamCity introduces 6 new permissions added to Project Administrators:
1) Enable / disable agents associated with project
2) Start / Stop cloud agent for project
3) Change agent run configuration policy for project
4) Administer project agent machines (e.g. reboot, view agent logs, etc.)
5) Remove project agent
6) Authorize project agent
These agent permissions are project-based. Additionally, these permissions can provide agent pool management rights: if a person is granted a permission to perform a certain agent management action for all projects within a pool, this user can perform this action on all agents in this pool.
If an agent within a pool is assigned to a project where no such permission is granted to the user, the pool management right is revoked.
The Project Administrator role will no longer include the Agent Manager role for the new TeamCity installations. The existing installations will not be affected by this change, but the new permissions are added to Project Managers and it is possible to exclude the inherited agent manager role manually.
Flaky Test Detection
TeamCity now supports flaky test detection. A flaky test is a test that is unstable (can exhibit both a passing and a failing result) with the same code.
Flaky test detection is based on the following heuristics:
- High flip rate (Frequent test status changes). A flip in TeamCity is a test status change — either from OK to Failure, or vice versa. The Flip Rate is the ratio of such "flips" to the invocation count of a given test, measured per agent, per build configuration, or over a certain time period (7 days by default). A test which constantly fails, while having a 100% failure rate, will have its flip rate close to zero; a test which "flips" each time it is invoked will have the flip rate close to 100%.
If the flip rate is too high, TeamCity will consider the test flaky.
- Different test status for build configurations with the same VCS change: if two builds of different configurations are run on the same VCS change and the test result is different in these builds, the test is reported as flaky. This may be an indication of environmental issues.
- If the status of a test 'flipped' in the new build with no changes, i.e. a previously successful test failed in a build without changes or a previously failing test passed in a build without changes, TeamCity will consider the test flaky.
- Different test status for multiple invocations in the same build: if the same test is invoked multiple times and the test status flips, TeamCity will consider the test flaky.
Such tests are displayed on the dedicated project tab, Flaky Tests, along with the total number of test failures, the flip rate for the given test and reasons for qualifying the test as a flaky one. You can also see if the test is flaky when viewing the expanded stacktrace for a failed test on the build results page.
As with any failed test, you can assign investigations for a flaky test (or multiple tests). For flaky tests the resolution method is automatically set to 'Manual'; otherwise the investigation will be automatically removed once the test is successful, which does not mean that the flaky test has been fixed.
Note that is branches are configured for a VCS Root, flaky tests are detected for the default branch only.
New Create project / Create build configuration buttons
The new Create subproject and Create build configuration buttons have a drop-down now and you can select whether you want to create a project from scratch (manually), from URL, or using the popular version control systems GitHub.com and Bitbucket. When one of the latter two is selected, TeamCity offers to configure a connection to the VCS hosting for the current project. When the connection is configured, TeamCity displays the list of the available repositories with their URLs. All you have to do is select a repository URL and proceed with the configuration.
REST API Enhancements
- Project and build configuration ordering is now exposed via REST API
- Maximum number of agents in a pool is now supported via REST API
- Project features are now exposed via REST API
- License list is now exposed through REST API
Bundled Tools Updates
Maven-related operations performed on the server-side are now moved to separate process
New option added to Subversion VCS root: Enable non-trusted SSL certificate; if this option is enabled, TeamCity will be able to connect to SVN servers without properly signed SSL certificate