4.09 Test Framework Support (R7)

Skip to end of metadata
Go to start of metadata

Most of ReSharper plug-ins involve support for different unit test frameworks. This page describes ways in which you can add unit test support to your own framework.

Overview

The first thing you need to understand about ReSharper testing is the way in which ReSharper actually finds tests. One would expect that the way of finding tests in ReSharper is by simply traversing the PSI (i.e., the syntax tree from parsed source code) looking for the appropriate attributes. However, on large solutions, this is too slow, which is why another approach exists: traversing the compiled assemblies and acquiring test information from there.

As a result, plugin writers need to support two separate test analysers - the metadata explorer and the file explorer.

Metadata Explorer

The metadata explorer is used to explore the metadata of compiled CLR assemblies to look for test elements. It is not applicable to non-CLR unit test frameworks such as QUnit.

In order to support the metadata explorer, plugin writers need to create a class decorated by the [ReSharper:MetadataUnitTestExplorer] interface and implementing the IUnitTestMetadataExplorer interface. This interface has two members:

  • The Provider property, which returns the unit test provider.
  • The ExploreAssembly() method, which is used to tell an explorer to explore the metadata for a particular assembly.

File Explorer

The file explorer acts in a way similar to the metadata explorer, the only difference being that instead of an ExploreAssembly() method it has an ExploreFile() method which, predictably, takes an IFile to explore.

A typical implementation of the ExploreFile() method is to simply take the file and call ProcessDescendants() on it, passing your own file explorer class which implements the IRecursiveElementProcessor interface. From then on, the idea is to traverse the PSI tree of the file (just like you would if you were writing an ordinary analyzer), identify the elements which constitute test fixtures, tests, and so on, and then construct appropriate IUnitTestElement entities.

IUnitTestElement

The IUnitTestElement interface is the core interface that is involved in unit testing. Essentially, this interface needs to be implemented by any class in your code that represents a test.

Currently, ReSharper currently has several base classes for test elements, such as MsTestElementBase or NUnitElementBase. These elements are further subclassed by the form in which a test element appears. Some examples are:

  • A fixture
  • A method
  • A row or test case

Other elements can be created to suit your needs. For example, if your tests are defined in fields, you could create a SomeFrameworkElementBase that implements IUnitTestElement and subsequently derive from it your SomeFrameworkFieldElement class.

If you look at the existing implementations of XxxElementBase classes, you will notice that each one of them takes a XxxTestProvider as a parameter to be injected in the constructor. (This provider is also returned in the Provider property.) This provider class is the class which actually explores the assembly and creates the tie-in between ReSharper and the unit testing framework of choice.

IUnitTestProvider

The unit test provider is a class which implements the IUnitTestProvider interface and is decorated with the UnitTestProvider attribute. The actual binding of the provider to a particular element happens in the IsElementOfKind() method. This method is used to determine whether the particular element is, in fact, an element of a particular kind. The possible kinds, which are part of the UnitTestElementKind enumeration, are as follows:

  • Unknown – we don’t know what this is
  • Test – the actual test which does does something. Here we mean something like a Test or a TestCase – both count as tests.
  • TestContainer – this represents a container of a set of tests. Typically, this matches a TestFixture, but if we have a test with several rows/test cases, then this would also cover a Test.
  • TestStuff – this is any element which relates to tests, such as all of the aforementioned elements plus elements such as SetUpFixture

There are, in fact, two overloads of IsElementKindOf(). One takes an IDeclaredElement, whereas another atakes a IUnitTestElement. The overload that takes an IUnitTestElement is simple – it simply implements a set of rules similar to the list above, for example:

The overload taking an IDeclaredElement is a bit more complicated. Essentially, this overload checks for the UnitTestElementKind on the actual code element. As a result, it is the responsibility of the developer to check that this is indeed the case.

Here’s an example. Let’s suppose that we want to determine whether something is a unit test. This would imply that our IDeclaredElement is:

  • An ITypeMember
  • An IMethod, since NUnit tests are kept in methods
  • Is public and not abstract
  • Is not generic
  • Has any attribute (direct or derived) from a set containing Test, TestCase, etc.

In order to determine whether a particular attribute has been applied to a method, we create declarations similar to the following

and we subsequently check that the attribute owner (i.e. our method) has this attribute using the HasInstanceAttribute() method. Determining derived types is a bit more complicated – take a look at the UnitTestAttributeCache class in ReSharper for an illustration of how this is handled.

RemoteRecursiveTaskRunner

We now come to what is arguably the most complicated part of all: the task runner which actually runs the unit tests. This class typically inherits from RecursiveRemoteTaskRunner and is expected to implement several methods from its parent types. Let’s take a look at some of them.

The ConfigureAppDomain() method is used to configure the AppDomain for running the tests. This method takes a TaskAppDomainConfiguration object and is expected to modify it, setting things like the priority or the apartment state. Some of these settings can be read from the configuration file (see NUnitTaskRunner for an example).

The ExecuteRecursive() method is the method that actually executes the tasks.

If you peek into the NUnitTaskRunner, you may see something strange: infrastructure code being acquired from resources and compiled before test execution. Please note that this approach is only necessary if you want one version of your plug-in to support many versions of your unit test framework. (And, even if you do that, you will have problems because of changing APIs.) Thus, if you are making a plug-in for your own unit test framework, your best bet is to simply keep the test framework and the plug-in in sync, i.e. release plug-ins that correspond to your unit test framework versions