Get the ReSharper binaries of the build you want to register into the experimental hive by running the regular installation. If you do not want to install R# for any of the main hives, uncheck all of the Visual Studio integration checkboxes. This will produce the needed set of binaries and omit any per-VS-version integration steps.
Register the R# binaries for the desired Visual Studio hive with the JetBrains.VSIntegration.Install.InstallExperimentalHive build task from the JetBrains.Platform.ReSharper.VSIntegration assembly. For an example task usage, see the attached InstallExperimentalHive.Proj file.
Note that the hive must already exist and that devenv /Setup for the appropriate /RootSuffix must be run manually.
Side-by-Side ReSharper Installations
You may want to install more than one R# instance on the same machine, but for different Visual Studio hives. Check for the upgrade options in the Advanced installation scenario.
The installation logs are very useful for diagnosing installation problems. The logs can be collected with an installer command line switch: msiexec /i ReSharperSetup.msi /Log Install.log
If you're installing more than one ReSharper side-by-side, make sure you specify a different installation location for each of the instances. This can be selected in the installation wizard, or specified in the INSTALLDIR installer property. Also, the installer will try to pick a unique directory on its own.