As previously mentioned there's no built-in way to pick a test agent automatically based on the selected test configuration, after much digging I have found a partial work around.
Theory
It turns out that like a lot of tfs components the test controller supports plugins. These inherit from TcmRunControllerPlugin and have 3 methods - Init, TestRunStarting and TestRunCompleted. Unlike other TFS Plugins they are not automatically detected and need to be manually added to the qtcontroller.config (in c:\program files (x86)\microsoft visual studio 11.0\common7\ide).
I found that on the TestRunConfiguration object there is a StringDictionary called AgentProperties, upon further inspection I noticed that it appeared to be used to filter test agents to find a suitable one to run the tests on. By default it has a filter for environment name, as a small test I added another hardcoded filter by overriding the Init method in a very simple plugin. To my surprise whenever tests were run they would now complain that no suitable agent could be found, by changing my filter it quickly came apparent that this method worked for machine tags.
By overriding the Init method and a bit more digging I managed to get a list of properties from the Test Configurations and put them as filters for the agent selected.
Limitations/Issues
I created a Test Configuration with values "IE:9" and another with "IE:10" and respectively labeled the web client machines with the same tags. Running the IE9 tests from MTM would always pick the machine tagged IE:9 and the IE10 ones would pick the machine tagged IE:10. Running both together would look for a machine labelled IE:9,10 which it wouldn't find and would fail. Unfortunately I have no way around this limitation currently.
When running coded ui tests via a lab build you can only select one Test Configuration anyway so this limitation is only a problem when manually running the tests in MTM.
The test controller plugins don't appear to be documented, which may mean this breaks in a future update.
Cross Browser Support
This idea can be extended to apply to other browsers too using the cross browser feature from update 1 simply create an extra machine for each browser you wish to test and label correctly. I'd probably change my labelling scheme from IE:9 to Browser:IE9 to be bit more consistant. On each machine add a system environment variable called browser set to the correct name, then in your coded UI test make sure you always grab this value and set BrowserWindow.CurrentBrowser to its value.
Advantages over other methods
- Tests can be run from MTM without needing to manually select a different environment to run it in (as long as you only run one Test Configuration at a time)
- Machines other than the test agents can be reused (eg database/webserver) meaning less hyper-v resources are needed
- Works fine with TFS2012's auto config of test clients which causes issues for the multiple test client role method
Potential Extensions (not sure if they're possible)
- A way to disable/enable this filtering (an extra property in a test configuration would easily do it)
- I'm still looking for ways to allow multiple Test Configurations to be run in one test run from MTM.
- Overriding the browser environment variable automatically so I can use the same virtual machine for IE, Chrome and Firefox (obviously still needing a second machine for IE9 vs IE10).
- Check if any other types of filters are available, I did see some xml floating around suggesting you may be able to filter based on Ram available.
