Tfs Automation - Automatically creating queries when new areas are created

Our testers got sick of setting up new queries whenever they created a new area in TFS so asked me if I could automate it.
The solution is broken into 2 parts:

AreaCreatedSubscriber

This is ISubscriber that watches for StructureChangedNotification events and queues a one time job to create the queries.
The reason I queued a job instead of just doing the work in this plugin was that when creating the query TFS was throwing an exception on validation (TF51011: The specified area path does not exist.) It seems that there is a slight delay between adding areas and them becoming valid for use in queries. I thought it was a caching issue but even when explicitly clearing the cache on the workitemstore it still happens, I found a post (New Area Path values are not available in work items) that suggested that tfs subscribes to its own events which then triggers a process of making the area available. This meant I needed to delay the creation of the queries until after all the events had completed, hence the job queue.
Also it is fairly good practice to push any event semi long running tasks into a job as it can delay actions for end users.
This gets installed to c:\Program Files\Microsoft Team Foundation Server 11.0\Application Tier\Web Services\bin\Plugins

CreateDefaultQueriesJob

The second part is the actual job, its also fairly simple (although crudely written). It reads the Area path out of the event data and creates a folder hierarchy for the queries to live in. It then checks if the queries already exist and if not creates them.
This job should probably be extended/cleanedup to read the queries from an external source, whether it be a WIQL file on the disk, some metadata stored on the team project, or just a special folder in source control.
This is installed into c:\Program Files\Microsoft Team Foundation Server 11.0\Application Tier\TFSJobAgent\plugins

Debugging

I had a bunch of issues trying to get this to work.
Firstly the TF51011 error - folders were appearing in the shared queries so I knew the plugin was doing something, luckily the exception was being logged to the event logs so it was really easy to find.
After moving the logic to a one time job I tried testing it only to find nothing happened. No folders created and nothing logged in the event logs. I figured that the event was still being fired so it was probably queueing the job, after a quick google I came across a blog article from Martin Hinshelwood debugging the TFS Active directory sync job this gave me a bunch of helpful debugging tips.
This lead me to the tfs project collection database
SELECT * FROM [Tfs_Enlighten].[dbo].[tbl_JobDefinition] where ExtensionName like '%CreateDefaultQueriesJob'
Returned rows but they weren't much help, jobs were being queued but I had no idea about if they were successful or not. I eventually discovered another table in the tfs_configuration database which contained the jobHistory
select * from tfs_configuration.dbo.tbl_JobHistory where jobid in (SELECT [JobId] FROM [Tfs_Enlighten].[dbo].[tbl_JobDefinition] where ExtensionName like '%CreateDefaultQueriesJob')
Also had one row for every time I queued the job, with the status result of 6, this time I needed to do some diving with ILSpy, which lead me to this enum

ExtensionNotFound = 6 - TFS couldn't find my job! I double and triple checked all my code, and started diving through the JobRunner code in TFS trying to figure out why it wasn't loading. I noticed a few hints of MEF in the code so tried adding an [Export] attribute to the class, still no luck. Enabling the trace log by editing c:\Program Files\Microsoft Team Foundation Server 11.0\Application Tier\TFSJobAgent\TfsJobAgent.exe.config I started seeing a curious exception about a plugin being added to a dictionary twice.

Especially interesting as it appears to check if the plugin is already registered in the dictionary before adding it, which makes me think they have a threading bug.
After removing the [Export] attribute and restarting the tfs job service a few times it seemed to come right.

Update: Apparently there's some new web access interfaces to view the status of TFS Jobs (and more), may have meant less database diving had I known at the time

Tfs Automation - Create Builds Automatically for New Solutions

I recently wrote a tfs server plugin that creates new builds automatically whenever you add or branch a solution file.  It's fairly simple, but worth sharing.

It takes advantage of another piece of code I had previously written (or perhaps found online, I honestly can't remember now) to create builds using the TFS api.

Installing it is very simple, just build the class into an assembly and copy it to c:\Program Files\Microsoft Team Foundation Server 11.0\Application Tier\Web Services\bin\Plugins you may also need to copy a couple of the referenced dlls to the same folder.

Creating a build using the Tfs API

Here is a snippet of code to create tfs builds using the API with a few sensible default settings.

Web Code Coverage for Manual Tests

Found a great little tool which I think may be new in VS/TFS 2012 (it was possible in previous versions but seemed much harder to setup)

 C:\Program Files (x86)\Microsoft Visual Studio 11.0\Team Tools\Dynamic Code Coverage Tools\CodeCoverage.exe collect /iis /output:c:\outputfile.coverage

 It will restart IIS and record a coverage file which can be later opened in visual studio. Great to set going when testers are doing any manual testing in a development environment

Very Simple TFS 2012 Web Access Plugin

Made a very basic tfs plugin to make the tfs logo a link back to the homepage.

Manifest.xml
clickable.min.js
Then zip them up together and upload it in the tfs admin section.

Obviously not exactly the best plugin in the world, considering we actually bypass the whole plugin system, but it works and may be of help to someone.

TFS Extensibility

In this post I hope to outline all the extensibility points in TFS 2012 and come up with a reason I might use it for the company I work for.  I would define this as both a major strength and weakness in TFS, there's a lot of powerful stuff exposed to developers but there's not a lot of documentation and a fair bit of it seems half complete which all results in very little adoption.  The community around customizing TFS seems fairly small in my opinion, but could be a major win for Microsoft.

Tfs Client API - .NET
A set of .NET classes to interact with the various webservices exposed by TFS, this is the main interaction point from which Visual studio and all components that don't need to be installed on the TFS server directly, for this reason it is extremely powerful.

We mainly use it to build a custom TFS web frontend for our clients for logging bugs which is a bit more cut down than the TFS web access. The need for this has reduced drastically with the release of TFS 2012 however it does mean we can integrate other systems giving our clients a one stop site for all their needs (rather than requiring them to login to 3-4 different systems).  We are careful that users can only see any work items that they create so that they don't require a special TFS CAL. The built in boards and backlog in TFS 2012 require a higher level CAL, but you can build your own backlog/board that works under the limited CAL.

We also make use of the API in a few ways to help manage the TFS Server. For example automatically granting permissions to TFS teams for any areas/iterations they're associated with, that way I can just assign people to project teams and they'll have the permissions they need.  Another example is a set of sanity checks which do things like making sure every solution in source control has an associated build.

Tarun Arora, Neno Loje, and Shai Raiten all blog extensively on this subject.

TFS Build Templates - XAML, .NET
TFS Builds in 2010 and higher are controlled by .NET Workflows, this includes a lot of the Lab management functionality.  There are lots of builtin activities you can add to your templates and you can write your own activities in Visual studio and include you (or use one of the many activities that are part of the community tfs build extensions).

I'm not a fan of the editor and using custom assemblies causes all kinds of pain just to get the custom activities into Visual Studio so you can add them.  You can't easily just grab a component off someones blog, add it to your template, test it and deploy it.  This gets even more complex when you've already customized your template so can't just replace the entire template and have to add their changes manually.

For the above reasons I typically try keep template customizations to a minimum.  I believe the only customization I have currently is to set the default build quality of a successful build so that I can make use of TFS Deployer to deploy the same binaries in turn to each environment for testing.  TFS Deployer then makes use of powershell scripts, which I find much easier to customise and test.

TFS Soap Subscriptions - SOAP
The tfs api (mentioned above) allows you to subscribe to a number of events via soap webservices, this allows your app to get notified when any of the following events occurs in TFS.

• Work item changes
• Check ins
• Build completes
• Build quality changes (this is what TFS Deployer uses)
• Project is created or deleted
• Branch moved
• Security Acl changes
• Iteration/Area changes

Note: The last 4 may be client events only, meaning only the client that triggers the event gets notified.

Back in 2005/2008 we used to use this notification service to send out emails when a workitem was assigned (instead of getting every developer to create a subscription themselves), in 2010 we migrated this to use the server events instead and finally replaced it with a single out of the box team subscription in tfs 2012.

These days I don't make use of any of these events myself (except obviously build quality change). I can imagine subscribing to area/iteration changes to automatically create folders in SharePoint, much like the proposed TFS Automation Platform from Martin Hishelwood, sadly this project never seemed to get off the ground.

TFS Server-side Event Handlers - .NET
Location: %Program Files%\Microsoft Team Foundation Server 11.0\Application Tier\Web Services\bin\Plugins

TFS 2010 introduced the ability to create server side event subscribers by implementing the ISubscriber interface and dropping the resulting assembly into the plugin directory and access the same events as the soap subscriptions.

These seemed a lot more reliable than the soap subscriptions, but had to process fairly quickly as it blocked TFS from finishing what it was doing. As mentioned earlier we used this to send email alerts for assigned work items, if you ever did a bulk update (100 work items) TFS would lock up and not respond for up to a minute.  The solution was either to spin up a background thread (which I believe had a chance to be killed) or queue a custom TFS Job.

Because TFS waits for you to process it means you can use events as Decision Points to allow or deny certain actions from occurring.  This means you can add extra custom validation to workitems and checkins, there are better ways to do both but sometimes you need access to code to do more advanced checks.

Apart from emailing notifications, I've seen a plugin that can associate workitems to checkins based on the comments (and heard of someone doing the reverse - setting comments based on the associated workitems).

Update: I created a few sample server side event plugins that automatically creates builds for new solution files, automatically creates workitem queries for new areas

MTM/TestCase server notifications are also available in 2012.

Check In Policies - .NET
Location: Developer machines

Code that can ensure particular requirements are met before a checkin can occur, examples:

• Project Compiles
• Tests pass
• Comments added
• WorkItem associated
• Passes StyleCop checks
• WorkItems associated are in particular areas/queries
• Regex patterns on filenames/contents

New policies can be created by extending the PolicyBase class, and deploying them to your development machines.

These are annoying to deploy as they get installed on each developers machine, not the TFS server. They became easier in VS2010 with the ability to install using a vsix, in Visual Studio 2012 it became possible to create private extension galleries to help deploy extensions within your enterprise.  However as far as I'm aware there is still no automated way to deploy one of these when your developers connect to a TFS server.

I try not discourage my developers from checking in so I keep the policies fairly light.  We currently require a comment, but encourage other practices just by word of mouth.  I think it would be possible to prompt the user if they want to create a code review request in 2012 on checkin, which may be a good way to introduce users to code review, however it would likely be annoying fast and probably still harder than just informing developers manually.

TFS Job Agent - .NET
Location: %ProgramFiles%\Microsoft Team Foundation Server 11.0\Application Tier\TFSJobAgent\plugins\

TFS runs a number of jobs in the background using its job agent and service (Processing the cube, email subscriptions, AD sync).  You can either write your own custom tasks by implementing the  ITeamFoundationJobExtension interface, or fire off existing tasks when you feel like it.

Custom Jobs are a good choice when you need to do a lot of work in the background, the problem I mentioned earlier when TFS hanging when trying to update 100+ workitems due to the email plugin could be solved by writing an event handler that queues a custom job.
Update: I created a sample server side job plugin that automatically creates workitem queries for new areas

Warehouse/Cube Customization - .NET
Location: %ProgramFiles%\Microsoft Team Foundation Server 11.0\Application Tier\TFSJobAgent\plugins\

TFS builds its own data warehouse from scratch, this is likely because work item fields can be configured to push information through to the warehouse/cube as either a dimension or measure.  TFS supports the interface IWarehouseAdapter which allows the various parts of TFS to add fields to the warehouse as it is getting built.  Developers can also take advantage of this interface and write their own custom data warehouse adapter.

The above example is regarding TFS 2008 which I have heard does not work in 2010, unfortunately there seems to be no real documentation from Microsoft and very few people trying to build a custom adapter.  I would suggest that maybe a tool like ILSpy and looking at the built in adapters would be the way to go.

In 2010 it looks like the warehouse processing was merged into the job service, and any IWarehouseAdapter have been converted to extend the abstract class WarehouseAdapter.  There's also a notion of WebhouseJobExtension which are the ITeamFoundationJobExtension jobs that presumably kick off the schema and data updates int the adapters.

Microsoft.TeamFoundation.WorkItemTracking.Adapter.WorkItemTrackingWarehouseAdapter looks to be a good example of a warehouse adapter.

Deployment Services - REST, .NET
TFS 2012 update 1 adds in a new build definition and services that allow you to deploy websites to azure. The code behind this actually looks a lot more powerful and gives me the impression that they're going to open it up for others to start publishing not only deployment methods but other services that can interact with TFS using OAuth.

Using WCF Storm I discovered the following webservices.

http://tfs:8080/tfs/{TeamProjectCollection}/services/v1.0/ConnectedServicesService.asmx

• CreateConnectedService
• QueryConnectedServices
• GetConnectedService

http://tfs:8080/tfs/{TeamProjectCollection}/Build/v4.0/BuildDeploymentService.asmx

• CreateDeploymentEnvironment
• GetDeploymentEnvironments

http://tfs:8080/tfs/{TeamProjectCollection}/services/v1.0/StrongBoxService.asmx

• GetDrawerContents
• GetString

Once you register a deployment service (eg Azure, or a custom built one) you can add environments to it with a custom set of properties (any keyvalue pair). You can then query all enviroments for a given service and pull properties out of its strong box drawer.

I haven't looked at the workflow activities at all to see if they would support calling a custom service.


Custom WorkItem Controls - .NET, Javascript
It's fairly well known that you can customize the fields for any workitem, however you can also write your own custom controls to display on workitems.

With the totally new Web Acess in TFS 2012 there's a new way to write custom controls.


There's a codeplex project that contains many work item tracking custom controls along with their source, and look like they may be updated for TFS 2012 (the visual studio integration side anyway).

Web Access Extensibility - Javascript
The Web access allows you to upload new javascript based plugins, apparently the team were not happy with the state of this at the time of release and are currently reworking them before they make many details public.

It also appears that the different sections of Web Access are built using the above plugins, two controllers (one for the view, one for json) and an AreaRegistration to register them. It may be possible to create new sections just by creating a new AreaRegistration and dropping it in the bin directory.


Update - below are some extensibility points I was either unaware of at the time of writing, or didn't think worth mentioning (visual studio extensions) as everyone already knows about them.

Test Controller Plugins

I've never seen anyone else use one of these, but they appear to exist and work well. I've written one to help filter available test agents by their machine tags for the current test config.

Diagnostic Data Adapters
Custom Adapter – Part 1


MTM Extensions
Example Extension TestScribe for MTM

Visual Studio Extensions
Developing Visual Studio Extensions

Summary
There's lots of extensibility points in TFS but a number of them seem half finished, or not documented.  It could be that most of them are only designed for internal Microsoft use, but it wouldn't take too much to make them very useful for other developers.

TFS Deployer - Marking builds as deployed in tfs 2012

TFS2012 allows you to mark builds as deployed, this is meant to work hand in hand with the azure deployment but I figured it should work with whatever TfsDeployer does too.

TFS always seems to throw an error afterwards which i haven't figured out quite yet, so i'm just wrapping it in a try/catch. Even if it didn't throw this exception, I think it will throw another one if you ever try mark the same build as deployed

This adds a new row in the "deployed" tab under builds in TFS 2012 web access (looks like visual studio doesn't support this quite yet).

It has a redeploy button which just triggers a new build, so it won't actually set the correct build quality to be auto deployed by tfs deployer.

Added a discussion to TFS Deployers discussion tab http://tfsdeployer.codeplex.com/discussions/397983

TFS 2012 - set user images to be same as active directory

I posted a question on Stackoverflow hoping that someone has an idea of how to use the user image api in tfs 2012.

http://stackoverflow.com/questions/12633732/api-to-update-users-image-identity-extended-properties-not-saving

Update
Figured it out myself

TFS 2012 - Groups Reviewed

When the company I work at first started using TFS 2005 we decided to have only one Team Project due to the limit of ~250 Team Projects. Today our root source folder for our main Team Project has a few hundred folders (clients) in it, so do our areas and iteration trees.  While it is a bit annoying to maintain, I am certainly thankful of that decision when it comes to upgrading versions of TFS.

So understandably when I heard about the new Team concept in TFS 2012 I got rather excited.  After playing with Teams for a little while in 2012 they are nice, but don't help with management as much as I was hoping for.

Things I think should be tweaked

• Work Items - "Assigned To Me" should filter by current team
• When creating an alert in a team, I should be able to filter work items by my team iteration or areas rather than selecting it manually
• Alerts that can be reused between teams (Email all team members when a work item is created in an area their team is associated to)
• Iteration/Area tree views in the admin section should default to filtered to the default area/iteration and what is below it
• Source view should select the first team favourite
• Builds should filter queued/completed by team favourites
• Easier way to assign security to areas based on groups (eg if a group is assocated to an area then allow the group a particular access level)
• Allow personal favourites per team
• At least some level of integration with reports
• Way to change the path of the area that is automatically created for a team

Obviously the extended view is required to setup favourites in the first place, but this can be a toggle on all views.  Nothing major, i just feel that it might be nice to restrict my default view a bit more when I'm meant to be looking at a team.

Maybe i'll look into the plugin framework a bit more and see what I can do.

TFS 2012 - Web Access Plugins

Update 1

Found a blog series on how to write custom work item controls (plugins) in javascript (which is quite old, oops).  Custom controls don't seem all that exciting to me, but there may be enough information there to develop something else with the information.

1. http://blogs.msdn.com/b/serkani/archive/2012/06/22/work-item-custom-control-development-in-tf-web-access-2012-development.aspx
2. http://blogs.msdn.com/b/serkani/archive/2012/06/22/work-item-custom-control-development-in-tf-web-access-2012-deployment.aspx
3. http://blogs.msdn.com/b/serkani/archive/2012/07/12/adding-content-files-to-work-item-custom-control-extension-package.aspx

Update 2

Tiago Pascoal has released a Plugin that enhances the Task Board
http://pascoal.net/2012/09/team-foundation-task-board-enhancer-version-0-3-released

Update 3

Wrote my own very basic plugin to make the logo a clickable link
http://bzbetty.blogspot.co.nz/2012/11/very-simple-tfs-2012-web-access-plugin.html

Introduction
While using the new WebAccess for TFS 2012 I noticed an interesting tab in the administration section named extensions, but it appears Microsoft haven't made the API public yet (1) so I thought I might have a quick dig to see what I could figure out.  Because the information has not been made public by Microsoft there's a good chance it will all change in upcoming TFS builds.

The Plugins
Plugins appear to be uploaded as a zip file containing an xml manifest file which defines its name, version, vendor and a few other properties. The author can also specify which other modules it loads after suggesting it has some form of dependency management.

All files included in the archive appear to be uploaded to a filestore somewhere, but the only required file that I could see was the manifest.xml.  Unfortunately whenever I tried making a extension I got the error message "TF400900: Invalid zip archive file : manifest.zip" upon upload, no resolution as of yet.

Registered plugins are added to a common xml file and serialized to a database or other persistent storage. Neither my TFS_Configuration nor Team Collection databases appeared to contain this file, more than likely it is created when the first plugin is uploaded.

There's a couple of properties and an ActionFilter that make it look like javascript plays a big part of these plugins. The website itself does reference a file named builtin-plugins.js, which then references a bunch of other javascript files by name.

The builtin javascript plugins align very nicely with the areas defined earlier, although not a 1 to 1 mapping.

I curious to see a lack of NuGet, it would have made a great plugin repository as seen in a number of projects now and would have provided a few major benefits such as better dependency management and a distribution channel. Although given Microsoft's recent announcements of other App Stores maybe they plan to take the same route with TFS.

Another point of interest is that TFSPreview does not appear to have this tab, which would be a curious ommision if they are just javascript scripts.

Portable Areas

TFS 2012 also seems to make heavy use of MVC areas, each area defined seems to be separated into its own assembly file containing the relevant controllers, viewmodels and other relevant code.  Some form of portable area code must have been developed, although i'm guessing nothing was taken from either MvcContrib nor Orchard.

Interestingly they don't appear to take advantage of the new WebAPI, nor do they use razor for their view engine. But they do typically contain a basic controller for serving views with little logic and an api controller to query via json.


While i haven't had the time to dig into this whole thing properly it looks like it should be possible to create complete new tabs/areas using MVC in the new webaccess, or to augment the existing ones using javascript plugins.  Considering how much of the UI appears to be created on the clientside this could be a very powerful little framework.  I hope Microsoft start releasing information about it soon, if not I'm sure a blogger will.

References
(1) http://social.msdn.microsoft.com/Forums/en-US/TFSvnext/thread/7696732e-767a-43c7-81d2-c318aeff41ed/