Integrating Matlab with Team Foundation Server 2010

I just managed to pull this integration off, and the process definitely deserves more web presence.

First, as of 2011, Matlab (I never understood the shout-ish MATLAB spelling) consumes only MSSCCI source control connections, which a Team Explorer installation does not supply by default. To correct that, install and run the Team Foundation Server MSSCCI Provider 2010 from the VS code gallery.  Beyond the binaries it installs – which seem to act as thin adapters to the real TFS functionality – it populates some registry keys, but mainly:

1. HKLM\SOFTWARE\SourceCodeControlProvider\InstalledSCCProviders:  a string value with a path to a TFS MSSCCI facade registry key

2. HKLM\SOFTWARE\Microsoft\Team Foundation Server MSSCCI Provider: the provider facade, containing at the very least the SCCServerName and SCCServerPath strings.

Now when source control is set from within Matlab/file/preferences, you can already see the TFS option:

It seems until Matlab R2007a, there were some issues with this long name, and you had to manually shorten it. I didn’t witness any such issues so I suppose they’re long resolved.

However – the work isn’t yet done: when you try and perform any source control operation on a file (or sometimes even just edit it), you get error TF249051:

Obviously Matlab still maintains somewhere the previous connection strings (in my case, to Perforce source control). Now this took a deeper dive with Process Monitor, but the culprit was eventually found.

Matlab places the innocent looking file mw.scc, somewhere deep in your user settings folder (in my case, C:\Users\Matlab\AppData\Roaming\MathWorks\MATLAB\R2010b\).

For P4 bindings, the file contains text like:

#Mathworks source code control preferences.
#Wed Aug 10 13:57:16 IDT 2011
d./work/matlab/dailytests.SccAuxPath=P4SCC\#myP4server\:1666\#\#myP4user\#\#myP4workspace
d./work/matlab/dailytests.SccProjectName=Perforce Project

And you have to empty it manually. After you do, Matlab will fill it with valid TFS connection strings, like:

#Mathworks source code control preferences.
#Thu Nov 24 14:43:44 IST 2011
d./work/matlab/dailytests.SccAuxPath=http\://myTFSserver\:8080/tfs/defaultcollection|yadda|yadda\\yadda
d./work/matlab/dailytests.SccProjectName=$/yadda/yadda/yadda

Hope this helps someone out there.

Source Control Binding, Part 2: SAK and SOURCE_CONTROL_SETTINGS_PROVIDER

In a previous post I shared one form of binding trouble, which had to do with MSSCCPRJ.scc location on disk. A side note there reminded me of another issue -

… The choice of binding location is controlled via a separate file – more in a future post.

- and lo, that future has just arrived.

Source Control (henceforth SC) info is maintained independently for projects and solutions. This by itself is understandable, as solutions can hold their own ‘solution items’. What’s less understandable is that some SC vendors try to apply the solution’s binding as a root to its projects, thus effectively pretending the solution has exclusive ownership over them – but that is an entirely different matter.

SC info amounts to 4 strings – SccProvider, SccProjectName, SccLocalPath and SccAuxPath, whose exact meaning can vary among SC vendors. These strings can be stored as fields in project/solution files, but that can make branching hard to handle properly – so it is highly preferable to store them externally, in a special file called MSSCCPRJ.scc.

When you do use MSSCCPRJ files (as you should) you’d see that the 4 SC fields include only  the string ‘SAK’. According to Alin Constantineasily the best online source on SC in Visual Studio – SAK probably stands for ‘Sumedh A. Kanetkar’ (a clear case of MZ envy). Anyway, it is just a flag that tells the IDE to look for the real SC bindings in the nearest MSSCCPRJ.

While MSSCCPRJ is opaque to the IDE, VS uses a host of its own files to store SC info – specifically project hint files (.vspscc) and solution hint files (.vssscc). Both files contain the field SOURCE_CONTROL_SETTINGS_PROVIDER, which can hold either ‘PROJECT’ or ‘PROVIDER’ – the latter meaning exactly that:

If the setting is "PROVIDER", VisualStudio expects your source control provider to create mssccprj.scc files that will contain the location in the scc database where the local project files are stored. VS will read in that case the bindings from the mssccprj.scc files.

As it turns out, the SAK declarations in the vcxproj/sln might clash with seemingly identical declarations in the vspscc/vssscc.

As I discovered some years ago, such a clash can cause some very thorny SC binding issues. Despite all sorts of online advice saying the contrary – I had to fix the hint files manually to contain ‘SOURCE_CONTROL_SETTINGS_PROVIDER = PROVIDER’, and thus be consistent with the project/solution files.

I was admittedly very hesitant about such manual modifications (and across half our dev machines, no less). Noting that Alin seems a top notch authority on SC apparatuses (apparati?) and that he’s quoted to be a heck of a nice guy, I simply mailed him and asked.

He replied within ~2 hours.

Hi Ofek,

Yes, the 2 settings are related.

Indeed, if the “SAK” strings are written in the project this means the real bindings can and will be provided by the source control provider and the setting in the vspscc files should be "SOURCE_CONTROL_SETTINGS_PROVIDER" = "PROVIDER" I haven’t worked for source control in the last 5 years and I don’t remember exactly what all these are used for. At a quick look it seems the setting matters only for project files (not for solutions), and it seems to be used when opening just the project file from source control.

Anyway, I don’t recommend changing these settings manually – they really depend on the capabilities of your source control provider. Setting them incorrectly or not matching the reality will cause your projects to end up uncontrolled. I suggest using ChangeSourceControl dialog whenever you need to change the bindings.

Alin

[Note: Alin explicitly permitted the publication of this correspondence, verbatim].


<aside> It is nowhere near obvious that a knowledgeable guy such as Alin Constantin dedicates this much effort and time to help out, either in online articles, forum answers or email responses (on matters which he stopped working in years before!). Beyond the personal kudos to Alin, I gotta say I feel some wider kudos is in order to Microsoft for actively encouraging this culture. I personally had quite a few similar experiences with other (even senior) MS employees. I cannot say the same for other tech giants. </aside>

"The solution appears to be under source control", or: How P4 messes up MSSCCPRJ.scc Location

For a long while we faced an annoying issue with source control binding:  we’d load some solution and this would pop up:

image

We’d then fix it using File/Source control/Change source control/Bind:

image 

And all seemed well – until we loaded the solution again, only to be greeted by the same error message.  We’d fix it again, load again, ad infinitum.

Googling around didn’t shed too much light. The p4 binding mechanism isn’t well documented (although where p4 lacks,  StackOverflow steps in). Someone described what might be a similar issue, but didn’t get any answers.

MSSCCPRJ.SCC

- is a text file that holds the actual binding info: the mapping of disk locations to source-control locations. (Note: strictly speaking, the binding info can be encoded inside sln/proj files, but it’s bad practice. The choice of binding location is controlled via a separate file – more in a future post).

When I search my disk for MSSCCPRJ  I find dozens (!) of instances scattered throughout my source folders, with contents similar to:

SCC = This is a source code control file

[yadda.vcxproj]
SCC_Aux_Path= "P4SCC#serverrnd:1666##Ofek##OfekPC"
SCC_Project_Name = Perforce Project

[yaddayadda.vcxproj]
SCC_Aux_Path= "P4SCC#serverrnd:1666##Ofek##OfekPC"
SCC_Project_Name = Perforce Project

MSDN lists some specifications the file must adhere too, but leave unspecified its location and scope – that is, the number of projects it describes and their relative-location-on-disk – and here exactly lies a loose end of the p4/VS integration. p4 knowledge base half-admits it:

- Use a Single Solution Model whenever possible. The Single Solution Model uses one .sln file as a container for all of the projects defined by your application

- Use a Consistent Folder Structure for solutions and projects. Ideally, keep solution files in the top directory and all of the projects in subdirectories.

These less-then-precise advice probably mean: avoid a state where a solution includes a project that does not lie below it on disk/source-control. This seems impossible to adhere to in real-life situations (how would you recycle libraries?), and is probably the indirect cause of the MSSCCPRJ mess.

The Problem

The direct cause is a discrepancy between the disk location where p4 places MSSCCPRJ during project bind, and the disk location where it searches MSSCCPRJ during solution load.

If you bind several projects simultaneously, seems p4 persists the binding info into an MSSCCPRJ at their common ancestor on disk. E.g., if you simultaneously bind D:\code\proj1\proj1.vcxproj, D:\code\proj2\proj2.vcxproj and  D:\code\proj3\proj3.vcxproj, the MSSCCPRJ would be generated (or modified) at D:\code. However it seems that during solution load, if p4 uses MSSCCPRJ at D:\code, it would never look for other MSSCCPRJ under D:\code subfolders.  So if, for example, some day you bind the single project D:\code\proj4\proj4.vcxproj, an additional MSSCCPRJ would be created at D:\code\proj4, and never be used during solution load, no matter how many times you re-bind the project using the change source control dialog.

The Solution (well, more like a workaround)

Just bind your projects individually. i.e., instead of clicking ‘bind’ on multi-selection:

image 

Repeatedly bind a single selection:

image

and separate MSSCCPRJ files would be created at every project folder. It can be a hassle for large solutions (I should know – I applied it to ~100 project solutions), but it’s a one-time hassle. For me, that seem to have solved the issue completely.

NOTE: we’re using Perforce 2007.1 and the issue may have been resolved since – although I find that improbable, as the issue seem to persist in the p4 knowledge base. The reason we’re not upgrading is that we’re migrating to TFS instead, and I can’t say I’m too sad about that.