Keeping all Visual Studio projects in sync with a library

Consider why your Applications need to work with different versions of the libraries. If you design your libraries properly, and make sure you don't break anything while upgrading them (use continuous integration and unit tests, as well as testing all dependent projects) then the best (simplest, cleanest) approach in most cases is simply to have all Applications run off the same version of the Libraries. I would go so far as to say that if your Applications can't run off the same Library version, that tells you that your Libraries are not designed/implemented/maintained properly, and you should reconsider your approach.

Another approach is to use a branch for each Application, so each has its own independent copy of the Libraries (use a relative reference between the projects and the libraries and then you can relocate the code in the branches). This allows the Applications to build off different versions of the libraries, but does have all the disadvantages of branching, though - two copies of the libraries so you have to be careful not to edit in the wrong branch; the potential for nasty merges, etc.

I'd recommend moving the Library projects into the Library. Your diagram shows the Application solutions building library code and pushing it up to the library, then sucking the built library files back down into the projects - a bi-directional copy, eek! Just place the library projects in the library and build the library, then reference that library from your projects - a unidirecitonal copy. You may think this would mean you have to build two solutions (Library and then Application), but you can actually add/remove library projects in your application Solution whenever you wish, so this layout does not necessitate a two-stage build process. But the logical distinction between the library and application is much cleaner.

The way we solve a similar problem is just what you describe in update 2, but where you suggest 'backup copies', we use clones and they are an integral part of our workflow.

My solution.

We use a combination of clones and named branches to allow us to control when the libraries for a particular application (solution in your example) are updated.

What we do is have a core repository for each of our libraries which contains the mainline branch of that library. When we create a new application we clone all of the relevant libraries (we use one mercurial repo per library, where a library may contain the library and it's test application) into directories beside the new application and reference the libraries in that application.

Any commits to the libraries for this application are committed to a branch named according to the application. When an application is complete, we review the changes to the library and then merge appropriate changes back into the mainline, creating a new mainline. This also has the advantage that in the future, you can easily identify which application caused a particular change to a library by looking at which branch that change was committed to.

Whenever we touch an application, we can chose whether to continue using the old libraries, or update to a more recent mainline version (or even another projects development version) of the library. Library changes aren't pushed out to every application which uses them (some might be legacy applications which haven't been touched for years and are working fine with the library they were suplied with), they are pulled in by applications which need the new facilities in a given version of the library.

Why do I do it this way?

The problem I'm trying to avoid here is one I constantly ran into at a previous company.

I would get my application working with a given library. Someone else would change the library for their application so that it did what they needed and the next time I went to edit my project, it wouldn't compile. I would then have to spend time fixing up my project to beghave in the way the library now wanted or spend time to revert inappropriate changes made to the library (which would then cascade back to the other developer).

With local project clones, every application gets to chose when it updates to a more recent library version and from a chance management perspective, we get to review changes to libraries due to application requirements before they get merged back into the mainline.

This is a workflow which would never have been possible with a traditional VCS like SourceSafe, but works beautifully with a DVCS like Hg.

Specific solutions to those elusive goals.

To try and answer your elusive goals

.3. Each repository contains all source code, including library projects

By cloning libraries into the directories close to your application, you get a set of repositories which can be managed together and possibly eventually will convert nicely into sub-repos of an 'application' super-repo.

.4. Everything is as automated as possible to minimize the risk of mistakes

I do this with a bunch of batch files which I store in my application directory.

For instance a given applications "_projects.bat" might contain:

@echo off
@shift
set APPDIR=%CD%
cd ..\Library1
%0 %1 %2 %3 %4 %5 %6 %7 %8 %9

cd ..\Library2
%0 %1 %2 %3 %4 %5 %6 %7 %8 %9

cd ..\Library3
%0 %1 %2 %3 %4 %5 %6 %7 %8 %9

cd %APPDIR%
%0 %1 %2 %3 %4 %5 %6 %7 %8 %9

pause

Then I have batch files to do useful synchronisation things like check what files would be pulled with "__incoming.bat:

@echo off
call _projects hg incoming

and check what would be pushed with "__outgoing.bat":

@echo off
call _projects hg outgoing

Or actually do those operations with "__push.bat":

@echo off
call _projects hg push

and "__pull.bat":

@echo off
call _projects hg pull

while "__update.bat" updates everything to the latest version (within the same branch, you still have to explicitly update outside the current branch):

@echo off
call _projects hg update

One of my favourites is "__ids.bat", which shows what changesets (and branched) are currently in use and critically, whether they have uncomitted changes outstanding:

@echo off
call _projects hg id

Whenever I come to start work on an application update, I check to see if there are updated libraries, pull down everything which is appropriate, update to the versions I want and then continue development.

Note the underscores in the filenames are just there to push them to the top of the directory lists, to make them easier to find. *8')

Future options.

Eventually I would like to fully automate this workflow by using sub-repos (so the state of the whole application and all of it's libraries are recorded in one application changeset), but while it is getting pretty stable in Mercurial, TortoiseHG integration doesn't seem to be a priority. I couldn't get the THG commit tool to do anything useful the last time I tried it (around 0.8.3). I really should retest with 0.9.3 but I can't remember seeing any notes about sub-repos in the changelogs, so it isn't worth it unless people tell me THG sub-repo support has been silently added.

As it is, the batch files work quite well. The next thing on my todo list is a "__clone.bat" which should make cloning the application and all of it's libraries a whole lot easier, but I haven't had the time to get that working.

If anyone fancies having a go though, I'd love to see a reply in this thread. *8')

Take care, hope this helps,

Mark..........