Software Development at Mocana — Version Control
Note: This post is part 1 of a two-part series. The second post in this series is here.
In this post, I’d like to give an overview of some of the tools we use for software development at Mocana, specifically around version control.
Mocana has a large Subversion-based source tree. Newer projects are using git, but it’s still somewhat of a mix. We have a large codebase, dating back to our founding in 2002 (though the first Subversion commit is an import from somewhat later). Managing all the moving pieces can be a challenge, and I wanted to share some of our experiences.
Migration from Subversion: A Phased Approach
The git-svn tool makes this fairly easy. On the server hosting the souce control, assuming it hosts both the git and Subversion repositories, you can first create a git-svn clone of the Subversion repository you want to migrate.
A git-svn repository is interesting in that you can actually use it to commit back to the original Subversion repository, but only if you do so from the same git repository that you originally cloned the Subersion source from. (you cannot commit to a Subversion server using a ‘git clone’ of a git-svn cloned repository) So this will be a one-way migration; in some ways, just what we want!
We created a special
mirror user on the source control server, which has access to check out from Subversion, and push to git. Then we created a git-svn directory inside its home directory, checked out each Subversion project we wanted to migrate, (using
git-svn clone) and verified that the repository contained what we expected it to contain.
Imagine the following directory structure:
Add a shell function as follows:
git svn fetch --all
git svn rebase
# Note: not using --mirror because it deletes
# refs in the remote repository.
# This'push' makes each Subversion branch
# a full-fledged git branch!
git push /git/$1 +refs/remotes/*:refs/heads/*
The way you run this shell function is to change to the ~/git-svn directory and run it against each Subversion checkout you want mirrored to git. (note: an [empty] repository under /git that corresponds to the name of each
git-svn clone must already be created.)
This can be run periodically (for example, in a crontab) to continually push any changes in the Subversion repository to git.
The beauty of this approach is that it allows teams who are working on git-based projects to push to their own branches, while retaining the ability to merge in changes from teams who are still using Subversion. (of course, if the teams using git create something that the teams using Subversion want to use, that’s a motivation for the Subversion-based teams to make the jump to git!)
Organizing git repositories
Complex software projects today often pull together components from many different source control repositories, all of which you may want to modify and/or build from source.
At Mocana, we had the same problem, and we turned to our experience working on the Android OS to help. When working on the source code for Android, you use a tool called repo to work with the source tree. The repo tool initializes a source code sandbox from a “manifestt.xml” file. (which itself exists inside a git repository)
We used the repo tool to gather together all the software repositories we needed in order to integrate the product. Here’s an example
manifest.xml file with just a few projects:
<?xml version="1.0" encoding="UTF-8"?>
This tells repo four important things. First, the source code should be fetched from
ssh://git.company.com/git/. (Note: you can also use a relative URL here. The Android project uses .. which would make it easier to mirror the repository to other servers. However, we only needed a single central server to be defined.)
Second, the default revision (branch to check out) is myproject-dev. You can also override this on a per-project basis, but usually it’s best to keep it simple. By using a branch specific to your project, you can increase the predictability of your project, assuming changes are going into that repository that you do not have direct control over. Smaller projects may want to simply use
master to get the latest from each repository, but using a consistent branch name specific to your project also makes it easier to branch the project later.
Third, three git repositories should be checked out:
Fourth, the layout of the source tree after running “repo init” and “repo sync” will look like this:
Simply save your manifest XML in a file called manifest.xml and commit it to a git repository of its own. Run repo init -u to fetch the manifest, and then run repo sync to get a sandbox.
The repo tool makes it easy to organize a set of git repositories in a flexible, predictable way, without directly specifying dependencies on specific versions of the other git repositories in your project. If you want to do a build, you can always run
repo manifest -r to output a version of the manifest with “hard-coded” repository revisions. You could subsequently replace your local manifest.xml with that file and do a
repo sync to restore your sandbox to a previous build.
Another factor in choosing to use
repo was the potential to use Gerrit for code review, an excellent code review tool that integrates with
Jenkins has a wide variety of plugins that can assist with other aspects of the software development process, as well.
Mike Pontillo is a Principal Engineer at Mocana.