GitMigration
Do you want just want to work on DebConf and skip all the gory details? Then click here to skip down to the appropriate section…
Over the last couple of years, more and more people wanted to migrate the DebConf repositories away from Subversion, and it was unanimously agreed to use Git instead.
The following document aims to make the transition easier. If you are already familiar with Git and you just want to know the repository URLs, skip to the bottom of the page, section #URLs_and_repos.
[edit] Getting started
[edit] Requirements
Please apt-get install git.
You may also want to install gitk, gitg or tig, which can be used to graphically inspect the history of a repository. This may be useful especially when working with Git for the first time.
[edit] Quick overview of the differences between Git and Subversion
Git and Subversion are both revision control systems and fundamentally do the same task (keep track of changes to files over time). However, tthere are three main differences between the two tools:
- When you issued the command svn commit, Subversion would record all the changes you made into one "revision". Git allows fine-grained control for what should become part of the commit transaction. If you want to commit all changes made to a file, type git add <filename>; if you want to select particular changes within a file, use git add -p, optionally with a filename.
- Subversion automatically pushes a commit to the server when you run svn commit, Git does not. With Git, commits need to be explicitly pushed, which you do by running git push. Git can be used offline, while Subversion cannot.
- As a consequence of Git's offline usability, it is possible that someone else pushes changes before you pushed your commits. In this case, you need to first pull the new commits, using git pull.
[edit] Getting you started quickly
The first thing to do is to identify yourself to Git:
git config --global user.name "Jane Q. Git" git config --global user.email "jane.git@example.org"
Instead of specifying --global, you can leave it out and the configuration will be stored in the current repository only.
If you prefer not to dive right into the cold water, you can create yourself two aliases to mimic the way Subversion worked:
git config --global branch.autosetuprebase true
git config --global alias.scommit '!scommit() { git commit -a "$@" && git push; }; scommit'
git config --global alias.update pull
git config --global alias.up pull
With those in place, you can use git scommit just like you used svn commit: it will create a commit encompassing all changes and push it to the remote repository right away. You can also use git update just like svn update and it will fetch new revisions from the remote. The last setting ensures that local changes are added on top of any changes fetched from the remote.
Most of the other commands that are available for svn are identical in Git: add, rm, diff, log, status.
[edit] Getting help
Please feel free to ask any questions about this migration or about how to use Git on the #debconf-team channel on irc.oftc.net.
If you prefer to use the mailing list, send an e-mail to debconf-team@lists.debconf.org and make sure to include "[Git]" in the subject to help people identify the message.
[edit] More information
Here are a few links that may also be of interest:
- Official Git crash course for Subversion users
- Git cheat sheet
- Git advocacy for the migration, including a list of traps to watch out for.
[edit] URLs and repos
The URLs and description of the individual repos follows, and there's a table helping you where to find stuff below, but first, there are some important things:
[edit] Important information
Important news will be published here, so make sure to revisit this page often, especially in case of problems
If you do experience a problem, please send a message to debconf-team@lists.debconf.org with "[git migration]" in subject line.
[edit] Changes made since the migration
The following is a list of changes made since the announcement of the completion of the migration (2014-04-02-1403):
| Timestamp | Repository | Re-clone? | Description |
|---|---|---|---|
| 2014-04-02-1616 | debconf-data/dc11 | Yes | History rewritten to remove huge print-quality files (sponsorship brochure, now on media.dc.o). |
| 2014-04-02-1632 | debconf-data/dc13 | Yes | History rewritten to remove venue evaluation photos (now on gallery.dc.o). |
| 2014-04-02-1718 | debconf-data/dc13 | Yes | History rewritten to remove more venue evaluation photos (now on gallery.dc.o). |
| 2014-04-02-2204 | debconf-data/misc | Yes | History rewritten to split out individual repos:
|
Instead of a re-clone, you can also run (this is for advanced users):
git tag -d debconf2git git fetch git checkout master git reset --hard origin/master git gc --prune=all
[edit] DebConf team repository
The debconf-team repository contains non-public information used by the team(s) to coordinate (except for accounting, see below). The same repository is used across different conferences to provide easier access to past information for new teams (this is an encouragement…).
You need to be a member of the debconf-team project on alioth.debian.org to be able to use this repository.
The following command is used to clone the respository:
git clone ssh://your-alioth-username@git.debian.org/git/debconf-team/debconf-team cd debconf-team
Please try to refrain from committing huge files to the repository. Scans of invoices and receipts should go into a conference's accounting-data repository, and sensitive information that you need to keep around for the conference, and possibly for a while thereafter should live in the conference's private repository, see below:
For a list of available (sub-)repositories, please have a look in the directory /git/debconf-team on git.debian.org.
[edit] DebConfX accounting data repositories
Scans of invoices and receipts should be put into a separate, per-conference repository, dcX-accounting-data, which you can then access like so:
git clone ssh://your-alioth-username@git.debian.org/git/debconf-team/dc13-accounting-data cd dc13-accounting-data
The main motivation here is to keep the size of the main team repository from exploding, and accounting data seems to be the worst offender here, based on past experience.
[edit] DebConfX private repositories
Personal information and other sensitive data, such as anything relating to visa applications of specific people, whould be put into dcX-private, and might be archived/purged at some point after the conference. This is to avoid infinite access to such data by every current and future team member.
git clone ssh://your-alioth-username@git.debian.org/git/debconf-team/dc14-private cd dc14-private
[edit] DebConf data repositories
The debconf-data repositories contain public information for each conference. Contrary to the Subversion repository, each conference has its own repository. For a list of available (sub-)repositories, please visit GitWeb, the simple HTTP interface, or have a look in the directory /git/debconf-data on git.debian.org.
If you are a member of the debconf-data project on alioth.debian.org, you can and should clone each repository using SSH:
git clone ssh://your-alioth-username@git.debian.org/git/debconf-data/dc14 cd dc14
If you just require read-only access, use
git clone git://anonscm.debian.org/git/debconf-data/dc14 cd dc14
instead.
[edit] Where do I find …?
| svn.debian.org/svn/… | git.debian.org/git/… | Notes |
|---|---|---|
| debconf-team | debconf-team | The debconf-team was mostly migrated as-is, with the exceptions of accounting and sensitive, personal data. Some of those have simply been removed, or split into their own repos: |
debconf-team/…
|
dcX-accounting-data/… | Accounting data (mostly large scans of receipts and invoices, etc.) have been split out into repositories for each conference, e.g. dc6-accounting-data. |
debconf-team/…
|
dcX-private/… | Personal data (such as used for visa applications) have been split out into repositories for each conference, e.g. dc14-private. |
debconf-data/…
|
debconf-data/… | Subdirectories have been split into their own repositories. |
| debconf-data/10year | debconf-data/10years | Dto., but renamed to match corresponding hostname. |
debconf-data/…
|
debconf-data/… | Subdirectories for each conference have also been split into their own repositories |
[edit] Help! Too many repositories!
If you don't like having 46 different repositories for your DebConf work and you wallow in memories of the days when there were just two, one for private and one for public data, here's a solution for you (which lets you have the best of all worlds: speed, size, Git, and simplicity).
[edit] Installing mr and basic setup
The tool to use is myrepos (mr), and the repositories have been enabled for its use:
% apt-get install myrepos
Before starting to use it, however, it helps to make sure that your SSH client knows how to log in to git.debian.org (e.g. which username to use, which key to present, etc.). So make sure there's a stanza like
Host git.debian.org User your_alioth_username #IdentityFile ~/.ssh/other_identity_if_not_id_rsa
in your ~/.ssh/config file.
Now we can start using mr:
Clone the debconf-team repository, as you normally would.
% git clone ssh://git.debian.org/git/debconf-team/debconf-team […] % cd debconf-team
Tell mr that you trust the contents of the .mrconfig file contained therein:
% echo \~${PWD#$HOME}/.mrconfig | tee -a ~/.mrtrust
~/debian/debconf/team/.mrconfig
Clone the debconf-data meta-repository to a place of your choosing. I call it a meta-repository, because it is empty but only serves as a parent to all the sub-repositories:
% git clone ssh://git.debian.org/git/debconf-data/debconf-data […]
If you're like me, you want debconf-data to live inside the debconf-team repository, which you could tell mr like this (assuming you want it to live in the subdirectory pub-data, I chose this name instead of debconf-data to draw attention to the public nature of data contained therein):
% echo -e "[DEFAULT]\ninclude = cat .mrconfig.pub-data\n" >> .mrconfig.local % mr up […] mr checkout: ~/debian/debconf/debconf-team/pub-data Cloning into '~/debian/debconf/debconf-team/pub-data'... […] mr update: finished (1 ok; 11 skipped)
(The skip is needed due to #743576.)
Tell mr that you also trust the contents of the .mrconfig file contained in the repository just checked out:
% cd debconf-data # or pub-data
% echo \~${PWD#$HOME}/.mrconfig | tee -a ~/.mrtrust
~/debian/debconf/team/pub-data/.mrconfig
I will assume you chose pub-data too in the following…
[edit] Interlude: directory structure
Providing automation with mr means that there are some choices that have been made which will affect you. In this case, there's a pre-defined directory structure, which can be described as follows:
- debconf-team is the base directory
- dcX-private and dcX-accounting-data repositories are placed inside the existing dcX directories, i.e. dcX/private and dcX/accounting-data.
- All the debconf-data sub-repositories become subdirectories of wherever you put the meta-repository, e.g. pub-data.
Here's what a complete tree would look like, at least at time of writing (abridged to only show relevant directories, and with debconf-data residing in debconf-team/pub-data):
. ├── dc1 ├── dc10 │ ├── accounting-data ├── dc11 │ ├── accounting-data ├── dc12 │ ├── accounting-data ├── dc13 │ ├── accounting-data ├── dc14 │ ├── accounting-data │ ├── private ├── dc15 ├── dc2 ├── dc3 ├── dc4 ├── dc5 ├── dc6 │ ├── accounting-data ├── dc7 │ ├── accounting-data ├── dc8 │ ├── accounting-data ├── dc9 │ ├── accounting-data ├── dc-admin ├── debconf2git ├── kgb-client ├── pub-data │ ├── 10years │ ├── bcn2014.mini │ ├── blog │ ├── db │ ├── dc0 │ ├── dc1 │ ├── dc10 │ ├── dc11 │ ├── dc12 │ ├── dc13 │ ├── dc14 │ ├── dc15 │ ├── dc2 │ ├── dc3 │ ├── dc4 │ ├── dc5 │ ├── dc6 │ ├── dc7 │ ├── dc8 │ ├── dc9 │ ├── fr2010.mini │ ├── fr2012.mini │ ├── in2010.mini │ ├── in2011.mini │ ├── in2013.mini │ ├── media.debconf.org │ ├── miniconf10 │ ├── misc │ ├── ni2013.mini │ ├── penta │ ├── reports │ └── www └── video-hw
However, you need not check out all of these repositories to get work done! Keep reading…
[edit] Adding sub-repositories for mr to manage
What you have now is a complete debconf-team clone (without private and accounting-data) in the current directory, as well as ./pub-data prepared to hold debconf-data contents. If you run mr up now, it will tell you about "skipped" repositories:
% mr up […] mr update: finished (2 ok; 43 skipped)
The reason for this high number of skips is that mr has been configured to only manage repositories that have been cloned beforehand. So say, for instance, you wanted to have access to debconf-team/dc14-private and debconf-data/blog, you would just do:
% mr -fd dc14/private checkout mr checkout: ~/debian/debconf/debconf-team/dc14/private Cloning into 'private'... […] mr checkout: finished (1 ok) % mr -fd pub-data/blog checkout mr checkout: ~/debian/debconf/debconf-team/pub-data/blog Cloning into 'blog'... […] mr checkout: finished (1 ok)
Subsequent invocations of mr will now recurse over the two base repositories (debconf-team and pub-data), as well as the two additional repositories checked out:
% mr up […] mr update: finished (4 ok; 41 skipped)
[edit] I want it all
If you don't want to select individual sub-repositories to clone and manage, but just let mr handle them all, put the following snippet into .mrconfig.local:
[DEFAULT]
lib = lazy() { false; }
Now running mr up will clone any repositories that aren't already present.
[edit] If you're already using mr
Those of you already using mr could just add the debconf-team repository to an existing .mrconfig file and enable chaining, e.g.
[debconf/team] checkout = git clone ssh://git.debian.org/git/debconf-team/debconf-team $MR_REPO chain = true
[edit] Next steps
And now you can use mr pretty much like you used svn before, as it does the recursion for you, e.g. the following commands all pretty much do what you expect:
- mr update
- mr status
- mr commit (actually, in our case: mr record && mr push)
- mr log and mr diff
Have a look at the mr(1) manpage for more information. You can define aliases, or new commands, hooks, run checkouts in parallel, and other things.
If you do want to make local changes to .mrconfig, please put them into .mrconfig.local (and do not commit that).
[edit] I just want to work on DC15!
You don't care about all of the above and you just want to get started? Alright! If you're okay using and learning about mr (possibly in addition to Git), read on. If you don't want to learn about mr, skip to the second-next section.
[edit] Using mr
[edit] Setup
sudo apt-get install git myrepos gitk
cd ~/path/to/my/debconf/workspace
git clone ssh://git.debian.org/git/debconf-team/debconf-team
cd debconf-team
echo -e '[DEFAULT]\ninclude = cat .mrconfig.pub-data\n' > .mrconfig.local
echo ${PWD}/.mrconfig | sed -e "s,$HOME,~," >> ~/.mrtrust
echo ${PWD}/pub-data/.mrconfig | sed -e "s,$HOME,~," >> ~/.mrtrust
mr up
[edit] Picking repos
All DC15 team-related stuff is already in ./dc15.
If you want to work on accounting or visa (or other personal stufF), you add the corresponding sub-repos like this:
mr -fd dc15/accounting-data co mr -fd dc15/private co
You will also need the public data for DC15:
mr -fd pub-data/dc15 co
And depending on whether you will be working on the blog, media data, reports, do any or all of the following:
mr -fd pub-data/blog co mr -fd pub-data/media.debconf.org co mr -fd pub-data/reports co
[edit] The result
This is what your tree will now look like (abrigded):
. ├── dc10 ├── dc11 │ […] ├── dc15 │ ├── accounting-data # debconf-team/dc15-accounting-data │ ├── contracts │ ├── logos │ ├── private # debconf-team/dc15-private │ └── visa ├── dc-admin ├── pub-data │ ├── blog # debconf-data/blog │ ├── dc15 # debconf-data/dc15 │ ├── media.debconf.org # debconf-data/media.debconf.org │ └── reports # debconf-data/reports └── video-hw
[edit] I do not want to learn about mr
Fine!
sudo apt-get install git gitk cd ~/path/to/my/debconf/workspace git clone ssh://git.debian.org/git/debconf-team/debconf-team git clone ssh://git.debian.org/git/debconf-team/dc15-private git clone ssh://git.debian.org/git/debconf-team/dc15-accounting-data git clone ssh://git.debian.org/git/debconf-data/dc15 dc15-data git clone ssh://git.debian.org/git/debconf-data/blog git clone ssh://git.debian.org/git/debconf-data/reports git clone ssh://git.debian.org/git/debconf-team/logos git clone ssh://git.debian.org/git/debconf-data/media.debconf.org
And now your tree will look like this:
. ├── blog ├── dc15-data ├── dc15-private ├── dc15-accounting-data ├── debconf-team ├── logos ├── media.debconf.org └── reports
[edit] Alright, so how do I…?
[edit] … tell Git about a new file I want to add to the repo?
git add newfile # git commit # git push
[edit] … remove a file
git rm file-to-remove # git commit # git push
[edit] … obtain information on the status of things
git status
The output of git status also contains some helpful information about why certain files are listed where they are, along with relevant commands. It helps to run a recent version of Git!
[edit] … commit my changes? Git says "nothing to commit"
Git is slightly different from Subversion in that you have to "stage" changes. At first, this might seem like a nuisance, but over time, you'll come to appreciate this!
If you just want to commit all changes made (like Subversion would), you can do either of the following two:
git add -u && git commit git commit -a
If you want to select only specific files to become part of a commit:
git add file1 file2 git commit
If you want to select specific lines (!) out of all the changes you have made (you can even edit the commits and effectively select words!):
git add -p [file1] git commit
If you have staged changes and you have changed your mind, you can unstage them like so:
git reset HEAD -- file
[edit]
"I committed changes, but noone else in the team can see them!"
Commiting is a local action with Git (meaning you can also do it offline!).
[edit] If you have an Alioth account with access to the repository…
To publish your commits, you can just push them to the repository:
git push
Quick note, in case you worry about pushing changes: don't; everything can be undone. However, if you are unsure and would prefer to send your changes as proposals via e-mail, please skip to the next section.
Git will report success with a bunch of lines ending e.g.
To ssh://git.debian.org/git/debconf-team/debconf-team 067866e..cf54341 master -> master
It may also fail with the following:
To ssh://git.debian.org/git/debconf-team/debconf-team ! [rejected] master -> master (non-fast-forward) […]
This simply means that someone else has pushed changes, which you need to integrate locally first:
git pull --rebase # possibly resolve conflicts at this stage git push
If you have branch.autosetuprebase set (see #Getting_you_started_quickly), then you can leave off --rebase. Rebasing on pull means that the history will be linearised, rather than different branches merged, you can read up on what that means e.g. here.
[edit] If you do not have an Alioth account with access to the repository…
Your local clone is a full-featured Git repository, meaning you can make regular commits. If you cannot push them directly to the repository, you can still share your changes, by e-mail. This is also a good way to share commits you want others to look at, or that should be discussed.
Let's assume you made a number of commits you would like to share. You can display those as follows:
git log --oneline origin/master..
If the output is not what you expect and you do not want to send all commits, then adjust the specification. For instance, to look at just the last commit, or the last two commits, you'd say:
git log --oneline HEAD^.. git log --oneline HEAD^^..
The ^ (caret) character denots the parent of a commit, and HEAD is the current commit. The .. afterwards just tell Git "everything since…". This is all explained with high complexity in the git-rev-parse(1) manpage.
Once you figure out which selector describes the set of commits to share, you just fire off another command:
git send-email --to debconf-team@lists.debconf.org origin/master
and follow the instructions. If you want to write an introductory message to your patch or patch series, throw a --compose in there:
git send-email --to debconf-team@lists.debconf.org --compose origin/master
If you do this often, you could configure the default recipient in your Git clone's configuration:
git config sendemail.to debconf-team@lists.debconf.org git send-email origin/master
Have a look at the git-send-email(1) manpage for more settings, if you care.
You might also want to know that git send-email employes git format-patch to create the patches, so if you want to export your commits to files, have a look at that tool, as well.
[edit] … discard my changes
To completely discard your changes to a file, run
git checkout -- file
You can also just discard changes on a line-by-line basis:
git checkout -p [-- file]
[edit] … resolve conflicts?
After git pull, you may encounter some conflicts which you need to resolve. Unfortunately, the process is different depending on whether you rebased or not. While team policy is to rebase locally, we'll cover both cases for completeness:
[edit] Without rebasing
You should really be using git pull --rebase. To set that as default, run
git config branch.master.rebase true git config branch.autosetuprebase true # to set to true for new branches
Confllict resolution is required when git pull says:
CONFLICT (content): Merge conflict in a_file Automatic merge failed; fix conflicts and then commit the result.
You can inspect this further like so:
% git status On branch master Your branch and 'origin/master' have diverged, and have 1 and 1 different commit each, respectively. (use "git pull" to merge the remote branch into yours) You have unmerged paths. (fix conflicts and run "git commit") Unmerged paths: (use "git add <file>..." to mark resolution) both modified: a_file […]
To resolve, edit a_file until it has the desired contents (make sure not to remove the changes someone else did!), and then commit the merge:
editor a_file git add a_file git commit git push
You may also want to experiment with git mergetool.
[edit] With rebasing
A merge conflict using rebasing looks like this:
Falling back to patching base and 3-way merge... Auto-merging a_file CONFLICT (content): Merge conflict in a_file Failed to merge in the changes. Patch failed at 0001 . The copy of the patch that failed is found in: /home/madduck/debian/debconf/team/.git/rebase-apply/patch When you have resolved this problem, run "git rebase --continue". If you prefer to skip this patch, run "git rebase --skip" instead. To check out the original branch and stop rebasing, run "git rebase --abort".
and using git status:
rebase in progress; onto 37ab086 You are currently rebasing branch 'master' on '37ab086'. (fix conflicts and then run "git rebase --continue") (use "git rebase --skip" to skip this patch) (use "git rebase --abort" to check out the original branch) Unmerged paths: (use "git reset HEAD <file>..." to unstage) (use "git add <file>..." to mark resolution) both modified: a_file […]
To resolve, you edit the file, stage changes, but you do not commit; instead, you continue the rebase:
editor a_file # or git mergetool git add a_file git rebase --continue
If instead you see a message like this, even though you used git add:
No changes - did you forget to use 'git add'?
then your changes may have been obsoleted by the changes someone else made. It's best to verify that, and then run
git rebase --skip
Finally, when the changes have been integrated, run
git push
[edit] … stash changes away temporarily?
"I have all these local changes, but I need to work on something else…?"
git stash
Now your changes have been "stashed away", and you can pull, edit, add, commit, push as you want.
When you want to resume your work, run
git stash pop
Weeeeh! \o/
[edit] Miscellaneous
[edit] Ignoring IRC notifications
Commits to the repositories will be reported to the debconf-team IRC channel. If you feel flooded at times (e.g. because you are working on DC14 and don't care about commits related to DC15), you could tell your IRC client to ignore those lines.
For instance, here is how you would tell irssi to stop bothering you with DC15-related commit notifications:
/ignore -regexp -pattern "debconf-(team |data/)dc15" -channels #debconf-team *!kgb@kgb*.bot.oftc.net PUBLIC
