Splitting the Specs to a Separate Project with git Submodules?
Posted by rue, Thu Mar 13 01:47:00 UTC 2008
If we have a separate RubySpec repository which is included to Rubinius by means of git’s submodule support (mounted at spec/ruby/) there are two types of committers: those who work with RubySpec only and those who work with Rubinius only. The workflow for a RubySpec-only committer is very simple:
- Clone the main RubySpec to a local working copy.
- Make changes.
- Push changes to RubySpec mainline or make a patch if no commit bit.
That is the end for that committer. Eventually one of the Rubinius committers will pull in the changes to the Rubinius mainline. For a Rubinius committer wanting to make a change to specs, the process is a bit different. One note to make is that having RubySpecs as a separate project imposes one extra step to the normal workflow even if not making any changes to the specs. When first cloning the rbx repo, one must run `git submodule init && git submodule update` to pull in the specs—which is obviously needed to just RUN the specs. Then, if a `git pull` contains submodule changes, another update is usually required with `git submodule update`.
Rubinius Committers
If a person working with the Rubinius repo wants to make changes or add a new spec or whatever, they can of course choose to just work on a separate RubySpec checkout. If the committer does not have a commit bit to RubySpecs, they MUST use a separate repository. That works exactly as described above.
If, on the other hand, we have an individual who is both a Rubinius and RubySpecs committer and wants to work directly in the Rubinius checkout, then the workflow looks like this:
- `cd spec/ruby`
- `git checkout master`
- Make and commit changes. (Use branches if you like but come back to master.)
- Push changes to mainline RubySpecs (just `git push`.) <—THIS IS IMPORTANT
- `cd ../..`
- `git add spec/ruby` <—NO TRAILING /, that overwrites the submodule
- Commit.
- Push to mainline Rubinius.
At this point the updated submodule becomes available to others working with the Rubinius repo using `git pull && git submodule update`.
Full Example
Here is a long example of the entire process, covering some ground that TFM did not seem to do. Please send in any corrections and/or improvements.1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 |
# We will work from scratch to illustrate the process
$ cd ~/code/tmp
# OK, first set up our submodule
$ mkdir rubyspec
$ cd rubyspec
$ git init
Initialized empty Git repository in .git/
$ echo 'rubyspec' > README
$ git add README
$ git commit -a -m 'Initial import of rubyspec'
Created initial commit 4c43e66: Initial import of rubyspec
1 files changed, 1 insertions(+), 0 deletions(-)
create mode 100644 README
$ cd ..
# OK, now set up the 'public' rubyspec repo (rubyspec.com or whatever)
$ git clone --bare rubyspec rubyspec.git
Initialized empty Git repository in /Users/user/code/tmp/rubyspec.git/
# Next, we create the superproject, obviously the public rbx repo
# already exists in reality.
$ mkdir rbx
$ cd rbx
$ git init
Initialized empty Git repository in .git/
$ echo 'rbx' > README
$ git add README
$ git commit -a -m 'RBX import'
Created initial commit 43b9035: RBX import
1 files changed, 1 insertions(+), 0 deletions(-)
create mode 100644 README
$ cd ..
# OK, same deal here, make a 'public' repo (git://git.rubini.us in reality)
$ git clone --bare rbx rbx.git
Initialized empty Git repository in /Users/user/code/tmp/rbx.git/
# Now, let us get rid of the originals so we have a more realistic situation
$ rm -rf rbx rubyspec
$ ls
rbx.git rubyspec.git
# Next we want to set up the submodule link, and we need a real rbx repo
$ git clone rbx.git my_rbx
Initialized empty Git repository in /Users/user/code/tmp/my_rbx/.git/
$ cd my_rbx
$ ls
README
# Now we associate the submodule (in reality we would use the URI, not local path)
$ mkdir spec
# We can 'mount' the submodule anywhere in the repo, in this case it goes in spec/ruby/
$ git submodule add ~/code/tmp/rubyspec.git spec/ruby
Initialized empty Git repository in /Users/user/code/tmp/my_rbx/spec/ruby/.git/
$ ls -la
total 16
drwxr-xr-x 4 user group 204 Mar 12 19:39 .
drwxr-xr-x 7 user group 238 Mar 12 19:37 ..
drwxr-xr-x 7 user group 374 Mar 12 19:39 .git
-rw-r--r-- 1 user group 85 Mar 12 19:39 .gitmodules
-rw-r--r-- 1 user group 4 Mar 12 19:37 README
drwxr-xr-x 3 user group 102 Mar 12 19:39 spec
# .gitmodules has the tracking info
$ cat .gitmodules
[submodule "spec/ruby"]
path = spec/ruby
url = /Users/user/code/tmp/rubyspec.git
# Our rubyspec repo should be in place now
$ ls -la spec/ruby
total 8
drwxr-xr-x 3 user group 136 Mar 12 19:39 .
drwxr-xr-x 3 user group 102 Mar 12 19:39 ..
drwxr-xr-x 7 user group 374 Mar 12 19:39 .git
-rw-r--r-- 1 user group 9 Mar 12 19:39 README
$ cat spec/ruby/README
rubyspec
# Now we need to record the submodule in the superproject
$ git status
On branch master
Changes to be committed:
(use "git reset HEAD <file>..." to unstage)
new file: .gitmodules
new file: spec/ruby
$ git commit -m 'Imported rubyspec submodule'
Created commit 0ae5cd3: Imported rubyspec submodule
2 files changed, 4 insertions(+), 0 deletions(-)
create mode 100644 .gitmodules
create mode 160000 spec/ruby
# And then record this over at the mainline
$ git push
Counting objects: 5, done.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (4/4), 439 bytes, done.
Total 4 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (4/4), done.
To /Users/user/code/tmp/rbx.git
43b9035..0ae5cd3 master -> master
$ cd ..
# Alright. Now, our test situation is that Joe just hacks on rubyspec and
# Bob hacks on rubyspec as a part of rbx!
# Let us create Bob's repo quick first:
$ git clone rbx.git bobs_rbx
$ cd bobs_rbx
$ ls -la
total 8
drwxr-xr-x 4 user group 204 Mar 12 19:47 .
drwxr-xr-x 9 user group 306 Mar 12 19:44 ..
drwxr-xr-x 7 user group 374 Mar 12 19:44 .git
-rw-r--r-- 1 user group 85 Mar 12 19:44 .gitmodules
-rw-r--r-- 1 user group 0 Mar 12 19:45 README
drwxr-xr-x 3 user group 102 Mar 12 19:44 spec
# Now, as you can see, a pristine checkout of a superproject will not actually contain the submodule code
$ ls -la spec/ruby
total 0
drwxr-xr-x 2 user group 68 Mar 12 19:44 .
drwxr-xr-x 3 user group 102 Mar 12 19:44 ..
# We need to first init to properly link the modules and then update to pull the data in
$ git submodule init
Submodule 'spec/ruby' (/Users/user/code/tmp/rubyspec.git) registered for path 'spec/ruby'
$ git submodule update
Initialized empty Git repository in /Users/user/code/tmp/bobs_rbx/spec/ruby/.git/
Submodule path 'spec/ruby': checked out '4c43e665381054478aa7c82da1d1d63b5c258e4b'
$ ls -la spec/ruby
total 8
drwxr-xr-x 3 user group 136 Mar 12 19:48 .
drwxr-xr-x 3 user group 102 Mar 12 19:48 ..
drwxr-xr-x 7 user group 408 Mar 12 19:48 .git
-rw-r--r-- 1 user group 9 Mar 12 19:48 README
# OK, now it is there. Let us hop out of here for a bit.
$ cd ..
# Now, Joe is doing some work on his repo which is JUST the rubyspec submodule
$ git clone rubyspec.git joes_spec
Initialized empty Git repository in /Users/user/code/tmp/joes_spec/.git/
$ cd joes_spec
$ ls
README
# OK, Joe makes a change
$ echo 'new spec' > spec.rb
$ git add spec.rb
$ git commit -m 'Added spec.rb'
Created commit f380258: Added spec.rb
1 files changed, 1 insertions(+), 0 deletions(-)
create mode 100644 spec.rb
# Joe pushes to mainline of rubyspec
$ git push
Counting objects: 4, done.
Compressing objects: 100% (2/2), done.
Writing objects: 100% (3/3), 290 bytes, done.
Total 3 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
To /Users/user/code/tmp/rubyspec.git
4c43e66..f380258 master -> master
# Now, let us check in with Bob who has his clone of the rbx repo
$ cd bobs_rbx
# Bob does not have Joe's spec yet
$ ls -la spec/ruby/
total 8
drwxr-xr-x 3 user group 136 Mar 12 19:48 .
drwxr-xr-x 3 user group 102 Mar 12 19:48 ..
drwxr-xr-x 7 user group 408 Mar 12 19:48 .git
-rw-r--r-- 1 user group 9 Mar 12 19:48 README
# Updating does not seem to work either!
$ git submodule update
$ ls -la spec/ruby/
total 8
drwxr-xr-x 3 user group 136 Mar 12 19:48 .
drwxr-xr-x 3 user group 102 Mar 12 19:48 ..
drwxr-xr-x 7 user group 408 Mar 12 19:48 .git
-rw-r--r-- 1 user group 9 Mar 12 19:48 README
# Did Joe's change get registered? Let us check..
$ cd ..
$ git clone rubyspec.git check_joes_change
Initialized empty Git repository in /Users/user/code/tmp/check_joes_change/.git/
$ ls check_joes_change/
README spec.rb
# Yep, it is there! So what is going on?
# Right, we need to update the mainline rbx. Technically Bob could actually do this
# too but for now, let us pretend that some access control allows only me to make
# that update. So back to my repo.
$ cd my_rbx/
$
# Now, we need to grab the change first
$ cd spec/ruby/
$ ls
README
$ git status
On branch master
nothing to commit (working directory clean)
$ git pull
remote: Counting objects: 4, done.
remote: Compressing objects: 100% (2/2)remote: , done.
remote: Total 3 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
From /Users/user/code/tmp/rubyspec
4c43e66..f380258 master -> origin/master
Updating 4c43e66..f380258
Fast forward
spec.rb | 1 +
1 files changed, 1 insertions(+), 0 deletions(-)
create mode 100644 spec.rb
$ ls -la
total 16
drwxr-xr-x 3 user group 170 Mar 12 19:54 .
drwxr-xr-x 3 user group 102 Mar 12 19:39 ..
drwxr-xr-x 7 user group 442 Mar 12 19:54 .git
-rw-r--r-- 1 user group 9 Mar 12 19:39 README
-rw-r--r-- 1 user group 9 Mar 12 19:54 spec.rb
# OK, there it is!
$ cd ../../
# So, need to record this update in the superproject
$ git status
On branch master
Changed but not updated:
(use "git add <file>..." to update what will be committed)
modified: spec/ruby
no changes added to commit (use "git add" and/or "git commit -a")
$ git add spec/ruby
$ git commit -m 'Updated our rubyspec checkout to newest version'
Created commit 0d96517: Updated our rubyspec checkout to newest version
1 files changed, 1 insertions(+), 1 deletions(-)
# And then push this change to mainline
$ git push
Counting objects: 5, done.
Compressing objects: 100% (2/2), done.
Writing objects: 100% (3/3), 319 bytes, done.
Total 3 (delta 1), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
To /Users/user/code/tmp/rbx.git
0ae5cd3..0d96517 master -> master
$ cd ..
$ cd bobs_rbx/
# NOW Bob should be able to access it
$ git submodule update
# No? How about this:
$ git pull
remote: Counting objects: 5, done.
remote: Compressing objects: 100% (2/2), done.
remote: Total 3 (delta 1), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
From /Users/user/code/tmp/rbx
0ae5cd3..0d96517 master -> origin/master
Updating 0ae5cd3..0d96517
README: needs update
Fast forward
spec/ruby | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
# That by itself does not update yet
$ ls -la spec/ruby
total 8
drwxr-xr-x 3 user group 136 Mar 12 19:48 .
drwxr-xr-x 3 user group 102 Mar 12 19:48 ..
drwxr-xr-x 7 user group 408 Mar 12 19:48 .git
-rw-r--r-- 1 user group 9 Mar 12 19:48 README
# Now we must update
$ git submodule update
Submodule path 'spec/ruby': checked out 'f38025812763dd8ee44beebcc0c8a3997c9aac0f'
# And now it is there
$ ls -la spec/ruby
total 16
drwxr-xr-x 3 user group 170 Mar 12 19:57 .
drwxr-xr-x 3 user group 102 Mar 12 19:48 ..
drwxr-xr-x 7 user group 408 Mar 12 19:57 .git
-rw-r--r-- 1 user group 9 Mar 12 19:48 README
-rw-r--r-- 1 user group 9 Mar 12 19:57 spec.rb
# And there we go, properly updated now
# Of course, Bob can make a few changes now too. Here is the only potential problem.
# First, change something in rbx.
$ echo 'RTFM' > INSTALL
$ git add INSTALL
$ git commit -m 'Installation instructions'
Created commit c0e6667: Installation instructions
1 files changed, 1 insertions(+), 0 deletions(-)
create mode 100644 INSTALL
$ git push
Counting objects: 4, done.
Compressing objects: 100% (2/2), done.
Writing objects: 100% (3/3), 361 bytes, done.
Total 3 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
To /Users/user/code/tmp/rbx.git
0d96517..c0e6667 master -> master
# That was normal. Now, what if Bob wants to make changes to rubyspec?
# He can make the changes normally. If he has commit access, he can
# just hop over and do so.
$ cd spec/ruby
$ ls
README spec.rb
# One notable thing is that this is a detached by default to avoid some problems, so
# Bob must go to a real branch.
$ git status
Not currently on any branch.
nothing to commit (working directory clean)
$ git checkout master
Previous HEAD position was b292add... Added a spec
Switched to branch "master"
$ echo 'my spec' > spec2.rb
$ git add spec2.rb
$ git commit -m 'Added a spec'
Created commit b1e7bdb: Added a spec
1 files changed, 1 insertions(+), 0 deletions(-)
create mode 100644 spec2.rb
# Now Bob can just push it if he has access
$ git push
Counting objects: 4, done.
Compressing objects: 100% (2/2), done.
Writing objects: 100% (3/3), 313 bytes, done.
Total 3 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
To /Users/user/code/tmp/rubyspec.git
f380258..b1e7bdb master -> master
# As the data shows, we directly updated the rubyspec project
$
# Obviously, if Bob did not have commit rights, he would do the normal git-format-patch dance instead
$
# Now, the problem
$ cd ../..
$ git status
On branch master
Changed but not updated:
(use "git add <file>..." to update what will be committed)
modified: spec/ruby
no changes added to commit (use "git add" and/or "git commit -a")
# This shows the change we made--and adding it is perfectly safe so long
# as we do it correctly. This is incorrect:
$ git add spec/ruby/
$ git status
On branch master
Changes to be committed:
(use "git reset HEAD <file>..." to unstage)
deleted: spec/ruby
new file: spec/ruby/README
new file: spec/ruby/spec.rb
new file: spec/ruby/spec2.rb
$ git reset --hard HEAD
HEAD is now at c0e6667... Installation instructions
# OK, so that is not good. The trailing / has git thinking this is a new
# path which overwrites the submodule. So, without:
$ git add spec/ruby
$ git status
On branch master
Changes to be committed:
(use "git reset HEAD <file>..." to unstage)
modified: spec/ruby
# So that worked fine. Now we can update mainline normally
$ git commit -m 'New spec in rubyspec, updated here and pushed to rubyspec mainline'
Created commit 3749f4b: New spec in rubyspec, updated here and pushed to rubyspec mainline
1 files changed, 1 insertions(+), 1 deletions(-)
$ git push
Counting objects: 5, done.
Compressing objects: 100% (2/2), done.
Writing objects: 100% (3/3), 331 bytes, done.
Total 3 (delta 1), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
To /Users/user/code/tmp/rbx.git
c0e6667..3749f4b master -> master
$
|