git-clone: aggressively optimize local clone behaviour.

This changes the behaviour of cloning from a repository on the
local machine, by defaulting to "-l" (use hardlinks to share
files under .git/objects) and making "-l" a no-op.  A new
option, --no-hardlinks, is also added to cause file-level copy
of files under .git/objects while still avoiding the normal
"pack to pipe, then receive and index pack" network transfer
overhead.  The old behaviour of local cloning without -l nor -s
is availble by specifying the source repository with the newly
introduced file:///path/to/repo.git/ syntax (i.e. "same as
network" cloning).

 * With --no-hardlinks (i.e. have all .git/objects/ copied via
   cpio) would not catch the source repository corruption, and
   also risks corrupted recipient repository if an
   alpha-particle hits memory cell while indexing and resolving
   deltas.  As long as the recipient is created uncorrupted, you
   have a good back-up.

 * same-as-network is expensive, but it would catch the breakage
   of the source repository.  It still risks corrupted recipient
   repository due to hardware failure.  As long as the recipient
   is created uncorrupted, you have a good back-up.

 * The new default on the same filesystem, as long as the source
   repository is healthy, it is very likely that the recipient
   would be, too.  Also it is very cheap.  You do not get any
   back-up benefit, though.

None of the method is resilient against the source repository
corruption, so let's discount that from the comparison.  Then
the difference with and without --no-hardlinks matters primarily
if you value the back-up benefit or not.  If you want to use the
cloned repository as a back-up, then it is cheaper to do a clone
with --no-hardlinks and two git-fsck (source before clone,
recipient after clone) than same-as-network clone, especially as
you are likely to do a git-fsck on the recipient if you are so
paranoid anyway.

Which leads me to believe that being able to use file:/// is
probably a good idea, if only for testability, but probably of
little practical value.  We default to hardlinked clone for
everyday use, and paranoids can use --no-hardlinks as a way to
make a back-up.

Signed-off-by: Junio C Hamano <gitster@pobox.com>

diff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt
index a0a10e3e26830d59cbb6eaddaab60a83fa601aa9..227f092e268497aad9c16cff4d796dabf60401de 100644 (file)
--- a/Documentation/git-clone.txt
+++ b/Documentation/git-clone.txt
@@ -9,7 +9,8 @@ git-clone - Clone a repository into a new directory
 SYNOPSIS
 --------
 [verse]
-'git-clone' [--template=<template_directory>] [-l [-s]] [-q] [-n] [--bare]
+'git-clone' [--template=<template_directory>]
+         [-l] [-s] [--no-hardlinks] [-q] [-n] [--bare]
          [-o <name>] [-u <upload-pack>] [--reference <repository>]
          [--depth <depth>] <repository> [<directory>]
 
@@ -40,8 +41,19 @@ OPTIONS
        this flag bypasses normal "git aware" transport
        mechanism and clones the repository by making a copy of
        HEAD and everything under objects and refs directories.
-       The files under .git/objects/ directory are hardlinked
-       to save space when possible.
+       The files under `.git/objects/` directory are hardlinked
+       to save space when possible.  This is now the default when
+       the source repository is specified with `/path/to/repo`
+       syntax, so it essentially is a no-op option.  To force
+       copying instead of hardlinking (which may be desirable
+       if you are trying to make a back-up of your repository),
+       but still avoid the usual "git aware" transport
+       mechanism, `--no-hardlinks` can be used.
+
+--no-hardlinks::
+       Optimize the cloning process from a repository on a
+       local filesystem by copying files under `.git/objects`
+       directory.
 
 --shared::
 -s::

diff --git a/Documentation/urls.txt b/Documentation/urls.txt
index 781df4174b7493eb905d0c87c73751bd4bc77b75..b38145faff384344fde354152db186275ebeb5ca 100644 (file)
--- a/Documentation/urls.txt
+++ b/Documentation/urls.txt
@@ -15,11 +15,11 @@ to name the remote repository:
 - ssh://{startsb}user@{endsb}host.xz/~/path/to/repo.git
 ===============================================================
 
-SSH is the default transport protocol.  You can optionally specify
-which user to log-in as, and an alternate, scp-like syntax is also
-supported.  Both syntaxes support username expansion,
-as does the native git protocol. The following three are
-identical to the last three above, respectively:
+SSH is the default transport protocol over the network.  You can
+optionally specify which user to log-in as, and an alternate,
+scp-like syntax is also supported.  Both syntaxes support
+username expansion, as does the native git protocol. The following
+three are identical to the last three above, respectively:
 
 ===============================================================
 - {startsb}user@{endsb}host.xz:/path/to/repo.git/
@@ -27,8 +27,12 @@ identical to the last three above, respectively:
 - {startsb}user@{endsb}host.xz:path/to/repo.git
 ===============================================================
 
-To sync with a local directory, use:
+To sync with a local directory, you can use:
 
 ===============================================================
 - /path/to/repo.git/
+- file:///path/to/repo.git/
 ===============================================================
+
+They are mostly equivalent, except when cloning.  See
+gitlink:git-clone[1] for details.

diff --git a/git-clone.sh b/git-clone.sh
index 09225540e607c5337f6a4807a570d7e296a555b9..4c9b1c9710c0fd55dfdbaf6bec6b77e0dae93562 100755 (executable)
--- a/git-clone.sh
+++ b/git-clone.sh
@@ -87,7 +87,7 @@ Perhaps git-update-server-info needs to be run there?"
 
 quiet=
 local=no
-use_local=no
+use_local_hardlink=yes
 local_shared=no
 unset template
 no_checkout=
@@ -108,9 +108,13 @@ while
          no_checkout=yes ;;
        *,--na|*,--nak|*,--nake|*,--naked|\
        *,-b|*,--b|*,--ba|*,--bar|*,--bare) bare=yes ;;
-       *,-l|*,--l|*,--lo|*,--loc|*,--loca|*,--local) use_local=yes ;;
+       *,-l|*,--l|*,--lo|*,--loc|*,--loca|*,--local)
+         use_local_hardlink=yes ;;
+       *,--no-h|*,--no-ha|*,--no-har|*,--no-hard|*,--no-hardl|\
+       *,--no-hardli|*,--no-hardlin|*,--no-hardlink|*,--no-hardlinks)
+         use_local_hardlink=no ;;
         *,-s|*,--s|*,--sh|*,--sha|*,--shar|*,--share|*,--shared)
-          local_shared=yes; use_local=yes ;;
+          local_shared=yes; ;;
        1,--template) usage ;;
        *,--template)
                shift; template="--template=$1" ;;
@@ -249,34 +253,36 @@ fi
 rm -f "$GIT_DIR/CLONE_HEAD"
 
 # We do local magic only when the user tells us to.
-case "$local,$use_local" in
-yes,yes)
+case "$local" in
+yes)
        ( cd "$repo/objects" ) ||
-               die "-l flag seen but repository '$repo' is not local."
+               die "cannot chdir to local '$repo/objects'."
 
-       case "$local_shared" in
-       no)
-           # See if we can hardlink and drop "l" if not.
-           sample_file=$(cd "$repo" && \
-                         find objects -type f -print | sed -e 1q)
-
-           # objects directory should not be empty since we are cloning!
-           test -f "$repo/$sample_file" || exit
-
-           l=
-           if ln "$repo/$sample_file" "$GIT_DIR/objects/sample" 2>/dev/null
-           then
-                   l=l
-           fi &&
-           rm -f "$GIT_DIR/objects/sample" &&
-           cd "$repo" &&
-           find objects -depth -print | cpio -pumd$l "$GIT_DIR/" || exit 1
-           ;;
-       yes)
-           mkdir -p "$GIT_DIR/objects/info"
-           echo "$repo/objects" >> "$GIT_DIR/objects/info/alternates"
-           ;;
-       esac
+       if test "$local_shared" = yes
+       then
+               mkdir -p "$GIT_DIR/objects/info"
+               echo "$repo/objects" >>"$GIT_DIR/objects/info/alternates"
+       else
+               l= &&
+               if test "$use_local_hardlink" = yes
+               then
+                       # See if we can hardlink and drop "l" if not.
+                       sample_file=$(cd "$repo" && \
+                                     find objects -type f -print | sed -e 1q)
+                       # objects directory should not be empty because
+                       # we are cloning!
+                       test -f "$repo/$sample_file" || exit
+                       if ln "$repo/$sample_file" "$GIT_DIR/objects/sample" 2>/dev/null
+                       then
+                               rm -f "$GIT_DIR/objects/sample"
+                               l=l
+                       else
+                               echo >&2 "Warning: -l asked but cannot hardlink to $repo"
+                       fi
+               fi &&
+               cd "$repo" &&
+               find objects -depth -print | cpio -pumd$l "$GIT_DIR/" || exit 1
+       fi
        git-ls-remote "$repo" >"$GIT_DIR/CLONE_HEAD" || exit 1
        ;;
 *)

diff --git a/t/t5500-fetch-pack.sh b/t/t5500-fetch-pack.sh
index 7da515361a8af4554760f1564f72e508a9bff2a5..7b6798d8b50f878c8957a60c058f6ad307f72789 100755 (executable)
--- a/t/t5500-fetch-pack.sh
+++ b/t/t5500-fetch-pack.sh
@@ -129,7 +129,7 @@ pull_to_client 2nd "B" $((64*3))
 
 pull_to_client 3rd "A" $((1*3)) # old fails
 
-test_expect_success "clone shallow" "git-clone --depth 2 . shallow"
+test_expect_success "clone shallow" "git-clone --depth 2 file://`pwd`/. shallow"
 
 (cd shallow; git count-objects -v) > count.shallow
 

diff --git a/t/t5700-clone-reference.sh b/t/t5700-clone-reference.sh
index 6d432525934980050f5c0d1683a1bc5e220b6e18..4e93aaab02e7b84b4bcf6ac70515e6cf52f0dabc 100755 (executable)
--- a/t/t5700-clone-reference.sh
+++ b/t/t5700-clone-reference.sh
@@ -51,7 +51,7 @@ diff expected current'
 cd "$base_dir"
 
 test_expect_success 'cloning with reference (no -l -s)' \
-'git clone --reference B A D'
+'git clone --reference B file://`pwd`/A D'
 
 cd "$base_dir"
 

diff --git a/t/t5701-clone-local.sh b/t/t5701-clone-local.sh
index b0933274db4ee73341bcc55e7d98fcddc3bae920..a3026ec4fcc653ff6a5a14bd84c1b3cc92078b6b 100755 (executable)
--- a/t/t5701-clone-local.sh
+++ b/t/t5701-clone-local.sh
@@ -43,4 +43,21 @@ test_expect_success 'local clone from x.git that does not exist' '
        fi
 '
 
+test_expect_success 'With -no-hardlinks, local will make a copy' '
+       cd "$D" &&
+       git clone --bare --no-hardlinks x w &&
+       cd w &&
+       linked=$(find objects -type f ! -links 1 | wc -l) &&
+       test "$linked" = 0
+'
+
+test_expect_success 'Even without -l, local will make a hardlink' '
+       cd "$D" &&
+       rm -fr w &&
+       git clone -l --bare x w &&
+       cd w &&
+       copied=$(find objects -type f -links 1 | wc -l) &&
+       test "$copied" = 0
+'
+
 test_done