2008-12

On the otherwise-idle flubber:

$ git clone git://sources.redhat.com/git/glibc.git
Initialized empty Git repository in /media/data/home/tschwinge/tmp/glibc/glibc/.git/
remote: Generating pack...
remote: Done counting 380933 objects.
remote: Deltifying 380933 objects...
remote:  100% (380933/380933) done
remote: Total 380933 (delta 294166), reused 380686 (delta 294002)
Receiving objects: 100% (380933/380933), 70.31 MiB | 27 KiB/s, done.
Resolving deltas: 100% (294166/294166), done.
error: git-checkout-index: unable to create file iconvdata/ibm1122.c (Interrupted system call)
error: git-checkout-index: unable to create file localedata/charmaps/IBM862 (Interrupted system call)
Checking out files: 100% (10676/10676), done.
$ git status
# On branch master
# Changed but not updated:
#   (use "git add <file>..." to update what will be committed)
#
#       modified:   iconvdata/ibm1122.c
#       modified:   localedata/charmaps/IBM862
#
no changes added to commit (use "git add" and/or "git commit -a")
$ ls -l iconvdata/ibm1122.c localedata/charmaps/IBM862
-rw-r--r-- 1 tschwinge tschwinge 0 2008-12-15 15:49 iconvdata/ibm1122.c
-rw-r--r-- 1 tschwinge tschwinge 0 2008-12-15 15:49 localedata/charmaps/IBM862

So these files are indeed of zero-length in the checked-out tree. Is this Git's fault or something else's?

Fixing this situation is easy enough:

$ git checkout -- iconvdata/ibm1122.c localedata/charmaps/IBM862
$ git status
# On branch master
nothing to commit (working directory clean)

2010-03-16

Still seen.

2010-11-17

A very similar issue. The working tree had a lot of differences to HEAD.

tschwinge@grubber:~/tmp/gcc/hurd $ git reset --hard HEAD
error: unable to unlink old 'gcc/config/darwin.h' (Interrupted system call)
Checking out files: 100% (1149/1149), done.
fatal: Could not reset index file to revision 'HEAD'.
tschwinge@grubber:~/tmp/gcc/hurd $ git reset --hard HEAD
error: unable to unlink old 'gcc/config/iq2000/iq2000.md' (Interrupted system call)
error: git checkout-index: unable to create file gcc/config/lm32/lm32.c (File exists)
Checking out files: 100% (1149/1149), done.
fatal: Could not reset index file to revision 'HEAD'.
tschwinge@grubber:~/tmp/gcc/hurd $ ls -l gcc/config/iq2000/iq2000.md gcc/config/lm32/lm32.c
ls: cannot access gcc/config/iq2000/iq2000.md: No such file or directory
-rw-r--r-- 1 tschwinge tschwinge 32159 Nov 17 19:09 gcc/config/lm32/lm32.c
tschwinge@grubber:~/tmp/gcc/hurd $ git reset --hard HEAD
error: git checkout-index: unable to create file gcc/fortran/expr.c (Interrupted system call)
Checking out files: 100% (1149/1149), done.
fatal: Could not reset index file to revision 'HEAD'.
tschwinge@grubber:~/tmp/gcc/hurd $ git reset --hard HEAD
error: git checkout-index: unable to create file gcc/config/sol2.h (Interrupted system call)
Checking out files: 100% (1149/1149), done.
fatal: Could not reset index file to revision 'HEAD'.
tschwinge@grubber:~/tmp/gcc/hurd $ git reset --hard HEAD
error: unable to unlink old 'gcc/config/i386/i386.c' (Interrupted system call)
Checking out files: 100% (1149/1149), done.
fatal: Could not reset index file to revision 'HEAD'.
tschwinge@grubber:~/tmp/gcc/hurd $ git reset --hard HEAD
Checking out files: 100% (1149/1149), done.
HEAD is now at fe3e43c Merge commit 'refs/top-bases/hurd/master' into hurd/master

2010-12-22

grubber:

$ git remote update
Fetching savannah
remote: Counting objects: 582331, done.
remote: Compressing objects: 100% (124133/124133), done.
remote: Total 582331 (delta 460856), reused 578352 (delta 457598)
Receiving objects: 100% (582331/582331), 525.15 MiB | 204 KiB/s, done.
fatal: cannot pread pack file: Interrupted system call
fatal: index-pack failed
error: Could not fetch savannah

2011-06-10

coulomb.SCHWINGE, checking out binutils' master branch, starting from an empty working directory (after an external git push):

$ git checkout -f
fatal: cannot create directory at 'gas/testsuite/gas/bfin': Interrupted system call
$ git checkout -f
error: unable to create file gas/testsuite/gas/i386/ilp32/x86-64-sse4_1-intel.d (File exists)
warning: unable to unlink gas/testsuite/gas/m68k-coff: Operation not permitted
fatal: cannot create directory at 'gas/testsuite/gas/m68k-coff': Operation not permitted
$ git checkout -f
error: unable to create file gas/testsuite/gas/h8300/h8300.exp (File exists)
error: unable to create file gas/testsuite/gas/i386/x86-64-addr32-intel.d (File exists)
error: unable to create file gas/testsuite/gas/ia64/secname.d (File exists)
error: unable to create file gas/testsuite/gas/m68k/pr11676.s (File exists)
Checking out files: 100% (12315/12315), done.
$ git status
# On branch master
# Changes not staged for commit:
#   (use "git add <file>..." to update what will be committed)
#   (use "git checkout -- <file>..." to discard changes in working directory)
#
#       modified:   gas/testsuite/gas/h8300/h8300.exp
#       modified:   gas/testsuite/gas/i386/x86-64-addr32-intel.d
#       modified:   gas/testsuite/gas/ia64/secname.d
#       modified:   gas/testsuite/gas/m68k/pr11676.s
#
no changes added to commit (use "git add" and/or "git commit -a")
$ rm gas/testsuite/gas/h8300/h8300.exp gas/testsuite/gas/i386/x86-64-addr32-intel.d gas/testsuite/gas/ia64/secname.d gas/testsuite/gas/m68k/pr11676.s
$ git checkout -f
$ git status
# On branch master
nothing to commit (working directory clean)

Analysis

2011-06-13

Running git checkout -f under GDB:

error: git checkout-index: unable to create file gas/testsuite/gas/cris/string-1.s (File exists)
error: git checkout-index: unable to create file gas/testsuite/gas/i386/x86-64-sse-check.d (File exists)
error: git checkout-index: unable to create file gas/testsuite/gas/i386/x86-64-sse4_1.d (File exists)
error: git checkout-index: unable to create file gas/testsuite/gas/ppc/astest.d (File exists)
error: git checkout-index: unable to create file gas/testsuite/gas/tic6x/reloc-bad-4.s (File exists)
warning: unable to unlink include/cgen: Operation not permitted
fatal: cannot create directory at 'include/cgen': Operation not permitted

Again:

error: git checkout-index: unable to create file gas/config/te-vxworks.h (File exists)
error: git checkout-index: unable to create file gas/testsuite/gas/cris/string-1.s (File exists)
error: git checkout-index: unable to create file gas/testsuite/gas/d10v/warning-019.s (File exists)
error: git checkout-index: unable to create file gas/testsuite/gas/i860/dual03.s (File exists)
error: git checkout-index: unable to create file ld/testsuite/ld-mmix/sec-7a.s (File exists)
warning: unable to unlink ld/testsuite/ld-powerpc: Operation not permitted
fatal: cannot create directory at 'ld/testsuite/ld-powerpc': Operation not permitted

And: git duplicated content.

All these (very likely) have the same root cause: SA_RESTART restarting too much.

With git checkout, Git uses in progress.c a SIGALRM handler (SA_RESTART; invoked every second via setitimer(ITIMER_REAL)) to display status messages: x % already checked out.

To avoid the status update signals every second, in [git]/progress.c:start_progress_delay we can just return NULL (manually in GDB, for example), then both the error: git checkout-index and the duplicated content issues go away.

I'm guessing that when returning from a SA_RESTART signal handler, too much of the ``syscall''s is being restarted. For example, if a file has already been created, the restarted creation attempt would fail: File exists. If data has been written, it might get written again (duplication issue). Then, there are cases where unlink apparently returns EINTR, which is not kosher either. Etc.

Do we have problems with SA_RESTART vs. the atomicity of our syscall-alikes?

IRC, freenode, #hurd, 2013-01-30

<braunr> hm, let's try to clone a huge repository
<braunr> hm, cloning a whole linux repo, and still no problem :)
<pinotree> weren't most/all the issues at unpack time?
<braunr> i don't remember
<braunr> we'll see when it gets there
<braunr> the longest part is "resolving deltas", for which ext2fs is
  clearly the big bottleneck (no I/O, page-cache only, but still)
<braunr> pinotree: well, slow, but no error

IRC, freenode, #hurd, 2013-01-31

<braunr> fyi, i've tried several checkouts of big repositories, and never
  got a single error
<braunr> youpi: looks like the recent fixes also solved some git issues we
  had
<braunr> i could clone big repositories without any problem
<youpi> cool :)