From d57e490af31dff940052a5929fe05947c7ad03ad Mon Sep 17 00:00:00 2001 From: Jonathan Nieder Date: Wed, 11 Apr 2012 16:25:01 -0500 Subject: [PATCH] fast-import doc: cat-blob and ls responses need to be consumed quickly If fast-import's command pipe and the frontend's cat-blob/ls response pipe are both filled, there can be a deadlock. Luckily all existing frontends consume any pending cat-blob/ls responses completely before writing the next command. Document the requirements so future frontend authors and users can be spared from the problem, too. It is not always easy to catch that kind of bug by testing. To set the scene, add some words of explanation to help the novice understand that "cat-blob" and "ls" output are meant for consumption by the frontend. Reported-by: Junio C Hamano Signed-off-by: Jonathan Nieder Signed-off-by: Junio C Hamano --- Documentation/git-fast-import.txt | 42 ++++++++++++++++++++++++++++--- 1 file changed, 39 insertions(+), 3 deletions(-) diff --git a/Documentation/git-fast-import.txt b/Documentation/git-fast-import.txt index ec6ef3119..b52dca513 100644 --- a/Documentation/git-fast-import.txt +++ b/Documentation/git-fast-import.txt @@ -98,9 +98,10 @@ OPTIONS options. --cat-blob-fd=:: - Specify the file descriptor that will be written to - when the `cat-blob` command is encountered in the stream. - The default behaviour is to write to `stdout`. + Write responses to `cat-blob` and `ls` queries to the + file descriptor instead of `stdout`. Allows `progress` + output intended for the end-user to be separated from other + output. --done:: Require a `done` command at the end of the stream. @@ -942,6 +943,9 @@ This command can be used anywhere in the stream that comments are accepted. In particular, the `cat-blob` command can be used in the middle of a commit but not in the middle of a `data` command. +See ``Responses To Commands'' below for details about how to read +this output safely. + `ls` ~~~~ Prints information about the object at a path to a file descriptor @@ -991,6 +995,9 @@ instead report missing SP LF ==== +See ``Responses To Commands'' below for details about how to read +this output safely. + `feature` ~~~~~~~~~ Require that fast-import supports the specified feature, or abort if @@ -1079,6 +1086,35 @@ If the `--done` command line option or `feature done` command is in use, the `done` command is mandatory and marks the end of the stream. +Responses To Commands +--------------------- +New objects written by fast-import are not available immediately. +Most fast-import commands have no visible effect until the next +checkpoint (or completion). The frontend can send commands to +fill fast-import's input pipe without worrying about how quickly +they will take effect, which improves performance by simplifying +scheduling. + +For some frontends, though, it is useful to be able to read back +data from the current repository as it is being updated (for +example when the source material describes objects in terms of +patches to be applied to previously imported objects). This can +be accomplished by connecting the frontend and fast-import via +bidirectional pipes: + +==== + mkfifo fast-import-output + frontend fast-import-output +==== + +A frontend set up this way can use `progress`, `ls`, and `cat-blob` +commands to read information from the import in progress. + +To avoid deadlock, such frontends must completely consume any +pending output from `progress`, `ls`, and `cat-blob` before +performing writes to fast-import that might block. + Crash Reports ------------- If fast-import is supplied invalid input it will terminate with a -- 2.26.2