results, such as branch names or file names with leading or trailing\r
spaces in their name, or early termination of fast-import when it encounters\r
unexpected input.</p>\r
+<h3>Stream Comments</h3>\r
+<p>To aid in debugging frontends fast-import ignores any line that\r
+begins with <tt>#</tt> (ASCII pound/hash) up to and including the line\r
+ending <tt>LF</tt>. A comment line may contain any sequence of bytes\r
+that does not contain an LF and therefore may be used to include\r
+any detailed debugging information that might be specific to the\r
+frontend and useful when inspecting a fast-import data stream.</p>\r
<h3>Date Formats</h3>\r
<p>The following date formats are supported. A frontend should select\r
the format it will use for this import by passing the format name\r
an import.\r
</p>\r
</dd>\r
+<dt>\r
+<tt>progress</tt>\r
+</dt>\r
+<dd>\r
+<p>\r
+ Causes fast-import to echo the entire line to its own\r
+ standard output. This command is optional and is not needed\r
+ to perform an import.\r
+</p>\r
+</dd>\r
</dl>\r
<h3><tt>commit</tt></h3>\r
<p>Create or update a branch with a new commit, recording one logical\r
('from' SP <committish> LF)?\r
('merge' SP <committish> LF)?\r
(filemodify | filedelete | filecopy | filerename | filedeleteall)*\r
- LF</tt></pre>\r
+ LF?</tt></pre>\r
</div></div>\r
<p>where <tt><ref></tt> is the name of the branch to make the commit on.\r
Typically branch names are prefixed with <tt>refs/heads/</tt> in\r
all <tt>filemodify</tt>, <tt>filecopy</tt> and <tt>filerename</tt> commands in the same\r
commit, as <tt>filedeleteall</tt>\r
wipes the branch clean (see below).</p>\r
+<p>The <tt>LF</tt> after the command is optional (it used to be required).</p>\r
<h4><tt>author</tt></h4>\r
<p>An <tt>author</tt> command may optionally appear, if the author information\r
might differ from the committer information. If <tt>author</tt> is omitted\r
<div class="content">\r
<pre><tt> 'reset' SP <ref> LF\r
('from' SP <committish> LF)?\r
- LF</tt></pre>\r
+ LF?</tt></pre>\r
</div></div>\r
<p>For a detailed description of <tt><ref></tt> and <tt><committish></tt> see above\r
under <tt>commit</tt> and <tt>from</tt>.</p>\r
+<p>The <tt>LF</tt> after the command is optional (it used to be required).</p>\r
<p>The <tt>reset</tt> command can also be used to create lightweight\r
(non-annotated) tags. For example:</p>\r
<div class="exampleblock">\r
intended for production-quality conversions should always use the\r
exact byte count format, as it is more robust and performs better.\r
The delimited format is intended primarily for testing fast-import.</p>\r
+<p>Comment lines appearing within the <tt><raw></tt> part of <tt>data</tt> commands\r
+are always taken to be part of the body of the data and are therefore\r
+never ignored by fast-import. This makes it safe to import any\r
+file/message content whose lines might start with <tt>#</tt>.</p>\r
<dl>\r
<dt>\r
Exact byte count format\r
<div class="literalblock">\r
<div class="content">\r
<pre><tt> 'data' SP <count> LF\r
- <raw> LF</tt></pre>\r
+ <raw> LF?</tt></pre>\r
</div></div>\r
<p>where <tt><count></tt> is the exact number of bytes appearing within\r
<tt><raw></tt>. The value of <tt><count></tt> is expressed as an ASCII decimal\r
integer. The <tt>LF</tt> on either side of <tt><raw></tt> is not\r
included in <tt><count></tt> and will not be included in the imported data.</p>\r
+<p>The <tt>LF</tt> after <tt><raw></tt> is optional (it used to be required) but\r
+recommended. Always including it makes debugging a fast-import\r
+stream easier as the next command always starts in column 0\r
+of the next line, even if <tt><raw></tt> did not end with an <tt>LF</tt>.</p>\r
</dd>\r
<dt>\r
Delimited format\r
<div class="content">\r
<pre><tt> 'data' SP '<<' <delim> LF\r
<raw> LF\r
- <delim> LF</tt></pre>\r
+ <delim> LF\r
+ LF?</tt></pre>\r
</div></div>\r
<p>where <tt><delim></tt> is the chosen delimiter string. The string <tt><delim></tt>\r
must not appear on a line by itself within <tt><raw></tt>, as otherwise\r
immediately trailing <tt><raw></tt> is part of <tt><raw></tt>. This is one of\r
the limitations of the delimited format, it is impossible to supply\r
a data chunk which does not have an LF as its last byte.</p>\r
+<p>The <tt>LF</tt> after <tt><delim> LF</tt> is optional (it used to be required).</p>\r
</dd>\r
</dl>\r
<h3><tt>checkpoint</tt></h3>\r
<div class="literalblock">\r
<div class="content">\r
<pre><tt> 'checkpoint' LF\r
- LF</tt></pre>\r
+ LF?</tt></pre>\r
</div></div>\r
<p>Note that fast-import automatically switches packfiles when the current\r
packfile reaches --max-pack-size, or 4 GiB, whichever limit is\r
process access to a branch. However given that a 30 GiB Subversion\r
repository can be loaded into Git through fast-import in about 3 hours,\r
explicit checkpointing may not be necessary.</p>\r
+<p>The <tt>LF</tt> after the command is optional (it used to be required).</p>\r
+<h3><tt>progress</tt></h3>\r
+<p>Causes fast-import to print the entire <tt>progress</tt> line unmodified to\r
+its standard output channel (file descriptor 1) when the command is\r
+processed from the input stream. The command otherwise has no impact\r
+on the current import, or on any of fast-import's internal state.</p>\r
+<div class="literalblock">\r
+<div class="content">\r
+<pre><tt> 'progress' SP <any> LF\r
+ LF?</tt></pre>\r
+</div></div>\r
+<p>The <tt><any></tt> part of the command may contain any sequence of bytes\r
+that does not contain <tt>LF</tt>. The <tt>LF</tt> after the command is optional.\r
+Callers may wish to process the output through a tool such as sed to\r
+remove the leading part of the line, for example:</p>\r
+<div class="exampleblock">\r
+<div class="exampleblock-content">\r
+<div class="literalblock">\r
+<div class="content">\r
+<pre><tt>frontend | git-fast-import | sed 's/^progress //'</tt></pre>\r
+</div></div>\r
+</div></div>\r
+<p>Placing a <tt>progress</tt> command immediately after a <tt>checkpoint</tt> will\r
+inform the reader when the <tt>checkpoint</tt> has been completed and it\r
+can safely access the refs that fast-import updated.</p>\r
</div>\r
<h2>Tips and Tricks</h2>\r
<div class="sectionbody">\r
This will take longer, but will also produce a smaller packfile.\r
You only need to expend the effort once, and everyone using your\r
project will benefit from the smaller repository.</p>\r
+<h3>Include Some Progress Messages</h3>\r
+<p>Every once in a while have your frontend emit a <tt>progress</tt> message\r
+to fast-import. The contents of the messages are entirely free-form,\r
+so one suggestion would be to output the current month and year\r
+each time the current commit date moves into the next month.\r
+Your users will feel better knowing how much of the data stream\r
+has been processed.</p>\r
</div>\r
<h2>Packfile Optimization</h2>\r
<div class="sectionbody">\r
</div>\r
<div id="footer">\r
<div id="footer-text">\r
-Last updated 19-Jul-2007 02:09:39 UTC\r
+Last updated 19-Aug-2007 19:15:24 UTC\r
</div>\r
</div>\r
</body>\r
spaces in their name, or early termination of fast-import when it encounters
unexpected input.
+Stream Comments
+~~~~~~~~~~~~~~~
+To aid in debugging frontends fast-import ignores any line that
+begins with `#` (ASCII pound/hash) up to and including the line
+ending `LF`. A comment line may contain any sequence of bytes
+that does not contain an LF and therefore may be used to include
+any detailed debugging information that might be specific to the
+frontend and useful when inspecting a fast-import data stream.
+
Date Formats
~~~~~~~~~~~~
The following date formats are supported. A frontend should select
This command is optional and is not needed to perform
an import.
+`progress`::
+ Causes fast-import to echo the entire line to its own
+ standard output. This command is optional and is not needed
+ to perform an import.
+
`commit`
~~~~~~~~
Create or update a branch with a new commit, recording one logical
('from' SP <committish> LF)?
('merge' SP <committish> LF)?
(filemodify | filedelete | filecopy | filerename | filedeleteall)*
- LF
+ LF?
....
where `<ref>` is the name of the branch to make the commit on.
commit, as `filedeleteall`
wipes the branch clean (see below).
+The `LF` after the command is optional (it used to be required).
+
`author`
^^^^^^^^
An `author` command may optionally appear, if the author information
....
'reset' SP <ref> LF
('from' SP <committish> LF)?
- LF
+ LF?
....
For a detailed description of `<ref>` and `<committish>` see above
under `commit` and `from`.
+The `LF` after the command is optional (it used to be required).
+
The `reset` command can also be used to create lightweight
(non-annotated) tags. For example:
exact byte count format, as it is more robust and performs better.
The delimited format is intended primarily for testing fast-import.
+Comment lines appearing within the `<raw>` part of `data` commands
+are always taken to be part of the body of the data and are therefore
+never ignored by fast-import. This makes it safe to import any
+file/message content whose lines might start with `#`.
+
Exact byte count format::
The frontend must specify the number of bytes of data.
+
....
'data' SP <count> LF
- <raw> LF
+ <raw> LF?
....
+
where `<count>` is the exact number of bytes appearing within
`<raw>`. The value of `<count>` is expressed as an ASCII decimal
integer. The `LF` on either side of `<raw>` is not
included in `<count>` and will not be included in the imported data.
++
+The `LF` after `<raw>` is optional (it used to be required) but
+recommended. Always including it makes debugging a fast-import
+stream easier as the next command always starts in column 0
+of the next line, even if `<raw>` did not end with an `LF`.
Delimited format::
A delimiter string is used to mark the end of the data.
'data' SP '<<' <delim> LF
<raw> LF
<delim> LF
+ LF?
....
+
where `<delim>` is the chosen delimiter string. The string `<delim>`
immediately trailing `<raw>` is part of `<raw>`. This is one of
the limitations of the delimited format, it is impossible to supply
a data chunk which does not have an LF as its last byte.
++
+The `LF` after `<delim> LF` is optional (it used to be required).
`checkpoint`
~~~~~~~~~~~~
....
'checkpoint' LF
- LF
+ LF?
....
Note that fast-import automatically switches packfiles when the current
repository can be loaded into Git through fast-import in about 3 hours,
explicit checkpointing may not be necessary.
+The `LF` after the command is optional (it used to be required).
+
+`progress`
+~~~~~~~~~~
+Causes fast-import to print the entire `progress` line unmodified to
+its standard output channel (file descriptor 1) when the command is
+processed from the input stream. The command otherwise has no impact
+on the current import, or on any of fast-import's internal state.
+
+....
+ 'progress' SP <any> LF
+ LF?
+....
+
+The `<any>` part of the command may contain any sequence of bytes
+that does not contain `LF`. The `LF` after the command is optional.
+Callers may wish to process the output through a tool such as sed to
+remove the leading part of the line, for example:
+
+====
+ frontend | git-fast-import | sed 's/^progress //'
+====
+
+Placing a `progress` command immediately after a `checkpoint` will
+inform the reader when the `checkpoint` has been completed and it
+can safely access the refs that fast-import updated.
Tips and Tricks
---------------
You only need to expend the effort once, and everyone using your
project will benefit from the smaller repository.
+Include Some Progress Messages
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Every once in a while have your frontend emit a `progress` message
+to fast-import. The contents of the messages are entirely free-form,
+so one suggestion would be to output the current month and year
+each time the current commit date moves into the next month.
+Your users will feel better knowing how much of the data stream
+has been processed.
+
Packfile Optimization
---------------------
projects / git.git / commitdiff