.\" It was generated using the DocBook XSL Stylesheets (version 1.69.1).
.\" Instead of manually editing it, you probably should edit the DocBook XML
.\" source for it and then use the DocBook XSL Stylesheets to regenerate it.
-.TH "GIT\-FAST\-IMPORT" "1" "07/19/2007" "Git 1.5.3.rc2.19.gc4fba" "Git Manual"
+.TH "GIT\-FAST\-IMPORT" "1" "08/19/2007" "Git 1.5.3.rc5.40.g2f82" "Git Manual"
.\" disable hyphenation
.nh
.\" disable justification (adjust text to left margin only)
With the exception of raw file data (which Git does not interpret) the fast\-import input format is text (ASCII) based. This text based format simplifies development and debugging of frontend programs, especially when a higher level language such as Perl, Python or Ruby is being used.
fast\-import is very strict about its input. Where we say SP below we mean \fBexactly\fR one space. Likewise LF means one (and only one) linefeed. Supplying additional whitespace characters will cause unexpected results, such as branch names or file names with leading or trailing spaces in their name, or early termination of fast\-import when it encounters unexpected input.
+.SS "Stream Comments"
+To aid in debugging frontends fast\-import ignores any line that begins with # (ASCII pound/hash) up to and including the line ending LF. A comment line may contain any sequence of bytes that does not contain an LF and therefore may be used to include any detailed debugging information that might be specific to the frontend and useful when inspecting a fast\-import data stream.
.SS "Date Formats"
The following date formats are supported. A frontend should select the format it will use for this import by passing the format name in the \-\-date\-format=<fmt> command line option.
.TP
.TP
checkpoint
Forces fast\-import to close the current packfile, generate its unique SHA\-1 checksum and index, and start a new packfile. This command is optional and is not needed to perform an import.
+.TP
+progress
+Causes fast\-import to echo the entire line to its own standard output. This command is optional and is not needed to perform an import.
.SS "commit"
Create or update a branch with a new commit, recording one logical change to the project.
.sp
('from' SP <committish> LF)?
('merge' SP <committish> LF)?
(filemodify | filedelete | filecopy | filerename | filedeleteall)*
- LF
+ LF?
.fi
where <ref> is the name of the branch to make the commit on. Typically branch names are prefixed with refs/heads/ in Git, so importing the CVS branch symbol RELENG\-1_0 would use refs/heads/RELENG\-1_0 for the value of <ref>. The value of <ref> must be a valid refname in Git. As LF is not valid in a Git refname, no quoting or escaping syntax is supported here.
The data command following committer must supply the commit message (see below for data command syntax). To import an empty commit message use a 0 length data. Commit messages are free\-form and are not interpreted by Git. Currently they must be encoded in UTF\-8, as fast\-import does not permit other encodings to be specified.
Zero or more filemodify, filedelete, filecopy, filerename and filedeleteall commands may be included to update the contents of the branch prior to creating the commit. These commands may be supplied in any order. However it is recommended that a filedeleteall command preceed all filemodify, filecopy and filerename commands in the same commit, as filedeleteall wipes the branch clean (see below).
+
+The LF after the command is optional (it used to be required).
.sp
.it 1 an-trap
.nr an-no-space-flag 1
.nf
'reset' SP <ref> LF
('from' SP <committish> LF)?
- LF
+ LF?
.fi
For a detailed description of <ref> and <committish> see above under commit and from.
+The LF after the command is optional (it used to be required).
+
The reset command can also be used to create lightweight (non\-annotated) tags. For example:
.IP
.sp
The mark command is optional here as some frontends have chosen to generate the Git SHA\-1 for the blob on their own, and feed that directly to commit. This is typically more work than its worth however, as marks are inexpensive to store and easy to use.
.SS "data"
Supplies raw data (for use as blob/file content, commit messages, or annotated tag messages) to fast\-import. Data can be supplied using an exact byte count or delimited with a terminating line. Real frontends intended for production\-quality conversions should always use the exact byte count format, as it is more robust and performs better. The delimited format is intended primarily for testing fast\-import.
+
+Comment lines appearing within the <raw> part of data commands are always taken to be part of the body of the data and are therefore never ignored by fast\-import. This makes it safe to import any file/message content whose lines might start with #.
.TP
Exact byte count format
The frontend must specify the number of bytes of data.
.sp
.nf
'data' SP <count> LF
- <raw> LF
+ <raw> LF?
.fi
where <count> is the exact number of bytes appearing within <raw>. The value of <count> is expressed as an ASCII decimal integer. The LF on either side of <raw> is not included in <count> and will not be included in the imported data.
+
+The LF after <raw> is optional (it used to be required) but recommended. Always including it makes debugging a fast\-import stream easier as the next command always starts in column 0 of the next line, even if <raw> did not end with an LF.
.TP
Delimited format
A delimiter string is used to mark the end of the data. fast\-import will compute the length by searching for the delimiter. This format is primarly useful for testing and is not recommended for real data.
'data' SP '<<' <delim> LF
<raw> LF
<delim> LF
+ LF?
.fi
where <delim> is the chosen delimiter string. The string <delim> must not appear on a line by itself within <raw>, as otherwise fast\-import will think the data ends earlier than it really does. The LF immediately trailing <raw> is part of <raw>. This is one of the limitations of the delimited format, it is impossible to supply a data chunk which does not have an LF as its last byte.
+
+The LF after <delim> LF is optional (it used to be required).
.SS "checkpoint"
Forces fast\-import to close the current packfile, start a new one, and to save out all current branch refs, tags and marks.
.sp
.nf
'checkpoint' LF
- LF
+ LF?
.fi
Note that fast\-import automatically switches packfiles when the current packfile reaches \-\-max\-pack\-size, or 4 GiB, whichever limit is smaller. During an automatic packfile switch fast\-import does not update the branch refs, tags or marks.
As a checkpoint can require a significant amount of CPU time and disk IO (to compute the overall pack SHA\-1 checksum, generate the corresponding index file, and update the refs) it can easily take several minutes for a single checkpoint command to complete.
Frontends may choose to issue checkpoints during extremely large and long running imports, or when they need to allow another Git process access to a branch. However given that a 30 GiB Subversion repository can be loaded into Git through fast\-import in about 3 hours, explicit checkpointing may not be necessary.
+
+The LF after the command is optional (it used to be required).
+.SS "progress"
+Causes fast\-import to print the entire progress line unmodified to its standard output channel (file descriptor 1) when the command is processed from the input stream. The command otherwise has no impact on the current import, or on any of fast\-import's internal state.
+.sp
+.nf
+ 'progress' SP <any> LF
+ LF?
+.fi
+The <any> part of the command may contain any sequence of bytes that does not contain LF. The LF after the command is optional. Callers may wish to process the output through a tool such as sed to remove the leading part of the line, for example:
+.IP
+.sp
+.nf
+frontend | git\-fast\-import | sed 's/^progress //'
+.fi
+Placing a progress command immediately after a checkpoint will inform the reader when the checkpoint has been completed and it can safely access the refs that fast\-import updated.
.SH "TIPS AND TRICKS"
The following tips and tricks have been collected from various users of fast\-import, and are offered here as suggestions.
.SS "Use One Mark Per Commit"
If you choose to wait for the repack, don't try to run benchmarks or performance tests until repacking is completed. fast\-import outputs suboptimal packfiles that are simply never seen in real use situations.
.SS "Repacking Historical Data"
If you are repacking very old imported data (e.g. older than the last year), consider expending some extra CPU time and supplying \-\-window=50 (or higher) when you run \fBgit\-repack\fR(1). This will take longer, but will also produce a smaller packfile. You only need to expend the effort once, and everyone using your project will benefit from the smaller repository.
+.SS "Include Some Progress Messages"
+Every once in a while have your frontend emit a progress message to fast\-import. The contents of the messages are entirely free\-form, so one suggestion would be to output the current month and year each time the current commit date moves into the next month. Your users will feel better knowing how much of the data stream has been processed.
.SH "PACKFILE OPTIMIZATION"
When packing a blob fast\-import always attempts to deltify against the last blob written. Unless specifically arranged for by the frontend, this will probably not be a prior version of the same file, so the generated delta will not be the smallest possible. The resulting packfile will be compressed, but will not be optimal.
projects / git.git / commitdiff