--- /dev/null
+<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"\r
+ "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">\r
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en">\r
+<head>\r
+<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />\r
+<meta name="generator" content="AsciiDoc 7.0.2" />\r
+<style type="text/css">\r
+/* Debug borders */\r
+p, li, dt, dd, div, pre, h1, h2, h3, h4, h5, h6 {\r
+/*\r
+ border: 1px solid red;\r
+*/\r
+}\r
+\r
+body {\r
+ margin: 1em 5% 1em 5%;\r
+}\r
+\r
+a { color: blue; }\r
+a:visited { color: fuchsia; }\r
+\r
+em {\r
+ font-style: italic;\r
+}\r
+\r
+strong {\r
+ font-weight: bold;\r
+}\r
+\r
+tt {\r
+ color: navy;\r
+}\r
+\r
+h1, h2, h3, h4, h5, h6 {\r
+ color: #527bbd;\r
+ font-family: sans-serif;\r
+ margin-top: 1.2em;\r
+ margin-bottom: 0.5em;\r
+ line-height: 1.3;\r
+}\r
+\r
+h1 {\r
+ border-bottom: 2px solid silver;\r
+}\r
+h2 {\r
+ border-bottom: 2px solid silver;\r
+ padding-top: 0.5em;\r
+}\r
+\r
+div.sectionbody {\r
+ font-family: serif;\r
+ margin-left: 0;\r
+}\r
+\r
+hr {\r
+ border: 1px solid silver;\r
+}\r
+\r
+p {\r
+ margin-top: 0.5em;\r
+ margin-bottom: 0.5em;\r
+}\r
+\r
+pre {\r
+ padding: 0;\r
+ margin: 0;\r
+}\r
+\r
+span#author {\r
+ color: #527bbd;\r
+ font-family: sans-serif;\r
+ font-weight: bold;\r
+ font-size: 1.2em;\r
+}\r
+span#email {\r
+}\r
+span#revision {\r
+ font-family: sans-serif;\r
+}\r
+\r
+div#footer {\r
+ font-family: sans-serif;\r
+ font-size: small;\r
+ border-top: 2px solid silver;\r
+ padding-top: 0.5em;\r
+ margin-top: 4.0em;\r
+}\r
+div#footer-text {\r
+ float: left;\r
+ padding-bottom: 0.5em;\r
+}\r
+div#footer-badges {\r
+ float: right;\r
+ padding-bottom: 0.5em;\r
+}\r
+\r
+div#preamble,\r
+div.tableblock, div.imageblock, div.exampleblock, div.verseblock,\r
+div.quoteblock, div.literalblock, div.listingblock, div.sidebarblock,\r
+div.admonitionblock {\r
+ margin-right: 10%;\r
+ margin-top: 1.5em;\r
+ margin-bottom: 1.5em;\r
+}\r
+div.admonitionblock {\r
+ margin-top: 2.5em;\r
+ margin-bottom: 2.5em;\r
+}\r
+\r
+div.content { /* Block element content. */\r
+ padding: 0;\r
+}\r
+\r
+/* Block element titles. */\r
+div.title, caption.title {\r
+ font-family: sans-serif;\r
+ font-weight: bold;\r
+ text-align: left;\r
+ margin-top: 1.0em;\r
+ margin-bottom: 0.5em;\r
+}\r
+div.title + * {\r
+ margin-top: 0;\r
+}\r
+\r
+td div.title:first-child {\r
+ margin-top: 0.0em;\r
+}\r
+div.content div.title:first-child {\r
+ margin-top: 0.0em;\r
+}\r
+div.content + div.title {\r
+ margin-top: 0.0em;\r
+}\r
+\r
+div.sidebarblock > div.content {\r
+ background: #ffffee;\r
+ border: 1px solid silver;\r
+ padding: 0.5em;\r
+}\r
+\r
+div.listingblock > div.content {\r
+ border: 1px solid silver;\r
+ background: #f4f4f4;\r
+ padding: 0.5em;\r
+}\r
+\r
+div.quoteblock > div.content {\r
+ padding-left: 2.0em;\r
+}\r
+div.quoteblock .attribution {\r
+ text-align: right;\r
+}\r
+\r
+div.admonitionblock .icon {\r
+ vertical-align: top;\r
+ font-size: 1.1em;\r
+ font-weight: bold;\r
+ text-decoration: underline;\r
+ color: #527bbd;\r
+ padding-right: 0.5em;\r
+}\r
+div.admonitionblock td.content {\r
+ padding-left: 0.5em;\r
+ border-left: 2px solid silver;\r
+}\r
+\r
+div.exampleblock > div.content {\r
+ border-left: 2px solid silver;\r
+ padding: 0.5em;\r
+}\r
+\r
+div.verseblock div.content {\r
+ white-space: pre;\r
+}\r
+\r
+div.imageblock div.content { padding-left: 0; }\r
+div.imageblock img { border: 1px solid silver; }\r
+span.image img { border-style: none; }\r
+\r
+dl {\r
+ margin-top: 0.8em;\r
+ margin-bottom: 0.8em;\r
+}\r
+dt {\r
+ margin-top: 0.5em;\r
+ margin-bottom: 0;\r
+ font-style: italic;\r
+}\r
+dd > *:first-child {\r
+ margin-top: 0;\r
+}\r
+\r
+ul, ol {\r
+ list-style-position: outside;\r
+}\r
+ol.olist2 {\r
+ list-style-type: lower-alpha;\r
+}\r
+\r
+div.tableblock > table {\r
+ border-color: #527bbd;\r
+ border-width: 3px;\r
+}\r
+thead {\r
+ font-family: sans-serif;\r
+ font-weight: bold;\r
+}\r
+tfoot {\r
+ font-weight: bold;\r
+}\r
+\r
+div.hlist {\r
+ margin-top: 0.8em;\r
+ margin-bottom: 0.8em;\r
+}\r
+td.hlist1 {\r
+ vertical-align: top;\r
+ font-style: italic;\r
+ padding-right: 0.8em;\r
+}\r
+td.hlist2 {\r
+ vertical-align: top;\r
+}\r
+\r
+@media print {\r
+ div#footer-badges { display: none; }\r
+}\r
+include::./stylesheets/xhtml11-manpage.css[]\r
+/* Workarounds for IE6's broken and incomplete CSS2. */\r
+\r
+div.sidebar-content {\r
+ background: #ffffee;\r
+ border: 1px solid silver;\r
+ padding: 0.5em;\r
+}\r
+div.sidebar-title, div.image-title {\r
+ font-family: sans-serif;\r
+ font-weight: bold;\r
+ margin-top: 0.0em;\r
+ margin-bottom: 0.5em;\r
+}\r
+\r
+div.listingblock div.content {\r
+ border: 1px solid silver;\r
+ background: #f4f4f4;\r
+ padding: 0.5em;\r
+}\r
+\r
+div.quoteblock-content {\r
+ padding-left: 2.0em;\r
+}\r
+\r
+div.exampleblock-content {\r
+ border-left: 2px solid silver;\r
+ padding-left: 0.5em;\r
+}\r
+</style>\r
+<title>git-fast-import(1)</title>\r
+</head>\r
+<body>\r
+<div id="header">\r
+<h1>\r
+git-fast-import(1) Manual Page\r
+</h1>\r
+<h2>NAME</h2>\r
+<div class="sectionbody">\r
+<p>git-fast-import -\r
+ Backend for fast Git data importers.\r
+</p>\r
+</div>\r
+</div>\r
+<h2>SYNOPSIS</h2>\r
+<div class="sectionbody">\r
+<p>frontend | <em>git-fast-import</em> [options]</p>\r
+</div>\r
+<h2>DESCRIPTION</h2>\r
+<div class="sectionbody">\r
+<p>This program is usually not what the end user wants to run directly.\r
+Most end users want to use one of the existing frontend programs,\r
+which parses a specific type of foreign source and feeds the contents\r
+stored there to git-fast-import (gfi).</p>\r
+<p>gfi reads a mixed command/data stream from standard input and\r
+writes one or more packfiles directly into the current repository.\r
+When EOF is received on standard input, fast import writes out\r
+updated branch and tag refs, fully updating the current repository\r
+with the newly imported data.</p>\r
+<p>The gfi backend itself can import into an empty repository (one that\r
+has already been initialized by <a href="git-init.html">git-init(1)</a>) or incrementally\r
+update an existing populated repository. Whether or not incremental\r
+imports are supported from a particular foreign source depends on\r
+the frontend program in use.</p>\r
+</div>\r
+<h2>OPTIONS</h2>\r
+<div class="sectionbody">\r
+<dl>\r
+<dt>\r
+--date-format=<fmt>\r
+</dt>\r
+<dd>\r
+<p>\r
+ Specify the type of dates the frontend will supply to\r
+ gfi within <tt>author</tt>, <tt>committer</tt> and <tt>tagger</tt> commands.\r
+ See “Date Formats” below for details about which formats\r
+ are supported, and their syntax.\r
+</p>\r
+</dd>\r
+<dt>\r
+--force\r
+</dt>\r
+<dd>\r
+<p>\r
+ Force updating modified existing branches, even if doing\r
+ so would cause commits to be lost (as the new commit does\r
+ not contain the old commit).\r
+</p>\r
+</dd>\r
+<dt>\r
+--max-pack-size=<n>\r
+</dt>\r
+<dd>\r
+<p>\r
+ Maximum size of each output packfile, expressed in MiB.\r
+ The default is 4096 (4 GiB) as that is the maximum allowed\r
+ packfile size (due to file format limitations). Some\r
+ importers may wish to lower this, such as to ensure the\r
+ resulting packfiles fit on CDs.\r
+</p>\r
+</dd>\r
+<dt>\r
+--depth=<n>\r
+</dt>\r
+<dd>\r
+<p>\r
+ Maximum delta depth, for blob and tree deltification.\r
+ Default is 10.\r
+</p>\r
+</dd>\r
+<dt>\r
+--active-branches=<n>\r
+</dt>\r
+<dd>\r
+<p>\r
+ Maximum number of branches to maintain active at once.\r
+ See “Memory Utilization” below for details. Default is 5.\r
+</p>\r
+</dd>\r
+<dt>\r
+--export-marks=<file>\r
+</dt>\r
+<dd>\r
+<p>\r
+ Dumps the internal marks table to <file> when complete.\r
+ Marks are written one per line as <tt>:markid SHA-1</tt>.\r
+ Frontends can use this file to validate imports after they\r
+ have been completed.\r
+</p>\r
+</dd>\r
+</dl>\r
+</div>\r
+<h2>Performance</h2>\r
+<div class="sectionbody">\r
+<p>The design of gfi allows it to import large projects in a minimum\r
+amount of memory usage and processing time. Assuming the frontend\r
+is able to keep up with gfi and feed it a constant stream of data,\r
+import times for projects holding 10+ years of history and containing\r
+100,000+ individual commits are generally completed in just 1-2\r
+hours on quite modest (~$2,000 USD) hardware.</p>\r
+<p>Most bottlenecks appear to be in foreign source data access (the\r
+source just cannot extract revisions fast enough) or disk IO (gfi\r
+writes as fast as the disk will take the data). Imports will run\r
+faster if the source data is stored on a different drive than the\r
+destination Git repository (due to less IO contention).</p>\r
+</div>\r
+<h2>Development Cost</h2>\r
+<div class="sectionbody">\r
+<p>A typical frontend for gfi tends to weigh in at approximately 200\r
+lines of Perl/Python/Ruby code. Most developers have been able to\r
+create working importers in just a couple of hours, even though it\r
+is their first exposure to gfi, and sometimes even to Git. This is\r
+an ideal situation, given that most conversion tools are throw-away\r
+(use once, and never look back).</p>\r
+</div>\r
+<h2>Parallel Operation</h2>\r
+<div class="sectionbody">\r
+<p>Like <tt>git-push</tt> or <tt>git-fetch</tt>, imports handled by gfi are safe to\r
+run alongside parallel <tt>git repack -a -d</tt> or <tt>git gc</tt> invocations,\r
+or any other Git operation (including <tt>git prune</tt>, as loose objects\r
+are never used by gfi).</p>\r
+<p>gfi does not lock the branch or tag refs it is actively importing.\r
+After the import, during its ref update phase, gfi tests each\r
+existing branch ref to verify the update will be a fast-forward\r
+update (the commit stored in the ref is contained in the new\r
+history of the commit to be written). If the update is not a\r
+fast-forward update, gfi will skip updating that ref and instead\r
+prints a warning message. gfi will always attempt to update all\r
+branch refs, and does not stop on the first failure.</p>\r
+<p>Branch updates can be forced with <tt>--force</tt>, but its recommended that\r
+this only be used on an otherwise quiet repository. Using <tt>--force</tt>\r
+is not necessary for an initial import into an empty repository.</p>\r
+</div>\r
+<h2>Technical Discussion</h2>\r
+<div class="sectionbody">\r
+<p>gfi tracks a set of branches in memory. Any branch can be created\r
+or modified at any point during the import process by sending a\r
+<tt>commit</tt> command on the input stream. This design allows a frontend\r
+program to process an unlimited number of branches simultaneously,\r
+generating commits in the order they are available from the source\r
+data. It also simplifies the frontend programs considerably.</p>\r
+<p>gfi does not use or alter the current working directory, or any\r
+file within it. (It does however update the current Git repository,\r
+as referenced by <tt>GIT_DIR</tt>.) Therefore an import frontend may use\r
+the working directory for its own purposes, such as extracting file\r
+revisions from the foreign source. This ignorance of the working\r
+directory also allows gfi to run very quickly, as it does not\r
+need to perform any costly file update operations when switching\r
+between branches.</p>\r
+</div>\r
+<h2>Input Format</h2>\r
+<div class="sectionbody">\r
+<p>With the exception of raw file data (which Git does not interpret)\r
+the gfi input format is text (ASCII) based. This text based\r
+format simplifies development and debugging of frontend programs,\r
+especially when a higher level language such as Perl, Python or\r
+Ruby is being used.</p>\r
+<p>gfi is very strict about its input. Where we say SP below we mean\r
+<strong>exactly</strong> one space. Likewise LF means one (and only one) linefeed.\r
+Supplying additional whitespace characters will cause unexpected\r
+results, such as branch names or file names with leading or trailing\r
+spaces in their name, or early termination of gfi when it encounters\r
+unexpected input.</p>\r
+<h3>Date Formats</h3>\r
+<p>The following date formats are supported. A frontend should select\r
+the format it will use for this import by passing the format name\r
+in the <tt>--date-format=<fmt></tt> command line option.</p>\r
+<dl>\r
+<dt>\r
+<tt>raw</tt>\r
+</dt>\r
+<dd>\r
+<p>\r
+ This is the Git native format and is <tt><time> SP <tz></tt>.\r
+ It is also gfi's default format, if <tt>--date-format</tt> was\r
+ not specified.\r
+</p>\r
+<p>The time of the event is specified by <tt><time></tt> as the number of\r
+seconds since the UNIX epoch (midnight, Jan 1, 1970, UTC) and is\r
+written as an ASCII decimal integer.</p>\r
+<p>The timezone is specified by <tt><tz></tt> as a positive or negative offset\r
+from UTC. For example EST (which is typically 5 hours behind GMT)\r
+would be expressed in <tt><tz></tt> by “-0500” while GMT is “+0000”.</p>\r
+<p>If the timezone is not available in the source material, use\r
+“+0000”, or the most common local timezone. For example many\r
+organizations have a CVS repository which has only ever been accessed\r
+by users who are located in the same location and timezone. In this\r
+case the user's timezone can be easily assumed.</p>\r
+<p>Unlike the <tt>rfc2822</tt> format, this format is very strict. Any\r
+variation in formatting will cause gfi to reject the value.</p>\r
+</dd>\r
+<dt>\r
+<tt>rfc2822</tt>\r
+</dt>\r
+<dd>\r
+<p>\r
+ This is the standard email format as described by RFC 2822.\r
+</p>\r
+<p>An example value is “Tue Feb 6 11:22:18 2007 -0500”. The Git\r
+parser is accurate, but a little on the lenient side. Its the\r
+same parser used by <a href="git-am.html">git-am(1)</a> when applying patches\r
+received from email.</p>\r
+<p>Some malformed strings may be accepted as valid dates. In some of\r
+these cases Git will still be able to obtain the correct date from\r
+the malformed string. There are also some types of malformed\r
+strings which Git will parse wrong, and yet consider valid.\r
+Seriously malformed strings will be rejected.</p>\r
+<p>If the source material is formatted in RFC 2822 style dates,\r
+the frontend should let gfi handle the parsing and conversion\r
+(rather than attempting to do it itself) as the Git parser has\r
+been well tested in the wild.</p>\r
+<p>Frontends should prefer the <tt>raw</tt> format if the source material\r
+is already in UNIX-epoch format, or is easily convertible to\r
+that format, as there is no ambiguity in parsing.</p>\r
+</dd>\r
+<dt>\r
+<tt>now</tt>\r
+</dt>\r
+<dd>\r
+<p>\r
+ Always use the current time and timezone. The literal\r
+ <tt>now</tt> must always be supplied for <tt><when></tt>.\r
+</p>\r
+<p>This is a toy format. The current time and timezone of this system\r
+is always copied into the identity string at the time it is being\r
+created by gfi. There is no way to specify a different time or\r
+timezone.</p>\r
+<p>This particular format is supplied as its short to implement and\r
+may be useful to a process that wants to create a new commit\r
+right now, without needing to use a working directory or\r
+<a href="git-update-index.html">git-update-index(1)</a>.</p>\r
+<p>If separate <tt>author</tt> and <tt>committer</tt> commands are used in a <tt>commit</tt>\r
+the timestamps may not match, as the system clock will be polled\r
+twice (once for each command). The only way to ensure that both\r
+author and committer identity information has the same timestamp\r
+is to omit <tt>author</tt> (thus copying from <tt>committer</tt>) or to use a\r
+date format other than <tt>now</tt>.</p>\r
+</dd>\r
+</dl>\r
+<h3>Commands</h3>\r
+<p>gfi accepts several commands to update the current repository\r
+and control the current import process. More detailed discussion\r
+(with examples) of each command follows later.</p>\r
+<dl>\r
+<dt>\r
+<tt>commit</tt>\r
+</dt>\r
+<dd>\r
+<p>\r
+ Creates a new branch or updates an existing branch by\r
+ creating a new commit and updating the branch to point at\r
+ the newly created commit.\r
+</p>\r
+</dd>\r
+<dt>\r
+<tt>tag</tt>\r
+</dt>\r
+<dd>\r
+<p>\r
+ Creates an annotated tag object from an existing commit or\r
+ branch. Lightweight tags are not supported by this command,\r
+ as they are not recommended for recording meaningful points\r
+ in time.\r
+</p>\r
+</dd>\r
+<dt>\r
+<tt>reset</tt>\r
+</dt>\r
+<dd>\r
+<p>\r
+ Reset an existing branch (or a new branch) to a specific\r
+ revision. This command must be used to change a branch to\r
+ a specific revision without making a commit on it.\r
+</p>\r
+</dd>\r
+<dt>\r
+<tt>blob</tt>\r
+</dt>\r
+<dd>\r
+<p>\r
+ Convert raw file data into a blob, for future use in a\r
+ <tt>commit</tt> command. This command is optional and is not\r
+ needed to perform an import.\r
+</p>\r
+</dd>\r
+<dt>\r
+<tt>checkpoint</tt>\r
+</dt>\r
+<dd>\r
+<p>\r
+ Forces gfi to close the current packfile, generate its\r
+ unique SHA-1 checksum and index, and start a new packfile.\r
+ This command is optional and is not needed to perform\r
+ an import.\r
+</p>\r
+</dd>\r
+</dl>\r
+<h3><tt>commit</tt></h3>\r
+<p>Create or update a branch with a new commit, recording one logical\r
+change to the project.</p>\r
+<div class="literalblock">\r
+<div class="content">\r
+<pre><tt> 'commit' SP <ref> LF\r
+ mark?\r
+ ('author' SP <name> SP LT <email> GT SP <when> LF)?\r
+ 'committer' SP <name> SP LT <email> GT SP <when> LF\r
+ data\r
+ ('from' SP <committish> LF)?\r
+ ('merge' SP <committish> LF)?\r
+ (filemodify | filedelete)*\r
+ LF</tt></pre>\r
+</div></div>\r
+<p>where <tt><ref></tt> is the name of the branch to make the commit on.\r
+Typically branch names are prefixed with <tt>refs/heads/</tt> in\r
+Git, so importing the CVS branch symbol <tt>RELENG-1_0</tt> would use\r
+<tt>refs/heads/RELENG-1_0</tt> for the value of <tt><ref></tt>. The value of\r
+<tt><ref></tt> must be a valid refname in Git. As <tt>LF</tt> is not valid in\r
+a Git refname, no quoting or escaping syntax is supported here.</p>\r
+<p>A <tt>mark</tt> command may optionally appear, requesting gfi to save a\r
+reference to the newly created commit for future use by the frontend\r
+(see below for format). It is very common for frontends to mark\r
+every commit they create, thereby allowing future branch creation\r
+from any imported commit.</p>\r
+<p>The <tt>data</tt> command following <tt>committer</tt> must supply the commit\r
+message (see below for <tt>data</tt> command syntax). To import an empty\r
+commit message use a 0 length data. Commit messages are free-form\r
+and are not interpreted by Git. Currently they must be encoded in\r
+UTF-8, as gfi does not permit other encodings to be specified.</p>\r
+<p>Zero or more <tt>filemodify</tt> and <tt>filedelete</tt> commands may be\r
+included to update the contents of the branch prior to the commit.\r
+These commands can be supplied in any order, gfi is not sensitive\r
+to pathname or operation ordering.</p>\r
+<h4><tt>author</tt></h4>\r
+<p>An <tt>author</tt> command may optionally appear, if the author information\r
+might differ from the committer information. If <tt>author</tt> is omitted\r
+then gfi will automatically use the committer's information for\r
+the author portion of the commit. See below for a description of\r
+the fields in <tt>author</tt>, as they are identical to <tt>committer</tt>.</p>\r
+<h4><tt>committer</tt></h4>\r
+<p>The <tt>committer</tt> command indicates who made this commit, and when\r
+they made it.</p>\r
+<p>Here <tt><name></tt> is the person's display name (for example\r
+“Com M Itter”) and <tt><email></tt> is the person's email address\r
+(“cm@example.com”). <tt>LT</tt> and <tt>GT</tt> are the literal less-than (\x3c)\r
+and greater-than (\x3e) symbols. These are required to delimit\r
+the email address from the other fields in the line. Note that\r
+<tt><name></tt> is free-form and may contain any sequence of bytes, except\r
+<tt>LT</tt> and <tt>LF</tt>. It is typically UTF-8 encoded.</p>\r
+<p>The time of the change is specified by <tt><when></tt> using the date format\r
+that was selected by the <tt>--date-format=<fmt></tt> command line option.\r
+See “Date Formats” above for the set of supported formats, and\r
+their syntax.</p>\r
+<h4><tt>from</tt></h4>\r
+<p>Only valid for the first commit made on this branch by this\r
+gfi process. The <tt>from</tt> command is used to specify the commit\r
+to initialize this branch from. This revision will be the first\r
+ancestor of the new commit.</p>\r
+<p>Omitting the <tt>from</tt> command in the first commit of a new branch will\r
+cause gfi to create that commit with no ancestor. This tends to be\r
+desired only for the initial commit of a project. Omitting the\r
+<tt>from</tt> command on existing branches is required, as the current\r
+commit on that branch is automatically assumed to be the first\r
+ancestor of the new commit.</p>\r
+<p>As <tt>LF</tt> is not valid in a Git refname or SHA-1 expression, no\r
+quoting or escaping syntax is supported within <tt><committish></tt>.</p>\r
+<p>Here <tt><committish></tt> is any of the following:</p>\r
+<ul>\r
+<li>\r
+<p>\r
+The name of an existing branch already in gfi's internal branch\r
+ table. If gfi doesn't know the name, its treated as a SHA-1\r
+ expression.\r
+</p>\r
+</li>\r
+<li>\r
+<p>\r
+A mark reference, <tt>:<idnum></tt>, where <tt><idnum></tt> is the mark number.\r
+</p>\r
+<p>The reason gfi uses <tt>:</tt> to denote a mark reference is this character\r
+is not legal in a Git branch name. The leading <tt>:</tt> makes it easy\r
+to distingush between the mark 42 (<tt>:42</tt>) and the branch 42 (<tt>42</tt>\r
+or <tt>refs/heads/42</tt>), or an abbreviated SHA-1 which happened to\r
+consist only of base-10 digits.</p>\r
+<p>Marks must be declared (via <tt>mark</tt>) before they can be used.</p>\r
+</li>\r
+<li>\r
+<p>\r
+A complete 40 byte or abbreviated commit SHA-1 in hex.\r
+</p>\r
+</li>\r
+<li>\r
+<p>\r
+Any valid Git SHA-1 expression that resolves to a commit. See\r
+ “SPECIFYING REVISIONS” in <a href="git-rev-parse.html">git-rev-parse(1)</a> for details.\r
+</p>\r
+</li>\r
+</ul>\r
+<p>The special case of restarting an incremental import from the\r
+current branch value should be written as:</p>\r
+<div class="listingblock">\r
+<div class="content">\r
+<pre><tt> from refs/heads/branch^0</tt></pre>\r
+</div></div>\r
+<p>The <tt>^0</tt> suffix is necessary as gfi does not permit a branch to\r
+start from itself, and the branch is created in memory before the\r
+<tt>from</tt> command is even read from the input. Adding <tt>^0</tt> will force\r
+gfi to resolve the commit through Git's revision parsing library,\r
+rather than its internal branch table, thereby loading in the\r
+existing value of the branch.</p>\r
+<h4><tt>merge</tt></h4>\r
+<p>Includes one additional ancestor commit, and makes the current\r
+commit a merge commit. An unlimited number of <tt>merge</tt> commands per\r
+commit are permitted by gfi, thereby establishing an n-way merge.\r
+However Git's other tools never create commits with more than 15\r
+additional ancestors (forming a 16-way merge). For this reason\r
+it is suggested that frontends do not use more than 15 <tt>merge</tt>\r
+commands per commit.</p>\r
+<p>Here <tt><committish></tt> is any of the commit specification expressions\r
+also accepted by <tt>from</tt> (see above).</p>\r
+<h4><tt>filemodify</tt></h4>\r
+<p>Included in a <tt>commit</tt> command to add a new file or change the\r
+content of an existing file. This command has two different means\r
+of specifying the content of the file.</p>\r
+<dl>\r
+<dt>\r
+External data format\r
+</dt>\r
+<dd>\r
+<p>\r
+ The data content for the file was already supplied by a prior\r
+ <tt>blob</tt> command. The frontend just needs to connect it.\r
+</p>\r
+<div class="literalblock">\r
+<div class="content">\r
+<pre><tt> 'M' SP <mode> SP <dataref> SP <path> LF</tt></pre>\r
+</div></div>\r
+<p>Here <tt><dataref></tt> can be either a mark reference (<tt>:<idnum></tt>)\r
+set by a prior <tt>blob</tt> command, or a full 40-byte SHA-1 of an\r
+existing Git blob object.</p>\r
+</dd>\r
+<dt>\r
+Inline data format\r
+</dt>\r
+<dd>\r
+<p>\r
+ The data content for the file has not been supplied yet.\r
+ The frontend wants to supply it as part of this modify\r
+ command.\r
+</p>\r
+<div class="literalblock">\r
+<div class="content">\r
+<pre><tt> 'M' SP <mode> SP 'inline' SP <path> LF\r
+ data</tt></pre>\r
+</div></div>\r
+<p>See below for a detailed description of the <tt>data</tt> command.</p>\r
+</dd>\r
+</dl>\r
+<p>In both formats <tt><mode></tt> is the type of file entry, specified\r
+in octal. Git only supports the following modes:</p>\r
+<ul>\r
+<li>\r
+<p>\r
+<tt>100644</tt> or <tt>644</tt>: A normal (not-executable) file. The majority\r
+ of files in most projects use this mode. If in doubt, this is\r
+ what you want.\r
+</p>\r
+</li>\r
+<li>\r
+<p>\r
+<tt>100755</tt> or <tt>755</tt>: A normal, but executable, file.\r
+</p>\r
+</li>\r
+<li>\r
+<p>\r
+<tt>120000</tt>: A symlink, the content of the file will be the link target.\r
+</p>\r
+</li>\r
+</ul>\r
+<p>In both formats <tt><path></tt> is the complete path of the file to be added\r
+(if not already existing) or modified (if already existing).</p>\r
+<p>A <tt><path></tt> string must use UNIX-style directory seperators (forward\r
+slash <tt>/</tt>), may contain any byte other than <tt>LF</tt>, and must not\r
+start with double quote (<tt>"</tt>).</p>\r
+<p>If an <tt>LF</tt> or double quote must be encoded into <tt><path></tt> shell-style\r
+quoting should be used, e.g. <tt>"path/with\n and \" in it"</tt>.</p>\r
+<p>The value of <tt><path></tt> must be in canoncial form. That is it must not:</p>\r
+<ul>\r
+<li>\r
+<p>\r
+contain an empty directory component (e.g. <tt>foo//bar</tt> is invalid),\r
+</p>\r
+</li>\r
+<li>\r
+<p>\r
+end with a directory seperator (e.g. <tt>foo/</tt> is invalid),\r
+</p>\r
+</li>\r
+<li>\r
+<p>\r
+start with a directory seperator (e.g. <tt>/foo</tt> is invalid),\r
+</p>\r
+</li>\r
+<li>\r
+<p>\r
+contain the special component <tt>.</tt> or <tt>..</tt> (e.g. <tt>foo/./bar</tt> and\r
+ <tt>foo/../bar</tt> are invalid).\r
+</p>\r
+</li>\r
+</ul>\r
+<p>It is recommended that <tt><path></tt> always be encoded using UTF-8.</p>\r
+<h4><tt>filedelete</tt></h4>\r
+<p>Included in a <tt>commit</tt> command to remove a file from the branch.\r
+If the file removal makes its directory empty, the directory will\r
+be automatically removed too. This cascades up the tree until the\r
+first non-empty directory or the root is reached.</p>\r
+<div class="literalblock">\r
+<div class="content">\r
+<pre><tt> 'D' SP <path> LF</tt></pre>\r
+</div></div>\r
+<p>here <tt><path></tt> is the complete path of the file to be removed.\r
+See <tt>filemodify</tt> above for a detailed description of <tt><path></tt>.</p>\r
+<h3><tt>mark</tt></h3>\r
+<p>Arranges for gfi to save a reference to the current object, allowing\r
+the frontend to recall this object at a future point in time, without\r
+knowing its SHA-1. Here the current object is the object creation\r
+command the <tt>mark</tt> command appears within. This can be <tt>commit</tt>,\r
+<tt>tag</tt>, and <tt>blob</tt>, but <tt>commit</tt> is the most common usage.</p>\r
+<div class="literalblock">\r
+<div class="content">\r
+<pre><tt> 'mark' SP ':' <idnum> LF</tt></pre>\r
+</div></div>\r
+<p>where <tt><idnum></tt> is the number assigned by the frontend to this mark.\r
+The value of <tt><idnum></tt> is expressed as an ASCII decimal integer.\r
+The value 0 is reserved and cannot be used as\r
+a mark. Only values greater than or equal to 1 may be used as marks.</p>\r
+<p>New marks are created automatically. Existing marks can be moved\r
+to another object simply by reusing the same <tt><idnum></tt> in another\r
+<tt>mark</tt> command.</p>\r
+<h3><tt>tag</tt></h3>\r
+<p>Creates an annotated tag referring to a specific commit. To create\r
+lightweight (non-annotated) tags see the <tt>reset</tt> command below.</p>\r
+<div class="literalblock">\r
+<div class="content">\r
+<pre><tt> 'tag' SP <name> LF\r
+ 'from' SP <committish> LF\r
+ 'tagger' SP <name> SP LT <email> GT SP <when> LF\r
+ data\r
+ LF</tt></pre>\r
+</div></div>\r
+<p>where <tt><name></tt> is the name of the tag to create.</p>\r
+<p>Tag names are automatically prefixed with <tt>refs/tags/</tt> when stored\r
+in Git, so importing the CVS branch symbol <tt>RELENG-1_0-FINAL</tt> would\r
+use just <tt>RELENG-1_0-FINAL</tt> for <tt><name></tt>, and gfi will write the\r
+corresponding ref as <tt>refs/tags/RELENG-1_0-FINAL</tt>.</p>\r
+<p>The value of <tt><name></tt> must be a valid refname in Git and therefore\r
+may contain forward slashes. As <tt>LF</tt> is not valid in a Git refname,\r
+no quoting or escaping syntax is supported here.</p>\r
+<p>The <tt>from</tt> command is the same as in the <tt>commit</tt> command; see\r
+above for details.</p>\r
+<p>The <tt>tagger</tt> command uses the same format as <tt>committer</tt> within\r
+<tt>commit</tt>; again see above for details.</p>\r
+<p>The <tt>data</tt> command following <tt>tagger</tt> must supply the annotated tag\r
+message (see below for <tt>data</tt> command syntax). To import an empty\r
+tag message use a 0 length data. Tag messages are free-form and are\r
+not interpreted by Git. Currently they must be encoded in UTF-8,\r
+as gfi does not permit other encodings to be specified.</p>\r
+<p>Signing annotated tags during import from within gfi is not\r
+supported. Trying to include your own PGP/GPG signature is not\r
+recommended, as the frontend does not (easily) have access to the\r
+complete set of bytes which normally goes into such a signature.\r
+If signing is required, create lightweight tags from within gfi with\r
+<tt>reset</tt>, then create the annotated versions of those tags offline\r
+with the standard <a href="git-tag.html">git-tag(1)</a> process.</p>\r
+<h3><tt>reset</tt></h3>\r
+<p>Creates (or recreates) the named branch, optionally starting from\r
+a specific revision. The reset command allows a frontend to issue\r
+a new <tt>from</tt> command for an existing branch, or to create a new\r
+branch from an existing commit without creating a new commit.</p>\r
+<div class="literalblock">\r
+<div class="content">\r
+<pre><tt> 'reset' SP <ref> LF\r
+ ('from' SP <committish> LF)?\r
+ LF</tt></pre>\r
+</div></div>\r
+<p>For a detailed description of <tt><ref></tt> and <tt><committish></tt> see above\r
+under <tt>commit</tt> and <tt>from</tt>.</p>\r
+<p>The <tt>reset</tt> command can also be used to create lightweight\r
+(non-annotated) tags. For example:</p>\r
+<div class="exampleblock">\r
+<div class="exampleblock-content">\r
+<div class="literalblock">\r
+<div class="content">\r
+<pre><tt>reset refs/tags/938\r
+from :938</tt></pre>\r
+</div></div>\r
+</div></div>\r
+<p>would create the lightweight tag <tt>refs/tags/938</tt> referring to\r
+whatever commit mark <tt>:938</tt> references.</p>\r
+<h3><tt>blob</tt></h3>\r
+<p>Requests writing one file revision to the packfile. The revision\r
+is not connected to any commit; this connection must be formed in\r
+a subsequent <tt>commit</tt> command by referencing the blob through an\r
+assigned mark.</p>\r
+<div class="literalblock">\r
+<div class="content">\r
+<pre><tt> 'blob' LF\r
+ mark?\r
+ data</tt></pre>\r
+</div></div>\r
+<p>The mark command is optional here as some frontends have chosen\r
+to generate the Git SHA-1 for the blob on their own, and feed that\r
+directly to <tt>commit</tt>. This is typically more work than its worth\r
+however, as marks are inexpensive to store and easy to use.</p>\r
+<h3><tt>data</tt></h3>\r
+<p>Supplies raw data (for use as blob/file content, commit messages, or\r
+annotated tag messages) to gfi. Data can be supplied using an exact\r
+byte count or delimited with a terminating line. Real frontends\r
+intended for production-quality conversions should always use the\r
+exact byte count format, as it is more robust and performs better.\r
+The delimited format is intended primarily for testing gfi.</p>\r
+<dl>\r
+<dt>\r
+Exact byte count format\r
+</dt>\r
+<dd>\r
+<p>\r
+ The frontend must specify the number of bytes of data.\r
+</p>\r
+<div class="literalblock">\r
+<div class="content">\r
+<pre><tt> 'data' SP <count> LF\r
+ <raw> LF</tt></pre>\r
+</div></div>\r
+<p>where <tt><count></tt> is the exact number of bytes appearing within\r
+<tt><raw></tt>. The value of <tt><count></tt> is expressed as an ASCII decimal\r
+integer. The <tt>LF</tt> on either side of <tt><raw></tt> is not\r
+included in <tt><count></tt> and will not be included in the imported data.</p>\r
+</dd>\r
+<dt>\r
+Delimited format\r
+</dt>\r
+<dd>\r
+<p>\r
+ A delimiter string is used to mark the end of the data.\r
+ gfi will compute the length by searching for the delimiter.\r
+ This format is primarly useful for testing and is not\r
+ recommended for real data.\r
+</p>\r
+<div class="literalblock">\r
+<div class="content">\r
+<pre><tt> 'data' SP '<<' <delim> LF\r
+ <raw> LF\r
+ <delim> LF</tt></pre>\r
+</div></div>\r
+<p>where <tt><delim></tt> is the chosen delimiter string. The string <tt><delim></tt>\r
+must not appear on a line by itself within <tt><raw></tt>, as otherwise\r
+gfi will think the data ends earlier than it really does. The <tt>LF</tt>\r
+immediately trailing <tt><raw></tt> is part of <tt><raw></tt>. This is one of\r
+the limitations of the delimited format, it is impossible to supply\r
+a data chunk which does not have an LF as its last byte.</p>\r
+</dd>\r
+</dl>\r
+<h3><tt>checkpoint</tt></h3>\r
+<p>Forces gfi to close the current packfile and start a new one.\r
+As this requires a significant amount of CPU time and disk IO\r
+(to compute the overall pack SHA-1 checksum and generate the\r
+corresponding index file) it can easily take several minutes for\r
+a single <tt>checkpoint</tt> command to complete.</p>\r
+<div class="literalblock">\r
+<div class="content">\r
+<pre><tt> 'checkpoint' LF\r
+ LF</tt></pre>\r
+</div></div>\r
+</div>\r
+<h2>Packfile Optimization</h2>\r
+<div class="sectionbody">\r
+<p>When packing a blob gfi always attempts to deltify against the last\r
+blob written. Unless specifically arranged for by the frontend,\r
+this will probably not be a prior version of the same file, so the\r
+generated delta will not be the smallest possible. The resulting\r
+packfile will be compressed, but will not be optimal.</p>\r
+<p>Frontends which have efficient access to all revisions of a\r
+single file (for example reading an RCS/CVS ,v file) can choose\r
+to supply all revisions of that file as a sequence of consecutive\r
+<tt>blob</tt> commands. This allows gfi to deltify the different file\r
+revisions against each other, saving space in the final packfile.\r
+Marks can be used to later identify individual file revisions during\r
+a sequence of <tt>commit</tt> commands.</p>\r
+<p>The packfile(s) created by gfi do not encourage good disk access\r
+patterns. This is caused by gfi writing the data in the order\r
+it is received on standard input, while Git typically organizes\r
+data within packfiles to make the most recent (current tip) data\r
+appear before historical data. Git also clusters commits together,\r
+speeding up revision traversal through better cache locality.</p>\r
+<p>For this reason it is strongly recommended that users repack the\r
+repository with <tt>git repack -a -d</tt> after gfi completes, allowing\r
+Git to reorganize the packfiles for faster data access. If blob\r
+deltas are suboptimal (see above) then also adding the <tt>-f</tt> option\r
+to force recomputation of all deltas can significantly reduce the\r
+final packfile size (30-50% smaller can be quite typical).</p>\r
+</div>\r
+<h2>Memory Utilization</h2>\r
+<div class="sectionbody">\r
+<p>There are a number of factors which affect how much memory gfi\r
+requires to perform an import. Like critical sections of core\r
+Git, gfi uses its own memory allocators to ammortize any overheads\r
+associated with malloc. In practice gfi tends to ammoritize any\r
+malloc overheads to 0, due to its use of large block allocations.</p>\r
+<h3>per object</h3>\r
+<p>gfi maintains an in-memory structure for every object written in\r
+this execution. On a 32 bit system the structure is 32 bytes,\r
+on a 64 bit system the structure is 40 bytes (due to the larger\r
+pointer sizes). Objects in the table are not deallocated until\r
+gfi terminates. Importing 2 million objects on a 32 bit system\r
+will require approximately 64 MiB of memory.</p>\r
+<p>The object table is actually a hashtable keyed on the object name\r
+(the unique SHA-1). This storage configuration allows gfi to reuse\r
+an existing or already written object and avoid writing duplicates\r
+to the output packfile. Duplicate blobs are surprisingly common\r
+in an import, typically due to branch merges in the source.</p>\r
+<h3>per mark</h3>\r
+<p>Marks are stored in a sparse array, using 1 pointer (4 bytes or 8\r
+bytes, depending on pointer size) per mark. Although the array\r
+is sparse, frontends are still strongly encouraged to use marks\r
+between 1 and n, where n is the total number of marks required for\r
+this import.</p>\r
+<h3>per branch</h3>\r
+<p>Branches are classified as active and inactive. The memory usage\r
+of the two classes is significantly different.</p>\r
+<p>Inactive branches are stored in a structure which uses 96 or 120\r
+bytes (32 bit or 64 bit systems, respectively), plus the length of\r
+the branch name (typically under 200 bytes), per branch. gfi will\r
+easily handle as many as 10,000 inactive branches in under 2 MiB\r
+of memory.</p>\r
+<p>Active branches have the same overhead as inactive branches, but\r
+also contain copies of every tree that has been recently modified on\r
+that branch. If subtree <tt>include</tt> has not been modified since the\r
+branch became active, its contents will not be loaded into memory,\r
+but if subtree <tt>src</tt> has been modified by a commit since the branch\r
+became active, then its contents will be loaded in memory.</p>\r
+<p>As active branches store metadata about the files contained on that\r
+branch, their in-memory storage size can grow to a considerable size\r
+(see below).</p>\r
+<p>gfi automatically moves active branches to inactive status based on\r
+a simple least-recently-used algorithm. The LRU chain is updated on\r
+each <tt>commit</tt> command. The maximum number of active branches can be\r
+increased or decreased on the command line with <tt>--active-branches=</tt>.</p>\r
+<h3>per active tree</h3>\r
+<p>Trees (aka directories) use just 12 bytes of memory on top of the\r
+memory required for their entries (see “per active file” below).\r
+The cost of a tree is virtually 0, as its overhead ammortizes out\r
+over the individual file entries.</p>\r
+<h3>per active file entry</h3>\r
+<p>Files (and pointers to subtrees) within active trees require 52 or 64\r
+bytes (32/64 bit platforms) per entry. To conserve space, file and\r
+tree names are pooled in a common string table, allowing the filename\r
+“Makefile” to use just 16 bytes (after including the string header\r
+overhead) no matter how many times it occurs within the project.</p>\r
+<p>The active branch LRU, when coupled with the filename string pool\r
+and lazy loading of subtrees, allows gfi to efficiently import\r
+projects with 2,000+ branches and 45,114+ files in a very limited\r
+memory footprint (less than 2.7 MiB per active branch).</p>\r
+</div>\r
+<h2>Author</h2>\r
+<div class="sectionbody">\r
+<p>Written by Shawn O. Pearce <spearce@spearce.org>.</p>\r
+</div>\r
+<h2>Documentation</h2>\r
+<div class="sectionbody">\r
+<p>Documentation by Shawn O. Pearce <spearce@spearce.org>.</p>\r
+</div>\r
+<h2>GIT</h2>\r
+<div class="sectionbody">\r
+<p>Part of the <a href="git.html">git(7)</a> suite</p>\r
+</div>\r
+<div id="footer">\r
+<div id="footer-text">\r
+Last updated 07-Feb-2007 05:52:25 UTC\r
+</div>\r
+</div>\r
+</body>\r
+</html>\r
--- /dev/null
+git-fast-import(1)
+==================
+
+NAME
+----
+git-fast-import - Backend for fast Git data importers.
+
+
+SYNOPSIS
+--------
+frontend | 'git-fast-import' [options]
+
+DESCRIPTION
+-----------
+This program is usually not what the end user wants to run directly.
+Most end users want to use one of the existing frontend programs,
+which parses a specific type of foreign source and feeds the contents
+stored there to git-fast-import (gfi).
+
+gfi reads a mixed command/data stream from standard input and
+writes one or more packfiles directly into the current repository.
+When EOF is received on standard input, fast import writes out
+updated branch and tag refs, fully updating the current repository
+with the newly imported data.
+
+The gfi backend itself can import into an empty repository (one that
+has already been initialized by gitlink:git-init[1]) or incrementally
+update an existing populated repository. Whether or not incremental
+imports are supported from a particular foreign source depends on
+the frontend program in use.
+
+
+OPTIONS
+-------
+--date-format=<fmt>::
+ Specify the type of dates the frontend will supply to
+ gfi within `author`, `committer` and `tagger` commands.
+ See ``Date Formats'' below for details about which formats
+ are supported, and their syntax.
+
+--force::
+ Force updating modified existing branches, even if doing
+ so would cause commits to be lost (as the new commit does
+ not contain the old commit).
+
+--max-pack-size=<n>::
+ Maximum size of each output packfile, expressed in MiB.
+ The default is 4096 (4 GiB) as that is the maximum allowed
+ packfile size (due to file format limitations). Some
+ importers may wish to lower this, such as to ensure the
+ resulting packfiles fit on CDs.
+
+--depth=<n>::
+ Maximum delta depth, for blob and tree deltification.
+ Default is 10.
+
+--active-branches=<n>::
+ Maximum number of branches to maintain active at once.
+ See ``Memory Utilization'' below for details. Default is 5.
+
+--export-marks=<file>::
+ Dumps the internal marks table to <file> when complete.
+ Marks are written one per line as `:markid SHA-1`.
+ Frontends can use this file to validate imports after they
+ have been completed.
+
+Performance
+-----------
+The design of gfi allows it to import large projects in a minimum
+amount of memory usage and processing time. Assuming the frontend
+is able to keep up with gfi and feed it a constant stream of data,
+import times for projects holding 10+ years of history and containing
+100,000+ individual commits are generally completed in just 1-2
+hours on quite modest (~$2,000 USD) hardware.
+
+Most bottlenecks appear to be in foreign source data access (the
+source just cannot extract revisions fast enough) or disk IO (gfi
+writes as fast as the disk will take the data). Imports will run
+faster if the source data is stored on a different drive than the
+destination Git repository (due to less IO contention).
+
+
+Development Cost
+----------------
+A typical frontend for gfi tends to weigh in at approximately 200
+lines of Perl/Python/Ruby code. Most developers have been able to
+create working importers in just a couple of hours, even though it
+is their first exposure to gfi, and sometimes even to Git. This is
+an ideal situation, given that most conversion tools are throw-away
+(use once, and never look back).
+
+
+Parallel Operation
+------------------
+Like `git-push` or `git-fetch`, imports handled by gfi are safe to
+run alongside parallel `git repack -a -d` or `git gc` invocations,
+or any other Git operation (including `git prune`, as loose objects
+are never used by gfi).
+
+gfi does not lock the branch or tag refs it is actively importing.
+After the import, during its ref update phase, gfi tests each
+existing branch ref to verify the update will be a fast-forward
+update (the commit stored in the ref is contained in the new
+history of the commit to be written). If the update is not a
+fast-forward update, gfi will skip updating that ref and instead
+prints a warning message. gfi will always attempt to update all
+branch refs, and does not stop on the first failure.
+
+Branch updates can be forced with `--force`, but its recommended that
+this only be used on an otherwise quiet repository. Using `--force`
+is not necessary for an initial import into an empty repository.
+
+
+Technical Discussion
+--------------------
+gfi tracks a set of branches in memory. Any branch can be created
+or modified at any point during the import process by sending a
+`commit` command on the input stream. This design allows a frontend
+program to process an unlimited number of branches simultaneously,
+generating commits in the order they are available from the source
+data. It also simplifies the frontend programs considerably.
+
+gfi does not use or alter the current working directory, or any
+file within it. (It does however update the current Git repository,
+as referenced by `GIT_DIR`.) Therefore an import frontend may use
+the working directory for its own purposes, such as extracting file
+revisions from the foreign source. This ignorance of the working
+directory also allows gfi to run very quickly, as it does not
+need to perform any costly file update operations when switching
+between branches.
+
+Input Format
+------------
+With the exception of raw file data (which Git does not interpret)
+the gfi input format is text (ASCII) based. This text based
+format simplifies development and debugging of frontend programs,
+especially when a higher level language such as Perl, Python or
+Ruby is being used.
+
+gfi is very strict about its input. Where we say SP below we mean
+*exactly* one space. Likewise LF means one (and only one) linefeed.
+Supplying additional whitespace characters will cause unexpected
+results, such as branch names or file names with leading or trailing
+spaces in their name, or early termination of gfi when it encounters
+unexpected input.
+
+Date Formats
+~~~~~~~~~~~~
+The following date formats are supported. A frontend should select
+the format it will use for this import by passing the format name
+in the `--date-format=<fmt>` command line option.
+
+`raw`::
+ This is the Git native format and is `<time> SP <tz>`.
+ It is also gfi's default format, if `--date-format` was
+ not specified.
++
+The time of the event is specified by `<time>` as the number of
+seconds since the UNIX epoch (midnight, Jan 1, 1970, UTC) and is
+written as an ASCII decimal integer.
++
+The timezone is specified by `<tz>` as a positive or negative offset
+from UTC. For example EST (which is typically 5 hours behind GMT)
+would be expressed in `<tz>` by ``-0500'' while GMT is ``+0000''.
++
+If the timezone is not available in the source material, use
+``+0000'', or the most common local timezone. For example many
+organizations have a CVS repository which has only ever been accessed
+by users who are located in the same location and timezone. In this
+case the user's timezone can be easily assumed.
++
+Unlike the `rfc2822` format, this format is very strict. Any
+variation in formatting will cause gfi to reject the value.
+
+`rfc2822`::
+ This is the standard email format as described by RFC 2822.
++
+An example value is ``Tue Feb 6 11:22:18 2007 -0500''. The Git
+parser is accurate, but a little on the lenient side. Its the
+same parser used by gitlink:git-am[1] when applying patches
+received from email.
++
+Some malformed strings may be accepted as valid dates. In some of
+these cases Git will still be able to obtain the correct date from
+the malformed string. There are also some types of malformed
+strings which Git will parse wrong, and yet consider valid.
+Seriously malformed strings will be rejected.
++
+If the source material is formatted in RFC 2822 style dates,
+the frontend should let gfi handle the parsing and conversion
+(rather than attempting to do it itself) as the Git parser has
+been well tested in the wild.
++
+Frontends should prefer the `raw` format if the source material
+is already in UNIX-epoch format, or is easily convertible to
+that format, as there is no ambiguity in parsing.
+
+`now`::
+ Always use the current time and timezone. The literal
+ `now` must always be supplied for `<when>`.
++
+This is a toy format. The current time and timezone of this system
+is always copied into the identity string at the time it is being
+created by gfi. There is no way to specify a different time or
+timezone.
++
+This particular format is supplied as its short to implement and
+may be useful to a process that wants to create a new commit
+right now, without needing to use a working directory or
+gitlink:git-update-index[1].
++
+If separate `author` and `committer` commands are used in a `commit`
+the timestamps may not match, as the system clock will be polled
+twice (once for each command). The only way to ensure that both
+author and committer identity information has the same timestamp
+is to omit `author` (thus copying from `committer`) or to use a
+date format other than `now`.
+
+Commands
+~~~~~~~~
+gfi accepts several commands to update the current repository
+and control the current import process. More detailed discussion
+(with examples) of each command follows later.
+
+`commit`::
+ Creates a new branch or updates an existing branch by
+ creating a new commit and updating the branch to point at
+ the newly created commit.
+
+`tag`::
+ Creates an annotated tag object from an existing commit or
+ branch. Lightweight tags are not supported by this command,
+ as they are not recommended for recording meaningful points
+ in time.
+
+`reset`::
+ Reset an existing branch (or a new branch) to a specific
+ revision. This command must be used to change a branch to
+ a specific revision without making a commit on it.
+
+`blob`::
+ Convert raw file data into a blob, for future use in a
+ `commit` command. This command is optional and is not
+ needed to perform an import.
+
+`checkpoint`::
+ Forces gfi to close the current packfile, generate its
+ unique SHA-1 checksum and index, and start a new packfile.
+ This command is optional and is not needed to perform
+ an import.
+
+`commit`
+~~~~~~~~
+Create or update a branch with a new commit, recording one logical
+change to the project.
+
+....
+ 'commit' SP <ref> LF
+ mark?
+ ('author' SP <name> SP LT <email> GT SP <when> LF)?
+ 'committer' SP <name> SP LT <email> GT SP <when> LF
+ data
+ ('from' SP <committish> LF)?
+ ('merge' SP <committish> LF)?
+ (filemodify | filedelete)*
+ LF
+....
+
+where `<ref>` is the name of the branch to make the commit on.
+Typically branch names are prefixed with `refs/heads/` in
+Git, so importing the CVS branch symbol `RELENG-1_0` would use
+`refs/heads/RELENG-1_0` for the value of `<ref>`. The value of
+`<ref>` must be a valid refname in Git. As `LF` is not valid in
+a Git refname, no quoting or escaping syntax is supported here.
+
+A `mark` command may optionally appear, requesting gfi to save a
+reference to the newly created commit for future use by the frontend
+(see below for format). It is very common for frontends to mark
+every commit they create, thereby allowing future branch creation
+from any imported commit.
+
+The `data` command following `committer` must supply the commit
+message (see below for `data` command syntax). To import an empty
+commit message use a 0 length data. Commit messages are free-form
+and are not interpreted by Git. Currently they must be encoded in
+UTF-8, as gfi does not permit other encodings to be specified.
+
+Zero or more `filemodify` and `filedelete` commands may be
+included to update the contents of the branch prior to the commit.
+These commands can be supplied in any order, gfi is not sensitive
+to pathname or operation ordering.
+
+`author`
+^^^^^^^^
+An `author` command may optionally appear, if the author information
+might differ from the committer information. If `author` is omitted
+then gfi will automatically use the committer's information for
+the author portion of the commit. See below for a description of
+the fields in `author`, as they are identical to `committer`.
+
+`committer`
+^^^^^^^^^^^
+The `committer` command indicates who made this commit, and when
+they made it.
+
+Here `<name>` is the person's display name (for example
+``Com M Itter'') and `<email>` is the person's email address
+(``cm@example.com''). `LT` and `GT` are the literal less-than (\x3c)
+and greater-than (\x3e) symbols. These are required to delimit
+the email address from the other fields in the line. Note that
+`<name>` is free-form and may contain any sequence of bytes, except
+`LT` and `LF`. It is typically UTF-8 encoded.
+
+The time of the change is specified by `<when>` using the date format
+that was selected by the `--date-format=<fmt>` command line option.
+See ``Date Formats'' above for the set of supported formats, and
+their syntax.
+
+`from`
+^^^^^^
+Only valid for the first commit made on this branch by this
+gfi process. The `from` command is used to specify the commit
+to initialize this branch from. This revision will be the first
+ancestor of the new commit.
+
+Omitting the `from` command in the first commit of a new branch will
+cause gfi to create that commit with no ancestor. This tends to be
+desired only for the initial commit of a project. Omitting the
+`from` command on existing branches is required, as the current
+commit on that branch is automatically assumed to be the first
+ancestor of the new commit.
+
+As `LF` is not valid in a Git refname or SHA-1 expression, no
+quoting or escaping syntax is supported within `<committish>`.
+
+Here `<committish>` is any of the following:
+
+* The name of an existing branch already in gfi's internal branch
+ table. If gfi doesn't know the name, its treated as a SHA-1
+ expression.
+
+* A mark reference, `:<idnum>`, where `<idnum>` is the mark number.
++
+The reason gfi uses `:` to denote a mark reference is this character
+is not legal in a Git branch name. The leading `:` makes it easy
+to distingush between the mark 42 (`:42`) and the branch 42 (`42`
+or `refs/heads/42`), or an abbreviated SHA-1 which happened to
+consist only of base-10 digits.
++
+Marks must be declared (via `mark`) before they can be used.
+
+* A complete 40 byte or abbreviated commit SHA-1 in hex.
+
+* Any valid Git SHA-1 expression that resolves to a commit. See
+ ``SPECIFYING REVISIONS'' in gitlink:git-rev-parse[1] for details.
+
+The special case of restarting an incremental import from the
+current branch value should be written as:
+----
+ from refs/heads/branch^0
+----
+The `^0` suffix is necessary as gfi does not permit a branch to
+start from itself, and the branch is created in memory before the
+`from` command is even read from the input. Adding `^0` will force
+gfi to resolve the commit through Git's revision parsing library,
+rather than its internal branch table, thereby loading in the
+existing value of the branch.
+
+`merge`
+^^^^^^^
+Includes one additional ancestor commit, and makes the current
+commit a merge commit. An unlimited number of `merge` commands per
+commit are permitted by gfi, thereby establishing an n-way merge.
+However Git's other tools never create commits with more than 15
+additional ancestors (forming a 16-way merge). For this reason
+it is suggested that frontends do not use more than 15 `merge`
+commands per commit.
+
+Here `<committish>` is any of the commit specification expressions
+also accepted by `from` (see above).
+
+`filemodify`
+^^^^^^^^^^^^
+Included in a `commit` command to add a new file or change the
+content of an existing file. This command has two different means
+of specifying the content of the file.
+
+External data format::
+ The data content for the file was already supplied by a prior
+ `blob` command. The frontend just needs to connect it.
++
+....
+ 'M' SP <mode> SP <dataref> SP <path> LF
+....
++
+Here `<dataref>` can be either a mark reference (`:<idnum>`)
+set by a prior `blob` command, or a full 40-byte SHA-1 of an
+existing Git blob object.
+
+Inline data format::
+ The data content for the file has not been supplied yet.
+ The frontend wants to supply it as part of this modify
+ command.
++
+....
+ 'M' SP <mode> SP 'inline' SP <path> LF
+ data
+....
++
+See below for a detailed description of the `data` command.
+
+In both formats `<mode>` is the type of file entry, specified
+in octal. Git only supports the following modes:
+
+* `100644` or `644`: A normal (not-executable) file. The majority
+ of files in most projects use this mode. If in doubt, this is
+ what you want.
+* `100755` or `755`: A normal, but executable, file.
+* `120000`: A symlink, the content of the file will be the link target.
+
+In both formats `<path>` is the complete path of the file to be added
+(if not already existing) or modified (if already existing).
+
+A `<path>` string must use UNIX-style directory seperators (forward
+slash `/`), may contain any byte other than `LF`, and must not
+start with double quote (`"`).
+
+If an `LF` or double quote must be encoded into `<path>` shell-style
+quoting should be used, e.g. `"path/with\n and \" in it"`.
+
+The value of `<path>` must be in canoncial form. That is it must not:
+
+* contain an empty directory component (e.g. `foo//bar` is invalid),
+* end with a directory seperator (e.g. `foo/` is invalid),
+* start with a directory seperator (e.g. `/foo` is invalid),
+* contain the special component `.` or `..` (e.g. `foo/./bar` and
+ `foo/../bar` are invalid).
+
+It is recommended that `<path>` always be encoded using UTF-8.
+
+`filedelete`
+^^^^^^^^^^^^
+Included in a `commit` command to remove a file from the branch.
+If the file removal makes its directory empty, the directory will
+be automatically removed too. This cascades up the tree until the
+first non-empty directory or the root is reached.
+
+....
+ 'D' SP <path> LF
+....
+
+here `<path>` is the complete path of the file to be removed.
+See `filemodify` above for a detailed description of `<path>`.
+
+`mark`
+~~~~~~
+Arranges for gfi to save a reference to the current object, allowing
+the frontend to recall this object at a future point in time, without
+knowing its SHA-1. Here the current object is the object creation
+command the `mark` command appears within. This can be `commit`,
+`tag`, and `blob`, but `commit` is the most common usage.
+
+....
+ 'mark' SP ':' <idnum> LF
+....
+
+where `<idnum>` is the number assigned by the frontend to this mark.
+The value of `<idnum>` is expressed as an ASCII decimal integer.
+The value 0 is reserved and cannot be used as
+a mark. Only values greater than or equal to 1 may be used as marks.
+
+New marks are created automatically. Existing marks can be moved
+to another object simply by reusing the same `<idnum>` in another
+`mark` command.
+
+`tag`
+~~~~~
+Creates an annotated tag referring to a specific commit. To create
+lightweight (non-annotated) tags see the `reset` command below.
+
+....
+ 'tag' SP <name> LF
+ 'from' SP <committish> LF
+ 'tagger' SP <name> SP LT <email> GT SP <when> LF
+ data
+ LF
+....
+
+where `<name>` is the name of the tag to create.
+
+Tag names are automatically prefixed with `refs/tags/` when stored
+in Git, so importing the CVS branch symbol `RELENG-1_0-FINAL` would
+use just `RELENG-1_0-FINAL` for `<name>`, and gfi will write the
+corresponding ref as `refs/tags/RELENG-1_0-FINAL`.
+
+The value of `<name>` must be a valid refname in Git and therefore
+may contain forward slashes. As `LF` is not valid in a Git refname,
+no quoting or escaping syntax is supported here.
+
+The `from` command is the same as in the `commit` command; see
+above for details.
+
+The `tagger` command uses the same format as `committer` within
+`commit`; again see above for details.
+
+The `data` command following `tagger` must supply the annotated tag
+message (see below for `data` command syntax). To import an empty
+tag message use a 0 length data. Tag messages are free-form and are
+not interpreted by Git. Currently they must be encoded in UTF-8,
+as gfi does not permit other encodings to be specified.
+
+Signing annotated tags during import from within gfi is not
+supported. Trying to include your own PGP/GPG signature is not
+recommended, as the frontend does not (easily) have access to the
+complete set of bytes which normally goes into such a signature.
+If signing is required, create lightweight tags from within gfi with
+`reset`, then create the annotated versions of those tags offline
+with the standard gitlink:git-tag[1] process.
+
+`reset`
+~~~~~~~
+Creates (or recreates) the named branch, optionally starting from
+a specific revision. The reset command allows a frontend to issue
+a new `from` command for an existing branch, or to create a new
+branch from an existing commit without creating a new commit.
+
+....
+ 'reset' SP <ref> LF
+ ('from' SP <committish> LF)?
+ LF
+....
+
+For a detailed description of `<ref>` and `<committish>` see above
+under `commit` and `from`.
+
+The `reset` command can also be used to create lightweight
+(non-annotated) tags. For example:
+
+====
+ reset refs/tags/938
+ from :938
+====
+
+would create the lightweight tag `refs/tags/938` referring to
+whatever commit mark `:938` references.
+
+`blob`
+~~~~~~
+Requests writing one file revision to the packfile. The revision
+is not connected to any commit; this connection must be formed in
+a subsequent `commit` command by referencing the blob through an
+assigned mark.
+
+....
+ 'blob' LF
+ mark?
+ data
+....
+
+The mark command is optional here as some frontends have chosen
+to generate the Git SHA-1 for the blob on their own, and feed that
+directly to `commit`. This is typically more work than its worth
+however, as marks are inexpensive to store and easy to use.
+
+`data`
+~~~~~~
+Supplies raw data (for use as blob/file content, commit messages, or
+annotated tag messages) to gfi. Data can be supplied using an exact
+byte count or delimited with a terminating line. Real frontends
+intended for production-quality conversions should always use the
+exact byte count format, as it is more robust and performs better.
+The delimited format is intended primarily for testing gfi.
+
+Exact byte count format::
+ The frontend must specify the number of bytes of data.
++
+....
+ 'data' SP <count> LF
+ <raw> LF
+....
++
+where `<count>` is the exact number of bytes appearing within
+`<raw>`. The value of `<count>` is expressed as an ASCII decimal
+integer. The `LF` on either side of `<raw>` is not
+included in `<count>` and will not be included in the imported data.
+
+Delimited format::
+ A delimiter string is used to mark the end of the data.
+ gfi will compute the length by searching for the delimiter.
+ This format is primarly useful for testing and is not
+ recommended for real data.
++
+....
+ 'data' SP '<<' <delim> LF
+ <raw> LF
+ <delim> LF
+....
++
+where `<delim>` is the chosen delimiter string. The string `<delim>`
+must not appear on a line by itself within `<raw>`, as otherwise
+gfi will think the data ends earlier than it really does. The `LF`
+immediately trailing `<raw>` is part of `<raw>`. This is one of
+the limitations of the delimited format, it is impossible to supply
+a data chunk which does not have an LF as its last byte.
+
+`checkpoint`
+~~~~~~~~~~~~
+Forces gfi to close the current packfile and start a new one.
+As this requires a significant amount of CPU time and disk IO
+(to compute the overall pack SHA-1 checksum and generate the
+corresponding index file) it can easily take several minutes for
+a single `checkpoint` command to complete.
+
+....
+ 'checkpoint' LF
+ LF
+....
+
+Packfile Optimization
+---------------------
+When packing a blob gfi always attempts to deltify against the last
+blob written. Unless specifically arranged for by the frontend,
+this will probably not be a prior version of the same file, so the
+generated delta will not be the smallest possible. The resulting
+packfile will be compressed, but will not be optimal.
+
+Frontends which have efficient access to all revisions of a
+single file (for example reading an RCS/CVS ,v file) can choose
+to supply all revisions of that file as a sequence of consecutive
+`blob` commands. This allows gfi to deltify the different file
+revisions against each other, saving space in the final packfile.
+Marks can be used to later identify individual file revisions during
+a sequence of `commit` commands.
+
+The packfile(s) created by gfi do not encourage good disk access
+patterns. This is caused by gfi writing the data in the order
+it is received on standard input, while Git typically organizes
+data within packfiles to make the most recent (current tip) data
+appear before historical data. Git also clusters commits together,
+speeding up revision traversal through better cache locality.
+
+For this reason it is strongly recommended that users repack the
+repository with `git repack -a -d` after gfi completes, allowing
+Git to reorganize the packfiles for faster data access. If blob
+deltas are suboptimal (see above) then also adding the `-f` option
+to force recomputation of all deltas can significantly reduce the
+final packfile size (30-50% smaller can be quite typical).
+
+Memory Utilization
+------------------
+There are a number of factors which affect how much memory gfi
+requires to perform an import. Like critical sections of core
+Git, gfi uses its own memory allocators to ammortize any overheads
+associated with malloc. In practice gfi tends to ammoritize any
+malloc overheads to 0, due to its use of large block allocations.
+
+per object
+~~~~~~~~~~
+gfi maintains an in-memory structure for every object written in
+this execution. On a 32 bit system the structure is 32 bytes,
+on a 64 bit system the structure is 40 bytes (due to the larger
+pointer sizes). Objects in the table are not deallocated until
+gfi terminates. Importing 2 million objects on a 32 bit system
+will require approximately 64 MiB of memory.
+
+The object table is actually a hashtable keyed on the object name
+(the unique SHA-1). This storage configuration allows gfi to reuse
+an existing or already written object and avoid writing duplicates
+to the output packfile. Duplicate blobs are surprisingly common
+in an import, typically due to branch merges in the source.
+
+per mark
+~~~~~~~~
+Marks are stored in a sparse array, using 1 pointer (4 bytes or 8
+bytes, depending on pointer size) per mark. Although the array
+is sparse, frontends are still strongly encouraged to use marks
+between 1 and n, where n is the total number of marks required for
+this import.
+
+per branch
+~~~~~~~~~~
+Branches are classified as active and inactive. The memory usage
+of the two classes is significantly different.
+
+Inactive branches are stored in a structure which uses 96 or 120
+bytes (32 bit or 64 bit systems, respectively), plus the length of
+the branch name (typically under 200 bytes), per branch. gfi will
+easily handle as many as 10,000 inactive branches in under 2 MiB
+of memory.
+
+Active branches have the same overhead as inactive branches, but
+also contain copies of every tree that has been recently modified on
+that branch. If subtree `include` has not been modified since the
+branch became active, its contents will not be loaded into memory,
+but if subtree `src` has been modified by a commit since the branch
+became active, then its contents will be loaded in memory.
+
+As active branches store metadata about the files contained on that
+branch, their in-memory storage size can grow to a considerable size
+(see below).
+
+gfi automatically moves active branches to inactive status based on
+a simple least-recently-used algorithm. The LRU chain is updated on
+each `commit` command. The maximum number of active branches can be
+increased or decreased on the command line with `--active-branches=`.
+
+per active tree
+~~~~~~~~~~~~~~~
+Trees (aka directories) use just 12 bytes of memory on top of the
+memory required for their entries (see ``per active file'' below).
+The cost of a tree is virtually 0, as its overhead ammortizes out
+over the individual file entries.
+
+per active file entry
+~~~~~~~~~~~~~~~~~~~~~
+Files (and pointers to subtrees) within active trees require 52 or 64
+bytes (32/64 bit platforms) per entry. To conserve space, file and
+tree names are pooled in a common string table, allowing the filename
+``Makefile'' to use just 16 bytes (after including the string header
+overhead) no matter how many times it occurs within the project.
+
+The active branch LRU, when coupled with the filename string pool
+and lazy loading of subtrees, allows gfi to efficiently import
+projects with 2,000+ branches and 45,114+ files in a very limited
+memory footprint (less than 2.7 MiB per active branch).
+
+
+Author
+------
+Written by Shawn O. Pearce <spearce@spearce.org>.
+
+Documentation
+--------------
+Documentation by Shawn O. Pearce <spearce@spearce.org>.
+
+GIT
+---
+Part of the gitlink:git[7] suite
+
projects / git.git / commitdiff