From: Jani Nikula <jani@nikula.org>
To: notmuch@notmuchmail.org
Subject: nmfirehose - a crude nmbug companion tool
User-Agent: Notmuch/0.11.1+295~g780f284 (http://notmuchmail.org) Emacs/23.3.1
	(i686-pc-linux-gnu)
Date: Sun, 11 Mar 2012 02:01:34 +0200
Message-ID: <87wr6sypi9.fsf@nikula.org>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="=-=-="
Precedence: list

--=-=-=


Hi all -

There was some talk on IRC about maintaining a development branch
containing the "maybe ready" patches [1] to better expose them to
real-life use before merging to master.

I set out to automate this a little, and ended up with nmfirehose. It's
a shell script that can be used to grab a bunch of patches from the
notmuch mail store based on queries, and apply them to a repository. In
particular, it can grab the "maybe ready" patches and apply them against
notmuch master.

Inevitable conflicts are handled by applying patches in oldest first
order, and dropping any conflicting series. An email thread approximates
a patch series; more than one patch series in a thread are folded into
one series.

Patches are applied incrementally, on top of each other by default, but
there's also support for non-incremental per-series apply against master
to see if there are conflicts. This is useful for the "review"
query. Conflicting patches can be automatically tagged if so desired
(but I don't recommend pushing these to nmbug without human review).

How well any of this works very much depends on how accurately the patch
messages are tagged.

Some examples (run within a notmuch git repo, with up-to-date nmbug tags
in the database):

# take nmbug maybe-ready patches and apply them against origin/master
$ nmfirehose

# use local branch firehose and force use it even if existing
$ nmfirehose -l firehose -F

# keep going after conflicts without reverting whole series
$ nmfirehose -k

# check each series in review queue against master, and tag conflicting
$ nmfirehose -q review -n -t

# help
$ nmfirehose -h

I originally planned on putting this to cron and pushing the result
(supported out-of-the-box) to some public repo, but in the end I think
people can use this locally just as well, at the pace they want.

The script tries not to screw up anything in your notmuch repo, but be
careful nonetheless.

Script attached; comments welcome. Is this worth adding to contrib?


BR,
Jani.

[1] http://nmbug.tethera.net/status/

--=-=-=
Content-Type: application/octet-stream
Content-Disposition: attachment; filename=nmfirehose
Content-Transfer-Encoding: base64
Content-Description: nmfirehose

IyEvYmluL3NoCiMgQ29weXJpZ2h0IChjKSAyMDEyIEphbmkgTmlrdWxhCiMgTGljZW5zZTogc2Ft
ZSBhcyBub3RtdWNoCgojIFByZWRlZmluZWQgcXVlcmllcy4gU2VlIGh0dHA6Ly9ubWJ1Zy50ZXRo
ZXJhLm5ldC9zdGF0dXMvClFVRVJZX01BWUJFX1JFQURZPSJ0YWc6bm90bXVjaDo6cGF0Y2ggYW5k
IG5vdCB0YWc6bm90bXVjaDo6cHVzaGVkIGFuZCBub3QgdGFnOm5vdG11Y2g6Om9ic29sZXRlIGFu
ZCBub3QgdGFnOm5vdG11Y2g6OndpcCBhbmQgbm90IHRhZzpub3RtdWNoOjpzdGFsZSBhbmQgbm90
IHRhZzpub3RtdWNoOjpjb250cmliIGFuZCBub3QgdGFnOm5vdG11Y2g6Om1vcmVpbmZvIGFuZCBu
b3QgdGFnOm5vdG11Y2g6OnB5dGhvbiBhbmQgbm90IHRhZzpub3RtdWNoOjp2aW0gYW5kIG5vdCB0
YWc6bm90bXVjaDo6d29udGZpeCBhbmQgbm90IHRhZzpub3RtdWNoOjpuZWVkcy1yZXZpZXciCgpR
VUVSWV9SRVZJRVc9InRhZzpub3RtdWNoOjpwYXRjaCBhbmQgbm90IHRhZzpub3RtdWNoOjpwdXNo
ZWQgYW5kIG5vdCB0YWc6bm90bXVjaDo6b2Jzb2xldGUgYW5kIG5vdCB0YWc6bm90bXVjaDo6c3Rh
bGUgYW5kIG5vdCB0YWc6bm90bXVjaDo6d29udGZpeCBhbmQgKHRhZzpub3RtdWNoOjptb3JlaW5m
byBvciB0YWc6bm90bXVjaDo6bmVlZHMtcmV2aWV3KSIKCiMgRGVmYXVsdHMgZm9yIGNvbW1hbmQg
bGluZSBvcHRpb25zCiMgbG9jYWwKTE9DQUxfQlJBTkNIPSJmaXJlaG9zZS1gZGF0ZSArJUYuJUgl
TSVTYCIKRk9SQ0VfTE9DQUw9CkdJVF9DSEVDS09VVF9NT0RFPSItYiIKUVVFUlk9IiRRVUVSWV9N
QVlCRV9SRUFEWSIKIyB1cHN0cmVhbQpET19GRVRDSD0KVVBTVFJFQU1fTkFNRT0ib3JpZ2luIgpV
UFNUUkVBTV9CUkFOQ0g9Im1hc3RlciIKIyBwdXNoaW5nCkRPX1BVU0g9CkdJVF9QVVNIX0ZPUkNF
PQpSRU1PVEVfQlJBTkNIPSJmaXJlaG9zZSIKUkVNT1RFX05BTUU9ImZpcmVob3NlIgojIGNvbmZs
aWN0IG1hbmFnZW1lbnQKRE9fTk9OX0lOQ1JFTUVOVEFMPQpET19LRUVQX0dPSU5HPQpET19UQUc9
CkNPTkZMSUNUX1RBRz0ibWF5YmUtc3RhbGUiCgp1c2FnZSgpCnsKICAgIGNhdCA8PEVPRgpubWZp
cmVob3NlIFtvcHRpb25zIC4uLl0KCkFwcGx5IHBhdGNoZXMgb24gdG9wIG9mIGFuIHVwc3RyZWFt
IHJlcG9zaXRvcnkgZnJvbSBhIG5vdG11Y2ggbWFpbCBzdG9yZSBiYXNlZApvbiBub3RtdWNoIHF1
ZXJpZXMuCgpSdW4gaW4gYSBnaXQgdHJlZSB3aXRoIHN1aXRhYmxlIHJlbW90ZShzKSBzZXQgKCRV
UFNUUkVBTV9OQU1FIGlzIHRoZSBkZWZhdWx0CnVwc3RyZWFtIHJlbW90ZSkuCgpBbiBlbWFpbCB0
aHJlYWQgaXMgZXhwZWN0ZWQgdG8gY29udGFpbiBleGFjdGx5IG9uZSBwYXRjaCBzZXJpZXMuIE11
bHRpcGxlIHNlcmllcwppbiBhIHRocmVhZCBhcmUgY29uc2lkZXJlZCBvbmUgc2VyaWVzLiBUaGUg
cGF0Y2ggb3JkZXIgaW4gYSBzZXJpZXMgaXMgYmFzZWQgb24Kbm90bXVjaCBzZWFyY2ggLS1zb3J0
PW9sZGVzdC1maXJzdC4KCkxvY2FsOgogIC1sIExPQ0FMX0JSQU5DSCAgICAgbG9jYWwgYnJhbmNo
IHRvIHVzZSAoZGVmYXVsdDogJExPQ0FMX0JSQU5DSCkKICAtRiAgICAgICAgICAgICAgICAgIGZv
cmNlIHVzZSBsb2NhbCBicmFuY2ggZXZlbiBpZiBleGlzdGluZwogIC1xIFFVRVJZX05BTUUgICAg
ICAgcHJlZGVmaW5lZCBxdWVyeSB0byB1c2UsIHNlZSBiZWxvdyAoZGVmYXVsdDogbWF5YmUtcmVh
ZHkpCiAgLVEgUVVFUlkgICAgICAgICAgICBxdWVyeSB0byB1c2UgKGRlZmF1bHQ6IHNhbWUgYXMg
LXEgbWF5YmUtcmVhZHkpCgpVcHN0cmVhbSBiYXNlbGluZSB0byBhcHBseSBwYXRjaGVzIG9uOgog
IC1mICAgICAgICAgICAgICAgICAgZmV0Y2ggdXBzdHJlYW0gZmlyc3QKICAtdSBVUFNUUkVBTV9C
UkFOQ0ggIHVwc3RyZWFtIGJyYW5jaCAoZGVmYXVsdDogJFVQU1RSRUFNX0JSQU5DSCkKICAtVSBV
UFNUUkVBTV9OQU1FICAgIHVwc3RyZWFtIHJlbW90ZSBuYW1lIChkZWZhdWx0OiAkVVBTVFJFQU1f
TkFNRSkKClB1c2hpbmcgdGhlIHJlc3VsdDoKICAtcCAgICAgICAgICAgICAgICAgIHB1c2ggdGhl
IHJlc3VsdCBhZnRlcndhcmRzCiAgLVAgICAgICAgICAgICAgICAgICBmb3JjZSBwdXNoIHRoZSBy
ZXN1bHQgYWZ0ZXJ3YXJkcwogIC1yIFJFTU9URV9CUkFOQ0ggICAgcmVtb3RlIGJyYW5jaCB0byBw
dXNoIHRvIChkZWZhdWx0OiAkUkVNT1RFX0JSQU5DSCkKICAtUiBSRU1PVEVfTkFNRSAgICAgIHJl
bW90ZSBuYW1lIHRvIHB1c2ggdG8gKGRlZmF1bHQ6ICRSRU1PVEVfTkFNRSkKCkNvbmZsaWN0IG1h
bmFnZW1lbnQ6CiAgLW4gICAgICAgICAgICAgICAgICBub24taW5jcmVtZW50YWwsIGFwcGx5IGVh
Y2ggc2VyaWVzIG9uIHVwc3RyZWFtCiAgLWsgICAgICAgICAgICAgICAgICBrZWVwIGdvaW5nIGFm
dGVyIGNvbmZsaWN0cyB3aXRob3V0IHJldmVydGluZyBzZXJpZXMKICAtdCAgICAgICAgICAgICAg
ICAgIHRhZyBjb25mbGljdGluZyBwYXRjaGVzCiAgLVQgVEFHICAgICAgICAgICAgICB0YWcgZm9y
IGNvbmZsaWN0aW5nIHBhdGNoZXMgKGRlZmF1bHQ6ICRDT05GTElDVF9UQUcpCgpQcmVkZWZpbmVk
IHF1ZXJpZXM6CgptYXliZS1yZWFkeSA9ICRRVUVSWV9NQVlCRV9SRUFEWQoKcmV2aWV3ID0gJFFV
RVJZX1JFVklFVwpFT0YKICAgIGV4aXQgMAp9CgplcnJvcigpCnsKICAgIGVjaG8gIm5tZmlyZWhv
c2U6ICQqIiA+JjIKICAgIGV4aXQgMQp9CgpzZXQgLWV1Cgp3aGlsZSBnZXRvcHRzIGw6RnE6UTpm
dTpVOnBQcjpSOm5rdFQ6aCBvcHQ7IGRvCiAgICBjYXNlICIkb3B0IiBpbgoJIyBsb2NhbAoJbCkg
TE9DQUxfQlJBTkNIPSIkT1BUQVJHIjs7CglGKSBGT1JDRV9MT0NBTD0xOyBHSVRfQ0hFQ0tPVVRf
TU9ERT0iLUIiOzsKCXEpCgkgICAgaWYgWyAiJE9QVEFSRyIgPSAibWF5YmUtcmVhZHkiIF07IHRo
ZW4KCQlRVUVSWT0iJFFVRVJZX01BWUJFX1JFQURZIgoJICAgIGVsaWYgWyAiJE9QVEFSRyIgPSAi
cmV2aWV3IiBdOyB0aGVuCgkJUVVFUlk9IiRRVUVSWV9SRVZJRVciCgkgICAgZWxzZQoJCWVycm9y
ICJ1bmtub3duIHByZWRlZmluZWQgcXVlcnkgJyRPUFRBUkcnIgoJICAgIGZpCgkgICAgOzsKCVEp
IFFVRVJZPSIkT1BUQVJHIjs7CgkjIHVwc3RyZWFtCglmKSBET19GRVRDSD0xOzsKCXUpIFVQU1RS
RUFNX0JSQU5DSD0iJE9QVEFSRyI7OwoJVSkgVVBTVFJFQU1fTkFNRT0iJE9QVEFSRyI7OwoJIyBw
dXNoaW5nCglwKSBET19QVVNIPTE7OwoJUCkgRE9fUFVTSD0xOyBHSVRfUFVTSF9GT1JDRT0iLS1m
b3JjZSI7OwoJcikgUkVNT1RFX0JSQU5DSD0iJE9QVEFSRyI7OwoJUikgUkVNT1RFX05BTUU9IiRP
UFRBUkciOzsKCSMgY29uZmxpY3QgbWFuYWdlbWVudAoJbikgRE9fTk9OX0lOQ1JFTUVOVEFMPTE7
OwoJaykgRE9fS0VFUF9HT0lORz0xOzsKCXQpIERPX1RBRz0xOzsKCVQpIENPTkZMSUNUX1RBRz0i
JE9QVEFSRyI7OwoJIyBoZWxwCgloKSB1c2FnZTs7CglcPykgZXJyb3IgInRyeSAtaCBmb3IgdXNh
Z2UiOzsKICAgIGVzYWMKZG9uZQpzaGlmdCBgZXhwciAkT1BUSU5EIC0gMWAKCmlmIFsgIiQjIiAh
PSAiMCIgXTsgdGhlbgogICAgZXJyb3IgInVuc3VwcG9ydGVkIHBvc2l0aW9uYWwgYXJndW1lbnQo
cykgJyQqJzsgdHJ5IC1oIGZvciB1c2FnZSIKZmkKCmlmIFsgLW4gIiRET19QVVNIIiAtYSAtbiAi
JERPX05PTl9JTkNSRU1FTlRBTCIgXTsgdGhlbgogICAgZXJyb3IgIm5vbi1pbmNyZW1lbnRhbCBw
dXNoIGlzIG5vdCBzdXBwb3J0ZWQiCmZpCgojIE5PVEU6IFRoaXMgZG9lcyBub3QgY2F0Y2ggYWxs
IHRoZSB3YXlzIHVzZXJzIGNhbiBzaG9vdCB0aGVtc2VsdmVzIGluIHRoZSBmZWV0LgppZiAhIGdp
dCBkaWZmIC0tcXVpZXQgfHwgISBnaXQgZGlmZiAtLWNhY2hlZCAtLXF1aWV0OyB0aGVuCiAgICBl
cnJvciAidW5jb21taXR0ZWQgbG9jYWwgY2hhbmdlcyIKZmkKCiMgRklYTUU6IENoZWNrIHRoYXQg
d2UncmUgaW4gYSBnaXQtZGlyIGFuZCB3b3JrLWRpciBhbmQgbm90IGJhcmUuIGdpdCByZXYtcGFy
c2UuCgppZiBbIC1uICIkRE9fRkVUQ0giIF07IHRoZW4KICAgIGdpdCBmZXRjaCAkVVBTVFJFQU1f
TkFNRQpmaQoKUkVTRVRfSEVBRD0kVVBTVFJFQU1fTkFNRS8kVVBTVFJFQU1fQlJBTkNICgojIE5P
VEU6IFRoaXMgZG9lcyBub3QgY2F0Y2ggYWxsIHRoZSB3YXlzIHVzZXJzIGNhbiBzaG9vdCB0aGVt
c2VsdmVzIGluIHRoZSBmZWV0LgppZiBbICJgZ2l0IG5hbWUtcmV2IC0tbmFtZS1vbmx5IEhFQURg
IiA9ICIkTE9DQUxfQlJBTkNIIiBdOyB0aGVuCiAgICBpZiBbIC1uICIkRk9SQ0VfTE9DQUwiIF07
IHRoZW4KCWdpdCByZXNldCAtLWhhcmQgJFJFU0VUX0hFQUQKICAgIGVsc2UKCWVycm9yICJhbHJl
YWR5IG9uIGJyYW5jaCAkTE9DQUxfQlJBTkNILCB0cnkgLUYgdG8gZm9yY2UiCiAgICBmaQplbHNl
CiAgICBnaXQgY2hlY2tvdXQgJEdJVF9DSEVDS09VVF9NT0RFICRMT0NBTF9CUkFOQ0ggJFVQU1RS
RUFNX05BTUUvJFVQU1RSRUFNX0JSQU5DSApmaQoKY2xlYW51cCgpCnsKICAgIHJ2PSQ/CiAgICAj
IEZJWE1FOiBjb3VsZCBkbyBtb3JlIGNsZXZlciB0aGluZ3MgaGVyZQogICAgZ2l0IGFtIC0tYWJv
cnQgPi9kZXYvbnVsbCAyPiYxCiAgICBleGl0ICRydgp9CnRyYXAgY2xlYW51cCAxIDIgMyA2IDE1
CgpUSFJFQURTPWBub3RtdWNoIHNlYXJjaCAtLW91dHB1dD10aHJlYWRzIC0tc29ydD1vbGRlc3Qt
Zmlyc3QgJFFVRVJZYAoKZm9yIHRocmVhZCBpbiAkVEhSRUFEUzsgZG8KICAgIE1FU1NBR0VTPWBu
b3RtdWNoIHNlYXJjaCAtLW91dHB1dD1tZXNzYWdlcyAtLXNvcnQ9b2xkZXN0LWZpcnN0ICRRVUVS
WSBBTkQgJHRocmVhZGAKICAgIHN1YmplY3Q9CgogICAgZm9yIG1lc3NhZ2UgaW4gJE1FU1NBR0VT
OyBkbwoJaWYgWyAteiAiJHN1YmplY3QiIF07IHRoZW4KCSAgICBzdWJqZWN0PWBub3RtdWNoIHNo
b3cgLS1mb3JtYXQ9cmF3ICRtZXNzYWdlIHwgZ3JlcCAtbSAxICJeU3ViamVjdDogIiB8IHNlZCAn
cy9eU3ViamVjdDogLy8nYAoJICAgIGVjaG8gIiR0aHJlYWQ6IFwiJHN1YmplY3RcIjoiCglmaQoK
CWlmIG5vdG11Y2ggc2hvdyAtLWZvcm1hdD1tYm94ICRtZXNzYWdlIHwgZ2l0IGFtIC0gPi9kZXYv
bnVsbCAyPiYxOyB0aGVuCgkgICAgZWNobyAiKyAkbWVzc2FnZSBhcHBsaWVkLiIKCWVsc2UKCSAg
ICBnaXQgYW0gLS1hYm9ydAoKCSAgICBpZiBbIC1uICIkRE9fVEFHIiAtYSAtbiAiJENPTkZMSUNU
X1RBRyIgXTsgdGhlbgoJCW5vdG11Y2ggdGFnICskQ09ORkxJQ1RfVEFHIC0tICRtZXNzYWdlCgkg
ICAgZmkKCgkgICAgaWYgWyAtbiAiJERPX0tFRVBfR09JTkciIF07IHRoZW4KCQllY2hvICIkbWVz
c2FnZSBjb25mbGljdHMsIHNraXBwaW5nLiIKCSAgICBlbHNlCgkJaWYgWyAtbiAiJERPX05PTl9J
TkNSRU1FTlRBTCIgXTsgdGhlbgoJCSAgICBlY2hvICItICRtZXNzYWdlIGNvbmZsaWN0cy4iCgkJ
ICAgICMgYnJhbmNoIHdpbGwgYmUgcmVzZXQgYW55d2F5CgkJZWxzZQoJCSAgICBlY2hvICItICRt
ZXNzYWdlIGNvbmZsaWN0cywgcmV2ZXJ0aW5nIHRocmVhZC4iCgkJICAgIGdpdCByZXNldCAtLWhh
cmQgJFJFU0VUX0hFQUQKCQlmaQoJCWJyZWFrCgkgICAgZmkKCWZpCiAgICBkb25lCgogICAgaWYg
WyAtbiAiJERPX05PTl9JTkNSRU1FTlRBTCIgXTsgdGhlbgoJZWNobyAibm9uLWluY3JlbWVudGFs
IG1vZGUsIHJldmVydGluZyB0byAkUkVTRVRfSEVBRC4iCglnaXQgcmVzZXQgLS1oYXJkICRSRVNF
VF9IRUFECiAgICBlbHNlCglSRVNFVF9IRUFEPWBnaXQgcmV2LXBhcnNlIEhFQURgCiAgICBmaQpk
b25lCgppZiBbIC1uICIkRE9fUFVTSCIgXTsgdGhlbgogICAgZ2l0IHB1c2ggJEdJVF9QVVNIX0ZP
UkNFICRSRU1PVEVfTkFNRSAkTE9DQUxfQlJBTkNIOiRSRU1PVEVfQlJBTkNICmZpCg==
--=-=-=--