From: Austin Clements Date: Sun, 22 Apr 2012 04:11:10 +0000 (+2000) Subject: Re: [PATCH 1/3] new: Consistently treat fatal errors as fatal X-Git-Url: http://git.tremily.us/gitweb.cgi?a=commitdiff_plain;h=f3664e7bf83077850ad086e5bc3de931961b07a8;p=notmuch-archives.git Re: [PATCH 1/3] new: Consistently treat fatal errors as fatal --- diff --git a/d9/5007d468ed78f3a735adb00b27ab8adb3a6b1f b/d9/5007d468ed78f3a735adb00b27ab8adb3a6b1f new file mode 100644 index 000000000..fd24635e3 --- /dev/null +++ b/d9/5007d468ed78f3a735adb00b27ab8adb3a6b1f @@ -0,0 +1,174 @@ +Return-Path: +X-Original-To: notmuch@notmuchmail.org +Delivered-To: notmuch@notmuchmail.org +Received: from localhost (localhost [127.0.0.1]) + by olra.theworths.org (Postfix) with ESMTP id 74F7F431FAF + for ; Sat, 21 Apr 2012 21:11:14 -0700 (PDT) +X-Virus-Scanned: Debian amavisd-new at olra.theworths.org +X-Spam-Flag: NO +X-Spam-Score: -0.7 +X-Spam-Level: +X-Spam-Status: No, score=-0.7 tagged_above=-999 required=5 + tests=[RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled +Received: from olra.theworths.org ([127.0.0.1]) + by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024) + with ESMTP id 1CWY6GUiW+Ma for ; + Sat, 21 Apr 2012 21:11:13 -0700 (PDT) +Received: from dmz-mailsec-scanner-8.mit.edu (DMZ-MAILSEC-SCANNER-8.MIT.EDU + [18.7.68.37]) + by olra.theworths.org (Postfix) with ESMTP id 9BA20431FAE + for ; Sat, 21 Apr 2012 21:11:13 -0700 (PDT) +X-AuditID: 12074425-b7f4a6d0000008e0-50-4f9384e1b031 +Received: from mailhub-auth-1.mit.edu ( [18.9.21.35]) + by dmz-mailsec-scanner-8.mit.edu (Symantec Messaging Gateway) with SMTP + id 1A.8B.02272.1E4839F4; Sun, 22 Apr 2012 00:11:13 -0400 (EDT) +Received: from outgoing.mit.edu (OUTGOING-AUTH.MIT.EDU [18.7.22.103]) + by mailhub-auth-1.mit.edu (8.13.8/8.9.2) with ESMTP id q3M4BCEV005381; + Sun, 22 Apr 2012 00:11:13 -0400 +Received: from awakening.csail.mit.edu (awakening.csail.mit.edu [18.26.4.91]) + (authenticated bits=0) + (User authenticated as amdragon@ATHENA.MIT.EDU) + by outgoing.mit.edu (8.13.6/8.12.4) with ESMTP id q3M4BAas010145 + (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=NOT); + Sun, 22 Apr 2012 00:11:11 -0400 (EDT) +Received: from amthrax by awakening.csail.mit.edu with local (Exim 4.77) + (envelope-from ) + id 1SLo8k-0005Q9-7z; Sun, 22 Apr 2012 00:11:10 -0400 +Date: Sun, 22 Apr 2012 00:11:10 -0400 +From: Austin Clements +To: Mark Walters +Subject: Re: [PATCH 1/3] new: Consistently treat fatal errors as fatal +Message-ID: <20120422041109.GA17810@mit.edu> +References: <1330357759-1337-1-git-send-email-amdragon@mit.edu> + <1330357759-1337-2-git-send-email-amdragon@mit.edu> + <87zkab4qs9.fsf@qmul.ac.uk> +MIME-Version: 1.0 +Content-Type: text/plain; charset=us-ascii +Content-Disposition: inline +In-Reply-To: <87zkab4qs9.fsf@qmul.ac.uk> +User-Agent: Mutt/1.5.21 (2010-09-15) +X-Brightmail-Tracker: + H4sIAAAAAAAAA+NgFmpmleLIzCtJLcpLzFFi42IR4hRV1n3YMtnfYOZMNovVc3ksrt+cyezA + 5LFz1l12j2erbjEHMEVx2aSk5mSWpRbp2yVwZWyYMoGx4LNMRf/GW2wNjP9Euxg5OSQETCQ2 + rtjDCmGLSVy4t56ti5GLQ0hgH6PE9TvrWCCcDYwSiy+2MUE4J5kkth1cwwrhLGGU2Hf/NFAZ + BweLgKpEyyw/kFFsAhoS2/YvZwSxRQR0JG4fWsAOYjMLSEt8+93MBGILC7hJzJ+xmRnE5gWq + uXH0CSPEzMmMEofmnodKCEqcnPmEBaJZS+LGv5dMILtABi3/xwES5gTatX3zbDYQW1RARWLK + yW1sExiFZiHpnoWkexZC9wJG5lWMsim5Vbq5iZk5xanJusXJiXl5qUW6Fnq5mSV6qSmlmxhB + Yc3uorqDccIhpUOMAhyMSjy8L00m+wuxJpYVV+YeYpTkYFIS5dWvBArxJeWnVGYkFmfEF5Xm + pBYfYpTgYFYS4f0sApTjTUmsrEotyodJSXOwKInzamq98xMSSE8sSc1OTS1ILYLJynBwKEnw + egLjV0iwKDU9tSItM6cEIc3EwQkynAdo+OtmkOHFBYm5xZnpEPlTjIpS4ryRIM0CIImM0jy4 + XljaecUoDvSKMK8RSBUPMGXBdb8CGswENDghZhLI4JJEhJRUA2PpxQeTD62YPcH4i9Qu9zMb + /Sd+P3f48FFN0eMXG1iPTZ3Fuk0/RVd7rW5Adta+5d65Fm+qw6uftN6davFR68HvnQ6Sa1vc + a/kdHjfcfbTgdcd1bW+pfxvDebatbf74tFbap1nRmO3r1RdpndkbkxUWyHzYqv7IcnKg033x + TXxMTVKXNv78Y+SqxFKckWioxVxUnAgAsTYadRYDAAA= +Cc: notmuch@notmuchmail.org +X-BeenThere: notmuch@notmuchmail.org +X-Mailman-Version: 2.1.13 +Precedence: list +List-Id: "Use and development of the notmuch mail system." + +List-Unsubscribe: , + +List-Archive: +List-Post: +List-Help: +List-Subscribe: , + +X-List-Received-Date: Sun, 22 Apr 2012 04:11:14 -0000 + +Quoth Mark Walters on Apr 16 at 4:53 pm: +> +> On Mon, 27 Feb 2012, Austin Clements wrote: +> > Previously, fatal errors in add_files_recursive were not treated as +> > fatal by its callers (including itself!) and add_files_recursive +> > sometimes returned errors on non-fatal conditions. This makes +> > add_files_recursive errors consistently fatal and updates all callers +> > to treat them as fatal. +> +> Hi I have attempted to review this but am feeling a little out of my +> depth. This patch seems fine except for one thing I am unsure about: +> +> > --- +> > notmuch-new.c | 13 ++++++++----- +> > 1 files changed, 8 insertions(+), 5 deletions(-) +> > +> > diff --git a/notmuch-new.c b/notmuch-new.c +> > index 4f13535..bd9786a 100644 +> > --- a/notmuch-new.c +> > +++ b/notmuch-new.c +> > @@ -308,7 +308,6 @@ add_files_recursive (notmuch_database_t *notmuch, +> > if (num_fs_entries == -1) { +> > fprintf (stderr, "Error opening directory %s: %s\n", +> > path, strerror (errno)); +> > - ret = NOTMUCH_STATUS_FILE_ERROR; +> > goto DONE; +> > } +> > +> +> If I understand this, then this change makes a failure to open a +> directory non-fatal. In the comment before the function it says +> +> * Also, we don't immediately act on file/directory removal since we +> * must ensure that in the case of a rename that the new filename is +> * added before the old filename is removed, (so that no information +> * is lost from the database). +> +> I am worried that we could fail to find some files because of the +> file error above, and then we delete them from the database. Maybe this +> could only happen if those emails have just been moved to this +> file-error directory? + +Hmm. This won't *actively* remove files, since that only happens if +we successfully scan a directory and find a message that's in the +database but not in that directory (*not* scanning the directory won't +add anything to the remove list). However, you are right that if a +message is moved from some other directory into a directory that we +fail to open, that message will be deleted "by omission". + +I've added the error back, along with a comment explaining it. + +> Best wishes +> +> Mark +> +> > @@ -351,8 +350,10 @@ add_files_recursive (notmuch_database_t *notmuch, +> > +> > next = talloc_asprintf (notmuch, "%s/%s", path, entry->d_name); +> > status = add_files_recursive (notmuch, next, state); +> > - if (status && ret == NOTMUCH_STATUS_SUCCESS) +> > + if (status) { +> > ret = status; +> > + goto DONE; +> > + } +> > talloc_free (next); +> > next = NULL; +> > } +> > @@ -933,6 +934,8 @@ notmuch_new_command (void *ctx, int argc, char *argv[]) +> > } +> > +> > ret = add_files (notmuch, db_path, &add_files_state); +> > + if (ret) +> > + goto DONE; +> > +> > gettimeofday (&tv_start, NULL); +> > for (f = add_files_state.removed_files->head; f && !interrupted; f = f->next) { +> > @@ -965,6 +968,7 @@ notmuch_new_command (void *ctx, int argc, char *argv[]) +> > } +> > } +> > +> > + DONE: +> > talloc_free (add_files_state.removed_files); +> > talloc_free (add_files_state.removed_directories); +> > talloc_free (add_files_state.directory_mtimes); +> > @@ -1012,10 +1016,9 @@ notmuch_new_command (void *ctx, int argc, char *argv[]) +> > +> > printf ("\n"); +> > +> > - if (ret) { +> > - printf ("\nNote: At least one error was encountered: %s\n", +> > + if (ret) +> > + printf ("\nNote: A fatal error was encountered: %s\n", +> > notmuch_status_to_string (ret)); +> > - } +> > +> > notmuch_database_close (notmuch);