Return-Path: X-Original-To: notmuch@notmuchmail.org Delivered-To: notmuch@notmuchmail.org Received: from localhost (localhost [127.0.0.1]) by olra.theworths.org (Postfix) with ESMTP id 060E6431FBC for ; Sat, 21 Nov 2009 16:57:17 -0800 (PST) X-Virus-Scanned: Debian amavisd-new at olra.theworths.org Received: from olra.theworths.org ([127.0.0.1]) by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id q6pPGsttVYh8 for ; Sat, 21 Nov 2009 16:57:16 -0800 (PST) Received: from azsmga101.ch.intel.com (mga07.intel.com [143.182.124.22]) by olra.theworths.org (Postfix) with ESMTP id 25DB7431FAE for ; Sat, 21 Nov 2009 16:57:16 -0800 (PST) Received: from azsmga001.ch.intel.com ([10.2.17.19]) by azsmga101.ch.intel.com with ESMTP; 21 Nov 2009 16:57:15 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="4.47,265,1257148800"; d="scan'208";a="214309534" Received: from unknown (HELO localhost.localdomain) ([10.255.16.190]) by azsmga001.ch.intel.com with ESMTP; 21 Nov 2009 16:57:14 -0800 From: Chris Wilson To: notmuch@notmuchmail.org Date: Sun, 22 Nov 2009 00:57:10 +0000 Message-Id: <1258851430-28732-1-git-send-email-chris@chris-wilson.co.uk> X-Mailer: git-send-email 1.6.5.3 Subject: [notmuch] [PATCH] notmuch-new: Eliminate tallocs whilst construct filenames. X-BeenThere: notmuch@notmuchmail.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Nov 2009 00:57:17 -0000 The majority of filenames will fit within PATH_MAX [4096] (because that's a hard limit imposed by the filesystems) so we can avoid an allocation per lookup and thereby eliminate a large proportion of the overhead of scanning a maildir. Signed-off-by: Chris Wilson --- notmuch-new.c | 75 ++++++++++++++++++++++++++++++++++---------------------- 1 files changed, 45 insertions(+), 30 deletions(-) diff --git a/notmuch-new.c b/notmuch-new.c index 0dd2784..13559d1 100644 --- a/notmuch-new.c +++ b/notmuch-new.c @@ -107,6 +107,7 @@ add_files_recursive (notmuch_database_t *notmuch, add_files_state_t *state) { DIR *dir = NULL; + char buf[4096]; struct dirent *entry = NULL; char *next = NULL; time_t path_mtime, path_dbtime; @@ -114,6 +115,7 @@ add_files_recursive (notmuch_database_t *notmuch, notmuch_message_t *message = NULL; struct dirent **namelist = NULL; int num_entries; + int path_len, dname_len; /* If we're told to, we bail out on encountering a read-only * directory, (with this being a clear clue from the user to @@ -140,6 +142,12 @@ add_files_recursive (notmuch_database_t *notmuch, int i=0; + path_len = strlen (path); + if (path_len + 2 < (int) sizeof (buf)) { + memcpy (buf, path, path_len); + buf[path_len] = '/'; + } + while (!interrupted) { if (i == num_entries) break; @@ -164,37 +172,42 @@ add_files_recursive (notmuch_database_t *notmuch, continue; } - next = talloc_asprintf (notmuch, "%s/%s", path, entry->d_name); + dname_len = strlen (entry->d_name); + if (path_len + dname_len + 2 < (int) sizeof (buf)) { + memcpy (buf + path_len + 1, entry->d_name, dname_len); + buf[path_len + dname_len + 1] = '\0'; + next = buf; + } else { + next = talloc_asprintf (notmuch, "%s/%s", path, entry->d_name); + } if (stat (next, st)) { fprintf (stderr, "Error reading %s: %s\n", next, strerror (errno)); ret = NOTMUCH_STATUS_FILE_ERROR; - continue; - } - - if (S_ISREG (st->st_mode)) { - /* If the file hasn't been modified since the last - * add_files, then we need not look at it. */ - if (path_dbtime == 0 || st->st_mtime > path_dbtime) { - state->processed_files++; - - status = notmuch_database_add_message (notmuch, next, &message); - switch (status) { - /* success */ + } else { + if (S_ISREG (st->st_mode)) { + /* If the file hasn't been modified since the last + * add_files, then we need not look at it. */ + if (path_dbtime == 0 || st->st_mtime > path_dbtime) { + state->processed_files++; + + status = notmuch_database_add_message (notmuch, next, &message); + switch (status) { + /* success */ case NOTMUCH_STATUS_SUCCESS: state->added_messages++; tag_inbox_and_unread (message); break; - /* Non-fatal issues (go on to next file) */ + /* Non-fatal issues (go on to next file) */ case NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID: - /* Stay silent on this one. */ + /* Stay silent on this one. */ break; case NOTMUCH_STATUS_FILE_NOT_EMAIL: fprintf (stderr, "Note: Ignoring non-mail file: %s\n", next); break; - /* Fatal issues. Don't process anymore. */ + /* Fatal issues. Don't process anymore. */ case NOTMUCH_STATUS_READONLY_DATABASE: case NOTMUCH_STATUS_XAPIAN_EXCEPTION: case NOTMUCH_STATUS_OUT_OF_MEMORY: @@ -210,25 +223,27 @@ add_files_recursive (notmuch_database_t *notmuch, case NOTMUCH_STATUS_LAST_STATUS: INTERNAL_ERROR ("add_message returned unexpected value: %d", status); goto DONE; - } + } - if (message) { - notmuch_message_destroy (message); - message = NULL; - } + if (message) { + notmuch_message_destroy (message); + message = NULL; + } - if (do_add_files_print_progress) { - do_add_files_print_progress = 0; - add_files_print_progress (state); + if (do_add_files_print_progress) { + do_add_files_print_progress = 0; + add_files_print_progress (state); + } } + } else if (S_ISDIR (st->st_mode)) { + status = add_files_recursive (notmuch, next, st, state); + if (status && ret == NOTMUCH_STATUS_SUCCESS) + ret = status; } - } else if (S_ISDIR (st->st_mode)) { - status = add_files_recursive (notmuch, next, st, state); - if (status && ret == NOTMUCH_STATUS_SUCCESS) - ret = status; } - talloc_free (next); + if (next != buf) + talloc_free (next); next = NULL; } @@ -237,7 +252,7 @@ add_files_recursive (notmuch_database_t *notmuch, ret = status; DONE: - if (next) + if (next != buf) talloc_free (next); if (entry) free (entry); -- 1.6.5.3