From: Gaute Hope Date: Sun, 28 Feb 2016 12:32:46 +0000 (+0000) Subject: Re: how do the different frontends deal with displaying large queries? X-Git-Url: http://git.tremily.us/?a=commitdiff_plain;h=d70c5139e02507885dbd35e7ae6c3eda7dd09ec1;p=notmuch-archives.git Re: how do the different frontends deal with displaying large queries? --- diff --git a/34/dad1d92c86f261c2f5c855eb74433e6b5583fa b/34/dad1d92c86f261c2f5c855eb74433e6b5583fa new file mode 100644 index 000000000..8d07c4f07 --- /dev/null +++ b/34/dad1d92c86f261c2f5c855eb74433e6b5583fa @@ -0,0 +1,123 @@ +Return-Path: +X-Original-To: notmuch@notmuchmail.org +Delivered-To: notmuch@notmuchmail.org +Received: from localhost (localhost [127.0.0.1]) + by arlo.cworth.org (Postfix) with ESMTP id AEC396DE0924 + for ; Sun, 28 Feb 2016 04:32:56 -0800 (PST) +X-Virus-Scanned: Debian amavisd-new at cworth.org +X-Spam-Flag: NO +X-Spam-Score: -0.468 +X-Spam-Level: +X-Spam-Status: No, score=-0.468 tagged_above=-999 required=5 tests=[AWL=0.252, + DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_LOW=-0.7, + RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01] autolearn=disabled +Received: from arlo.cworth.org ([127.0.0.1]) + by localhost (arlo.cworth.org [127.0.0.1]) (amavisd-new, port 10024) + with ESMTP id rJ0iI7U4Vkln for ; + Sun, 28 Feb 2016 04:32:54 -0800 (PST) +Received: from mail-lf0-f49.google.com (mail-lf0-f49.google.com + [209.85.215.49]) + by arlo.cworth.org (Postfix) with ESMTPS id A87FA6DE00DF + for ; Sun, 28 Feb 2016 04:32:53 -0800 (PST) +Received: by mail-lf0-f49.google.com with SMTP id m1so76641930lfg.0 + for ; Sun, 28 Feb 2016 04:32:53 -0800 (PST) +DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; + d=gaute-vetsj-com.20150623.gappssmtp.com; s=20150623; + h=date:from:subject:to:references:in-reply-to:user-agent:message-id + :mime-version:content-transfer-encoding; + bh=NYKPKjKEkcPieDZ89dXiUbu1RNkbVqY7xq8fRbvtQAw=; + b=lvHqDhUKIzv9QbRkgfxJ2hK1oSmAZTKCE8Ljfzv49MSeoJgqa/bVPLRwl5YTh8eReH + SNAHy8fT/0jmmNylO2rFg6IDyizinogTsqnRDuaRJWQLo7AmpbrXj0jIh2rR1pxb5Ado + BW3wtE7oyqsQWZPmEYgao74yl/2ayWd393d+o18q6XvzpdtTzJNRycRSEevX4mSJrV6o + x9OEeucVvjdJ4tMX3v00ijhS28lc48jA1lOEWEg5ek8o1jkAeTGQMq383jO5f6TjGsub + 5ycLv7Pm+UpdfPbCZgohGyd87QHIrkh7M+usa929wK0eHY2JydDa49UcI4ppOjpg7WoO + MOiQ== +X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; + d=1e100.net; s=20130820; + h=x-gm-message-state:date:from:subject:to:references:in-reply-to + :user-agent:message-id:mime-version:content-transfer-encoding; + bh=NYKPKjKEkcPieDZ89dXiUbu1RNkbVqY7xq8fRbvtQAw=; + b=NNmonmF2pdk3GSPYZ9UZJiiHAXZBrfvAoX7iY60sYHXNOJKe9oSyyhVBqBZvpztCRw + Z5Gwe4ZHRa2RMEheOEQ6A82lcwpa7BRwN0N0KJq93zKMXAND6iPeICEDbiOu85/naACG + SosZQZomgUdXP5TJTgWRqrgj5zzBmyt2Ot0TbACCaZrVsqy5mjCNBd8mUqOgVypuJk3s + +hQtA2bUgtLbOT4Q4110NL3pF8qbdtNbZwhzWh2K3CGGKKDbeg3m9B72EsfbRCtQ3o7m + FQBYofrLc5r9xSUUDxD5LETw2p9tzJ+LgdK9nyfQFGtJOgwgO6yxf4iLRwTIvf6m5UZW + UwIQ== +X-Gm-Message-State: + AD7BkJLZ+FgTDxawuzmSBQ2aXvUFKKOgi7YpGvUWQFyCU5pN9R+8trLBWSFfHV7kTW5Lag== +X-Received: by 10.25.144.12 with SMTP id s12mr3830130lfd.114.1456662770212; + Sun, 28 Feb 2016 04:32:50 -0800 (PST) +Received: from localhost (c6878BF51.dhcp.as2116.net. [81.191.120.104]) + by smtp.gmail.com with ESMTPSA id xf10sm3136470lbb.23.2016.02.28.04.32.47 + (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); + Sun, 28 Feb 2016 04:32:48 -0800 (PST) +Date: Sun, 28 Feb 2016 12:32:46 +0000 +From: Gaute Hope +Subject: Re: how do the different frontends deal with displaying large + queries? +To: David Bremner , notmuch@notmuchmail.org +References: <1456296267-astroid-0-gqidxpyulu-23762@strange> + <87ziuqj8xl.fsf@tesseract.cs.unb.ca> + <1456487129-astroid-1-zemv9qfkbh-18628@strange> + <877fhq5ncl.fsf@zancas.localnet> +In-Reply-To: <877fhq5ncl.fsf@zancas.localnet> +User-Agent: astroid/v0.5-71-g4efc3da1 (https://github.com/gauteh/astroid) +Message-Id: <1456661143-astroid-1-q87qney5lk-1529@strange> +MIME-Version: 1.0 +Content-Type: text/plain; charset=utf-8; format=flowed +Content-Transfer-Encoding: quoted-printable +X-BeenThere: notmuch@notmuchmail.org +X-Mailman-Version: 2.1.20 +Precedence: list +List-Id: "Use and development of the notmuch mail system." + +List-Unsubscribe: , + +List-Archive: +List-Post: +List-Help: +List-Subscribe: , + +X-List-Received-Date: Sun, 28 Feb 2016 12:32:56 -0000 + +David Bremner writes on February 27, 2016 13:50: +> Gaute Hope writes: +>=20 +>> +>> While loading the threads, if I make enough modifications to the +>> database while the query is still loading (specifically 1: removing the +>> unread tag from a thread, and 2: adding the unread tag to the same threa= +d) +>> apparently the query is invalidated in some way, and I get a hard crash +>> in `notmuch_tags_get ()`. +>> +>> The thread in question has alreaday been loaded and is displayed. +>> +>> Do you deal with this issue in some way? +>> +>=20 +> I think we "deal with" it by not leaving the database open very +> long. The query is dumped as s-expr by a seperate process. Or perhaps we +> just aren't doing the same level of concurrent operations, I don't know. + +Neither do I, I load the threads in the background. But for a long query +that takes a while. The database is only kept open for the duration it +takes to load the threads in the query. + +How long is the db open? Would you notice if it crashed? I'm guessing +the abort is triggered by an Xapian::ModifiedDatabase somewhere. + +I am only able to consistently reproduce the error if I change the tags +of a thread not yet loaded, a (somewhat messy) test case is available +here: + +https://github.com/gauteh/astroid/blob/ti-stateless-query/test/test_notmuch= +_standalone.cc + +If nothing else, it demonstrates the difficulty of handling these +exceptions in functions that don't yet catch them, since they could +happen along many of the internal steps. + +Regards, Gaute + +=