From: Pieter Praet <pieter@praet.org>
To: Tomi Ollila <tomi.ollila@iki.fi>,
	Jameson Graef Rollins <jrollins@finestructure.net>,
	Notmuch Mail <notmuch@notmuchmail.org>
Subject: Re: [PATCH] test: use dtach(1) instead of screen(1) in emacs tests.
In-Reply-To: <yf6ipmrdqs2.fsf@taco2.nixu.fi>
References: <yf6mxc3ab4t.fsf@taco2.nixu.fi>
	<1320963737-1666-1-git-send-email-jrollins@finestructure.net>
	<87hb2btsv2.fsf@praet.org> <yf6ipmrdqs2.fsf@taco2.nixu.fi>
User-Agent: Notmuch/0.9+76~g2fd88e6 (http://notmuchmail.org) Emacs/23.3.1
	(x86_64-unknown-linux-gnu)
Date: Tue, 15 Nov 2011 17:53:57 +0100
Message-ID: <87aa7xjpi2.fsf@praet.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Precedence: list

On Fri, 11 Nov 2011 10:09:17 +0200, Tomi Ollila <tomi.ollila@iki.fi> wrote:
> On Fri, 11 Nov 2011 01:17:37 +0100, Pieter Praet <pieter@praet.org> wrote:
> > On Thu, 10 Nov 2011 14:22:17 -0800, Jameson Graef Rollins <jrollins@finestructure.net> wrote:
> > > From: Tomi Ollila <tomi.ollila@iki.fi>
> > > 
> > > dtach is lighter than screen and is not setuid/setgid program so
> > > TMPDIR does not get reset by dynamic loader when executed.
> > > 
> > 
> > `dtach' may be lighter than `screen', but contrary to my expectations,
> > it appears to be far less efficient:
> > 
> > |      | original [1] | w/ screen [2] | w/ dtach [3] |
> > |------+--------------+---------------+--------------|
> > | real |    0m17.570s |     0m20.132s |    0m28.567s |
> > | user |     0m2.453s |      0m2.593s |     0m2.753s |
> > | sys  |     0m1.700s |      0m1.577s |     0m1.753s |
> 
> This sounds interesting; how were these runs measured?
> Just using shell builtin time command ?
> 

Correct.

> Does time measure the time of the process and it's subprocesses
> or just the process. In any way 20 vs 28 second 'real' time difference
> could indicate that there are more other processes running and
> the context switch time affects user and system time.
> 

I just ran this rather crude ad-hoc script while off to make some coffee :)

  #+begin_src sh
    revs=(749abb74 760b311b tmp)
    loop=5
    cd ~/src/dev/notmuch
    until [ ${loop} == 0 ] ; do
        for i in ${revs[*]} ; do
            git co ${i}
            rm -rf /dev/shm/notmuch/*
            make clean >/dev/null 2>&1
            make >/dev/null 2>&1
            sync
            time make test OPTIONS="--root=/dev/shm/notmuch" | tail -
        done
        loop=$[${loop}-1]
    done
  #+end_src

So, I try to start with a blank slate and then simply time the entire
test suite.  This for each of the relevant commits ("tmp" is a branch on
top of 760b311b, containing only the dtach patch), five times in a row.

The results remained consistent between iterations (17s, 20s, 28s "real"
time, without exception), so they should be fairly accurate.

> Or, dtach does some stupid io things. Or screen does some advanced
> optimizations. Or emacs works more effectively when TERM=screen.
> Or the 'virtual' window size emacs sees is unsuitable...
> 
> 
> > 
> > > Signed-off-by: Jameson Graef Rollins <jrollins@finestructure.net>
> > > ---
> > > This tweaks the original patch slightly by removing the no-longer needed
> > > screenrc variables.
> > > 
> > >  test/test-lib.sh |    5 ++---
> > >  1 files changed, 2 insertions(+), 3 deletions(-)
> > > 
> > > diff --git a/test/test-lib.sh b/test/test-lib.sh
> > > index c81c709..c0fe037 100755
> > > --- a/test/test-lib.sh
> > > +++ b/test/test-lib.sh
> > > @@ -50,8 +50,6 @@ TZ=UTC
> > >  TERM=dumb
> > >  export LANG LC_ALL PAGER TERM TZ
> > >  GIT_TEST_CMP=${GIT_TEST_CMP:-diff -u}
> > > -export SCREENRC=/dev/null
> > > -export SYSSCREENRC=/dev/null
> > >  
> > >  # Protect ourselves from common misconfiguration to export
> > >  # CDPATH into the environment
> > > @@ -844,7 +842,8 @@ test_emacs () {
> > >  	if [ -z "$EMACS_SERVER" ]; then
> > >  		EMACS_SERVER="notmuch-test-suite-$$"
> > >  		# start a detached screen session with an emacs server
> > > -		screen -S "$EMACS_SERVER" -d -m "$TMP_DIRECTORY/run_emacs" \
> > > +		TERM=xterm dtach -n "$TMP_DIRECTORY/dtach-emacs-socket-$$" \
> > 
> > It's worth noting that $TERM is only set to "xterm" to deceive `dtach',
> > so thankfully, xterm (and all its deps) aren't actually required to run
> > the tests.
> 
> Actually, emacs SIGSEGVs if there is unsuitable TERM in use. I don't know
> what it originally was.
> 

Hmm, I may have jumped to conclusions there.  I'm glad the part about
being spared from xterm and friends was right, though... :)

> > 
> > > +			"$TMP_DIRECTORY/run_emacs" \
> > >  			--no-window-system \
> > >  			--eval "(setq server-name \"$EMACS_SERVER\")" \
> > >  			--eval '(server-start)' \
> > > -- 
> > > 1.7.7.1
> > > 
> > > _______________________________________________
> > > notmuch mailing list
> > > notmuch@notmuchmail.org
> > > http://notmuchmail.org/mailman/listinfo/notmuch
> > 
> > 
> > Peace
> > 
> > -- 
> > Pieter
> > 
> > [1] @ commit 749abb74
> > [2] @ commit 760b311b
> > [3] @ commit 760b311b + id:"1320963737-1666-1-git-send-email-jrollins@finestructure.net"
> > _______________________________________________
> > notmuch mailing list
> > notmuch@notmuchmail.org
> > http://notmuchmail.org/mailman/listinfo/notmuch
> > 
> 
> Tomi


Peace

-- 
Pieter