Date: Wed, 19 Feb 2014 14:54:57 -0500
From: Austin Clements <amdragon@MIT.EDU>
To: Tomi Ollila <tomi.ollila@iki.fi>
Subject: Re: [PATCH 1/1] emacs: always write emacs/.eldeps when the target is
	remade
Message-ID: <20140219195457.GD13537@mit.edu>
References: <1390645309-2326-1-git-send-email-tomi.ollila@iki.fi>
	<20140213071026.GX4375@mit.edu> <m2sirn1nik.fsf@guru.guru-group.fi>
	<20140213074944.GY4375@mit.edu> <m2bny9ebvk.fsf@guru.guru-group.fi>
	<20140214215736.GD4375@mit.edu>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20140214215736.GD4375@mit.edu>
User-Agent: Mutt/1.5.21 (2010-09-15)
Cc: notmuch@notmuchmail.org
Precedence: list

Quoth myself on Feb 14 at  4:58 pm:
> Quoth Tomi Ollila on Feb 14 at  9:24 pm:
> > On Thu, Feb 13 2014, Austin Clements <amdragon@MIT.EDU> wrote:
> > 
> > > Quoth Tomi Ollila on Feb 13 at  9:26 am:
> > >> On Thu, Feb 13 2014, Austin Clements <amdragon@MIT.EDU> wrote:
> > >> 
> > >> > Quoth Tomi Ollila on Jan 25 at 12:21 pm:
> > >> >> So that the target is newer than its prerequisites.
> > >> >> ---
> > >> >>  emacs/Makefile.local | 3 +--
> > >> >>  1 file changed, 1 insertion(+), 2 deletions(-)
> > >> >> 
> > >> >> diff --git a/emacs/Makefile.local b/emacs/Makefile.local
> > >> >> index 42bfbd9..d5d402e 100644
> > >> >> --- a/emacs/Makefile.local
> > >> >> +++ b/emacs/Makefile.local
> > >> >> @@ -32,8 +32,7 @@ emacs_bytecode = $(emacs_sources:.el=.elc)
> > >> >>  ifeq ($(HAVE_EMACS),1)
> > >> >>  $(dir)/.eldeps: $(dir)/Makefile.local $(dir)/make-deps.el $(emacs_sources)
> > >> >>  	$(call quiet,EMACS) --directory emacs -batch -l make-deps.el \
> > >> >> -		-f batch-make-deps $(emacs_sources) > $@.tmp && \
> > >> >> -		(cmp -s $@.tmp $@ || mv $@.tmp $@)
> > >> >> +		-f batch-make-deps $(emacs_sources) > $@.tmp && mv $@.tmp $@
> > >> >>  -include $(dir)/.eldeps
> > >> >>  endif
> > >> >>  CLEAN+=$(dir)/.eldeps $(dir)/.eldeps.tmp
> > >> >
> > >> > Is this just so the rule doesn't get run again on the next make
> > >> > invocation (unless, of course, a dependent changed)?
> > >> 
> > >> Basically yes. I did multiple builds in rapid succession when developing
> > >> something and this thing confused me quite a lot in the beginning...
> > >> 
> > >> > Interestingly, if any of the dependents have changed, but in ways that
> > >> > don't affect .eldeps, this change will make the build more expensive
> > >> > because it will trigger a make restart after .eldeps is updated.
> > >> 
> > >> I wondered what was the reason for the recipe you've chosen here, has
> > >> it something to do with inodes or something ;) (and were going to ask
> > >> an alternative to touch (or even : > $@) the target...
> > >> 
> > >> ... but now I understand. E.g. change in notmuch-lib.el will cause *all* 
> > >> .el files to be re-bytecompiled after this change. 
> > >
> > > This is unrelated to the cmp in the recipe.  A change in
> > > notmuch-lib.el *must* cause almost all .el files to be recompiled
> > > because they almost all require it and may use macros from it.  If you
> > > touch, e.g., notmuch-hello.el, you'll see that very little is rebuilt
> > > (with or without this patch).
> > >
> > >> An alternative to this patch would be adding a message to the rule
> > >> which informs user to touch .eldeps to avoid re-doing .eldeps if
> > >> that irritates one :D
> > >> 
> > >> something like:
> > >> 
> > >> (cmp -s $@.tmp $@ && \
> > >>      echo "touch $@ to avoid redoing this target" || mv $@.tmp $@)
> > >
> > > I think this patch is good as it is; I was just pointing out that the
> > > change also has a performance drawback in some situations.  (Knowing
> > > make, there's probably *some* way to get the best of both worlds and
> > > it's probably really ugly.)
> > 
> > Ah, now I understand, because emacs/Makefile.local includes emacs/.eldeps
> > when it notices mtime of emacs/.eldeps changes it restarts itself...
> > 
> > ... to compare with current behaviour:
> > 
> > $ make
> > ...
> > $ touch emacs/notmuch.el
> > $ make
> > ...
> > $ make
> > Use "make V=1" to see the verbose compile lines.
> > EMACS emacs/.eldeps
> > make: Nothing to be done for `all'.
> > $ make
> > Use "make V=1" to see the verbose compile lines.
> > EMACS emacs/.eldeps
> > make: Nothing to be done for `all'.
> > $ make
> > Use "make V=1" to see the verbose compile lines.
> > EMACS emacs/.eldeps
> > make: Nothing to be done for `all'.
> > 
> > It will rerun emacs -L emacs -batch -l make-deps.el -f batch-make-deps ...
> > over and over again...
> > 
> > I thought I figured out a solution involving .eldeps.stamp file but
> > that turned out to be no-go... (circular dependency netween .eldeps.stamp
> > and Makefile.local).
> > 
> > So, with this if one edits emacs/* files and keeps building byte-compiled
> > versions often, make will both run EMACS emacs/.eldeps and restarting
> > -- and when moving to edit other files both of the above stops. Without
> > this change after editing emacs/* files EMACS emacs/.eldeps will be
> > executed until it is touched or make clean executed (I wonder how make
> > handles rebuilding emacs/.eldeps while doing make clean ;)
> > 
> > 
> > Tomi
> 
> Here's a way that works, but it's more complicated than your patch and
> doesn't seem to perform any better:
> 
> diff --git a/emacs/Makefile.local b/emacs/Makefile.local
> index 42bfbd9..8a1f1c2 100644
> --- a/emacs/Makefile.local
> +++ b/emacs/Makefile.local
> @@ -29,14 +29,28 @@ emacs_bytecode = $(emacs_sources:.el=.elc)
>  # the byte compiler may load an old .elc file when processing a
>  # "require" or we may fail to rebuild a .elc that depended on a macro
>  # from an updated file.
> +#
> +# We do this in two rules.  .eldeps is the "real" dependencies file
> +# and will be rebuilt if and only if a .el source file changes.  We
> +# could include this directly, but then whenever a .el file changed
> +# (and .eldeps's mtime changed), make would restart immediately after
> +# the include below.  To avoid this, we create a second file,
> +# .eldeps.x, that we only update when its *contents* differ from
> +# .eldeps.  .eldeps will often have an mtime older than .eldeps.x (any
> +# .el file change that does not change dependencies will cause this),
> +# so the .eldeps.x rule will often run, but it executes quickly and
> +# quietly and will not trigger a make restart unless the dependencies
> +# actually change.

Here's a version of this comment that gets to the point directly:

# We could include .eldeps directly, but that would cause a make
# restart whenever any .el file was modified, even if dependencies
# didn't change, because the mtime of .eldeps will change.  Instead,
# we include a second file, .eldeps.x, which we ensure always has the
# same content as .eldeps, but its mtime only changes when dependency
# information changes, in which case a make restart is necessary
# anyway.

>  ifeq ($(HAVE_EMACS),1)
>  $(dir)/.eldeps: $(dir)/Makefile.local $(dir)/make-deps.el $(emacs_sources)
>  	$(call quiet,EMACS) --directory emacs -batch -l make-deps.el \
>  		-f batch-make-deps $(emacs_sources) > $@.tmp && \
> -		(cmp -s $@.tmp $@ || mv $@.tmp $@)
> --include $(dir)/.eldeps
> +		mv $@.tmp $@
> +$(dir)/.eldeps.x: $(dir)/.eldeps
> +	@cmp -s $^ $@ || cp $^ $@
> +-include $(dir)/.eldeps.x
>  endif
> -CLEAN+=$(dir)/.eldeps $(dir)/.eldeps.tmp
> +CLEAN+=$(dir)/.eldeps $(dir)/.eldeps.tmp $(dir)/.eldeps.x
>  
>  ifeq ($(HAVE_EMACS),1)
>  %.elc: %.el $(global_deps)