Fix issues with combining unicode srcdirs and source files.

A short story:

  Once there was a unicode string, let's call him Srcdir.

  Along came a crufy old File::Find, who went through a tree and pasted each
  of the leaves in turn onto Srcdir. But this 90's relic didn't decode the
  leaves -- despite some of them using unicode! Poor Srcdir, with these
  leaves stuck on him, tainted them with his nice unicode-ness. They didn't
  look like leaves at all, but instead garbage.

In other words, perl's unicode support sucks mightily, and drives
us all to drink and bad storytelling. But we knew that..

So, srcdir is not normally flagged as unicode, because typically it's pure
ascii. And in that case, things work ok; File::Find finds filenames, which
are not yet decoded to unicode, and appends them to the srcdir, and then
decode_utf8 happily converts the whole thing.

But, if the srcdir does contain utf8 characters, that breaks. Or, if a Yaml
setup file is used, Yaml::Syck's implicitunicode sets the unicode flag of
*all* strings, even those containing only ascii. In either case, srcdir
has the unicode flag set; a non-decoded filename is appended, and
decode_utf8 sees the flag and does *nothing*. The result is that the
filename is not decoded, so looks valid and gets skipped.

File::Find only sticks the directory and filenames together in no_chdir
mode .. but we need that mode for security. In order to retain the
security, and avoid the problem, I made it not pass srcdir to File::Find.
Instead, chdir to the srcdir, and pass ".". Since "." is ascii, the problem
is avoided.

Note that it takes care to chdir back to the starting location. Because
the user may have specified relative paths and so staying in the srcdir
might break. A relative path could even be specifed for an underlay dir, so
it chdirs back after each.

diff --git a/IkiWiki/Render.pm b/IkiWiki/Render.pm
index f9fbc801f750419c76ca301dbcaac1ec62d32aa6..0e7aa9a48a2bd5ddb4267ca948e3f6eba6ba121a 100644 (file)
--- a/IkiWiki/Render.pm
+++ b/IkiWiki/Render.pm
@@ -292,12 +292,11 @@ sub find_src_files () {
        eval q{use File::Find};
        error($@) if $@;
 
-       my ($page, $dir, $underlay);
+       my ($page, $underlay);
        my $helper=sub {
                my $file=decode_utf8($_);
-
                return if -l $file || -d _;
-               $file=~s/^\Q$dir\E\/?//;
+               $file=~s/^\Q.\/\E//;
                return if ! length $file;
                $page = pagename($file);
                if (! exists $pagesources{$page} &&
@@ -330,17 +329,27 @@ sub find_src_files () {
                }
        };
 
+       eval q{use Cwd};
+       die $@ if $@;
+       my $origdir=getcwd();
+
+       chdir($config{srcdir}) || die "chdir: $!";
        find({
                no_chdir => 1,
                wanted => $helper,
-       }, $dir=$config{srcdir});
+       }, '.');
+       chdir($origdir) || die "chdir: $!";
+
        $underlay=1;
        foreach (@{$config{underlaydirs}}, $config{underlaydir}) {
+               chdir($_) || die "chdir: $!";
                find({
                        no_chdir => 1,
                        wanted => $helper,
-               }, $dir=$_);
+               }, '.');
+               chdir($origdir) || die "chdir: $!";
        };
+
        return \@files, \%pages;
 }
 

diff --git a/debian/changelog b/debian/changelog
index bddedeafaee8109335ba2a148ab819bb6da1aa0b..16f14b3a11c57694715b51e98942c32073906b67 100644 (file)
--- a/debian/changelog
+++ b/debian/changelog
@@ -16,6 +16,7 @@ ikiwiki (3.20100611) UNRELEASED; urgency=low
   * editpage, comments: Fix broken links in sidebar (due to forcebaseurl).
     (Thanks, privat)
   * calendar: Tune archive_pagespec to only match pages, not other files.
+  * Fix issues with combining unicode srcdirs and source files.
 
  -- Joey Hess <joeyh@debian.org>  Fri, 11 Jun 2010 13:39:15 -0400