check for invalid utf-8, and toss it back to avoid crashes

Since ikiwiki uses open :utf8, perl assumes that files contain valid utf-8.
If it turns out to be malformed it may later crash while processing strings
read from them, with 'Malformed UTF-8 character (fatal)'.

As at least a quick fix, use utf8::valid as soon as data is read, and if
it's not valid, call encode_utf8 on the string, thus clearing the utf-8
flag. This may cause follow-on encoding problems, but will avoid this
crash, and the input file was broken anyway, so GIGO is a reasonable
response. (I looked at calling decode_utf8 after, but it seemed to cause
more trouble than it was worth. BTW, use open ':encoding(utf8)' avaoids
this problem, but the corrupted data later causes Storable to crash when
writing the index.)

This is a quick fix, clearly imperfect:
- It might be better to explicitly call decode_utf8 when reading files,
  rather than using the IO layer.
- Data read other than by readfile() can still sneak in bad utf-8. While
  ikiwiki does very little file input not using it, stdin for the CGI
  would be one way.

diff --git a/IkiWiki.pm b/IkiWiki.pm
index 5e21e7090d09693a0f835d90145df8504142b367..735dc97b14f32dce588410e00b9063342054a668 100644 (file)
--- a/IkiWiki.pm
+++ b/IkiWiki.pm
@@ -721,6 +721,10 @@ sub readfile ($;$$) { #{{{
        binmode($in) if ($binary);
        return \*$in if $wantfd;
        my $ret=<$in>;
+       # check for invalid utf-8, and toss it back to avoid crashes
+       if (! utf8::valid($ret)) {
+               $ret=encode_utf8($ret);
+       }
        close $in || error("failed to read $file: $!");
        return $ret;
 } #}}}

diff --git a/debian/changelog b/debian/changelog
index 99f35482e942e6d75269f89eb00a08aa139f83c2..3838a3e902e8c92717d4f73fffd9babe34b2bf63 100644 (file)
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,9 @@
+ikiwiki (2.70) UNRELEASED; urgency=low
+
+  * Avoid crash on malformed utf-8 discovered by intrigeri.
+
+ -- Joey Hess <joeyh@debian.org>  Wed, 12 Nov 2008 17:30:33 -0500
+
 ikiwiki (2.69) unstable; urgency=low
 
   * Avoid multiple ikiwiki cgi processes piling up, eating all memory,

diff --git a/doc/security.mdwn b/doc/security.mdwn
index 0841abf495a3d62580bfa849bbc8bafe13d58497..1bc7b9e60a6c626826403bb2d4a12368992e7de2 100644 (file)
--- a/doc/security.mdwn
+++ b/doc/security.mdwn
@@ -407,3 +407,12 @@ discovered on 30 May 2008 and fixed the same day. ([[!cve CVE-2008-0169]])
 
 I recommend upgrading to 2.48 immediatly if your wiki allows both password
 and openid logins.
+
+## Malformed UTF-8 DOS
+
+Feeding ikiwiki page sources containing certian forms of malformed UTF-8
+can cause it to crash. This can potentially be used for a denial of service
+attack.
+
+intrigeri discovered this problem on 12 Nov 2008 and a patch put in place
+later that day.