From: Armin Ronacher Date: Tue, 27 Mar 2007 19:31:24 +0000 (+0200) Subject: [svn] reworked the jinja escaping system, removed getattr concatenating and documente... X-Git-Tag: 2.0rc1~392 X-Git-Url: http://git.tremily.us/?a=commitdiff_plain;h=ae16fd01b33f96ff6a4661d33b91e48c3bba235d;p=jinja2.git [svn] reworked the jinja escaping system, removed getattr concatenating and documented internal jinja functions. --HG-- branch : trunk --- diff --git a/docs/src/contextenv.txt b/docs/src/contextenv.txt index bdf2228..f362a64 100644 --- a/docs/src/contextenv.txt +++ b/docs/src/contextenv.txt @@ -57,6 +57,9 @@ in the template evaluation code you may want to override: unicode. The filters for the names are stored on ``self.filters`` in a dict. Missing filters should raise a `FilterNotFound` exception. + **Warning** this is a jinja internal method. The actual implementation + and function signature might change. + **def** `perform_test` *(self, context, testname, args, value, invert)*: Like `apply_filters` you usually don't override this one. It's the @@ -69,10 +72,16 @@ in the template evaluation code you may want to override: Missing tests should raise a `TestNotFound` exception. -**def** `get_attribute` *(self, obj, attributes)*: + **Warning** this is a jinja internal method. The actual implementation + and function signature might change. + +**def** `get_attribute` *(self, obj, attribute)*: + + Get `attribute` from the object provided. The default implementation + performs security tests. - Get `attributes` from the object provided. The default implementation - performs security tests for each attribute. + **Warning** this is a jinja internal method. The actual implementation + and function signature might change. **def** `call_function` *(self, f, context, args, kwargs, dyn_args, dyn_kwargs)*: @@ -83,13 +92,22 @@ in the template evaluation code you may want to override: The default implementation performs some security checks. + **Warning** this is a jinja internal method. The actual implementation + and function signature might change. + **def** `call_function_simple` *(self, f, context)*: Like `call_function` but without arguments. -**def** `finish_var` *(self, value)*: + **Warning** this is a jinja internal method. The actual implementation + and function signature might change. + +**def** `finish_var` *(self, value, ctx)*: Postprocess a variable before it's sent to the template. + + **Warning** this is a jinja internal method. The actual implementation + and function signature might change. .. admonition:: Note diff --git a/docs/src/devintro.txt b/docs/src/devintro.txt index f843834..ca9859b 100644 --- a/docs/src/devintro.txt +++ b/docs/src/devintro.txt @@ -51,6 +51,18 @@ Here the possible initialization parameters: escaping methods. If you don't want to escape a string you have to wrap it in a ``Markup`` object from the ``jinja.datastructure`` module. + If `auto_escape` is ``True`` there will be also + a ``Markup`` object in the template namespace + to define partial html fragments. Note that we do + not recomment this feature, see also the comment + below. +`default_filters` list of tuples in the form (``filter_name``, + ``arguments``) where ``filter_name`` is the + name of a registered filter and ``arguments`` + a tuple with the filter arguments. The filters + specified here will always be applied when + printing data to the template. + *new in jinja 1.1* `template_charset` The charset of the templates. Defaults to ``'utf-8'``. `charset` Charset of all string input data. Defaults @@ -87,6 +99,22 @@ addition to the initialization values: There are also some internal functions on the environment used by the template evaluation code to keep it sandboxed. +Automatic Escaping +================== + +Jinja provides a way for automatic escaping, but we do not recommend using it. +Because Jinja was designed as multi purpose template engine there are some +issues with automatic escaping. For example filters don't deal with markup +data. Also you can easily bypass the automatic escaping so it's not something +you can expect to "just work". Also there is a huge overhead when escaping +everything. + +The best idea is to think about which data already contains html, which will +probably contain (eg: every user input, etc) etc. And live with self escaping. + +That's usually a much better idea. + + Loading Templates From Files ============================ diff --git a/jinja/__init__.py b/jinja/__init__.py index e87d195..8178f5c 100644 --- a/jinja/__init__.py +++ b/jinja/__init__.py @@ -1,12 +1,70 @@ # -*- coding: utf-8 -*- """ - Jinja Sandboxed Template Engine - ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + jinja + ~~~~~ + + Jinja is a `sandboxed`_ template engine written in pure Python. It + provides a `Django`_ like non-XML syntax and compiles templates into + executable python code. It's basically a combination of Django templates + and python code. + + Nutshell + -------- + + Here a small example of a Jinja template:: + + {% extends 'base.html' %} + {% block title %}Memberlist{% endblock %} + {% block content %} + + {% endblock %} + + Philosophy + ---------- + + Application logic is for the controller but don't try to make the life + for the template designer too hard by giving him too few functionality. + + For more informations visit the new `jinja webpage`_ and `documentation`_. + + Note + ---- + + This is the Jinja 1.0 release which is completely incompatible with the + old "pre 1.0" branch. The old branch will still receive security updates + and bugfixes but the 1.0 branch will be the only version that receives + support. + + If you have an application that uses Jinja 0.9 and won't be updated in + the near future the best idea is to ship a Jinja 0.9 checkout together + with the application. + + The `Jinja trunk`_ is installable via `easy_install` with ``easy_install + Jinja==dev``. + + .. _sandboxed: http://en.wikipedia.org/wiki/Sandbox_(computer_security) + .. _Django: http://www.djangoproject.com/ + .. _jinja webpage: http://jinja.pocoo.org/ + .. _documentation: http://jinja.pocoo.org/documentation/index.html + .. _Jinja trunk: http://trac.pocoo.org/repos/jinja/trunk#egg=Jinja-dev + :copyright: 2007 by Armin Ronacher. :license: BSD, see LICENSE for more details. """ + from jinja.environment import Environment +from jinja.datastructure import Markup +from jinja.utils import from_string from jinja.plugin import jinja_plugin_factory as template_plugin_factory -from jinja.loaders import * -from_string = Environment().from_string +from jinja.loaders import FileSystemLoader, PackageLoader, DictLoader, \ + ChoiceLoader, FunctionLoader + + +__all__ = ['Environment', 'Markup', 'FileSystemLoader', 'PackageLoader', + 'DictLoader', 'ChoiceLoader', 'FunctionLoader', 'from_string', + 'template_plugin_factory'] diff --git a/jinja/datastructure.py b/jinja/datastructure.py index a4c6388..3b8448e 100644 --- a/jinja/datastructure.py +++ b/jinja/datastructure.py @@ -16,6 +16,9 @@ except NameError: from sets import Set as set from jinja.exceptions import TemplateSyntaxError, TemplateRuntimeError +from cgi import escape + +_known_safe_types = set([int, long, float]) def contextcallable(f): @@ -122,12 +125,18 @@ class Deferred(object): class Markup(unicode): """ - Mark a string as safe for XML. If the environment uses the - auto_escape option values marked as `Markup` aren't escaped. + Compatibility for Pylons and probably some other frameworks. """ - def __repr__(self): - return 'Markup(%s)' % unicode.__repr__(self) + def __html__(self): + return unicode(self) + + +class TemplateData(Markup): + """ + Subclass of unicode to mark objects that are coming from the + template. The autoescape filter can use that. + """ class Context(object): diff --git a/jinja/environment.py b/jinja/environment.py index 0b2b13a..aaec799 100644 --- a/jinja/environment.py +++ b/jinja/environment.py @@ -12,9 +12,9 @@ import re from jinja.lexer import Lexer from jinja.parser import Parser from jinja.loaders import LoaderWrapper -from jinja.datastructure import Undefined, Context, Markup, FakeTranslator +from jinja.datastructure import Undefined, Markup, Context, FakeTranslator from jinja.utils import escape, collect_translations -from jinja.exceptions import FilterNotFound, TestNotFound, SecurityException +from jinja.exceptions import FilterNotFound, TestNotFound from jinja.defaults import DEFAULT_FILTERS, DEFAULT_TESTS, DEFAULT_NAMESPACE @@ -32,6 +32,7 @@ class Environment(object): comment_end_string='#}', trim_blocks=False, auto_escape=False, + default_filters=None, template_charset='utf-8', charset='utf-8', namespace=None, @@ -55,13 +56,18 @@ class Environment(object): self.loader = loader self.filters = filters is None and DEFAULT_FILTERS.copy() or filters self.tests = tests is None and DEFAULT_TESTS.copy() or tests - self.auto_escape = auto_escape + self.default_filters = default_filters or [] self.context_class = context_class # global namespace self.globals = namespace is None and DEFAULT_NAMESPACE.copy() \ or namespace + # jinja 1.0 compatibility + if auto_escape: + self.default_filters.append(('escape', (True,))) + self.globals['Markup'] = Markup + # create lexer self.lexer = Lexer(self) @@ -151,22 +157,22 @@ class Environment(object): return not rv return bool(rv) - def get_attribute(self, obj, attributes): + def get_attribute(self, obj, name): """ - Get some attributes from an object. + Get one attribute from an object. """ - node = obj - for name in attributes: - try: - node = node[name] - except (TypeError, KeyError, IndexError): - if not hasattr(node, name): - return Undefined - r = getattr(obj, 'jinja_allowed_attributes', None) - if r is not None and name not in r: - raise SecurityException('unsafe attributed %r accessed' % name) - node = getattr(node, name) - return node + try: + return obj[name] + except (TypeError, KeyError, IndexError): + if name[:2] == name[-2:] == '__': + return Undefined + if not hasattr(obj, name): + return Undefined + r = getattr(obj, 'jinja_allowed_attributes', None) + if r is not None and name not in r: + return Undefined + return getattr(obj, name) + return Undefined def call_function(self, f, context, args, kwargs, dyn_args, dyn_kwargs): """ @@ -179,7 +185,7 @@ class Environment(object): kwargs.update(dyn_kwargs) if getattr(f, 'jinja_unsafe_call', False) or \ getattr(f, 'alters_data', False): - raise SecurityException('unsafe function %r called' % f.__name__) + return Undefined if getattr(f, 'jinja_context_callable', False): args = (self, context) + args return f(*args, **kwargs) @@ -191,12 +197,12 @@ class Environment(object): """ if getattr(f, 'jinja_unsafe_call', False) or \ getattr(f, 'alters_data', False): - raise SecurityException('unsafe function %r called' % f.__name__) + return Undefined if getattr(f, 'jinja_context_callable', False): return f(self, context) return f() - def finish_var(self, value): + def finish_var(self, value, ctx): """ As long as no write_var function is passed to the template evaluator the source generated by the python translator will @@ -204,10 +210,8 @@ class Environment(object): """ if value is Undefined or value is None: return u'' - elif isinstance(value, (int, float, Markup, bool)): - return unicode(value) - elif not isinstance(value, unicode): - value = self.to_unicode(value) - if self.auto_escape: - return escape(value, True) - return value + val = self.to_unicode(value) + # apply default filters + if self.default_filters: + val = self.apply_filters(val, ctx, self.default_filters) + return val diff --git a/jinja/exceptions.py b/jinja/exceptions.py index 1994467..e8e6c47 100644 --- a/jinja/exceptions.py +++ b/jinja/exceptions.py @@ -17,6 +17,8 @@ class TemplateError(RuntimeError): class SecurityException(TemplateError): """ Raise if the template designer tried to do something dangerous. + + Not used any more. exists for backwards compatibility. """ diff --git a/jinja/filters.py b/jinja/filters.py index be32113..76dd1be 100644 --- a/jinja/filters.py +++ b/jinja/filters.py @@ -10,8 +10,8 @@ """ from random import choice from urllib import urlencode, quote -from jinja.utils import escape, urlize -from jinja.datastructure import Undefined +from jinja.utils import urlize, escape +from jinja.datastructure import Undefined, Markup, TemplateData from jinja.exceptions import FilterArgumentError @@ -68,12 +68,12 @@ def do_replace(s, old, new, count=None): """ if not isinstance(old, basestring) or \ not isinstance(new, basestring): - raise FilterArgumentException('the replace filter requires ' - 'string replacement arguments') + raise FilterArgumentError('the replace filter requires ' + 'string replacement arguments') elif not isinstance(count, (int, long)): - raise FilterArgumentException('the count parameter of the ' - 'replace filter requires ' - 'an integer') + raise FilterArgumentError('the count parameter of the ' + 'replace filter requires ' + 'an integer') if count is None: return s.replace(old, new) return s.replace(old, new, count) @@ -96,7 +96,7 @@ def do_lower(s): do_lower = stringfilter(do_lower) -def do_escape(s, attribute=False): +def do_escape(attribute=False): """ XML escape ``&``, ``<``, and ``>`` in a string of data. If the optional parameter is `true` this filter will also convert @@ -105,8 +105,13 @@ def do_escape(s, attribute=False): This method will have no effect it the value is already escaped. """ - return escape(s, attribute) -do_escape = stringfilter(do_escape) + def wrapped(env, context, s): + if isinstance(s, TemplateData): + return s + elif hasattr(s, '__html__'): + return s.__html__() + return escape(env.to_unicode(s), attribute) + return wrapped def do_capitalize(s): diff --git a/jinja/loaders.py b/jinja/loaders.py index 2acbcf5..b11e9f9 100644 --- a/jinja/loaders.py +++ b/jinja/loaders.py @@ -25,6 +25,7 @@ except ImportError: resource_exists = resource_string = resource_filename = None +#: when updating this, update the listing in the jinja package too __all__ = ['FileSystemLoader', 'PackageLoader', 'DictLoader', 'ChoiceLoader', 'FunctionLoader'] diff --git a/jinja/parser.py b/jinja/parser.py index e447733..b647398 100644 --- a/jinja/parser.py +++ b/jinja/parser.py @@ -266,7 +266,7 @@ class Parser(object): block_name = tokens.pop(0) if block_name[1] != 'name': raise TemplateSyntaxError('expected \'name\', got %r' % - block_name[1], lineno, seilf.filename) + block_name[1], lineno, self.filename) # disallow keywords if not block_name[2].endswith('_'): raise TemplateSyntaxError('illegal use of keyword %r ' diff --git a/jinja/translators/python.py b/jinja/translators/python.py index 1c5de59..407960b 100644 --- a/jinja/translators/python.py +++ b/jinja/translators/python.py @@ -204,8 +204,8 @@ class PythonTranslator(Translator): n.lineno) args.append(self.handle_node(arg)) if n.star_args is not None or n.dstar_args is not None: - raise TemplateSynaxError('*args / **kwargs is not supported ' - 'for filters', n.lineno) + raise TemplateSyntaxError('*args / **kwargs is not supported ' + 'for filters', n.lineno) filters.append('(%r, %s)' % ( n.node.name, _to_tuple(args) @@ -240,7 +240,9 @@ class PythonTranslator(Translator): def translate(self): self.reset() - return self.handle_node(self.node) + rv = self.handle_node(self.node) + print rv + return rv # -- jinja nodes @@ -467,9 +469,9 @@ class PythonTranslator(Translator): self.indention -= 1 if hardcoded: - write('yield finish_var(context.current[%r].cycle())' % name) + write('yield finish_var(context.current[%r].cycle(), context)' % name) else: - write('yield finish_var(context.current[%r].cycle(%s))' % ( + write('yield finish_var(context.current[%r].cycle(%s), context)' % ( name, self.handle_node(node.seq) )) @@ -483,7 +485,7 @@ class PythonTranslator(Translator): nodeinfo = self.nodeinfo(node) or '' if nodeinfo: nodeinfo = self.indent(nodeinfo) + '\n' - return nodeinfo + self.indent('yield finish_var(%s)' % + return nodeinfo + self.indent('yield finish_var(%s, context)' % self.handle_node(node.variable)) def handle_macro(self, node): @@ -656,8 +658,8 @@ class PythonTranslator(Translator): n.lineno) args.append(self.handle_node(arg)) if n.star_args is not None or n.dstar_args is not None: - raise TemplateSynaxError('*args / **kwargs is not supported ' - 'for tests', n.lineno) + raise TemplateSyntaxError('*args / **kwargs is not supported ' + 'for tests', n.lineno) else: raise TemplateSyntaxError('is operator requires a test name' ' as operand', node.lineno) @@ -698,27 +700,18 @@ class PythonTranslator(Translator): self.handle_node(node.expr), self.handle_node(node.subs[0]) ) - return 'get_attribute(%s, (%s,))' % ( + return 'get_attribute(%s, %s)' % ( self.handle_node(node.expr), self.handle_node(node.subs[0]) ) def handle_getattr(self, node): """ - Handle hardcoded attribute access. foo.bar + Handle hardcoded attribute access. """ - expr = node.expr - - # chain getattrs for speed reasons - path = [repr(node.attrname)] - while node.expr.__class__ is ast.Getattr: - node = node.expr - path.append(repr(node.attrname)) - path.reverse() - - return 'get_attribute(%s, %s)' % ( + return 'get_attribute(%s, %r)' % ( self.handle_node(node.expr), - _to_tuple(path) + node.attrname ) def handle_ass_tuple(self, node): diff --git a/jinja/utils.py b/jinja/utils.py index 2c6fbd8..34a0e84 100644 --- a/jinja/utils.py +++ b/jinja/utils.py @@ -14,10 +14,11 @@ import re import sys import string +import cgi from types import MethodType, FunctionType from compiler.ast import CallFunc, Name, Const from jinja.nodes import Trans -from jinja.datastructure import Markup, Context +from jinja.datastructure import Context, TemplateData try: from collections import deque @@ -29,18 +30,6 @@ MAX_RANGE = 1000000 _debug_info_re = re.compile(r'^\s*\# DEBUG\(filename=(.*?), lineno=(.*?)\)$') -_escape_pairs = { - '&': '&', - '<': '<', - '>': '>', - '"': '"' -} - -_escape_res = ( - re.compile('(&|<|>|")'), - re.compile('(&|<|>)') -) - _integer_re = re.compile('^(\d+)$') _word_split_re = re.compile(r'(\s+)') @@ -54,13 +43,10 @@ _punctuation_re = re.compile( _simple_email_re = re.compile(r'^\S+@[a-zA-Z0-9._-]+\.[a-zA-Z0-9._-]+$') +#: used by from_string as cache +_from_string_env = None -def escape(x, attribute=False): - """ - Escape an object x. - """ - return Markup(_escape_res[not attribute].sub(lambda m: - _escape_pairs[m.group()], x)) +escape = cgi.escape def urlize(text, trim_url_limit=None, nofollow=False): @@ -108,6 +94,17 @@ def urlize(text, trim_url_limit=None, nofollow=False): return u''.join(words) +def from_string(source): + """ + Create a template from the template source. + """ + global _from_string_env + if _from_string_env is None: + from jinja.environment import Environment + _from_string_env = Environment() + return _from_string_env.from_string(source) + + def debug_context(env, context): """ Use this function in templates to get a printed context. @@ -157,7 +154,7 @@ def buffereater(f): (macros, filter sections etc) """ def wrapped(*args, **kwargs): - return capture_generator(f(*args, **kwargs)) + return TemplateData(capture_generator(f(*args, **kwargs))) return wrapped diff --git a/setup.py b/setup.py index bbab812..631bdb2 100644 --- a/setup.py +++ b/setup.py @@ -1,59 +1,10 @@ # -*- coding: utf-8 -*- -""" -Jinja -===== - -Jinja is a `sandboxed`_ template engine written in pure Python. It provides a -`Django`_ like non-XML syntax and compiles templates into executable python code. -It's basically a combination of Django templates and python code. - -Nutshell --------- - -Here a small example of a Jinja template:: - - {% extends 'base.html' %} - {% block title %}Memberlist{% endblock %} - {% block content %} - - {% endblock %} - -Philosophy ----------- - -Application logic is for the controller but don't try to make the life for the -template designer too hard by giving him too few functionality. - -For more informations visit the new `jinja webpage`_ and `documentation`_. - -Note ----- - -This is the Jinja 1.0 release which is completely incompatible with the old -"pre 1.0" branch. The old branch will still receive security updates and -bugfixes but the 1.0 branch will be the only version that receives support. - -If you have an application that uses Jinja 0.9 and won't be updated in the -near future the best idea is to ship a Jinja 0.9 checkout together with -the application. - -The `Jinja trunk`_ is installable via `easy_install` with ``easy_install -Jinja==dev``. - -.. _sandboxed: http://en.wikipedia.org/wiki/Sandbox_%28computer_security%29 -.. _Django: http://www.djangoproject.com/ -.. _jinja webpage: http://jinja.pocoo.org/ -.. _documentation: http://jinja.pocoo.org/documentation/index.html -.. _Jinja trunk: http://trac.pocoo.org/repos/jinja/trunk#egg=Jinja-dev -""" +import jinja import os import ez_setup ez_setup.use_setuptools() from setuptools import setup +from inspect import getdoc def list_files(path): @@ -74,7 +25,7 @@ setup( author_email = 'armin.ronacher@active-4.com', description = 'A small but fast and easy to use stand-alone template ' 'engine written in pure python.', - long_description = __doc__, + long_description = getdoc(jinja), # jinja is egg safe. But because we distribute the documentation # in form of html and txt files it's a better idea to extract the files zip_safe = False,