- developer friendly traceback is now toggleable
-- silent variable name failure is now toggleable
+- the variable failure is now pluggable by replacing the undefined
+ singleton for an environment instance
- fixed issue with old-style classes not implementing `__getitem__`
(thanks to Axel Böhm for discovering that bug)
- You can now use the environment to just tokenize a template. This can
be useful for syntax highlighting or other purposes.
-- added optional c-implementation of the context baseclass.
+- added optional C-implementation of the context baseclass.
+
+- you can now use optional parentheses around macro defintions. Thus it's
+ possible to write ``{% macro foo(a, b, c) %}`` instead of ``{% macro
+ foo a, b, c %}``.
+
+- additional macro arguments now end up in `varargs`.
- implemented `{% call %}` - unsure if this makes it into the final release.
+- it's not possible to stream templates.
+
Version 1.0
-----------
TODO List for Jinja
===================
-- Improve the context lookup (maybe with an optional C extension)
+1.1:
-- `include` and `extends` should work with dynamic data too. In order to
- support this the blocks should be stored as importable functions in the
- generated source.
+ - Improve the context lookup (maybe with an optional C extension) [DONE]
+ - make Undefined exchangeable [DONE]
+ - implement block.super [DONE]
+ - Implement a `IntrospectionPrinter` that works like pprint but it outputs
+ either plain text or html. It would also have to cover changing names of
+ the special builtins True, False etc to lowercase in order to not
+ confuse people.
+ - decide on `{% call %}`
+ - speed up jinja import
+ - add optional zlib compression of template bytecode
+ - write more unittests!!!!
+
+1.2:
+
+ - `include` and `extends` should work with dynamic data too. In order to
+ support this the blocks should be stored as importable functions in the
+ generated source.
+ - add support for `{% include myfile = 'myfile.html' %}` and give the
+ template designer to access variables and macros defined in the template
+ root or requirements namespace.
--- /dev/null
+=======================
+Context and Environment
+=======================
+
+The two central objects in Jinja are the `Environment` and `Context`. Both
+are designed to be subclassed by applications if they need to extend Jinja.
+
+Environment
+===========
+
+The initialization parameters are already covered in the `Quickstart`_ thus
+not repeated here.
+
+But beside those configurable instance variables there are some functions used
+in the template evaluation code you may want to override:
+
+**def** `parse` *(source, filename)*:
+
+ Parse the sourcecode and return the abstract syntax tree. This tree of
+ nodes is used by the `translators`_ to convert the template into
+ executable source- or bytecode.
+
+**def** `lex` *(source, filename)*:
+
+ Tokenize the given sourcecode and return a generator of tuples in the
+ form ``(lineno, token, value)``. The filename is just used in the
+ exceptions raised.
+
+ **New in Jinja 1.1**
+
+**def** `from_string` *(source)*:
+
+ Load and parse a template source and translate it into eval-able Python
+ code. This code is wrapped within a `Template` class that allows you to
+ render it.
+
+**def** `get_template` *(name)*:
+
+ Load a template from a loader. If the template does not exist, you will
+ get a `jinja.exceptions.TemplateNotFound` exception.
+
+**def** `to_unicode` *(self, value)*:
+
+ Called to convert variables to unicode. Per default this checks if the
+ value is already unicode. If not it's converted to unicode using the
+ charset defined on the environment.
+
+ Also `None` is converted into an empty string per default.
+
+**def** `get_translator` *(self, context)*:
+
+ Return the translator used for i18n. A translator is an object that
+ provides the two functions ``gettext(string)`` and
+ ``ngettext(singular, plural, n)``. Both of those functions have to
+ behave like the `ugettext` and `nugettext` functions described in the
+ python `gettext documentation`_.
+
+ If you don't provide a translator a default one is used to switch
+ between singular and plural forms.
+
+ Have a look at the `i18n`_ section for more information.
+
+**def** `get_translations` *(self, name)*:
+
+ Get the translations for the template `name`. Only works if a loader
+ is present. See the `i18n`_ section for more details.
+
+**def** `get_translations_for_string` *(self, string)*:
+
+ Get the translations for the string `string`. This works also if no
+ loader is present and can be used to lookup translation strings from
+ templates that are loaded from dynamic resources like databases.
+
+**def** `apply_filters` *(self, value, context, filters)*:
+
+ Now this function is a bit tricky and you usually don't have to override
+ it. It's used to apply filters on a value. The Jinja expression
+ ``{{ foo|escape|replace('a', 'b') }}`` calls the function with the
+ value of `foo` as first parameter, the current context as second and
+ a list of filters as third. The list looks like this:
+
+ .. sourcecode:: python
+
+ [('escape', ()), ('replace', (u'a', u'b'))]
+
+ As you can see the filter `escape` is called without arguments whereas
+ `replace` is called with the two literal strings ``a`` and ``b``, both
+ unicode. The filters for the names are stored on ``self.filters`` in a
+ dict. Missing filters should raise a `FilterNotFound` exception.
+
+ **Warning** this is a Jinja internal method. The actual implementation
+ and function signature might change.
+
+**def** `perform_test` *(self, context, testname, args, value, invert)*:
+
+ Like `apply_filters` you usually don't override this one. It's the
+ callback function for tests (``foo is bar`` / ``foo is not bar``).
+
+ The first parameter is the current contex, the second the name of
+ the test to perform. the third a tuple of arguments, the fourth is
+ the value to test. The last one is `True` if the test was performed
+ with the `is not` operator, `False` if with the `is` operator.
+
+ Missing tests should raise a `TestNotFound` exception.
+
+ **Warning** this is a Jinja internal method. The actual implementation
+ and function signature might change.
+
+**def** `get_attribute` *(self, obj, attribute)*:
+
+ Get `attribute` from the object provided. The default implementation
+ performs security tests.
+
+ **Warning** this is a Jinja internal method. The actual implementation
+ and function signature might change.
+
+**def** `get_attributes` *(self, obj, attributes)*:
+
+ Get some attributes from the object. If `attributes` is an empty
+ sequence the object itself is returned unchanged.
+
+**def** `call_function` *(self, f, context, args, kwargs, dyn_args, dyn_kwargs)*:
+
+ Call a function `f` with the arguments `args`, `kwargs`, `dyn_args` and
+ `dyn_kwargs` where `args` is a tuple and `kwargs` a dict. If `dyn_args`
+ is not `None` you have to add it to the arguments, if `dyn_kwargs` is
+ not `None` you have to update the `kwargs` with it.
+
+ The default implementation performs some security checks.
+
+ **Warning** this is a Jinja internal method. The actual implementation
+ and function signature might change.
+
+**def** `call_function_simple` *(self, f, context)*:
+
+ Like `call_function` but without arguments.
+
+ **Warning** this is a Jinja internal method. The actual implementation
+ and function signature might change.
+
+**def** `finish_var` *(self, value, ctx)*:
+
+ Postprocess a variable before it's sent to the template.
+
+ **Warning** this is a Jinja internal method. The actual implementation
+ and function signature might change.
+
+.. admonition:: Note
+
+ The Enviornment class is defined in `jinja.environment.Environment`
+ but imported into the `jinja` package because it's often used.
+
+Context
+=======
+
+Jinja wraps the variables passed to the template in a special class called a
+context. This context supports variables on multiple layers and lazy (deferred)
+objects. Often your application has a request object, database connection
+object or something similar you want to access in filters, functions etc.
+
+The default context object is defined in `jinja.datastructure`. If you want
+to provide your own context object always subclass the default one. This
+ensures that the class continues working after Jinja upgrades.
+
+Beacause of that you can easily subclass a context to add additional variables
+or to change the way it behaves.
+
+**def** `pop` *(self)*:
+
+ Pop the outermost layer and return it.
+
+**def** `push` *(self, data=None)*:
+
+ Push a dict to the stack or an empty layer.
+
+ Has to return the pushed object.
+
+**def** `to_dict` *(self)*:
+
+ Flatten the context and convert it into a dict.
+
+**def** `__getitem__` *(self, name)*:
+
+ Resolve an item. Per default this also resolves `Deferred` objects.
+
+**def** `__setitem__` *(self, name, value)*:
+
+ Set an item in the outermost layer.
+
+**def** `__delitem__` *(self, name)*:
+
+ Delete an item in the outermost layer. Do not raise exceptions if
+ the value does not exist.
+
+**def** `__contains__` *(self, name)*:
+
+ Return `True` if `name` exists in the context.
+
+**attribute** `cache`:
+
+ The cache is a dict which can be used by filters, test functions
+ and global objects to cache data. It's also used by the environment
+ to cache often used tests and filters.
+
+**attribute** `translate_func`:
+
+ This property is created on first access and returns a translation
+ function used by the rendering process to translate strings with the
+ translator defined on the environment.
+
+.. admonition:: Note
+
+ The context uses a stack of dicts internally to represent the
+ layers of variables. It contains at least 3 levels available on
+ the context with some attributes. Those are:
+
+ `globals`:
+
+ The reference to the global namespace of the environment.
+ It's the lowest namespace on the stack and thus immutable
+
+ `initial`:
+
+ The initial namespace. Contains the values passed to the
+ context in the render function. It also contains the resolved
+ deferred values for bot the `initial` and the `globals`
+ namespace.
+
+ `current`:
+
+ The reference to the current active namespace. When the
+ context is initialized this automatically points to an
+ empty namespace.
+
+ The number of layers on the stack are theoretically unlimited.
+ Some elements in the template language like loops, blocks,
+ macros and others push and pop the layer on entering and leaving
+ the section.
+
+ This is done in order to keep the namespace clean.
+
+ Note that since Jinja 1.1 the context object is a subclass of the
+ `BaseContext`, a much simpler class that just implements a stack
+ like namespace for python. If the `_speedups` extension was
+ compiled for jinja the base class will be
+ `jinja._speedups.BaseContext` otherwise `jinja._native.BaseContext`.
+
+ Since you cannot reproduce completely the same semantics in python
+ and the C API there are some things you should keep in mind:
+
+ - The `stack` attribute of the context maps to the real layers
+ on the stack, thus you can modify the items but the list as
+ such is meant to be read only.
+
+ - `globals`, `current` and `initial` are read only attributes that
+ map to layers on the stack which you can of course modify.
+
+
+Exceptions
+==========
+
+During parsing and evaluation Jinja raises a couple of Jinja specific
+exceptions. All of those exceptions are defined in the `jinja.exceptions`
+module and are subclasses of the `TemplateError` class defined there.
+
+Here a list of exceptions that could occur:
+
+`SecurityException`:
+
+ An exception that is raised if the template tried to access something
+ it should not access. In the default configuration this exception
+ will get caught in the Jinja rendering process and silenced.
+
+ If however the environment is configured to not silently fail it
+ could happen that this exception reaches the application.
+
+`FilterNotFound`:
+
+ Raised if the template tried to apply a filter that does not exist.
+ Since this exception is a subclass of `KeyError` too you can catch
+ it this way too.
+
+`FilterArgumentError`:
+
+ Raised if the filter received an argument that it couldn't handle.
+ It's a subclass of `TypeError` too so you can catch it this way too.
+
+`TestNotFound`:
+
+ Raised if the template tried to perform a test that does not exist.
+ Since this exception is a subclass of `KeyError` too you can catch
+ it this way too.
+
+`TestArgumentError`:
+
+ Raised if a test function received an argument that it couldn't handle.
+ It's a subclass of `TypeError` too so you can catch it this way too.
+
+`TemplateNotFound`:
+
+ Raised if a template does not exist. Subclass of `IOError` too.
+
+`TemplateSyntaxError`:
+
+ Subclass of `SyntaxError` and used to indicate an syntax error.
+
+`TemplateRuntimeError`:
+
+ Generic runtime error exception which can occour at various places.
+
+
+.. _i18n: i18n.txt
+.. _translators: translators.txt
+.. _Quickstart: devintro.txt
+.. _gettext documentation: http://docs.python.org/lib/module-gettext.html
*new in Jinja 1.1*
-`flush`
-
- Jinja 1.1 includes a new stream interface which can be used to speed
- up the generation of big templates by streaming them to the client.
- Per default Jinja flushes after 40 actions automatically. If you
- want to force a flush at a special point you can use `flush()` to
- do so. (eg, end of a loop iteration etc)
-
- .. sourcecode:: jinja
-
- {% for items in seq %}
- {% for item in items %}
- <li>{{ item|e }}</li>
- {% endfor %}
- {{ flush() }}
- {% endfor %}
-
- *new in Jinja 1.1*
-
Loops
=====
{{ show_dialog('Warning', 'something went wrong i guess') }}
+*Improvements in Jinja 1.1*:
+
+ Starting with Jinja 1.1 it's possible to use optional parentheses
+ around the macro arguments:
+
+ .. sourcecode:: html+jinja
+
+ {% macro foo(a, b) %}
+ ...
+ {% endmacro %}
+
+ Additionally extra arguments passed to the macro end up in the
+ special variable `varargs`. So you can have a macro like this:
+
+ .. sourcecode:: html+jinja
+
+ {% macro make_list() %}
+ {% if varargs %}
+ <ul>
+ {% for item in varargs %}
+ <li>{{ item|e }}</li>
+ {% endfor %}
+ </ul>
+ {% endif %}
+ {% endmacro %}
+
+ {{ make_list("John", "Jane", "Marcus", "Heinrich") }}
+
+ If a macro parameter is called `varargs` the additional extra
+ arguments are not accessible.
+
+
Inheritance
===========
Defined macros appear on the context as variables. Because of this, they are
affected by the scoping too. A macro defined inside of a macro is just available
-in those two macros (the macro itself and the macro it's defined in). For `set`
-and `macro` two additional rules exist: If a macro is defined in an extended
-template but outside of a visible block (thus outside of any block) will be
-available in all blocks below. This allows you to use `include` statements to
-load often used macros at once.
+in those two macros (the macro itself and the macro it's defined in).
+
+Template Globals
+================
+
+A special threatment exists for template code outside of visible blocks in
+child templates. This code will be executed **before** the layout template
+code. Thus it can be used to propagate values back to the layout template or
+import macros from templates for rendering.
+
+Such code can output data but it won't appear in the final rendering. So no
+additional whitespace will pollute the template.
+
+Because this code is executed before the actual layout template code it's
+possible that the layout code overrides some of those variables. Usually
+this is not a problem because of different variable names but it can be
+a problem if you plan to specify default values.
+
+In that case you have to test if the variable is not defined before setting
+it:
+
+.. sourcecode:: jinja
+
+ {% if not page_title %}
+ {% set page_title = 'Default Page Title' %}
+ {% endif %}
+
+You can of course also use the `|default` filter.
+
+.. admonition:: Explanation
+
+ This template stored as `a.html`:
+
+ .. sourcecode:: html+jinja
+
+ <title>{{ title|default('Untitled') }}</title>
+ <body>{% block body %}{% endblock %}
+
+ ...and this child template stored as `b.html`:
+
+ .. sourcecode:: html+jinja
+
+ {% extends 'a.html' %}
+ {% include 'macros.tmpl' %}
+ {% set title = 'My Page' %}
+ {% block body %}{{ wrap(42) }}{% endblock %}
+
+ ...and this code in `macros.tmpl`:
+
+ .. sourcecode:: html+jinja
+
+ {% macro wrap(text) %}
+ [{{ text }}]
+ {% endmacro %}
+
+ ..will translate to something with the same semantics as this (just
+ that the value is not stored in a variable):
+
+ .. sourcecode:: html+jinja
+
+ {% filter capture('captured', true) %}
+ {% macro wrap(text) %}
+ [{{ text }}]
+ {% endmacro %}
+ {% set title='My Page' %}
+ {% endfilter %}
+ <title>{{ title|default('Untitled') }}</title>
+ <body>
+ {{ wrap(42) }}
+ </body>
+
+.. admonition:: Note
+
+ This implementation was improved in Jinja 1.1. In Jinja 1.0 blocks that
+ were not top-level were not propagated to the layout template. This made
+ it impossible to use conditional expressions for inclusion in non root
+ templates.
+
Undefined Variables
===================
undefined variables raise an exception. This is different in Jinja. There is a
special value called `undefined` that represents values that do not exist.
-This special variable works complete different from any variables you maybe
-know. If you print it using ``{{ variable }}`` it will not appear because it's
-literally empty. If you try to iterate over it, it will work. But no items
-are returned. Comparing this value to any other value results in `false`.
-Even if you compare it to itself:
+Depending on the configuration it will behave different.
+
+In order to check if a value is defined you can use the `defined` test:
.. sourcecode:: jinja
- {{ undefined == undefined }}
- will return false. Not even undefined is undefined :)
- Use `is defined` / `is not defined`:
+ {{ myvariable is not defined }}
+ will return true if the variable does not exist.
- {{ undefined is not defined }}
- will return true.
+`SilentUndefined`:
-There are also some additional rules regarding this special value. Any
-mathematical operators (``+``, ``-``, ``*``, ``/``) return the operand
-as result:
+ The silent `undefined` is the default behavior. The `undefined` object works
+ complete different from any variables you maybe know. If you print it
+ using ``{{ variable }}`` it will not appear because it's literally empty.
+ If you try to iterate over it, it will work. But no items are returned.
-.. sourcecode:: jinja
+ In order to check if a value is defined you can use the `defined` test:
+
+ There are also some additional rules regarding this special value. Any
+ mathematical operators (``+``, ``-``, ``*``, ``/``) return the operand
+ as result:
- {{ undefined + "foo" }}
- returns "foo"
+ .. sourcecode:: jinja
- {{ undefined - 42 }}
- returns 42. Note: not -42!
+ {{ undefined + "foo" }}
+ returns "foo"
-In any expression `undefined` evaluates to `false`. It has no length, all
-attribute calls return undefined, calling too:
+ {{ undefined - 42 }}
+ returns 42. Note: not -42!
-.. sourcecode:: jinja
+ In any expression `undefined` evaluates to `false`. It has no length, all
+ attribute calls return undefined, calling too:
- {{ undefined.attribute().attribute_too[42] }}
- still returns `undefined`.
+ .. sourcecode:: jinja
+
+ {{ undefined.attribute().attribute_too[42] }}
+ still returns `undefined`.
-Starting with Jinja 1.1 it's possible to control the `undefined` behavior
-in the Jinja environment, thus on the application side. If you receive
-a `TemplateRuntimeError` when operating on an undefined variable the
-environment was created with `silent` disabled.
+`ComplainingUndefined`:
-In a non silent environment the error you get tells you the line number
-and the name of the variable / attribute which could not be found.
+ Starting with Jinja 1.1 it's possible to replace the default undefined
+ object with different values. The other common undefined object which
+ comes with Jinja is the `ComplainingUndefined` object.
+
+ It raises exceptions as soon as you either render it or want to iterate
+ over it or try to access attributes etc.
+
Escaping
========
</div>
{% endmacro %}
- Called the normal way `caller` will be undefined and thus return nothing
- in silent mode or raise an exception in non silent. But if you call it
- with the new `{% call %}` tag you can pass it some data:
+ Called the normal way `caller` will be undefined, but if you call it
+ using the new `{% call %}` tag you can pass it some data:
.. sourcecode:: html+jinja
There are also some internal functions on the environment used by the template
evaluation code to keep it sandboxed.
+Undefined Values
+================
+
+If a template designer tries to access a not defined value the return value
+will be the `undefined_singleton` specified in the environment. The default
+one is the `SilentUndefined` which fails in no case. Additionally there is
+a special undefined type called the `ComplainingUndefined` which is located
+in the `jinja.datastructure` module. It will raise exceptions when compared
+with other types, or when rendered.
+
+Theoretically you can provide your own singleton by subclassing
+`AbstractUndefindedType` and creating an instance of it using `make_undefined`
+(both located in `jinja.datastructure`) but those two types should cover
+the basic use cases. The `Undefined` object in that module exists for
+backwards compatibility and is an alias for `SilentUndefined`.
+
Automatic Escaping
==================
to pass the stream to the WSGI interface in order to keep the amount of
memory used low and deliver the output to the browser as fast as possible.
-The streaming works quite simple. Whenever an item is returned by the
-internal generator it's passed to a `TemplateStream` which buffers 40 events
-before yielding them. Because WSGI usually slows down on too many flushings
-this is the integrated solution for this problem. The template designer
-can further control that behavior by placing a ``{{ flush() }}`` call in
-the template. Whenever the `TemplateStream` encounters an object marked as
-flushable it stops buffering and yields the item.
-
-Note that some constructs are not part of the stream. For example blocks
-that are rendered using ``{{ super() }}`` are yielded as one flushable item.
-
-This restriction exists because template designers may want to operate on
-the return value of a macro, block etc.
-
-The following constructs yield their subcontents as one event: ``blocks``
-that are not the current one (eg, blocks called using ``{{ super() }}``),
-macros, filtered sections.
-
-The TemplateStream
-==================
-
-You can get a new `TemplateStream` by calling the `stream()` function on
-a template like you would do for rendering. The `TemplateStream` behaves
-like a normal generator/iterator. However, as long as the stream is not
-started you can modify the buffer threshold. Once the stream started
-streaming it looses the `threshold` attribute.
-
-Explicit flushing:
+The streaming interface is straightforward. Instead of using `render()` you
+can call `stream()` which does pretty much the same but doesn't return a
+string but a `TemplateStream`:
.. sourcecode:: pycon
- >>> tmpl = evironment.from_string("<ul>{% for item in seq %}"
- ... "<li>{{ item }}</li>{{ flush() }}{% endfor %}</ul>")
- >>> s = tmpl.stream(seq=range(3))
- >>> s.next()
- u'<ul><li>0</li>'
- >>> s.next()
- u'<li>1</li>'
- >>> s.next()
- u'<li>2</li>'
- >>> s.next()
- u'</ul>'
- >>>
-
-Implicit flushing after 6 events:
+ >>> tmpl = env.from_string("<ul>{% for item in seq %}\n <li>{{ loop.index "
+ ... "}} - {{ item }}</li>\n{%- endfor %}</ul>")
+ >>> stream = tmpl.stream(seq=range(4))
+ >>> stream.next()
+ '<ul>'
+ >>> stream.next()
+ u'\n <li>1 - 0</li>'
+ >>> stream.next()
+ u'\n <li>2 - 1</li>'
+ >>> stream.next()
+ u'\n <li>3 - 2</li>'
+ >>> stream.next()
+ u'\n <li>4 - 3</li>'
+ >>> stream.next()
+ '</ul>'
+
+As you can see each iteration is threated as template event here. But also
+other tags trigger events. Basically every tag yields one event, the
+`print` tag too. The only exception is the variable substitution syntax which
+is inserted into the template text data.
+
+Because some protocols like `WSGI` flush after each iteration if passed as
+response iterable it's better to buffer some events internally. You can do
+this by enable buffering using `enable_buffering` and passing it the buffer
+size which must be greater than one:
.. sourcecode:: pycon
- >>> tmpl = environment.from_string("<ul>{% for item in seq %}"
- ... "<li>{{ item }}</li>{% endfor %}</ul>")
- >>> s = tmpl.stream(seq=range(6))
- >>> s.threshold = 6
- >>> s.next()
- u'<ul><li>0</li><li>1'
- >>> s.next()
- u'</li><li>2</li><li>3'
- >>> s.next()
- u'</li><li>4</li><li>5'
- >>> s.next()
- u'</li></ul>'
+ >>> stream.enable_buffering(size=3)
+ >>> stream.next()
+ u'<ul>\n <li>1 - 0</li>\n <li>2 - 1</li>'
+ >>> stream.next()
+ u'\n <li>3 - 2</li>\n <li>4 - 3</li></ul>'
-General `TemplateStream` behavior:
+This will buffer 3 events before yielding. Disabling buffering works using the
+`disable_buffering` method. You can enable and disable buffering on the fly also
+if you have already iterated over that stream. To check if you are in buffered
+or unbuffered mode you can use the `.buffered` property:
.. sourcecode:: pycon
- >>> s = tmpl.stream(seq=range(6))
- >>> s.started
+ >>> stream.buffered
False
- >>> s.threshold
- 40
- >>> s.next()
- u'<ul><li>0</li><li>1</li><li>2</li><li>3</li><li>4</li><li>5</li></ul>'
- >>> s.started
+ >>> stream.enable_buffering(20)
+ >>> stream.buffered
True
- >>> s.threshold
- Traceback (most recent call last):
- File "<stdin>", line 1, in <module>
- AttributeError: 'TemplateStream' object has no attribute 'threshold'
+ >>> stream.disable_buffering()
+ >>> stream.buffered
+ False
+.. admonition:: Note
-Stream Control
-==============
+ Jinja uses buffering internally for some constructs like macros. A macro
+ call is yielded as one event, independently of the internal structure.
-The stream control is designed so that it's completely transparent. When used
-in non stream mode the invisible flush tokens disappear. In order to flush
-the stream after calling a specific function all you have to do is to wrap
-the return value in a `jinja.datastructure.Flush` object. This however
-bypasses the automatic filtering system and converts the value to unicode.
+ The same applies to recursive for loops and `{% filter %}` tags.
PythonTranslator.translate = debug_translate
-def p(x, f=None):
+def p(x=None, f=None):
+ if x is None and f is not None:
+ x = e.loader.get_source(f)
print PythonTranslator(e, Parser(e, x, f).parse()).translate()
def l(x):
for item in e.lexer.tokenize(x):
print '%5s %-20s %r' % item
+
+if __name__ == '__main__':
+ if len(sys.argv) > 1:
+ from jinja import FileSystemLoader
+ e.loader = FileSystemLoader(sys.argv[1])
+ if len(sys.argv) > 2:
+ p(f=sys.argv[2])
+ else:
+ p(sys.stdin.read())
:license: BSD, see LICENSE for more details.
"""
from jinja.datastructure import Deferred, Undefined
-from jinja.exceptions import TemplateRuntimeError
class BaseContext(object):
- def __init__(self, silent, globals, initial):
- self._silent = silent
+ def __init__(self, undefined_singleton, globals, initial):
+ self._undefined_singleton = undefined_singleton
self.current = current = {}
self.stack = [globals, initial, current]
self._push = self.stack.append
else:
d[name] = rv
return rv
- if self._silent:
- return Undefined
- raise TemplateRuntimeError('%r is not defined' % name)
+ return self._undefined_singleton
def __setitem__(self, name, value):
"""
#include <structmember.h>
/* Set by init_constants to real values */
-static PyObject *Undefined, *Deferred, *TemplateRuntimeError;
-static Py_UNICODE *amp, *lt, *gt, *qt;
+static PyObject *Deferred;
/**
* Internal struct used by BaseContext to store the
struct StackLayer *initial; /* initial values */
struct StackLayer *current; /* current values */
long stacksize; /* current size of the stack */
- int silent; /* boolean value for silent failure */
+ PyObject *undefined_singleton; /* the singleton returned on missing values */
} BaseContext;
/**
PyObject *datastructure = PyImport_ImportModule("jinja.datastructure");
if (!datastructure)
return 0;
- PyObject *exceptions = PyImport_ImportModule("jinja.exceptions");
- if (!exceptions) {
- Py_DECREF(datastructure);
- return 0;
- }
- Undefined = PyObject_GetAttrString(datastructure, "Undefined");
Deferred = PyObject_GetAttrString(datastructure, "Deferred");
- TemplateRuntimeError = PyObject_GetAttrString(exceptions, "TemplateRuntimeError");
-
- amp = ((PyUnicodeObject*)PyUnicode_DecodeASCII("&", 5, NULL))->str;
- lt = ((PyUnicodeObject*)PyUnicode_DecodeASCII("<", 4, NULL))->str;
- gt = ((PyUnicodeObject*)PyUnicode_DecodeASCII(">", 4, NULL))->str;
- qt = ((PyUnicodeObject*)PyUnicode_DecodeASCII(""", 6, NULL))->str;
-
Py_DECREF(datastructure);
- Py_DECREF(exceptions);
return 1;
}
-/**
- * SGML/XML escape something.
- *
- * XXX: this is awefully slow for non unicode objects because they
- * get converted to unicode first.
- */
-static PyObject*
-escape(PyObject *self, PyObject *args)
-{
- PyUnicodeObject *in, *out;
- Py_UNICODE *outp;
- int i, len;
-
- PyObject *text = NULL, *use_quotes = NULL;
-
- if (!PyArg_ParseTuple(args, "O|O", &text, &use_quotes))
- return NULL;
- int quotes = use_quotes && PyObject_IsTrue(use_quotes);
- in = (PyUnicodeObject*)PyObject_Unicode(text);
- if (!in)
- return NULL;
-
- /* First we need to figure out how long the escaped string will be */
- len = 0;
- for (i = 0; i < in->length; i++) {
- switch (in->str[i]) {
- case '&':
- len += 5;
- break;
- case '"':
- len += quotes ? 6 : 1;
- break;
- case '<':
- case '>':
- len += 4;
- break;
- default:
- len++;
- }
- }
-
- /* Do we need to escape anything at all? */
- if (len == in->length)
- return (PyObject*)in;
-
- out = (PyUnicodeObject*)PyUnicode_FromUnicode(NULL, len);
- if (!out) {
- Py_DECREF(in);
- return NULL;
- }
-
- outp = out->str;
- for (i = 0; i < in->length; i++) {
- switch (in->str[i]) {
- case '&':
- Py_UNICODE_COPY(outp, amp, 5);
- outp += 5;
- break;
- case '"':
- if (quotes) {
- Py_UNICODE_COPY(outp, qt, 6);
- outp += 6;
- }
- else
- *outp++ = in->str[i];
- break;
- case '<':
- Py_UNICODE_COPY(outp, lt, 4);
- outp += 4;
- break;
- case '>':
- Py_UNICODE_COPY(outp, gt, 4);
- outp += 4;
- break;
- default:
- *outp++ = in->str[i];
- };
- }
-
- Py_DECREF(in);
- return (PyObject*)out;
-}
-
/**
* Deallocator for BaseContext.
*
/**
* Initializes the BaseContext.
*
- * Like the native python class it takes a flag telling the context
- * to either fail silently with Undefined or raising a TemplateRuntimeError.
+ * Like the native python class it takes a reference to the undefined
+ * singleton which will be used for undefined values.
* The other two arguments are the global namespace and the initial
* namespace which usually contains the values passed to the render
* function of the template. Both must be dicts.
static int
BaseContext_init(BaseContext *self, PyObject *args, PyObject *kwds)
{
- PyObject *silent = NULL, *globals = NULL, *initial = NULL;
+ PyObject *undefined = NULL, *globals = NULL, *initial = NULL;
- static char *kwlist[] = {"silent", "globals", "initial", NULL};
- if (!PyArg_ParseTupleAndKeywords(args, kwds, "OOO", kwlist,
- &silent, &globals, &initial))
+ if (!PyArg_ParseTuple(args, "OOO", &undefined, &globals, &initial))
return -1;
if (!PyDict_Check(globals) || !PyDict_Check(initial)) {
PyErr_SetString(PyExc_TypeError, "stack layers must be dicts.");
return -1;
}
- self->silent = PyObject_IsTrue(silent);
- if (self->silent == -1)
- return -1;
-
self->current = PyMem_Malloc(sizeof(struct StackLayer));
self->current->prev = NULL;
self->current->dict = PyDict_New();
Py_INCREF(globals);
self->initial->prev = self->globals;
+ self->undefined_singleton = undefined;
+ Py_INCREF(undefined);
+
self->stacksize = 3;
return 0;
}
int isdeferred;
struct StackLayer *current = self->current;
- if (!PyString_Check(item))
+ /* allow unicode keys as long as they are ascii keys */
+ if (PyUnicode_CheckExact(item)) {
+ item = PyUnicode_AsASCIIString(item);
+ if (!item)
+ goto missing;
+ }
+ else if (!PyString_Check(item))
goto missing;
/* disallow access to internal jinja values */
}
missing:
- if (self->silent) {
- Py_INCREF(Undefined);
- return Undefined;
- }
- if (name)
- PyErr_Format(TemplateRuntimeError, "'%s' is not defined", name);
- else
- PyErr_SetString(TemplateRuntimeError, "value is not a string");
- return NULL;
+ Py_INCREF(self->undefined_singleton);
+ return self->undefined_singleton;
}
/**
char *name;
struct StackLayer *current = self->current;
- if (!PyString_Check(item))
+ /* allow unicode objects as keys as long as they are ASCII */
+ if (PyUnicode_CheckExact(item)) {
+ item = PyUnicode_AsASCIIString(item);
+ if (!item)
+ return 0;
+ }
+ else if (!PyString_Check(item))
return 0;
name = PyString_AS_STRING(item);
static int
BaseContext_setitem(BaseContext *self, PyObject *item, PyObject *value)
{
- if (!PyString_Check(item)) {
- PyErr_SetString(PyExc_TypeError, "expected string argument");
- return -1;
+ /* allow unicode objects as keys as long as they are ASCII */
+ if (PyUnicode_CheckExact(item)) {
+ item = PyUnicode_AsASCIIString(item);
+ if (!item) {
+ PyErr_Clear();
+ goto error;
+ }
}
+ else if (!PyString_Check(item))
+ goto error;
if (!value)
return PyDict_DelItem(self->current->dict, item);
return PyDict_SetItem(self->current->dict, item, value);
+
+error:
+ PyErr_SetString(PyExc_TypeError, "expected string argument");
+ return -1;
}
/**
};
static PyMethodDef module_methods[] = {
- {"escape", (PyCFunction)escape, METH_VARARGS,
- "escape(s, quotes=False) -> string\n\n"
- "SGML/XML a string."},
{NULL, NULL, 0, NULL} /* Sentinel */
};
return f
-class UndefinedType(object):
+def make_undefined(implementation):
"""
- An object that does not exist.
+ Creates an undefined singleton based on a given implementation.
+ It performs some tests that make sure the undefined type implements
+ everything it should.
+ """
+ self = object.__new__(implementation)
+ self.__reduce__()
+ return self
+
+
+class AbstractUndefinedType(object):
+ """
+ Base class for any undefined type.
"""
__slots__ = ()
raise TypeError('cannot create %r instances' %
self.__class__.__name__)
+ def __setattr__(self, name, value):
+ raise AttributeError('%r object has no attribute %r' % (
+ self.__class__.__name__,
+ name
+ ))
+
+ def __eq__(self, other):
+ return self is other
+
+ def __ne__(self, other):
+ return self is not other
+
+ def __copy__(self):
+ return self
+ __deepcopy__ = __copy__
+
+ def __reduce__(self):
+ raise TypeError('undefined objects have to provide a __reduce__')
+
+
+class SilentUndefinedType(AbstractUndefinedType):
+ """
+ An object that does not exist.
+ """
+ __slots__ = ()
+
def __add__(self, other):
"""Any operator returns the operand."""
return other
"""The unicode representation is empty."""
return u''
- def __repr__(self):
- """The representation is ``'Undefined'``"""
- return 'Undefined'
-
def __int__(self):
"""Converting `Undefined` to an integer ends up in ``0``"""
return 0
"""Converting `Undefined` to an float ends up in ``0.0``"""
return 0.0
- def __eq__(self, other):
- """`Undefined` is not equal to anything else."""
- return False
-
- def __ne__(self, other):
- """`Undefined` is not equal to anything else."""
- return True
-
def __call__(self, *args, **kwargs):
"""Calling `Undefined` returns `Undefined`"""
return self
- def __copy__(self):
- """Return a copy."""
- return self
+ def __reduce__(self):
+ """Helper for pickle."""
+ return 'SilentUndefined'
- def __deepcopy__(self, memo):
- """Return a deepcopy."""
- return self
+
+class ComplainingUndefinedType(AbstractUndefinedType):
+ """
+ An object that does not exist.
+ """
+ __slots__ = ()
+
+ def __iter__(self):
+ """Iterating over `Undefined` returns an empty iterator."""
+ if False:
+ yield None
+
+ def __nonzero__(self):
+ """`Undefined` is considered boolean `False`"""
+ return False
+
+ def __str__(self):
+ """The string representation raises an error."""
+ raise TemplateRuntimeError('Undefined object rendered')
+
+ def __unicode__(self):
+ """The unicode representation raises an error."""
+ self.__str__()
+
+ def __call__(self, *args, **kwargs):
+ """Calling `Undefined` returns `Undefined`"""
+ raise TemplateRuntimeError('Undefined object called')
def __reduce__(self):
"""Helper for pickle."""
- return 'Undefined'
+ return 'ComplainingUndefined'
-#: the singleton instance of UndefinedType
-Undefined = object.__new__(UndefinedType)
+#: the singleton instances for the undefined objects
+SilentUndefined = make_undefined(SilentUndefinedType)
+ComplainingUndefined = make_undefined(ComplainingUndefinedType)
+
+#: jinja 1.0 compatibility
+Undefined = SilentUndefined
+UndefinedType = SilentUndefinedType
class FakeTranslator(object):
"""
-class Flush(TemplateData):
- """
- After a string marked as Flush the stream will stop buffering.
- """
-
- jinja_no_finalization = True
-
-
# import these here because those modules import Deferred and Undefined
# from this module.
try:
def __init__(self, *args, **kwargs):
environment = args[0]
- super(Context, self).__init__(environment.silent,
+ super(Context, self).__init__(environment.undefined_singleton,
environment.globals,
dict(*args[1:], **kwargs))
self._translate_func = None
class TemplateStream(object):
"""
- Pass it a template generator and it will buffer a few items
- before yielding them as one item. Useful when working with WSGI
- because a Jinja template yields far too many items per default.
-
- The `TemplateStream` class looks for the invisble `Flush`
- markers sent by the template to find out when it should stop
- buffering.
+ Wraps a genererator for outputing template streams.
"""
def __init__(self, gen):
- self._next = gen.next
- self._threshold = None
- self.threshold = 40
+ self._gen = gen
+ self._next = gen._next
+ self.buffered = False
+
+ def disable_buffering(self):
+ """
+ Disable the output buffering.
+ """
+ self._next = self._gen.next
+ self.buffered = False
+
+ def enable_buffering(self, size=5):
+ """
+ Enable buffering. Buffer `size` items before
+ yielding them.
+ """
+ if size <= 1:
+ raise ValueError('buffer size too small')
+ self.buffered = True
+
+ def buffering_next():
+ buf = []
+ c_size = 0
+ push = buf.append
+ next = self._gen.next
+
+ try:
+ while True:
+ item = next()
+ if item:
+ push(item)
+ c_size += 1
+ if c_size >= size:
+ raise StopIteration()
+ except StopIteration:
+ if not size:
+ raise
+ return u''.join(buf)
+
+ self._next = buffering_next
def __iter__(self):
return self
- def started(self):
- return self._threshold is not None
- started = property(started)
-
def next(self):
- if self._threshold is None:
- self._threshold = t = self.threshold
- del self.threshold
- else:
- t = self._threshold
- buf = []
- size = 0
- push = buf.append
- next = self._next
-
- try:
- while True:
- item = next()
- if item:
- push(item)
- size += 1
- if (size and item.__class__ is Flush) or size >= t:
- raise StopIteration()
- except StopIteration:
- pass
- if not size:
- raise StopIteration()
- return u''.join(buf)
+ return self._next()
from jinja.filters import FILTERS as DEFAULT_FILTERS
from jinja.tests import TESTS as DEFAULT_TESTS
from jinja.utils import debug_helper, safe_range, generate_lorem_ipsum, \
- watch_changes, flush
+ watch_changes
__all__ = ['DEFAULT_FILTERS', 'DEFAULT_TESTS', 'DEFAULT_NAMESPACE']
'range': safe_range,
'debug': debug_helper,
'lipsum': generate_lorem_ipsum,
- 'watchchanges': watch_changes,
- 'flush': flush
+ 'watchchanges': watch_changes
}
from jinja.lexer import Lexer
from jinja.parser import Parser
from jinja.loaders import LoaderWrapper
-from jinja.datastructure import Undefined, Markup, Context, FakeTranslator
+from jinja.datastructure import SilentUndefined, Markup, Context, FakeTranslator
from jinja.utils import collect_translations, get_attribute
from jinja.exceptions import FilterNotFound, TestNotFound, \
SecurityException, TemplateSyntaxError, TemplateRuntimeError
filters=None,
tests=None,
context_class=Context,
- silent=True,
+ undefined_singleton=SilentUndefined,
friendly_traceback=True):
"""
Here the possible initialization parameters:
`context_class` the context class this template should use.
See the `Context` documentation for more
details.
- `silent` set this to `False` if you want to receive
- errors for missing template variables or
- attributes. Defaults to `False`. *new in
- Jinja 1.1*
+ `undefined_singleton` The singleton value that is used for missing
+ variables. *new in Jinja 1.1*
`friendly_traceback` Set this to `False` to disable the developer
friendly traceback rewriting. Whenever an
runtime or syntax error occours jinja will
self.tests = tests is None and DEFAULT_TESTS.copy() or tests
self.default_filters = default_filters or []
self.context_class = context_class
- self.silent = silent
+ self.undefined_singleton = undefined_singleton
self.friendly_traceback = friendly_traceback
# global namespace
"""
Convert a value to unicode with the rules defined on the environment.
"""
- if value in (None, Undefined):
+ # undefined and None expand to ""
+ if value in (None, self.undefined_singleton):
return u''
+ # things that are already unicode can pass. As long as nobody
+ # does ugly things with the class it works for jinja too
elif isinstance(value, unicode):
return value
- else:
- try:
- return unicode(value)
- except UnicodeError:
- return str(value).decode(self.charset, 'ignore')
+ # otherwise try to use __unicode__ or decode __str__
+ try:
+ return unicode(value)
+ except UnicodeError:
+ return str(value).decode(self.charset, 'ignore')
def get_translator(self, context):
"""
return get_attribute(obj, name)
except (AttributeError, SecurityException):
pass
- if self.silent:
- return Undefined
- raise TemplateRuntimeError('attribute %r or object %r not defined' % (
- name, obj))
+ if obj is self.undefined_singleton:
+ return getattr(self.undefined_singleton, name)
+ return self.undefined_singleton
def get_attributes(self, obj, attributes):
"""
kwargs.update(dyn_kwargs)
if getattr(f, 'jinja_unsafe_call', False) or \
getattr(f, 'alters_data', False):
- return Undefined
+ return self.undefined_singleton
if getattr(f, 'jinja_context_callable', False):
args = (self, context) + args
return f(*args, **kwargs)
"""
if getattr(f, 'jinja_unsafe_call', False) or \
getattr(f, 'alters_data', False):
- return Undefined
+ return self.undefined_singleton
if getattr(f, 'jinja_context_callable', False):
return f(self, context)
return f()
evaluator the source generated by the python translator will
call this function for all variables.
"""
- if value is Undefined or value is None:
+ if value is None:
return u''
+ elif value is self.undefined_singleton:
+ return unicode(value)
elif getattr(value, 'jinja_no_finalization', False):
return value
val = self.to_unicode(value)
from random import choice
from urllib import urlencode, quote
from jinja.utils import urlize, escape
-from jinja.datastructure import Undefined, Markup, TemplateData
+from jinja.datastructure import Markup, TemplateData
from jinja.exceptions import FilterArgumentError
return s
elif hasattr(s, '__html__'):
return s.__html__()
- #: small speedup
+ #: small speedup, do not convert to unicode if we already
+ #: have an unicode object.
if s.__class__ is not unicode:
s = env.to_unicode(s)
return e(s, attribute)
raise TypeError('a dict is required')
result = []
for key, value in d.iteritems():
- if value not in (None, Undefined):
+ if value not in (None, env.undefined_singleton):
result.append(u'%s="%s"' % (
e(env.to_unicode(key)),
e(env.to_unicode(value), True)
{{ ''|default('the string was empty', true) }}
"""
def wrapped(env, context, value):
- if (boolean and not value) or value in (Undefined, None):
+ if (boolean and not value) or value in (env.undefined_singleton, None):
return default_value
return value
return wrapped
try:
return iter(seq).next()
except StopIteration:
- if env.silent:
- return Undefined
- raise TemplateRuntimeError('%r is empty' % seq)
+ return env.undefined_singleton[0]
return wrapped
try:
return iter(_reversed(seq)).next()
except StopIteration:
- if env.silent:
- return Undefined
- raise TemplateRuntimeError('%r is empty' % seq)
+ return env.undefined_singleton[-1]
return wrapped
try:
return choice(seq)
except IndexError:
- if env.silent:
- return Undefined
- raise TemplateRuntimeError('%r is empty' % seq)
+ return env.undefined_singleton[0]
return wrapped
def wrapped(env, context, value):
context[name] = value
if clean:
- return Undefined
+ return TemplateData()
return value
return wrapped
return 'Text(%r)' % (self.text,)
+class DynamicText(Node):
+ """
+ Note that represents normal text with string formattings.
+ Those are used for texts that contain variables. The attribute
+ `variables` contains a list of Print tags and nothing else.
+ """
+
+ def __init__(self, lineno, format_string, variables):
+ self.lineno = lineno
+ self.format_string = format_string
+ self.variables = variables
+
+ def get_items(self):
+ return [self.format_string] + list(self.variables)
+
+ def __repr__(self):
+ return 'DynamicText(%r, %r)' % (
+ self.text,
+ self.variables
+ )
+
+
class NodeList(list, Node):
"""
A node that stores multiple childnodes.
#: set of directives that are only available in a certain
#: context.
- self.context_directives = set(['elif', 'else', 'endblock',
- 'endfilter', 'endfor', 'endif', 'endmacro', 'endraw',
- 'endtrans', 'pluralize'
+ self.context_directives = set([
+ 'elif', 'else', 'endblock', 'endfilter', 'endfor', 'endif',
+ 'endmacro', 'endraw', 'endtrans', 'pluralize'
])
self.tokenstream = environment.lexer.tokenize(source, filename)
def handle_macro_directive(self, lineno, gen):
"""
- Handle {% macro foo bar, baz %}.
+ Handle {% macro foo bar, baz %} as well as
+ {% macro foo(bar, baz) %}.
"""
try:
macro_name = gen.next()
'as macro name.' % macro_name[2],
lineno, self.filename)
- ast = self.parse_python(lineno, gen, 'def %s(%%s):pass' %
+ # make the punctuation around arguments optional
+ arg_list = list(gen)
+ if arg_list and arg_list[0][1:] == ('operator', '(') and \
+ arg_list[-1][1:] == ('operator', ')'):
+ arg_list = arg_list[1:-1]
+
+ ast = self.parse_python(lineno, arg_list, 'def %s(%%s):pass' %
str(macro_name[2][:-1]))
body = self.subparse(end_of_macro, True)
self.close_remaining_block()
"""
def finish():
"""Helper function to remove unused nodelists."""
+ if data_buffer:
+ flush_data_buffer()
if len(result) == 1:
return result[0]
return result
+ def flush_data_buffer():
+ """Helper function to write the contents of the buffer
+ to the result nodelist."""
+ format_string = []
+ insertions = []
+ for item in data_buffer:
+ if item[0] == 'variable':
+ p = self.handle_print_directive(*item[1:])
+ format_string.append('%s')
+ insertions.append(p)
+ else:
+ format_string.append(item[2].replace('%', '%%'))
+ # if we do not have insertions yield it as text node
+ if not insertions:
+ result.append(nodes.Text(data_buffer[0][1],
+ (u''.join(format_string)).replace('%%', '%')))
+ # if we do not have any text data we yield some variable nodes
+ elif len(insertions) == len(format_string):
+ result.extend(insertions)
+ # otherwise we go with a dynamic text node
+ else:
+ result.append(nodes.DynamicText(data_buffer[0][1],
+ u''.join(format_string),
+ insertions))
+ # clear the buffer
+ del data_buffer[:]
+
lineno = self.tokenstream.last[0]
result = nodes.NodeList(lineno)
+ data_buffer = []
for lineno, token, data in self.tokenstream:
# comments
if token == 'comment_begin':
# parse everything till the end of it.
elif token == 'variable_begin':
gen = self.tokenstream.fetch_until(end_of_variable, True)
- result.append(self.directives['print'](lineno, gen))
+ data_buffer.append(('variable', lineno, tuple(gen)))
# this token marks the start of a block. like for variables
# just parse everything until the end of the block
elif token == 'block_begin':
+ # if we have something in the buffer we write the
+ # data back
+ if data_buffer:
+ flush_data_buffer()
+
gen = self.tokenstream.fetch_until(end_of_block, True)
try:
lineno, token, data = gen.next()
# tokens just exist in block or variable sections. (if the
# tokenizer is not brocken)
elif token in 'data':
- result.append(nodes.Text(lineno, data))
+ data_buffer.append(('text', lineno, data))
# so this should be unreachable code
else:
:license: BSD, see LICENSE for more details.
"""
import re
-from jinja.datastructure import Undefined
number_re = re.compile(r'^-?\d+(\.\d+)?$')
See also the ``default`` filter.
"""
- return lambda e, c, v: v is not Undefined
+ return lambda e, c, v: v is not e.undefined_singleton
def test_lower():
It also adds debug symbols used by the traceback toolkit implemented
in `jinja.utils`.
+ Implementation Details
+ ======================
+
+ Some of the semantics are handled directly in the translator in order
+ to speed up the parsing process. For example the semantics for the
+ `is` operator are handled here, changing this would require and
+ additional traversing of the node tree in the parser. Thus the actual
+ translation process can raise a `TemplateSyntaxError` too.
+
+ It might sound strange but the translator tries to keep the generated
+ code readable as much as possible. This simplifies debugging the Jinja
+ core a lot. The additional processing overhead is just relevant for
+ the translation process, the additional comments and whitespace won't
+ appear in the saved bytecode.
+
:copyright: 2007 by Armin Ronacher.
:license: BSD, see LICENSE for more details.
"""
_debug_re = re.compile(r'^\s*\# DEBUG\(filename=(?P<filename>.*?), '
r'lineno=(?P<lineno>\d+)\)$')
+# For Python2.3 compatibiilty
+try:
+ set
+except NameError:
+ from sets import Set as set
+# For Python 2.3/2.4 compatibility
try:
GeneratorExit
except NameError:
class GeneratorExit(Exception):
- """For python2.3/python2.4 compatibility"""
-
+ pass
+# For Pythons without conditional expressions
try:
- set
-except NameError:
- from sets import Set as set
-
-
-def _to_tuple(args):
- """
- Return a tuple repr without nested repr.
- """
- return '(%s%s)' % (
- ', '.join(args),
- len(args) == 1 and ',' or ''
- )
+ exec '0 if 0 else 0'
+ have_conditional_expr = True
+except SyntaxError:
+ have_conditional_expr = False
class Template(object):
'true': 'True',
'false': 'False',
'none': 'None',
- 'undefined': 'Undefined'
+ 'undefined': 'undefined_singleton'
}
#: bind the nodes to the callback functions. There are
# jinja nodes
nodes.Template: self.handle_template,
nodes.Text: self.handle_template_text,
+ nodes.DynamicText: self.handle_dynamic_text,
nodes.NodeList: self.handle_node_list,
nodes.ForLoop: self.handle_for_loop,
nodes.IfCondition: self.handle_if_condition,
"""
return (' ' * (self.indention * 4)) + text
+ def to_tuple(self, args):
+ """
+ Return a tuple repr without nested repr.
+ """
+ return '(%s%s)' % (
+ ', '.join(args),
+ len(args) == 1 and ',' or ''
+ )
+
def nodeinfo(self, node, force=False):
"""
Return a comment that helds the node informations or None
n.filename)
filters.append('(%r, %s)' % (
n.node.name,
- _to_tuple(args)
+ self.to_tuple(args)
))
elif n.__class__ is ast.Name:
filters.append('(%r, ())' % n.name)
'hardcoded function name from the '
'filter namespace',
n.lineno, n.filename)
- return 'apply_filters(%s, context, %s)' % (s, _to_tuple(filters))
+ self.used_shortcuts.add('apply_filters')
+ return 'apply_filters(%s, context, %s)' % (s, self.to_tuple(filters))
def handle_node(self, node):
"""
"""
Reset translation variables such as indention or cycle id
"""
+ #: current level of indention
self.indention = 0
+ #: each {% cycle %} tag has a unique ID which increments
+ #: automatically for each tag.
self.last_cycle_id = 0
+ #: set of used shortcuts jinja has to make local automatically
+ self.used_shortcuts = set(['undefined_singleton'])
+ #: set of used datastructures jinja has to import
+ self.used_data_structures = set()
+ #: set of used utils jinja has to import
+ self.used_utils = set()
+ #: flags for runtime error
+ self.require_runtime_error = False
def translate(self):
"""
# update the blocks there. Once this is done we drop the current
# template in favor of the new one. Do that until we found the
# root template.
- requirements_todo = []
parent = None
overwrites = {}
blocks = {}
+ requirements = []
+ outer_filename = node.filename or '<template>'
+
+ # this set is required in order to not add blocks to the block
+ # dict a second time if they were not overridden in one template
+ # in the template chain.
+ already_registered_block = set()
while node.extends is not None:
- # handle all requirements but not those from the
- # root template. The root template renders everything so
- # there is no need for additional requirements
- if node not in requirements_todo:
- requirements_todo.append(node)
+ # the direct child nodes in a template that are not blocks
+ # are processed as template globals, thus executed *before*
+ # the master layout template is loaded. This can be used
+ # for further processing. The output of those nodes does
+ # not appear in the final template.
+ requirements += [child for child in node.getChildNodes()
+ if child.__class__ not in (nodes.Text,
+ nodes.Block, nodes.Extends)]
# load the template we inherit from and add not known blocks
parent = self.environment.loader.parse(node.extends.template,
# an overwritten block for the parent template. handle that
# override in the template and register it in the deferred
# block dict.
- if n.name in overwrites:
+ if n.name in overwrites and not n in already_registered_block:
blocks.setdefault(n.name, []).append(n.clone())
n.replace(overwrites[n.name])
+ already_registered_block.add(n)
# make the parent node the new node
node = parent
- # look up requirements
- requirements = []
- for req in requirements_todo:
- for n in req:
- if n.__class__ in (nodes.Set, nodes.Macro, nodes.Include):
- requirements.append(n)
-
- # aliases boilerplate
- aliases = ['%s = environment.%s' % (item, item) for item in
- ['get_attribute', 'perform_test', 'apply_filters',
- 'call_function', 'call_function_simple', 'finish_var']]
-
- # bootstrapping code
- lines = [
- '# Essential imports\n'
- 'from __future__ import division\n'
- 'from jinja.datastructure import Undefined, LoopContext, '
- 'CycleContext, SuperBlock\n'
- 'from jinja.utils import buffereater\n'
- 'from jinja.exceptions import TemplateRuntimeError\n\n'
- '# Local aliases for some speedup\n'
- '%s\n'
- '__name__ = %r\n\n'
- 'def generate(context):\n'
- ' assert environment is context.environment\n'
- ' ctx_push = context.push\n'
- ' ctx_pop = context.pop' % (
- '\n'.join(aliases),
- node.filename
- )
- ]
-
- # we have requirements? add them here.
- body_lines = []
+ # handle requirements code
if requirements:
+ requirement_lines = [
+ 'def bootstrap(context):',
+ ' ctx_push = context.push',
+ ' ctx_pop = context.pop'
+ ]
+ has_requirements = False
for n in requirements:
- body_lines.append(self.handle_node(n))
+ requirement_lines.append(self.handle_node(n))
+ requirement_lines.append(' if 0: yield None\n')
- # the template body
- body_lines.extend([self.handle_node(n) for n in node])
-
- # add body lines and "generator hook"
- lines.extend(body_lines)
- lines.append(' if 0: yield None')
+ # handle body in order to get the used shortcuts
+ body_lines = [self.handle_node(n) for n in node]
- # add the missing blocks
+ # same for blocks in callables
+ block_lines = []
block_items = blocks.items()
block_items.sort()
dict_lines = []
# ensure that the indention is correct
self.indention = 1
func_name = 'block_%s_%s' % (name, idx)
- lines.extend([
- '\ndef %s(context):' % func_name,
- ' ctx_push = context.push',
- ' ctx_pop = context.pop',
- ' if 0: yield None'
- ])
- lines.append(self.indent(self.nodeinfo(item, True)))
- lines.append(self.handle_block(item, idx + 1))
- tmp.append('buffereater(%s)' % func_name)
+ data = self.handle_block(item, idx + 1)
+ # blocks with data
+ if data:
+ block_lines.extend([
+ 'def %s(context):' % func_name,
+ ' ctx_push = context.push',
+ ' ctx_pop = context.pop',
+ self.indent(self.nodeinfo(item, True)),
+ data,
+ ' if 0: yield None\n'
+ ])
+ tmp.append('buffereater(%s)' % func_name)
+ self.used_utils.add('buffereater')
+ # blocks without data, can default to something
+ # from utils
+ else:
+ tmp.append('empty_block')
+ self.used_utils.add('empty_block')
dict_lines.append(' %r: %s' % (
str(name),
- _to_tuple(tmp)
+ self.to_tuple(tmp)
))
+ # aliases boilerplate
+ aliases = ['%s = environment.%s' % (item, item) for item in
+ ['get_attribute', 'perform_test', 'apply_filters',
+ 'call_function', 'call_function_simple', 'finish_var',
+ 'undefined_singleton'] if item in self.used_shortcuts]
+
+ # bootstrapping code
+ lines = ['# Essential imports', 'from __future__ import division']
+ if self.used_utils:
+ lines.append('from jinja.utils import %s' % ', '.join(self.used_utils))
+ if self.require_runtime_error:
+ lines.append('from jinja.exceptions import TemplateRuntimeError')
+ if self.used_data_structures:
+ lines.append('from jinja.datastructure import %s' % ', '.
+ join(self.used_data_structures))
+ lines.extend([
+ '\n# Aliases for some speedup\n'
+ '%s\n\n'
+ '# Name for disabled debugging\n'
+ '__name__ = %r\n\n'
+ 'def generate(context):\n'
+ ' assert environment is context.environment\n'
+ ' ctx_push = context.push\n'
+ ' ctx_pop = context.pop' % (
+ '\n'.join([
+ '%s = environment.%s' % (item, item) for item in
+ ['get_attribute', 'perform_test', 'apply_filters',
+ 'call_function', 'call_function_simple', 'finish_var',
+ 'undefined_singleton'] if item in self.used_shortcuts
+ ]),
+ outer_filename
+ )
+ ])
+
+ # the template body
+ if requirements:
+ lines.append(' for item in bootstrap(context): pass')
+ lines.extend(body_lines)
+ lines.append(' if 0: yield None\n')
+
+ # now write the bootstrapping (requirements) core if there is one
+ if requirements:
+ lines.append('# Bootstrapping code')
+ lines.extend(requirement_lines)
+
# blocks must always be defined. even if it's empty. some
# features depend on it
- lines.append('\n# Block mapping and debug information')
+ if block_lines:
+ lines.append('# Superable blocks')
+ lines.extend(block_lines)
+ lines.append('# Block mapping')
if dict_lines:
- lines.append('blocks = {\n%s\n}' % ',\n'.join(dict_lines))
+ lines.append('blocks = {\n%s\n}\n' % ',\n'.join(dict_lines))
else:
- lines.append('blocks = {}')
+ lines.append('blocks = {}\n')
# now get the real source lines and map the debugging symbols
debug_mapping = []
if m is not None:
d = m.groupdict()
filename = d['filename'] or None
+ if isinstance(filename, unicode):
+ filename = filename.encode('utf-8')
if filename in file_mapping:
file_id = file_mapping[filename]
else:
result.append(line)
# now print file mapping and debug info
+ result.append('\n# Debug Information')
file_mapping = file_mapping.items()
file_mapping.sort(lambda a, b: cmp(a[1], b[1]))
for filename, file_id in file_mapping:
result.append('%s = %r' % (file_id, filename))
- result.append('debug_info = [%s]' % ', '.join(debug_mapping))
+ result.append('debug_info = %s' % self.to_tuple(debug_mapping))
return '\n'.join(result)
def handle_template_text(self, node):
return self.indent(self.nodeinfo(node)) + '\n' +\
self.indent('yield %r' % data)
+ def handle_dynamic_text(self, node):
+ """
+ Like `handle_template_text` but for nodes of the type
+ `DynamicText`.
+ """
+ buf = []
+ write = lambda x: buf.append(self.indent(x))
+ self.used_shortcuts.add('finish_var')
+
+ write(self.nodeinfo(node))
+ write('yield %r %% (' % node.format_string)
+ self.indention += 1
+ for var in node.variables:
+ write(self.nodeinfo(var))
+ write('finish_var(%s, context)' % self.handle_node(var.variable) + ',')
+ self.indention -= 1
+ write(')')
+
+ return '\n'.join(buf)
+
def handle_node_list(self, node):
"""
In some situations we might have a node list. It's just
Handle a for loop. Pretty basic, just that we give the else
clause a different behavior.
"""
+ self.used_data_structures.add('LoopContext')
buf = []
write = lambda x: buf.append(self.indent(x))
write(self.nodeinfo(node))
# recursive loops
if node.recursive:
- write('def forloop(seq):')
+ write('def loop(seq):')
self.indention += 1
write('for %s in context[\'loop\'].push(seq):' %
self.handle_node(node.item),
write('if 0: yield None')
self.indention -= 1
write('context[\'loop\'] = LoopContext(None, context[\'loop\'], '
- 'buffereater(forloop))')
- write('for item in forloop(%s):' % self.handle_node(node.seq))
+ 'buffereater(loop))')
+ self.used_utils.add('buffereater')
+ write('for item in loop(%s):' % self.handle_node(node.seq))
self.indention += 1
write('yield item')
self.indention -= 1
"""
Handle the cycle tag.
"""
+ self.used_data_structures.add('CycleContext')
name = '::cycle_%x' % self.last_cycle_id
self.last_cycle_id += 1
buf = []
if node.seq.__class__ in (ast.Tuple, ast.List):
write('context.current[%r] = CycleContext(%s)' % (
name,
- _to_tuple([self.handle_node(n) for n in node.seq.nodes])
+ self.to_tuple([self.handle_node(n) for n in node.seq.nodes])
))
hardcoded = True
else:
hardcoded = False
self.indention -= 1
+ self.used_shortcuts.add('finish_var')
if hardcoded:
write('yield finish_var(context.current[%r].cycle(), '
'context)' % name)
"""
Handle a print statement.
"""
+ self.used_shortcuts.add('finish_var')
return self.indent(self.nodeinfo(node)) + '\n' +\
self.indent('yield finish_var(%s, context)' %
self.handle_node(node.variable))
# collect macro arguments
arg_items = []
caller_overridden = False
+
+ # if we have conditional expressions available in that python
+ # build (for example cpython > 2.4) we can use them, they
+ # will perform slightly better.
+ if have_conditional_expr:
+ arg_tmpl = '\'%(name)s\': args[%(pos)d] if argcount > %(pos)d ' \
+ 'else %(default)s'
+ # otherwise go with the and/or tuple hack:
+ else:
+ arg_tmpl = '\'%(name)s\': (argcount > %(pos)d and '\
+ '(args[%(pos)d],) or (%(default)s,))[0]'
+
if node.arguments:
+ varargs_init = '\'varargs\': args[%d:]' % len(node.arguments)
write('argcount = len(args)')
for idx, (name, n) in enumerate(node.arguments):
- arg_items.append('\'%s\': (argcount > %d and (args[%d],) '
- 'or (%s,))[0]' % (
- name,
- idx,
- idx,
- n is None and 'Undefined' or self.handle_node(n)
- ))
+ arg_items.append(arg_tmpl % {
+ 'name': name,
+ 'pos': idx,
+ 'default': n is None and 'undefined_singleton' or
+ self.handle_node(n)
+ })
if name == 'caller':
caller_overridden = True
+ elif name == 'varargs':
+ varargs_init = None
+ else:
+ varargs_init = '\'varargs\': args'
+
if caller_overridden:
write('kw.pop(\'caller\', None)')
else:
- arg_items.append('\'caller\': kw.pop(\'caller\', Undefined)')
- write('ctx_push({%s})' % ', '.join(arg_items))
+ arg_items.append('\'caller\': kw.pop(\'caller\', undefined_singleton)')
+ if varargs_init:
+ arg_items.append(varargs_init)
+
+ write('ctx_push({%s})' % ',\n '.join([
+ idx and self.indent(item) or item for idx, item
+ in enumerate(arg_items)
+ ]))
# disallow any keyword arguments
write('if kw:')
self.indention += 1
write('raise TemplateRuntimeError(\'%s got an unexpected keyword '
'argument %%r\' %% iter(kw).next())' % node.name)
+ self.require_runtime_error = True
self.indention -= 1
write(self.nodeinfo(node.body))
- buf.append(self.handle_node(node.body))
+ data = self.handle_node(node.body)
+ if data:
+ buf.append(data)
write('ctx_pop()')
write('if 0: yield None')
self.indention -= 1
buf.append(self.indent('context[%r] = buffereater(macro)' %
node.name))
+ self.used_utils.add('buffereater')
return '\n'.join(buf)
write('def call(**kwargs):')
self.indention += 1
write('ctx_push(kwargs)')
- buf.append(self.handle_node(node.body))
+ data = self.handle_node(node.body)
+ if data:
+ buf.append(data)
write('ctx_pop()')
write('if 0: yield None')
self.indention -= 1
write('yield ' + self.handle_call_func(node.expr,
{'caller': 'buffereater(call)'}))
+ self.used_utils.add('buffereater')
return '\n'.join(buf)
self.indention += 1
write('ctx_push()')
write(self.nodeinfo(node.body))
- buf.append(self.handle_node(node.body))
+ data = self.handle_node(node.body)
+ if data:
+ buf.append(data)
write('ctx_pop()')
write('if 0: yield None')
self.indention -= 1
write('yield %s' % self.filter('buffereater(filtered)()',
node.filters))
+ self.used_utils.add('buffereater')
return '\n'.join(buf)
def handle_block(self, node, level=0):
if not rv:
return ''
+ self.used_data_structures.add('SuperBlock')
buf = []
write = lambda x: buf.append(self.indent(x))
level
))
write(self.nodeinfo(node.body))
- buf.append(self.handle_node(node.body))
+ buf.append(rv)
write('ctx_pop()')
return '\n'.join(buf)
raise TemplateSyntaxError('is operator requires a test name'
' as operand', node.lineno,
node.filename)
+ self.used_shortcuts.add('perform_test')
return 'perform_test(context, %r, %s, %s, %s)' % (
name,
- _to_tuple(args),
+ self.to_tuple(args),
self.handle_node(node.expr),
node.ops[0][0] == 'is not'
)
self.handle_node(node.expr),
self.handle_node(node.subs[0])
)
+ self.used_shortcuts.add('get_attribute')
return 'get_attribute(%s, %s)' % (
self.handle_node(node.expr),
self.handle_node(node.subs[0])
"""
Handle hardcoded attribute access.
"""
+ self.used_shortcuts.add('get_attribute')
return 'get_attribute(%s, %r)' % (
self.handle_node(node.expr),
node.attrname
"""
Tuple unpacking loops.
"""
- return _to_tuple([self.handle_node(n) for n in node.nodes])
+ return self.to_tuple([self.handle_node(n) for n in node.nodes])
def handle_bitor(self, node):
"""
else:
args.append(self.handle_node(arg))
if extra_kwargs:
- kwargs.update(extra_kwargs)
+ kwargs.update(extra_kwargs)
if not (args or kwargs or star_args or dstar_args or extra_kwargs):
+ self.used_shortcuts.add('call_function_simple')
return 'call_function_simple(%s, context)' % \
self.handle_node(node.node)
+ self.used_shortcuts.add('call_function')
return 'call_function(%s, context, %s, {%s}, %s, %s)' % (
self.handle_node(node.node),
- _to_tuple(args),
+ self.to_tuple(args),
', '.join(['%r: %s' % i for i in kwargs.iteritems()]),
star_args,
dstar_args
from types import MethodType, FunctionType
from compiler.ast import CallFunc, Name, Const
from jinja.nodes import Trans
-from jinja.datastructure import Context, Flush
+from jinja.datastructure import Context
from jinja.exceptions import SecurityException, TemplateNotFound
-#: the python2.4 version of deque is missing the remove method
-#: because a for loop with a lookup for the missing value written
-#: in python is slower we just use deque if we have python2.5 or higher
-if sys.version_info >= (2, 5):
+# the python2.4 version of deque is missing the remove method
+# because a for loop with a lookup for the missing value written
+# in python is slower we just use deque if we have python2.5 or higher
+try:
from collections import deque
-else:
+ deque.remove
+except (ImportError, AttributeError):
deque = None
+# support for python 2.3/2.4
try:
set
except NameError:
#: used by from_string as cache
_from_string_env = None
+
+def escape(s, quote=None):
+ """
+ SGML/XML escape an unicode object.
+ """
+ s = s.replace("&", "&").replace("<", "<").replace(">", ">")
+ if not quote:
+ return s
+ return s.replace('"', """)
+
+
def urlize(text, trim_url_limit=None, nofollow=False):
"""
Converts any URLs in text into clickable links. Works on http://,
return u'\n'.join([u'<p>%s</p>' % escape(x) for x in result])
-def flush():
- """
- Yield a flush marker.
- """
- return Flush()
-
-
def watch_changes(env, context, iterable, *attributes):
"""
Wise replacement for ``{% ifchanged %}``.
# python2.4 and lower has a bug regarding joining of broken generators.
# because of the runtime debugging system we have to keep track of the
# number of frames to skip. that's what RUNTIME_EXCEPTION_OFFSET is for.
-if sys.version_info < (2, 5):
- capture_generator = lambda gen: u''.join(tuple(gen))
- RUNTIME_EXCEPTION_OFFSET = 2
-
-# this should be faster and used in python2.5 and higher
-else:
- capture_generator = u''.join
- RUNTIME_EXCEPTION_OFFSET = 1
+try:
+ _test_singleton = object()
+ def _test_gen_bug():
+ raise TypeError(_test_singleton)
+ yield None
+ ''.join(_test_gen_bug())
+except TypeError, e:
+ if e.args and e.args[0] is _test_singleton:
+ capture_generator = u''.join
+ RUNTIME_EXCEPTION_OFFSET = 1
+ else:
+ capture_generator = lambda gen: u''.join(tuple(gen))
+ RUNTIME_EXCEPTION_OFFSET = 2
+del _test_singleton, _test_gen_bug
def buffereater(f):
(macros, filter sections etc)
"""
def wrapped(*args, **kwargs):
- return Flush(capture_generator(f(*args, **kwargs)))
+ return capture_generator(f(*args, **kwargs))
return wrapped
+def empty_block(context):
+ """
+ An empty callable that just returns an empty decorator.
+ Used to represent empty blocks.
+ """
+ if 0: yield None
+
+
def fake_template_exception(exception, filename, lineno, source,
context_or_env):
"""
rv._mapping = deepcopy(self._mapping)
rv._queue = deepcopy(self._queue)
return rv
-
-
-# escaping function. Use this only if you escape unicode
-# objects. in all other cases it's likely that the cgi.escape
-# function performs better.
-try:
- from jinja._speedups import escape
-except ImportError:
- from cgi import escape
except ImportError:
have_mako = False
-table = [dict(a='1',b='2',c='3',d='4',e='5',f='6',g='7',h='8',i='9',j='10')
+table = [dict(zip('abcdefghij', map(unicode,range(1, 11))))
for x in range(1000)]
genshi_tmpl = MarkupTemplate("""
--- /dev/null
+# test file for block super support
+import jdebug
+from jinja import Environment, DictLoader
+
+env = Environment(loader=DictLoader({
+ 'a': '''\
+<title>{{ title|e }}</title>
+<body>
+ {% block body %}Default{% endblock %}
+</body>
+''',
+ 'b': '''
+{% set foo = 42 %}
+''',
+ 'c': '''
+{% extends 'a' %}
+{% if true %}
+ {% set title = "foo" %}
+{% endif %}
+{% include 'b' %}
+{% include 'tools' %}
+{% block body %}
+ hehe, this comes from b: {{ foo }}
+
+ Say hello to the former block content:
+ {{ say_hello(super()) }}
+{% endblock %}
+''',
+ 'tools': '''
+{% macro say_hello name -%}
+ Hello {{ name }}!
+{%- endmacro %}
+'''
+}))
+
+tmpl = env.get_template('c')
+print tmpl.render()
projects / jinja2.git / commitdiff