Implement tokenization in a C extension.

Conflicts:
	History.md
	lib/liquid/tags/assign.rb
	lib/liquid/tags/capture.rb
	lib/liquid/tags/decrement.rb
	lib/liquid/tags/if.rb

.gitignore

@@ -1,3 +1,12 @@
+*~
 *.gem
 *.swp
 pkg
+*.rbc
+.rvmrc
+.ruby-version
+Gemfile.lock
+/ext/liquid/Makefile
+*.o
+*.bundle
+/tmp

.travis.yml

@@ -0,0 +1,16 @@
+rvm:
+  - 1.9.3
+  - 2.0.0
+  - 2.1.0
+  - jruby-19mode
+  - jruby-head
+  - rbx-19mode
+matrix:
+  allow_failures:
+    - rvm: rbx-19mode
+    - rvm: jruby-head
+
+script: "rake test"
+
+notifications:
+  disable: true

CHANGELOG

@@ -1,46 +0,0 @@
-* Make context and assign work the same
-
-* Ruby 1.9.1 bugfixes
-
-* Fix LiquidView for Rails 2.2. Fix local assigns for all versions of Rails
-
-* Fixed gem install rake task
-* Improve Error encapsulation in liquid by maintaining a own set of exceptions instead of relying on ruby build ins
-
-* Added If with or / and expressions
-
-* Implemented .to_liquid for all objects which can be passed to liquid like Strings Arrays Hashes Numerics and Booleans. To export new objects to liquid just implement .to_liquid on them and return objects which themselves have .to_liquid methods.
-
-* Added more tags to standard library
-
-* Added include tag ( like partials in rails )
-
-* [...] Gazillion of detail improvements
-
-* Added strainers as filter hosts for better security [Tobias Luetke]
-
-* Fixed that rails integration would call filter with the wrong "self" [Michael Geary]
-
-* Fixed bad error reporting when a filter called a method which doesn't exist. Liquid told you that it couldn't find the filter which was obviously misleading [Tobias Luetke]
-
-* Removed count helper from standard lib. use size [Tobias Luetke]
-
-* Fixed bug with string filter parameters failing to tolerate commas in strings. [Paul Hammond]
-
-* Improved filter parameters. Filter parameters are now context sensitive; Types are resolved according to the rules of the context. Multiple parameters are now separated by the Liquid::ArgumentSeparator: , by default [Paul Hammond]
-
-  {{ 'Typo' | link_to: 'http://typo.leetsoft.com', 'Typo - a modern weblog engine' }}
-
-
-* Added Liquid::Drop. A base class which you can use for exporting proxy objects to liquid which can acquire more data when used in liquid. [Tobias Luetke]
-
-  class ProductDrop < Liquid::Drop
-    def top_sales
-       Shop.current.products.find(:all, :order => 'sales', :limit => 10 )
-    end
-  end
-  t = Liquid::Template.parse( ' {% for product in product.top_sales %} {{ product.name }} {% endfor %} '  )
-  t.render('product' => ProductDrop.new )
-
-
-* Added filter parameters support. Example: {{ date | format_date: "%Y" }} [Paul Hammond]

CONTRIBUTING.md

@@ -0,0 +1,26 @@
+# How to contribute
+
+## Things we will merge
+
+* Bugfixes
+* Performance improvements
+* Features which are likely to be useful to the majority of Liquid users
+
+## Things we won't merge
+
+* Code which introduces considerable performance degrations
+* Code which touches performance critical parts of Liquid and comes without benchmarks
+* Features which are not important for most people (we want to keep the core Liquid code small and tidy)
+* Features which can easily be implemented on top of Liquid (for example as a custom filter or custom filesystem)
+* Code which comes without tests
+* Code which breaks existing tests
+
+## Workflow
+
+* Fork the Liquid repository
+* Create a new branch in your fork
+* If it makes sense, add tests for your code and run a performance benchmark
+* Make sure all tests pass
+* Create a pull request
+* In the description, ping one of [@boourns](https://github.com/boourns), [@fw42](https://github.com/fw42), [@camilo](https://github.com/camilo), [@dylanahsmith](https://github.com/dylanahsmith), or [@arthurnn](https://github.com/arthurnn) and ask for a code review.
+

Gemfile

@@ -0,0 +1,3 @@
+source 'https://rubygems.org'
+
+gemspec

History.md

@@ -0,0 +1,142 @@
+# Liquid Version History
+
+## 3.0.0 / not yet released / branch "master"
+
+* ...
+* Freeze lots of string literals for new Ruby 2.1 optimization, see #297 [Florian Weingarten, fw42]
+* Allow newlines in tags and variables, see #324 [Dylan Thacker-Smith, dylanahsmith]
+* Tag#parse is called after initialize, which now takes options instead of tokens as the 3rd argument. See #321 [Dylan Thacker-Smith, dylanahsmith]
+* Raise `Liquid::ArgumentError` instead of `::ArgumentError` when filter has wrong number of arguments #309 [Bogdan Gusiev, bogdan]
+* Add a to_s default for liquid drops, see #306 [Adam Doeler, releod]
+* Add strip, lstrip, and rstrip to standard filters [Florian Weingarten, fw42]
+* Make if, for & case tags return complete and consistent nodelists, see #250 [Nick Jones, dntj]
+* Prevent arbitrary method invocation on condition objects, see #274 [Dylan Thacker-Smith, dylanahsmith]
+* Don't call to_sym when creating conditions for security reasons, see #273 [Bouke van der Bijl, bouk]
+* Fix resource counting bug with respond_to?(:length), see #263 [Florian Weingarten, fw42]
+* Allow specifying custom patterns for template filenames, see #284 [Andrei Gladkyi, agladkyi]
+* Allow drops to optimize loading a slice of elements, see #282 [Tom Burns, boourns]
+* Support for passing variables to snippets in subdirs, see #271 [Joost Hietbrink, joost]
+* Add a class cache to avoid runtime extend calls, see #249 [James Tucker, raggi]
+* Remove some legacy Ruby 1.8 compatibility code, see #276 [Florian Weingarten, fw42]
+* Add default filter to standard filters, see #267 [Derrick Reimer, djreimer]
+* Add optional strict parsing and warn parsing, see #235 [Tristan Hume, trishume]
+* Add I18n syntax error translation, see #241 [Simon Hørup Eskildsen, Sirupsen]
+* Make sort filter work on enumerable drops, see #239 [Florian Weingarten, fw42]
+* Fix clashing method names in enumerable drops, see #238 [Florian Weingarten, fw42]
+* Make map filter work on enumerable drops, see #233 [Florian Weingarten, fw42]
+* Improved whitespace stripping for blank blocks, related to #216 [Florian Weingarten, fw42]
+
+## 2.6.0 / 2013-11-25 / branch "2.6-stable"
+
+IMPORTANT: Liquid 2.6 is going to be the last version of Liquid which maintains explicit Ruby 1.8 compatability.
+The following releases will only be tested against Ruby 1.9 and Ruby 2.0 and are likely to break on Ruby 1.8.
+
+* Bugfix for #106: fix example servlet [gnowoel]
+* Bugfix for #97: strip_html filter supports multi-line tags [Jo Liss, joliss]
+* Bugfix for #114: strip_html filter supports style tags [James Allardice, jamesallardice]
+* Bugfix for #117: 'now' support for date filter in Ruby 1.9 [Notre Dame Webgroup, ndwebgroup]
+* Bugfix for #166: truncate filter on UTF-8 strings with Ruby 1.8 [Florian Weingarten, fw42]
+* Bugfix for #204: 'raw' parsing bug [Florian Weingarten, fw42]
+* Bugfix for #150: 'for' parsing bug [Peter Schröder, phoet]
+* Bugfix for #126: Strip CRLF in strip_newline [Peter Schröder, phoet]
+* Bugfix for #174, "can't convert Fixnum into String" for "replace" [wǒ_is神仙, jsw0528]
+* Allow a Liquid::Drop to be passed into Template#render [Daniel Huckstep, darkhelmet]
+* Resource limits [Florian Weingarten, fw42]
+* Add reverse filter [Jay Strybis, unreal]
+* Add utf-8 support
+* Use array instead of Hash to keep the registered filters [Tasos Stathopoulos, astathopoulos]
+* Cache tokenized partial templates [Tom Burns, boourns]
+* Avoid warnings in Ruby 1.9.3 [Marcus Stollsteimer, stomar]
+* Better documentation for 'include' tag (closes #163) [Peter Schröder, phoet]
+* Use of BigDecimal on filters to have better precision (closes #155) [Arthur Nogueira Neves, arthurnn]
+
+## 2.5.4 / 2013-11-11 / branch "2.5-stable"
+
+* Fix "can't convert Fixnum into String" for "replace", see #173, [wǒ_is神仙, jsw0528]
+
+## 2.5.3 / 2013-10-09
+
+* #232, #234, #237: Fix map filter bugs [Florian Weingarten, fw42]
+
+## 2.5.2 / 2013-09-03 / deleted
+
+Yanked from rubygems, as it contained too many changes that broke compatibility. Those changes will be on following major releases.
+
+## 2.5.1 / 2013-07-24
+
+* #230: Fix security issue with map filter, Use invoke_drop in map filter [Florian Weingarten, fw42]
+
+## 2.5.0 / 2013-03-06
+
+* Prevent Object methods from being called on drops
+* Avoid symbol injection from liquid
+* Added break and continue statements
+* Fix filter parser for args without space separators
+* Add support for filter keyword arguments
+
+
+## 2.4.0 / 2012-08-03
+
+* Performance improvements
+* Allow filters in `assign`
+* Add `modulo` filter
+* Ruby 1.8, 1.9, and Rubinius compatibility fixes
+* Add support for `quoted['references']` in `tablerow`
+* Add support for Enumerable to `tablerow`
+* `strip_html` filter removes html comments
+
+
+## 2.3.0 / 2011-10-16
+
+* Several speed/memory improvements
+* Numerous bug fixes
+* Added support for MRI 1.9, Rubinius, and JRuby
+* Added support for integer drop parameters
+* Added epoch support to `date` filter
+* New `raw` tag that suppresses parsing
+* Added `else` option to `for` tag
+* New `increment` tag
+* New `split` filter
+
+
+## 2.2.1 / 2010-08-23
+
+* Added support for literal tags
+
+
+## 2.2.0 / 2010-08-22
+
+* Compatible with Ruby 1.8.7, 1.9.1 and 1.9.2-p0
+* Merged some changed made by the community
+
+
+## 1.9.0 / 2008-03-04
+
+* Fixed gem install rake task
+* Improve Error encapsulation in liquid by maintaining a own set of exceptions instead of relying on ruby build ins
+
+
+## Before 1.9.0
+
+* Added If with or / and expressions
+* Implemented .to_liquid for all objects which can be passed to liquid like Strings Arrays Hashes Numerics and Booleans. To export new objects to liquid just implement .to_liquid on them and return objects which themselves have .to_liquid methods.
+* Added more tags to standard library
+* Added include tag ( like partials in rails )
+* [...] Gazillion of detail improvements
+* Added strainers as filter hosts for better security [Tobias Luetke]
+* Fixed that rails integration would call filter with the wrong "self" [Michael Geary]
+* Fixed bad error reporting when a filter called a method which doesn't exist. Liquid told you that it couldn't find the filter which was obviously misleading [Tobias Luetke]
+* Removed count helper from standard lib. use size [Tobias Luetke]
+* Fixed bug with string filter parameters failing to tolerate commas in strings. [Paul Hammond]
+* Improved filter parameters. Filter parameters are now context sensitive; Types are resolved according to the rules of the context. Multiple parameters are now separated by the Liquid::ArgumentSeparator: , by default [Paul Hammond]
+    {{ 'Typo' | link_to: 'http://typo.leetsoft.com', 'Typo - a modern weblog engine' }}
+* Added Liquid::Drop. A base class which you can use for exporting proxy objects to liquid which can acquire more data when used in liquid. [Tobias Luetke]
+
+  class ProductDrop < Liquid::Drop
+    def top_sales
+       Shop.current.products.find(:all, :order => 'sales', :limit => 10 )
+    end
+  end
+  t = Liquid::Template.parse( ' {% for product in product.top_sales %} {{ product.name }} {% endfor %} '  )
+  t.render('product' => ProductDrop.new )
+* Added filter parameters support. Example: {{ date | format_date: "%Y" }} [Paul Hammond]

History.txt

@@ -1,53 +0,0 @@
-2.2.1 / 2010-08-23
-
-* Added support for literal tags
-
-2.2.0 / 2010-08-22
-
-* Compatible with Ruby 1.8.7, 1.9.1 and 1.9.2-p0
-* Merged some changed made by the community
-
-1.9.0 / 2008-03-04
-
-* Fixed gem install rake task
-* Improve Error encapsulation in liquid by maintaining a own set of exceptions instead of relying on ruby build ins
-
-Before 1.9.0
-
-* Added If with or / and expressions
-
-* Implemented .to_liquid for all objects which can be passed to liquid like Strings Arrays Hashes Numerics and Booleans. To export new objects to liquid just implement .to_liquid on them and return objects which themselves have .to_liquid methods. 
-
-* Added more tags to standard library
-
-* Added include tag ( like partials in rails )
-
-* [...] Gazillion of detail improvements
-
-* Added strainers as filter hosts for better security [Tobias Luetke]
-
-* Fixed that rails integration would call filter with the wrong "self" [Michael Geary]
-
-* Fixed bad error reporting when a filter called a method which doesn't exist. Liquid told you that it couldn't find the filter which was obviously misleading [Tobias Luetke]
-
-* Removed count helper from standard lib. use size [Tobias Luetke]
-
-* Fixed bug with string filter parameters failing to tolerate commas in strings. [Paul Hammond]
-
-* Improved filter parameters. Filter parameters are now context sensitive; Types are resolved according to the rules of the context. Multiple parameters are now separated by the Liquid::ArgumentSeparator: , by default [Paul Hammond]
-	
-	{{ 'Typo' | link_to: 'http://typo.leetsoft.com', 'Typo - a modern weblog engine' }}
-	
-
-* Added Liquid::Drop. A base class which you can use for exporting proxy objects to liquid which can acquire more data when used in liquid. [Tobias Luetke] 
-
-  class ProductDrop < Liquid::Drop
-    def top_sales
-       Shop.current.products.find(:all, :order => 'sales', :limit => 10 )
-    end
-  end   
-  t = Liquid::Template.parse( ' {% for product in product.top_sales %} {{ product.name }} {% endfor %} '  )
-  t.render('product' => ProductDrop.new )
-
-
-* Added filter parameters support. Example: {{ date | format_date: "%Y" }} [Paul Hammond]

Manifest.txt

@@ -1,34 +0,0 @@
-CHANGELOG
-History.txt
-MIT-LICENSE
-Manifest.txt
-README.md
-Rakefile
-init.rb
-lib/extras/liquid_view.rb
-lib/liquid.rb
-lib/liquid/block.rb
-lib/liquid/condition.rb
-lib/liquid/context.rb
-lib/liquid/document.rb
-lib/liquid/drop.rb
-lib/liquid/errors.rb
-lib/liquid/extensions.rb
-lib/liquid/file_system.rb
-lib/liquid/htmltags.rb
-lib/liquid/module_ex.rb
-lib/liquid/standardfilters.rb
-lib/liquid/strainer.rb
-lib/liquid/tag.rb
-lib/liquid/tags/assign.rb
-lib/liquid/tags/capture.rb
-lib/liquid/tags/case.rb
-lib/liquid/tags/comment.rb
-lib/liquid/tags/cycle.rb
-lib/liquid/tags/for.rb
-lib/liquid/tags/if.rb
-lib/liquid/tags/ifchanged.rb
-lib/liquid/tags/include.rb
-lib/liquid/tags/unless.rb
-lib/liquid/template.rb
-lib/liquid/variable.rb

README.md

@@ -1,42 +1,73 @@
+[![Build Status](https://secure.travis-ci.org/Shopify/liquid.png?branch=master)](http://travis-ci.org/Shopify/liquid)
 # Liquid template engine
 
+* [Contributing guidelines](CONTRIBUTING.md)
+* [Version history](History.md)
+* [Liquid documentation from Shopify](http://docs.shopify.com/themes/liquid-basics)
+* [Liquid Wiki at GitHub](https://github.com/Shopify/liquid/wiki)
+* [Website](http://liquidmarkup.org/)
+
 ## Introduction
 
-Liquid is a template engine which I wrote for very specific requirements
+Liquid is a template engine which was written with very specific requirements:
 
 * It has to have beautiful and simple markup. Template engines which don't produce good looking markup are no fun to use.
 * It needs to be non evaling and secure. Liquid templates are made so that users can edit them. You don't want to run code on your server which your users wrote.
-* It has to be stateless. Compile and render steps have to be seperate so that the expensive parsing and compiling can be done once and later on you can just render it passing in a hash with local variables and objects.
+* It has to be stateless. Compile and render steps have to be separate so that the expensive parsing and compiling can be done once and later on you can just render it passing in a hash with local variables and objects.
 
-## Why should I use Liquid
+## Why you should use Liquid
 
 * You want to allow your users to edit the appearance of your application but don't want them to run **insecure code on your server**.
-* You want to render templates directly from the database
+* You want to render templates directly from the database.
-* You like smarty (PHP) style template engines
+* You like smarty (PHP) style template engines.
-* You need a template engine which does HTML just as well as emails
+* You need a template engine which does HTML just as well as emails.
-* You don't like the markup of your current templating engine
+* You don't like the markup of your current templating engine.
 
 ## What does it look like?
 
-<code>
+```html
-  <ul id="products">
+<ul id="products">
   {% for product in products %}
     <li>
-        <h2>{{product.name}}</h2>
+      <h2>{{ product.name }}</h2>
-        Only {{product.price | price }}
+      Only {{ product.price | price }}
 
-        {{product.description | prettyprint | paragraph }}
+      {{ product.description | prettyprint | paragraph }}
     </li>
   {% endfor %}
-  </ul>
+</ul>
-</code>
+```
 
-## Howto use Liquid
+## How to use Liquid
 
 Liquid supports a very simple API based around the Liquid::Template class.
 For standard use you can just pass it the content of a file and call render with a parameters hash.
 
-<pre>
+```ruby
 @template = Liquid::Template.parse("hi {{name}}") # Parses and compiles the template
-@template.render( 'name' => 'tobi' )              # => "hi tobi"
+@template.render('name' => 'tobi')                # => "hi tobi"
-</pre>
+```
+
+### Error Modes
+
+Setting the error mode of Liquid lets you specify how strictly you want your templates to be interpreted.
+Normally the parser is very lax and will accept almost anything without error. Unfortunately this can make
+it very hard to debug and can lead to unexpected behaviour. 
+
+Liquid also comes with a stricter parser that can be used when editing templates to give better error messages
+when templates are invalid. You can enable this new parser like this:
+
+```ruby
+Liquid::Template.error_mode = :strict # Raises a SyntaxError when invalid syntax is used
+Liquid::Template.error_mode = :warn # Adds errors to template.errors but continues as normal
+Liquid::Template.error_mode = :lax # The default mode, accepts almost anything.
+```
+
+If you want to set the error mode only on specific templates you can pass `:error_mode` as an option to `parse`:
+```ruby
+Liquid::Template.parse(source, :error_mode => :strict)
+```
+This is useful for doing things like enabling strict mode only in the theme editor.
+
+It is recommended that you enable `:strict` or `:warn` mode on new apps to stop invalid templates from being created.
+It is also recommended that you use it in the template editors of existing apps to give editors better error messages.

Rakefile

@@ -1,43 +1,85 @@
-#!/usr/bin/env ruby
-$:.unshift File.join(File.dirname(__FILE__), 'test') unless $:.include? File.join(File.dirname(__FILE__), 'test')
-
-require 'rubygems'
 require 'rake'
 require 'rake/testtask'
-require 'rake/gempackagetask'
+$LOAD_PATH.unshift File.expand_path("../lib", __FILE__)
+require "liquid/version"
 
 task :default => 'test'
 
-Rake::TestTask.new(:test) do |t|
+desc 'run test suite with default parser'
+Rake::TestTask.new(:base_test) do |t|
   t.libs << '.' << 'lib' << 'test'
-  t.pattern = 'test/lib/**/*_test.rb'
+  t.test_files = FileList['test/liquid/**/*_test.rb']
   t.verbose = false
 end
 
-gemspec = eval(File.read('liquid.gemspec'))
+desc 'run test suite with warn error mode'
-Rake::GemPackageTask.new(gemspec) do |pkg|
+task :warn_test do
-  pkg.gem_spec = gemspec
+  ENV['LIQUID_PARSER_MODE'] = 'warn'
+  Rake::Task['base_test'].invoke
 end
 
-desc "build the gem and release it to rubygems.org"
+desc 'runs test suite with both strict and lax parsers'
-task :release => :gem do
+task :test do
-  sh "gem push pkg/liquid-#{gemspec.version}.gem"
+  ENV['LIQUID_PARSER_MODE'] = 'lax'
+  Rake::Task['base_test'].invoke
+  ENV['LIQUID_PARSER_MODE'] = 'strict'
+  Rake::Task['base_test'].reenable
+  Rake::Task['base_test'].invoke
 end
 
+task :gem => :build
+task :build do
+  system "gem build liquid.gemspec"
+end
+
+task :install => :build do
+  system "gem install liquid-#{Liquid::VERSION}.gem"
+end
+
+task :release => :build do
+  system "git tag -a v#{Liquid::VERSION} -m 'Tagging #{Liquid::VERSION}'"
+  system "git push --tags"
+  system "gem push liquid-#{Liquid::VERSION}.gem"
+  system "rm liquid-#{Liquid::VERSION}.gem"
+end
+
+namespace :benchmark do
+
+  desc "Run the liquid benchmark with lax parsing"
+  task :run do
+    ruby "./performance/benchmark.rb lax"
+  end
+
+  desc "Run the liquid benchmark with strict parsing"
+  task :strict do
+    ruby "./performance/benchmark.rb strict"
+  end
+end
+
+
 namespace :profile do
 
-  task :default => [:run]
+  desc "Run the liquid profile/performance coverage"
-
-  desc "Run the liquid profile/perforamce coverage"
   task :run do
-
+    ruby "./performance/profile.rb"
-    ruby "performance/shopify.rb"
-
   end
 
-  desc "Run KCacheGrind"
+  desc "Run the liquid profile/performance coverage with strict parsing"
-  task :grind => :run  do
+  task :strict do
-    system "kcachegrind /tmp/liquid.rubyprof_calltreeprinter.txt"
+    ruby "./performance/profile.rb strict"
   end
 
 end
+
+desc "Run example"
+task :example do
+  ruby "-w -d -Ilib example/server/server.rb"
+end
+
+if defined?(RUBY_ENGINE) && RUBY_ENGINE == 'ruby'
+  require 'rake/extensiontask'
+  Rake::ExtensionTask.new "liquid" do |ext|
+    ext.lib_dir = "lib/liquid"
+  end
+  Rake::Task[:test].prerequisites << :compile
+end

example/server/example_servlet.rb

@@ -23,7 +23,7 @@ class Servlet < LiquidServlet
   end
 
   def products
-    { 'products' => products_list, 'section' => 'Snowboards', 'cool_products' => true}    
+    { 'products' => products_list, 'description' => description, 'section' => 'Snowboards', 'cool_products' => true}
   end
 
   private
@@ -34,4 +34,8 @@ class Servlet < LiquidServlet
     {'name' => 'Arbor Diamond', 'price' => 59900, 'description' => 'the *arbor diamond* is a made up product because im obsessed with arbor and have no creativity'}]
   end
 
+  def description
+    "List of Products ~ This is a list of products with price and description."
+  end
+
 end

example/server/liquid_servlet.rb

@@ -13,7 +13,7 @@ class LiquidServlet < WEBrick::HTTPServlet::AbstractServlet
   def handle(type, req, res)
     @request, @response = req, res
 
-    @request.path_info =~ /(\w+)$/
+    @request.path_info =~ /(\w+)\z/
     @action = $1 || 'index'
     @assigns = send(@action) if respond_to?(@action)
 

example/server/server.rb

@@ -1,9 +1,11 @@
 require 'webrick'
 require 'rexml/document'
 
-require File.dirname(__FILE__) + '/../../lib/liquid'
+DIR = File.expand_path(File.dirname(__FILE__))
-require File.dirname(__FILE__) + '/liquid_servlet'
+
-require File.dirname(__FILE__) + '/example_servlet'
+require DIR + '/../../lib/liquid'
+require DIR + '/liquid_servlet'
+require DIR + '/example_servlet'
 
 # Setup webrick
 server = WEBrick::HTTPServer.new( :Port => ARGV[1] || 3000 )

example/server/templates/products.liquid

@@ -17,7 +17,11 @@
 
   <body>
 
-	  <h1>There are currently {{products | count}} products in the {{section}} catalog</h1>
+    <h1>{{ description | split: '~' | first }}</h1>
+
+    <h2>{{ description | split: '~' | last }}</h2>
+
+    <h2>There are currently {{products | count}} products in the {{section}} catalog</h2>
 
     {% if cool_products %}
       Cool products :)

ext/liquid/extconf.rb

@@ -0,0 +1,4 @@
+require 'mkmf'
+$CFLAGS << ' -Wall -Werror'
+$warnflags.gsub!(/-Wdeclaration-after-statement/, "")
+create_makefile("liquid/liquid")

ext/liquid/liquid.c

@@ -0,0 +1,9 @@
+#include "liquid.h"
+
+VALUE mLiquid;
+
+void Init_liquid(void)
+{
+    mLiquid = rb_define_module("Liquid");
+    init_liquid_tokenizer();
+}

ext/liquid/liquid.h

@@ -0,0 +1,11 @@
+#ifndef LIQUID_H
+#define LIQUID_H
+
+#include <ruby.h>
+#include <stdbool.h>
+
+#include "tokenizer.h"
+
+extern VALUE mLiquid;
+
+#endif

ext/liquid/tokenizer.c

@@ -0,0 +1,137 @@
+#include "liquid.h"
+
+VALUE cLiquidTokenizer;
+
+static void tokenizer_mark(void *ptr) {
+    tokenizer_t *tokenizer = ptr;
+    rb_gc_mark(tokenizer->source);
+}
+
+static void tokenizer_free(void *ptr)
+{
+    tokenizer_t *tokenizer = ptr;
+    xfree(tokenizer);
+}
+
+static size_t tokenizer_memsize(const void *ptr)
+{
+    return ptr ? sizeof(tokenizer_t) : 0;
+}
+
+const rb_data_type_t tokenizer_data_type = {
+    "liquid_tokenizer",
+    {tokenizer_mark, tokenizer_free, tokenizer_memsize,},
+#ifdef RUBY_TYPED_FREE_IMMEDIATELY
+    NULL, NULL, RUBY_TYPED_FREE_IMMEDIATELY
+#endif
+};
+
+static VALUE tokenizer_allocate(VALUE klass)
+{
+    VALUE obj;
+    tokenizer_t *tokenizer;
+
+    obj = TypedData_Make_Struct(klass, tokenizer_t, &tokenizer_data_type, tokenizer);
+    tokenizer->source = Qnil;
+    return obj;
+}
+
+static VALUE tokenizer_initialize_method(VALUE self, VALUE source)
+{
+    tokenizer_t *tokenizer;
+
+    Check_Type(source, T_STRING);
+    Tokenizer_Get_Struct(self, tokenizer);
+    source = rb_str_dup_frozen(source);
+    tokenizer->source = source;
+    tokenizer->cursor = RSTRING_PTR(source);
+    tokenizer->length = RSTRING_LEN(source);
+    return Qnil;
+}
+
+void tokenizer_next(tokenizer_t *tokenizer, token_t *token)
+{
+    if (tokenizer->length <= 0) {
+        memset(token, 0, sizeof(*token));
+        return;
+    }
+
+    const char *cursor = tokenizer->cursor;
+    const char *last = cursor + tokenizer->length - 1;
+
+    token->str = cursor;
+    token->type = TOKEN_STRING;
+
+    while (cursor < last) {
+        if (*cursor++ != '{')
+            continue;
+
+        char c = *cursor++;
+        if (c != '%' && c != '{')
+            continue;
+        if (cursor - tokenizer->cursor > 2) {
+            token->type = TOKEN_STRING;
+            cursor -= 2;
+            goto found;
+        }
+        token->type = TOKEN_INVALID;
+        if (c == '%') {
+            while (cursor < last) {
+                if (*cursor++ != '%')
+                    continue;
+                c = *cursor++;
+                while (c == '%' && cursor <= last)
+                    c = *cursor++;
+                if (c != '}')
+                    continue;
+                token->type = TOKEN_TAG;
+                goto found;
+            }
+            // unterminated tag
+            cursor = tokenizer->cursor + 2;
+            goto found;
+        } else {
+            while (cursor < last) {
+                if (*cursor++ != '}')
+                    continue;
+                if (*cursor++ != '}') {
+                    // variable incomplete end, used to end raw tags
+                    cursor--;
+                    goto found;
+                }
+                token->type = TOKEN_VARIABLE;
+                goto found;
+            }
+            // unterminated variable
+            cursor = tokenizer->cursor + 2;
+            goto found;
+        }
+    }
+    cursor = last + 1;
+found:
+    token->length = cursor - tokenizer->cursor;
+    tokenizer->cursor += token->length;
+    tokenizer->length -= token->length;
+}
+
+static VALUE tokenizer_next_method(VALUE self)
+{
+    tokenizer_t *tokenizer;
+    Tokenizer_Get_Struct(self, tokenizer);
+
+    token_t token;
+    tokenizer_next(tokenizer, &token);
+    if (token.type == TOKEN_NONE)
+        return Qnil;
+
+    return rb_str_new(token.str, token.length);
+}
+
+void init_liquid_tokenizer()
+{
+    cLiquidTokenizer = rb_define_class_under(mLiquid, "Tokenizer", rb_cObject);
+    rb_define_alloc_func(cLiquidTokenizer, tokenizer_allocate);
+    rb_define_method(cLiquidTokenizer, "initialize", tokenizer_initialize_method, 1);
+    rb_define_method(cLiquidTokenizer, "next", tokenizer_next_method, 0);
+    rb_define_alias(cLiquidTokenizer, "shift", "next");
+}

ext/liquid/tokenizer.h

@@ -0,0 +1,31 @@
+#ifndef LIQUID_TOKENIZER_H
+#define LIQUID_TOKENIZER_H
+
+enum token_type {
+    TOKEN_NONE,
+    TOKEN_INVALID,
+    TOKEN_STRING,
+    TOKEN_TAG,
+    TOKEN_VARIABLE
+};
+
+typedef struct token {
+    enum token_type type;
+    const char *str;
+    long length;
+} token_t;
+
+typedef struct tokenizer {
+    VALUE source;
+    const char *cursor;
+    long length;
+} tokenizer_t;
+
+extern VALUE cLiquidTokenizer;
+extern const rb_data_type_t tokenizer_data_type;
+#define Tokenizer_Get_Struct(obj, sval) TypedData_Get_Struct(obj, tokenizer_t, &tokenizer_data_type, sval)
+
+void init_liquid_tokenizer();
+void tokenizer_next(tokenizer_t *tokenizer, token_t *token);
+
+#endif

lib/liquid.rb

@@ -19,38 +19,31 @@
 # OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
 # WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
 
-$LOAD_PATH.unshift(File.dirname(__FILE__))
-
 module Liquid
   FilterSeparator             = /\|/
-  ArgumentSeparator           = ','
+  ArgumentSeparator           = ','.freeze
-  FilterArgumentSeparator     = ':'
+  FilterArgumentSeparator     = ':'.freeze
-  VariableAttributeSeparator  = '.'
+  VariableAttributeSeparator  = '.'.freeze
   TagStart                    = /\{\%/
   TagEnd                      = /\%\}/
   VariableSignature           = /\(?[\w\-\.\[\]]\)?/
   VariableSegment             = /[\w\-]/
   VariableStart               = /\{\{/
   VariableEnd                 = /\}\}/
-  VariableIncompleteEnd       = /\}\}?/
   QuotedString                = /"[^"]*"|'[^']*'/
-  QuotedFragment              = /#{QuotedString}|(?:[^\s,\|'"]|#{QuotedString})+/
+  QuotedFragment              = /#{QuotedString}|(?:[^\s,\|'"]|#{QuotedString})+/o
-  StrictQuotedFragment        = /"[^"]+"|'[^']+'|[^\s,\|,\:,\,]+/
+  TagAttributes               = /(\w+)\s*\:\s*(#{QuotedFragment})/o
-  FirstFilterArgument         = /#{FilterArgumentSeparator}(?:#{StrictQuotedFragment})/
+  VariableParser              = /\[[^\]]+\]|#{VariableSegment}+\??/o
-  OtherFilterArgument         = /#{ArgumentSeparator}(?:#{StrictQuotedFragment})/
-  SpacelessFilter             = /^(?:'[^']+'|"[^"]+"|[^'"])*#{FilterSeparator}(?:#{StrictQuotedFragment})(?:#{FirstFilterArgument}(?:#{OtherFilterArgument})*)?/
-  Expression                  = /(?:#{QuotedFragment}(?:#{SpacelessFilter})*)/
-  TagAttributes               = /(\w+)\s*\:\s*(#{QuotedFragment})/
-  AnyStartingTag              = /\{\{|\{\%/
-  PartialTemplateParser       = /#{TagStart}.*?#{TagEnd}|#{VariableStart}.*?#{VariableIncompleteEnd}/
-  TemplateParser              = /(#{PartialTemplateParser}|#{AnyStartingTag})/
-  VariableParser              = /\[[^\]]+\]|#{VariableSegment}+\??/
-  LiteralShorthand            = /^(?:\{\{\{\s?)(.*?)(?:\s*\}\}\})$/
 end
 
+require "liquid/version"
+require 'liquid/lexer'
+require 'liquid/parser'
+require 'liquid/i18n'
 require 'liquid/drop'
 require 'liquid/extensions'
 require 'liquid/errors'
+require 'liquid/interrupts'
 require 'liquid/strainer'
 require 'liquid/context'
 require 'liquid/tag'
@@ -59,11 +52,17 @@ require 'liquid/document'
 require 'liquid/variable'
 require 'liquid/file_system'
 require 'liquid/template'
-require 'liquid/htmltags'
 require 'liquid/standardfilters'
 require 'liquid/condition'
 require 'liquid/module_ex'
+require 'liquid/utils'
 
 # Load all the tags of the standard library
 #
 Dir[File.dirname(__FILE__) + '/liquid/tags/*.rb'].each { |f| require f }
+
+if defined?(RUBY_ENGINE) && RUBY_ENGINE == 'ruby'
+  require 'liquid/liquid'
+else
+  require 'liquid/tokenizer'
+end

lib/liquid/block.rb

@@ -1,22 +1,28 @@
 module Liquid
-
   class Block < Tag
-    IsTag             = /^#{TagStart}/
+    IsTag             = /\A#{TagStart}/o
-    IsVariable        = /^#{VariableStart}/
+    IsVariable        = /\A#{VariableStart}/o
-    FullToken         = /^#{TagStart}\s*(\w+)\s*(.*)?#{TagEnd}$/
+    FullToken         = /\A#{TagStart}\s*(\w+)\s*(.*)?#{TagEnd}\z/om
-    ContentOfVariable = /^#{VariableStart}(.*)#{VariableEnd}$/
+    ContentOfVariable = /\A#{VariableStart}(.*)#{VariableEnd}\z/om
+
+    def blank?
+      @blank || false
+    end
 
     def parse(tokens)
+      @blank = true
       @nodelist ||= []
       @nodelist.clear
 
-      while token = tokens.shift
+      # All child tags of the current block.
+      @children = []
 
+      while token = tokens.shift
         case token
         when IsTag
           if token =~ FullToken
 
-            # if we found the proper block delimitor just end parsing here and let the outer block
+            # if we found the proper block delimiter just end parsing here and let the outer block
             # proceed
             if block_delimiter == $1
               end_tag
@@ -25,41 +31,63 @@ module Liquid
 
             # fetch the tag from registered blocks
             if tag = Template.tags[$1]
-              @nodelist << tag.new($1, $2, tokens)
+              new_tag = tag.parse($1, $2, tokens, @options)
+              @blank &&= new_tag.blank?
+              @nodelist << new_tag
+              @children << new_tag
             else
               # this tag is not registered with the system
               # pass it to the current block for special handling or error reporting
               unknown_tag($1, $2, tokens)
             end
           else
-            raise SyntaxError, "Tag '#{token}' was not properly terminated with regexp: #{TagEnd.inspect} "
+            raise SyntaxError.new(options[:locale].t("errors.syntax.tag_termination".freeze, :token => token, :tag_end => TagEnd.inspect))
           end
         when IsVariable
-          @nodelist << create_variable(token)
+          new_var = create_variable(token)
-        when ''
+          @nodelist << new_var
+          @children << new_var
+          @blank = false
+        when ''.freeze
           # pass
         else
           @nodelist << token
+          @blank &&= (token =~ /\A\s*\z/)
         end
       end
 
-      # Make sure that its ok to end parsing in the current block.
+      # Make sure that it's ok to end parsing in the current block.
-      # Effectively this method will throw and exception unless the current block is
+      # Effectively this method will throw an exception unless the current block is
       # of type Document
       assert_missing_delimitation!
     end
 
+    # warnings of this block and all sub-tags
+    def warnings
+      all_warnings = []
+      all_warnings.concat(@warnings) if @warnings
+
+      (@children || []).each do |node|
+        all_warnings.concat(node.warnings || [])
+      end
+
+      all_warnings
+    end
+
     def end_tag
     end
 
     def unknown_tag(tag, params, tokens)
       case tag
-      when 'else'
+      when 'else'.freeze
-        raise SyntaxError, "#{block_name} tag does not expect else tag"
+        raise SyntaxError.new(options[:locale].t("errors.syntax.unexpected_else".freeze,
-      when 'end'
+                                                 :block_name => block_name))
-        raise SyntaxError, "'end' is not a valid delimiter for #{block_name} tags. use #{block_delimiter}"
+      when 'end'.freeze
+        raise SyntaxError.new(options[:locale].t("errors.syntax.invalid_delimiter".freeze,
+                                                 :block_name => block_name,
+                                                 :block_delimiter => block_delimiter))
       else
-        raise SyntaxError, "Unknown tag '#{tag}'"
+        raise SyntaxError.new(options[:locale].t("errors.syntax.unknown_tag".freeze, :tag => tag))
       end
     end
 
@@ -73,9 +101,9 @@ module Liquid
 
     def create_variable(token)
       token.scan(ContentOfVariable) do |content|
-        return Variable.new(content.first)
+        return Variable.new(content.first, @options)
       end
-      raise SyntaxError.new("Variable '#{token}' was not properly terminated with regexp: #{VariableEnd.inspect} ")
+      raise SyntaxError.new(options[:locale].t("errors.syntax.variable_termination".freeze, :token => token, :tag_end => VariableEnd.inspect))
     end
 
     def render(context)
@@ -85,17 +113,44 @@ module Liquid
     protected
 
     def assert_missing_delimitation!
-      raise SyntaxError.new("#{block_name} tag was never closed")
+      raise SyntaxError.new(options[:locale].t("errors.syntax.tag_never_closed".freeze, :block_name => block_name))
     end
 
     def render_all(list, context)
-      list.collect do |token|
+      output = []
+      context.resource_limits[:render_length_current] = 0
+      context.resource_limits[:render_score_current] += list.length
+
+      list.each do |token|
+        # Break out if we have any unhanded interrupts.
+        break if context.has_interrupt?
+
         begin
-          token.respond_to?(:render) ? token.render(context) : token
+          # If we get an Interrupt that means the block must stop processing. An
-        rescue Exception => e
+          # Interrupt is any command that stops block execution such as {% break %}
-          context.handle_error(e)
+          # or {% continue %}
+          if token.is_a? Continue or token.is_a? Break
+            context.push_interrupt(token.interrupt)
+            break
+          end
+
+          token_output = (token.respond_to?(:render) ? token.render(context) : token)
+          context.increment_used_resources(:render_length_current, token_output)
+          if context.resource_limits_reached?
+            context.resource_limits[:reached] = true
+            raise MemoryError.new("Memory limits exceeded".freeze)
+          end
+          unless token.is_a?(Block) && token.blank?
+            output << token_output
+          end
+        rescue MemoryError => e
+          raise e
+        rescue ::StandardError => e
+          output << (context.handle_error(e))
         end
       end
+
+      output.join
     end
   end
 end

lib/liquid/condition.rb

@@ -8,14 +8,14 @@ module Liquid
   #
   class Condition #:nodoc:
     @@operators = {
-      '==' => lambda { |cond, left, right|  cond.send(:equal_variables, left, right) },
+      '=='.freeze => lambda { |cond, left, right|  cond.send(:equal_variables, left, right) },
-      '!=' => lambda { |cond, left, right| !cond.send(:equal_variables, left, right) },
+      '!='.freeze => lambda { |cond, left, right| !cond.send(:equal_variables, left, right) },
-      '<>' => lambda { |cond, left, right| !cond.send(:equal_variables, left, right) },
+      '<>'.freeze => lambda { |cond, left, right| !cond.send(:equal_variables, left, right) },
-      '<'  => :<,
+      '<'.freeze  => :<,
-      '>'  => :>,
+      '>'.freeze  => :>,
-      '>=' => :>=,
+      '>='.freeze => :>=,
-      '<=' => :<=,
+      '<='.freeze => :<=,
-      'contains' => lambda { |cond, left, right| left && right ? left.include?(right) : false }
+      'contains'.freeze => lambda { |cond, left, right| left && right ? left.include?(right) : false }
     }
 
     def self.operators
@@ -61,7 +61,7 @@ module Liquid
     end
 
     def inspect
-      "#<Condition #{[@left, @operator, @right].compact.join(' ')}>"
+      "#<Condition #{[@left, @operator, @right].compact.join(' '.freeze)}>"
     end
 
     private

lib/liquid/context.rb

@@ -13,19 +13,37 @@ module Liquid
   #
   #   context['bob']  #=> nil  class Context
   class Context
-    attr_reader :scopes, :errors, :registers, :environments
+    attr_reader :scopes, :errors, :registers, :environments, :resource_limits
 
-    def initialize(environments = {}, outer_scope = {}, registers = {}, rethrow_errors = false)
+    def initialize(environments = {}, outer_scope = {}, registers = {}, rethrow_errors = false, resource_limits = {})
       @environments    = [environments].flatten
       @scopes          = [(outer_scope || {})]
       @registers       = registers
       @errors          = []
       @rethrow_errors  = rethrow_errors
+      @resource_limits = (resource_limits || {}).merge!({ :render_score_current => 0, :assign_score_current => 0 })
       squash_instance_assigns_with_environments
+
+      @interrupts = []
+      @filters = []
+    end
+
+    def increment_used_resources(key, obj)
+      @resource_limits[key] += if obj.kind_of?(String) || obj.kind_of?(Array) || obj.kind_of?(Hash)
+        obj.length
+      else
+        1
+      end
+    end
+
+    def resource_limits_reached?
+      (@resource_limits[:render_length_limit] && @resource_limits[:render_length_current] > @resource_limits[:render_length_limit]) ||
+      (@resource_limits[:render_score_limit]  && @resource_limits[:render_score_current]  > @resource_limits[:render_score_limit] ) ||
+      (@resource_limits[:assign_score_limit]  && @resource_limits[:assign_score_current]  > @resource_limits[:assign_score_limit] )
     end
 
     def strainer
-      @strainer ||= Strainer.create(self)
+      @strainer ||= Strainer.create(self, @filters)
     end
 
     # Adds filters to this context.
@@ -34,11 +52,36 @@ module Liquid
     # for that
     def add_filters(filters)
       filters = [filters].flatten.compact
-
       filters.each do |f|
         raise ArgumentError, "Expected module but got: #{f.class}" unless f.is_a?(Module)
+        Strainer.add_known_filter(f)
+      end
+
+      # If strainer is already setup then there's no choice but to use a runtime
+      # extend call. If strainer is not yet created, we can utilize strainers
+      # cached class based API, which avoids busting the method cache.
+      if @strainer
+        filters.each do |f|
           strainer.extend(f)
         end
+      else
+        @filters.concat filters
+      end
+    end
+
+    # are there any not handled interrupts?
+    def has_interrupt?
+      @interrupts.any?
+    end
+
+    # push an interrupt to the stack. this interrupt is considered not handled.
+    def push_interrupt(e)
+      @interrupts.push(e)
+    end
+
+    # pop an interrupt from the stack
+    def pop_interrupt
+      @interrupts.pop
     end
 
     def handle_error(e)
@@ -54,17 +97,13 @@ module Liquid
     end
 
     def invoke(method, *args)
-      if strainer.respond_to?(method)
+      strainer.invoke(method, *args)
-        strainer.__send__(method, *args)
-      else
-        args.first
-      end
     end
 
     # Push new local scope on the stack. use <tt>Context#stack</tt> instead
     def push(new_scope={})
-      raise StackLevelError, "Nesting too deep" if @scopes.length > 100
       @scopes.unshift(new_scope)
+      raise StackLevelError, "Nesting too deep".freeze if @scopes.length > 100
     end
 
     # Merge a hash of variables in the current local scope
@@ -86,19 +125,13 @@ module Liquid
     #   end
     #
     #   context['var]  #=> nil
-    def stack(new_scope={},&block)
+    def stack(new_scope={})
-      result = nil
       push(new_scope)
-
+      yield
-      begin
-        result = yield
     ensure
       pop
     end
 
-      result
-    end
-
     def clear_instance_assigns
       @scopes[0] = {}
     end
@@ -117,6 +150,14 @@ module Liquid
     end
 
     private
+      LITERALS = {
+        nil => nil, 'nil'.freeze => nil, 'null'.freeze => nil, ''.freeze => nil,
+        'true'.freeze  => true,
+        'false'.freeze => false,
+        'blank'.freeze => :blank?,
+        'empty'.freeze => :empty?
+      }
+
       # Look up variable, either resolve directly after considering the name. We can directly handle
       # Strings, digits, floats and booleans (true,false).
       # If no match is made we lookup the variable in the current scope and
@@ -126,35 +167,30 @@ module Liquid
       # Example:
       #   products == empty #=> products.empty?
       def resolve(key)
+        if LITERALS.key?(key)
+          LITERALS[key]
+        else
           case key
-        when nil, 'nil', 'null', ''
+          when /\A'(.*)'\z/m # Single quoted strings
-          nil
+            $1
-        when 'true'
+          when /\A"(.*)"\z/m # Double quoted strings
-          true
+            $1
-        when 'false'
+          when /\A(-?\d+)\z/ # Integer and floats
-          false
-        when 'blank'
-          :blank?
-        when 'empty' # Single quoted strings
-          :empty?
-        when /^'(.*)'$/ # Double quoted strings
-          $1.to_s
-        when /^"(.*)"$/ # Integer and floats
-          $1.to_s
-        when /^(\d+)$/ # Ranges
             $1.to_i
-        when /^\((\S+)\.\.(\S+)\)$/ # Floats
+          when /\A\((\S+)\.\.(\S+)\)\z/ # Ranges
             (resolve($1).to_i..resolve($2).to_i)
-        when /^(\d[\d\.]+)$/
+          when /\A(-?\d[\d\.]+)\z/ # Floats
             $1.to_f
           else
             variable(key)
           end
         end
+      end
 
       # Fetches an object starting at the local scope and then moving up the hierachy
       def find_variable(key)
         scope = @scopes.find { |s| s.has_key?(key) }
+        variable = nil
 
         if scope.nil?
           @environments.each do |e|
@@ -182,7 +218,7 @@ module Liquid
       #  assert_equal 'tobi', @context['hash["name"]']
       def variable(markup)
         parts = markup.scan(VariableParser)
-        square_bracketed = /^\[(.*)\]$/
+        square_bracketed = /\A\[(.*)\]\z/m
 
         first_part = parts.shift
 
@@ -208,7 +244,7 @@ module Liquid
               # Some special cases. If the part wasn't in square brackets and
               # no key with the same name was found we interpret following calls
               # as commands and call them on the current object
-            elsif !part_resolved and object.respond_to?(part) and ['size', 'first', 'last'].include?(part)
+            elsif !part_resolved and object.respond_to?(part) and ['size'.freeze, 'first'.freeze, 'last'.freeze].include?(part)
 
               object = object.send(part.intern).to_liquid
 

lib/liquid/document.rb

@@ -1,11 +1,11 @@
 module Liquid
   class Document < Block
+    def self.parse(tokens, options={})
       # we don't need markup to open this block
-    def initialize(tokens)
+      super(nil, nil, tokens, options)
-      parse(tokens)
     end
 
-    # There isn't a real delimter
+    # There isn't a real delimiter
     def block_delimiter
       []
     end

lib/liquid/drop.rb

@@ -1,10 +1,12 @@
+require 'set'
+
 module Liquid
 
-  # A drop in liquid is a class which allows you to to export DOM like things to liquid
+  # A drop in liquid is a class which allows you to export DOM like things to liquid.
   # Methods of drops are callable.
-  # The main use for liquid drops is the implement lazy loaded objects.
+  # The main use for liquid drops is to implement lazy loaded objects.
   # If you would like to make data available to the web designers which you don't want loaded unless needed then
-  # a drop is a great way to do that
+  # a drop is a great way to do that.
   #
   # Example:
   #
@@ -18,23 +20,23 @@ module Liquid
   #   tmpl.render('product' => ProductDrop.new ) # will invoke top_sales query.
   #
   # Your drop can either implement the methods sans any parameters or implement the before_method(name) method which is a
-  # catch all
+  # catch all.
   class Drop
     attr_writer :context
 
+    EMPTY_STRING = ''.freeze
+
     # Catch all for the method
     def before_method(method)
       nil
     end
 
     # called by liquid to invoke a drop
-    def invoke_drop(method)
+    def invoke_drop(method_or_key)
-      # for backward compatibility with Ruby 1.8
+      if method_or_key && method_or_key != EMPTY_STRING && self.class.invokable?(method_or_key)
-      methods = self.class.public_instance_methods.map { |m| m.to_s }
+        send(method_or_key)
-      if methods.include?(method.to_s)
-        send(method.to_sym)
       else
-        before_method(method)
+        before_method(method_or_key)
       end
     end
 
@@ -42,10 +44,34 @@ module Liquid
       true
     end
 
+    def inspect
+      self.class.to_s
+    end
+
     def to_liquid
       self
     end
 
+    def to_s
+      self.class.name
+    end
+
     alias :[] :invoke_drop
+
+    private
+
+    # Check for method existence without invoking respond_to?, which creates symbols
+    def self.invokable?(method_name)
+      unless @invokable_methods
+        blacklist = Liquid::Drop.public_instance_methods + [:each]
+        if include?(Enumerable)
+          blacklist += Enumerable.public_instance_methods
+          blacklist -= [:sort, :count, :first, :min, :max, :include?]
+        end
+        whitelist = [:to_liquid] + (public_instance_methods - blacklist)
+        @invokable_methods = Set.new(whitelist.map(&:to_s))
+      end
+      @invokable_methods.include?(method_name.to_s)
+    end
   end
 end

lib/liquid/errors.rb

@@ -8,4 +8,5 @@ module Liquid
   class StandardError < Error; end
   class SyntaxError < Error; end
   class StackLevelError < Error; end
+  class MemoryError < Error; end
 end

lib/liquid/extensions.rb

@@ -43,14 +43,20 @@ class Date  # :nodoc:
   end
 end
 
-def true.to_liquid  # :nodoc:
+class TrueClass
+  def to_liquid  # :nodoc:
     self
+  end
 end
 
-def false.to_liquid # :nodoc:
+class FalseClass
+  def to_liquid # :nodoc:
     self
+  end
 end
 
-def nil.to_liquid # :nodoc:
+class NilClass
+  def to_liquid # :nodoc:
     self
+  end
 end

lib/liquid/file_system.rb

@@ -1,5 +1,5 @@
 module Liquid
-  # A Liquid file system is way to let your templates retrieve other templates for use with the include tag.
+  # A Liquid file system is a way to let your templates retrieve other templates for use with the include tag.
   #
   # You can implement subclasses that retrieve templates from the database, from the file system using a different
   # path structure, you can provide them as hard-coded inline strings, or any manner that you see fit.
@@ -14,7 +14,7 @@ module Liquid
   # This will parse the template with a LocalFileSystem implementation rooted at 'template_path'.
   class BlankFileSystem
     # Called by Liquid to retrieve a template file
-    def read_template_file(template_path)
+    def read_template_file(template_path, context)
       raise FileSystemError, "This liquid context does not allow includes."
     end
   end
@@ -31,14 +31,25 @@ module Liquid
   # file_system.full_path("mypartial")       # => "/some/path/_mypartial.liquid"
   # file_system.full_path("dir/mypartial")   # => "/some/path/dir/_mypartial.liquid"
   #
+  # Optionally in the second argument you can specify a custom pattern for template filenames.
+  # The Kernel::sprintf format specification is used.
+  # Default pattern is "_%s.liquid".
+  #
+  # Example:
+  #
+  # file_system = Liquid::LocalFileSystem.new("/some/path", "%s.html")
+  #
+  # file_system.full_path("index") # => "/some/path/index.html"
+  #
   class LocalFileSystem
     attr_accessor :root
 
-    def initialize(root)
+    def initialize(root, pattern = "_%s.liquid".freeze)
       @root = root
+      @pattern = pattern
     end
 
-    def read_template_file(template_path)
+    def read_template_file(template_path, context)
       full_path = full_path(template_path)
       raise FileSystemError, "No such template '#{template_path}'" unless File.exists?(full_path)
 
@@ -46,15 +57,15 @@ module Liquid
     end
 
     def full_path(template_path)
-      raise FileSystemError, "Illegal template name '#{template_path}'" unless template_path =~ /^[^.\/][a-zA-Z0-9_\/]+$/
+      raise FileSystemError, "Illegal template name '#{template_path}'" unless template_path =~ /\A[^.\/][a-zA-Z0-9_\/]+\z/
 
-      full_path = if template_path.include?('/')
+      full_path = if template_path.include?('/'.freeze)
-        File.join(root, File.dirname(template_path), "_#{File.basename(template_path)}.liquid")
+        File.join(root, File.dirname(template_path), @pattern % File.basename(template_path))
       else
-        File.join(root, "_#{template_path}.liquid")
+        File.join(root, @pattern % template_path)
       end
 
-      raise FileSystemError, "Illegal template path '#{File.expand_path(full_path)}'" unless File.expand_path(full_path) =~ /^#{File.expand_path(root)}/
+      raise FileSystemError, "Illegal template path '#{File.expand_path(full_path)}'" unless File.expand_path(full_path) =~ /\A#{File.expand_path(root)}/
 
       full_path
     end

lib/liquid/htmltags.rb

@@ -1,74 +0,0 @@
-module Liquid
-  class TableRow < Block
-    Syntax = /(\w+)\s+in\s+(#{VariableSignature}+)/
-
-    def initialize(tag_name, markup, tokens)
-      if markup =~ Syntax
-        @variable_name = $1
-        @collection_name = $2
-        @attributes = {}
-        markup.scan(TagAttributes) do |key, value|
-          @attributes[key] = value
-        end
-      else
-        raise SyntaxError.new("Syntax Error in 'table_row loop' - Valid syntax: table_row [item] in [collection] cols=3")
-      end
-
-      super
-    end
-
-    def render(context)
-      collection = context[@collection_name] or return ''
-
-      if @attributes['limit'] or @attributes['offset']
-        limit = context[@attributes['limit']] || -1
-        offset = context[@attributes['offset']] || 0
-        collection = collection[offset.to_i..(limit.to_i + offset.to_i - 1)]
-      end
-
-      length = collection.length
-
-      cols = context[@attributes['cols']].to_i
-
-      row = 1
-      col = 0
-
-      result = ["<tr class=\"row1\">\n"]
-      context.stack do
-
-        collection.each_with_index do |item, index|
-          context[@variable_name] = item
-          context['tablerowloop'] = {
-            'length'  => length,
-            'index'   => index + 1,
-            'index0'  => index,
-            'col'     => col + 1,
-            'col0'    => col,
-            'index0'  => index,
-            'rindex'  => length - index,
-            'rindex0' => length - index -1,
-            'first'   => (index == 0),
-            'last'    => (index == length - 1),
-            'col_first' => (col == 0),
-            'col_last'  => (col == cols - 1)
-          }
-
-
-          col += 1
-
-          result << ["<td class=\"col#{col}\">"] + render_all(@nodelist, context) + ['</td>']
-
-          if col == cols and not (index == length - 1)
-            col  = 0
-            row += 1
-            result << ["</tr>\n<tr class=\"row#{row}\">"]
-          end
-
-        end
-      end
-      result + ["</tr>\n"]
-    end
-  end
-
-  Template.register_tag('tablerow', TableRow)
-end

lib/liquid/i18n.rb

@@ -0,0 +1,39 @@
+require 'yaml'
+
+module Liquid
+  class I18n
+    DEFAULT_LOCALE = File.join(File.expand_path(File.dirname(__FILE__)), "locales", "en.yml")
+
+    class TranslationError < StandardError
+    end
+
+    attr_reader :path
+
+    def initialize(path = DEFAULT_LOCALE)
+      @path = path
+    end
+
+    def translate(name, vars = {})
+      interpolate(deep_fetch_translation(name), vars)
+    end
+    alias_method :t, :translate
+
+    def locale
+      @locale ||= YAML.load_file(@path)
+    end
+
+    private
+    def interpolate(name, vars)
+      name.gsub(/%\{(\w+)\}/) {
+        # raise TranslationError, "Undefined key #{$1} for interpolation in translation #{name}"  unless vars[$1.to_sym]
+        "#{vars[$1.to_sym]}"
+      }
+    end
+
+    def deep_fetch_translation(name)
+      name.split('.'.freeze).reduce(locale) do |level, cur|
+        level[cur] or raise TranslationError, "Translation for #{name} does not exist in locale #{path}"
+      end
+    end
+  end
+end

lib/liquid/interrupts.rb

@@ -0,0 +1,17 @@
+module Liquid
+
+  # An interrupt is any command that breaks processing of a block (ex: a for loop).
+  class Interrupt
+    attr_reader :message
+
+    def initialize(message=nil)
+      @message = message || "interrupt".freeze
+    end
+  end
+
+  # Interrupt that is thrown whenever a {% break %} is called.
+  class BreakInterrupt < Interrupt; end
+
+  # Interrupt that is thrown whenever a {% continue %} is called.
+  class ContinueInterrupt < Interrupt; end
+end

lib/liquid/lexer.rb

@@ -0,0 +1,51 @@
+require "strscan"
+module Liquid
+  class Lexer
+    SPECIALS = {
+      '|'.freeze => :pipe,
+      '.'.freeze => :dot,
+      ':'.freeze => :colon,
+      ','.freeze => :comma,
+      '['.freeze => :open_square,
+      ']'.freeze => :close_square,
+      '('.freeze => :open_round,
+      ')'.freeze => :close_round
+    }
+    IDENTIFIER = /[\w\-?!]+/
+    SINGLE_STRING_LITERAL = /'[^\']*'/
+    DOUBLE_STRING_LITERAL = /"[^\"]*"/
+    NUMBER_LITERAL = /-?\d+(\.\d+)?/
+    DOTDOT = /\.\./
+    COMPARISON_OPERATOR = /==|!=|<>|<=?|>=?|contains/
+
+    def initialize(input)
+      @ss = StringScanner.new(input.rstrip)
+    end
+
+    def tokenize
+      @output = []
+
+      while !@ss.eos?
+        @ss.skip(/\s*/)
+        tok = case
+        when t = @ss.scan(COMPARISON_OPERATOR) then [:comparison, t]
+        when t = @ss.scan(SINGLE_STRING_LITERAL) then [:string, t]
+        when t = @ss.scan(DOUBLE_STRING_LITERAL) then [:string, t]
+        when t = @ss.scan(NUMBER_LITERAL) then [:number, t]
+        when t = @ss.scan(IDENTIFIER) then [:id, t]
+        when t = @ss.scan(DOTDOT) then [:dotdot, t]
+        else
+          c = @ss.getch
+          if s = SPECIALS[c]
+            [s,c]
+          else
+            raise SyntaxError, "Unexpected character #{c}"
+          end
+        end
+        @output << tok
+      end
+
+      @output << [:end_of_string]
+    end
+  end
+end

lib/liquid/locales/en.yml

@@ -0,0 +1,22 @@
+---
+  errors:
+    syntax:
+      assign: "Syntax Error in 'assign' - Valid syntax: assign [var] = [source]"
+      capture: "Syntax Error in 'capture' - Valid syntax: capture [var]"
+      case: "Syntax Error in 'case' - Valid syntax: case [condition]"
+      case_invalid_when: "Syntax Error in tag 'case' - Valid when condition: {% when [condition] [or condition2...] %}"
+      case_invalid_else: "Syntax Error in tag 'case' - Valid else condition: {% else %} (no parameters) "
+      cycle: "Syntax Error in 'cycle' - Valid syntax: cycle [name :] var [, var2, var3 ...]"
+      for: "Syntax Error in 'for loop' - Valid syntax: for [item] in [collection]"
+      for_invalid_in: "For loops require an 'in' clause"
+      for_invalid_attribute: "Invalid attribute in for loop. Valid attributes are limit and offset"
+      if: "Syntax Error in tag 'if' - Valid syntax: if [expression]"
+      include: "Error in tag 'include' - Valid syntax: include '[template]' (with|for) [object|collection]"
+      unknown_tag: "Unknown tag '%{tag}'"
+      invalid_delimiter: "'end' is not a valid delimiter for %{block_name} tags. use %{block_delimiter}"
+      unexpected_else: "%{block_name} tag does not expect else tag"
+      tag_termination: "Tag '%{token}' was not properly terminated with regexp: %{tag_end}"
+      variable_termination: "Variable '%{token}' was not properly terminated with regexp: %{tag_end}"
+      tag_never_closed: "'%{block_name}' tag was never closed"
+      meta_syntax_error: "Liquid syntax error: #{e.message}"
+      table_row: "Syntax Error in 'table_row loop' - Valid syntax: table_row [item] in [collection] cols=3"

lib/liquid/module_ex.rb

@@ -2,7 +2,7 @@
 # This library is free software. It may be used, redistributed and/or modified
 # under the same terms as Ruby itself
 #
-# This extension is usesd in order to expose the object of the implementing class
+# This extension is used in order to expose the object of the implementing class
 # to liquid as it were a Drop. It also limits the liquid-callable methods of the instance
 # to the allowed method passed with the liquid_methods call
 # Example:

lib/liquid/parser.rb

@@ -0,0 +1,90 @@
+module Liquid
+  class Parser
+    def initialize(input)
+      l = Lexer.new(input)
+      @tokens = l.tokenize
+      @p = 0 # pointer to current location
+    end
+
+    def jump(point)
+      @p = point
+    end
+
+    def consume(type = nil)
+      token = @tokens[@p]
+      if type && token[0] != type
+        raise SyntaxError, "Expected #{type} but found #{@tokens[@p].first}"
+      end
+      @p += 1
+      token[1]
+    end
+
+    # Only consumes the token if it matches the type
+    # Returns the token's contents if it was consumed
+    # or false otherwise.
+    def consume?(type)
+      token = @tokens[@p]
+      return false unless token && token[0] == type
+      @p += 1
+      token[1]
+    end
+
+    # Like consume? Except for an :id token of a certain name
+    def id?(str)
+      token = @tokens[@p]
+      return false unless token && token[0] == :id
+      return false unless token[1] == str
+      @p += 1
+      token[1]
+    end
+
+    def look(type, ahead = 0)
+      tok = @tokens[@p + ahead]
+      return false unless tok
+      tok[0] == type
+    end
+
+    def expression
+      token = @tokens[@p]
+      if token[0] == :id
+        variable_signature
+      elsif [:string, :number].include? token[0]
+        consume
+      elsif token.first == :open_round
+        consume
+        first = expression
+        consume(:dotdot)
+        last = expression
+        consume(:close_round)
+        "(#{first}..#{last})"
+      else
+        raise SyntaxError, "#{token} is not a valid expression"
+      end
+    end
+
+    def argument
+      str = ""
+      # might be a keyword argument (identifier: expression)
+      if look(:id) && look(:colon, 1)
+        str << consume << consume << ' '.freeze
+      end
+
+      str << expression
+      str
+    end
+
+    def variable_signature
+      str = consume(:id)
+      if look(:open_square)
+        str << consume
+        str << expression
+        str << consume(:close_square)
+      end
+      if look(:dot)
+        str << consume
+        str << variable_signature
+      end
+      str
+    end
+  end
+end

lib/liquid/standardfilters.rb

@@ -1,21 +1,29 @@
 require 'cgi'
+require 'bigdecimal'
 
 module Liquid
 
   module StandardFilters
+    HTML_ESCAPE = {
+      '&'.freeze => '&'.freeze,
+      '>'.freeze => '>'.freeze,
+      '<'.freeze => '&lt;'.freeze,
+      '"'.freeze => '&quot;'.freeze,
+      "'".freeze => '&#39;'.freeze
+    }
+    HTML_ESCAPE_ONCE_REGEXP = /["><']|&(?!([a-zA-Z]+|(#\d+));)/
 
     # Return the size of an array or of an string
     def size(input)
-
       input.respond_to?(:size) ? input.size : 0
     end
 
-    # convert a input string to DOWNCASE
+    # convert an input string to DOWNCASE
     def downcase(input)
       input.to_s.downcase
     end
 
-    # convert a input string to UPCASE
+    # convert an input string to UPCASE
     def upcase(input)
       input.to_s.upcase
     end
@@ -30,83 +38,113 @@ module Liquid
     end
 
     def escape_once(input)
-      ActionView::Helpers::TagHelper.escape_once(input) rescue input
+      input.to_s.gsub(HTML_ESCAPE_ONCE_REGEXP, HTML_ESCAPE)
     end
 
     alias_method :h, :escape
 
     # Truncate a string down to x characters
-    def truncate(input, length = 50, truncate_string = "...")
+    def truncate(input, length = 50, truncate_string = "...".freeze)
       if input.nil? then return end
       l = length.to_i - truncate_string.length
       l = 0 if l < 0
       input.length > length.to_i ? input[0...l] + truncate_string : input
     end
 
-    def truncatewords(input, words = 15, truncate_string = "...")
+    def truncatewords(input, words = 15, truncate_string = "...".freeze)
       if input.nil? then return end
       wordlist = input.to_s.split
       l = words.to_i - 1
       l = 0 if l < 0
-      wordlist.length > l ? wordlist[0..l].join(" ") + truncate_string : input
+      wordlist.length > l ? wordlist[0..l].join(" ".freeze) + truncate_string : input
+    end
+
+    # Split input string into an array of substrings separated by given pattern.
+    #
+    # Example:
+    #   <div class="summary">{{ post | split '//' | first }}</div>
+    #
+    def split(input, pattern)
+      input.split(pattern)
+    end
+
+    def strip(input)
+      input.to_s.strip
+    end
+
+    def lstrip(input)
+      input.to_s.lstrip
+    end
+
+    def rstrip(input)
+      input.to_s.rstrip
     end
 
     def strip_html(input)
-      input.to_s.gsub(/<script.*?<\/script>/, '').gsub(/<.*?>/, '')
+      empty = ''.freeze
+      input.to_s.gsub(/<script.*?<\/script>/m, empty).gsub(/<!--.*?-->/m, empty).gsub(/<style.*?<\/style>/m, empty).gsub(/<.*?>/m, empty)
     end
 
     # Remove all newlines from the string
     def strip_newlines(input)
-      input.to_s.gsub(/\n/, '')
+      input.to_s.gsub(/\r?\n/, ''.freeze)
     end
 
-
     # Join elements of the array with certain character between them
-    def join(input, glue = ' ')
+    def join(input, glue = ' '.freeze)
       [input].flatten.join(glue)
     end
 
     # Sort elements of the array
     # provide optional property with which to sort an array of hashes or drops
     def sort(input, property = nil)
-      ary = [input].flatten
+      ary = flatten_if_necessary(input)
       if property.nil?
         ary.sort
-      elsif ary.first.respond_to?('[]') and !ary.first[property].nil?
+      elsif ary.first.respond_to?('[]'.freeze) and !ary.first[property].nil?
         ary.sort {|a,b| a[property] <=> b[property] }
       elsif ary.first.respond_to?(property)
         ary.sort {|a,b| a.send(property) <=> b.send(property) }
       end
     end
 
+    # Reverse the elements of an array
+    def reverse(input)
+      ary = [input].flatten
+      ary.reverse
+    end
+
     # map/collect on a given property
     def map(input, property)
-      ary = [input].flatten
+      flatten_if_necessary(input).map do |e|
-      if ary.first.respond_to?('[]') and !ary.first[property].nil?
+        e = e.call if e.is_a?(Proc)
-        ary.map {|e| e[property] }
+
-      elsif ary.first.respond_to?(property)
+        if property == "to_liquid".freeze
-        ary.map {|e| e.send(property) }
+          e
+        elsif e.respond_to?(:[])
+          e[property]
+        end
       end
     end
 
     # Replace occurrences of a string with another
-    def replace(input, string, replacement = '')
+    def replace(input, string, replacement = ''.freeze)
-      input.to_s.gsub(string, replacement)
+      input.to_s.gsub(string, replacement.to_s)
     end
 
     # Replace the first occurrences of a string with another
-    def replace_first(input, string, replacement = '')
+    def replace_first(input, string, replacement = ''.freeze)
-      input.to_s.sub(string, replacement)
+      input.to_s.sub(string, replacement.to_s)
     end
 
     # remove a substring
     def remove(input, string)
-      input.to_s.gsub(string, '')
+      input.to_s.gsub(string, ''.freeze)
     end
 
     # remove the first occurrences of a substring
     def remove_first(input, string)
-      input.to_s.sub(string, '')
+      input.to_s.sub(string, ''.freeze)
     end
 
     # add one string to another
@@ -121,7 +159,7 @@ module Liquid
 
     # Add <br /> tags in front of all newlines in input string
     def newline_to_br(input)
-      input.to_s.gsub(/\n/, "<br />\n")
+      input.to_s.gsub(/\n/, "<br />\n".freeze)
     end
 
     # Reformat a date
@@ -158,14 +196,27 @@ module Liquid
         return input.to_s
       end
 
-      date = input.is_a?(String) ? Time.parse(input) : input
+      if ((input.is_a?(String) && !/\A\d+\z/.match(input.to_s).nil?) || input.is_a?(Integer)) && input.to_i > 0
+        input = Time.at(input.to_i)
+      end
+
+      date = if input.is_a?(String)
+        case input.downcase
+        when 'now'.freeze, 'today'.freeze
+          Time.now
+        else
+          Time.parse(input)
+        end
+      else
+        input
+      end
 
       if date.respond_to?(:strftime)
         date.strftime(format.to_s)
       else
         input
       end
-    rescue => e
+    rescue
       input
     end
 
@@ -189,37 +240,63 @@ module Liquid
 
     # addition
     def plus(input, operand)
-      to_number(input) + to_number(operand)
+      apply_operation(input, operand, :+)
     end
 
     # subtraction
     def minus(input, operand)
-      to_number(input) - to_number(operand)
+      apply_operation(input, operand, :-)
     end
 
     # multiplication
     def times(input, operand)
-      to_number(input) * to_number(operand)
+      apply_operation(input, operand, :*)
     end
 
     # division
     def divided_by(input, operand)
-      to_number(input) / to_number(operand)
+      apply_operation(input, operand, :/)
+    end
+
+    def modulo(input, operand)
+      apply_operation(input, operand, :%)
+    end
+
+    def default(input, default_value = "".freeze)
+      is_blank = input.respond_to?(:empty?) ? input.empty? : !input
+      is_blank ? default_value : input
     end
 
     private
 
+    def flatten_if_necessary(input)
+      ary = if input.is_a?(Array)
+        input.flatten
+      elsif input.is_a?(Enumerable) && !input.is_a?(Hash)
+        input
+      else
+        [input].flatten
+      end
+      ary.map{ |e| e.respond_to?(:to_liquid) ? e.to_liquid : e }
+    end
+
     def to_number(obj)
       case obj
+      when Float
+        BigDecimal.new(obj.to_s)
       when Numeric
         obj
       when String
-          (obj.strip =~ /^\d+\.\d+$/) ? obj.to_f : obj.to_i
+        (obj.strip =~ /\A\d+\.\d+\z/) ? BigDecimal.new(obj) : obj.to_i
       else
         0
       end
     end
 
+    def apply_operation(input, operand, operation)
+      result = to_number(input).send(operation, to_number(operand))
+      result.is_a?(BigDecimal) ? result.to_f : result
+    end
   end
 
   Template.register_filter(StandardFilters)

lib/liquid/strainer.rb

@@ -2,24 +2,20 @@ require 'set'
 
 module Liquid
 
-  parent_object = if defined? BlankObject
-    BlankObject
-  else
-    Object
-  end
-
   # Strainer is the parent class for the filters system.
-  # New filters are mixed into the strainer class which is then instanciated for each liquid template render run.
+  # New filters are mixed into the strainer class which is then instantiated for each liquid template render run.
   #
-  # One of the strainer's responsibilities is to keep malicious method calls out
+  # The Strainer only allows method calls defined in filters given to it via Strainer.global_filter,
-  class Strainer < parent_object #:nodoc:
+  # Context#add_filters or Template.register_filter
-    INTERNAL_METHOD = /^__/
+  class Strainer #:nodoc:
-    @@required_methods = Set.new([:__id__, :__send__, :respond_to?, :extend, :methods, :class, :object_id])
+    @@filters = []
-
+    @@known_filters = Set.new
-    # Ruby 1.9.2 introduces Object#respond_to_missing?, which is invoked by Object#respond_to?
+    @@known_methods = Set.new
-    @@required_methods << :respond_to_missing? if Object.respond_to? :respond_to_missing?
+    @@strainer_class_cache = Hash.new do |hash, filters|
-
+      hash[filters] = Class.new(Strainer) do
-    @@filters = {}
+        filters.each { |f| include f }
+      end
+    end
 
     def initialize(context)
       @context = context
@@ -27,28 +23,41 @@ module Liquid
 
     def self.global_filter(filter)
       raise ArgumentError, "Passed filter is not a module" unless filter.is_a?(Module)
-      @@filters[filter.name] = filter
+      add_known_filter(filter)
+      @@filters << filter unless @@filters.include?(filter)
     end
 
-    def self.create(context)
+    def self.add_known_filter(filter)
-      strainer = Strainer.new(context)
+      unless @@known_filters.include?(filter)
-      @@filters.each { |k,m| strainer.extend(m) }
+        @@method_blacklist ||= Set.new(Strainer.instance_methods.map(&:to_s))
-      strainer
+        new_methods = filter.instance_methods.map(&:to_s)
+        new_methods.reject!{ |m| @@method_blacklist.include?(m) }
+        @@known_methods.merge(new_methods)
+        @@known_filters.add(filter)
+      end
     end
 
-    def respond_to?(method, include_private = false)
+    def self.strainer_class_cache
-      method_name = method.to_s
+      @@strainer_class_cache
-      return false if method_name =~ INTERNAL_METHOD
-      return false if @@required_methods.include?(method_name)
-      super
     end
 
-    # remove all standard methods from the bucket so circumvent security
+    def self.create(context, filters = [])
-    # problems
+      filters = @@filters + filters
-    instance_methods.each do |m|
+      strainer_class_cache[filters].new(context)
-      unless @@required_methods.include?(m.to_sym)
-        undef_method m
     end
+
+    def invoke(method, *args)
+      if invokable?(method)
+        send(method, *args)
+      else
+        args.first
+      end
+    rescue ::ArgumentError => e
+      raise Liquid::ArgumentError.new(e.message)
+    end
+
+    def invokable?(method)
+      @@known_methods.include?(method.to_s) && respond_to?(method)
     end
   end
 end

lib/liquid/tag.rb

@@ -1,13 +1,22 @@
 module Liquid
-
   class Tag
+    attr_accessor :options
+    attr_reader :nodelist, :warnings
 
-    attr_accessor :nodelist
+    class << self
+      def parse(tag_name, markup, tokens, options)
+        tag = new(tag_name, markup, options)
+        tag.parse(tokens)
+        tag
+      end
 
-    def initialize(tag_name, markup, tokens)
+      private :new
+    end
+
+    def initialize(tag_name, markup, options)
       @tag_name   = tag_name
       @markup     = markup
-      parse(tokens)
+      @options    = options
     end
 
     def parse(tokens)
@@ -18,9 +27,34 @@ module Liquid
     end
 
     def render(context)
-      ''
+      ''.freeze
     end
 
-  end # Tag
+    def blank?
+      @blank || false
+    end
 
-end # Tag
+    def parse_with_selected_parser(markup)
+      case @options[:error_mode] || Template.error_mode
+      when :strict then strict_parse_with_error_context(markup)
+      when :lax    then lax_parse(markup)
+      when :warn
+        begin
+          return strict_parse_with_error_context(markup)
+        rescue SyntaxError => e
+          @warnings ||= []
+          @warnings << e
+          return lax_parse(markup)
+        end
+      end
+    end
+
+    private
+    def strict_parse_with_error_context(markup)
+      strict_parse(markup)
+    rescue SyntaxError => e
+      e.message << " in \"#{markup.strip}\"" 
+      raise e
+    end
+  end # Tag
+end # Liquid

lib/liquid/tags/assign.rb

@@ -9,25 +9,29 @@ module Liquid
   #  {{ foo }}
   #
   class Assign < Tag
-    Syntax = /(#{VariableSignature}+)\s*=\s*(#{QuotedFragment}+)/   
+    Syntax = /(#{VariableSignature}+)\s*=\s*(.*)\s*/om
 
-    def initialize(tag_name, markup, tokens)          
+    def initialize(tag_name, markup, options)
+      super
       if markup =~ Syntax
         @to = $1
-        @from = $2
+        @from = Variable.new($2)
       else
-        raise SyntaxError.new("Syntax Error in 'assign' - Valid syntax: assign [var] = [source]")
+        raise SyntaxError.new options[:locale].t("errors.syntax.assign".freeze)
       end
-      
-      super      
     end
 
     def render(context)
-       context.scopes.last[@to] = context[@from]
+      val = @from.render(context)
-       ''
+      context.scopes.last[@to] = val
+      context.increment_used_resources(:assign_score_current, val)
+      ''.freeze
     end
 
+    def blank?
+      true
+    end
   end
 
-  Template.register_tag('assign', Assign)  
+  Template.register_tag('assign'.freeze, Assign)
 end

lib/liquid/tags/break.rb

@@ -0,0 +1,21 @@
+module Liquid
+
+  # Break tag to be used to break out of a for loop.
+  #
+  # == Basic Usage:
+  #    {% for item in collection %}
+  #      {% if item.condition %}
+  #        {% break %}
+  #      {% endif %}
+  #    {% endfor %}
+  #
+  class Break < Tag
+
+    def interrupt
+      BreakInterrupt.new
+    end
+
+  end
+
+  Template.register_tag('break'.freeze, Break)
+end

lib/liquid/tags/capture.rb

@@ -14,22 +14,26 @@ module Liquid
   class Capture < Block
     Syntax = /(\w+)/
 
-    def initialize(tag_name, markup, tokens)
+    def initialize(tag_name, markup, options)
+      super
       if markup =~ Syntax
         @to = $1
       else
-        raise SyntaxError.new("Syntax Error in 'capture' - Valid syntax: capture [var]")
+        raise SyntaxError.new(options[:locale].t("errors.syntax.capture"))
       end
-
-      super
     end
 
     def render(context)
       output = super
-      context.scopes.last[@to] = output.join
+      context.scopes.last[@to] = output
-      ''
+      context.increment_used_resources(:assign_score_current, output)
+      ''.freeze
+    end
+
+    def blank?
+      true
     end
   end
 
-  Template.register_tag('capture', Capture)
+  Template.register_tag('capture'.freeze, Capture)
 end

lib/liquid/tags/case.rb

@@ -1,26 +1,29 @@
 module Liquid
   class Case < Block
-    Syntax     = /(#{QuotedFragment})/
+    Syntax     = /(#{QuotedFragment})/o
-    WhenSyntax = /(#{QuotedFragment})(?:(?:\s+or\s+|\s*\,\s*)(#{QuotedFragment}.*))?/
+    WhenSyntax = /(#{QuotedFragment})(?:(?:\s+or\s+|\s*\,\s*)(#{QuotedFragment}.*))?/om
 
-    def initialize(tag_name, markup, tokens)      
+    def initialize(tag_name, markup, options)
+      super
       @blocks = []
 
       if markup =~ Syntax
         @left = $1
       else
-        raise SyntaxError.new("Syntax Error in tag 'case' - Valid syntax: case [condition]")
+        raise SyntaxError.new(options[:locale].t("errors.syntax.case".freeze))
+      end
     end
 
-      super
+    def nodelist
+      @blocks.map(&:attachment).flatten
     end
 
     def unknown_tag(tag, markup, tokens)
       @nodelist = []
       case tag
-      when 'when'
+      when 'when'.freeze
         record_when_condition(markup)
-      when 'else'
+      when 'else'.freeze
         record_else_condition(markup)
       else
         super
@@ -31,22 +34,18 @@ module Liquid
       context.stack do
         execute_else_block = true
 
-        @blocks.inject([]) do |output, block|
+        output = ''
-      
+        @blocks.each do |block|
           if block.else?
-            
             return render_all(block.attachment, context) if execute_else_block
-            
           elsif block.evaluate(context)
-            
             execute_else_block = false
-            output += render_all(block.attachment, context)                    
+            output << render_all(block.attachment, context)
+          end
         end
-      
         output
       end
     end
-    end
 
     private
 
@@ -54,30 +53,27 @@ module Liquid
       while markup
         # Create a new nodelist and assign it to the new block
         if not markup =~ WhenSyntax
-      	  raise SyntaxError.new("Syntax Error in tag 'case' - Valid when condition: {% when [condition] [or condition2...] %} ")
+          raise SyntaxError.new(options[:locale].t("errors.syntax.case_invalid_when".freeze))
         end
 
         markup = $2
 
-      	block = Condition.new(@left, '==', $1)        
+        block = Condition.new(@left, '=='.freeze, $1)
         block.attach(@nodelist)
         @blocks.push(block)
       end
     end
 
     def record_else_condition(markup)
-
       if not markup.strip.empty?
-        raise SyntaxError.new("Syntax Error in tag 'case' - Valid else condition: {% else %} (no parameters) ")
+        raise SyntaxError.new(options[:locale].t("errors.syntax.case_invalid_else".freeze))
       end
 
       block = ElseCondition.new
       block.attach(@nodelist)
       @blocks << block
     end
-    
-        
   end
 
-  Template.register_tag('case', Case)
+  Template.register_tag('case'.freeze, Case)
 end

lib/liquid/tags/comment.rb

@@ -1,9 +1,16 @@
 module Liquid
   class Comment < Block
     def render(context)
-      ''
+      ''.freeze
+    end
+
+    def unknown_tag(tag, markup, tokens)
+    end
+
+    def blank?
+      true
     end
   end
 
-  Template.register_tag('comment', Comment)
+  Template.register_tag('comment'.freeze, Comment)
 end

lib/liquid/tags/continue.rb

@@ -0,0 +1,18 @@
+module Liquid
+  # Continue tag to be used to break out of a for loop.
+  #
+  # == Basic Usage:
+  #    {% for item in collection %}
+  #      {% if item.condition %}
+  #        {% continue %}
+  #      {% endif %}
+  #    {% endfor %}
+  #
+  class Continue < Tag
+    def interrupt
+      ContinueInterrupt.new
+    end
+  end
+
+  Template.register_tag('continue'.freeze, Continue)
+end

lib/liquid/tags/cycle.rb

@@ -1,5 +1,4 @@
 module Liquid
-  
   # Cycle is usually used within a loop to alternate between values, like colors or DOM classes.
   #
   #   {% for item in items %}
@@ -13,10 +12,11 @@ module Liquid
   #    <div class="green"> Item five</div>
   #
   class Cycle < Tag
-    SimpleSyntax = /^#{QuotedFragment}+/        
+    SimpleSyntax = /\A#{QuotedFragment}+/o
-    NamedSyntax  = /^(#{QuotedFragment})\s*\:\s*(.*)/
+    NamedSyntax  = /\A(#{QuotedFragment})\s*\:\s*(.*)/om
 
-    def initialize(tag_name, markup, tokens)      
+    def initialize(tag_name, markup, options)
+      super
       case markup
       when NamedSyntax
         @variables = variables_from_string($2)
@@ -25,9 +25,8 @@ module Liquid
         @variables = variables_from_string(markup)
         @name = "'#{@variables.to_s}'"
       else
-        raise SyntaxError.new("Syntax Error in 'cycle' - Valid syntax: cycle [name :] var [, var2, var3 ...]")
+        raise SyntaxError.new(options[:locale].t("errors.syntax.cycle".freeze))
       end
-      super    
     end
 
     def render(context)
@@ -44,15 +43,17 @@ module Liquid
       end
     end
 
-    private
+    def blank?
+      false
+    end
 
+    private
     def variables_from_string(markup)
       markup.split(',').collect do |var|
-    	  var =~ /\s*(#{QuotedFragment})\s*/
+        var =~ /\s*(#{QuotedFragment})\s*/o
         $1 ? $1 : nil
       end.compact
     end
-  
   end
 
   Template.register_tag('cycle', Cycle)

lib/liquid/tags/decrement.rb

@@ -0,0 +1,38 @@
+module Liquid
+
+  # decrement is used in a place where one needs to insert a counter
+  #     into a template, and needs the counter to survive across
+  #     multiple instantiations of the template.
+  #     NOTE: decrement is a pre-decrement, --i,
+  #           while increment is post:      i++.
+  #
+  #     (To achieve the survival, the application must keep the context)
+  #
+  #     if the variable does not exist, it is created with value 0.
+
+  #   Hello: {% decrement variable %}
+  #
+  # gives you:
+  #
+  #    Hello: -1
+  #    Hello: -2
+  #    Hello: -3
+  #
+  class Decrement < Tag
+    def initialize(tag_name, markup, options)
+      super
+      @variable = markup.strip
+    end
+
+    def render(context)
+      value = context.environments.first[@variable] ||= 0
+      value = value - 1
+      context.environments.first[@variable] = value
+      value.to_s
+    end
+
+    private
+  end
+
+  Template.register_tag('decrement'.freeze, Decrement)
+end

lib/liquid/tags/for.rb

@@ -13,6 +13,8 @@ module Liquid
   #      <div {% if forloop.first %}class="first"{% endif %}>
   #        Item {{ forloop.index }}: {{ item.name }}
   #      </div>
+  #    {% else %}
+  #      There is nothing in the collection.
   #    {% endfor %}
   #
   # You can also define a limit and offset much like SQL.  Remember
@@ -42,9 +44,88 @@ module Liquid
   # forloop.last:: Returns true if the item is the last item.
   #
   class For < Block
-    Syntax = /(\w+)\s+in\s+(#{QuotedFragment}+)\s*(reversed)?/   
+    Syntax = /\A(#{VariableSegment}+)\s+in\s+(#{QuotedFragment}+)\s*(reversed)?/o
 
-    def initialize(tag_name, markup, tokens)
+    def initialize(tag_name, markup, options)
+      super
+      parse_with_selected_parser(markup)
+      @nodelist = @for_block = []
+    end
+
+    def nodelist
+      if @else_block
+        @for_block + @else_block
+      else
+        @for_block
+      end
+    end
+
+    def unknown_tag(tag, markup, tokens)
+      return super unless tag == 'else'.freeze
+      @nodelist = @else_block = []
+    end
+
+    def render(context)
+      context.registers[:for] ||= Hash.new(0)
+
+      collection = context[@collection_name]
+      collection = collection.to_a if collection.is_a?(Range)
+
+      # Maintains Ruby 1.8.7 String#each behaviour on 1.9
+      return render_else(context) unless iterable?(collection)
+
+      from = if @attributes['offset'.freeze] == 'continue'.freeze
+        context.registers[:for][@name].to_i
+      else
+        context[@attributes['offset'.freeze]].to_i
+      end
+
+      limit = context[@attributes['limit'.freeze]]
+      to    = limit ? limit.to_i + from : nil
+
+      segment = Utils.slice_collection(collection, from, to)
+
+      return render_else(context) if segment.empty?
+
+      segment.reverse! if @reversed
+
+      result = ''
+
+      length = segment.length
+
+      # Store our progress through the collection for the continue flag
+      context.registers[:for][@name] = from + segment.length
+
+      context.stack do
+        segment.each_with_index do |item, index|
+          context[@variable_name] = item
+          context['forloop'.freeze] = {
+            'name'.freeze    => @name,
+            'length'.freeze  => length,
+            'index'.freeze   => index + 1,
+            'index0'.freeze  => index,
+            'rindex'.freeze  => length - index,
+            'rindex0'.freeze => length - index - 1,
+            'first'.freeze   => (index == 0),
+            'last'.freeze    => (index == length - 1)
+          }
+
+          result << render_all(@for_block, context)
+
+          # Handle any interrupts if they exist.
+          if context.has_interrupt?
+            interrupt = context.pop_interrupt
+            break if interrupt.is_a? BreakInterrupt
+            next if interrupt.is_a? ContinueInterrupt
+          end
+        end
+      end
+      result
+    end
+
+    protected
+
+    def lax_parse(markup)
       if markup =~ Syntax
         @variable_name = $1
         @collection_name = $2
@@ -55,82 +136,40 @@ module Liquid
           @attributes[key] = value
         end
       else
-        raise SyntaxError.new("Syntax Error in 'for loop' - Valid syntax: for [item] in [collection]")
+        raise SyntaxError.new(options[:locale].t("errors.syntax.for".freeze))
-      end
-
-      super
-    end
-  
-    def render(context)        
-      context.registers[:for] ||= Hash.new(0)
-    
-      collection = context[@collection_name]
-      collection = collection.to_a if collection.is_a?(Range)
-    
-      return '' unless collection.respond_to?(:each) 
-                                                 
-      from = if @attributes['offset'] == 'continue'
-        context.registers[:for][@name].to_i
-      else
-        context[@attributes['offset']].to_i
-      end
-        
-      limit = context[@attributes['limit']]
-      to    = limit ? limit.to_i + from : nil  
-          
-                       
-      segment = slice_collection_using_each(collection, from, to)      
-      
-      return '' if segment.empty?
-      
-      segment.reverse! if @reversed
-
-      result = []
-        
-      length = segment.length            
-            
-      # Store our progress through the collection for the continue flag
-      context.registers[:for][@name] = from + segment.length
-              
-      context.stack do 
-        segment.each_with_index do |item, index|     
-          context[@variable_name] = item
-          context['forloop'] = {
-            'name'    => @name,
-            'length'  => length,
-            'index'   => index + 1, 
-            'index0'  => index, 
-            'rindex'  => length - index,
-            'rindex0' => length - index -1,
-            'first'   => (index == 0),
-            'last'    => (index == length - 1) }
-
-          result << render_all(@nodelist, context)
-        end
-      end
-      result     
-    end          
-        
-    def slice_collection_using_each(collection, from, to)       
-      segments = []      
-      index = 0      
-      yielded = 0
-      collection.each do |item|         
-                
-        if to && to <= index
-          break
-        end
-        
-        if from <= index                               
-          segments << item
-        end                    
-                
-        index += 1
-      end    
-
-      segments
       end
     end
 
-  Template.register_tag('for', For)
+    def strict_parse(markup)
+      p = Parser.new(markup)
+      @variable_name = p.consume(:id)
+      raise SyntaxError.new(options[:locale].t("errors.syntax.for_invalid_in".freeze))  unless p.id?('in'.freeze)
+      @collection_name = p.expression
+      @name = "#{@variable_name}-#{@collection_name}"
+      @reversed = p.id?('reversed'.freeze)
+
+      @attributes = {}
+      while p.look(:id) && p.look(:colon, 1)
+        unless attribute = p.id?('limit'.freeze) || p.id?('offset'.freeze)
+          raise SyntaxError.new(options[:locale].t("errors.syntax.for_invalid_attribute".freeze))
+        end
+        p.consume
+        val = p.expression
+        @attributes[attribute] = val
+      end
+      p.consume(:end_of_string)
+    end
+
+    private
+
+    def render_else(context)
+      return @else_block ? [render_all(@else_block, context)] : ''.freeze
+    end
+
+    def iterable?(collection)
+      collection.respond_to?(:each) || Utils.non_blank_string?(collection)
+    end
+  end
+
+  Template.register_tag('for'.freeze, For)
 end

lib/liquid/tags/if.rb

@@ -1,5 +1,4 @@
 module Liquid
-
   # If is the conditional block
   #
   #   {% if user.admin %}
@@ -10,22 +9,23 @@ module Liquid
   #
   #    There are {% if count < 5 %} less {% else %} more {% endif %} items than you need.
   #
-  #
   class If < Block
-    SyntaxHelp = "Syntax Error in tag 'if' - Valid syntax: if [expression]"
+    Syntax = /(#{QuotedFragment})\s*([=!<>a-z_]+)?\s*(#{QuotedFragment})?/o
-    Syntax = /(#{QuotedFragment})\s*([=!<>a-z_]+)?\s*(#{QuotedFragment})?/
+    ExpressionsAndOperators = /(?:\b(?:\s?and\s?|\s?or\s?)\b|(?:\s*(?!\b(?:\s?and\s?|\s?or\s?)\b)(?:#{QuotedFragment}|\S+)\s*)+)/o
-    ExpressionsAndOperators = /(?:\b(?:\s?and\s?|\s?or\s?)\b|(?:\s*(?!\b(?:\s?and\s?|\s?or\s?)\b)(?:#{QuotedFragment}|\S+)\s*)+)/
+    BOOLEAN_OPERATORS = %w(and or)
-
-    def initialize(tag_name, markup, tokens)
-      @blocks = []
-
-      push_block('if', markup)
 
+    def initialize(tag_name, markup, options)
       super
+      @blocks = []
+      push_block('if'.freeze, markup)
+    end
+
+    def nodelist
+      @blocks.map(&:attachment).flatten
     end
 
     def unknown_tag(tag, markup, tokens)
-      if ['elsif', 'else'].include?(tag)
+      if ['elsif'.freeze, 'else'.freeze].include?(tag)
         push_block(tag, markup)
       else
         super
@@ -39,41 +39,68 @@ module Liquid
             return render_all(block.attachment, context)
           end
         end
-        ''
+        ''.freeze
       end
     end
 
     private
 
       def push_block(tag, markup)
-        block = if tag == 'else'
+        block = if tag == 'else'.freeze
           ElseCondition.new
         else
-
+          parse_with_selected_parser(markup)
-          expressions = markup.scan(ExpressionsAndOperators).reverse
-          raise(SyntaxError, SyntaxHelp) unless expressions.shift =~ Syntax
-
-          condition = Condition.new($1, $2, $3)
-
-          while not expressions.empty?
-            operator = (expressions.shift).to_s.strip
-
-            raise(SyntaxError, SyntaxHelp) unless expressions.shift.to_s =~ Syntax
-
-            new_condition = Condition.new($1, $2, $3)
-            new_condition.send(operator.to_sym, condition)
-            condition = new_condition
-          end
-
-          condition
         end
 
         @blocks.push(block)
         @nodelist = block.attach(Array.new)
       end
 
+      def lax_parse(markup)
+        expressions = markup.scan(ExpressionsAndOperators).reverse
+        raise(SyntaxError.new(options[:locale].t("errors.syntax.if".freeze))) unless expressions.shift =~ Syntax
 
+        condition = Condition.new($1, $2, $3)
+
+        while not expressions.empty?
+          operator = (expressions.shift).to_s.strip
+
+          raise(SyntaxError.new(options[:locale].t("errors.syntax.if".freeze))) unless expressions.shift.to_s =~ Syntax
+
+          new_condition = Condition.new($1, $2, $3)
+          raise(SyntaxError.new(options[:locale].t("errors.syntax.if".freeze))) unless BOOLEAN_OPERATORS.include?(operator)
+          new_condition.send(operator, condition)
+          condition = new_condition
         end
 
-  Template.register_tag('if', If)
+        condition
+      end
+
+      def strict_parse(markup)
+        p = Parser.new(markup)
+
+        condition = parse_comparison(p)
+
+        while op = (p.id?('and'.freeze) || p.id?('or'.freeze))
+          new_cond = parse_comparison(p)
+          new_cond.send(op, condition)
+          condition = new_cond
+        end
+        p.consume(:end_of_string)
+
+        condition
+      end
+
+      def parse_comparison(p)
+        a = p.expression
+        if op = p.consume?(:comparison)
+          b = p.expression
+          Condition.new(a, op, b)
+        else
+          Condition.new(a)
+        end
+      end
+  end
+
+  Template.register_tag('if'.freeze, If)
 end

lib/liquid/tags/ifchanged.rb

@@ -10,11 +10,11 @@ module Liquid
           context.registers[:ifchanged] = output
           output
         else
-          ''
+          ''.freeze
         end
       end
     end
   end
 
-  Template.register_tag('ifchanged', Ifchanged)  
+  Template.register_tag('ifchanged'.freeze, Ifchanged)
 end

lib/liquid/tags/include.rb

@@ -1,8 +1,25 @@
 module Liquid
-  class Include < Tag
-    Syntax = /(#{QuotedFragment}+)(\s+(?:with|for)\s+(#{QuotedFragment}+))?/
 
-    def initialize(tag_name, markup, tokens)      
+  # Include allows templates to relate with other templates
+  #
+  # Simply include another template:
+  #
+  #   {% include 'product' %}
+  #
+  # Include a template with a local variable:
+  #
+  #   {% include 'product' with products[0] %}
+  #
+  # Include a template for a collection:
+  #
+  #   {% include 'product' for products %}
+  #
+  class Include < Tag
+    Syntax = /(#{QuotedFragment}+)(\s+(?:with|for)\s+(#{QuotedFragment}+))?/o
+
+    def initialize(tag_name, markup, options)
+      super
+
       if markup =~ Syntax
 
         @template_name = $1
@@ -14,20 +31,19 @@ module Liquid
         end
 
       else
-        raise SyntaxError.new("Error in tag 'include' - Valid syntax: include '[template]' (with|for) [object|collection]")
+        raise SyntaxError.new(options[:locale].t("errors.syntax.include".freeze))
       end
-
-      super
     end
 
     def parse(tokens)
     end
 
-    def render(context)      
+    def blank?
-      file_system = context.registers[:file_system] || Liquid::Template.file_system
+      false
-      source  = file_system.read_template_file(context[@template_name])      
+    end
-      partial = Liquid::Template.parse(source)      
 
+    def render(context)
+      partial = load_cached_partial(context)
       variable = context[@variable_name || @template_name[1..-2]]
 
       context.stack do
@@ -35,22 +51,48 @@ module Liquid
           context[key] = context[value]
         end
 
+        context_variable_name = @template_name[1..-2].split('/'.freeze).last
         if variable.is_a?(Array)
-          
+          variable.collect do |var|
-          variable.collect do |variable|            
+            context[context_variable_name] = var
-            context[@template_name[1..-2]] = variable
             partial.render(context)
           end
-
         else
-                    
+          context[context_variable_name] = variable
-          context[@template_name[1..-2]] = variable
           partial.render(context)
-          
-        end
         end
       end
     end
 
-  Template.register_tag('include', Include)  
+    private
+      def load_cached_partial(context)
+        cached_partials = context.registers[:cached_partials] || {}
+        template_name = context[@template_name]
+
+        if cached = cached_partials[template_name]
+          return cached
+        end
+        source = read_template_from_file_system(context)
+        partial = Liquid::Template.parse(source)
+        cached_partials[template_name] = partial
+        context.registers[:cached_partials] = cached_partials
+        partial
+      end
+
+      def read_template_from_file_system(context)
+        file_system = context.registers[:file_system] || Liquid::Template.file_system
+
+        # make read_template_file call backwards-compatible.
+        case file_system.method(:read_template_file).arity
+        when 1
+          file_system.read_template_file(context[@template_name])
+        when 2
+          file_system.read_template_file(context[@template_name], context)
+        else
+          raise ArgumentError, "file_system.read_template_file expects two parameters: (template_name, context)"
+        end
+      end
+  end
+
+  Template.register_tag('include'.freeze, Include)
 end

lib/liquid/tags/increment.rb

@@ -0,0 +1,35 @@
+module Liquid
+  # increment is used in a place where one needs to insert a counter
+  #     into a template, and needs the counter to survive across
+  #     multiple instantiations of the template.
+  #     (To achieve the survival, the application must keep the context)
+  #
+  #     if the variable does not exist, it is created with value 0.
+  #
+  #   Hello: {% increment variable %}
+  #
+  # gives you:
+  #
+  #    Hello: 0
+  #    Hello: 1
+  #    Hello: 2
+  #
+  class Increment < Tag
+    def initialize(tag_name, markup, options)
+      super
+      @variable = markup.strip
+    end
+
+    def render(context)
+      value = context.environments.first[@variable] ||= 0
+      context.environments.first[@variable] = value + 1
+      value.to_s
+    end
+
+    def blank?
+      false
+    end
+  end
+
+  Template.register_tag('increment'.freeze, Increment)
+end

lib/liquid/tags/literal.rb

@@ -1,42 +0,0 @@
-module Liquid
-
-  class Literal < Block
-
-    # Class methods
-
-    # Converts a shorthand Liquid literal into its long representation.
-    #
-    # Currently the Template parser only knows how to handle the long version.
-    # So, it always checks if it is in the presence of a literal, in which case it gets converted through this method.
-    #
-    # Example:
-    #   Liquid::Literal "{{{ hello world }}}" #=> "{% literal %} hello world {% endliteral %}"
-    def self.from_shorthand(literal)
-      literal =~ LiteralShorthand ? "{% literal %}#{$1}{% endliteral %}" : literal
-    end
-
-    # Public instance methods
-
-    def parse(tokens) # :nodoc:
-      @nodelist ||= []
-      @nodelist.clear
-
-      while token = tokens.shift
-        if token =~ FullToken && block_delimiter == $1
-          end_tag
-          return
-        else
-          @nodelist << token
-        end
-      end
-
-      # Make sure that its ok to end parsing in the current block.
-      # Effectively this method will throw and exception unless the current block is
-      # of type Document
-      assert_missing_delimitation!
-    end # parse
-
-  end
-
-  Template.register_tag('literal', Literal)
-end

lib/liquid/tags/raw.rb

@@ -0,0 +1,22 @@
+module Liquid
+  class Raw < Block
+    FullTokenPossiblyInvalid = /\A(.*)#{TagStart}\s*(\w+)\s*(.*)?#{TagEnd}\z/om
+
+    def parse(tokens)
+      @nodelist ||= []
+      @nodelist.clear
+      while token = tokens.shift
+        if token =~ FullTokenPossiblyInvalid
+          @nodelist << $1 if $1 != "".freeze
+          if block_delimiter == $2
+            end_tag
+            return
+          end
+        end
+        @nodelist << token if not token.empty?
+      end
+    end
+  end
+
+  Template.register_tag('raw'.freeze, Raw)
+end

lib/liquid/tags/table_row.rb

@@ -0,0 +1,73 @@
+module Liquid
+  class TableRow < Block
+    Syntax = /(\w+)\s+in\s+(#{QuotedFragment}+)/o
+
+    def initialize(tag_name, markup, options)
+      super
+      if markup =~ Syntax
+        @variable_name = $1
+        @collection_name = $2
+        @attributes = {}
+        markup.scan(TagAttributes) do |key, value|
+          @attributes[key] = value
+        end
+      else
+        raise SyntaxError.new(options[:locale].t("errors.syntax.table_row".freeze))
+      end
+    end
+
+    def render(context)
+      collection = context[@collection_name] or return ''.freeze
+
+      from = @attributes['offset'.freeze] ? context[@attributes['offset'.freeze]].to_i : 0
+      to = @attributes['limit'.freeze] ? from + context[@attributes['limit'.freeze]].to_i : nil
+
+      collection = Utils.slice_collection(collection, from, to)
+
+      length = collection.length
+
+      cols = context[@attributes['cols'.freeze]].to_i
+
+      row = 1
+      col = 0
+
+      result = "<tr class=\"row1\">\n"
+      context.stack do
+
+        collection.each_with_index do |item, index|
+          context[@variable_name] = item
+          context['tablerowloop'.freeze] = {
+            'length'.freeze    => length,
+            'index'.freeze     => index + 1,
+            'index0'.freeze    => index,
+            'col'.freeze       => col + 1,
+            'col0'.freeze      => col,
+            'index0'.freeze    => index,
+            'rindex'.freeze    => length - index,
+            'rindex0'.freeze   => length - index - 1,
+            'first'.freeze     => (index == 0),
+            'last'.freeze      => (index == length - 1),
+            'col_first'.freeze => (col == 0),
+            'col_last'.freeze  => (col == cols - 1)
+          }
+
+
+          col += 1
+
+          result << "<td class=\"col#{col}\">" << render_all(@nodelist, context) << '</td>'
+
+          if col == cols and (index != length - 1)
+            col  = 0
+            row += 1
+            result << "</tr>\n<tr class=\"row#{row}\">"
+          end
+
+        end
+      end
+      result << "</tr>\n"
+      result
+    end
+  end
+
+  Template.register_tag('tablerow'.freeze, TableRow)
+end

lib/liquid/tags/unless.rb

@@ -1,7 +1,6 @@
 require File.dirname(__FILE__) + '/if'
 
 module Liquid
-
   # Unless is a conditional just like 'if' but works on the inverse logic.
   #
   #   {% unless x < 0 %} x is greater than zero {% end %}
@@ -11,9 +10,9 @@ module Liquid
       context.stack do
 
         # First condition is interpreted backwards ( if not )
-        block = @blocks.first
+        first_block = @blocks.first
-        unless block.evaluate(context)
+        unless first_block.evaluate(context)
-          return render_all(block.attachment, context)            
+          return render_all(first_block.attachment, context)
         end
 
         # After the first condition unless works just like if
@@ -23,11 +22,10 @@ module Liquid
           end
         end
 
-        ''
+        ''.freeze
       end
     end
   end
 
-
+  Template.register_tag('unless'.freeze, Unless)
-  Template.register_tag('unless', Unless)
 end

lib/liquid/template.rb

@@ -14,7 +14,11 @@ module Liquid
   #   template.render('user_name' => 'bob')
   #
   class Template
-    attr_accessor :root
+    DEFAULT_OPTIONS = {
+      :locale => I18n.new
+    }
+
+    attr_accessor :root, :resource_limits
     @@file_system = BlankFileSystem.new
 
     class << self
@@ -34,6 +38,18 @@ module Liquid
         @tags ||= {}
       end
 
+      # Sets how strict the parser should be.
+      # :lax acts like liquid 2.5 and silently ignores malformed tags in most cases.
+      # :warn is the default and will give deprecation warnings when invalid syntax is used.
+      # :strict will enforce correct syntax.
+      def error_mode=(mode)
+        @error_mode = mode
+      end
+
+      def error_mode
+        @error_mode || :lax
+      end
+
       # Pass a module with filter methods which should be available
       # to all liquid views. Good for registering the standard library
       def register_filter(mod)
@@ -41,24 +57,31 @@ module Liquid
       end
 
       # creates a new <tt>Template</tt> object from liquid source code
-      def parse(source)
+      def parse(source, options = {})
         template = Template.new
-        template.parse(source)
+        template.parse(source, options)
         template
       end
     end
 
     # creates a new <tt>Template</tt> from an array of tokens. Use <tt>Template.parse</tt> instead
     def initialize
+      @resource_limits = {}
     end
 
     # Parse source code.
     # Returns self for easy chaining
-    def parse(source)
+    def parse(source, options = {})
-      @root = Document.new(tokenize(Liquid::Literal.from_shorthand(source)))
+      @root = Document.parse(tokenize(source), DEFAULT_OPTIONS.merge(options))
+      @warnings = nil
       self
     end
 
+    def warnings
+      return [] unless @root
+      @warnings ||= @root.warnings
+    end
+
     def registers
       @registers ||= {}
     end
@@ -87,17 +110,20 @@ module Liquid
     #    filters and tags and might be useful to integrate liquid more with its host application
     #
     def render(*args)
-      return '' if @root.nil?
+      return ''.freeze if @root.nil?
 
       context = case args.first
       when Liquid::Context
         args.shift
+      when Liquid::Drop
+        drop = args.shift
+        drop.context = Context.new([drop, assigns], instance_assigns, registers, @rethrow_errors, @resource_limits)
       when Hash
-        Context.new([args.shift, assigns], instance_assigns, registers, @rethrow_errors)
+        Context.new([args.shift, assigns], instance_assigns, registers, @rethrow_errors, @resource_limits)
       when nil
-        Context.new(assigns, instance_assigns, registers, @rethrow_errors)
+        Context.new(assigns, instance_assigns, registers, @rethrow_errors, @resource_limits)
       else
-        raise ArgumentError, "Expect Hash or Liquid::Context as parameter"
+        raise ArgumentError, "Expected Hash or Liquid::Context as parameter"
       end
 
       case args.last
@@ -120,8 +146,11 @@ module Liquid
 
       begin
         # render the nodelist.
-        # for performance reasons we get a array back here. join will make a string out of it
+        # for performance reasons we get an array back here. join will make a string out of it.
-        @root.render(context).join
+        result = @root.render(context)
+        result.respond_to?(:join) ? result.join : result
+      rescue Liquid::MemoryError => e
+        context.handle_error(e)
       ensure
         @errors = context.errors
       end
@@ -133,16 +162,9 @@ module Liquid
 
     private
 
-    # Uses the <tt>Liquid::TemplateParser</tt> regexp to tokenize the passed source
     def tokenize(source)
       source = source.source if source.respond_to?(:source)
-      return [] if source.to_s.empty?
+      Tokenizer.new(source.to_s)
-      tokens = source.split(TemplateParser)
-
-      # removes the rogue empty element at the beginning of the array
-      tokens.shift if tokens[0] and tokens[0].empty?
-
-      tokens
     end
 
   end

lib/liquid/tokenizer.rb

@@ -0,0 +1,20 @@
+module Liquid
+  class Tokenizer
+    VariableIncompleteEnd = /\}\}?/
+    AnyStartingTag        = /\{\{|\{\%/
+    PartialTemplateParser = /#{TagStart}.*?#{TagEnd}|#{VariableStart}.*?#{VariableIncompleteEnd}/om
+    TemplateParser        = /(#{PartialTemplateParser}|#{AnyStartingTag})/om
+
+    def initialize(source)
+      @tokens = source.split(TemplateParser)
+
+      # removes the rogue empty element at the beginning of the array
+      @tokens.shift if @tokens[0] && @tokens[0].empty?
+    end
+
+    def next
+      @tokens.shift
+    end
+    alias_method :shift, :next
+  end
+end

lib/liquid/utils.rb

@@ -0,0 +1,39 @@
+module Liquid
+  module Utils
+
+    def self.slice_collection(collection, from, to)
+      if (from != 0 || to != nil) && collection.respond_to?(:load_slice)
+        collection.load_slice(from, to)
+      else
+        slice_collection_using_each(collection, from, to)
+      end
+    end
+
+    def self.non_blank_string?(collection)
+      collection.is_a?(String) && collection != ''.freeze
+    end
+
+    def self.slice_collection_using_each(collection, from, to)
+      segments = []
+      index = 0
+
+      # Maintains Ruby 1.8.7 String#each behaviour on 1.9
+      return [collection] if non_blank_string?(collection)
+
+      collection.each do |item|
+
+        if to && to <= index
+          break
+        end
+
+        if from <= index
+          segments << item
+        end
+
+        index += 1
+      end
+
+      segments
+    end
+  end
+end

lib/liquid/variable.rb

@@ -11,34 +11,92 @@ module Liquid
   #   {{ user | link }}
   #
   class Variable
-    FilterParser = /(?:#{FilterSeparator}|(?:\s*(?!(?:#{FilterSeparator}))(?:#{QuotedFragment}|\S+)\s*)+)/
+    FilterParser = /(?:#{FilterSeparator}|(?:\s*(?:#{QuotedFragment}|#{ArgumentSeparator})\s*)+)/o
-    attr_accessor :filters, :name
+    EasyParse = /\A *(\w+(?:\.\w+)*) *\z/
+    attr_accessor :filters, :name, :warnings
 
-    def initialize(markup)
+    def initialize(markup, options = {})
       @markup  = markup
       @name    = nil
+      @options = options || {}
+
+      case @options[:error_mode] || Template.error_mode
+      when :strict then strict_parse(markup)
+      when :lax    then lax_parse(markup)
+      when :warn
+        begin
+          strict_parse(markup)
+        rescue SyntaxError => e
+          @warnings ||= []
+          @warnings << e
+          lax_parse(markup)
+        end
+      end
+    end
+
+    def lax_parse(markup)
       @filters = []
-      if match = markup.match(/\s*(#{QuotedFragment})(.*)/)
+      if match = markup.match(/\s*(#{QuotedFragment})(.*)/om)
         @name = match[1]
-        if match[2].match(/#{FilterSeparator}\s*(.*)/)
+        if match[2].match(/#{FilterSeparator}\s*(.*)/om)
           filters = Regexp.last_match(1).scan(FilterParser)
           filters.each do |f|
             if matches = f.match(/\s*(\w+)/)
               filtername = matches[1]
-              filterargs = f.scan(/(?:#{FilterArgumentSeparator}|#{ArgumentSeparator})\s*(#{QuotedFragment})/).flatten
+              filterargs = f.scan(/(?:#{FilterArgumentSeparator}|#{ArgumentSeparator})\s*((?:\w+\s*\:\s*)?#{QuotedFragment})/o).flatten
-              @filters << [filtername.to_sym, filterargs]
+              @filters << [filtername, filterargs]
             end
           end
         end
       end
     end
 
-    def render(context)
+    def strict_parse(markup)
-      return '' if @name.nil?
+      # Very simple valid cases
-      @filters.inject(context[@name]) do |output, filter|
+      if markup =~ EasyParse
-        filterargs = filter[1].to_a.collect do |a|
+        @name = $1
-         context[a]
+        @filters = []
+        return
       end
+
+      @filters = []
+      p = Parser.new(markup)
+      # Could be just filters with no input
+      @name = p.look(:pipe) ? ''.freeze : p.expression
+      while p.consume?(:pipe)
+        filtername = p.consume(:id)
+        filterargs = p.consume?(:colon) ? parse_filterargs(p) : []
+        @filters << [filtername, filterargs]
+      end
+      p.consume(:end_of_string)
+    rescue SyntaxError => e
+      e.message << " in \"{{#{markup}}}\""
+      raise e
+    end
+
+    def parse_filterargs(p)
+      # first argument
+      filterargs = [p.argument]
+      # followed by comma separated others
+      while p.consume?(:comma)
+        filterargs << p.argument
+      end
+      filterargs
+    end
+
+    def render(context)
+      return ''.freeze if @name.nil?
+      @filters.inject(context[@name]) do |output, filter|
+        filterargs = []
+        keyword_args = {}
+        filter[1].to_a.each do |a|
+          if matches = a.match(/\A#{TagAttributes}\z/o)
+            keyword_args[matches[1]] = context[matches[2]]
+          else
+            filterargs << context[a]
+          end
+        end
+        filterargs << keyword_args unless keyword_args.empty?
         begin
           output = context.invoke(filter[0], output, *filterargs)
         rescue FilterNotFound

lib/liquid/version.rb

@@ -0,0 +1,4 @@
+# encoding: utf-8
+module Liquid
+  VERSION = "3.0.0".freeze
+end

liquid.gemspec

@@ -1,23 +1,34 @@
-# -*- encoding: utf-8 -*-
+# encoding: utf-8
+lib = File.expand_path('../lib/', __FILE__)
+$:.unshift lib unless $:.include?(lib)
+
+require "liquid/version"
 
 Gem::Specification.new do |s|
-  s.platform    = Gem::Platform::RUBY
-
   s.name        = "liquid"
-  s.version     = '2.2.2'
+  s.version     = Liquid::VERSION
+  s.platform    = Gem::Platform::RUBY
   s.summary     = "A secure, non-evaling end user template engine with aesthetic markup."
-
   s.authors     = ["Tobias Luetke"]
   s.email       = ["tobi@leetsoft.com"]
   s.homepage    = "http://www.liquidmarkup.org"
-
+  s.license     = "MIT"
-  s.description = "A secure, non-evaling end user template engine with aesthetic markup."
+  #s.description = "A secure, non-evaling end user template engine with aesthetic markup."
 
   s.required_rubygems_version = ">= 1.3.7"
 
-  s.files             = Dir.glob("{lib}/**/*") + %w(MIT-LICENSE README.md)
+  s.test_files  = Dir.glob("{test}/**/*")
+  s.files       = Dir.glob("{lib,ext}/**/*") + %w(MIT-LICENSE README.md)
 
-  s.extra_rdoc_files  = ["History.txt", "Manifest.txt", "README.md"]
+  s.extra_rdoc_files  = ["History.md", "README.md"]
 
-  s.require_path = 'lib'
+  s.require_path = "lib"
+
+  s.add_development_dependency 'rake'
+  s.add_development_dependency 'activesupport'
+  if defined?(RUBY_ENGINE) && RUBY_ENGINE == 'ruby'
+    s.extensions  = ['ext/liquid/extconf.rb']
+    s.add_development_dependency 'rake-compiler'
+    s.add_development_dependency 'stackprof' if Gem::Version.new(RUBY_VERSION) >= Gem::Version.new("2.1.0")
+  end
 end

performance/benchmark.rb

@@ -0,0 +1,12 @@
+require 'rubygems'
+require 'benchmark'
+require File.dirname(__FILE__) + '/theme_runner'
+
+Liquid::Template.error_mode = ARGV.first.to_sym if ARGV.first
+profiler = ThemeRunner.new
+
+Benchmark.bmbm do |x|
+  x.report("parse:")   { 100.times { profiler.compile } }
+  x.report("parse & run:")   { 100.times { profiler.run } }
+end
+

performance/profile.rb

@@ -0,0 +1,13 @@
+require 'stackprof' rescue fail("install stackprof extension/gem")
+require File.dirname(__FILE__) + '/theme_runner'
+
+Liquid::Template.error_mode = ARGV.first.to_sym if ARGV.first
+profiler = ThemeRunner.new
+profiler.run
+results = StackProf.run(mode: :cpu) do
+  100.times do
+    profiler.run
+  end
+end
+StackProf::Report.new(results).print_text(false, 20)
+File.write(ENV['FILENAME'], Marshal.dump(results)) if ENV['FILENAME']

performance/shopify.rb

@@ -1,92 +0,0 @@
-# This profiler run simulates Shopify.
-# We are looking in the tests directory for liquid files and render them within the designated layout file.
-# We will also export a substantial database to liquid which the templates can render values of. 
-# All this is to make the benchmark as non syntetic as possible. All templates and tests are lifted from
-# direct real-world usage and the profiler measures code that looks very similar to the way it looks in 
-# Shopify which is likely the biggest user of liquid in the world which something to the tune of several 
-# million Template#render calls a day. 
-
-require 'rubygems'
-require 'active_support'
-require 'yaml'
-require 'digest/md5'
-require File.dirname(__FILE__) + '/shopify/liquid'
-require File.dirname(__FILE__) + '/shopify/database.rb'
-
-require "ruby-prof" rescue fail("install ruby-prof extension/gem")
-
-class ThemeProfiler
-  
-  # Load all templates into memory, do this now so that 
-  # we don't profile IO. 
-  def initialize
-    @tests = Dir[File.dirname(__FILE__) + '/tests/**/*.liquid'].collect do |test|
-      next if File.basename(test) == 'theme.liquid'
-      
-      theme_path = File.dirname(test) + '/theme.liquid'
-      
-      [File.read(test), (File.file?(theme_path) ? File.read(theme_path) : nil), test]
-    end.compact
-  end  
-  
-  
-  def profile
-    RubyProf.measure_mode = RubyProf::WALL_TIME
-    
-    # Dup assigns because will make some changes to them
-    assigns = Database.tables.dup
-    
-    @tests.each do |liquid, layout, template_name|
-      
-      # Compute page_tempalte outside of profiler run, uninteresting to profiler
-      html = nil
-      page_template = File.basename(template_name, File.extname(template_name))
-      
-      # Profile compiling and rendering both
-      RubyProf.resume { html = compile_and_render(liquid, layout, assigns, page_template) }
-      
-      # return the result and the MD5 of the content, this can be used to detect regressions between liquid version
-      $stdout.puts "* rendered template %s, content: %s" % [template_name, Digest::MD5.hexdigest(html)]
-      
-      # Uncomment to dump html files to /tmp so that you can inspect for errors
-      # File.open("/tmp/#{File.basename(template_name)}.html", "w+") { |fp| fp <<html}
-    end
-  
-    RubyProf.stop    
-  end
-  
-  def compile_and_render(template, layout, assigns, page_template)    
-    tmpl = Liquid::Template.new
-    tmpl.assigns['page_title'] = 'Page title'
-    tmpl.assigns['template'] = page_template
-    
-    content_for_layout = tmpl.parse(template).render(assigns)
-    
-    if layout
-      assigns['content_for_layout'] = content_for_layout      
-      tmpl.parse(layout).render(assigns)
-    else
-      content_for_layout
-    end    
-  end
-end
-
-
-  profiler = ThemeProfiler.new
-
-  puts 'Running profiler...'
-
-  results  = profiler.profile
-  
-  puts 'Success'
-  puts
-
-[RubyProf::FlatPrinter, RubyProf::GraphPrinter, RubyProf::GraphHtmlPrinter, RubyProf::CallTreePrinter].each do |klass|
-  filename = (ENV['TMP'] || '/tmp') + (klass.name.include?('Html') ? "/liquid.#{klass.name.downcase}.html" : "/liquid.#{klass.name.downcase}.txt")
-  filename.gsub!(/:+/, '_')
-  File.open(filename, "w+") { |fp| klass.new(results).print(fp) }
-  $stderr.puts "wrote #{klass.name} output to #{filename}"
-end
-
-
-

performance/shopify/comment_form.rb

@@ -1,15 +1,15 @@
 class CommentForm < Liquid::Block
   Syntax = /(#{Liquid::VariableSignature}+)/
 
-  def initialize(tag_name, markup, tokens)
+  def initialize(tag_name, markup, options)
+    super
+
     if markup =~ Syntax
       @variable_name = $1
       @attributes = {}
     else
       raise SyntaxError.new("Syntax Error in 'comment_form' - Valid syntax: comment_form [article]")
     end
-    
-    super
   end
 
   def render(context)

performance/shopify/liquid.rb

@@ -1,3 +1,4 @@
+$:.unshift File.dirname(__FILE__) + '/../../lib'
 require File.dirname(__FILE__) + '/../../lib/liquid'
 
 require File.dirname(__FILE__) + '/comment_form'

performance/shopify/paginate.rb

@@ -1,7 +1,9 @@
 class Paginate < Liquid::Block
   Syntax     = /(#{Liquid::QuotedFragment})\s*(by\s*(\d+))?/
 
-  def initialize(tag_name, markup, tokens)
+  def initialize(tag_name, markup, options)
+    super
+
     @nodelist = []
 
     if markup =~ Syntax
@@ -19,8 +21,6 @@ class Paginate < Liquid::Block
     else
       raise SyntaxError.new("Syntax Error in tag 'paginate' - Valid syntax: paginate [collection] by number")
     end
-  
-    super
   end
 
   def render(context)

performance/shopify/shop_filter.rb

@@ -45,16 +45,16 @@ module ShopFilter
   end
 
   def url_for_vendor(vendor_title)
-    "/collections/#{vendor_title.to_handle}"
+    "/collections/#{to_handle(vendor_title)}"
   end
 
   def url_for_type(type_title)
-    "/collections/#{type_title.to_handle}"
+    "/collections/#{to_handle(type_title)}"
   end
 
   def product_img_url(url, style = 'small')
 
-    unless url =~ /^products\/([\w\-\_]+)\.(\w{2,4})/
+    unless url =~ /\Aproducts\/([\w\-\_]+)\.(\w{2,4})/
       raise ArgumentError, 'filter "size" can only be called on product images'
     end
 
@@ -95,4 +95,16 @@ module ShopFilter
     input == 1 ? singular : plural
   end
 
+  private
+
+  def to_handle(str)
+    result = str.dup
+    result.downcase!
+    result.delete!("'\"()[]")
+    result.gsub!(/\W+/, '-')
+    result.gsub!(/-+\z/, '') if result[-1] == '-'
+    result.gsub!(/\A-+/, '') if result[0] == '-'
+    result
+  end
+
 end

performance/tests/dropify/index.liquid

@@ -28,7 +28,7 @@
   {% else %}
     <div class="article-body textile">
     In <em>Admin &gt; Blogs &amp; Pages</em>, create a page with the handle <strong><code>frontpage</code></strong> and it will show up here.<br />
-	  {{ "Learn more about handles" | link_to "http://wiki.shopify.com/Handle" }}
+    {{ "Learn more about handles" | link_to: "http://wiki.shopify.com/Handle" }}
     </div>
   {% endif %}
 

performance/tests/ripen/index.liquid

@@ -26,7 +26,7 @@
       {{ article.content }}
     {% else %}
       In <em>Admin &gt; Blogs &amp; Pages</em>, create a page with the handle <strong><code>frontpage</code></strong> and it will show up here.<br />
-  	  {{ "Learn more about handles" | link_to "http://wiki.shopify.com/Handle" }}
+      {{ "Learn more about handles" | link_to: "http://wiki.shopify.com/Handle" }}
     {% endif %}
   </div>
 </div>

performance/tests/vogue/index.liquid

@@ -5,7 +5,7 @@
       {{ article.content }}
     {% else %}
       In <em>Admin &gt; Blogs &amp; Pages</em>, create a page with the handle <strong><code>frontpage</code></strong> and it will show up here.<br />
-  	  {{ "Learn more about handles" | link_to "http://wiki.shopify.com/Handle" }}
+      {{ "Learn more about handles" | link_to: "http://wiki.shopify.com/Handle" }}
     {% endif %}
   </div>
 

performance/theme_runner.rb

@@ -0,0 +1,86 @@
+# This profiler run simulates Shopify.
+# We are looking in the tests directory for liquid files and render them within the designated layout file.
+# We will also export a substantial database to liquid which the templates can render values of.
+# All this is to make the benchmark as non syntetic as possible. All templates and tests are lifted from
+# direct real-world usage and the profiler measures code that looks very similar to the way it looks in
+# Shopify which is likely the biggest user of liquid in the world which something to the tune of several
+# million Template#render calls a day.
+
+require 'rubygems'
+require 'active_support'
+require 'active_support/json'
+require 'yaml'
+require 'digest/md5'
+require File.dirname(__FILE__) + '/shopify/liquid'
+require File.dirname(__FILE__) + '/shopify/database.rb'
+
+class ThemeRunner
+  class FileSystem
+
+    def initialize(path)
+      @path = path
+    end
+
+    # Called by Liquid to retrieve a template file
+    def read_template_file(template_path, context)
+      File.read(@path + '/' + template_path + '.liquid')
+    end
+  end
+
+  # Load all templates into memory, do this now so that
+  # we don't profile IO.
+  def initialize
+    @tests = Dir[File.dirname(__FILE__) + '/tests/**/*.liquid'].collect do |test|
+      next if File.basename(test) == 'theme.liquid'
+
+      theme_path = File.dirname(test) + '/theme.liquid'
+
+      [File.read(test), (File.file?(theme_path) ? File.read(theme_path) : nil), test]
+    end.compact
+  end
+
+  def compile
+    # Dup assigns because will make some changes to them
+
+    @tests.each do |liquid, layout, template_name|
+
+      tmpl = Liquid::Template.new
+      tmpl.parse(liquid)
+      tmpl = Liquid::Template.new
+      tmpl.parse(layout)
+    end
+  end
+
+   def run
+    # Dup assigns because will make some changes to them
+    assigns = Database.tables.dup
+
+    @tests.each do |liquid, layout, template_name|
+
+      # Compute page_tempalte outside of profiler run, uninteresting to profiler
+      page_template = File.basename(template_name, File.extname(template_name))
+      compile_and_render(liquid, layout, assigns, page_template, template_name)
+
+    end
+  end
+
+
+  def compile_and_render(template, layout, assigns, page_template, template_file)
+    tmpl = Liquid::Template.new
+    tmpl.assigns['page_title'] = 'Page title'
+    tmpl.assigns['template'] = page_template
+    tmpl.registers[:file_system] = ThemeRunner::FileSystem.new(File.dirname(template_file))
+
+    content_for_layout = tmpl.parse(template).render!(assigns)
+
+    if layout
+      assigns['content_for_layout'] = content_for_layout
+      tmpl.parse(layout).render!(assigns)
+    else
+      content_for_layout
+    end
+  end
+end
+
+
+

test/extra/breakpoint.rb

@@ -1,547 +0,0 @@
-# The Breakpoint library provides the convenience of
-# being able to inspect and modify state, diagnose
-# bugs all via IRB by simply setting breakpoints in
-# your applications by the call of a method.
-#
-# This library was written and is supported by me,
-# Florian Gross. I can be reached at flgr@ccan.de
-# and enjoy getting feedback about my libraries.
-#
-# The whole library (including breakpoint_client.rb
-# and binding_of_caller.rb) is licensed under the
-# same license that Ruby uses. (Which is currently
-# either the GNU General Public License or a custom
-# one that allows for commercial usage.) If you for
-# some good reason need to use this under another
-# license please contact me.
-
-require 'irb'
-require 'caller'
-require 'drb'
-require 'drb/acl'
-require 'thread'
-
-module Breakpoint
-  id = %q$Id: breakpoint.rb 52 2005-02-26 19:43:19Z flgr $
-  current_version = id.split(" ")[2]
-  unless defined?(Version)
-    # The Version of ruby-breakpoint you are using as String of the
-    # 1.2.3 form where the digits stand for release, major and minor
-    # version respectively.
-    Version = "0.5.0"
-  end
-
-  extend self
-
-  # This will pop up an interactive ruby session at a
-  # pre-defined break point in a Ruby application. In
-  # this session you can examine the environment of
-  # the break point.
-  #
-  # You can get a list of variables in the context using
-  # local_variables via +local_variables+. You can then
-  # examine their values by typing their names.
-  #
-  # You can have a look at the call stack via +caller+.
-  #
-  # The source code around the location where the breakpoint
-  # was executed can be examined via +source_lines+. Its
-  # argument specifies how much lines of context to display.
-  # The default amount of context is 5 lines. Note that
-  # the call to +source_lines+ can raise an exception when
-  # it isn't able to read in the source code.
-  #
-  # breakpoints can also return a value. They will execute
-  # a supplied block for getting a default return value.
-  # A custom value can be returned from the session by doing
-  # +throw(:debug_return, value)+.
-  #
-  # You can also give names to break points which will be
-  # used in the message that is displayed upon execution 
-  # of them.
-  #
-  # Here's a sample of how breakpoints should be placed:
-  #
-  #   class Person
-  #     def initialize(name, age)
-  #       @name, @age = name, age
-  #       breakpoint("Person#initialize")
-  #     end
-  #
-  #     attr_reader :age
-  #     def name
-  #       breakpoint("Person#name") { @name }
-  #     end
-  #   end
-  #
-  #   person = Person.new("Random Person", 23)
-  #   puts "Name: #{person.name}"
-  #
-  # And here is a sample debug session:
-  #
-  #   Executing break point "Person#initialize" at file.rb:4 in `initialize'
-  #   irb(#<Person:0x292fbe8>):001:0> local_variables
-  #   => ["name", "age", "_", "__"]
-  #   irb(#<Person:0x292fbe8>):002:0> [name, age]
-  #   => ["Random Person", 23]
-  #   irb(#<Person:0x292fbe8>):003:0> [@name, @age]
-  #   => ["Random Person", 23]
-  #   irb(#<Person:0x292fbe8>):004:0> self
-  #   => #<Person:0x292fbe8 @age=23, @name="Random Person">
-  #   irb(#<Person:0x292fbe8>):005:0> @age += 1; self
-  #   => #<Person:0x292fbe8 @age=24, @name="Random Person">
-  #   irb(#<Person:0x292fbe8>):006:0> exit
-  #   Executing break point "Person#name" at file.rb:9 in `name'
-  #   irb(#<Person:0x292fbe8>):001:0> throw(:debug_return, "Overriden name")
-  #   Name: Overriden name
-  #
-  # Breakpoint sessions will automatically have a few
-  # convenience methods available. See Breakpoint::CommandBundle
-  # for a list of them.
-  #
-  # Breakpoints can also be used remotely over sockets.
-  # This is implemented by running part of the IRB session
-  # in the application and part of it in a special client.
-  # You have to call Breakpoint.activate_drb to enable
-  # support for remote breakpoints and then run
-  # breakpoint_client.rb which is distributed with this
-  # library. See the documentation of Breakpoint.activate_drb
-  # for details.
-  def breakpoint(id = nil, context = nil, &block)
-    callstack = caller
-    callstack.slice!(0, 3) if callstack.first["breakpoint"]
-    file, line, method = *callstack.first.match(/^(.+?):(\d+)(?::in `(.*?)')?/).captures
-
-    message = "Executing break point " + (id ? "#{id.inspect} " : "") +
-              "at #{file}:#{line}" + (method ? " in `#{method}'" : "")
-
-    if context then
-      return handle_breakpoint(context, message, file, line, &block)
-    end
-
-    Binding.of_caller do |binding_context|
-      handle_breakpoint(binding_context, message, file, line, &block)
-    end
-  end
-
-  # These commands are automatically available in all breakpoint shells.
-  module CommandBundle
-    # Proxy to a Breakpoint client. Lets you directly execute code
-    # in the context of the client.
-    class Client
-      def initialize(eval_handler) # :nodoc:
-        eval_handler.untaint
-        @eval_handler = eval_handler
-      end
-
-      instance_methods.each do |method|
-        next if method[/^__.+__$/]
-        undef_method method
-      end
-
-      # Executes the specified code at the client.
-      def eval(code)
-        @eval_handler.call(code)
-      end
-
-      # Will execute the specified statement at the client.
-      def method_missing(method, *args, &block)
-        if args.empty? and not block
-          result = eval "#{method}"
-        else
-          # This is a bit ugly. The alternative would be using an
-          # eval context instead of an eval handler for executing
-          # the code at the client. The problem with that approach
-          # is that we would have to handle special expressions
-          # like "self", "nil" or constants ourself which is hard.
-          remote = eval %{
-            result = lambda { |block, *args| #{method}(*args, &block) }
-            def result.call_with_block(*args, &block)
-              call(block, *args)
-            end
-            result
-          }
-          remote.call_with_block(*args, &block)
-        end
-
-        return result
-      end
-    end
-
-    # Returns the source code surrounding the location where the
-    # breakpoint was issued.
-    def source_lines(context = 5, return_line_numbers = false)
-      lines = File.readlines(@__bp_file).map { |line| line.chomp }
-
-      break_line = @__bp_line
-      start_line = [break_line - context, 1].max
-      end_line = break_line + context
-
-      result = lines[(start_line - 1) .. (end_line - 1)]
-
-      if return_line_numbers then
-        return [start_line, break_line, result]
-      else
-        return result
-      end
-    end
-
-    # Lets an object that will forward method calls to the breakpoint
-    # client. This is useful for outputting longer things at the client
-    # and so on. You can for example do these things:
-    #
-    #   client.puts "Hello" # outputs "Hello" at client console
-    #   # outputs "Hello" into the file temp.txt at the client
-    #   client.File.open("temp.txt", "w") { |f| f.puts "Hello" } 
-    def client()
-      if Breakpoint.use_drb? then
-        sleep(0.5) until Breakpoint.drb_service.eval_handler
-        Client.new(Breakpoint.drb_service.eval_handler)
-      else
-        Client.new(lambda { |code| eval(code, TOPLEVEL_BINDING) })
-      end
-    end
-  end
-
-  def handle_breakpoint(context, message, file = "", line = "", &block) # :nodoc:
-    catch(:debug_return) do |value|
-      eval(%{
-        @__bp_file = #{file.inspect}
-        @__bp_line = #{line}
-        extend Breakpoint::CommandBundle
-        extend DRbUndumped if self
-      }, context) rescue nil
-
-      if not use_drb? then
-        puts message
-        IRB.start(nil, IRB::WorkSpace.new(context))
-      else
-        @drb_service.add_breakpoint(context, message)
-      end
-
-      block.call if block
-    end
-  end
-
-  # These exceptions will be raised on failed asserts
-  # if Breakpoint.asserts_cause_exceptions is set to
-  # true.
-  class FailedAssertError < RuntimeError
-  end
-
-  # This asserts that the block evaluates to true.
-  # If it doesn't evaluate to true a breakpoint will
-  # automatically be created at that execution point.
-  #
-  # You can disable assert checking in production
-  # code by setting Breakpoint.optimize_asserts to
-  # true. (It will still be enabled when Ruby is run
-  # via the -d argument.)
-  #
-  # Example:
-  #   person_name = "Foobar"
-  #   assert { not person_name.nil? }
-  #
-  # Note: If you want to use this method from an
-  # unit test, you will have to call it by its full
-  # name, Breakpoint.assert.
-  def assert(context = nil, &condition)
-    return if Breakpoint.optimize_asserts and not $DEBUG
-    return if yield
-
-    callstack = caller
-    callstack.slice!(0, 3) if callstack.first["assert"]
-    file, line, method = *callstack.first.match(/^(.+?):(\d+)(?::in `(.*?)')?/).captures
-
-    message = "Assert failed at #{file}:#{line}#{" in `#{method}'" if method}."
-
-    if Breakpoint.asserts_cause_exceptions and not $DEBUG then
-      raise(Breakpoint::FailedAssertError, message)
-    end
-
-    message += " Executing implicit breakpoint."
-
-    if context then
-      return handle_breakpoint(context, message, file, line)
-    end
-
-    Binding.of_caller do |context|
-      handle_breakpoint(context, message, file, line)
-    end
-  end
-
-  # Whether asserts should be ignored if not in debug mode.
-  # Debug mode can be enabled by running ruby with the -d
-  # switch or by setting $DEBUG to true.
-  attr_accessor :optimize_asserts
-  self.optimize_asserts = false
-
-  # Whether an Exception should be raised on failed asserts
-  # in non-$DEBUG code or not. By default this is disabled.
-  attr_accessor :asserts_cause_exceptions
-  self.asserts_cause_exceptions = false
-  @use_drb = false
-
-  attr_reader :drb_service # :nodoc:
-
-  class DRbService # :nodoc:
-    include DRbUndumped
-
-    def initialize
-      @handler = @eval_handler = @collision_handler = nil
-
-      IRB.instance_eval { @CONF[:RC] = true }
-      IRB.run_config
-    end
-
-    def collision
-      sleep(0.5) until @collision_handler
-
-      @collision_handler.untaint
-
-      @collision_handler.call
-    end
-
-    def ping() end
-
-    def add_breakpoint(context, message)
-      workspace = IRB::WorkSpace.new(context)
-      workspace.extend(DRbUndumped)
-
-      sleep(0.5) until @handler
-
-      @handler.untaint
-      @handler.call(workspace, message)
-    rescue Errno::ECONNREFUSED, DRb::DRbConnError
-      raise if Breakpoint.use_drb? 
-    end
-
-    attr_accessor :handler, :eval_handler, :collision_handler
-  end
-
-  # Will run Breakpoint in DRb mode. This will spawn a server
-  # that can be attached to via the breakpoint-client command
-  # whenever a breakpoint is executed. This is useful when you
-  # are debugging CGI applications or other applications where
-  # you can't access debug sessions via the standard input and
-  # output of your application.
-  #
-  # You can specify an URI where the DRb server will run at.
-  # This way you can specify the port the server runs on. The
-  # default URI is druby://localhost:42531.
-  #
-  # Please note that breakpoints will be skipped silently in
-  # case the DRb server can not spawned. (This can happen if
-  # the port is already used by another instance of your
-  # application on CGI or another application.)
-  #
-  # Also note that by default this will only allow access
-  # from localhost. You can however specify a list of
-  # allowed hosts or nil (to allow access from everywhere).
-  # But that will still not protect you from somebody
-  # reading the data as it goes through the net.
-  #
-  # A good approach for getting security and remote access
-  # is setting up an SSH tunnel between the DRb service
-  # and the client. This is usually done like this:
-  #
-  # $ ssh -L20000:127.0.0.1:20000 -R10000:127.0.0.1:10000 example.com
-  # (This will connect port 20000 at the client side to port
-  # 20000 at the server side, and port 10000 at the server
-  # side to port 10000 at the client side.)
-  #
-  # After that do this on the server side: (the code being debugged)
-  # Breakpoint.activate_drb("druby://127.0.0.1:20000", "localhost")
-  #
-  # And at the client side:
-  # ruby breakpoint_client.rb -c druby://127.0.0.1:10000 -s druby://127.0.0.1:20000
-  #
-  # Running through such a SSH proxy will also let you use 
-  # breakpoint.rb in case you are behind a firewall.
-  #
-  # Detailed information about running DRb through firewalls is
-  # available at http://www.rubygarden.org/ruby?DrbTutorial
-  #
-  # == Security considerations
-  # Usually you will be fine when using the default druby:// URI and the default
-  # access control list. However, if you are sitting on a machine where there are
-  # local users that you likely can not trust (this is the case for example on
-  # most web hosts which have multiple users sitting on the same physical machine)
-  # you will be better off by doing client/server communication through a unix
-  # socket. This can be accomplished by calling with a drbunix:/ style URI, e.g.
-  # <code>Breakpoint.activate_drb('drbunix:/tmp/breakpoint_server')</code>. This
-  # will only work on Unix based platforms.
-  def activate_drb(uri = nil, allowed_hosts = ['localhost', '127.0.0.1', '::1'],
-    ignore_collisions = false)
-
-    return false if @use_drb
-
-    uri ||= 'druby://localhost:42531'
-
-    if allowed_hosts then
-      acl = ["deny", "all"]
-
-      Array(allowed_hosts).each do |host|
-        acl += ["allow", host]
-      end
-
-      DRb.install_acl(ACL.new(acl))
-    end
-
-    @use_drb = true
-    @drb_service = DRbService.new
-    did_collision = false
-    begin
-      @service = DRb.start_service(uri, @drb_service)
-    rescue Errno::EADDRINUSE
-      if ignore_collisions then
-        nil
-      else
-        # The port is already occupied by another
-        # Breakpoint service. We will try to tell
-        # the old service that we want its port.
-        # It will then forward that request to the
-        # user and retry.
-        unless did_collision then
-          DRbObject.new(nil, uri).collision
-          did_collision = true
-        end
-        sleep(10)
-        retry
-      end
-    end
-
-    return true
-  end
-
-  # Deactivates a running Breakpoint service.
-  def deactivate_drb
-    Thread.exclusive do
-      @service.stop_service unless @service.nil?
-      @service = nil
-      @use_drb = false
-      @drb_service = nil
-    end
-  end
-
-  # Returns true when Breakpoints are used over DRb.
-  # Breakpoint.activate_drb causes this to be true.
-  def use_drb?
-    @use_drb == true
-  end
-end
-
-module IRB # :nodoc:
-  class << self; remove_method :start; end
-  def self.start(ap_path = nil, main_context = nil, workspace = nil)
-    $0 = File::basename(ap_path, ".rb") if ap_path
-
-    # suppress some warnings about redefined constants
-    old_verbose, $VERBOSE = $VERBOSE, nil
-    IRB.setup(ap_path)
-    $VERBOSE = old_verbose
-
-    if @CONF[:SCRIPT] then
-      irb = Irb.new(main_context, @CONF[:SCRIPT])
-    else
-      irb = Irb.new(main_context)
-    end
-
-    if workspace then
-      irb.context.workspace = workspace
-    end
-
-    @CONF[:IRB_RC].call(irb.context) if @CONF[:IRB_RC]
-    @CONF[:MAIN_CONTEXT] = irb.context
-
-    old_sigint = trap("SIGINT") do
-      begin
-        irb.signal_handle
-      rescue RubyLex::TerminateLineInput
-        # ignored
-      end
-    end
-    
-    catch(:IRB_EXIT) do
-      irb.eval_input
-    end
-  ensure
-    trap("SIGINT", old_sigint)
-  end
-
-  class << self
-    alias :old_CurrentContext :CurrentContext
-    remove_method :CurrentContext
-    remove_method :parse_opts
-  end
-
-  def IRB.CurrentContext
-    if old_CurrentContext.nil? and Breakpoint.use_drb? then
-      result = Object.new
-      def result.last_value; end
-      return result
-    else
-      old_CurrentContext
-    end
-  end
-  def IRB.parse_opts() end
-
-  class Context # :nodoc:
-    alias :old_evaluate :evaluate
-    def evaluate(line, line_no)
-      if line.chomp == "exit" then
-        exit
-      else
-        old_evaluate(line, line_no)
-      end
-    end
-  end
-
-  class WorkSpace # :nodoc:
-    alias :old_evaluate :evaluate
-
-    def evaluate(*args)
-      if Breakpoint.use_drb? then
-        result = old_evaluate(*args)
-        if args[0] != :no_proxy and
-          not [true, false, nil].include?(result)
-        then
-          result.extend(DRbUndumped) rescue nil
-        end
-        return result
-      else
-        old_evaluate(*args)
-      end
-    end
-  end
-
-  module InputCompletor # :nodoc:
-    def self.eval(code, context, *more)
-      # Big hack, this assumes that InputCompletor
-      # will only call eval() when it wants code
-      # to be executed in the IRB context.
-      IRB.conf[:MAIN_CONTEXT].workspace.evaluate(:no_proxy, code, *more)
-    end
-  end
-end
-
-module DRb # :nodoc:
-  class DRbObject # :nodoc:
-    undef :inspect if method_defined?(:inspect)
-    undef :clone if method_defined?(:clone)
-  end
-end
-
-# See Breakpoint.breakpoint
-def breakpoint(id = nil, &block)
-  Binding.of_caller do |context|
-    Breakpoint.breakpoint(id, context, &block)
-  end
-end
-
-# See Breakpoint.assert
-def assert(&block)
-  Binding.of_caller do |context|
-    Breakpoint.assert(context, &block)
-  end
-end

test/extra/caller.rb

@@ -1,80 +0,0 @@
-class Continuation # :nodoc:
-  def self.create(*args, &block) # :nodoc:
-    cc = nil; result = callcc {|c| cc = c; block.call(cc) if block and args.empty?}
-    result ||= args
-    return *[cc, *result]
-  end
-end
-
-class Binding; end # for RDoc
-# This method returns the binding of the method that called your
-# method. It will raise an Exception when you're not inside a method.
-#
-# It's used like this:
-#   def inc_counter(amount = 1)
-#     Binding.of_caller do |binding|
-#       # Create a lambda that will increase the variable 'counter'
-#       # in the caller of this method when called.
-#       inc = eval("lambda { |arg| counter += arg }", binding)
-#       # We can refer to amount from inside this block safely.
-#       inc.call(amount)
-#     end
-#     # No other statements can go here. Put them inside the block.
-#   end
-#   counter = 0
-#   2.times { inc_counter }
-#   counter # => 2
-#
-# Binding.of_caller must be the last statement in the method.
-# This means that you will have to put everything you want to
-# do after the call to Binding.of_caller into the block of it.
-# This should be no problem however, because Ruby has closures.
-# If you don't do this an Exception will be raised. Because of
-# the way that Binding.of_caller is implemented it has to be
-# done this way.
-def Binding.of_caller(&block)
-  old_critical = Thread.critical
-  Thread.critical = true
-  count = 0
-  cc, result, error, extra_data = Continuation.create(nil, nil)
-  error.call if error
-
-  tracer = lambda do |*args|
-    type, context, extra_data = args[0], args[4], args
-    if type == "return"
-      count += 1
-      # First this method and then calling one will return --
-      # the trace event of the second event gets the context
-      # of the method which called the method that called this
-      # method.
-      if count == 2
-        # It would be nice if we could restore the trace_func
-        # that was set before we swapped in our own one, but
-        # this is impossible without overloading set_trace_func
-        # in current Ruby.
-        set_trace_func(nil)
-        cc.call(eval("binding", context), nil, extra_data)
-      end
-    elsif type == "line" then
-      nil
-    elsif type == "c-return" and extra_data[3] == :set_trace_func then
-      nil
-    else
-      set_trace_func(nil)
-      error_msg = "Binding.of_caller used in non-method context or " +
-        "trailing statements of method using it aren't in the block."
-      cc.call(nil, lambda { raise(ArgumentError, error_msg) }, nil)
-    end
-  end
-
-  unless result
-    set_trace_func(tracer)
-    return nil
-  else
-    Thread.critical = old_critical
-    case block.arity
-      when 1 then yield(result)
-      else yield(result, extra_data)        
-    end
-  end
-end

test/fixtures/en_locale.yml

@@ -0,0 +1,9 @@
+---
+  simple: "less is more"
+  whatever: "something %{something}"
+  errors:
+    i18n:
+      undefined_interpolation: "undefined key %{key}"
+      unknown_translation: "translation '%{name}' wasn't found"
+    syntax:
+      oops: "something wasn't right"

test/lib/liquid/drop_test.rb

@@ -1,153 +0,0 @@
-require 'test_helper'
-
-class ContextDrop < Liquid::Drop
-  def scopes
-    @context.scopes.size
-  end
-
-  def scopes_as_array
-    (1..@context.scopes.size).to_a
-  end
-
-  def loop_pos
-    @context['forloop.index']
-  end
-
-  def break
-    Breakpoint.breakpoint
-  end
-
-  def before_method(method)
-    return @context[method]
-  end
-end
-
-class ProductDrop < Liquid::Drop
-
-  class TextDrop < Liquid::Drop
-    def array
-      ['text1', 'text2']
-    end
-
-    def text
-      'text1'
-    end
-  end
-
-  class CatchallDrop < Liquid::Drop
-    def before_method(method)
-      return 'method: ' << method
-    end
-  end
-
-  def texts
-    TextDrop.new
-  end
-
-  def catchall
-    CatchallDrop.new
-  end
-
-  def context
-    ContextDrop.new
-  end
-
-  protected
-    def callmenot
-      "protected"
-    end
-end
-
-class EnumerableDrop < Liquid::Drop
-
-  def size
-    3
-  end
-
-  def each
-    yield 1
-    yield 2
-    yield 3
-  end
-end
-
-class DropsTest < Test::Unit::TestCase
-  include Liquid
-
-  def test_product_drop
-
-    assert_nothing_raised do
-      tpl = Liquid::Template.parse( '  '  )
-      tpl.render('product' => ProductDrop.new)
-    end
-  end
-
-  def test_text_drop
-    output = Liquid::Template.parse( ' {{ product.texts.text }} '  ).render('product' => ProductDrop.new)
-    assert_equal ' text1 ', output
-
-  end
-
-  def test_text_drop
-    output = Liquid::Template.parse( ' {{ product.catchall.unknown }} '  ).render('product' => ProductDrop.new)
-    assert_equal ' method: unknown ', output
-
-  end
-
-  def test_text_array_drop
-    output = Liquid::Template.parse( '{% for text in product.texts.array %} {{text}} {% endfor %}'  ).render('product' => ProductDrop.new)
-    assert_equal ' text1  text2 ', output
-  end
-
-  def test_context_drop
-    output = Liquid::Template.parse( ' {{ context.bar }} '  ).render('context' => ContextDrop.new, 'bar' => "carrot")
-    assert_equal ' carrot ', output
-  end
-
-  def test_nested_context_drop
-    output = Liquid::Template.parse( ' {{ product.context.foo }} '  ).render('product' => ProductDrop.new, 'foo' => "monkey")
-    assert_equal ' monkey ', output
-  end
-
-  def test_protected
-    output = Liquid::Template.parse( ' {{ product.callmenot }} '  ).render('product' => ProductDrop.new)
-    assert_equal '  ', output
-  end
-
-  def test_scope
-    assert_equal '1', Liquid::Template.parse( '{{ context.scopes }}'  ).render('context' => ContextDrop.new)
-    assert_equal '2', Liquid::Template.parse( '{%for i in dummy%}{{ context.scopes }}{%endfor%}'  ).render('context' => ContextDrop.new, 'dummy' => [1])
-    assert_equal '3', Liquid::Template.parse( '{%for i in dummy%}{%for i in dummy%}{{ context.scopes }}{%endfor%}{%endfor%}'  ).render('context' => ContextDrop.new, 'dummy' => [1])
-  end
-
-  def test_scope_though_proc
-    assert_equal '1', Liquid::Template.parse( '{{ s }}'  ).render('context' => ContextDrop.new, 's' => Proc.new{|c| c['context.scopes'] })
-    assert_equal '2', Liquid::Template.parse( '{%for i in dummy%}{{ s }}{%endfor%}'  ).render('context' => ContextDrop.new, 's' => Proc.new{|c| c['context.scopes'] }, 'dummy' => [1])
-    assert_equal '3', Liquid::Template.parse( '{%for i in dummy%}{%for i in dummy%}{{ s }}{%endfor%}{%endfor%}'  ).render('context' => ContextDrop.new, 's' => Proc.new{|c| c['context.scopes'] }, 'dummy' => [1])
-  end
-
-  def test_scope_with_assigns
-    assert_equal 'variable', Liquid::Template.parse( '{% assign a = "variable"%}{{a}}'  ).render('context' => ContextDrop.new)
-    assert_equal 'variable', Liquid::Template.parse( '{% assign a = "variable"%}{%for i in dummy%}{{a}}{%endfor%}'  ).render('context' => ContextDrop.new, 'dummy' => [1])
-    assert_equal 'test', Liquid::Template.parse( '{% assign header_gif = "test"%}{{header_gif}}'  ).render('context' => ContextDrop.new)
-    assert_equal 'test', Liquid::Template.parse( "{% assign header_gif = 'test'%}{{header_gif}}"  ).render('context' => ContextDrop.new)
-  end
-
-  def test_scope_from_tags
-    assert_equal '1', Liquid::Template.parse( '{% for i in context.scopes_as_array %}{{i}}{% endfor %}'  ).render('context' => ContextDrop.new, 'dummy' => [1])
-    assert_equal '12', Liquid::Template.parse( '{%for a in dummy%}{% for i in context.scopes_as_array %}{{i}}{% endfor %}{% endfor %}'  ).render('context' => ContextDrop.new, 'dummy' => [1])
-    assert_equal '123', Liquid::Template.parse( '{%for a in dummy%}{%for a in dummy%}{% for i in context.scopes_as_array %}{{i}}{% endfor %}{% endfor %}{% endfor %}'  ).render('context' => ContextDrop.new, 'dummy' => [1])
-  end
-
-  def test_access_context_from_drop
-    assert_equal '123', Liquid::Template.parse( '{%for a in dummy%}{{ context.loop_pos }}{% endfor %}'  ).render('context' => ContextDrop.new, 'dummy' => [1,2,3])
-  end
-
-  def test_enumerable_drop
-    assert_equal '123', Liquid::Template.parse( '{% for c in collection %}{{c}}{% endfor %}').render('collection' => EnumerableDrop.new)
-  end
-
-  def test_enumerable_drop_size
-    assert_equal '3', Liquid::Template.parse( '{{collection.size}}').render('collection' => EnumerableDrop.new)
-  end
-end # DropsTest

test/lib/liquid/html_tag_test.rb

@@ -1,29 +0,0 @@
-require 'test_helper'
-
-class HtmlTagTest < Test::Unit::TestCase
-  include Liquid
-
-  def test_html_table
-
-    assert_template_result("<tr class=\"row1\">\n<td class=\"col1\"> 1 </td><td class=\"col2\"> 2 </td><td class=\"col3\"> 3 </td></tr>\n<tr class=\"row2\"><td class=\"col1\"> 4 </td><td class=\"col2\"> 5 </td><td class=\"col3\"> 6 </td></tr>\n",
-                           '{% tablerow n in numbers cols:3%} {{n}} {% endtablerow %}',
-                           'numbers' => [1,2,3,4,5,6])
-
-    assert_template_result("<tr class=\"row1\">\n</tr>\n",
-                           '{% tablerow n in numbers cols:3%} {{n}} {% endtablerow %}',
-                           'numbers' => [])
-  end
-
-  def test_html_table_with_different_cols
-    assert_template_result("<tr class=\"row1\">\n<td class=\"col1\"> 1 </td><td class=\"col2\"> 2 </td><td class=\"col3\"> 3 </td><td class=\"col4\"> 4 </td><td class=\"col5\"> 5 </td></tr>\n<tr class=\"row2\"><td class=\"col1\"> 6 </td></tr>\n",
-                           '{% tablerow n in numbers cols:5%} {{n}} {% endtablerow %}',
-                           'numbers' => [1,2,3,4,5,6])
-
-  end
-
-  def test_html_col_counter
-    assert_template_result("<tr class=\"row1\">\n<td class=\"col1\">1</td><td class=\"col2\">2</td></tr>\n<tr class=\"row2\"><td class=\"col1\">1</td><td class=\"col2\">2</td></tr>\n<tr class=\"row3\"><td class=\"col1\">1</td><td class=\"col2\">2</td></tr>\n",
-                           '{% tablerow n in numbers cols:2%}{{tablerowloop.col}}{% endtablerow %}',
-                           'numbers' => [1,2,3,4,5,6])
-  end
-end # HtmlTagTest

test/lib/liquid/include_tag_test.rb

@@ -1,127 +0,0 @@
-require 'test_helper'
-
-class TestFileSystem
-  def read_template_file(template_path)
-    case template_path
-    when "product"
-      "Product: {{ product.title }} "
-
-    when "locale_variables"
-      "Locale: {{echo1}} {{echo2}}"
-
-    when "variant"
-      "Variant: {{ variant.title }}"
-
-    when "nested_template"
-      "{% include 'header' %} {% include 'body' %} {% include 'footer' %}"
-
-    when "body"
-      "body {% include 'body_detail' %}"
-
-    when "nested_product_template"
-      "Product: {{ nested_product_template.title }} {%include 'details'%} "
-
-    when "recursively_nested_template"
-      "-{% include 'recursively_nested_template' %}"
-
-    when "pick_a_source"
-      "from TestFileSystem"
-
-    else
-      template_path
-    end
-  end
-end
-
-class OtherFileSystem
-  def read_template_file(template_path)
-    'from OtherFileSystem'
-  end
-end
-
-class IncludeTagTest < Test::Unit::TestCase
-  include Liquid
-
-  def setup
-    Liquid::Template.file_system = TestFileSystem.new
-  end
-
-  def test_include_tag_looks_for_file_system_in_registers_first
-    assert_equal 'from OtherFileSystem',
-      Template.parse("{% include 'pick_a_source' %}").render({}, :registers => {:file_system => OtherFileSystem.new})
-  end
-
-
-  def test_include_tag_with
-    assert_equal "Product: Draft 151cm ",
-      Template.parse("{% include 'product' with products[0] %}").render( "products" => [ {'title' => 'Draft 151cm'}, {'title' => 'Element 155cm'} ]  )
-  end
-
-  def test_include_tag_with_default_name
-    assert_equal "Product: Draft 151cm ",
-      Template.parse("{% include 'product' %}").render( "product" => {'title' => 'Draft 151cm'}  )
-  end
-
-  def test_include_tag_for
-
-    assert_equal "Product: Draft 151cm Product: Element 155cm ",
-      Template.parse("{% include 'product' for products %}").render( "products" => [ {'title' => 'Draft 151cm'}, {'title' => 'Element 155cm'} ]  )
-  end
-
-  def test_include_tag_with_local_variables
-    assert_equal "Locale: test123 ",
-      Template.parse("{% include 'locale_variables' echo1: 'test123' %}").render
-  end
-
-  def test_include_tag_with_multiple_local_variables
-    assert_equal "Locale: test123 test321",
-      Template.parse("{% include 'locale_variables' echo1: 'test123', echo2: 'test321' %}").render
-  end
-
-  def test_include_tag_with_multiple_local_variables_from_context
-    assert_equal "Locale: test123 test321",
-      Template.parse("{% include 'locale_variables' echo1: echo1, echo2: more_echos.echo2 %}").render('echo1' => 'test123', 'more_echos' => { "echo2" => 'test321'})
-  end
-
-  def test_nested_include_tag
-    assert_equal "body body_detail",
-      Template.parse("{% include 'body' %}").render
-
-    assert_equal "header body body_detail footer",
-      Template.parse("{% include 'nested_template' %}").render
-  end
-
-  def test_nested_include_with_variable
-
-    assert_equal "Product: Draft 151cm details ",
-      Template.parse("{% include 'nested_product_template' with product %}").render("product" => {"title" => 'Draft 151cm'})
-
-    assert_equal "Product: Draft 151cm details Product: Element 155cm details ",
-      Template.parse("{% include 'nested_product_template' for products %}").render("products" => [{"title" => 'Draft 151cm'}, {"title" => 'Element 155cm'}])
-
-  end
-
-  def test_recursively_included_template_does_not_produce_endless_loop
-
-    infinite_file_system = Class.new do
-      def read_template_file(template_path)
-        "-{% include 'loop' %}"
-      end
-    end
-
-    Liquid::Template.file_system = infinite_file_system.new
-
-    assert_raise(Liquid::StackLevelError) do
-      Template.parse("{% include 'loop' %}").render!
-    end
-
-  end
-
-  def test_dynamically_choosen_template
-
-    assert_equal "Test123", Template.parse("{% include template %}").render("template" => 'Test123')
-    assert_equal "Test321", Template.parse("{% include template %}").render("template" => 'Test321')
-
-    assert_equal "Product: Draft 151cm ", Template.parse("{% include template for product %}").render("template" => 'product', 'product' => { 'title' => 'Draft 151cm'})
-  end
-end # IncludeTagTest

test/lib/liquid/parsing_quirks_test.rb

@@ -1,52 +0,0 @@
-require 'test_helper'
-
-class ParsingQuirksTest < Test::Unit::TestCase
-  include Liquid
-
-  def test_error_with_css
-    text = %| div { font-weight: bold; } |
-    template = Template.parse(text)
-
-    assert_equal text, template.render
-    assert_equal [String], template.root.nodelist.collect {|i| i.class}
-  end
-
-  def test_raise_on_single_close_bracet
-    assert_raise(SyntaxError) do
-      Template.parse("text {{method} oh nos!")
-    end
-  end
-
-  def test_raise_on_label_and_no_close_bracets
-    assert_raise(SyntaxError) do
-      Template.parse("TEST {{ ")
-    end
-  end
-
-  def test_raise_on_label_and_no_close_bracets_percent
-    assert_raise(SyntaxError) do
-      Template.parse("TEST {% ")
-    end
-  end
-
-  def test_error_on_empty_filter
-    assert_nothing_raised do
-      Template.parse("{{test |a|b|}}")
-      Template.parse("{{test}}")
-      Template.parse("{{|test|}}")
-    end
-  end
-
-  def test_meaningless_parens
-    assigns = {'b' => 'bar', 'c' => 'baz'}
-    markup = "a == 'foo' or (b == 'bar' and c == 'baz') or false"
-    assert_template_result(' YES ',"{% if #{markup} %} YES {% endif %}", assigns)
-  end
-
-  def test_unexpected_characters_silently_eat_logic
-    markup = "true && false"
-    assert_template_result(' YES ',"{% if #{markup} %} YES {% endif %}")
-    markup = "false || true"
-    assert_template_result('',"{% if #{markup} %} YES {% endif %}")
-  end
-end # ParsingQuirksTest

test/lib/liquid/security_test.rb

@@ -1,41 +0,0 @@
-require 'test_helper'
-
-module SecurityFilter
-  def add_one(input)
-    "#{input} + 1"
-  end
-end
-
-class SecurityTest < Test::Unit::TestCase
-  include Liquid
-
-  def test_no_instance_eval
-    text = %( {{ '1+1' | instance_eval }} )
-    expected = %| 1+1 |
-
-    assert_equal expected, Template.parse(text).render(@assigns)
-  end
-
-  def test_no_existing_instance_eval
-    text = %( {{ '1+1' | __instance_eval__ }} )
-    expected = %| 1+1 |
-
-    assert_equal expected, Template.parse(text).render(@assigns)
-  end
-
-
-  def test_no_instance_eval_after_mixing_in_new_filter
-    text = %( {{ '1+1' | instance_eval }} )
-    expected = %| 1+1 |
-
-    assert_equal expected, Template.parse(text).render(@assigns)
-  end
-
-
-  def test_no_instance_eval_later_in_chain
-    text = %( {{ '1+1' | add_one | instance_eval }} )
-    expected = %| 1+1 + 1 |
-
-    assert_equal expected, Template.parse(text).render(@assigns, :filters => SecurityFilter)
-  end
-end # SecurityTest

test/lib/liquid/strainer_test.rb

@@ -1,25 +0,0 @@
-require 'test_helper'
-
-class StrainerTest < Test::Unit::TestCase
-  include Liquid
-
-  def test_strainer
-    strainer = Strainer.create(nil)
-    assert_equal false, strainer.respond_to?('__test__')
-    assert_equal false, strainer.respond_to?('test')
-    assert_equal false, strainer.respond_to?('instance_eval')
-    assert_equal false, strainer.respond_to?('__send__')
-    assert_equal true, strainer.respond_to?('size') # from the standard lib
-  end
-
-  def test_should_respond_to_two_parameters
-    strainer = Strainer.create(nil)
-    assert_equal true, strainer.respond_to?('size', false)
-  end
-
-  # Asserts that Object#respond_to_missing? is not being undefined in Ruby versions where it has been implemented
-  # Currently this method is only present in Ruby v1.9.2, or higher
-  def test_object_respond_to_missing
-    assert_equal Object.respond_to?(:respond_to_missing?), Strainer.create(nil).respond_to?(:respond_to_missing?)
-  end
-end # StrainerTest

test/lib/liquid/tags/literal_test.rb

@@ -1,39 +0,0 @@
-require 'test_helper'
-
-class LiteralTagTest < Test::Unit::TestCase
-  include Liquid
-
-  def test_empty_literal
-    assert_template_result '', '{% literal %}{% endliteral %}'
-    assert_template_result '', '{{{}}}'
-  end
-
-  def test_simple_literal_value
-    assert_template_result 'howdy',
-                           '{% literal %}howdy{% endliteral %}'
-  end
-
-  def test_literals_ignore_liquid_markup
-    expected = %({% if 'gnomeslab' contain 'liquid' %}yes{ % endif %})
-    template = %({% literal %}#{expected}{% endliteral %})
-
-    assert_template_result expected, template
-  end
-
-  def test_shorthand_syntax
-    expected = %({% if 'gnomeslab' contain 'liquid' %}yes{ % endif %})
-    template = %({{{#{expected}}}})
-
-    assert_template_result expected, template
-  end
-
-  # Class methods
-  def test_from_shorthand
-    assert_equal '{% literal %}gnomeslab{% endliteral %}', Liquid::Literal.from_shorthand('{{{gnomeslab}}}')
-  end
-
-  def test_from_shorthand_ignores_improper_syntax
-    text = "{% if 'hi' == 'hi' %}hi{% endif %}"
-    assert_equal text, Liquid::Literal.from_shorthand(text)
-  end
-end # AssignTest

test/lib/liquid/tags/statements_test.rb

@@ -1,134 +0,0 @@
-require 'test_helper'
-
-class StatementsTest < Test::Unit::TestCase
-  include Liquid
-
-  def test_true_eql_true
-    text = %| {% if true == true %} true {% else %} false {% endif %} |
-    expected = %|  true  |
-    assert_equal expected, Template.parse(text).render
-  end
-
-  def test_true_not_eql_true
-    text = %| {% if true != true %} true {% else %} false {% endif %} |
-    expected = %|  false  |
-    assert_equal expected, Template.parse(text).render
-  end
-
-  def test_true_lq_true
-    text = %| {% if 0 > 0 %} true {% else %} false {% endif %} |
-    expected = %|  false  |
-    assert_equal expected, Template.parse(text).render
-  end
-
-  def test_one_lq_zero
-    text = %| {% if 1 > 0 %} true {% else %} false {% endif %} |
-    expected = %|  true  |
-    assert_equal expected, Template.parse(text).render
-  end
-
-  def test_zero_lq_one
-    text = %| {% if 0 < 1 %} true {% else %} false {% endif %} |
-    expected = %|  true  |
-    assert_equal expected, Template.parse(text).render
-  end
-
-  def test_zero_lq_or_equal_one
-    text = %| {% if 0 <= 0 %} true {% else %} false {% endif %} |
-    expected = %|  true  |
-    assert_equal expected, Template.parse(text).render
-  end
-
-  def test_zero_lq_or_equal_one_involving_nil
-    text = %| {% if null <= 0 %} true {% else %} false {% endif %} |
-    expected = %|  false  |
-    assert_equal expected, Template.parse(text).render
-
-
-    text = %| {% if 0 <= null %} true {% else %} false {% endif %} |
-    expected = %|  false  |
-    assert_equal expected, Template.parse(text).render
-  end
-
-  def test_zero_lqq_or_equal_one
-    text = %| {% if 0 >= 0 %} true {% else %} false {% endif %} |
-    expected = %|  true  |
-    assert_equal expected, Template.parse(text).render
-  end
-
-  def test_strings
-    text = %| {% if 'test' == 'test' %} true {% else %} false {% endif %} |
-    expected = %|  true  |
-    assert_equal expected, Template.parse(text).render
-  end
-
-  def test_strings_not_equal
-    text = %| {% if 'test' != 'test' %} true {% else %} false {% endif %} |
-    expected = %|  false  |
-    assert_equal expected, Template.parse(text).render
-  end
-
-  def test_var_strings_equal
-    text = %| {% if var == "hello there!" %} true {% else %} false {% endif %} |
-    expected = %|  true  |
-    assert_equal expected, Template.parse(text).render('var' => 'hello there!')
-  end
-
-  def test_var_strings_are_not_equal
-    text = %| {% if "hello there!" == var %} true {% else %} false {% endif %} |
-    expected = %|  true  |
-    assert_equal expected, Template.parse(text).render('var' => 'hello there!')
-  end
-
-  def test_var_and_long_string_are_equal
-    text = %| {% if var == 'hello there!' %} true {% else %} false {% endif %} |
-    expected = %|  true  |
-    assert_equal expected, Template.parse(text).render('var' => 'hello there!')
-  end
-
-
-  def test_var_and_long_string_are_equal_backwards
-    text = %| {% if 'hello there!' == var %} true {% else %} false {% endif %} |
-    expected = %|  true  |
-    assert_equal expected, Template.parse(text).render('var' => 'hello there!')
-  end
-
-  #def test_is_nil
-  #  text = %| {% if var != nil %} true {% else %} false {% end %} |
-  #  @template.assigns = { 'var' => 'hello there!'}
-  #  expected = %|  true  |
-  #  assert_equal expected, @template.parse(text)
-  #end
-
-  def test_is_collection_empty
-    text = %| {% if array == empty %} true {% else %} false {% endif %} |
-    expected = %|  true  |
-    assert_equal expected, Template.parse(text).render('array' => [])
-  end
-
-  def test_is_not_collection_empty
-    text = %| {% if array == empty %} true {% else %} false {% endif %} |
-    expected = %|  false  |
-    assert_equal expected, Template.parse(text).render('array' => [1,2,3])
-  end
-
-  def test_nil
-    text = %| {% if var == nil %} true {% else %} false {% endif %} |
-    expected = %|  true  |
-    assert_equal expected, Template.parse(text).render('var' => nil)
-
-    text = %| {% if var == null %} true {% else %} false {% endif %} |
-    expected = %|  true  |
-    assert_equal expected, Template.parse(text).render('var' => nil)
-  end
-
-  def test_not_nil
-    text = %| {% if var != nil %} true {% else %} false {% endif %} |
-    expected = %|  true  |
-    assert_equal expected, Template.parse(text).render('var' => 1 )
-
-    text = %| {% if var != null %} true {% else %} false {% endif %} |
-    expected = %|  true  |
-    assert_equal expected, Template.parse(text).render('var' => 1 )
-  end
-end # StatementsTest

test/lib/liquid/template_test.rb

@@ -1,74 +0,0 @@
-require 'test_helper'
-
-class TemplateTest < Test::Unit::TestCase
-  include Liquid
-
-  def test_tokenize_strings
-    assert_equal [' '], Template.new.send(:tokenize, ' ')
-    assert_equal ['hello world'], Template.new.send(:tokenize, 'hello world')
-  end
-
-  def test_tokenize_variables
-    assert_equal ['{{funk}}'], Template.new.send(:tokenize, '{{funk}}')
-    assert_equal [' ', '{{funk}}', ' '], Template.new.send(:tokenize, ' {{funk}} ')
-    assert_equal [' ', '{{funk}}', ' ', '{{so}}', ' ', '{{brother}}', ' '], Template.new.send(:tokenize, ' {{funk}} {{so}} {{brother}} ')
-    assert_equal [' ', '{{  funk  }}', ' '], Template.new.send(:tokenize, ' {{  funk  }} ')
-  end
-
-  def test_tokenize_blocks
-    assert_equal ['{%comment%}'], Template.new.send(:tokenize, '{%comment%}')
-    assert_equal [' ', '{%comment%}', ' '], Template.new.send(:tokenize, ' {%comment%} ')
-
-    assert_equal [' ', '{%comment%}', ' ', '{%endcomment%}', ' '], Template.new.send(:tokenize, ' {%comment%} {%endcomment%} ')
-    assert_equal ['  ', '{% comment %}', ' ', '{% endcomment %}', ' '], Template.new.send(:tokenize, "  {% comment %} {% endcomment %} ")
-  end
-
-  def test_instance_assigns_persist_on_same_template_object_between_parses
-    t = Template.new
-    assert_equal 'from instance assigns', t.parse("{% assign foo = 'from instance assigns' %}{{ foo }}").render
-    assert_equal 'from instance assigns', t.parse("{{ foo }}").render
-  end
-
-  def test_instance_assigns_persist_on_same_template_parsing_between_renders
-    t = Template.new.parse("{{ foo }}{% assign foo = 'foo' %}{{ foo }}")
-    assert_equal 'foo', t.render
-    assert_equal 'foofoo', t.render
-  end
-
-  def test_custom_assigns_do_not_persist_on_same_template
-    t = Template.new
-    assert_equal 'from custom assigns', t.parse("{{ foo }}").render('foo' => 'from custom assigns')
-    assert_equal '', t.parse("{{ foo }}").render
-  end
-
-  def test_custom_assigns_squash_instance_assigns
-    t = Template.new
-    assert_equal 'from instance assigns', t.parse("{% assign foo = 'from instance assigns' %}{{ foo }}").render
-    assert_equal 'from custom assigns', t.parse("{{ foo }}").render('foo' => 'from custom assigns')
-  end
-
-  def test_persistent_assigns_squash_instance_assigns
-    t = Template.new
-    assert_equal 'from instance assigns', t.parse("{% assign foo = 'from instance assigns' %}{{ foo }}").render
-    t.assigns['foo'] = 'from persistent assigns'
-    assert_equal 'from persistent assigns', t.parse("{{ foo }}").render
-  end
-
-  def test_lambda_is_called_once_from_persistent_assigns_over_multiple_parses_and_renders
-    t = Template.new
-    t.assigns['number'] = lambda { @global ||= 0; @global += 1 }
-    assert_equal '1', t.parse("{{number}}").render
-    assert_equal '1', t.parse("{{number}}").render
-    assert_equal '1', t.render
-    @global = nil
-  end
-
-  def test_lambda_is_called_once_from_custom_assigns_over_multiple_parses_and_renders
-    t = Template.new
-    assigns = {'number' => lambda { @global ||= 0; @global += 1 }}
-    assert_equal '1', t.parse("{{number}}").render(assigns)
-    assert_equal '1', t.parse("{{number}}").render(assigns)
-    assert_equal '1', t.render(assigns)
-    @global = nil
-  end
-end # TemplateTest

test/liquid/assign_test.rb

@@ -12,4 +12,16 @@ class AssignTest < Test::Unit::TestCase
                            '{% assign foo = values %}.{{ foo[1] }}.',
                            'values' => %w{foo bar baz})
   end
+
+  def test_assign_with_filter
+    assert_template_result('.bar.',
+                           '{% assign foo = values | split: "," %}.{{ foo[1] }}.',
+                           'values' => "foo,bar,baz")
+  end
+
+  def test_assign_syntax_error
+    assert_match_syntax_error(/assign/,
+                       '{% assign foo not values %}.',
+                       'values' => "foo,bar,baz")
+  end
 end # AssignTest

test/liquid/blank_test.rb

@@ -0,0 +1,106 @@
+require 'test_helper'
+
+class FoobarTag < Liquid::Tag
+  def render(*args)
+    " "
+  end
+
+  Liquid::Template.register_tag('foobar', FoobarTag)
+end
+
+class BlankTestFileSystem
+  def read_template_file(template_path, context)
+    template_path
+  end
+end
+
+class BlankTest < Test::Unit::TestCase
+  include Liquid
+  N = 10
+
+  def wrap_in_for(body)
+    "{% for i in (1..#{N}) %}#{body}{% endfor %}"
+  end
+
+  def wrap_in_if(body)
+    "{% if true %}#{body}{% endif %}"
+  end
+
+  def wrap(body)
+    wrap_in_for(body) + wrap_in_if(body)
+  end
+
+  def test_new_tags_are_not_blank_by_default
+    assert_template_result(" "*N, wrap_in_for("{% foobar %}"))
+  end
+
+  def test_loops_are_blank
+    assert_template_result("", wrap_in_for(" "))
+  end
+
+  def test_if_else_are_blank
+    assert_template_result("", "{% if true %} {% elsif false %} {% else %} {% endif %}")
+  end
+
+  def test_unless_is_blank
+    assert_template_result("", wrap("{% unless true %} {% endunless %}"))
+  end
+
+  def test_mark_as_blank_only_during_parsing
+    assert_template_result(" "*(N+1), wrap(" {% if false %} this never happens, but still, this block is not blank {% endif %}"))
+  end
+
+  def test_comments_are_blank
+    assert_template_result("", wrap(" {% comment %} whatever {% endcomment %} "))
+  end
+
+  def test_captures_are_blank
+    assert_template_result("", wrap(" {% capture foo %} whatever {% endcapture %} "))
+  end
+
+  def test_nested_blocks_are_blank_but_only_if_all_children_are
+    assert_template_result("", wrap(wrap(" ")))
+    assert_template_result("\n       but this is not "*(N+1),
+      wrap(%q{{% if true %} {% comment %} this is blank {% endcomment %} {% endif %}
+      {% if true %} but this is not {% endif %}}))
+  end
+
+  def test_assigns_are_blank
+    assert_template_result("", wrap(' {% assign foo = "bar" %} '))
+  end
+
+  def test_whitespace_is_blank
+    assert_template_result("", wrap(" "))
+    assert_template_result("", wrap("\t"))
+  end
+
+  def test_whitespace_is_not_blank_if_other_stuff_is_present
+    body = "     x "
+    assert_template_result(body*(N+1), wrap(body))
+  end
+
+  def test_increment_is_not_blank
+    assert_template_result(" 0"*2*(N+1), wrap("{% assign foo = 0 %} {% increment foo %} {% decrement foo %}"))
+  end
+
+  def test_cycle_is_not_blank
+    assert_template_result("  "*((N+1)/2)+" ", wrap("{% cycle ' ', ' ' %}"))
+  end
+
+  def test_raw_is_not_blank
+    assert_template_result("  "*(N+1), wrap(" {% raw %} {% endraw %}"))
+  end
+
+  def test_include_is_blank
+    Liquid::Template.file_system = BlankTestFileSystem.new
+    assert_template_result "foobar"*(N+1), wrap("{% include 'foobar' %}")
+    assert_template_result " foobar "*(N+1), wrap("{% include ' foobar ' %}")
+    assert_template_result "   "*(N+1), wrap(" {% include ' ' %} ")
+  end
+
+  def test_case_is_blank
+    assert_template_result("", wrap(" {% assign foo = 'bar' %} {% case foo %} {% when 'bar' %} {% when 'whatever' %} {% else %} {% endcase %} "))
+    assert_template_result("", wrap(" {% assign foo = 'else' %} {% case foo %} {% when 'bar' %} {% when 'whatever' %} {% else %} {% endcase %} "))
+    assert_template_result("   x  "*(N+1), wrap(" {% assign foo = 'else' %} {% case foo %} {% when 'bar' %} {% when 'whatever' %} {% else %} x {% endcase %} "))
+  end
+end

test/liquid/block_test.rb

@@ -1,6 +1,6 @@
 require 'test_helper'
 
-class VariableTest < Test::Unit::TestCase
+class BlockTest < Test::Unit::TestCase
   include Liquid
 
   def test_blankspace

test/liquid/capture_test.rb

@@ -19,7 +19,7 @@ class CaptureTest < Test::Unit::TestCase
     {{var}}
     END_TEMPLATE
     template = Template.parse(template_source)
-    rendered = template.render
+    rendered = template.render!
     assert_equal "test-string", rendered.gsub(/\s/, '')
   end
 
@@ -34,7 +34,7 @@ class CaptureTest < Test::Unit::TestCase
     {{ first }}-{{ second }}
     END_TEMPLATE
     template = Template.parse(template_source)
-    rendered = template.render
+    rendered = template.render!
     assert_equal "3-3", rendered.gsub(/\s/, '')
   end
 end # CaptureTest

test/liquid/condition_test.rb

@@ -18,6 +18,11 @@ class ConditionTest < Test::Unit::TestCase
     assert_evalutes_true '2', '>=', '1'
     assert_evalutes_true '1', '<=', '2'
     assert_evalutes_true '1', '<=', '1'
+    # negative numbers
+    assert_evalutes_true '1', '>', '-1'
+    assert_evalutes_true '-1', '<', '1'
+    assert_evalutes_true '1.0', '>', '-1.0'
+    assert_evalutes_true '-1.0', '<', '1.0'
   end
 
   def test_default_operators_evalute_false

test/liquid/context_test.rb

@@ -176,8 +176,8 @@ class ContextTest < Test::Unit::TestCase
     end
 
     Template.register_filter(global)
-    assert_equal 'Global test', Template.parse("{{'test' | notice }}").render
+    assert_equal 'Global test', Template.parse("{{'test' | notice }}").render!
-    assert_equal 'Local test', Template.parse("{{'test' | notice }}").render({}, :filters => [local])
+    assert_equal 'Local test', Template.parse("{{'test' | notice }}").render!({}, :filters => [local])
   end
 
   def test_only_intended_filters_make_it_there
@@ -189,10 +189,10 @@ class ContextTest < Test::Unit::TestCase
     end
 
     context = Context.new
-    methods_before = context.strainer.methods.map { |method| method.to_s }
+    assert_equal "Wookie", context.invoke("hi", "Wookie")
+
     context.add_filters(filter)
-    methods_after = context.strainer.methods.map { |method| method.to_s }
+    assert_equal "Wookie hi!", context.invoke("hi", "Wookie")
-    assert_equal (methods_before + ["hi"]).sort, methods_after.sort
   end
 
   def test_add_item_in_outer_scope

test/liquid/drop_test.rb

@@ -0,0 +1,245 @@
+require 'test_helper'
+
+class ContextDrop < Liquid::Drop
+  def scopes
+    @context.scopes.size
+  end
+
+  def scopes_as_array
+    (1..@context.scopes.size).to_a
+  end
+
+  def loop_pos
+    @context['forloop.index']
+  end
+
+  def before_method(method)
+    return @context[method]
+  end
+end
+
+class ProductDrop < Liquid::Drop
+
+  class TextDrop < Liquid::Drop
+    def array
+      ['text1', 'text2']
+    end
+
+    def text
+      'text1'
+    end
+  end
+
+  class CatchallDrop < Liquid::Drop
+    def before_method(method)
+      return 'method: ' << method.to_s
+    end
+  end
+
+  def texts
+    TextDrop.new
+  end
+
+  def catchall
+    CatchallDrop.new
+  end
+
+  def context
+    ContextDrop.new
+  end
+
+  protected
+    def callmenot
+      "protected"
+    end
+end
+
+class EnumerableDrop < Liquid::Drop
+  def before_method(method)
+    method
+  end
+
+  def size
+    3
+  end
+
+  def first
+    1
+  end
+
+  def count
+    3
+  end
+
+  def min
+    1
+  end
+
+  def max
+    3
+  end
+
+  def each
+    yield 1
+    yield 2
+    yield 3
+  end
+end
+
+class RealEnumerableDrop < Liquid::Drop
+  include Enumerable
+
+  def before_method(method)
+    method
+  end
+
+  def each
+    yield 1
+    yield 2
+    yield 3
+  end
+end
+
+class DropsTest < Test::Unit::TestCase
+  include Liquid
+
+  def test_product_drop
+    assert_nothing_raised do
+      tpl = Liquid::Template.parse( '  '  )
+      tpl.render!('product' => ProductDrop.new)
+    end
+  end
+
+  def test_drop_does_only_respond_to_whitelisted_methods
+    assert_equal "", Liquid::Template.parse("{{ product.inspect }}").render!('product' => ProductDrop.new)
+    assert_equal "", Liquid::Template.parse("{{ product.pretty_inspect }}").render!('product' => ProductDrop.new)
+    assert_equal "", Liquid::Template.parse("{{ product.whatever }}").render!('product' => ProductDrop.new)
+    assert_equal "", Liquid::Template.parse('{{ product | map: "inspect" }}').render!('product' => ProductDrop.new)
+    assert_equal "", Liquid::Template.parse('{{ product | map: "pretty_inspect" }}').render!('product' => ProductDrop.new)
+    assert_equal "", Liquid::Template.parse('{{ product | map: "whatever" }}').render!('product' => ProductDrop.new)
+  end
+
+  def test_drops_respond_to_to_liquid
+    assert_equal "text1", Liquid::Template.parse("{{ product.to_liquid.texts.text }}").render!('product' => ProductDrop.new)
+    assert_equal "text1", Liquid::Template.parse('{{ product | map: "to_liquid" | map: "texts" | map: "text" }}').render!('product' => ProductDrop.new)
+  end
+
+  def test_text_drop
+    output = Liquid::Template.parse( ' {{ product.texts.text }} '  ).render!('product' => ProductDrop.new)
+    assert_equal ' text1 ', output
+  end
+
+  def test_unknown_method
+    output = Liquid::Template.parse( ' {{ product.catchall.unknown }} '  ).render!('product' => ProductDrop.new)
+    assert_equal ' method: unknown ', output
+  end
+
+  def test_integer_argument_drop
+    output = Liquid::Template.parse( ' {{ product.catchall[8] }} '  ).render!('product' => ProductDrop.new)
+    assert_equal ' method: 8 ', output
+  end
+
+  def test_text_array_drop
+    output = Liquid::Template.parse( '{% for text in product.texts.array %} {{text}} {% endfor %}'  ).render!('product' => ProductDrop.new)
+    assert_equal ' text1  text2 ', output
+  end
+
+  def test_context_drop
+    output = Liquid::Template.parse( ' {{ context.bar }} '  ).render!('context' => ContextDrop.new, 'bar' => "carrot")
+    assert_equal ' carrot ', output
+  end
+
+  def test_nested_context_drop
+    output = Liquid::Template.parse( ' {{ product.context.foo }} '  ).render!('product' => ProductDrop.new, 'foo' => "monkey")
+    assert_equal ' monkey ', output
+  end
+
+  def test_protected
+    output = Liquid::Template.parse( ' {{ product.callmenot }} '  ).render!('product' => ProductDrop.new)
+    assert_equal '  ', output
+  end
+
+  def test_object_methods_not_allowed
+    [:dup, :clone, :singleton_class, :eval, :class_eval, :inspect].each do |method|
+      output = Liquid::Template.parse(" {{ product.#{method} }} ").render!('product' => ProductDrop.new)
+      assert_equal '  ', output
+    end
+  end
+
+  def test_scope
+    assert_equal '1', Liquid::Template.parse( '{{ context.scopes }}'  ).render!('context' => ContextDrop.new)
+    assert_equal '2', Liquid::Template.parse( '{%for i in dummy%}{{ context.scopes }}{%endfor%}'  ).render!('context' => ContextDrop.new, 'dummy' => [1])
+    assert_equal '3', Liquid::Template.parse( '{%for i in dummy%}{%for i in dummy%}{{ context.scopes }}{%endfor%}{%endfor%}'  ).render!('context' => ContextDrop.new, 'dummy' => [1])
+  end
+
+  def test_scope_though_proc
+    assert_equal '1', Liquid::Template.parse( '{{ s }}'  ).render!('context' => ContextDrop.new, 's' => Proc.new{|c| c['context.scopes'] })
+    assert_equal '2', Liquid::Template.parse( '{%for i in dummy%}{{ s }}{%endfor%}'  ).render!('context' => ContextDrop.new, 's' => Proc.new{|c| c['context.scopes'] }, 'dummy' => [1])
+    assert_equal '3', Liquid::Template.parse( '{%for i in dummy%}{%for i in dummy%}{{ s }}{%endfor%}{%endfor%}'  ).render!('context' => ContextDrop.new, 's' => Proc.new{|c| c['context.scopes'] }, 'dummy' => [1])
+  end
+
+  def test_scope_with_assigns
+    assert_equal 'variable', Liquid::Template.parse( '{% assign a = "variable"%}{{a}}'  ).render!('context' => ContextDrop.new)
+    assert_equal 'variable', Liquid::Template.parse( '{% assign a = "variable"%}{%for i in dummy%}{{a}}{%endfor%}'  ).render!('context' => ContextDrop.new, 'dummy' => [1])
+    assert_equal 'test', Liquid::Template.parse( '{% assign header_gif = "test"%}{{header_gif}}'  ).render!('context' => ContextDrop.new)
+    assert_equal 'test', Liquid::Template.parse( "{% assign header_gif = 'test'%}{{header_gif}}"  ).render!('context' => ContextDrop.new)
+  end
+
+  def test_scope_from_tags
+    assert_equal '1', Liquid::Template.parse( '{% for i in context.scopes_as_array %}{{i}}{% endfor %}'  ).render!('context' => ContextDrop.new, 'dummy' => [1])
+    assert_equal '12', Liquid::Template.parse( '{%for a in dummy%}{% for i in context.scopes_as_array %}{{i}}{% endfor %}{% endfor %}'  ).render!('context' => ContextDrop.new, 'dummy' => [1])
+    assert_equal '123', Liquid::Template.parse( '{%for a in dummy%}{%for a in dummy%}{% for i in context.scopes_as_array %}{{i}}{% endfor %}{% endfor %}{% endfor %}'  ).render!('context' => ContextDrop.new, 'dummy' => [1])
+  end
+
+  def test_access_context_from_drop
+    assert_equal '123', Liquid::Template.parse( '{%for a in dummy%}{{ context.loop_pos }}{% endfor %}'  ).render!('context' => ContextDrop.new, 'dummy' => [1,2,3])
+  end
+
+  def test_enumerable_drop
+    assert_equal '123', Liquid::Template.parse( '{% for c in collection %}{{c}}{% endfor %}').render!('collection' => EnumerableDrop.new)
+  end
+
+  def test_enumerable_drop_size
+    assert_equal '3', Liquid::Template.parse( '{{collection.size}}').render!('collection' => EnumerableDrop.new)
+  end
+
+  def test_enumerable_drop_will_invoke_before_method_for_clashing_method_names
+    ["select", "each", "map", "cycle"].each do |method|
+      assert_equal method.to_s, Liquid::Template.parse("{{collection.#{method}}}").render!('collection' => EnumerableDrop.new)
+      assert_equal method.to_s, Liquid::Template.parse("{{collection[\"#{method}\"]}}").render!('collection' => EnumerableDrop.new)
+      assert_equal method.to_s, Liquid::Template.parse("{{collection.#{method}}}").render!('collection' => RealEnumerableDrop.new)
+      assert_equal method.to_s, Liquid::Template.parse("{{collection[\"#{method}\"]}}").render!('collection' => RealEnumerableDrop.new)
+    end
+  end
+
+  def test_some_enumerable_methods_still_get_invoked
+    [ :count, :max ].each do |method|
+      assert_equal "3", Liquid::Template.parse("{{collection.#{method}}}").render!('collection' => RealEnumerableDrop.new)
+      assert_equal "3", Liquid::Template.parse("{{collection[\"#{method}\"]}}").render!('collection' => RealEnumerableDrop.new)
+      assert_equal "3", Liquid::Template.parse("{{collection.#{method}}}").render!('collection' => EnumerableDrop.new)
+      assert_equal "3", Liquid::Template.parse("{{collection[\"#{method}\"]}}").render!('collection' => EnumerableDrop.new)
+    end
+
+    assert_equal "yes", Liquid::Template.parse("{% if collection contains 3 %}yes{% endif %}").render!('collection' => RealEnumerableDrop.new)
+
+    [ :min, :first ].each do |method|
+      assert_equal "1", Liquid::Template.parse("{{collection.#{method}}}").render!('collection' => RealEnumerableDrop.new)
+      assert_equal "1", Liquid::Template.parse("{{collection[\"#{method}\"]}}").render!('collection' => RealEnumerableDrop.new)
+      assert_equal "1", Liquid::Template.parse("{{collection.#{method}}}").render!('collection' => EnumerableDrop.new)
+      assert_equal "1", Liquid::Template.parse("{{collection[\"#{method}\"]}}").render!('collection' => EnumerableDrop.new)
+    end
+  end
+
+  def test_empty_string_value_access
+    assert_equal '', Liquid::Template.parse('{{ product[value] }}').render!('product' => ProductDrop.new, 'value' => '')
+  end
+
+  def test_nil_value_access
+    assert_equal '', Liquid::Template.parse('{{ product[value] }}').render!('product' => ProductDrop.new, 'value' => nil)
+  end
+
+  def test_default_to_s_on_drops
+    assert_equal 'ProductDrop', Liquid::Template.parse("{{ product }}").render!('product' => ProductDrop.new)
+    assert_equal 'EnumerableDrop', Liquid::Template.parse('{{ collection }}').render!('collection' => EnumerableDrop.new)
+  end
+end # DropsTest

test/liquid/error_handling_test.rb

@@ -13,6 +13,10 @@ class ErrorDrop < Liquid::Drop
     raise Liquid::SyntaxError, 'syntax error'
   end
 
+  def exception
+    raise Exception, 'exception'
+  end
+
 end
 
 class ErrorHandlingTest < Test::Unit::TestCase
@@ -54,16 +58,53 @@ class ErrorHandlingTest < Test::Unit::TestCase
 
   def test_missing_endtag_parse_time_error
     assert_raise(Liquid::SyntaxError) do
-      template = Liquid::Template.parse(' {% for a in b %} ... ')
+      Liquid::Template.parse(' {% for a in b %} ... ')
     end
   end
 
   def test_unrecognized_operator
+    with_error_mode(:strict) do
+      assert_raise(SyntaxError) do
+        Liquid::Template.parse(' {% if 1 =! 2 %}ok{% endif %} ')
+      end
+    end
+  end
+  
+  def test_lax_unrecognized_operator
     assert_nothing_raised do
-      template = Liquid::Template.parse(' {% if 1 =! 2 %}ok{% endif %} ')
+      template = Liquid::Template.parse(' {% if 1 =! 2 %}ok{% endif %} ', :error_mode => :lax)
       assert_equal ' Liquid error: Unknown operator =! ', template.render
       assert_equal 1, template.errors.size
       assert_equal Liquid::ArgumentError, template.errors.first.class
     end
   end
+
+  def test_strict_error_messages
+    err = assert_raise(SyntaxError) do
+      Liquid::Template.parse(' {% if 1 =! 2 %}ok{% endif %} ', :error_mode => :strict)
+    end
+    assert_equal 'Unexpected character = in "1 =! 2"', err.message
+
+    err = assert_raise(SyntaxError) do
+      Liquid::Template.parse('{{%%%}}', :error_mode => :strict)
+    end
+    assert_equal 'Unexpected character % in "{{%%%}}"', err.message
+  end
+
+  def test_warnings
+    template = Liquid::Template.parse('{% if ~~~ %}{{%%%}}{% else %}{{ hello. }}{% endif %}', :error_mode => :warn)
+    assert_equal 3, template.warnings.size
+    assert_equal 'Unexpected character ~ in "~~~"', template.warnings[0].message
+    assert_equal 'Unexpected character % in "{{%%%}}"', template.warnings[1].message
+    assert_equal 'Expected id but found end_of_string in "{{ hello. }}"', template.warnings[2].message
+    assert_equal '', template.render
+  end
+
+  # Liquid should not catch Exceptions that are not subclasses of StandardError, like Interrupt and NoMemoryError
+  def test_exceptions_propagate
+    assert_raise Exception do
+      template = Liquid::Template.parse( ' {{ errors.exception }} '  )
+      template.render('errors' => ErrorDrop.new)
+    end
+  end
 end # ErrorHandlingTest

test/liquid/file_system_test.rb

@@ -5,7 +5,7 @@ class FileSystemTest < Test::Unit::TestCase
 
   def test_default
     assert_raise(FileSystemError) do
-      BlankFileSystem.new.read_template_file("dummy")
+      BlankFileSystem.new.read_template_file("dummy", {'dummy'=>'smarty'})
     end
   end
 
@@ -26,4 +26,10 @@ class FileSystemTest < Test::Unit::TestCase
       file_system.full_path("/etc/passwd")
     end
   end
+
+  def test_custom_template_filename_patterns
+    file_system = Liquid::LocalFileSystem.new("/some/path", "%s.html")
+    assert_equal "/some/path/mypartial.html"    , file_system.full_path("mypartial")
+    assert_equal "/some/path/dir/mypartial.html", file_system.full_path("dir/mypartial")
+  end
 end # FileSystemTest

test/liquid/filter_test.rb

@@ -16,6 +16,12 @@ module CanadianMoneyFilter
   end
 end
 
+module SubstituteFilter
+  def substitute(input, params={})
+    input.gsub(/%\{(\w+)\}/) { |match| params[$1] }
+  end
+end
+
 class FiltersTest < Test::Unit::TestCase
   include Liquid
 
@@ -75,6 +81,12 @@ class FiltersTest < Test::Unit::TestCase
     assert_equal "bla blub", Variable.new("var | strip_html").render(@context)
   end
 
+  def test_strip_html_ignore_comments_with_html
+    @context['var'] = "<!-- split and some <ul> tag --><b>bla blub</a>"
+
+    assert_equal "bla blub", Variable.new("var | strip_html").render(@context)
+  end
+
   def test_capitalize
     @context['var'] = "blub"
 
@@ -86,6 +98,13 @@ class FiltersTest < Test::Unit::TestCase
 
     assert_equal 1000, Variable.new("var | xyzzy").render(@context)
   end
+
+  def test_filter_with_keyword_arguments
+    @context['surname'] = 'john'
+    @context.add_filters(SubstituteFilter)
+    output = Variable.new(%! 'hello %{first_name}, %{last_name}' | substitute: first_name: surname, last_name: 'doe' !).render(@context)
+    assert_equal 'hello john, doe', output
+  end
 end
 
 class FiltersInTemplate < Test::Unit::TestCase
@@ -94,13 +113,13 @@ class FiltersInTemplate < Test::Unit::TestCase
   def test_local_global
     Template.register_filter(MoneyFilter)
 
-    assert_equal " 1000$ ", Template.parse("{{1000 | money}}").render(nil, nil)
+    assert_equal " 1000$ ", Template.parse("{{1000 | money}}").render!(nil, nil)
-    assert_equal " 1000$ CAD ", Template.parse("{{1000 | money}}").render(nil, :filters => CanadianMoneyFilter)
+    assert_equal " 1000$ CAD ", Template.parse("{{1000 | money}}").render!(nil, :filters => CanadianMoneyFilter)
-    assert_equal " 1000$ CAD ", Template.parse("{{1000 | money}}").render(nil, :filters => [CanadianMoneyFilter])
+    assert_equal " 1000$ CAD ", Template.parse("{{1000 | money}}").render!(nil, :filters => [CanadianMoneyFilter])
   end
 
   def test_local_filter_with_deprecated_syntax
-    assert_equal " 1000$ CAD ", Template.parse("{{1000 | money}}").render(nil, CanadianMoneyFilter)
+    assert_equal " 1000$ CAD ", Template.parse("{{1000 | money}}").render!(nil, CanadianMoneyFilter)
-    assert_equal " 1000$ CAD ", Template.parse("{{1000 | money}}").render(nil, [CanadianMoneyFilter])
+    assert_equal " 1000$ CAD ", Template.parse("{{1000 | money}}").render!(nil, [CanadianMoneyFilter])
   end
 end # FiltersTest

test/liquid/hash_ordering_test.rb

@@ -0,0 +1,25 @@
+require 'test_helper'
+
+module MoneyFilter
+  def money(input)
+    sprintf(' %d$ ', input)
+  end
+end
+
+module CanadianMoneyFilter
+  def money(input)
+    sprintf(' %d$ CAD ', input)
+  end
+end
+
+class HashOrderingTest < Test::Unit::TestCase
+  include Liquid
+
+ def test_global_register_order
+    Template.register_filter(MoneyFilter)
+    Template.register_filter(CanadianMoneyFilter)
+
+    assert_equal " 1000$ CAD ", Template.parse("{{1000 | money}}").render(nil, nil)
+ end
+
+end

test/liquid/i18n_test.rb

@@ -0,0 +1,37 @@
+require 'test_helper'
+
+class I18nTest < Test::Unit::TestCase
+  include Liquid
+
+  def setup
+    @i18n = I18n.new(fixture("en_locale.yml"))
+  end
+
+  def test_simple_translate_string
+    assert_equal "less is more", @i18n.translate("simple")
+  end
+
+  def test_nested_translate_string
+    assert_equal "something wasn't right", @i18n.translate("errors.syntax.oops")
+  end
+
+  def test_single_string_interpolation
+    assert_equal "something different", @i18n.translate("whatever", :something => "different")
+  end
+
+  # def test_raises_translation_error_on_undefined_interpolation_key
+  #   assert_raise I18n::TranslationError do
+  #     @i18n.translate("whatever", :oopstypos => "yes")
+  #   end
+  # end
+  
+  def test_raises_unknown_translation
+    assert_raise I18n::TranslationError do
+      @i18n.translate("doesnt_exist")
+    end
+  end
+
+  def test_sets_default_path_to_en
+    assert_equal I18n::DEFAULT_LOCALE, I18n.new.path
+  end
+end

test/liquid/lexer_test.rb

@@ -0,0 +1,48 @@
+require 'test_helper'
+
+class LexerTest < Test::Unit::TestCase
+  include Liquid
+
+  def test_strings
+    tokens = Lexer.new(%! 'this is a test""' "wat 'lol'"!).tokenize
+    assert_equal [[:string,%!'this is a test""'!], [:string, %!"wat 'lol'"!], [:end_of_string]], tokens
+  end
+
+  def test_integer
+    tokens = Lexer.new('hi 50').tokenize
+    assert_equal [[:id,'hi'], [:number, '50'], [:end_of_string]], tokens
+  end
+
+  def test_float
+    tokens = Lexer.new('hi 5.0').tokenize
+    assert_equal [[:id,'hi'], [:number, '5.0'], [:end_of_string]], tokens
+  end
+
+  def test_comparison
+    tokens = Lexer.new('== <> contains').tokenize
+    assert_equal [[:comparison,'=='], [:comparison, '<>'], [:comparison, 'contains'], [:end_of_string]], tokens
+  end
+
+  def test_specials
+    tokens = Lexer.new('| .:').tokenize
+    assert_equal [[:pipe, '|'], [:dot, '.'], [:colon, ':'], [:end_of_string]], tokens
+    tokens = Lexer.new('[,]').tokenize
+    assert_equal [[:open_square, '['], [:comma, ','], [:close_square, ']'], [:end_of_string]], tokens
+  end
+
+  def test_fancy_identifiers
+    tokens = Lexer.new('hi! five?').tokenize
+    assert_equal [[:id,'hi!'], [:id, 'five?'], [:end_of_string]], tokens
+  end
+
+  def test_whitespace
+    tokens = Lexer.new("five|\n\t ==").tokenize
+    assert_equal [[:id,'five'], [:pipe, '|'], [:comparison, '=='], [:end_of_string]], tokens
+  end
+
+  def test_unexpected_character
+    assert_raises(SyntaxError) do
+      Lexer.new("%").tokenize
+    end
+  end
+end

test/liquid/module_ex_test.rb

@@ -77,11 +77,11 @@ class ModuleExTest < Test::Unit::TestCase
   end
 
   def test_should_use_regular_objects_as_drops
-    assert_equal 'allowedA', Liquid::Template.parse("{{ a.allowedA }}").render('a'=>@a)
+    assert_template_result 'allowedA', "{{ a.allowedA }}", 'a'=>@a
-    assert_equal 'allowedB', Liquid::Template.parse("{{ a.chainedB.allowedB }}").render('a'=>@a)
+    assert_template_result 'allowedB', "{{ a.chainedB.allowedB }}", 'a'=>@a
-    assert_equal 'allowedC', Liquid::Template.parse("{{ a.chainedB.chainedC.allowedC }}").render('a'=>@a)
+    assert_template_result 'allowedC', "{{ a.chainedB.chainedC.allowedC }}", 'a'=>@a
-    assert_equal 'another_allowedC', Liquid::Template.parse("{{ a.chainedB.chainedC.another_allowedC }}").render('a'=>@a)
+    assert_template_result 'another_allowedC', "{{ a.chainedB.chainedC.another_allowedC }}", 'a'=>@a
-    assert_equal '', Liquid::Template.parse("{{ a.restricted }}").render('a'=>@a)
+    assert_template_result '', "{{ a.restricted }}", 'a'=>@a
-    assert_equal '', Liquid::Template.parse("{{ a.unknown }}").render('a'=>@a)
+    assert_template_result '', "{{ a.unknown }}", 'a'=>@a
   end
 end # ModuleExTest

test/liquid/output_test.rb

@@ -41,76 +41,76 @@ class OutputTest < Test::Unit::TestCase
     text = %| {{best_cars}} |
 
     expected = %| bmw |
-    assert_equal expected, Template.parse(text).render(@assigns)
+    assert_equal expected, Template.parse(text).render!(@assigns)
   end
 
   def test_variable_traversing
     text = %| {{car.bmw}} {{car.gm}} {{car.bmw}} |
 
     expected = %| good bad good |
-    assert_equal expected, Template.parse(text).render(@assigns)
+    assert_equal expected, Template.parse(text).render!(@assigns)
   end
 
   def test_variable_piping
     text = %( {{ car.gm | make_funny }} )
     expected = %| LOL |
 
-    assert_equal expected, Template.parse(text).render(@assigns, :filters => [FunnyFilter])
+    assert_equal expected, Template.parse(text).render!(@assigns, :filters => [FunnyFilter])
   end
 
   def test_variable_piping_with_input
     text = %( {{ car.gm | cite_funny }} )
     expected = %| LOL: bad |
 
-    assert_equal expected, Template.parse(text).render(@assigns, :filters => [FunnyFilter])
+    assert_equal expected, Template.parse(text).render!(@assigns, :filters => [FunnyFilter])
   end
 
   def test_variable_piping_with_args
     text = %! {{ car.gm | add_smiley : ':-(' }} !
     expected = %| bad :-( |
 
-    assert_equal expected, Template.parse(text).render(@assigns, :filters => [FunnyFilter])
+    assert_equal expected, Template.parse(text).render!(@assigns, :filters => [FunnyFilter])
   end
 
   def test_variable_piping_with_no_args
     text = %! {{ car.gm | add_smiley }} !
     expected = %| bad :-) |
 
-    assert_equal expected, Template.parse(text).render(@assigns, :filters => [FunnyFilter])
+    assert_equal expected, Template.parse(text).render!(@assigns, :filters => [FunnyFilter])
   end
 
   def test_multiple_variable_piping_with_args
     text = %! {{ car.gm | add_smiley : ':-(' | add_smiley : ':-('}} !
     expected = %| bad :-( :-( |
 
-    assert_equal expected, Template.parse(text).render(@assigns, :filters => [FunnyFilter])
+    assert_equal expected, Template.parse(text).render!(@assigns, :filters => [FunnyFilter])
   end
 
-  def test_variable_piping_with_args
+  def test_variable_piping_with_multiple_args
     text = %! {{ car.gm | add_tag : 'span', 'bar'}} !
     expected = %| <span id="bar">bad</span> |
 
-    assert_equal expected, Template.parse(text).render(@assigns, :filters => [FunnyFilter])
+    assert_equal expected, Template.parse(text).render!(@assigns, :filters => [FunnyFilter])
   end
 
   def test_variable_piping_with_variable_args
     text = %! {{ car.gm | add_tag : 'span', car.bmw}} !
     expected = %| <span id="good">bad</span> |
 
-    assert_equal expected, Template.parse(text).render(@assigns, :filters => [FunnyFilter])
+    assert_equal expected, Template.parse(text).render!(@assigns, :filters => [FunnyFilter])
   end
 
   def test_multiple_pipings
     text = %( {{ best_cars | cite_funny | paragraph }} )
     expected = %| <p>LOL: bmw</p> |
 
-    assert_equal expected, Template.parse(text).render(@assigns, :filters => [FunnyFilter])
+    assert_equal expected, Template.parse(text).render!(@assigns, :filters => [FunnyFilter])
   end
 
   def test_link_to
     text = %( {{ 'Typo' | link_to: 'http://typo.leetsoft.com' }} )
     expected = %| <a href="http://typo.leetsoft.com">Typo</a> |
 
-    assert_equal expected, Template.parse(text).render(@assigns, :filters => [FunnyFilter])
+    assert_equal expected, Template.parse(text).render!(@assigns, :filters => [FunnyFilter])
   end
 end # OutputTest

test/liquid/parser_test.rb

@@ -0,0 +1,82 @@
+require 'test_helper'
+
+class ParserTest < Test::Unit::TestCase
+  include Liquid
+
+  def test_consume
+    p = Parser.new("wat: 7")
+    assert_equal 'wat', p.consume(:id)
+    assert_equal ':', p.consume(:colon)
+    assert_equal '7', p.consume(:number)
+  end
+
+  def test_jump
+    p = Parser.new("wat: 7")
+    p.jump(2)
+    assert_equal '7', p.consume(:number)
+  end
+
+  def test_consume?
+    p = Parser.new("wat: 7")
+    assert_equal 'wat', p.consume?(:id)
+    assert_equal false, p.consume?(:dot)
+    assert_equal ':', p.consume(:colon)
+    assert_equal '7', p.consume?(:number)
+  end
+
+  def test_id?
+    p = Parser.new("wat 6 Peter Hegemon")
+    assert_equal 'wat', p.id?('wat')
+    assert_equal false, p.id?('endgame')
+    assert_equal '6', p.consume(:number)
+    assert_equal 'Peter', p.id?('Peter')
+    assert_equal false, p.id?('Achilles')
+  end
+
+  def test_look
+    p = Parser.new("wat 6 Peter Hegemon")
+    assert_equal true, p.look(:id)
+    assert_equal 'wat', p.consume(:id)
+    assert_equal false, p.look(:comparison)
+    assert_equal true, p.look(:number)
+    assert_equal true, p.look(:id, 1)
+    assert_equal false, p.look(:number, 1)
+  end
+
+  def test_expressions
+    p = Parser.new("hi.there hi[5].! hi.there.bob")
+    assert_equal 'hi.there', p.expression
+    assert_equal 'hi[5].!', p.expression
+    assert_equal 'hi.there.bob', p.expression
+
+    p = Parser.new("567 6.0 'lol' \"wut\"")
+    assert_equal '567', p.expression
+    assert_equal '6.0', p.expression
+    assert_equal "'lol'", p.expression
+    assert_equal '"wut"', p.expression
+  end
+
+  def test_ranges
+    p = Parser.new("(5..7) (1.5..9.6) (young..old) (hi[5].wat..old)")
+    assert_equal '(5..7)', p.expression
+    assert_equal '(1.5..9.6)', p.expression
+    assert_equal '(young..old)', p.expression
+    assert_equal '(hi[5].wat..old)', p.expression
+  end
+
+  def test_arguments
+    p = Parser.new("filter: hi.there[5], keyarg: 7")
+    assert_equal 'filter', p.consume(:id)
+    assert_equal ':', p.consume(:colon)
+    assert_equal 'hi.there[5]', p.argument
+    assert_equal ',', p.consume(:comma)
+    assert_equal 'keyarg: 7', p.argument
+  end
+
+  def test_invalid_expression
+    assert_raises(SyntaxError) do
+      p = Parser.new("==")
+      p.expression
+    end
+  end
+end