Fix error

Add I18n syntax error translation

History.md

@@ -3,15 +3,16 @@
 IMPORTANT: Liquid 2.6 is going to be the last version of Liquid which maintains explicit Ruby 1.8 compatability.
 The following releases will only be tested against Ruby 1.9 and Ruby 2.0 and are likely to break on Ruby 1.8.
 
-## 2.6.1 / 2014-01-10 / branch "2-6-stable"
-
-Security fix, cherry-picked from master (4e14a65):
-* Don't call to_sym when creating conditions for security reasons, see #273 [Bouke van der Bijl, bouk]
-* Prevent arbitrary method invocation on condition objects, see #274 [Dylan Thacker-Smith, dylanahsmith]
-
-## 2.6.0 / 2013-11-25
+## 2.6.0 / Master branch (not yet released)
 
 * ...
+* Add optional strict parsing and warn parsing, see #235 [Tristan Hume, trishume]
+* Add I18n syntax error translation, see #241 [Simon Hørup Eskildsen, Sirupsen]
+* Make sort filter work on enumerable drops, see #239 [Florian Weingarten, fw42]
+* Fix clashing method names in enumerable drops, see #238 [Florian Weingarten, fw42]
+* Make map filter work on enumerable drops, see #233 [Florian Weingarten, fw42]
+* Fix security issue with map filter, see #230, #232, #234, #237 [Florian Weingarten, fw42]
+* Improved whitespace stripping for blank blocks, related to #216 [Florian Weingarten, fw42]
 * Bugfix for #106: fix example servlet [gnowoel]
 * Bugfix for #97: strip_html filter supports multi-line tags [Jo Liss, joliss]
 * Bugfix for #114: strip_html filter supports style tags [James Allardice, jamesallardice]
@@ -20,7 +21,6 @@ Security fix, cherry-picked from master (4e14a65):
 * Bugfix for #204: 'raw' parsing bug [Florian Weingarten, fw42]
 * Bugfix for #150: 'for' parsing bug [Peter Schröder, phoet]
 * Bugfix for #126: Strip CRLF in strip_newline [Peter Schröder, phoet]
-* Bugfix for #174, "can't convert Fixnum into String" for "replace" [wǒ_is神仙, jsw0528]
 * Allow a Liquid::Drop to be passed into Template#render [Daniel Huckstep, darkhelmet]
 * Resource limits [Florian Weingarten, fw42]
 * Add reverse filter [Jay Strybis, unreal]
@@ -31,21 +31,6 @@ Security fix, cherry-picked from master (4e14a65):
 * Better documentation for 'include' tag (closes #163) [Peter Schröder, phoet]
 * Use of BigDecimal on filters to have better precision (closes #155) [Arthur Nogueira Neves, arthurnn]
 
-## 2.5.4 / 2013-11-11 / branch "2.5-stable"
-
-* Fix "can't convert Fixnum into String" for "replace", see #173, [wǒ_is神仙, jsw0528]
-
-## 2.5.3 / 2013-10-09
-
-* #232, #234, #237: Fix map filter bugs [Florian Weingarten, fw42]
-
-## 2.5.2 / 2013-09-03 / deleted
-
-Yanked from rubygems, as it contained too many changes that broke compatibility. Those changes will be on following major releases.
-
-## 2.5.1 / 2013-07-24
-
-* #230: Fix security issue with map filter, Use invoke_drop in map filter [Florian Weingarten, fw42]
 
 ## 2.5.0 / 2013-03-06
 

README.md

@@ -4,6 +4,7 @@
 * [Version history](History.md)
 * [Liquid documentation from Shopify](http://docs.shopify.com/themes/liquid-basics)
 * [Liquid Wiki from Shopify](http://wiki.shopify.com/Liquid)
+* [Liquid Wiki at GitHub](https://github.com/Shopify/liquid/wiki)
 * [Website](http://liquidmarkup.org/)
 
 ## Introduction
@@ -47,4 +48,28 @@ For standard use you can just pass it the content of a file and call render with
 @template.render('name' => 'tobi')                # => "hi tobi"
 ```
 
+### Error Modes
+
+Setting the error mode of Liquid lets you specify how strictly you want your templates to be interpreted.
+Normally the parser is very lax and will accept almost anything without error. Unfortunately this can make
+it very hard to debug and can lead to unexpected behaviour. 
+
+Liquid also comes with a stricter parser that can be used when editing templates to give better error messages
+when templates are invalid. You can enable this new parser like this:
+
+```ruby
+Liquid::Template.error_mode = :strict # Raises a SyntaxError when invalid syntax is used
+Liquid::Template.error_mode = :warn # Adds errors to template.errors but continues as normal
+Liquid::Template.error_mode = :lax # The default mode, accepts almost anything.
+```
+
+If you want to set the error mode only on specific templates you can pass `:error_mode` as an option to `parse`:
+```ruby
+Liquid::Template.parse(source, :error_mode => :strict)
+```
+This is useful for doing things like enabling strict mode only in the theme editor.
+
+It is recommended that you enable `:strict` or `:warn` mode on new apps to stop invalid templates from being created.
+It is also recommended that you use it in the template editors of existing apps to give editors better error messages.
+
 [![Build Status](https://secure.travis-ci.org/Shopify/liquid.png)](http://travis-ci.org/Shopify/liquid)

Rakefile

@@ -1,39 +1,55 @@
+#!/usr/bin/env ruby
+
+require 'rubygems'
 require 'rake'
 require 'rake/testtask'
-$LOAD_PATH.unshift File.expand_path("../lib", __FILE__)
-require "liquid/version"
+require 'rubygems/package_task'
 
 task :default => 'test'
 
-Rake::TestTask.new(:test) do |t|
+desc 'run test suite with default parser'
+Rake::TestTask.new(:base_test) do |t|
   t.libs << '.' << 'lib' << 'test'
   t.test_files = FileList['test/liquid/**/*_test.rb']
   t.verbose = false
 end
 
-task :gem => :build
-task :build do
-  system "gem build liquid.gemspec"
+desc 'run test suite with warn error mode'
+task :warn_test do
+  ENV['LIQUID_PARSER_MODE'] = 'warn'
+  Rake::Task['base_test'].invoke
 end
 
-task :install => :build do
-  system "gem install liquid-#{Liquid::VERSION}.gem"
+desc 'runs test suite with both strict and lax parsers'
+task :test do
+  ENV['LIQUID_PARSER_MODE'] = 'lax'
+  Rake::Task['base_test'].invoke
+  ENV['LIQUID_PARSER_MODE'] = 'strict'
+  Rake::Task['base_test'].reenable
+  Rake::Task['base_test'].invoke
 end
 
-task :release => :build do
-  system "git tag -a v#{Liquid::VERSION} -m 'Tagging #{Liquid::VERSION}'"
-  system "git push --tags"
-  system "gem push liquid-#{Liquid::VERSION}.gem"
-  system "rm liquid-#{Liquid::VERSION}.gem"
+gemspec = eval(File.read('liquid.gemspec'))
+Gem::PackageTask.new(gemspec) do |pkg|
+  pkg.gem_spec = gemspec
+end
+
+desc "Build the gem and release it to rubygems.org"
+task :release => :gem do
+  sh "gem push pkg/liquid-#{gemspec.version}.gem"
 end
 
 namespace :benchmark do
 
-  desc "Run the liquid benchmark"
+  desc "Run the liquid benchmark with lax parsing"
   task :run do
-    ruby "./performance/benchmark.rb"
+    ruby "./performance/benchmark.rb lax"
   end
 
+  desc "Run the liquid benchmark with strict parsing"
+  task :strict do
+    ruby "./performance/benchmark.rb strict"
+  end
 end
 
 

lib/liquid.rb

@@ -46,6 +46,9 @@ module Liquid
 end
 
 require "liquid/version"
+require 'liquid/lexer'
+require 'liquid/parser'
+require 'liquid/i18n'
 require 'liquid/drop'
 require 'liquid/extensions'
 require 'liquid/errors'

lib/liquid/block.rb

@@ -1,17 +1,23 @@
 module Liquid
-
   class Block < Tag
     IsTag             = /^#{TagStart}/o
     IsVariable        = /^#{VariableStart}/o
     FullToken         = /^#{TagStart}\s*(\w+)\s*(.*)?#{TagEnd}$/o
     ContentOfVariable = /^#{VariableStart}(.*)#{VariableEnd}$/o
 
+    def blank?
+      @blank || false
+    end
+
     def parse(tokens)
+      @blank = true
       @nodelist ||= []
       @nodelist.clear
 
-      while token = tokens.shift
+      # All child tags of the current block.
+      @children = []
 
+      while token = tokens.shift
         case token
         when IsTag
           if token =~ FullToken
@@ -25,21 +31,28 @@ module Liquid
 
             # fetch the tag from registered blocks
             if tag = Template.tags[$1]
-              @nodelist << tag.new($1, $2, tokens)
+              new_tag = tag.new_with_options($1, $2, tokens, @options || {})
+              @blank &&= new_tag.blank?
+              @nodelist << new_tag
+              @children << new_tag
             else
               # this tag is not registered with the system
               # pass it to the current block for special handling or error reporting
               unknown_tag($1, $2, tokens)
             end
           else
-            raise SyntaxError, "Tag '#{token}' was not properly terminated with regexp: #{TagEnd.inspect} "
+            raise SyntaxError.new(options[:locale].t("errors.syntax.tag_termination", :token => token, :tag_end => TagEnd.inspect))
           end
         when IsVariable
-          @nodelist << create_variable(token)
+          new_var = create_variable(token)
+          @nodelist << new_var
+          @children << new_var
+          @blank = false
         when ''
           # pass
         else
           @nodelist << token
+          @blank &&= (token =~ /\A\s*\z/)
         end
       end
 
@@ -49,17 +62,32 @@ module Liquid
       assert_missing_delimitation!
     end
 
+    # warnings of this block and all sub-tags
+    def warnings
+      all_warnings = []
+      all_warnings.concat(@warnings) if @warnings
+
+      (@children || []).each do |node|
+        all_warnings.concat(node.warnings || [])
+      end
+
+      all_warnings
+    end
+
     def end_tag
     end
 
     def unknown_tag(tag, params, tokens)
       case tag
       when 'else'
-        raise SyntaxError, "#{block_name} tag does not expect else tag"
+        raise SyntaxError.new(options[:locale].t("errors.syntax.unexpected_else", 
+                                                 :block_name => block_name))
       when 'end'
-        raise SyntaxError, "'end' is not a valid delimiter for #{block_name} tags. use #{block_delimiter}"
+        raise SyntaxError.new(options[:locale].t("errors.syntax.invalid_delimiter", 
+                                                 :block_name => block_name, 
+                                                 :block_delimiter => block_delimiter))
       else
-        raise SyntaxError, "Unknown tag '#{tag}'"
+        raise SyntaxError.new(options[:locale].t("errors.syntax.unknown_tag", :tag => tag))
       end
     end
 
@@ -73,9 +101,9 @@ module Liquid
 
     def create_variable(token)
       token.scan(ContentOfVariable) do |content|
-        return Variable.new(content.first)
+        return Variable.new(content.first, @options)
       end
-      raise SyntaxError.new("Variable '#{token}' was not properly terminated with regexp: #{VariableEnd.inspect} ")
+      raise SyntaxError.new(options[:locale].t("errors.syntax.variable_termination", :token => token, :tag_end => VariableEnd.inspect))
     end
 
     def render(context)
@@ -85,7 +113,7 @@ module Liquid
     protected
 
     def assert_missing_delimitation!
-      raise SyntaxError.new("#{block_name} tag was never closed")
+      raise SyntaxError.new(options[:locale].t("errors.syntax.tag_never_closed", :block_name => block_name))
     end
 
     def render_all(list, context)
@@ -112,7 +140,9 @@ module Liquid
             context.resource_limits[:reached] = true
             raise MemoryError.new("Memory limits exceeded")
           end
-          output << token_output
+          unless token.is_a?(Block) && token.blank?
+            output << token_output
+          end
         rescue MemoryError => e
           raise e
         rescue ::StandardError => e

lib/liquid/document.rb

@@ -1,7 +1,8 @@
 module Liquid
   class Document < Block
     # we don't need markup to open this block
-    def initialize(tokens)
+    def initialize(tokens, options = {})
+      @options = options
       parse(tokens)
     end
 

lib/liquid/drop.rb

@@ -44,6 +44,10 @@ module Liquid
       true
     end
 
+    def inspect
+      self.class.to_s
+    end
+
     def to_liquid
       self
     end
@@ -54,7 +58,16 @@ module Liquid
 
     # Check for method existence without invoking respond_to?, which creates symbols
     def self.invokable?(method_name)
-      @invokable_methods ||= Set.new(["to_liquid"] + (public_instance_methods - Liquid::Drop.public_instance_methods).map(&:to_s))
+      unless @invokable_methods
+        # Ruby 1.8 compatibility: call to_s on method names (which are strings in 1.8, but already symbols in 1.9)
+        blacklist = (Liquid::Drop.public_instance_methods + [:each]).map(&:to_s)
+        if include?(Enumerable)
+          blacklist += Enumerable.public_instance_methods.map(&:to_s)
+          blacklist -= [:sort, :count, :first, :min, :max, :include?].map(&:to_s)
+        end
+        whitelist = [:to_liquid] + (public_instance_methods.map(&:to_s) - blacklist.map(&:to_s))
+        @invokable_methods = Set.new(whitelist.map(&:to_s))
+      end
       @invokable_methods.include?(method_name.to_s)
     end
   end

lib/liquid/htmltags.rb

@@ -11,7 +11,7 @@ module Liquid
           @attributes[key] = value
         end
       else
-        raise SyntaxError.new("Syntax Error in 'table_row loop' - Valid syntax: table_row [item] in [collection] cols=3")
+        raise SyntaxError.new(options[:locale].t("errors.syntax.table_row"))
       end
 
       super
@@ -43,6 +43,7 @@ module Liquid
             'index0'  => index,
             'col'     => col + 1,
             'col0'    => col,
+            'index0'  => index,
             'rindex'  => length - index,
             'rindex0' => length - index - 1,
             'first'   => (index == 0),

lib/liquid/i18n.rb

@@ -0,0 +1,39 @@
+require 'yaml'
+
+module Liquid
+  class I18n
+    DEFAULT_LOCALE = File.join(File.expand_path(File.dirname(__FILE__)), "locales", "en.yml")
+
+    class TranslationError < StandardError
+    end
+    
+    attr_reader :path
+
+    def initialize(path = DEFAULT_LOCALE)
+      @path = path
+    end
+
+    def translate(name, vars = {})
+      interpolate(deep_fetch_translation(name), vars)
+    end
+    alias_method :t, :translate
+
+    def locale
+      @locale ||= YAML.load_file(@path)
+    end
+
+    private
+    def interpolate(name, vars)
+      name.gsub(/%{(\w+)}/) {
+        # raise TranslationError, "Undefined key #{$1} for interpolation in translation #{name}"  unless vars[$1.to_sym]
+        "#{vars[$1.to_sym]}"
+      }
+    end
+
+    def deep_fetch_translation(name)
+      name.split('.').reduce(locale) do |level, cur|
+        level[cur] or raise TranslationError, "Translation for #{name} does not exist in locale #{path}"
+      end
+    end
+  end
+end

lib/liquid/lexer.rb

@@ -0,0 +1,49 @@
+require "strscan"
+module Liquid
+  class Lexer
+    SPECIALS = {
+      '|' => :pipe,
+      '.' => :dot,
+      ':' => :colon,
+      ',' => :comma,
+      '[' => :open_square,
+      ']' => :close_square,
+      '(' => :open_round,
+      ')' => :close_round
+    }
+    IDENTIFIER = /[\w\-?!]+/
+    SINGLE_STRING_LITERAL = /'[^\']*'/
+    DOUBLE_STRING_LITERAL = /"[^\"]*"/
+    NUMBER_LITERAL = /-?\d+(\.\d+)?/
+    COMPARISON_OPERATOR = /==|!=|<>|<=?|>=?|contains/
+
+    def initialize(input)
+      @ss = StringScanner.new(input.rstrip)
+    end
+
+    def tokenize
+      @output = []
+
+      while !@ss.eos?
+        @ss.skip(/\s*/)
+        tok = case
+        when t = @ss.scan(COMPARISON_OPERATOR) then [:comparison, t]
+        when t = @ss.scan(SINGLE_STRING_LITERAL) then [:string, t]
+        when t = @ss.scan(DOUBLE_STRING_LITERAL) then [:string, t]
+        when t = @ss.scan(NUMBER_LITERAL) then [:number, t]
+        when t = @ss.scan(IDENTIFIER) then [:id, t]
+        else
+          c = @ss.getch
+          if s = SPECIALS[c]
+            [s,c]
+          else
+            raise SyntaxError, "Unexpected character #{c}"
+          end
+        end
+        @output << tok
+      end
+
+      @output << [:end_of_string]
+    end
+  end
+end

lib/liquid/locales/en.yml

@@ -0,0 +1,22 @@
+---
+  errors:
+    syntax:
+      assign: "Syntax Error in 'assign' - Valid syntax: assign [var] = [source]"
+      capture: "Syntax Error in 'capture' - Valid syntax: capture [var]"
+      case: "Syntax Error in 'case' - Valid syntax: case [condition]"
+      case_invalid_when: "Syntax Error in tag 'case' - Valid when condition: {% when [condition] [or condition2...] %}"
+      case_invalid_else: "Syntax Error in tag 'case' - Valid else condition: {% else %} (no parameters) "
+      cycle: "Syntax Error in 'cycle' - Valid syntax: cycle [name :] var [, var2, var3 ...]"
+      for: "Syntax Error in 'for loop' - Valid syntax: for [item] in [collection]"
+      for_invalid_in: "For loops require an 'in' clause"
+      for_invalid_attribute: "Invalid attribute in for loop. Valid attributes are limit and offset"
+      if: "Syntax Error in tag 'if' - Valid syntax: if [expression]"
+      include: "Error in tag 'include' - Valid syntax: include '[template]' (with|for) [object|collection]"
+      unknown_tag: "Unknown tag '%{tag}'"
+      invalid_delimiter: "'end' is not a valid delimiter for %{block_name} tags. use %{block_delimiter}"
+      unexpected_else: "%{block_name} tag does not expect else tag"
+      tag_termination: "Tag '%{token}' was not properly terminated with regexp: %{tag_end}"
+      variable_termination: "Variable '%{token}' was not properly terminated with regexp: %{tag_end}"
+      tag_never_closed: "'%{block_name}' tag was never closed"
+      meta_syntax_error: "Liquid syntax error: #{e.message}"
+      table_row: "Syntax Error in 'table_row loop' - Valid syntax: table_row [item] in [collection] cols=3"

lib/liquid/parser.rb

@@ -0,0 +1,90 @@
+module Liquid
+  class Parser
+    def initialize(input)
+      l = Lexer.new(input)
+      @tokens = l.tokenize
+      @p = 0 # pointer to current location
+    end
+
+    def jump(point)
+      @p = point
+    end
+
+    def consume(type = nil)
+      token = @tokens[@p]
+      if type && token[0] != type
+        raise SyntaxError, "Expected #{type} but found #{@tokens[@p].first}"
+      end
+      @p += 1
+      token[1]
+    end
+
+    # Only consumes the token if it matches the type
+    # Returns the token's contents if it was consumed
+    # or false otherwise.
+    def consume?(type)
+      token = @tokens[@p]
+      return false unless token && token[0] == type
+      @p += 1
+      token[1]
+    end
+
+    # Like consume? Except for an :id token of a certain name
+    def id?(str)
+      token = @tokens[@p]
+      return false unless token && token[0] == :id
+      return false unless token[1] == str
+      @p += 1
+      token[1]
+    end
+
+    def look(type, ahead = 0)
+      tok = @tokens[@p + ahead]
+      return false unless tok
+      tok[0] == type
+    end
+
+    def expression
+      token = @tokens[@p]
+      if token[0] == :id
+        variable_signature
+      elsif [:string, :number].include? token[0]
+        consume
+      elsif token.first == :open_round
+        consume
+        first = expression
+        consume(:dot)
+        consume(:dot)
+        last = expression
+        consume(:close_round)
+        "(#{first}..#{last})"
+      else
+        raise SyntaxError, "#{token} is not a valid expression"
+      end
+    end
+
+    def argument
+      str = ""
+      # might be a keyword argument (identifier: expression)
+      if look(:id) && look(:colon, 1)
+        str << consume << consume << ' '
+      end
+
+      str << expression
+    end
+
+    def variable_signature
+      str = consume(:id)
+      if look(:open_square)
+        str << consume
+        str << expression
+        str << consume(:close_square)
+      end
+      if look(:dot)
+        str << consume
+        str << variable_signature
+      end
+      str
+    end
+  end
+end

lib/liquid/standardfilters.rb

@@ -81,7 +81,7 @@ module Liquid
     # Sort elements of the array
     # provide optional property with which to sort an array of hashes or drops
     def sort(input, property = nil)
-      ary = [input].flatten
+      ary = flatten_if_necessary(input)
       if property.nil?
         ary.sort
       elsif ary.first.respond_to?('[]') and !ary.first[property].nil?
@@ -99,10 +99,8 @@ module Liquid
 
     # map/collect on a given property
     def map(input, property)
-      ary = [input].flatten
-      ary.map do |e|
+      flatten_if_necessary(input).map do |e|
         e = e.call if e.is_a?(Proc)
-        e = e.to_liquid if e.respond_to?(:to_liquid)
 
         if property == "to_liquid"
           e
@@ -249,6 +247,17 @@ module Liquid
 
     private
 
+    def flatten_if_necessary(input)
+      ary = if input.is_a?(Array)
+        input.flatten
+      elsif input.kind_of?(Enumerable)
+        input
+      else
+        [input].flatten
+      end
+      ary.map{ |e| e.respond_to?(:to_liquid) ? e.to_liquid : e }
+    end
+
     def to_number(obj)
       case obj
       when Float

lib/liquid/tag.rb

@@ -1,12 +1,21 @@
 module Liquid
-
   class Tag
+    attr_accessor :nodelist, :options
+    attr_reader :warnings
 
-    attr_accessor :nodelist
+    def self.new_with_options(tag_name, markup, tokens, options)
+      # Forgive me Matz for I have sinned. I know this code is weird
+      # but it was necessary to maintain API compatibility.
+      new_tag = self.allocate
+      new_tag.options = options
+      new_tag.send(:initialize, tag_name, markup, tokens)
+      new_tag
+    end
 
     def initialize(tag_name, markup, tokens)
       @tag_name   = tag_name
       @markup     = markup
+      @options    ||= {} # needs || because might be set before initialize
       parse(tokens)
     end
 
@@ -21,6 +30,31 @@ module Liquid
       ''
     end
 
-  end # Tag
+    def blank?
+      @blank || true
+    end
 
+    def parse_with_selected_parser(markup)
+      case @options[:error_mode] || Template.error_mode
+      when :strict then strict_parse_with_error_context(markup)
+      when :lax    then lax_parse(markup)
+      when :warn
+        begin
+          return strict_parse_with_error_context(markup)
+        rescue SyntaxError => e
+          @warnings ||= []
+          @warnings << e
+          return lax_parse(markup)
+        end
+      end
+    end
+
+    private
+    def strict_parse_with_error_context(markup)
+      strict_parse(markup)
+    rescue SyntaxError => e
+      e.message << " in \"#{markup.strip}\"" 
+      raise e
+    end
+  end # Tag
 end # Liquid

lib/liquid/tags/assign.rb

@@ -16,7 +16,7 @@ module Liquid
         @to = $1
         @from = Variable.new($2)
       else
-        raise SyntaxError.new("Syntax Error in 'assign' - Valid syntax: assign [var] = [source]")
+        raise SyntaxError.new options[:locale].t("errors.syntax.assign")
       end
 
       super

lib/liquid/tags/capture.rb

@@ -18,7 +18,7 @@ module Liquid
       if markup =~ Syntax
         @to = $1
       else
-        raise SyntaxError.new("Syntax Error in 'capture' - Valid syntax: capture [var]")
+        raise SyntaxError.new(options[:locale].t("errors.syntax.capture"))
       end
 
       super
@@ -30,6 +30,10 @@ module Liquid
       context.resource_limits[:assign_score_current] += (output.respond_to?(:length) ? output.length : 1)
       ''
     end
+
+    def blank?
+      true
+    end
   end
 
   Template.register_tag('capture', Capture)

lib/liquid/tags/case.rb

@@ -9,7 +9,7 @@ module Liquid
       if markup =~ Syntax
         @left = $1
       else
-        raise SyntaxError.new("Syntax Error in tag 'case' - Valid syntax: case [condition]")
+        raise SyntaxError.new(options[:locale].t("errors.syntax.case"))
       end
 
       super
@@ -50,7 +50,7 @@ module Liquid
       while markup
         # Create a new nodelist and assign it to the new block
         if not markup =~ WhenSyntax
-          raise SyntaxError.new("Syntax Error in tag 'case' - Valid when condition: {% when [condition] [or condition2...] %} ")
+          raise SyntaxError.new(options[:locale].t("errors.syntax.case_invalid_when"))
         end
 
         markup = $2
@@ -62,17 +62,14 @@ module Liquid
     end
 
     def record_else_condition(markup)
-
       if not markup.strip.empty?
-        raise SyntaxError.new("Syntax Error in tag 'case' - Valid else condition: {% else %} (no parameters) ")
+        raise SyntaxError.new(options[:locale].t("errors.syntax.case_invalid_else"))
       end
 
       block = ElseCondition.new
       block.attach(@nodelist)
       @blocks << block
     end
-
-
   end
 
   Template.register_tag('case', Case)

lib/liquid/tags/comment.rb

@@ -3,6 +3,10 @@ module Liquid
     def render(context)
       ''
     end
+
+    def blank?
+      true
+    end
   end
 
   Template.register_tag('comment', Comment)

lib/liquid/tags/cycle.rb

@@ -1,5 +1,4 @@
 module Liquid
-
   # Cycle is usually used within a loop to alternate between values, like colors or DOM classes.
   #
   #   {% for item in items %}
@@ -25,7 +24,7 @@ module Liquid
         @variables = variables_from_string(markup)
         @name = "'#{@variables.to_s}'"
       else
-        raise SyntaxError.new("Syntax Error in 'cycle' - Valid syntax: cycle [name :] var [, var2, var3 ...]")
+        raise SyntaxError.new(options[:locale].t("errors.syntax.cycle"))
       end
       super
     end
@@ -44,15 +43,17 @@ module Liquid
       end
     end
 
-    private
+    def blank?
+      false
+    end
 
+    private
     def variables_from_string(markup)
       markup.split(',').collect do |var|
         var =~ /\s*(#{QuotedFragment})\s*/o
         $1 ? $1 : nil
       end.compact
     end
-
   end
 
   Template.register_tag('cycle', Cycle)

lib/liquid/tags/for.rb

@@ -44,22 +44,10 @@ module Liquid
   # forloop.last:: Returns true if the item is the last item.
   #
   class For < Block
-    Syntax = /\A(\w+)\s+in\s+(#{QuotedFragment}+)\s*(reversed)?/o
+    Syntax = /\A(#{VariableSegment}+)\s+in\s+(#{QuotedFragment}+)\s*(reversed)?/o
 
     def initialize(tag_name, markup, tokens)
-      if markup =~ Syntax
-        @variable_name = $1
-        @collection_name = $2
-        @name = "#{$1}-#{$2}"
-        @reversed = $3
-        @attributes = {}
-        markup.scan(TagAttributes) do |key, value|
-          @attributes[key] = value
-        end
-      else
-        raise SyntaxError.new("Syntax Error in 'for loop' - Valid syntax: for [item] in [collection]")
-      end
-
+      parse_with_selected_parser(markup)
       @nodelist = @for_block = []
       super
     end
@@ -127,6 +115,43 @@ module Liquid
       result
     end
 
+    protected
+
+    def lax_parse(markup)
+      if markup =~ Syntax
+        @variable_name = $1
+        @collection_name = $2
+        @name = "#{$1}-#{$2}"
+        @reversed = $3
+        @attributes = {}
+        markup.scan(TagAttributes) do |key, value|
+          @attributes[key] = value
+        end
+      else
+        raise SyntaxError.new(options[:locale].t("errors.syntax.for"))
+      end
+    end
+
+    def strict_parse(markup)
+      p = Parser.new(markup)
+      @variable_name = p.consume(:id)
+      raise SyntaxError.new(options[:locale].t("errors.syntax.for_invalid_in"))  unless p.id?('in')
+      @collection_name = p.expression
+      @name = "#{@variable_name}-#{@collection_name}"
+      @reversed = p.id?('reversed')
+
+      @attributes = {}
+      while p.look(:id) && p.look(:colon, 1)
+        unless attribute = p.id?('limit') || p.id?('offset')
+          raise SyntaxError.new(options[:locale].t("errors.syntax.for_invalid_attribute"))
+        end
+        p.consume
+        val = p.expression
+        @attributes[attribute] = val
+      end
+      p.consume(:end_of_string)
+    end
+
     private
 
       def render_else(context)

lib/liquid/tags/if.rb

@@ -1,5 +1,4 @@
 module Liquid
-
   # If is the conditional block
   #
   #   {% if user.admin %}
@@ -10,18 +9,13 @@ module Liquid
   #
   #    There are {% if count < 5 %} less {% else %} more {% endif %} items than you need.
   #
-  #
   class If < Block
-    SyntaxHelp = "Syntax Error in tag 'if' - Valid syntax: if [expression]"
     Syntax = /(#{QuotedFragment})\s*([=!<>a-z_]+)?\s*(#{QuotedFragment})?/o
     ExpressionsAndOperators = /(?:\b(?:\s?and\s?|\s?or\s?)\b|(?:\s*(?!\b(?:\s?and\s?|\s?or\s?)\b)(?:#{QuotedFragment}|\S+)\s*)+)/o
-    BOOLEAN_OPERATORS = %w(and or)
 
     def initialize(tag_name, markup, tokens)
       @blocks = []
-
       push_block('if', markup)
-
       super
     end
 
@@ -50,29 +44,56 @@ module Liquid
         block = if tag == 'else'
           ElseCondition.new
         else
-
-          expressions = markup.scan(ExpressionsAndOperators).reverse
-          raise(SyntaxError, SyntaxHelp) unless expressions.shift =~ Syntax
-
-          condition = Condition.new($1, $2, $3)
-
-          while not expressions.empty?
-            operator = (expressions.shift).to_s.strip
-
-            raise(SyntaxError, SyntaxHelp) unless expressions.shift.to_s =~ Syntax
-
-            new_condition = Condition.new($1, $2, $3)
-            raise SyntaxError, "invalid boolean operator" unless BOOLEAN_OPERATORS.include?(operator)
-            new_condition.send(operator, condition)
-            condition = new_condition
-          end
-
-          condition
+          parse_with_selected_parser(markup)
         end
 
         @blocks.push(block)
         @nodelist = block.attach(Array.new)
       end
+
+      def lax_parse(markup)
+        expressions = markup.scan(ExpressionsAndOperators).reverse
+        raise(SyntaxError.new(options[:locale].t("errors.syntax.if"))) unless expressions.shift =~ Syntax
+
+        condition = Condition.new($1, $2, $3)
+
+        while not expressions.empty?
+          operator = (expressions.shift).to_s.strip
+
+          raise(SyntaxError.new(options[:locale].t("errors.syntax.if"))) unless expressions.shift.to_s =~ Syntax
+
+          new_condition = Condition.new($1, $2, $3)
+          new_condition.send(operator.to_sym, condition)
+          condition = new_condition
+        end
+
+        condition
+      end
+
+      def strict_parse(markup)
+        p = Parser.new(markup)
+
+        condition = parse_comparison(p)
+
+        while op = (p.id?('and') || p.id?('or'))
+          new_cond = parse_comparison(p)
+          new_cond.send(op, condition)
+          condition = new_cond
+        end
+        p.consume(:end_of_string)
+
+        condition
+      end
+
+      def parse_comparison(p)
+        a = p.expression
+        if op = p.consume?(:comparison)
+          b = p.expression
+          Condition.new(a, op, b)
+        else
+          Condition.new(a)
+        end
+      end
   end
 
   Template.register_tag('if', If)

lib/liquid/tags/include.rb

@@ -29,7 +29,7 @@ module Liquid
         end
 
       else
-        raise SyntaxError.new("Error in tag 'include' - Valid syntax: include '[template]' (with|for) [object|collection]")
+        raise SyntaxError.new(options[:locale].t("errors.syntax.include"))
       end
 
       super
@@ -38,6 +38,10 @@ module Liquid
     def parse(tokens)
     end
 
+    def blank?
+      false
+    end
+
     def render(context)
       partial = load_cached_partial(context)
       variable = context[@variable_name || @template_name[1..-2]]

lib/liquid/tags/increment.rb

@@ -1,12 +1,11 @@
 module Liquid
-
   # increment is used in a place where one needs to insert a counter
   #     into a template, and needs the counter to survive across
   #     multiple instantiations of the template.
   #     (To achieve the survival, the application must keep the context)
   #
   #     if the variable does not exist, it is created with value 0.
-
+  #
   #   Hello: {% increment variable %}
   #
   # gives you:
@@ -18,7 +17,6 @@ module Liquid
   class Increment < Tag
     def initialize(tag_name, markup, tokens)
       @variable = markup.strip
-
       super
     end
 
@@ -28,7 +26,9 @@ module Liquid
       value.to_s
     end
 
-    private
+    def blank?
+      false
+    end
   end
 
   Template.register_tag('increment', Increment)

lib/liquid/template.rb

@@ -14,6 +14,10 @@ module Liquid
   #   template.render('user_name' => 'bob')
   #
   class Template
+    DEFAULT_OPTIONS = {
+      :locale => I18n.new
+    }
+
     attr_accessor :root, :resource_limits
     @@file_system = BlankFileSystem.new
 
@@ -34,6 +38,18 @@ module Liquid
         @tags ||= {}
       end
 
+      # Sets how strict the parser should be.
+      # :lax acts like liquid 2.5 and silently ignores malformed tags in most cases.
+      # :warn is the default and will give deprecation warnings when invalid syntax is used.
+      # :strict will enforce correct syntax.
+      def error_mode=(mode)
+        @error_mode = mode
+      end
+
+      def error_mode
+        @error_mode || :lax
+      end
+
       # Pass a module with filter methods which should be available
       # to all liquid views. Good for registering the standard library
       def register_filter(mod)
@@ -41,9 +57,9 @@ module Liquid
       end
 
       # creates a new <tt>Template</tt> object from liquid source code
-      def parse(source)
+      def parse(source, options = {})
         template = Template.new
-        template.parse(source)
+        template.parse(source, options)
         template
       end
     end
@@ -55,11 +71,17 @@ module Liquid
 
     # Parse source code.
     # Returns self for easy chaining
-    def parse(source)
-      @root = Document.new(tokenize(source))
+    def parse(source, options = {})
+      @root = Document.new(tokenize(source), DEFAULT_OPTIONS.merge(options))
+      @warnings = nil
       self
     end
 
+    def warnings
+      return [] unless @root
+      @warnings ||= @root.warnings
+    end
+
     def registers
       @registers ||= {}
     end
@@ -101,7 +123,7 @@ module Liquid
       when nil
         Context.new(assigns, instance_assigns, registers, @rethrow_errors, @resource_limits)
       else
-        raise ArgumentError, "Expect Hash or Liquid::Context as parameter"
+        raise ArgumentError, "Expected Hash or Liquid::Context as parameter"
       end
 
       case args.last

lib/liquid/variable.rb

@@ -12,11 +12,30 @@ module Liquid
   #
   class Variable
     FilterParser = /(?:#{FilterSeparator}|(?:\s*(?:#{QuotedFragment}|#{ArgumentSeparator})\s*)+)/o
-    attr_accessor :filters, :name
+    EasyParse = /^ *(\w+(?:\.\w+)*) *$/
+    attr_accessor :filters, :name, :warnings
 
-    def initialize(markup)
+    def initialize(markup, options = {})
       @markup  = markup
       @name    = nil
+      @options = options || {}
+      
+
+      case @options[:error_mode] || Template.error_mode
+      when :strict then strict_parse(markup)
+      when :lax    then lax_parse(markup)
+      when :warn
+        begin
+          strict_parse(markup)
+        rescue SyntaxError => e
+          @warnings ||= []
+          @warnings << e
+          lax_parse(markup)
+        end
+      end
+    end
+
+    def lax_parse(markup)
       @filters = []
       if match = markup.match(/\s*(#{QuotedFragment})(.*)/o)
         @name = match[1]
@@ -33,6 +52,39 @@ module Liquid
       end
     end
 
+    def strict_parse(markup)
+      # Very simple valid cases
+      if markup =~ EasyParse
+        @name = $1
+        @filters = []
+        return
+      end
+
+      @filters = []
+      p = Parser.new(markup)
+      # Could be just filters with no input
+      @name = p.look(:pipe) ? '' : p.expression
+      while p.consume?(:pipe)
+        filtername = p.consume(:id)
+        filterargs = p.consume?(:colon) ? parse_filterargs(p) : []
+        @filters << [filtername, filterargs]
+      end
+      p.consume(:end_of_string)
+    rescue SyntaxError => e
+      e.message << " in \"{{#{markup}}}\""
+      raise e
+    end
+
+    def parse_filterargs(p)
+      # first argument
+      filterargs = [p.argument]
+      # followed by comma separated others
+      while p.consume?(:comma)
+        filterargs << p.argument
+      end
+      filterargs
+    end
+
     def render(context)
       return '' if @name.nil?
       @filters.inject(context[@name]) do |output, filter|

lib/liquid/version.rb

@@ -1,4 +1,4 @@
 # encoding: utf-8
 module Liquid
-  VERSION = "2.6.2"
+  VERSION = "2.6.0"
 end

liquid.gemspec

@@ -12,6 +12,7 @@ Gem::Specification.new do |s|
   s.authors     = ["Tobias Luetke"]
   s.email       = ["tobi@leetsoft.com"]
   s.homepage    = "http://www.liquidmarkup.org"
+  s.license     = "MIT"
   #s.description = "A secure, non-evaling end user template engine with aesthetic markup."
 
   s.required_rubygems_version = ">= 1.3.7"

performance/benchmark.rb

@@ -2,6 +2,7 @@ require 'rubygems'
 require 'benchmark'
 require File.dirname(__FILE__) + '/theme_runner'
 
+Liquid::Template.error_mode = ARGV.first.to_sym if ARGV.first
 profiler = ThemeRunner.new
 
 Benchmark.bmbm do |x|

performance/profile.rb

@@ -6,14 +6,14 @@ profiler = ThemeRunner.new
 
 puts 'Running profiler...'
 
-results  = profiler.run
+results  = profiler.run_profile
 
 puts 'Success'
 puts
 
-[RubyProf::FlatPrinter, RubyProf::GraphPrinter, RubyProf::GraphHtmlPrinter, RubyProf::CallTreePrinter].each do |klass|
+[RubyProf::FlatPrinter, RubyProf::GraphHtmlPrinter, RubyProf::CallTreePrinter, RubyProf::DotPrinter].each do |klass|
   filename = (ENV['TMP'] || '/tmp') + (klass.name.include?('Html') ? "/liquid.#{klass.name.downcase}.html" : "/callgrind.liquid.#{klass.name.downcase}.txt")
   filename.gsub!(/:+/, '_')
-  File.open(filename, "w+") { |fp| klass.new(results).print(fp, :print_file => true) }
+  File.open(filename, "w+") { |fp| klass.new(results).print(fp, :print_file => true, :min_percent => 3) }
   $stderr.puts "wrote #{klass.name} output to #{filename}"
 end

performance/tests/dropify/index.liquid

@@ -28,7 +28,7 @@
   {% else %}
     <div class="article-body textile">
     In <em>Admin &gt; Blogs &amp; Pages</em>, create a page with the handle <strong><code>frontpage</code></strong> and it will show up here.<br />
-    {{ "Learn more about handles" | link_to "http://wiki.shopify.com/Handle" }}
+    {{ "Learn more about handles" | link_to: "http://wiki.shopify.com/Handle" }}
     </div>
   {% endif %}
 

performance/tests/ripen/index.liquid

@@ -26,7 +26,7 @@
       {{ article.content }}
     {% else %}
       In <em>Admin &gt; Blogs &amp; Pages</em>, create a page with the handle <strong><code>frontpage</code></strong> and it will show up here.<br />
-      {{ "Learn more about handles" | link_to "http://wiki.shopify.com/Handle" }}
+      {{ "Learn more about handles" | link_to: "http://wiki.shopify.com/Handle" }}
     {% endif %}
   </div>
 </div>

performance/tests/vogue/index.liquid

@@ -5,7 +5,7 @@
       {{ article.content }}
     {% else %}
       In <em>Admin &gt; Blogs &amp; Pages</em>, create a page with the handle <strong><code>frontpage</code></strong> and it will show up here.<br />
-      {{ "Learn more about handles" | link_to "http://wiki.shopify.com/Handle" }}
+      {{ "Learn more about handles" | link_to: "http://wiki.shopify.com/Handle" }}
     {% endif %}
   </div>
 

test/fixtures/en_locale.yml

@@ -0,0 +1,9 @@
+---
+  simple: "less is more"
+  whatever: "something %{something}"
+  errors:
+    i18n:
+      undefined_interpolation: "undefined key %{key}"
+      unknown_translation: "translation '%{name}' wasn't found"
+    syntax:
+      oops: "something wasn't right"

test/liquid/assign_test.rb

@@ -18,4 +18,10 @@ class AssignTest < Test::Unit::TestCase
                            '{% assign foo = values | split: "," %}.{{ foo[1] }}.',
                            'values' => "foo,bar,baz")
   end
+
+  def test_assign_syntax_error
+    assert_match_syntax_error(/assign/,
+                       '{% assign foo not values %}.',
+                       'values' => "foo,bar,baz")
+  end
 end # AssignTest

test/liquid/blank_test.rb

@@ -0,0 +1,94 @@
+require 'test_helper'
+
+class BlankTestFileSystem
+  def read_template_file(template_path, context)
+    template_path
+  end
+end
+
+class BlankTest < Test::Unit::TestCase
+  include Liquid
+  N = 10
+
+  def wrap_in_for(body)
+    "{% for i in (1..#{N}) %}#{body}{% endfor %}"
+  end
+
+  def wrap_in_if(body)
+    "{% if true %}#{body}{% endif %}"
+  end
+
+  def wrap(body)
+    wrap_in_for(body) + wrap_in_if(body)
+  end
+
+  def test_loops_are_blank
+    assert_template_result("", wrap_in_for(" "))
+  end
+
+  def test_if_else_are_blank
+    assert_template_result("", "{% if true %} {% elsif false %} {% else %} {% endif %}")
+  end
+
+  def test_unless_is_blank
+    assert_template_result("", wrap("{% unless true %} {% endunless %}"))
+  end
+
+  def test_mark_as_blank_only_during_parsing
+    assert_template_result(" "*(N+1), wrap(" {% if false %} this never happens, but still, this block is not blank {% endif %}"))
+  end
+
+  def test_comments_are_blank
+    assert_template_result("", wrap(" {% comment %} whatever {% endcomment %} "))
+  end
+
+  def test_captures_are_blank
+    assert_template_result("", wrap(" {% capture foo %} whatever {% endcapture %} "))
+  end
+
+  def test_nested_blocks_are_blank_but_only_if_all_children_are
+    assert_template_result("", wrap(wrap(" ")))
+    assert_template_result("\n       but this is not "*(N+1),
+      wrap(%q{{% if true %} {% comment %} this is blank {% endcomment %} {% endif %}
+      {% if true %} but this is not {% endif %}}))
+  end
+
+  def test_assigns_are_blank
+    assert_template_result("", wrap(' {% assign foo = "bar" %} '))
+  end
+
+  def test_whitespace_is_blank
+    assert_template_result("", wrap(" "))
+    assert_template_result("", wrap("\t"))
+  end
+
+  def test_whitespace_is_not_blank_if_other_stuff_is_present
+    body = "     x "
+    assert_template_result(body*(N+1), wrap(body))
+  end
+
+  def test_increment_is_not_blank
+    assert_template_result(" 0"*2*(N+1), wrap("{% assign foo = 0 %} {% increment foo %} {% decrement foo %}"))
+  end
+
+  def test_cycle_is_not_blank
+    assert_template_result("  "*((N+1)/2)+" ", wrap("{% cycle ' ', ' ' %}"))
+  end
+
+  def test_raw_is_not_blank
+    assert_template_result("  "*(N+1), wrap(" {% raw %} {% endraw %}"))
+  end
+
+  def test_include_is_blank
+    Liquid::Template.file_system = BlankTestFileSystem.new
+    assert_equal "foobar"*(N+1), Template.parse(wrap("{% include 'foobar' %}")).render()
+    assert_equal " foobar "*(N+1), Template.parse(wrap("{% include ' foobar ' %}")).render()
+    assert_equal "   ", Template.parse(" {% include ' ' %} ").render()
+  end
+
+  def test_case_is_blank
+    assert_template_result("", wrap(" {% assign foo = 'bar' %} {% case foo %} {% when 'bar' %} {% when 'whatever' %} {% else %} {% endcase %} "))
+    assert_template_result("", wrap(" {% assign foo = 'else' %} {% case foo %} {% when 'bar' %} {% when 'whatever' %} {% else %} {% endcase %} "))
+    assert_template_result("   x  "*(N+1), wrap(" {% assign foo = 'else' %} {% case foo %} {% when 'bar' %} {% when 'whatever' %} {% else %} x {% endcase %} "))
+  end
+end

test/liquid/drop_test.rb

@@ -55,11 +55,44 @@ class ProductDrop < Liquid::Drop
 end
 
 class EnumerableDrop < Liquid::Drop
+  def before_method(method)
+    method
+  end
 
   def size
     3
   end
 
+  def first
+    1
+  end
+
+  def count
+    3
+  end
+
+  def min
+    1
+  end
+
+  def max
+    3
+  end
+
+  def each
+    yield 1
+    yield 2
+    yield 3
+  end
+end
+
+class RealEnumerableDrop < Liquid::Drop
+  include Enumerable
+
+  def before_method(method)
+    method
+  end
+
   def each
     yield 1
     yield 2
@@ -170,6 +203,33 @@ class DropsTest < Test::Unit::TestCase
     assert_equal '3', Liquid::Template.parse( '{{collection.size}}').render('collection' => EnumerableDrop.new)
   end
 
+  def test_enumerable_drop_will_invoke_before_method_for_clashing_method_names
+    ["select", "each", "map", "cycle"].each do |method|
+      assert_equal method.to_s, Liquid::Template.parse("{{collection.#{method}}}").render('collection' => EnumerableDrop.new)
+      assert_equal method.to_s, Liquid::Template.parse("{{collection[\"#{method}\"]}}").render('collection' => EnumerableDrop.new)
+      assert_equal method.to_s, Liquid::Template.parse("{{collection.#{method}}}").render('collection' => RealEnumerableDrop.new)
+      assert_equal method.to_s, Liquid::Template.parse("{{collection[\"#{method}\"]}}").render('collection' => RealEnumerableDrop.new)
+    end
+  end
+
+  def test_some_enumerable_methods_still_get_invoked
+    [ :count, :max ].each do |method|
+      assert_equal "3", Liquid::Template.parse("{{collection.#{method}}}").render('collection' => RealEnumerableDrop.new)
+      assert_equal "3", Liquid::Template.parse("{{collection[\"#{method}\"]}}").render('collection' => RealEnumerableDrop.new)
+      assert_equal "3", Liquid::Template.parse("{{collection.#{method}}}").render('collection' => EnumerableDrop.new)
+      assert_equal "3", Liquid::Template.parse("{{collection[\"#{method}\"]}}").render('collection' => EnumerableDrop.new)
+    end
+
+    assert_equal "yes", Liquid::Template.parse("{% if collection contains 3 %}yes{% endif %}").render('collection' => RealEnumerableDrop.new)
+
+    [ :min, :first ].each do |method|
+      assert_equal "1", Liquid::Template.parse("{{collection.#{method}}}").render('collection' => RealEnumerableDrop.new)
+      assert_equal "1", Liquid::Template.parse("{{collection[\"#{method}\"]}}").render('collection' => RealEnumerableDrop.new)
+      assert_equal "1", Liquid::Template.parse("{{collection.#{method}}}").render('collection' => EnumerableDrop.new)
+      assert_equal "1", Liquid::Template.parse("{{collection[\"#{method}\"]}}").render('collection' => EnumerableDrop.new)
+    end
+  end
+
   def test_empty_string_value_access
     assert_equal '', Liquid::Template.parse('{{ product[value] }}').render('product' => ProductDrop.new, 'value' => '')
   end

test/liquid/error_handling_test.rb

@@ -58,19 +58,48 @@ class ErrorHandlingTest < Test::Unit::TestCase
 
   def test_missing_endtag_parse_time_error
     assert_raise(Liquid::SyntaxError) do
-      template = Liquid::Template.parse(' {% for a in b %} ... ')
+      Liquid::Template.parse(' {% for a in b %} ... ')
     end
   end
 
   def test_unrecognized_operator
+    with_error_mode(:strict) do
+      assert_raise(SyntaxError) do
+        Liquid::Template.parse(' {% if 1 =! 2 %}ok{% endif %} ')
+      end
+    end
+  end
+  
+  def test_lax_unrecognized_operator
     assert_nothing_raised do
-      template = Liquid::Template.parse(' {% if 1 =! 2 %}ok{% endif %} ')
+      template = Liquid::Template.parse(' {% if 1 =! 2 %}ok{% endif %} ', :error_mode => :lax)
       assert_equal ' Liquid error: Unknown operator =! ', template.render
       assert_equal 1, template.errors.size
       assert_equal Liquid::ArgumentError, template.errors.first.class
     end
   end
 
+  def test_strict_error_messages
+    err = assert_raise(SyntaxError) do
+      Liquid::Template.parse(' {% if 1 =! 2 %}ok{% endif %} ', :error_mode => :strict)
+    end
+    assert_equal 'Unexpected character = in "1 =! 2"', err.message
+
+    err = assert_raise(SyntaxError) do
+      Liquid::Template.parse('{{%%%}}', :error_mode => :strict)
+    end
+    assert_equal 'Unexpected character % in "{{%%%}}"', err.message
+  end
+
+  def test_warnings
+    template = Liquid::Template.parse('{% if ~~~ %}{{%%%}}{% else %}{{ hello. }}{% endif %}', :error_mode => :warn)
+    assert_equal 3, template.warnings.size
+    assert_equal 'Unexpected character ~ in "~~~"', template.warnings[0].message
+    assert_equal 'Unexpected character % in "{{%%%}}"', template.warnings[1].message
+    assert_equal 'Expected id but found end_of_string in "{{ hello. }}"', template.warnings[2].message
+    assert_equal '', template.render
+  end
+
   # Liquid should not catch Exceptions that are not subclasses of StandardError, like Interrupt and NoMemoryError
   def test_exceptions_propagate
     assert_raise Exception do

test/liquid/i18n_test.rb

@@ -0,0 +1,37 @@
+require 'test_helper'
+
+class I18nTest < Test::Unit::TestCase
+  include Liquid
+
+  def setup
+    @i18n = I18n.new(fixture("en_locale.yml"))
+  end
+
+  def test_simple_translate_string
+    assert_equal "less is more", @i18n.translate("simple")
+  end
+
+  def test_nested_translate_string
+    assert_equal "something wasn't right", @i18n.translate("errors.syntax.oops")
+  end
+
+  def test_single_string_interpolation
+    assert_equal "something different", @i18n.translate("whatever", :something => "different")
+  end
+
+  # def test_raises_translation_error_on_undefined_interpolation_key
+  #   assert_raise I18n::TranslationError do
+  #     @i18n.translate("whatever", :oopstypos => "yes")
+  #   end
+  # end
+  
+  def test_raises_unknown_translation
+    assert_raise I18n::TranslationError do
+      @i18n.translate("doesnt_exist")
+    end
+  end
+
+  def test_sets_default_path_to_en
+    assert_equal I18n::DEFAULT_LOCALE, I18n.new.path
+  end
+end

test/liquid/lexer_test.rb

@@ -0,0 +1,48 @@
+require 'test_helper'
+
+class LexerTest < Test::Unit::TestCase
+  include Liquid
+
+  def test_strings
+    tokens = Lexer.new(%! 'this is a test""' "wat 'lol'"!).tokenize
+    assert_equal [[:string,%!'this is a test""'!], [:string, %!"wat 'lol'"!], [:end_of_string]], tokens
+  end
+
+  def test_integer
+    tokens = Lexer.new('hi 50').tokenize
+    assert_equal [[:id,'hi'], [:number, '50'], [:end_of_string]], tokens
+  end
+
+  def test_float
+    tokens = Lexer.new('hi 5.0').tokenize
+    assert_equal [[:id,'hi'], [:number, '5.0'], [:end_of_string]], tokens
+  end
+
+  def test_comparison
+    tokens = Lexer.new('== <> contains').tokenize
+    assert_equal [[:comparison,'=='], [:comparison, '<>'], [:comparison, 'contains'], [:end_of_string]], tokens
+  end
+
+  def test_specials
+    tokens = Lexer.new('| .:').tokenize
+    assert_equal [[:pipe, '|'], [:dot, '.'], [:colon, ':'], [:end_of_string]], tokens
+    tokens = Lexer.new('[,]').tokenize
+    assert_equal [[:open_square, '['], [:comma, ','], [:close_square, ']'], [:end_of_string]], tokens
+  end
+
+  def test_fancy_identifiers
+    tokens = Lexer.new('hi! five?').tokenize
+    assert_equal [[:id,'hi!'], [:id, 'five?'], [:end_of_string]], tokens
+  end
+
+  def test_whitespace
+    tokens = Lexer.new("five|\n\t ==").tokenize
+    assert_equal [[:id,'five'], [:pipe, '|'], [:comparison, '=='], [:end_of_string]], tokens
+  end
+
+  def test_unexpected_character
+    assert_raises(SyntaxError) do
+      Lexer.new("%").tokenize
+    end
+  end
+end

test/liquid/parser_test.rb

@@ -0,0 +1,74 @@
+require 'test_helper'
+
+class ParserTest < Test::Unit::TestCase
+  include Liquid
+
+  def test_consume
+    p = Parser.new("wat: 7")
+    assert_equal 'wat', p.consume(:id)
+    assert_equal ':', p.consume(:colon)
+    assert_equal '7', p.consume(:number)
+  end
+
+  def test_jump
+    p = Parser.new("wat: 7")
+    p.jump(2)
+    assert_equal '7', p.consume(:number)
+  end
+
+  def test_consume?
+    p = Parser.new("wat: 7")
+    assert_equal 'wat', p.consume?(:id)
+    assert_equal false, p.consume?(:dot)
+    assert_equal ':', p.consume(:colon)
+    assert_equal '7', p.consume?(:number)
+  end
+
+  def test_id?
+    p = Parser.new("wat 6 Peter Hegemon")
+    assert_equal 'wat', p.id?('wat')
+    assert_equal false, p.id?('endgame')
+    assert_equal '6', p.consume(:number)
+    assert_equal 'Peter', p.id?('Peter')
+    assert_equal false, p.id?('Achilles')
+  end
+
+  def test_look
+    p = Parser.new("wat 6 Peter Hegemon")
+    assert_equal true, p.look(:id)
+    assert_equal 'wat', p.consume(:id)
+    assert_equal false, p.look(:comparison)
+    assert_equal true, p.look(:number)
+    assert_equal true, p.look(:id, 1)
+    assert_equal false, p.look(:number, 1)
+  end
+
+  def test_expressions
+    p = Parser.new("hi.there hi[5].! hi.there.bob")
+    assert_equal 'hi.there', p.expression
+    assert_equal 'hi[5].!', p.expression
+    assert_equal 'hi.there.bob', p.expression
+
+    p = Parser.new("567 6.0 'lol' \"wut\"")
+    assert_equal '567', p.expression
+    assert_equal '6.0', p.expression
+    assert_equal "'lol'", p.expression
+    assert_equal '"wut"', p.expression
+  end
+
+  def test_arguments
+    p = Parser.new("filter: hi.there[5], keyarg: 7")
+    assert_equal 'filter', p.consume(:id)
+    assert_equal ':', p.consume(:colon)
+    assert_equal 'hi.there[5]', p.argument
+    assert_equal ',', p.consume(:comma)
+    assert_equal 'keyarg: 7', p.argument
+  end
+
+  def test_invalid_expression
+    assert_raises(SyntaxError) do
+      p = Parser.new("==")
+      p.expression
+    end
+  end
+end

test/liquid/parsing_quirks_test.rb

@@ -31,22 +31,60 @@ class ParsingQuirksTest < Test::Unit::TestCase
 
   def test_error_on_empty_filter
     assert_nothing_raised do
-      Template.parse("{{test |a|b|}}")
       Template.parse("{{test}}")
-      Template.parse("{{|test|}}")
+      Template.parse("{{|test}}")
+    end
+    with_error_mode(:strict) do
+      assert_raise(SyntaxError) do
+        Template.parse("{{test |a|b|}}")
+      end
     end
   end
 
-  def test_meaningless_parens
-    assigns = {'b' => 'bar', 'c' => 'baz'}
-    markup = "a == 'foo' or (b == 'bar' and c == 'baz') or false"
-    assert_template_result(' YES ',"{% if #{markup} %} YES {% endif %}", assigns)
+  def test_meaningless_parens_error
+    with_error_mode(:strict) do
+      assert_raise(SyntaxError) do
+        markup = "a == 'foo' or (b == 'bar' and c == 'baz') or false"
+        Template.parse("{% if #{markup} %} YES {% endif %}")
+      end
+    end
   end
 
-  def test_unexpected_characters_silently_eat_logic
-    markup = "true && false"
-    assert_template_result(' YES ',"{% if #{markup} %} YES {% endif %}")
-    markup = "false || true"
-    assert_template_result('',"{% if #{markup} %} YES {% endif %}")
+  def test_unexpected_characters_syntax_error
+    with_error_mode(:strict) do
+      assert_raise(SyntaxError) do
+        markup = "true && false"
+        Template.parse("{% if #{markup} %} YES {% endif %}")
+      end
+      assert_raise(SyntaxError) do
+        markup = "false || true"
+        Template.parse("{% if #{markup} %} YES {% endif %}")
+      end
+    end
+  end
+
+  def test_no_error_on_lax_empty_filter
+    assert_nothing_raised do
+      Template.parse("{{test |a|b|}}", :error_mode => :lax)
+      Template.parse("{{test}}", :error_mode => :lax)
+      Template.parse("{{|test|}}", :error_mode => :lax)
+    end
+  end
+
+  def test_meaningless_parens_lax
+    with_error_mode(:lax) do
+      assigns = {'b' => 'bar', 'c' => 'baz'}
+      markup = "a == 'foo' or (b == 'bar' and c == 'baz') or false"
+      assert_template_result(' YES ',"{% if #{markup} %} YES {% endif %}", assigns)
+    end
+  end
+
+  def test_unexpected_characters_silently_eat_logic_lax
+    with_error_mode(:lax) do
+      markup = "true && false"
+      assert_template_result(' YES ',"{% if #{markup} %} YES {% endif %}")
+      markup = "false || true"
+      assert_template_result('',"{% if #{markup} %} YES {% endif %}")
+    end
   end
 end # ParsingQuirksTest

test/liquid/standard_filter_test.rb

@@ -15,6 +15,10 @@ class TestThing
     "woot: #{@foo}"
   end
 
+  def [](whatever)
+    to_s
+  end
+
   def to_liquid
     @foo += 1
     self
@@ -27,6 +31,14 @@ class TestDrop < Liquid::Drop
   end
 end
 
+class TestEnumerable < Liquid::Drop
+  include Enumerable
+
+  def each(&block)
+    [ { "foo" => 1, "bar" => 2 }, { "foo" => 2, "bar" => 1 }, { "foo" => 3, "bar" => 3 } ].each(&block)
+  end
+end
+
 class StandardFiltersTest < Test::Unit::TestCase
   include Liquid
 
@@ -125,7 +137,12 @@ class StandardFiltersTest < Test::Unit::TestCase
 
   def test_map_calls_to_liquid
     t = TestThing.new
-    assert_equal "woot: 1", Liquid::Template.parse('{{ foo }}').render("foo" => t)
+    assert_equal "woot: 1", Liquid::Template.parse('{{ foo | map: "whatever" }}').render("foo" => [t])
+  end
+
+  def test_sort_calls_to_liquid
+    t = TestThing.new
+    assert_equal "woot: 1", Liquid::Template.parse('{{ foo | sort: "whatever" }}').render("foo" => [t])
   end
 
   def test_map_over_proc
@@ -135,6 +152,14 @@ class StandardFiltersTest < Test::Unit::TestCase
     assert_equal "testfoo", Liquid::Template.parse(templ).render("procs" => [p])
   end
 
+  def test_map_works_on_enumerables
+    assert_equal "123", Liquid::Template.parse('{{ foo | map: "foo" }}').render!("foo" => TestEnumerable.new)
+  end
+
+  def test_sort_works_on_enumerables
+    assert_equal "213", Liquid::Template.parse('{{ foo | sort: "bar" | map: "foo" }}').render!("foo" => TestEnumerable.new)
+  end
+
   def test_date
     assert_equal 'May', @filters.date(Time.parse("2006-05-05 10:00:00"), "%B")
     assert_equal 'June', @filters.date(Time.parse("2006-06-05 10:00:00"), "%B")

test/liquid/tags/if_else_tag_test.rb

@@ -157,10 +157,4 @@ class IfElseTagTest < Test::Unit::TestCase
     assert_template_result('yes',
                            %({% if 'gnomeslab-and-or-liquid' contains 'gnomeslab-and-or-liquid' %}yes{% endif %}))
   end
-
-  def test_operators_are_whitelisted
-    assert_raise(SyntaxError) do
-      assert_template_result('', %({% if 1 or throw or or 1 %}yes{% endif %}))
-    end
-  end
 end # IfElseTest

test/liquid/template_test.rb

@@ -143,4 +143,18 @@ class TemplateTest < Test::Unit::TestCase
     assert_equal 'bar', t.parse('{{bar}}').render(drop)
     assert_equal 'haha', t.parse("{{baz}}").render(drop)
   end
-end # TemplateTest
+
+  def test_sets_default_localization_in_document
+    t = Template.new
+    t.parse('')
+    assert_instance_of I18n, t.root.options[:locale]
+  end
+
+  def test_sets_default_localization_in_context_with_quick_initialization
+    t = Template.new
+    t.parse('{{foo}}', :locale => I18n.new(fixture("en_locale.yml")))
+
+    assert_instance_of I18n, t.root.options[:locale]
+    assert_equal fixture("en_locale.yml"), t.root.options[:locale].path
+  end
+end

test/liquid/variable_test.rb

@@ -73,8 +73,14 @@ class VariableTest < Test::Unit::TestCase
   end
 
   def test_symbol
-    var = Variable.new("http://disney.com/logo.gif | image: 'med' ")
-    assert_equal 'http://disney.com/logo.gif', var.name
+    var = Variable.new("http://disney.com/logo.gif | image: 'med' ", :error_mode => :lax)
+    assert_equal "http://disney.com/logo.gif", var.name
+    assert_equal [["image",["'med'"]]], var.filters
+  end
+
+  def test_string_to_filter
+    var = Variable.new("'http://disney.com/logo.gif' | image: 'med' ")
+    assert_equal "'http://disney.com/logo.gif'", var.name
     assert_equal [["image",["'med'"]]], var.filters
   end
 
@@ -115,10 +121,18 @@ class VariableTest < Test::Unit::TestCase
   end
 
   def test_lax_filter_argument_parsing
-    var = Variable.new(%! number_of_comments | pluralize: 'comment': 'comments' !)
+    var = Variable.new(%! number_of_comments | pluralize: 'comment': 'comments' !, :error_mode => :lax)
     assert_equal 'number_of_comments', var.name
     assert_equal [['pluralize',["'comment'","'comments'"]]], var.filters
   end
+
+  def test_strict_filter_argument_parsing
+    with_error_mode(:strict) do
+      assert_raises(SyntaxError) do
+        Variable.new(%! number_of_comments | pluralize: 'comment': 'comments' !)
+      end
+    end
+  end
 end
 
 

test/test_helper.rb

@@ -7,11 +7,26 @@ begin
 rescue LoadError
   puts "Couldn't load ruby-debug. gem install ruby-debug if you need it."
 end
-require File.join(File.dirname(__FILE__), '..', 'lib', 'liquid')
+
+$:.unshift(File.join(File.expand_path(File.dirname(__FILE__)), '..', 'lib'))
+require 'liquid.rb'
+
+mode = :strict
+if env_mode = ENV['LIQUID_PARSER_MODE']
+  puts "-- #{env_mode.upcase} ERROR MODE"
+  mode = env_mode.to_sym
+end
+Liquid::Template.error_mode = mode
 
 
 module Test
   module Unit
+    class TestCase
+      def fixture(name)
+        File.join(File.expand_path(File.dirname(__FILE__)), "fixtures", name)
+      end
+    end
+
     module Assertions
       include Liquid
 
@@ -24,6 +39,20 @@ module Test
 
         assert_match expected, Template.parse(template).render(assigns)
       end
+
+      def assert_match_syntax_error(match, template, registers = {})
+        exception = assert_raise(Liquid::SyntaxError) {
+          Template.parse(template).render(assigns)
+        }
+        assert_match match, exception.message
+      end
+
+      def with_error_mode(mode)
+        old_mode = Liquid::Template.error_mode
+        Liquid::Template.error_mode = mode
+        yield
+        Liquid::Template.error_mode = old_mode
+      end
     end # Assertions
   end # Unit
 end # Test