Also dump object distribution

Removed context from read_template_file, fixed tests to match new arity

.travis.yml

@@ -1,13 +1,17 @@
+language: ruby
+
 rvm:
-  - 1.9
   - 2.0
   - 2.1
-  - jruby-19mode
+  - 2.2
+  - ruby-head
   - jruby-head
   - rbx-2
+
+sudo: false
+
 matrix:
   allow_failures:
-    - rvm: rbx-2
     - rvm: jruby-head
 
 script: "rake test"

Gemfile

@@ -2,6 +2,7 @@ source 'https://rubygems.org'
 
 gemspec
 gem 'stackprof', platforms: :mri_21
+gem 'allocation_tracer', platforms: :mri_21
 
 group :test do
   gem 'spy', '0.4.1'

History.md

@@ -1,46 +1,69 @@
-# Liquid Version History
+# Liquid Change Log
 
-## 3.0.0 / not yet released / branch "master"
+## 4.0.0 / not yet released / branch "master"
+### Changed
+* Add forloop.parentloop as a reference to the parent loop (#520) [Justin Li, pushrax]
+* Block parsing moved to BlockBody class (#458) [Dylan Thacker-Smith, dylanahsmith]
+* Add concat filter to concatenate arrays (#429) [Diogo Beato, dvbeato]
+* Ruby 1.9 support dropped (#491) [Justin Li, pushrax]
+* Liquid::Template.file_system's read_template_file method is no longer passed the context. (#441) [James Reid-Smith, sunblaze]
+
+### Fixed
+* Fix capturing into variables with a hyphen in the name (#505) [Florian Weingarten, fw42]
+* Fix case sensitivity regression in date standard filter (#499) [Kelley Reynolds, kreynolds]
+* Disallow filters with no variable in strict mode (#475) [Justin Li, pushrax]
+* Disallow variable names in the strict parser that are not valid in the lax parser (#463) [Justin Li, pushrax]
+* Fix BlockBody#warnings taking exponential time to compute (#486) [Justin Li, pushrax]
+
+## 3.0.2 / 2015-04-24 / branch "3-0-stable"
+
+* Expose VariableLookup private members (#551) [Justin Li, pushrax]
+* Documentation fixes
+
+## 3.0.1 / 2015-01-23
+
+* Remove duplicate `index0` key in TableRow tag (#502) [Alfred Xing]
+
+## 3.0.0 / 2014-11-12
 
-* ...
 * Removed Block#end_tag. Instead, override parse with `super` followed by your code. See #446 [Dylan Thacker-Smith, dylanahsmith]
-* Fixed condition with wrong data types, see #423 [Bogdan Gusiev]
-* Add url_encode to standard filters, see #421 [Derrick Reimer, djreimer]
+* Fixed condition with wrong data types (#423) [Bogdan Gusiev]
+* Add url_encode to standard filters (#421) [Derrick Reimer, djreimer]
 * Add uniq to standard filters [Florian Weingarten, fw42]
-* Add exception_handler feature, see #397 and #254 [Bogdan Gusiev, bogdan and Florian Weingarten, fw42]
+* Add exception_handler feature (#397) and #254 [Bogdan Gusiev, bogdan and Florian Weingarten, fw42]
 * Optimize variable parsing to avoid repeated regex evaluation during template rendering #383 [Jason Hiltz-Laforge, jasonhl]
 * Optimize checking for block interrupts to reduce object allocation #380 [Jason Hiltz-Laforge, jasonhl] 
 * Properly set context rethrow_errors on render! #349 [Thierry Joyal, tjoyal]
-* Fix broken rendering of variables which are equal to false, see #345 [Florian Weingarten, fw42]
+* Fix broken rendering of variables which are equal to false (#345) [Florian Weingarten, fw42]
 * Remove ActionView template handler [Dylan Thacker-Smith, dylanahsmith]
-* Freeze lots of string literals for new Ruby 2.1 optimization, see #297 [Florian Weingarten, fw42]
-* Allow newlines in tags and variables, see #324 [Dylan Thacker-Smith, dylanahsmith]
+* Freeze lots of string literals for new Ruby 2.1 optimization (#297) [Florian Weingarten, fw42]
+* Allow newlines in tags and variables (#324) [Dylan Thacker-Smith, dylanahsmith]
 * Tag#parse is called after initialize, which now takes options instead of tokens as the 3rd argument. See #321 [Dylan Thacker-Smith, dylanahsmith]
 * Raise `Liquid::ArgumentError` instead of `::ArgumentError` when filter has wrong number of arguments #309 [Bogdan Gusiev, bogdan]
-* Add a to_s default for liquid drops, see #306 [Adam Doeler, releod]
+* Add a to_s default for liquid drops (#306) [Adam Doeler, releod]
 * Add strip, lstrip, and rstrip to standard filters [Florian Weingarten, fw42]
-* Make if, for & case tags return complete and consistent nodelists, see #250 [Nick Jones, dntj]
-* Prevent arbitrary method invocation on condition objects, see #274 [Dylan Thacker-Smith, dylanahsmith]
-* Don't call to_sym when creating conditions for security reasons, see #273 [Bouke van der Bijl, bouk]
-* Fix resource counting bug with respond_to?(:length), see #263 [Florian Weingarten, fw42]
-* Allow specifying custom patterns for template filenames, see #284 [Andrei Gladkyi, agladkyi]
-* Allow drops to optimize loading a slice of elements, see #282 [Tom Burns, boourns]
-* Support for passing variables to snippets in subdirs, see #271 [Joost Hietbrink, joost]
-* Add a class cache to avoid runtime extend calls, see #249 [James Tucker, raggi]
-* Remove some legacy Ruby 1.8 compatibility code, see #276 [Florian Weingarten, fw42]
-* Add default filter to standard filters, see #267 [Derrick Reimer, djreimer]
-* Add optional strict parsing and warn parsing, see #235 [Tristan Hume, trishume]
-* Add I18n syntax error translation, see #241 [Simon Hørup Eskildsen, Sirupsen]
-* Make sort filter work on enumerable drops, see #239 [Florian Weingarten, fw42]
-* Fix clashing method names in enumerable drops, see #238 [Florian Weingarten, fw42]
-* Make map filter work on enumerable drops, see #233 [Florian Weingarten, fw42]
+* Make if, for & case tags return complete and consistent nodelists (#250) [Nick Jones, dntj]
+* Prevent arbitrary method invocation on condition objects (#274) [Dylan Thacker-Smith, dylanahsmith]
+* Don't call to_sym when creating conditions for security reasons (#273) [Bouke van der Bijl, bouk]
+* Fix resource counting bug with respond_to?(:length) (#263) [Florian Weingarten, fw42]
+* Allow specifying custom patterns for template filenames (#284) [Andrei Gladkyi, agladkyi]
+* Allow drops to optimize loading a slice of elements (#282) [Tom Burns, boourns]
+* Support for passing variables to snippets in subdirs (#271) [Joost Hietbrink, joost]
+* Add a class cache to avoid runtime extend calls (#249) [James Tucker, raggi]
+* Remove some legacy Ruby 1.8 compatibility code (#276) [Florian Weingarten, fw42]
+* Add default filter to standard filters (#267) [Derrick Reimer, djreimer]
+* Add optional strict parsing and warn parsing (#235) [Tristan Hume, trishume]
+* Add I18n syntax error translation (#241) [Simon Hørup Eskildsen, Sirupsen]
+* Make sort filter work on enumerable drops (#239) [Florian Weingarten, fw42]
+* Fix clashing method names in enumerable drops (#238) [Florian Weingarten, fw42]
+* Make map filter work on enumerable drops (#233) [Florian Weingarten, fw42]
 * Improved whitespace stripping for blank blocks, related to #216 [Florian Weingarten, fw42]
 
 ## 2.6.1 / 2014-01-10 / branch "2-6-stable"
 
 Security fix, cherry-picked from master (4e14a65):
-* Don't call to_sym when creating conditions for security reasons, see #273 [Bouke van der Bijl, bouk]
-* Prevent arbitrary method invocation on condition objects, see #274 [Dylan Thacker-Smith, dylanahsmith]
+* Don't call to_sym when creating conditions for security reasons (#273) [Bouke van der Bijl, bouk]
+* Prevent arbitrary method invocation on condition objects (#274) [Dylan Thacker-Smith, dylanahsmith]
 
 ## 2.6.0 / 2013-11-25
 
@@ -69,12 +92,12 @@ The following releases will only be tested against Ruby 1.9 and Ruby 2.0 and are
 ## 2.5.5 / 2014-01-10 / branch "2-5-stable"
 
 Security fix, cherry-picked from master (4e14a65):
-* Don't call to_sym when creating conditions for security reasons, see #273 [Bouke van der Bijl, bouk]
-* Prevent arbitrary method invocation on condition objects, see #274 [Dylan Thacker-Smith, dylanahsmith]
+* Don't call to_sym when creating conditions for security reasons (#273) [Bouke van der Bijl, bouk]
+* Prevent arbitrary method invocation on condition objects (#274) [Dylan Thacker-Smith, dylanahsmith]
 
 ## 2.5.4 / 2013-11-11
 
-* Fix "can't convert Fixnum into String" for "replace", see #173, [wǒ_is神仙, jsw0528]
+* Fix "can't convert Fixnum into String" for "replace" (#173), [wǒ_is神仙, jsw0528]
 
 ## 2.5.3 / 2013-10-09
 

README.md

@@ -1,5 +1,5 @@
-[![Build Status](https://secure.travis-ci.org/Shopify/liquid.png?branch=master)](http://travis-ci.org/Shopify/liquid)
-[![Inline docs](http://inch-ci.org/github/Shopify/liquid.png)](http://inch-ci.org/github/Shopify/liquid)
+[![Build Status](https://api.travis-ci.org/Shopify/liquid.svg?branch=master)](http://travis-ci.org/Shopify/liquid)
+[![Inline docs](http://inch-ci.org/github/Shopify/liquid.svg?branch=master)](http://inch-ci.org/github/Shopify/liquid)
 
 # Liquid template engine
 

Rakefile

@@ -71,6 +71,20 @@ namespace :profile do
 
 end
 
+namespace :memory do
+
+  desc "Run the liquid memory tracer"
+  task :run do
+    ruby "./performance/memory.rb"
+  end
+
+  desc "Run the liquid memory tracer with strict parsing"
+  task :strict do
+    ruby "./performance/memory.rb strict"
+  end
+
+end
+
 desc "Run example"
 task :example do
   ruby "-w -d -Ilib example/server/server.rb"

example/server/example_servlet.rb

@@ -23,7 +23,7 @@ class Servlet < LiquidServlet
   end
 
   def products
-    { 'products' => products_list, 'description' => description, 'section' => 'Snowboards', 'cool_products' => true}
+    { 'products' => products_list, 'more_products' => more_products_list, 'description' => description, 'section' => 'Snowboards', 'cool_products' => true}
   end
 
   private
@@ -34,6 +34,11 @@ class Servlet < LiquidServlet
     {'name' => 'Arbor Diamond', 'price' => 59900, 'description' => 'the *arbor diamond* is a made up product because im obsessed with arbor and have no creativity'}]
   end
 
+  def more_products_list
+    [{'name' => 'Arbor Catalyst', 'price' => 39900, 'description' => 'the *arbor catalyst* is an advanced drop-through for freestyle and flatground performance and versatility' },
+    {'name' => 'Arbor Fish', 'price' => 40000, 'description' => 'the *arbor fish* is a compact pin that features an extended wheelbase and time-honored teardrop shape'}]
+  end
+
   def description
     "List of Products ~ This is a list of products with price and description."
   end

example/server/templates/products.liquid

@@ -16,12 +16,12 @@
   </head>
 
   <body>
-
+    {% assign all_products = products | concat: more_products  %}
     <h1>{{ description | split: '~' | first }}</h1>
 
     <h2>{{ description | split: '~' | last }}</h2>
 
-    <h2>There are currently {{products | count}} products in the {{section}} catalog</h2>
+    <h2>There are currently {{all_products | count}} products in the {{section}} catalog</h2>
 
     {% if cool_products %}
       Cool products :)
@@ -31,7 +31,7 @@
 
     <ul id="products">
 
-      {% for product in products %}
+      {% for product in all_products %}
         <li>
           <h2>{{product.name}}</h2>
           Only {{product.price | price }}

lib/liquid.rb

@@ -57,11 +57,13 @@ require 'liquid/context'
 require 'liquid/parser_switching'
 require 'liquid/tag'
 require 'liquid/block'
+require 'liquid/block_body'
 require 'liquid/document'
 require 'liquid/variable'
 require 'liquid/variable_lookup'
 require 'liquid/range_lookup'
 require 'liquid/file_system'
+require 'liquid/resource_limits'
 require 'liquid/template'
 require 'liquid/standardfilters'
 require 'liquid/condition'
@@ -72,6 +74,3 @@ require 'liquid/token'
 # Load all the tags of the standard library
 #
 Dir[File.dirname(__FILE__) + '/liquid/tags/*.rb'].each { |f| require f }
-
-require 'liquid/profiler'
-require 'liquid/profiler/hooks'

lib/liquid/block.rb

@@ -1,65 +1,26 @@
 module Liquid
   class Block < Tag
-    FullToken         = /\A#{TagStart}\s*(\w+)\s*(.*)?#{TagEnd}\z/om
-    ContentOfVariable = /\A#{VariableStart}(.*)#{VariableEnd}\z/om
-    TAGSTART = "{%".freeze
-    VARSTART = "{{".freeze
+    def initialize(tag_name, markup, options)
+      super
+      @blank = true
+    end
+
+    def parse(tokens)
+      @body = BlockBody.new
+      while more = parse_body(@body, tokens)
+      end
+    end
+
+    def render(context)
+      @body.render(context)
+    end
 
     def blank?
       @blank
     end
 
-    def parse(tokens)
-      @blank = true
-      @nodelist ||= []
-      @nodelist.clear
-
-      while token = tokens.shift
-        begin
-          unless token.empty?
-            case
-            when token.start_with?(TAGSTART)
-              if token =~ FullToken
-
-                # if we found the proper block delimiter just end parsing here and let the outer block
-                # proceed
-                return if block_delimiter == $1
-
-                # fetch the tag from registered blocks
-                if tag = Template.tags[$1]
-                  markup = token.is_a?(Token) ? token.child($2) : $2
-                  new_tag = tag.parse($1, markup, tokens, @options)
-                  new_tag.line_number = token.line_number if token.is_a?(Token)
-                  @blank &&= new_tag.blank?
-                  @nodelist << new_tag
-                else
-                  # this tag is not registered with the system
-                  # pass it to the current block for special handling or error reporting
-                  unknown_tag($1, $2, tokens)
-                end
-              else
-                raise SyntaxError.new(options[:locale].t("errors.syntax.tag_termination".freeze, :token => token, :tag_end => TagEnd.inspect))
-              end
-            when token.start_with?(VARSTART)
-              new_var = create_variable(token)
-              new_var.line_number = token.line_number if token.is_a?(Token)
-              @nodelist << new_var
-              @blank = false
-            else
-              @nodelist << token
-              @blank &&= (token =~ /\A\s*\z/)
-            end
-          end
-        rescue SyntaxError => e
-          e.set_line_number_from_token(token)
-          raise
-        end
-      end
-
-      # Make sure that it's ok to end parsing in the current block.
-      # Effectively this method will throw an exception unless the current block is
-      # of type Document
-      assert_missing_delimitation!
+    def nodelist
+      @body.nodelist
     end
 
     # warnings of this block and all sub-tags
@@ -96,65 +57,23 @@ module Liquid
       @block_delimiter ||= "end#{block_name}"
     end
 
-    def create_variable(token)
-      token.scan(ContentOfVariable) do |content|
-        markup = token.is_a?(Token) ? token.child(content.first) : content.first
-        return Variable.new(markup, @options)
-      end
-      raise SyntaxError.new(options[:locale].t("errors.syntax.variable_termination".freeze, :token => token, :tag_end => VariableEnd.inspect))
-    end
-
-    def render(context)
-      render_all(@nodelist, context)
-    end
-
     protected
 
-    def assert_missing_delimitation!
-      raise SyntaxError.new(options[:locale].t("errors.syntax.tag_never_closed".freeze, :block_name => block_name))
-    end
+    def parse_body(body, tokens)
+      body.parse(tokens, options) do |end_tag_name, end_tag_params|
+        @blank &&= body.blank?
 
-    def render_all(list, context)
-      output = []
-      context.resource_limits[:render_length_current] = 0
-      context.resource_limits[:render_score_current] += list.length
-
-      list.each do |token|
-        # Break out if we have any unhanded interrupts.
-        break if context.has_interrupt?
-
-        begin
-          # If we get an Interrupt that means the block must stop processing. An
-          # Interrupt is any command that stops block execution such as {% break %}
-          # or {% continue %}
-          if token.is_a? Continue or token.is_a? Break
-            context.push_interrupt(token.interrupt)
-            break
-          end
-
-          token_output = render_token(token, context)
-
-          unless token.is_a?(Block) && token.blank?
-            output << token_output
-          end
-        rescue MemoryError => e
-          raise e
-        rescue ::StandardError => e
-          output << (context.handle_error(e, token))
+        return false if end_tag_name == block_delimiter
+        unless end_tag_name
+          raise SyntaxError.new(@options[:locale].t("errors.syntax.tag_never_closed".freeze, :block_name => block_name))
         end
+
+        # this tag is not registered with the system
+        # pass it to the current block for special handling or error reporting
+        unknown_tag(end_tag_name, end_tag_params, tokens)
       end
 
-      output.join
-    end
-
-    def render_token(token, context)
-      token_output = (token.respond_to?(:render) ? token.render(context) : token)
-      context.increment_used_resources(:render_length_current, token_output)
-      if context.resource_limits_reached?
-        context.resource_limits[:reached] = true
-        raise MemoryError.new("Memory limits exceeded".freeze)
-      end
-      token_output
+      true
     end
   end
 end

lib/liquid/block_body.rb

@@ -0,0 +1,132 @@
+module Liquid
+  class BlockBody
+    FullToken = /\A#{TagStart}\s*(\w+)\s*(.*)?#{TagEnd}\z/om
+    ContentOfVariable = /\A#{VariableStart}(.*)#{VariableEnd}\z/om
+    TAGSTART = "{%".freeze
+    VARSTART = "{{".freeze
+
+    attr_reader :nodelist
+
+    def initialize
+      @nodelist = []
+      @blank = true
+    end
+
+    def parse(tokens, options)
+      while token = tokens.shift
+        begin
+          unless token.empty?
+            case
+            when token.start_with?(TAGSTART)
+              if token =~ FullToken
+                tag_name = $1
+                markup = $2
+                # fetch the tag from registered blocks
+                if tag = Template.tags[tag_name]
+                  markup = token.child(markup) if token.is_a?(Token)
+                  new_tag = tag.parse(tag_name, markup, tokens, options)
+                  new_tag.line_number = token.line_number if token.is_a?(Token)
+                  @blank &&= new_tag.blank?
+                  @nodelist << new_tag
+                else
+                  # end parsing if we reach an unknown tag and let the caller decide
+                  # determine how to proceed
+                  return yield tag_name, markup
+                end
+              else
+                raise_missing_tag_terminator(token, options)
+              end
+            when token.start_with?(VARSTART)
+              new_var = create_variable(token, options)
+              new_var.line_number = token.line_number if token.is_a?(Token)
+              @nodelist << new_var
+              @blank = false
+            else
+              @nodelist << token
+              @blank &&= !!(token =~ /\A\s*\z/)
+            end
+          end
+        rescue SyntaxError => e
+          e.set_line_number_from_token(token)
+          raise
+        end
+      end
+
+      yield nil, nil
+    end
+
+    def blank?
+      @blank
+    end
+
+    def warnings
+      all_warnings = []
+      nodelist.each do |node|
+        all_warnings.concat(node.warnings || []) if node.respond_to?(:warnings)
+      end
+      all_warnings
+    end
+
+    def render(context)
+      @output ||= []
+      @output.clear
+      context.resource_limits.render_score += @nodelist.length
+
+      @nodelist.each do |token|
+        # Break out if we have any unhanded interrupts.
+        break if context.has_interrupt?
+
+        begin
+          # If we get an Interrupt that means the block must stop processing. An
+          # Interrupt is any command that stops block execution such as {% break %}
+          # or {% continue %}
+          if token.is_a?(Continue) or token.is_a?(Break)
+            context.push_interrupt(token.interrupt)
+            break
+          end
+
+          token_output = render_token(token, context)
+
+          unless token.is_a?(Block) && token.blank?
+            @output << token_output
+          end
+        rescue MemoryError => e
+          raise e
+        rescue ::StandardError => e
+          @output << context.handle_error(e, token)
+        end
+      end
+
+      @output.join
+    end
+
+    private
+
+    def render_token(token, context)
+      token_output = (token.respond_to?(:render) ? token.render(context) : token)
+      token_str = token_output.is_a?(Array) ? token_output.join : token_output.to_s
+
+      context.resource_limits.render_length += token_str.length
+      if context.resource_limits.reached?
+        raise MemoryError.new("Memory limits exceeded".freeze)
+      end
+      token_str
+    end
+
+    def create_variable(token, options)
+      token.scan(ContentOfVariable) do |content|
+        markup = token.is_a?(Token) ? token.child(content.first) : content.first
+        return Variable.new(markup, options)
+      end
+      raise_missing_variable_terminator(token, options)
+    end
+
+    def raise_missing_tag_terminator(token, options)
+      raise SyntaxError.new(options[:locale].t("errors.syntax.tag_termination".freeze, :token => token, :tag_end => TagEnd.inspect))
+    end
+
+    def raise_missing_variable_terminator(token, options)
+      raise SyntaxError.new(options[:locale].t("errors.syntax.variable_termination".freeze, :token => token, :tag_end => VariableEnd.inspect))
+    end
+  end
+end

lib/liquid/context.rb

@@ -13,17 +13,14 @@ module Liquid
   #
   #   context['bob']  #=> nil  class Context
   class Context
-    attr_reader :scopes, :errors, :registers, :environments, :resource_limits
+    attr_reader :scopes, :registers, :environments, :resource_limits
     attr_accessor :exception_handler
 
     def initialize(environments = {}, outer_scope = {}, registers = {}, rethrow_errors = false, resource_limits = nil)
       @environments     = [environments].flatten
       @scopes           = [(outer_scope || {})]
       @registers        = registers
-      @errors           = []
-      @resource_limits  = resource_limits || Template.default_resource_limits.dup
-      @resource_limits[:render_score_current] = 0
-      @resource_limits[:assign_score_current] = 0
+      @resource_limits  = resource_limits || ResourceLimits.new(Template.default_resource_limits)
       squash_instance_assigns_with_environments
 
       @this_stack_used = false
@@ -32,22 +29,12 @@ module Liquid
         self.exception_handler = ->(e) { true }
       end
 
-      @interrupts = []
+      @interrupts = nil
       @filters = []
     end
 
-    def increment_used_resources(key, obj)
-      @resource_limits[key] += if obj.kind_of?(String) || obj.kind_of?(Array) || obj.kind_of?(Hash)
-        obj.length
-      else
-        1
-      end
-    end
-
-    def resource_limits_reached?
-      (@resource_limits[:render_length_limit] && @resource_limits[:render_length_current] > @resource_limits[:render_length_limit]) ||
-      (@resource_limits[:render_score_limit]  && @resource_limits[:render_score_current]  > @resource_limits[:render_score_limit] ) ||
-      (@resource_limits[:assign_score_limit]  && @resource_limits[:assign_score_current]  > @resource_limits[:assign_score_limit] )
+    def errors
+      @errors ||= []
     end
 
     def strainer
@@ -60,36 +47,23 @@ module Liquid
     # for that
     def add_filters(filters)
       filters = [filters].flatten.compact
-      filters.each do |f|
-        raise ArgumentError, "Expected module but got: #{f.class}" unless f.is_a?(Module)
-        Strainer.add_known_filter(f)
-      end
-
-      # If strainer is already setup then there's no choice but to use a runtime
-      # extend call. If strainer is not yet created, we can utilize strainers
-      # cached class based API, which avoids busting the method cache.
-      if @strainer
-        filters.each do |f|
-          strainer.extend(f)
-        end
-      else
-        @filters.concat filters
-      end
+      @filters += filters
+      @strainer = nil
     end
 
     # are there any not handled interrupts?
     def has_interrupt?
-      !@interrupts.empty?
+      @interrupts && !@interrupts.empty?
     end
 
     # push an interrupt to the stack. this interrupt is considered not handled.
     def push_interrupt(e)
-      @interrupts.push(e)
+      (@interrupts ||= []).push(e)
     end
 
     # pop an interrupt from the stack
     def pop_interrupt
-      @interrupts.pop
+      @interrupts.pop if @interrupts
     end
 
 

lib/liquid/document.rb

@@ -1,17 +1,24 @@
 module Liquid
-  class Document < Block
-    def self.parse(tokens, options={})
-      # we don't need markup to open this block
-      super(nil, nil, tokens, options)
+  class Document < BlockBody
+    def self.parse(tokens, options)
+      doc = new
+      doc.parse(tokens, options)
+      doc
     end
 
-    # There isn't a real delimiter
-    def block_delimiter
-      []
+    def parse(tokens, options)
+      super do |end_tag_name, end_tag_params|
+        unknown_tag(end_tag_name, options) if end_tag_name
+      end
     end
 
-    # Document blocks don't need to be terminated since they are not actually opened
-    def assert_missing_delimitation!
+    def unknown_tag(tag, options)
+      case tag
+      when 'else'.freeze, 'end'.freeze
+        raise SyntaxError.new(options[:locale].t("errors.syntax.unexpected_outer_tag".freeze, :tag => tag))
+      else
+        raise SyntaxError.new(options[:locale].t("errors.syntax.unknown_tag".freeze, :tag => tag))
+      end
     end
   end
 end

lib/liquid/drop.rb

@@ -62,6 +62,10 @@ module Liquid
 
     # Check for method existence without invoking respond_to?, which creates symbols
     def self.invokable?(method_name)
+      self.invokable_methods.include?(method_name.to_s)
+    end
+
+    def self.invokable_methods
       unless @invokable_methods
         blacklist = Liquid::Drop.public_instance_methods + [:each]
         if include?(Enumerable)
@@ -71,7 +75,7 @@ module Liquid
         whitelist = [:to_liquid] + (public_instance_methods - blacklist)
         @invokable_methods = Set.new(whitelist.map(&:to_s))
       end
-      @invokable_methods.include?(method_name.to_s)
+      @invokable_methods
     end
   end
 end

lib/liquid/file_system.rb

@@ -14,7 +14,7 @@ module Liquid
   # This will parse the template with a LocalFileSystem implementation rooted at 'template_path'.
   class BlankFileSystem
     # Called by Liquid to retrieve a template file
-    def read_template_file(template_path, context)
+    def read_template_file(template_path)
       raise FileSystemError, "This liquid context does not allow includes."
     end
   end
@@ -49,7 +49,7 @@ module Liquid
       @pattern = pattern
     end
 
-    def read_template_file(template_path, context)
+    def read_template_file(template_path)
       full_path = full_path(template_path)
       raise FileSystemError, "No such template '#{template_path}'" unless File.exists?(full_path)
 

lib/liquid/lexer.rb

@@ -13,7 +13,7 @@ module Liquid
       '?'.freeze => :question,
       '-'.freeze => :dash
     }
-    IDENTIFIER = /\w+/
+    IDENTIFIER = /[a-zA-Z_][\w-]*\??/
     SINGLE_STRING_LITERAL = /'[^\']*'/
     DOUBLE_STRING_LITERAL = /"[^\"]*"/
     NUMBER_LITERAL = /-?\d+(\.\d+)?/

lib/liquid/locales/en.yml

@@ -14,7 +14,8 @@
       include: "Error in tag 'include' - Valid syntax: include '[template]' (with|for) [object|collection]"
       unknown_tag: "Unknown tag '%{tag}'"
       invalid_delimiter: "'end' is not a valid delimiter for %{block_name} tags. use %{block_delimiter}"
-      unexpected_else: "%{block_name} tag does not expect else tag"
+      unexpected_else: "%{block_name} tag does not expect 'else' tag"
+      unexpected_outer_tag: "Unexpected outer '%{tag}' tag"
       tag_termination: "Tag '%{token}' was not properly terminated with regexp: %{tag_end}"
       variable_termination: "Variable '%{token}' was not properly terminated with regexp: %{tag_end}"
       tag_never_closed: "'%{block_name}' tag was never closed"

lib/liquid/module_ex.rb

@@ -7,16 +7,17 @@
 # to the allowed method passed with the liquid_methods call
 # Example:
 #
-# class SomeClass
-#   liquid_methods :an_allowed_method
+#   class SomeClass
+#     liquid_methods :an_allowed_method
 #
-#   def an_allowed_method
-#     'this comes from an allowed method'
+#     def an_allowed_method
+#       'this comes from an allowed method'
+#     end
+#
+#     def unallowed_method
+#       'this will never be an output'
+#     end
 #   end
-#   def unallowed_method
-#     'this will never be an output'
-#   end
-# end
 #
 # if you want to extend the drop to other methods you can defines more methods
 # in the class <YourClass>::LiquidDropClass
@@ -26,31 +27,33 @@
 #       'and this from another allowed method'
 #     end
 #   end
-# end
+#
 #
 # usage:
-# @something = SomeClass.new
+#   @something = SomeClass.new
 #
 # template:
-# {{something.an_allowed_method}}{{something.unallowed_method}} {{something.another_allowed_method}}
+#   {{something.an_allowed_method}}{{something.unallowed_method}} {{something.another_allowed_method}}
 #
 # output:
-# 'this comes from an allowed method and this from another allowed method'
+#   'this comes from an allowed method and this from another allowed method'
 #
 # You can also chain associations, by adding the liquid_method call in the
 # association models.
 #
 class Module
-
   def liquid_methods(*allowed_methods)
     drop_class = eval "class #{self.to_s}::LiquidDropClass < Liquid::Drop; self; end"
+    
     define_method :to_liquid do
       drop_class.new(self)
     end
+    
     drop_class.class_eval do
       def initialize(object)
         @object = object
       end
+      
       allowed_methods.each do |sym|
         define_method sym do
           @object.send sym
@@ -58,5 +61,4 @@ class Module
       end
     end
   end
-
 end

lib/liquid/parser.rb

@@ -75,13 +75,6 @@ module Liquid
 
     def variable_signature
       str = consume(:id)
-      while consume?(:dash)
-        str << "-".freeze
-        str << consume(:id)
-      end
-      if consume?(:question)
-        str << "?".freeze
-      end
       if look(:open_square)
         str << consume
         str << expression

lib/liquid/profiler.rb

@@ -1,9 +1,12 @@
+require 'liquid/profiler/hooks'
+
 module Liquid
 
   # Profiler enables support for profiling template rendering to help track down performance issues.
   #
-  # To enable profiling, pass the <tt>profile: true</tt> option to <tt>Liquid::Template.parse</tt>. Then, after
-  # <tt>Liquid::Template#render</tt> is called, the template object makes available an instance of this
+  # To enable profiling, first require 'liquid/profiler'.
+  # Then, to profile a parse/render cycle, pass the <tt>profile: true</tt> option to <tt>Liquid::Template.parse</tt>.
+  # After <tt>Liquid::Template#render</tt> is called, the template object makes available an instance of this
   # class via the <tt>Liquid::Template#profiler</tt> method.
   #
   #   template = Liquid::Template.parse(template_content, profile: true)

lib/liquid/profiler/hooks.rb

@@ -1,5 +1,5 @@
 module Liquid
-  class Block < Tag
+  class BlockBody
     def render_token_with_profiling(token, context)
       Profiler.profile_token_render(token) do
         render_token_without_profiling(token, context)

lib/liquid/resource_limits.rb

@@ -0,0 +1,23 @@
+module Liquid
+  class ResourceLimits
+    attr_accessor :render_length, :render_score, :assign_score,
+                  :render_length_limit, :render_score_limit, :assign_score_limit
+
+    def initialize(limits)
+      @render_length_limit = limits[:render_length_limit]
+      @render_score_limit = limits[:render_score_limit]
+      @assign_score_limit = limits[:assign_score_limit]
+      reset
+    end
+
+    def reached?
+      (@render_length_limit && @render_length > @render_length_limit) ||
+      (@render_score_limit  && @render_score  > @render_score_limit ) ||
+      (@assign_score_limit  && @assign_score  > @assign_score_limit )
+    end
+
+    def reset
+      @render_length = @render_score = @assign_score = 0
+    end
+  end
+end

lib/liquid/standardfilters.rb

@@ -177,6 +177,10 @@ module Liquid
       input.to_s + string.to_s
     end
 
+    def concat(input, array)
+      InputIterator.new(input).concat(array)
+    end
+
     # prepend a string to another
     def prepend(input, string)
       string.to_s + input.to_s
@@ -306,6 +310,8 @@ module Liquid
     def to_date(obj)
       return obj if obj.respond_to?(:strftime)
 
+      obj = obj.downcase if obj.is_a?(String)
+
       case obj
       when 'now'.freeze, 'today'.freeze
         Time.now
@@ -344,6 +350,10 @@ module Liquid
         to_a.join(glue)
       end
 
+      def concat(args)
+        to_a.concat args
+      end
+
       def reverse
         reverse_each.to_a
       end

lib/liquid/strainer.rb

@@ -8,12 +8,13 @@ module Liquid
   # The Strainer only allows method calls defined in filters given to it via Strainer.global_filter,
   # Context#add_filters or Template.register_filter
   class Strainer #:nodoc:
-    @@filters = []
-    @@known_filters = Set.new
-    @@known_methods = Set.new
+    @@global_strainer = Class.new(Strainer) do
+      @filter_methods = Set.new
+    end
     @@strainer_class_cache = Hash.new do |hash, filters|
-      hash[filters] = Class.new(Strainer) do
-        filters.each { |f| include f }
+      hash[filters] = Class.new(@@global_strainer) do
+        @filter_methods = @@global_strainer.filter_methods.dup
+        filters.each { |f| add_filter(f) }
       end
     end
 
@@ -21,33 +22,32 @@ module Liquid
       @context = context
     end
 
-    def self.global_filter(filter)
-      raise ArgumentError, "Passed filter is not a module" unless filter.is_a?(Module)
-      add_known_filter(filter)
-      @@filters << filter unless @@filters.include?(filter)
+    def self.filter_methods
+      @filter_methods
     end
 
-    def self.add_known_filter(filter)
-      unless @@known_filters.include?(filter)
-        @@method_blacklist ||= Set.new(Strainer.instance_methods.map(&:to_s))
-        new_methods = filter.instance_methods.map(&:to_s)
-        new_methods.reject!{ |m| @@method_blacklist.include?(m) }
-        @@known_methods.merge(new_methods)
-        @@known_filters.add(filter)
+    def self.add_filter(filter)
+      raise ArgumentError, "Expected module but got: #{f.class}" unless filter.is_a?(Module)
+      unless self.class.include?(filter)
+        self.send(:include, filter)
+        @filter_methods.merge(filter.public_instance_methods.map(&:to_s))
       end
     end
 
-    def self.strainer_class_cache
-      @@strainer_class_cache
+    def self.global_filter(filter)
+      @@global_strainer.add_filter(filter)
+    end
+
+    def self.invokable?(method)
+      @filter_methods.include?(method.to_s)
     end
 
     def self.create(context, filters = [])
-      filters = @@filters + filters
-      strainer_class_cache[filters].new(context)
+      @@strainer_class_cache[filters].new(context)
     end
 
     def invoke(method, *args)
-      if invokable?(method)
+      if self.class.invokable?(method)
         send(method, *args)
       else
         args.first
@@ -55,9 +55,5 @@ module Liquid
     rescue ::ArgumentError => e
       raise Liquid::ArgumentError.new(e.message)
     end
-
-    def invokable?(method)
-      @@known_methods.include?(method.to_s) && respond_to?(method)
-    end
   end
 end

lib/liquid/tags/assign.rb

@@ -25,7 +25,10 @@ module Liquid
     def render(context)
       val = @from.render(context)
       context.scopes.last[@to] = val
-      context.increment_used_resources(:assign_score_current, val)
+
+      inc = val.instance_of?(String) || val.instance_of?(Array) || val.instance_of?(Hash) ? val.length : 1
+      context.resource_limits.assign_score += inc
+
       ''.freeze
     end
 

lib/liquid/tags/capture.rb

@@ -11,7 +11,7 @@ module Liquid
   # in a sidebar or footer.
   #
   class Capture < Block
-    Syntax = /(\w+)/
+    Syntax = /(#{VariableSignature}+)/o
 
     def initialize(tag_name, markup, options)
       super
@@ -25,7 +25,7 @@ module Liquid
     def render(context)
       output = super
       context.scopes.last[@to] = output
-      context.increment_used_resources(:assign_score_current, output)
+      context.resource_limits.assign_score += output.length
       ''.freeze
     end
 

lib/liquid/tags/case.rb

@@ -14,12 +14,18 @@ module Liquid
       end
     end
 
+    def parse(tokens)
+      body = BlockBody.new
+      while more = parse_body(body, tokens)
+        body = @blocks.last.attachment
+      end
+    end
+
     def nodelist
-      @blocks.flat_map(&:attachment)
+      @blocks.map(&:attachment)
     end
 
     def unknown_tag(tag, markup, tokens)
-      @nodelist = []
       case tag
       when 'when'.freeze
         record_when_condition(markup)
@@ -37,10 +43,10 @@ module Liquid
         output = ''
         @blocks.each do |block|
           if block.else?
-            return render_all(block.attachment, context) if execute_else_block
+            return block.attachment.render(context) if execute_else_block
           elsif block.evaluate(context)
             execute_else_block = false
-            output << render_all(block.attachment, context)
+            output << block.attachment.render(context)
           end
         end
         output
@@ -50,8 +56,9 @@ module Liquid
     private
 
     def record_when_condition(markup)
+      body = BlockBody.new
+
       while markup
-        # Create a new nodelist and assign it to the new block
         if not markup =~ WhenSyntax
           raise SyntaxError.new(options[:locale].t("errors.syntax.case_invalid_when".freeze))
         end
@@ -59,8 +66,8 @@ module Liquid
         markup = $2
 
         block = Condition.new(@left, '=='.freeze, Expression.parse($1))
-        block.attach(@nodelist)
-        @blocks.push(block)
+        block.attach(body)
+        @blocks << block
       end
     end
 
@@ -70,7 +77,7 @@ module Liquid
       end
 
       block = ElseCondition.new
-      block.attach(@nodelist)
+      block.attach(BlockBody.new)
       @blocks << block
     end
   end

lib/liquid/tags/for.rb

@@ -42,6 +42,7 @@ module Liquid
   #                   where 0 is the last item.
   # forloop.first:: Returns true if the item is the first item.
   # forloop.last:: Returns true if the item is the last item.
+  # forloop.parentloop:: Provides access to the parent loop, if present.
   #
   class For < Block
     Syntax = /\A(#{VariableSegment}+)\s+in\s+(#{QuotedFragment}+)\s*(reversed)?/o
@@ -49,20 +50,22 @@ module Liquid
     def initialize(tag_name, markup, options)
       super
       parse_with_selected_parser(markup)
-      @nodelist = @for_block = []
+      @for_block = BlockBody.new
+    end
+
+    def parse(tokens)
+      if more = parse_body(@for_block, tokens)
+        parse_body(@else_block, tokens)
+      end
     end
 
     def nodelist
-      if @else_block
-        @for_block + @else_block
-      else
-        @for_block
-      end
+      @else_block ? [@for_block, @else_block] : [@for_block]
     end
 
     def unknown_tag(tag, markup, tokens)
       return super unless tag == 'else'.freeze
-      @nodelist = @else_block = []
+      @else_block = BlockBody.new
     end
 
     def render(context)
@@ -96,21 +99,24 @@ module Liquid
       # Store our progress through the collection for the continue flag
       context.registers[:for][@name] = from + segment.length
 
+      parent_loop = context['forloop'.freeze]
+
       context.stack do
         segment.each_with_index do |item, index|
           context[@variable_name] = item
           context['forloop'.freeze] = {
-            'name'.freeze    => @name,
-            'length'.freeze  => length,
-            'index'.freeze   => index + 1,
-            'index0'.freeze  => index,
-            'rindex'.freeze  => length - index,
-            'rindex0'.freeze => length - index - 1,
-            'first'.freeze   => (index == 0),
-            'last'.freeze    => (index == length - 1)
+            'name'.freeze       => @name,
+            'length'.freeze     => length,
+            'index'.freeze      => index + 1,
+            'index0'.freeze     => index,
+            'rindex'.freeze     => length - index,
+            'rindex0'.freeze    => length - index - 1,
+            'first'.freeze      => (index == 0),
+            'last'.freeze       => (index == length - 1),
+            'parentloop'.freeze => parent_loop
           }
 
-          result << render_all(@for_block, context)
+          result << @for_block.render(context)
 
           # Handle any interrupts if they exist.
           if context.has_interrupt?
@@ -175,7 +181,7 @@ module Liquid
     end
 
     def render_else(context)
-      return @else_block ? [render_all(@else_block, context)] : ''.freeze
+      @else_block ? @else_block.render(context) : ''.freeze
     end
 
     def iterable?(collection)

lib/liquid/tags/if.rb

@@ -20,8 +20,13 @@ module Liquid
       push_block('if'.freeze, markup)
     end
 
+    def parse(tokens)
+      while more = parse_body(@blocks.last.attachment, tokens)
+      end
+    end
+
     def nodelist
-      @blocks.flat_map(&:attachment)
+      @blocks.map(&:attachment)
     end
 
     def unknown_tag(tag, markup, tokens)
@@ -36,7 +41,7 @@ module Liquid
       context.stack do
         @blocks.each do |block|
           if block.evaluate(context)
-            return render_all(block.attachment, context)
+            return block.attachment.render(context)
           end
         end
         ''.freeze
@@ -53,7 +58,7 @@ module Liquid
         end
 
         @blocks.push(block)
-        @nodelist = block.attach(Array.new)
+        block.attach(BlockBody.new)
       end
 
       def lax_parse(markup)

lib/liquid/tags/include.rb

@@ -81,15 +81,7 @@ module Liquid
       def read_template_from_file_system(context)
         file_system = context.registers[:file_system] || Liquid::Template.file_system
 
-        # make read_template_file call backwards-compatible.
-        case file_system.method(:read_template_file).arity
-        when 1
-          file_system.read_template_file(context.evaluate(@template_name))
-        when 2
-          file_system.read_template_file(context.evaluate(@template_name), context)
-        else
-          raise ArgumentError, "file_system.read_template_file expects two parameters: (template_name, context)"
-        end
+        file_system.read_template_file(context.evaluate(@template_name))
       end
 
       def pass_options

lib/liquid/tags/raw.rb

@@ -3,16 +3,27 @@ module Liquid
     FullTokenPossiblyInvalid = /\A(.*)#{TagStart}\s*(\w+)\s*(.*)?#{TagEnd}\z/om
 
     def parse(tokens)
-      @nodelist ||= []
-      @nodelist.clear
+      @body = ''
       while token = tokens.shift
         if token =~ FullTokenPossiblyInvalid
-          @nodelist << $1 if $1 != "".freeze
+          @body << $1 if $1 != "".freeze
           return if block_delimiter == $2
         end
-        @nodelist << token if not token.empty?
+        @body << token if not token.empty?
       end
     end
+
+    def render(context)
+      @body
+    end
+
+    def nodelist
+      [@body]
+    end
+
+    def blank?
+      @body.empty?
+    end
   end
 
   Template.register_tag('raw'.freeze, Raw)

lib/liquid/tags/table_row.rb

@@ -42,7 +42,6 @@ module Liquid
             'index0'.freeze    => index,
             'col'.freeze       => col + 1,
             'col0'.freeze      => col,
-            'index0'.freeze    => index,
             'rindex'.freeze    => length - index,
             'rindex0'.freeze   => length - index - 1,
             'first'.freeze     => (index == 0),

lib/liquid/tags/unless.rb

@@ -3,7 +3,7 @@ require File.dirname(__FILE__) + '/if'
 module Liquid
   # Unless is a conditional just like 'if' but works on the inverse logic.
   #
-  #   {% unless x < 0 %} x is greater than zero {% end %}
+  #   {% unless x < 0 %} x is greater than zero {% endunless %}
   #
   class Unless < If
     def render(context)
@@ -12,13 +12,13 @@ module Liquid
         # First condition is interpreted backwards ( if not )
         first_block = @blocks.first
         unless first_block.evaluate(context)
-          return render_all(first_block.attachment, context)
+          return first_block.attachment.render(context)
         end
 
         # After the first condition unless works just like if
         @blocks[1..-1].each do |block|
           if block.evaluate(context)
-            return render_all(block.attachment, context)
+            return block.attachment.render(context)
           end
         end
 

lib/liquid/template.rb

@@ -18,7 +18,9 @@ module Liquid
       :locale => I18n.new
     }
 
-    attr_accessor :root, :resource_limits
+    attr_accessor :root
+    attr_reader :resource_limits
+
     @@file_system = BlankFileSystem.new
 
     class TagRegistry
@@ -110,7 +112,7 @@ module Liquid
     end
 
     def initialize
-      @resource_limits = self.class.default_resource_limits.dup
+      @resource_limits = ResourceLimits.new(self.class.default_resource_limits)
     end
 
     # Parse source code.
@@ -203,6 +205,9 @@ module Liquid
         context.add_filters(args.pop)
       end
 
+      # Retrying a render resets resource usage
+      context.resource_limits.reset
+
       begin
         # render the nodelist.
         # for performance reasons we get an array back here. join will make a string out of it.
@@ -250,6 +255,8 @@ module Liquid
 
     def with_profiling
       if @profiling && !@options[:included]
+        raise "Profiler not loaded, require 'liquid/profiler' first" unless defined?(Liquid::Profiler)
+
         @profiler = Profiler.new
         @profiler.start
 

lib/liquid/variable.rb

@@ -12,7 +12,6 @@ module Liquid
   #
   class Variable
     FilterParser = /(?:\s+|#{QuotedFragment}|#{ArgumentSeparator})+/o
-    EasyParse = /\A *(\w+(?:\.\w+)*) *\z/
     attr_accessor :filters, :name, :warnings
     attr_accessor :line_number
     include ParserSwitching
@@ -34,7 +33,8 @@ module Liquid
     end
 
     def lax_parse(markup)
-      @filters = []
+      @filters ||= []
+      @filters.clear
       if markup =~ /(#{QuotedFragment})(.*)/om
         name_markup = $1
         filter_markup = $2
@@ -53,17 +53,11 @@ module Liquid
     end
 
     def strict_parse(markup)
-      # Very simple valid cases
-      if markup =~ EasyParse
-        @name = Expression.parse($1)
-        @filters = []
-        return
-      end
-
-      @filters = []
+      @filters ||= []
+      @filters.clear
       p = Parser.new(markup)
-      # Could be just filters with no input
-      @name = p.look(:pipe) ? nil : Expression.parse(p.expression)
+
+      @name = Expression.parse(p.expression)
       while p.consume?(:pipe)
         filtername = p.consume(:id)
         filterargs = p.consume?(:colon) ? parse_filterargs(p) : []

lib/liquid/variable_lookup.rb

@@ -3,26 +3,27 @@ module Liquid
     SQUARE_BRACKETED = /\A\[(.*)\]\z/m
     COMMAND_METHODS = ['size'.freeze, 'first'.freeze, 'last'.freeze]
 
+    attr_reader :name, :lookups
+
     def self.parse(markup)
       new(markup)
     end
 
     def initialize(markup)
-      lookups = markup.scan(VariableParser)
+      @lookups = markup.scan(VariableParser)
 
-      name = lookups.shift
+      name = @lookups.shift
       if name =~ SQUARE_BRACKETED
         name = Expression.parse($1)
       end
       @name = name
 
-      @lookups = lookups
       @command_flags = 0
 
       @lookups.each_index do |i|
-        lookup = lookups[i]
+        lookup = @lookups[i]
         if lookup =~ SQUARE_BRACKETED
-          lookups[i] = Expression.parse($1)
+          @lookups[i] = Expression.parse($1)
         elsif COMMAND_METHODS.include?(lookup)
           @command_flags |= 1 << i
         end

lib/liquid/version.rb

@@ -1,4 +1,4 @@
 # encoding: utf-8
 module Liquid
-  VERSION = "3.0.0"
+  VERSION = "4.0.0.alpha"
 end

liquid.gemspec

@@ -9,7 +9,7 @@ Gem::Specification.new do |s|
   s.version     = Liquid::VERSION
   s.platform    = Gem::Platform::RUBY
   s.summary     = "A secure, non-evaling end user template engine with aesthetic markup."
-  s.authors     = ["Tobias Luetke"]
+  s.authors     = ["Tobias Lütke"]
   s.email       = ["tobi@leetsoft.com"]
   s.homepage    = "http://www.liquidmarkup.org"
   s.license     = "MIT"

performance/memory.rb

@@ -0,0 +1,19 @@
+at_exit do
+  p 'Objects distribution:'
+  require 'pp'
+  pp ObjectSpace.count_objects
+end
+
+require 'allocation_tracer' rescue fail("install allocation_tracer extension/gem")
+require File.dirname(__FILE__) + '/theme_runner'
+
+Liquid::Template.error_mode = ARGV.first.to_sym if ARGV.first
+profiler = ThemeRunner.new
+
+require 'allocation_tracer/trace'
+
+puts "Profiling memory usage..."
+
+200.times do
+  profiler.run
+end

performance/profile.rb

@@ -8,10 +8,17 @@ profiler.run
 [:cpu, :object].each do |profile_type|
   puts "Profiling in #{profile_type.to_s} mode..."
   results = StackProf.run(mode: profile_type) do
-    100.times do
+    200.times do
       profiler.run
     end
   end
+
+  if profile_type == :cpu && graph_filename = ENV['GRAPH_FILENAME']
+    File.open(graph_filename, 'w') do |f|
+      StackProf::Report.new(results).print_graphviz(nil, f)
+    end
+  end
+
   StackProf::Report.new(results).print_text(false, 20)
   File.write(ENV['FILENAME'] + "." + profile_type.to_s, Marshal.dump(results)) if ENV['FILENAME']
 end

performance/theme_runner.rb

@@ -17,7 +17,7 @@ class ThemeRunner
     end
 
     # Called by Liquid to retrieve a template file
-    def read_template_file(template_path, context)
+    def read_template_file(template_path)
       File.read(@path + '/' + template_path + '.liquid')
     end
   end

test/integration/assign_test.rb

@@ -3,6 +3,16 @@ require 'test_helper'
 class AssignTest < Minitest::Test
   include Liquid
 
+  def test_assign_with_hyphen_in_variable_name
+    template_source = <<-END_TEMPLATE
+    {% assign this-thing = 'Print this-thing' %}
+    {{ this-thing }}
+    END_TEMPLATE
+    template = Template.parse(template_source)
+    rendered = template.render!
+    assert_equal "Print this-thing", rendered.strip
+  end
+
   def test_assigned_variable
     assert_template_result('.foo.',
                            '{% assign foo = values %}.{{ foo[0] }}.',

test/integration/blank_test.rb

@@ -9,7 +9,7 @@ class FoobarTag < Liquid::Tag
 end
 
 class BlankTestFileSystem
-  def read_template_file(template_path, context)
+  def read_template_file(template_path)
     template_path
   end
 end

test/integration/capture_test.rb

@@ -7,6 +7,16 @@ class CaptureTest < Minitest::Test
     assert_template_result("test string", "{% capture 'var' %}test string{% endcapture %}{{var}}", {})
   end
 
+  def test_capture_with_hyphen_in_variable_name
+    template_source = <<-END_TEMPLATE
+    {% capture this-thing %}Print this-thing{% endcapture %}
+    {{ this-thing }}
+    END_TEMPLATE
+    template = Template.parse(template_source)
+    rendered = template.render!
+    assert_equal "Print this-thing", rendered.strip
+  end
+
   def test_capture_to_variable_from_outer_scope_if_existing
     template_source = <<-END_TEMPLATE
     {% assign var = '' %}

test/integration/document_test.rb

@@ -0,0 +1,19 @@
+require 'test_helper'
+
+class DocumentTest < Minitest::Test
+  include Liquid
+
+  def test_unexpected_outer_tag
+    exc = assert_raises(SyntaxError) do
+      Template.parse("{% else %}")
+    end
+    assert_equal exc.message, "Liquid syntax error: Unexpected outer 'else' tag"
+  end
+
+  def test_unknown_tag
+    exc = assert_raises(SyntaxError) do
+      Template.parse("{% foo %}")
+    end
+    assert_equal exc.message, "Liquid syntax error: Unknown tag 'foo'"
+  end
+end

test/integration/filter_test.rb

@@ -25,6 +25,12 @@ end
 class FiltersTest < Minitest::Test
   include Liquid
 
+  module OverrideObjectMethodFilter
+    def tap(input)
+      "tap overridden"
+    end
+  end
+
   def setup
     @context = Context.new
   end
@@ -105,6 +111,13 @@ class FiltersTest < Minitest::Test
     output = Variable.new(%! 'hello %{first_name}, %{last_name}' | substitute: first_name: surname, last_name: 'doe' !).render(@context)
     assert_equal 'hello john, doe', output
   end
+
+  def test_override_object_method_in_filter
+    assert_equal "tap overridden", Template.parse("{{var | tap}}").render!({ 'var' => 1000 }, :filters => [OverrideObjectMethodFilter])
+
+    # tap still treated as a non-existent filter
+    assert_equal "1000", Template.parse("{{var | tap}}").render!({ 'var' => 1000 })
+  end
 end
 
 class FiltersInTemplate < Minitest::Test

test/integration/parsing_quirks_test.rb

@@ -28,11 +28,14 @@ class ParsingQuirksTest < Minitest::Test
 
   def test_error_on_empty_filter
     assert Template.parse("{{test}}")
-    assert Template.parse("{{|test}}")
+
+    with_error_mode(:lax) do
+      assert Template.parse("{{|test}}")
+    end
+
     with_error_mode(:strict) do
-      assert_raises(SyntaxError) do
-        Template.parse("{{test |a|b|}}")
-      end
+      assert_raises(SyntaxError) { Template.parse("{{|test}}") }
+      assert_raises(SyntaxError) { Template.parse("{{test |a|b|}}") }
     end
   end
 
@@ -100,4 +103,17 @@ class ParsingQuirksTest < Minitest::Test
     end
   end
 
+  def test_invalid_variables_work
+    with_error_mode(:lax) do
+      assert_template_result('bar', "{% assign 123foo = 'bar' %}{{ 123foo }}")
+      assert_template_result('123', "{% assign 123 = 'bar' %}{{ 123 }}")
+    end
+  end
+
+  def test_extra_dots_in_ranges
+    with_error_mode(:lax) do
+      assert_template_result('12345', "{% for i in (1...5) %}{{ i }}{% endfor %}")
+    end
+  end
+
 end # ParsingQuirksTest

test/integration/render_profiling_test.rb

@@ -4,7 +4,7 @@ class RenderProfilingTest < Minitest::Test
   include Liquid
 
   class ProfilingFileSystem
-    def read_template_file(template_path, context)
+    def read_template_file(template_path)
       "Rendering template {% assign template_name = '#{template_path}'%}\n{{ template_name }}"
     end
   end

test/integration/standard_filter_test.rb

@@ -249,6 +249,7 @@ class StandardFiltersTest < Minitest::Test
     assert_equal "07/16/2004", @filters.date("Fri Jul 16 01:00:00 2004", "%m/%d/%Y")
     assert_equal "#{Date.today.year}", @filters.date('now', '%Y')
     assert_equal "#{Date.today.year}", @filters.date('today', '%Y')
+    assert_equal "#{Date.today.year}", @filters.date('Today', '%Y')
 
     assert_equal nil, @filters.date(nil, "%B")
 
@@ -358,6 +359,17 @@ class StandardFiltersTest < Minitest::Test
     assert_template_result('bcd',"{{ a | append: b}}",assigns)
   end
 
+  def test_concat
+    assert_equal [1, 2, 3, 4], @filters.concat([1, 2], [3, 4])
+    assert_equal [1, 2, 'a'],  @filters.concat([1, 2], ['a'])
+    assert_equal [1, 2, 10],   @filters.concat([1, 2], [10])
+
+    assert_raises(TypeError) do
+      # no implicit conversion of Fixnum into Array
+      @filters.concat([1, 2], 10)
+    end
+  end
+
   def test_prepend
     assigns = {'a' => 'bc', 'b' => 'a' }
     assert_template_result('abc',"{{ a | prepend: 'a'}}",assigns)

test/integration/tags/for_tag_test.rb

@@ -298,6 +298,22 @@ HERE
                 'string' => "test string")
   end
 
+  def test_for_parentloop_references_parent_loop
+    assert_template_result('1.1 1.2 1.3 2.1 2.2 2.3 ',
+                           '{% for inner in outer %}{% for k in inner %}' +
+                           '{{ forloop.parentloop.index }}.{{ forloop.index }} ' +
+                           '{% endfor %}{% endfor %}',
+                           'outer' => [[1, 1, 1], [1, 1, 1]])
+  end
+
+  def test_for_parentloop_nil_when_not_present
+    assert_template_result('.1 .2 ',
+                           '{% for inner in outer %}' +
+                           '{{ forloop.parentloop.index }}.{{ forloop.index }} ' +
+                           '{% endfor %}',
+                           'outer' => [[1, 1, 1], [1, 1, 1]])
+  end
+
   def test_blank_string_not_iterable
     assert_template_result('', "{% for char in characters %}I WILL NOT BE OUTPUT{% endfor %}", 'characters' => '')
   end

test/integration/tags/include_tag_test.rb

@@ -1,7 +1,7 @@
 require 'test_helper'
 
 class TestFileSystem
-  def read_template_file(template_path, context)
+  def read_template_file(template_path)
     case template_path
     when "product"
       "Product: {{ product.title }} "
@@ -37,14 +37,14 @@ class TestFileSystem
 end
 
 class OtherFileSystem
-  def read_template_file(template_path, context)
+  def read_template_file(template_path)
     'from OtherFileSystem'
   end
 end
 
 class CountingFileSystem
   attr_reader :count
-  def read_template_file(template_path, context)
+  def read_template_file(template_path)
     @count ||= 0
     @count += 1
     'from CountingFileSystem'
@@ -132,7 +132,7 @@ class IncludeTagTest < Minitest::Test
   def test_recursively_included_template_does_not_produce_endless_loop
 
     infinite_file_system = Class.new do
-      def read_template_file(template_path, context)
+      def read_template_file(template_path)
         "-{% include 'loop' %}"
       end
     end
@@ -145,18 +145,6 @@ class IncludeTagTest < Minitest::Test
 
   end
 
-  def test_backwards_compatability_support_for_overridden_read_template_file
-    infinite_file_system = Class.new do
-      def read_template_file(template_path) # testing only one argument here.
-        "- hi mom"
-      end
-    end
-
-    Liquid::Template.file_system = infinite_file_system.new
-
-    Template.parse("{% include 'hi_mom' %}").render!
-  end
-
   def test_dynamically_choosen_template
     assert_template_result "Test123", "{% include template %}", "template" => 'Test123'
     assert_template_result "Test321", "{% include template %}", "template" => 'Test321'

test/integration/template_test.rb

@@ -1,4 +1,5 @@
 require 'test_helper'
+require 'timeout'
 
 class TemplateContextDrop < Liquid::Drop
   def before_method(method)
@@ -37,6 +38,16 @@ class TemplateTest < Minitest::Test
     assert_equal 'from instance assigns', t.parse("{{ foo }}").render!
   end
 
+  def test_warnings_is_not_exponential_time
+    str = "false"
+    100.times do
+      str = "{% if true %}true{% else %}#{str}{% endif %}"
+    end
+
+    t = Template.parse(str)
+    assert_equal [], Timeout::timeout(1) { t.warnings }
+  end
+
   def test_instance_assigns_persist_on_same_template_parsing_between_renders
     t = Template.new.parse("{{ foo }}{% assign foo = 'foo' %}{{ foo }}")
     assert_equal 'foo', t.render!
@@ -82,69 +93,92 @@ class TemplateTest < Minitest::Test
 
   def test_resource_limits_works_with_custom_length_method
     t = Template.parse("{% assign foo = bar %}")
-    t.resource_limits = { :render_length_limit => 42 }
+    t.resource_limits.render_length_limit = 42
     assert_equal "", t.render!("bar" => SomethingWithLength.new)
   end
 
   def test_resource_limits_render_length
     t = Template.parse("0123456789")
-    t.resource_limits = { :render_length_limit => 5 }
+    t.resource_limits.render_length_limit = 5
     assert_equal "Liquid error: Memory limits exceeded", t.render()
-    assert t.resource_limits[:reached]
-    t.resource_limits = { :render_length_limit => 10 }
+    assert t.resource_limits.reached?
+
+    t.resource_limits.render_length_limit = 10
     assert_equal "0123456789", t.render!()
-    refute_nil t.resource_limits[:render_length_current]
+    refute_nil t.resource_limits.render_length
   end
 
   def test_resource_limits_render_score
     t = Template.parse("{% for a in (1..10) %} {% for a in (1..10) %} foo {% endfor %} {% endfor %}")
-    t.resource_limits = { :render_score_limit => 50 }
+    t.resource_limits.render_score_limit = 50
     assert_equal "Liquid error: Memory limits exceeded", t.render()
-    assert t.resource_limits[:reached]
+    assert t.resource_limits.reached?
+
     t = Template.parse("{% for a in (1..100) %} foo {% endfor %}")
-    t.resource_limits = { :render_score_limit => 50 }
+    t.resource_limits.render_score_limit = 50
     assert_equal "Liquid error: Memory limits exceeded", t.render()
-    assert t.resource_limits[:reached]
-    t.resource_limits = { :render_score_limit => 200 }
+    assert t.resource_limits.reached?
+
+    t.resource_limits.render_score_limit = 200
     assert_equal (" foo " * 100), t.render!()
-    refute_nil t.resource_limits[:render_score_current]
+    refute_nil t.resource_limits.render_score
   end
 
   def test_resource_limits_assign_score
     t = Template.parse("{% assign foo = 42 %}{% assign bar = 23 %}")
-    t.resource_limits = { :assign_score_limit => 1 }
+    t.resource_limits.assign_score_limit = 1
     assert_equal "Liquid error: Memory limits exceeded", t.render()
-    assert t.resource_limits[:reached]
-    t.resource_limits = { :assign_score_limit => 2 }
+    assert t.resource_limits.reached?
+
+    t.resource_limits.assign_score_limit = 2
     assert_equal "", t.render!()
-    refute_nil t.resource_limits[:assign_score_current]
+    refute_nil t.resource_limits.assign_score
   end
 
   def test_resource_limits_aborts_rendering_after_first_error
     t = Template.parse("{% for a in (1..100) %} foo1 {% endfor %} bar {% for a in (1..100) %} foo2 {% endfor %}")
-    t.resource_limits = { :render_score_limit => 50 }
+    t.resource_limits.render_score_limit = 50
     assert_equal "Liquid error: Memory limits exceeded", t.render()
-    assert t.resource_limits[:reached]
+    assert t.resource_limits.reached?
   end
 
   def test_resource_limits_hash_in_template_gets_updated_even_if_no_limits_are_set
     t = Template.parse("{% for a in (1..100) %} {% assign foo = 1 %} {% endfor %}")
     t.render!()
-    assert t.resource_limits[:assign_score_current] > 0
-    assert t.resource_limits[:render_score_current] > 0
-    assert t.resource_limits[:render_length_current] > 0
+    assert t.resource_limits.assign_score > 0
+    assert t.resource_limits.render_score > 0
+    assert t.resource_limits.render_length > 0
+  end
+
+  def test_render_length_persists_between_blocks
+    t = Template.parse("{% if true %}aaaa{% endif %}")
+    t.resource_limits.render_length_limit = 7
+    assert_equal "Liquid error: Memory limits exceeded", t.render()
+    t.resource_limits.render_length_limit = 8
+    assert_equal "aaaa", t.render()
+
+    t = Template.parse("{% if true %}aaaa{% endif %}{% if true %}bbb{% endif %}")
+    t.resource_limits.render_length_limit = 13
+    assert_equal "Liquid error: Memory limits exceeded", t.render()
+    t.resource_limits.render_length_limit = 14
+    assert_equal "aaaabbb", t.render()
+
+    t = Template.parse("{% if true %}a{% endif %}{% if true %}b{% endif %}{% if true %}a{% endif %}{% if true %}b{% endif %}{% if true %}a{% endif %}{% if true %}b{% endif %}")
+    t.resource_limits.render_length_limit = 5
+    assert_equal "Liquid error: Memory limits exceeded", t.render()
+    t.resource_limits.render_length_limit = 11
+    assert_equal "Liquid error: Memory limits exceeded", t.render()
+    t.resource_limits.render_length_limit = 12
+    assert_equal "ababab", t.render()
   end
 
   def test_default_resource_limits_unaffected_by_render_with_context
     context = Context.new
     t = Template.parse("{% for a in (1..100) %} {% assign foo = 1 %} {% endfor %}")
     t.render!(context)
-    assert context.resource_limits[:assign_score_current] > 0
-    assert context.resource_limits[:render_score_current] > 0
-    assert context.resource_limits[:render_length_current] > 0
-    refute Template.default_resource_limits.key?(:assign_score_current)
-    refute Template.default_resource_limits.key?(:render_score_current)
-    refute Template.default_resource_limits.key?(:render_length_current)
+    assert context.resource_limits.assign_score > 0
+    assert context.resource_limits.render_score > 0
+    assert context.resource_limits.render_length > 0
   end
 
   def test_can_use_drop_as_context

test/test_helper.rb

@@ -1,10 +1,12 @@
 #!/usr/bin/env ruby
 
+ENV["MT_NO_EXPECTATIONS"] = "1"
 require 'minitest/autorun'
 require 'spy/integration'
 
 $:.unshift(File.join(File.expand_path(File.dirname(__FILE__)), '..', 'lib'))
 require 'liquid.rb'
+require 'liquid/profiler'
 
 mode = :strict
 if env_mode = ENV['LIQUID_PARSER_MODE']
@@ -48,13 +50,19 @@ module Minitest
     end
 
     def with_global_filter(*globals)
-      original_filters = Array.new(Liquid::Strainer.class_variable_get(:@@filters))
+      original_global_strainer = Liquid::Strainer.class_variable_get(:@@global_strainer)
+      Liquid::Strainer.class_variable_set(:@@global_strainer, Class.new(Liquid::Strainer) do
+        @filter_methods = Set.new
+      end)
+      Liquid::Strainer.class_variable_get(:@@strainer_class_cache).clear
+
       globals.each do |global|
         Liquid::Template.register_filter(global)
       end
       yield
     ensure
-      Liquid::Strainer.class_variable_set(:@@filters, original_filters)
+      Liquid::Strainer.class_variable_get(:@@strainer_class_cache).clear
+      Liquid::Strainer.class_variable_set(:@@global_strainer, original_global_strainer)
     end
 
     def with_taint_mode(mode)

test/unit/context_unit_test.rb

@@ -465,6 +465,7 @@ class ContextUnitTest < Minitest::Test
     mock_empty = Spy.on_instance_method(Array, :empty?)
     mock_has_interrupt = Spy.on(@context, :has_interrupt?).and_call_through
 
+    @context.push_interrupt(StandardError.new)
     @context.has_interrupt?
 
     refute mock_any.has_been_called?

test/unit/file_system_unit_test.rb

@@ -5,7 +5,7 @@ class FileSystemUnitTest < Minitest::Test
 
   def test_default
     assert_raises(FileSystemError) do
-      BlankFileSystem.new.read_template_file("dummy", {'dummy'=>'smarty'})
+      BlankFileSystem.new.read_template_file("dummy")
     end
   end
 

test/unit/lexer_unit_test.rb

@@ -32,7 +32,10 @@ class LexerUnitTest < Minitest::Test
 
   def test_fancy_identifiers
     tokens = Lexer.new('hi five?').tokenize
-    assert_equal [[:id,'hi'], [:id, 'five'], [:question, '?'], [:end_of_string]], tokens
+    assert_equal [[:id, 'hi'], [:id, 'five?'], [:end_of_string]], tokens
+
+    tokens = Lexer.new('2foo').tokenize
+    assert_equal [[:number, '2'], [:id, 'foo'], [:end_of_string]], tokens
   end
 
   def test_whitespace

test/unit/strainer_unit_test.rb

@@ -31,11 +31,11 @@ class StrainerUnitTest < Minitest::Test
 
   def test_strainer_only_invokes_public_filter_methods
     strainer = Strainer.create(nil)
-    assert_equal false, strainer.invokable?('__test__')
-    assert_equal false, strainer.invokable?('test')
-    assert_equal false, strainer.invokable?('instance_eval')
-    assert_equal false, strainer.invokable?('__send__')
-    assert_equal true, strainer.invokable?('size') # from the standard lib
+    assert_equal false, strainer.class.invokable?('__test__')
+    assert_equal false, strainer.class.invokable?('test')
+    assert_equal false, strainer.class.invokable?('instance_eval')
+    assert_equal false, strainer.class.invokable?('__send__')
+    assert_equal true, strainer.class.invokable?('size') # from the standard lib
   end
 
   def test_strainer_returns_nil_if_no_filter_method_found
@@ -63,9 +63,7 @@ class StrainerUnitTest < Minitest::Test
     assert_kind_of Strainer, strainer
     assert_kind_of a, strainer
     assert_kind_of b, strainer
-    Strainer.class_variable_get(:@@filters).each do |m|
-      assert_kind_of m, strainer
-    end
+    assert_kind_of Liquid::StandardFilters, strainer
   end
 
 end # StrainerTest

test/unit/tags/case_tag_unit_test.rb

@@ -5,6 +5,6 @@ class CaseTagUnitTest < Minitest::Test
 
   def test_case_nodelist
     template = Liquid::Template.parse('{% case var %}{% when true %}WHEN{% else %}ELSE{% endcase %}')
-    assert_equal ['WHEN', 'ELSE'], template.root.nodelist[0].nodelist
+    assert_equal ['WHEN', 'ELSE'], template.root.nodelist[0].nodelist.map(&:nodelist).flatten
   end
 end

test/unit/tags/for_tag_unit_test.rb

@@ -3,11 +3,11 @@ require 'test_helper'
 class ForTagUnitTest < Minitest::Test
   def test_for_nodelist
     template = Liquid::Template.parse('{% for item in items %}FOR{% endfor %}')
-    assert_equal ['FOR'], template.root.nodelist[0].nodelist
+    assert_equal ['FOR'], template.root.nodelist[0].nodelist.map(&:nodelist).flatten
   end
 
   def test_for_else_nodelist
     template = Liquid::Template.parse('{% for item in items %}FOR{% else %}ELSE{% endfor %}')
-    assert_equal ['FOR', 'ELSE'], template.root.nodelist[0].nodelist
+    assert_equal ['FOR', 'ELSE'], template.root.nodelist[0].nodelist.map(&:nodelist).flatten
   end
 end

test/unit/tags/if_tag_unit_test.rb

@@ -3,6 +3,6 @@ require 'test_helper'
 class IfTagUnitTest < Minitest::Test
   def test_if_nodelist
     template = Liquid::Template.parse('{% if true %}IF{% else %}ELSE{% endif %}')
-    assert_equal ['IF', 'ELSE'], template.root.nodelist[0].nodelist
+    assert_equal ['IF', 'ELSE'], template.root.nodelist[0].nodelist.map(&:nodelist).flatten
   end
 end

test/unit/template_unit_test.rb

@@ -5,16 +5,17 @@ class TemplateUnitTest < Minitest::Test
 
   def test_sets_default_localization_in_document
     t = Template.new
-    t.parse('')
-    assert_instance_of I18n, t.root.options[:locale]
+    t.parse('{%comment%}{%endcomment%}')
+    assert_instance_of I18n, t.root.nodelist[0].options[:locale]
   end
 
   def test_sets_default_localization_in_context_with_quick_initialization
     t = Template.new
-    t.parse('{{foo}}', :locale => I18n.new(fixture("en_locale.yml")))
+    t.parse('{%comment%}{%endcomment%}', :locale => I18n.new(fixture("en_locale.yml")))
 
-    assert_instance_of I18n, t.root.options[:locale]
-    assert_equal fixture("en_locale.yml"), t.root.options[:locale].path
+    locale = t.root.nodelist[0].options[:locale]
+    assert_instance_of I18n, locale
+    assert_equal fixture("en_locale.yml"), locale.path
   end
 
   def test_with_cache_classes_tags_returns_the_same_class

test/unit/variable_unit_test.rb

@@ -102,6 +102,17 @@ class VariableUnitTest < Minitest::Test
     assert_equal 1000.01, var.name
   end
 
+  def test_dashes
+    assert_equal VariableLookup.new('foo-bar'), Variable.new('foo-bar').name
+    assert_equal VariableLookup.new('foo-bar-2'), Variable.new('foo-bar-2').name
+
+    with_error_mode :strict do
+      assert_raises(Liquid::SyntaxError) { Variable.new('foo - bar') }
+      assert_raises(Liquid::SyntaxError) { Variable.new('-foo') }
+      assert_raises(Liquid::SyntaxError) { Variable.new('2foo') }
+    end
+  end
+
   def test_string_with_special_chars
     var = Variable.new(%| 'hello! $!@.;"ddasd" ' |)
     assert_equal 'hello! $!@.;"ddasd" ', var.name
@@ -136,4 +147,10 @@ class VariableUnitTest < Minitest::Test
     var = Variable.new(%! name_of_variable | upcase !)
     assert_equal " name_of_variable | upcase ", var.raw
   end
+
+  def test_variable_lookup_interface
+    lookup = VariableLookup.new('a.b.c')
+    assert_equal 'a', lookup.name
+    assert_equal ['b', 'c'], lookup.lookups
+  end
 end