Including external files in a Jekyll template

Question

Is it possible to include a html file from another domain inside a Jekyll template? And if so what would the syntax be?

I'm not a Ruby or Jekyll developer, more or less asking on behalf of another so please forgive me if the answer is obvious! At least I couldn't quite find the answer with some initial research.

In essence we're trying to pull the markup of a footer from another domain, this is how production will work so we're actually just trying to simulate it in our template deliverables.

Cheers

Answer 1

You cannot do this within the template itself. However, you could define a custom Liquid tag that scrapes the markup of the remote page, and then put that tag into template. This would be in a file called e.g. plugins/remote_footer.rb

require 'nokogiri'
require 'open-uri'
require 'uri'

module Jekyll

  class RemoteFooterTag < Liquid::Tag

    def initialize(tag_name, markup, tokens)
      #markup is what is defined in the tag. Lets make it a URL so devs 
      #don't have to update code if the URL changes.
      url = markup

      #check if the URL is valid
      if url =~ URI::regexp
        #grab the remote document with nokogiri
        doc = Nokogiri::HTML(open(url))

        #search the document for the HTML element you want
        @node = doc.at_xpath("//div[@id='footer']")
      else
        raise 'Invalid URL passed to RemoteFooterTag'
      end

      super
    end

    def render(context)
      output = super
      if @node 
        node.to_s
      else
        "Something went wrong in RemoteFooterTag"
      end
    end
  end
end

Liquid::Template.register_tag('remote_footer', Jekyll::RemoteFooterTag)

And then in your template:

{% remote_footer http://google.com %}

I threw this together quickly and didn't check if it runs, but hopefully it's enough to work with. Keep in mind that this will run once when the liquid parser runs on the page, and if the remote element changes that will not be reflected until the Jekyll site is rebuilt.

Answer 2

I just stumbled into this problem and I couldn't find any working solution addressing all the use cases I had so I wrote my own plugin.

N.B. This is the first piece of ruby I ever wrote.

require 'nokogiri'
require 'open-uri'
require 'uri'

class Jekyll::IncludeRemoteTag < Jekyll::Tags::IncludeTag
  @@remote_cache = {}

  def initialize(tag_name, markup, tokens)
    super
    @url = @file
  end

  def validate_url(url)
    if url !~ URI::regexp
      raise ArgumentError.new <<-eos
Invalid syntax for include_remote tag. URL contains invalid characters or sequences:

#{url}

Valid syntax:

#{syntax_example}

eos
    end
  end

  def syntax_example
    "{% #{@tag_name} http://domain.ltd css=\".menu\" xpath=\"//div[@class='.menu']\" param=\"value\" param2=\"value\" %}"
  end

  def render(context)
    @url = render_variable(context) || @url
    validate_url(@url)

    if @params
      validate_params
      @params = parse_params(context)
    end

    xpath = @params['xpath']
    css = @params['css']

    if ! html = @@remote_cache["#{@url}_#{xpath}"]
      # fetch remote file
      page = Nokogiri::HTML(open(@url))

      # parse extract xpath/css fragments if necessary
      node = page.at_xpath(xpath) if xpath
      node = page.css(css) if css
      node = page if !node

      raise IOError.new "Error while parsing remote file '#{@url}': '#{xpath||css}' not found" if !node

      # cache result
      html = @@remote_cache["#{@url}_#{xpath}"] = node.to_s
    end

    begin
      partial = Liquid::Template.parse(html)

      context.stack do
        context['include'] = @params
        partial.render!(context)
      end
    rescue => e
      raise Jekyll::Tags::IncludeTagError.new e.message, @url
    end
  end
end

Liquid::Template.register_tag('include_remote', Jekyll::IncludeRemoteTag)

And you'd use it like this:

<!-- fetch header.html -->
{% assign url = 'http://mything.me/_includes/header.html' %}
{% include_remote {{ url }} %}

<!-- fetch menu.html and extract div.menu -->
{% include_remote 'http://mything.me/_includes/menu.html' css="div.menu" links=site.data.menu %}

<!-- fetch menu.html and extract div.menu (xpath version) -->
{% include_remote 'http://mything.me/_includes/menu.html' xpath="div[@class='menu']" links=site.data.menu %}

It basically works exactly like a normal include file but it's remote.

Available for download here: https://gist.github.com/kilianc/a6d87879735d4a68b34f

License MIT.

Answer 3

You can include Liquid tag plugin for Jekyll, it worked for me.

Add this in your site's Gemfile:

group :jekyll_plugins do
    gem 'jekyll-remote-include', :github => 'netrics/jekyll-remote-include'
end

Use the tag in your posts as follows:

{% remote_include https://raw.githubusercontent.com/jekyll/jekyll/master/README.markdown %}

Or

{% capture products %}
{% remote_include http://localhost:8888/products.json %}
{% endcapture %}

...

{{ products }}