Sitemap_generator fails to upload

Question

I've followed the instructions on a couple of pages for getting a sitemap to generate and be uploaded to my S3 Bucket. The sitemap is generating, but not uploading.

I'm using carrierwave for the upload, which is working fine for image uploads.

The key file seems to be config/sitemap.rb. Here's mine:

require 'rubygems'
require 'sitemap_generator'

# Set the host name for URL creation
SitemapGenerator::Sitemap.default_host = "https://www.driverhunt.com"

# pick a place safe to write the files
SitemapGenerator::Sitemap.public_path = 'tmp/'

# store on S3 using #Fog# Carrierwave
SitemapGenerator::Sitemap.adapter = SitemapGenerator::WaveAdapter.new
# SitemapGenerator::Sitemap.adapter = SitemapGenerator::S3Adapter.new
# This is a different problem to the one in the question, but using this second adaptor gives the error: "...lib/fog/storage.rb:27:in `new':  is not a recognized storage provider (ArgumentError)"

# inform the map cross-linking where to find the other maps
SitemapGenerator::Sitemap.sitemaps_host = "http://#{ENV['S3_BUCKET']}.s3.amazonaws.com/"

# pick a namespace within your bucket to organize your maps
SitemapGenerator::Sitemap.sitemaps_path = 'sitemaps/'

SitemapGenerator::Sitemap.create do
  add '/home', :changefreq => 'daily', :priority => 0.9
  # add '/contact_us', :changefreq => 'weekly'
end
# SitemapGenerator::Sitemap.ping_search_engines # Not needed if you use the rake tasks

What's going on? How do I debug a carrierwave upload?

score 4 · Answer 1 · answered Apr 08 '16 at 17:09

I will answer the question as your comment for the S3Adapter brought me to this topic while I was googling the not recognized provider. If you turn back on the comment using the S3Adapter and do the following you will get it working.

If you do not specify any fog ENV VARS for the fog-aws gem you will get the error:

ArgumentError:  is not a recognized provider

by using as an adapter the SitemapGenerator::S3Adapter.new

The setup you have got above is perfectly fine, just use the S3Adapter.new instead of the WaveAdapter! The error you are getting (and I was getting as well) is due to the fact that SitemapGenerator::S3Adapter uses fog-aws and in order to make it run by default you should have the following ENV VARS:

ENV['AWS_ACCESS_KEY_ID']     = XXX
ENV['AWS_SECRET_ACCESS_KEY'] = XXX
ENV['FOG_PROVIDER']          = AWS
ENV['FOG_DIRECTORY']         = your-bucket-name
ENV['FOG_REGION']            = your-bucket-region (ex: us-west-2)

If you are missing even one of the following you will get the error:

ArgumentError:  is not a recognized provider

Alternativelly, if you want to avoid using ENV VARS for some reason you should specify the values when you initialize your adapter as follows:

SitemapGenerator::Sitemap.adapter = SitemapGenerator::S3Adapter.new(fog_provider: 'AWS',
                                     aws_access_key_id: 'your-access-key-id',
                                     aws_secret_access_key: 'your-access-key',
                                     fog_directory: 'your-bucket',
                                     fog_region: 'your-aws-region')

However using just the above ENV VARS you will be fine and get your sitemap up and running. This setup was tested with sitemap_generator version: 5.1.0

For your question: The Image uploading works as it does not require the exact same configuration as the WaveAdapter. I am guessing that your carrierwave.rb file is missing the following:

 config.cache_dir = "#{Rails.root}/tmp/"
 config.permissions = 0666

The complete configuration for the carrierwave initializer can be found here: Generate Sitemaps on read only filesystems like Heroku (check if you are missing something or use the other adapter)

However, I believe that your problem has to do with missing ENV VARS from the production environment.

Sitemap_generator fails to upload

1 Answers1