Unit test for scraping instagram followers

Question

New on ruby/rails and currently in the process of learning it and writing specs. I've got these two methods for scraping instagram followers

def instagram_scraped_followers_count(instagram_username)
        url = "https://www.instagram.com/#{ CGI::escape(instagram_username) }/"
        body = HTTParty.get(url)

        match = body.match(/(?<=content\=")(?<count>\d+[,.]?\d*)(?<unit>[km])?(?=\sFollowers)/) if body
        match['count'].to_f * instagram_multiplier(match['unit']) if match
      end

      def instagram_multiplier(unit)
        case unit
        when 'm' then 1_000_000
        when 'k' then 1_000
        else 1
        end
      end

and added unit test for this as below:

  context '1,300 followers' do
    let(:html_body) { '<meta content="1,300 Followers, 1,408 Following, 395 Posts' }

    it 'returns 1,300' do
      expect(subject.stats[:instagram]).to eql(1_300.0)
    end
  end

My unit test is failing because of this context '1,300 followers' do and the error says that expected: 1300.0 got: 1.0 What did I miss? Is it because there's no condition for it in instagram_multiplier method?

score 3 · Accepted Answer · answered May 21 '19 at 08:10

match[:count].to_f of 1,300 return 1.

I would suggest you change it to match[:count].gsub(',', '').to_f where you replace the , with nothing then perform a .to_f

def instagram_scraped_followers_count(instagram_username)
  url = "https://www.instagram.com/#{ CGI::escape(instagram_username) }/"
  body = HTTParty.get(url)

  match = body.match(/(?<=content\=")(?<count>\d+[,.]?\d*)(?<unit>[km])?(?=\sFollowers)/) if body
  match['count'].gsub(',', '').to_f * instagram_multiplier(match['unit']) if match
end

def instagram_multiplier(unit)
  case unit
  when 'm' then 1_000_000
  when 'k' then 1_000
  else 1
  end
end

score 2 · Answer 2 · answered May 21 '19 at 08:44

If you want to mock HTTP calls, I suggest you use webmock, a gem tailored for that use case.

It will let you write something along the lines of:

let(:instagram_username) { 'whatever' }
let(:body) { '...' }

before do
  stub_request(:get, 'https://www.instragram.com/whatever/').and_return(body: body)
end

Further considerations

Instagram provides developers with an API. You might want to use this instead of parsing a HTML document with a regex.

There might even be a gem that does the Ruby heavy-lifting for you.

Unit test for scraping instagram followers

2 Answers2

Further considerations