0

I'm new to Scrapy and to python and when I try to import a class from items.py in VS Code I get the following error:

Exception has occurred: ModuleNotFoundError
No module named 'scraper.items'; 'scraper' is not a package

My folder structure: Folder structure

Tutorial I was trying to follow: https://www.youtube.com/watch?v=wyE4oDxScfE

scraper.py code:

import scrapy
from scrapy.crawler import CrawlerProcess
from scrapy.loader import ItemLoader
from scraper.items import ScraperItem

class ScrapySpider(scrapy.Spider):
    name = 'scraper'

def start_requests(self):
    yield scrapy.Request('https://www.thewhiskyexchange.com/c/639/bourbon-whiskey')
    
def parse(self, response):
    Item = ScraperItem()
    products = response.css('li.product-grid__item')
    
    for item in products:
        
            Item['name'] = item.css('p.product-card__name::text').get().strip(),
            Item['price'] = item.css('p.product-card__price::text').get().strip().replace('£', ''),
            Item['perL'] = item.css('p.product-card__unit-price::text').get().strip().replace('per litre', 'per L').replace('(', '').replace(')', '')
            
            yield item
        
    for x in range(2, 7):
        yield(scrapy.Request(f'https://www.thewhiskyexchange.com/c/639/bourbon-whiskey?pg={x}', callback=self.parse))

process = CrawlerProcess(settings = {
    'FEED_URI': 'whiskey.csv',
    'FEED_FORMAT': 'csv'
})

process.crawl(ScrapySpider)
process.start()

items.py code:

import scrapy


class ScraperItem(scrapy.Item):
    name = scrapy.Field()
    price = scrapy.Field()
    perL = scrapy.Field()

When I use:

..items import ScraperItem

I get the same error

Ego0r
  • 9
  • 1
  • 1
    probably because you are executing the script from outside of the project directory. move your terminal current working directory inside the scraper folder – Alexander Sep 24 '22 at 01:55

1 Answers1

0

the import needs to be relative to the execution directory. My guess is scraper.scraper.item will work.

Bindestrich
  • 9
  • 3