I am having issues with the eBay python sdk finding API

Question

I am trying to work on a project which collects data on smart devices. I have decided to use the eBay python SDK rather than rely on web scraping. I am having a few issues

1. When I make a request for a particular Item for example "iPhone x 64gb", I get a response which is a list of eBay listings. Among the listings, some of the listing items may come in the form of a.) A listing of an iPhone 6 which is not what I wanted. b.) A listing of two phones (e.g an iPhone x 64 Gb and 256gb versions). How do I filter through the mess?

2. The documentation for the python SDK is insufficient as I need more teachings on filtering the XML responses and also adding search filters to my API request.

3. I have to make multiple calls for the same item but for another page number that the response will send (max is 100 pages with 100 items per page). I usually see a lot of listings of the same item, the same price and their URLs point to the same seller. This probably won't help me make an accurate statistical analysis on metrics such as the daily average sale price of "iPhone x". How do I get a better sample data from the API as I won't be given all the "iPhone X" listings?

All the problems are encountered when using the finding API.

from ebaysdk.finding import Connection as find_connect
from statistics import mean, median
from bs4 import BeautifulSoup

APP_ID = 'Removed for privacy reasons'
# keywords = input("Enter search keywords(e.g 'white board'): ")

api = find_connect(appid=APP_ID, config_file=None,  siteid="EBAY-ENCA")
request = {
        'keywords': "Iphone x 64gb",
        'itemFilter': [
            {'name': 'Condition', 'value': 'Used'},
            {'name': 'currency', 'value': 'CAD'},
            {'name': 'minPrice', 'value': 100.0}
        ],
        'paginationInput': {
            'entriesPerPage': 100,
            'pageNumber': 1
        },
    }
response = api.execute('findItemsByKeywords', request)

# print(responses.dict())
soup = BeautifulSoup(response.content, 'lxml')
totalentries = int(soup.find('totalentries').text)
items = soup.find_all('item')

print(f"{totalentries} items found")

print_no = 0
prices = []
print(f"Current list is {len(items)} items long")
for item in items:
    cat = item.categoryname.string.lower()
    title = item.title.string.lower()
    price = int(round(float(item.currentprice.string)))
    url = item.viewitemurl.string.lower()

    print('-'*20)
    print(f"{cat}\n{title}\n{price}\n{url}\n")
    prices.append(price)
    print_no += 1

print(f"{print_no} items have been printed")
print(f"Average price is ${mean(prices)}. Median is ${median(prices)}")

I can receive an output such as

3242 items found
Current list is 100 items long

--------------------
# The problem about two different phones in one listing that I was talking about
cell phones & smartphones
apple iphone x silver & gray gsm unlocked 64gb or 256gb
600
https://www.ebay.ca/itm/apple-iphone-x-silver-gray-gsm-unlocked-64gb-256gb-/273580927268?var=572990606496

--------------------
# Basically a duplicate of the above listing
cell phones & smartphones
apple iphone x silver & gray gsm unlocked 64gb or 256gb
600
https://www.ebay.ca/itm/apple-iphone-x-silver-gray-gsm-unlocked-64gb-256gb-/273580927268?var=572990606496

--------------------
# I did not search for an iPhone 8
mobile phones
apple iphone 8 - 64gb - silver (unlocked) model a1863 
152
https://www.ebay.ca/itm/apple-iphone-8-64gb-silver-unlocked-model-a1863-/174235235608

--------------------
# This is what I wanted
cell phones & smartphones
apple iphone x 64gb silver unlocked 5.8 in ios smartphone-visible shadow/burn-in
460
https://www.ebay.ca/itm/apple-iphone-x-64gb-silver-unlocked-5-8-ios-smartphone-visible-shadow-burn-in-/174212340572?var=473126790373

--------------------
# X not Xs max
mobile phones
apple iphone xs max [64gb / 256gb /512gb] cheap unlocked [au stock] free express
1019
https://www.ebay.ca/itm/apple-iphone-xs-max-64gb-256gb-512gb-cheap-unlocked-au-stock-free-express-/324024310348?var=513068412663

100 items have been printed # removed most listings from output for brevity
Average price is $566.2. Median is $600

Answer 1

As a matter of fact, I have finished very similar project few months ago (for mobile company, pricings' statistical analysis as well). Here is my short and simple to implement repository: https://github.com/Brat-Pit/eBay

My general approach: get items' ID list using findItemsAdvanced(), then use GetMultipleItems() to get additional data.

But, back to Your questions:

Ad.1 If You want to somehow filter items, first get items' ID list (using price/description filters in findItemsAdvanced() for example). Then use ebaysdk.shopping and method GetMultipleItems(). You'll get access into item's attributes (like RAM memory, screen size, etc.)

Ad.2 That's true. About 80% of information I acquired from forums, 20% via documentation.

Ad.3 I understand Your point. Daily limit is 5000 queries. My solution was to order the data by relevance first using findItemsAdvanced() (sort the returned items according to a single specified sort order. Default: BestMatch. So You dont't need to do anything like sort) and then download only most 'popular' auctions (users are picking/buying them not without the reason and that's the main idea behind the scenes).

Hope it helps.

Answer 2

You could query by aspect instead of keyword. This should return more expected results.

api_request = {
    'keywords': "Iphone x",
    'itemFilter': [
        {'name': 'Condition', 'value': 'Used'},
        {'name': 'currency', 'value': 'CAD'},
        {'name': 'minPrice', 'value': 100.0}
    ],
    'aspectFilter': [
        {
            'aspectName': 'Storage Capacity',
            'aspectValueName': '16 GB',
        }
    ],
    'categoryId': 9355,
    'outputSelector': [
        'AspectHistogram',
    ],
    'paginationInput': {
        'entriesPerPage': 100,
        'pageNumber': 1
    },
    'sortOrder': 'BestMatch'
}