1

I am developing a affiliation website for products similar to http://www.mysmartprice.com or http://www.smartprix.com/ but I am facing problem in design is database.

My approach:

  • I first downloaded XML of each website which are provided by the ecommerce website itself. It contain the detail of products on that website.

Sample XML file

<Product><ProductID>41410399</ProductID>
<ProductSKU>B00LTUN476</ProductSKU>
<ProductName>Micromax Canvas HD Plus A190 (White)</ProductName>
<ProductDescription>8MP primary camera with full HD video recording, LED flash, 4x digital zoom, auto focus and 2MP front facing camera 5-inch HD capacitive touchscreen with 1280 x 720 pixels resolution Android v4.4.2 KitKat operating system with 1.5GHz hexa core processor, 1GB RAM and 8GB internal memory expandable up to 32GB 2000mAH battery providing talk-time up to 7 hours and standby time up to 250 hours on 2G networks 1 year manufacturer warranty for device and 6 months manufacturer warranty for in-box accessories including batteries from the date of purchase</ProductDescription>
<ProductPrice>8069.00</ProductPrice>
<ProductPriceCurrency>INR</ProductPriceCurrency>
<WasPrice>0.00</WasPrice>
<DiscountedPrice>0.00</DiscountedPrice>
<ProductURL>http://clk.omgt5.com/?AID=630405&amp;PID=13171&amp;Type=12&amp;r=http://www.amazon.in/dp/B00LTUN476/ref%3Dasc_df_B00LTUN47626216749/</ProductURL>
<PID>13171</PID>
<MID>661795</MID>
<ProductImageSmallURL>http://ecx.images-amazon.com/images/I/51EbrIOWRtL._SL75_.jpg</ProductImageSmallURL>
<ProductImageMediumURL>http://ecx.images-amazon.com/images/I/51EbrIOWRtL._SL160_.jpg</ProductImageMediumURL>
<ProductImageLargeURL>http://ecx.images-amazon.com/images/I/51EbrIOWRtL.jpg</ProductImageLargeURL>
<MPN>MCX_A190_WHE</MPN>
<StockAvailability>Usually dispatched within 24 hours</StockAvailability>
<Brand>Micromax</Brand>
<custom1>Shipping Charge INR -0.00</custom1>
<CategoryName>Smartphones</CategoryName>
<CategoryPathAsString>Root|Electronics|Categories|Mobiles &amp; Accessories|Smartphones &amp; Basic Mobiles|Smartphones|</CategoryPathAsString>
</Product>

  • I made separate table for each website. And one table for my website i.e my_table, which has product name and some details

  • Whenever a product is searched it fetched from my_table and it searches from other website tables and shows me the same product with its price on different websites.

My questions are:

  1. Is my approach correct?
  2. The ecommerce websites does not provide product description separately. They provide it in a paragraph manner. So how can I use filters in different products because manually inputting data is not feasible for 2 million products.
  3. How to make an API so that if there is any change in price of product on any web site it directly reflects on my website?
Shashwat
  • 41
  • 4
  • you need check their API (not yours) for price updates. If you want to scan those websites and update all the product information, it's not feasible to keep your database updated. – Tim3880 May 13 '15 at 14:50
  • I have no idea on API development. It is developed in which language? – Shashwat May 13 '15 at 20:06
  • Can you suggest me some links from where I can learn it. – Shashwat May 13 '15 at 20:07
  • Maybe API is not a good realistic idea and I don't know any websites is provide them to 3rd partners. In you case, you need parse the XML in your own code. I'll post an answer to include more details. – Tim3880 May 13 '15 at 20:39

1 Answers1

0

To query multiple websites and find their products and prices based on your customer's requests:

1) create a table of websites, the url address which you can make the queries

2) create a buffer table, (optional), to store the parsed products, prices and other information received from those websites

3) Write a parser to process each websites's response (XML) so you can generate a list of products, prices , details, and store them to your own buffer table. It's easy if all results are in XML format.

4) when your customer search "phone" on your website, your code will send request to each website in your website table, receive and parse the XML files you received, save the result to your own buffer table, and generate response to your customer. If you want to search more than one pages of any websites, you have to send them multiple requests with different page information.

5) If another customer search "phone" again, you can lookup your buffer table and provide results directly. You can decide how long you can trust the buffered results before send requests again.

6) you can do many things on your buffered table and provide more services and good user experiences.

7) if you want to, you can do some background "requesting" before your customer send you request to speed things up.

You need a) some good database design and handling b) pretty good programming (any of php , perl, .net, java) on sending http request, parsing XML
c) pretty good javascript programming to provide good user experience

Tim3880
  • 2,563
  • 1
  • 11
  • 14