1

I have to implement a Web Cralwer that visits the Linked Data on the Web. I have built a simple functionality for that. I have three queries for that:

  1. What are the seed URIs I should use. The web sites which provide data in RDF format and follow Tim Berners Lee Linked Data principles ??
  2. Generally what do you mean by round based approach for Web-Cralwers? I read about general Web Crawlers and found that round based approach should be followed.
  3. I am able to parse only web pages which can return RDF/XML data .Is it sufficient to crawl the Linked Data.
Kara
  • 6,115
  • 16
  • 50
  • 57
Prannoy Mittal
  • 1,525
  • 5
  • 21
  • 32

1 Answers1

5
  1. There's a couple of options, for example use all the URIs found in the Billion Triples Challenge dump as starting points, or all the resources listed in the lodcloud group on the Data Hub (can be retrieved through the CKAN API).
  2. Sorry, I don't know.
  3. No, RDF/XML is not sufficient, as many datasets published as linked data use other formats. You also want Turtle and RDFa. You can use Apache Any23, which understands all of the above. LDSpider is a crawler that uses Any23.
cygri
  • 9,412
  • 1
  • 25
  • 47