0

we are using watson discovery service to find answer for travel related questions. Unfortunately the documents set to be used here are not static but the travel related forum on the web. So we need Discovery to access those URL for annotating various components (entities, relations, sentiments etc) and later we can query based on the same to find the right link which has the answers. However I see discovery only support files as documents set and not an URL. I remember alchemy API and the new service NLU both has support for URL. Is there any way discovery can access the URL ? We have selected discovery service for our solution because of the query support it has which seems to be not there in NLU or Alchemy.

Manoj K Sardana
  • 153
  • 1
  • 4
  • 14
  • I am afraid you can't, because Watson Discovery is all about performing queries in a repository of indexed documents. However, you can set up a crawler to extract the information from the forum on the web, so you can perform queries on it - see https://console.bluemix.net/docs/services/discovery/getting-started.html#getting-started-with-the-api – Leo Nov 27 '17 at 10:47

1 Answers1

1

In order to access a public URL, you would need to create some kind of web crawler which converts the web site to the correct format that the data crawler can access.

More details here: https://www.ibm.com/watson/developercloud/doc/discovery/data-crawler-seeds.html

Simon O'Doherty
  • 9,259
  • 3
  • 26
  • 54
  • Thanks. This seems to be the only way. Unfortunately data crawler which is referred in Watson discovery does not support window platform. I may be able to mirror the site using wget for windows but that wont have any value as crawler does not support uploading to Watson discovery. I hope soon we will have support for window. – Manoj K Sardana Mar 26 '17 at 02:22