2

Im new to webapi2, I`v implemented a simple WebApi2 project, which unlocks some data from a third party webservice (which is called with credentials)

This works fine, but now Im looking for ways to avoid scrape content from our webapi. It isn`t sensitive data, but it can be beneficial for competitors.

We cant have our users log in, but im looking for ways to make it harder for someone to scrape the content from the webservice. Since the data is freely available from our website, I understand we cant make it 100% secure, but there must be something I can do to make it harder (and it seems like a problem people have countered before, I just cant seem to find some clear information)

Ive looked into CORS, but that seems more geared the other way around, allow request from other domains.

Maybe use some secret key in the backend, hash it with some datettime and header information, and let the javascript add that to the ajax calls, and check in the webapi methods if the token is valid?

Edit

Because there doesn`t seem to be an easy (and clean) way to stop abuse with token authenticiation, I decided to go with Ip throttling. Not ideal, but it helps:

I used this library and it works well: https://github.com/stefanprodan/WebApiThrottle

Dossie
  • 57
  • 6
  • "This works fine, but now Im looking for ways to avoid scrape content from our webapi. It isn`t sensitive data, but it can be beneficial for competitors." There's no way to avoid that if all your output is HTML based, one way tho, would be to present some of the data as an image format, but then again your competitors could use some form of OCR and bypass that. – MeTitus Jul 02 '16 at 13:00
  • Well, presenting the data as an image format will be a bit too much, and it will hurt user experience. What I would like to prevent, is their ability to easily automate request to the webapi. For example the following request is being used via ajax on domain.tld: domain.tld/api/getcategoriesbyid I would like to prevent (or perhaps make more difficult if that is more practical) users/servers to call this automaticly and store the data, or use the methods in their own applications. – Dossie Jul 02 '16 at 14:08
  • I honestly don't know how you would be able to achieve this if the web api is exposed to the general public. Your third party provides integrate with your web api isn't it? – MeTitus Jul 03 '16 at 12:07
  • 1
    Agreed, went with another solution, see my edit – Dossie Jul 12 '16 at 09:07

1 Answers1

1

A couple of ideas:

Shared Key. You mentioned this, but I would have them add the key to the Authorization header as a Bearer token. Then your API can simple compare the key for a know value and reject or accept the request.

Obfuscate the Id's. Rather than using ints that are easily automated, you can use Guids to prevent this. This would require your dataset to be updated to have Guids so it may not be feasible.

User Agent parsing. Look at the User Agent header and if it's from a browser or any other user agent you don't allow, reject the request. This is easily spoofed, but it's a quick thing to check.

Remember that security through obscurity is not security. You mentioned that you understand this wouldn't be secure, but simply want to make it more difficult and I think any of these methods would work. The Bearer token would be the most secure as you could give each client their own key and track usage. You could argue that the token route would actually be secure if you control who has the keys and keep them rotated. They other two options I mentioned are not secure by any means, but they'd get the job done.

ManOVision
  • 1,853
  • 1
  • 12
  • 14
  • "We cant have our users log in, but im looking for ways to make it harder for someone to scrape the content from the webservice. Since the data is freely available from our website" You did get this part right? – MeTitus Jul 03 '16 at 12:02
  • Thanks for the reply, for now im going with another solution, see my edit. Maybe I will look at this at a later time to implement some other features. – Dossie Jul 12 '16 at 09:10