-1

What I already have:

  • An asp.net core on .NET framework project which uses DocumentDB as its storage
  • An Azure WebJob which listens to a queue that my web project writes messages to for e-mail sending and other processing
  • Successfully deployed and running on Azure

This all works fine. In addition to the web project there is a Model and Data class library to separate the application into layers.

Currently, the web application invokes a web service and will save the result (a quite large xml document) in the cache and keep it there for 24 hours. This is not ideal as it takes a long time the first time. What I want instead is a nightly batch job which invokes this web service and then stores (overwrites) the response into persistent storage which the web application will then use instead.

I'm confused about which Azure "service" to use for this. What I have started on so far is another WebJob and the idea is to use the same DocumentDb storage to persist the web service response every night. However, I already have all the database repository etc. set up in the web application (Data class library), is it ok to just reference this project from the WebJob instead of having to rewrite some of the same code in the WebJob?

Is it better to use some of the other Azure storage options for this WebJob instead? Like Table Storage, Blob Storage etc? Basically the structure of the data received from the web service is very simple. For each item I just need to store a url, a title, description and unique product id. Obviously the web application needs to access this storage too by simply looking up the product id, and never writing to this storage.

Also, I'm not entirely sure if there is a better alternative than Azure WebJobs for this task, but it seems like the right approach.

Any feedback is appreciated. I'm generally just confused/overwhelmed by all the different services that Azure provides.

Force444
  • 3,321
  • 9
  • 39
  • 77

1 Answers1

1

I'll answer some of your questions...

A webjob works fine for this task. If you have a webservice that is always on adding another webjob seems like a good idea. If your webservice isn't always on, you could have a look at Azure Functions. Azure Functions is sometimes called webjobs 2.0.

As for storage in Document DB there is a file size limit for 2MB (give or take). So, you'll have to find another solution there. I think that Azure Tables also have limitations on storage size, so you'll have to split the file in smaller chunks. So, recommended solution is to go with Azure Blobs.

You'll find some good reading in this answer regarding Blobs vs Tables vs SQL - Getting started with Azure storage: Blobs vs Tables vs SQL Azure

iikkoo
  • 2,810
  • 6
  • 33
  • 38
  • Thanks. Can't I just save each product as a document, to bypass the storage limits? What consequences would this have? Basically when a user visits the site the application needs to retrieve the product information once, depending which product id the user chooses. – Force444 Nov 01 '17 at 14:17
  • 1
    I reread your question. Sure, you can do that, or you can use Azure Tables as a key/value storage. If you already have the Document DB connection and are paying for it, go ahead and use that. Build something simple and evalute. Stick to it if it suits your need, change otherwise... – iikkoo Nov 01 '17 at 14:32
  • 1
    Thanks a lot and apologies for the apparently very broad question. Didn't really know how to make it more specific without stripping away information about my setup! – Force444 Nov 01 '17 at 14:38