0

I wrote a Visual Basic .NET application in Visual Studio to parse MS Powerpoint and Word Files, transform slides to jpgs and to store the content in ElasticSearch. I want to ensure that when parsing a shared network drive that neither the server or my client dramatically slow down. How can I monitor the execution and adapt the processing accordingly? Any basic techniques to get me started?

Saurav Sahu
  • 13,038
  • 6
  • 64
  • 79
wuz
  • 483
  • 3
  • 16
  • Before we can tell you how to detect it you need to be much more specific what "slow down" means to you and what is a acceptable level of performance loss before you start throttling. – Scott Chamberlain Dec 24 '16 at 15:30
  • There's not enough detail in your question, to supply a useful answer. Whats the implementation of the Sever, what's the client? Monitor the execution of what? – Mick Dec 24 '16 at 16:10
  • Options are *very* limited. You think you are doing something special, but the people that write operating systems think that it is entirely normal and that the OS should just handle it. They are not wrong about that. You can tinker with the Thread.CurrentThread.Priority value but the odds you'll see any difference are low. – Hans Passant Dec 24 '16 at 17:02

1 Answers1

0

In my answer I assume that the load mainly consists of reading the PowerPoint and Word files. So enumerating directories, generating the JPGs and writing the result isn't an issue. If not, the approach can be extended.

A simple approach would be:

  1. Figure out how much IO load you want to generate at most, e.g. at most 5 MB/s. This is your read rate you do not want to exceed.

  2. Retrieve the time before you start processing a file.

  3. Retrieve the file size.

  4. After processing a file, take the time again and calculate the duration.

  5. When processing a file, you'll probably going over your read rate. So after processing a file, calculate how long you have to wait to fall under the read rate again and then wait. The calculation is basically

wait_time = file_size / read_rate - duration

Use matching units such as seconds for wait_time and duration, bytes for file_size and bytes per second for the read_rate.

If the wait_time is negative, skip the waiting.

Codo
  • 75,595
  • 17
  • 168
  • 206