0

We have different continuous Azure WebJobs (using WebJobs SDK) listening to Azure Service Bus topics which are running across multiple instances on an App Service Environment. Everyday we get multiple log files in the log folder D:\home\LogFiles\kudu\trace containing an exception as below: 

<step title="Error occurred" date="2016-12-14T18:26:38.860" instance="qwerty" type="error" text="System.IO.InternalBufferOverflowException: Too many changes at once in directory:D:\home\site\wwwroot\App_Data\jobs\continuous." >
  <step title="Cleanup Xml Logs" date="2016-12-14T18:26:38.907" /><!-- duration: 63ms -->
</step><!-- duration: 156ms -->

Is this something to worry about? Is there a way to avoid this exception to happen?

Paco de la Cruz
  • 2,066
  • 13
  • 22
  • Can you provide more details on the WebJob you've deployed? Is it composed of a large number of files? Do the errors you mentioned coincide with deployments? E.g. the theory would then be that your deployment of a large number of files would cause may FileSystemWatcher events, leading to the error I mention below in my answer. – mathewc Dec 17 '16 at 18:18
  • Thanks @mathewc for your answer and comments. The WebJob deployment is pretty standard, compared to other WebJobs I have deployed in the past. These errors are happening almost every day, and sometimes couple of times per day, and they are not related to deployments. What else could write files on **D:\home\LogFiles\kudu\trace**? – Paco de la Cruz Dec 18 '16 at 23:18
  • This isn't due to changes under the LogFiles directory. We only setup watchers you your job files dir (where your binaries/scripts are). Are you modifying any files in that directory at runtime? As I said, you don't really have anything to worry about here. However, if you want us to investigate this feel free to respond to the github issue #2275 that I linked to below, sharing your app name [directly or indirectly](https://github.com/projectkudu/kudu/wiki/Reporting-your-site-name-without-posting-it-publicly) with your issue comment details. – mathewc Dec 19 '16 at 00:05
  • Also, if you're satisfied with the answer below, please mark it as the answer :) – mathewc Dec 19 '16 at 00:07

1 Answers1

2

No this isn't something you need to worry about. Our WebJobs infrastructure uses FileSystemWatchers to watch your files for changes (e.g. when job bits/sources are updated, etc.). From time to time due to Azure Storage issues or other network gremlins the file shares that the FileWatcher is connected to might experience transient issues, causing the FileWatcher to throw errors.

We have logic to handle such transient errors (source code here) and we trace them to the log file you pointed out. We log this primarily for our own use to help us monitor/diagnose issues. The logs you mentioned above are coming from this codepath. We have an issue here in our repo to review the verbosity of these traces.

mathewc
  • 13,312
  • 2
  • 45
  • 53