Over the past few days I've been monitoring a windows service that I've created, as I was sure it has a memory leak. As it turns out, I'm correct - it's memory usage has increased from 41MB to 75MB over the last few days.
All this service does, is watch a file directory and each time a file is created there it uploads it onto our content management system (Microfocus Content Manager); some other miscellaneous tasks are being performed as well, such as writing to the event log if certain exceptions occur and writing messages about the upload status to a log file.
One idea I would like to try to use to find this leak is using something like the .NET CLR profiler, as proposed in answer to this question. However, the profiler doesn't seem to work for windows services (so I would somehow need to change the service I've made into a console application?) and I'm not sure it's able to profile the application as it runs?
Anyway, here is a full copy of the code for the windows service. I appreciate anyone who is able to take a read through this and see if there is something stupid I've done that is causing the leak, below this I'll talk about the areas I think might be causing the memory leak.
using HP.HPTRIM.SDK;
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Diagnostics;
using System.IO;
using System.Configuration;
using System.Linq;
using System.ServiceProcess;
using System.Text;
using System.Threading.Tasks;
using System.Threading;
namespace BailiffReturnsUploader
{
public partial class BailiffReturnsService : ServiceBase
{
private string FileWatchPath = ConfigurationManager.AppSettings["FileWatchPath"];
private string UploadLogPath = ConfigurationManager.AppSettings["UploadLogPath"];
private string UploadErrorLocation = ConfigurationManager.AppSettings["UploadErrorLocation"];
private string ContentManagerEnviro = ConfigurationManager.AppSettings["ContentManagerEnviro"];
public BailiffReturnsService()
{
InitializeComponent();
bailiffEventLogger = new EventLog();
if(!EventLog.SourceExists("BailiffReturnsSource"))
{
EventLog.CreateEventSource("BailiffReturnsSource", "BailiffReturnsLog");
}
bailiffEventLogger.Source = "BailiffReturnsSource";
bailiffEventLogger.Log = "BailiffReturnsLog";
}
protected override void OnStart(string[] args)
{
try
{
TrimApplication.Initialize();
BailiffReturnsFileWatcher = new FileSystemWatcher(FileWatchPath)
{
NotifyFilter = NotifyFilters.DirectoryName | NotifyFilters.FileName | NotifyFilters.Attributes,
Filter = "*.*"
};
// I think this is the problematic part, the event registration?
BailiffReturnsFileWatcher.Created += new FileSystemEventHandler(OnCreate);
BailiffReturnsFileWatcher.EnableRaisingEvents = true;
bailiffEventLogger.WriteEntry("Service has started", EventLogEntryType.Information);
}
catch (Exception ex)
{
bailiffEventLogger.WriteEntry(string.Format("Could not create file listener : {0}", ex.Message), EventLogEntryType.Error);
}
}
protected override void OnStop()
{
bailiffEventLogger.WriteEntry("Service has stopped", EventLogEntryType.Information);
Dispose();
}
/// <summary>
/// Handler for file
/// </summary>
/// <param name="sender"></param>
/// <param name="e"></param>
void OnCreate(object sender, FileSystemEventArgs e)
{
try
{
int attempts = 0;
FileInfo fileInfo = new FileInfo(e.FullPath);
while (IsFileLocked(fileInfo))
{
attempts++;
CreateUploadLog(UploadLogPath, string.Format("Info : {0} is locked, trying again. Attempt #{1}.", fileInfo.Name, attempts));
if (attempts == 5)
{
CreateUploadLog(UploadLogPath, string.Format("Error : {0} is locked, could not access file within 5 attempts.", fileInfo.Name));
bailiffEventLogger.WriteEntry(string.Format("Error : {0} is locked, could not access file within 5 attempts.", fileInfo.Name), EventLogEntryType.Error);
break;
}
Thread.Sleep(1500);
}
bool isSaveSuccess = SaveToTrim(e.FullPath);
if(isSaveSuccess)
{
DeleteFile(e.FullPath);
}
else
{
MoveFileToError(e.FullPath);
}
fileInfo = null;
Dispose();
}
catch (Exception ex)
{
bailiffEventLogger.WriteEntry(string.Format("Error while saving or deleting file : {0}", ex.Message), EventLogEntryType.Error);
}
}
/// <summary>
/// Attemps to upload file to content manager.
/// </summary>
/// <param name="path"></param>
/// <returns></returns>
bool SaveToTrim(string path)
{
string pathFileNoExt = Path.GetFileNameWithoutExtension(path);
try
{
string[] pathArgs = pathFileNoExt.Split(new string[] { "_" }, StringSplitOptions.RemoveEmptyEntries);
// Note for stack overflow: These classes and methods are part of an SDK provided by a 3rd party that I'm using to upload documents
// into their content management system. I'm not sure, but I don't think the memory leak is occuring at this part.
using (Database dbCntMgr = new Database { Id = ContentManagerEnviro })
{
// Connect to the content manager database.
try
{
dbCntMgr.Connect();
}
catch (Exception ex)
{
bailiffEventLogger.WriteEntry(ex.Message, EventLogEntryType.Error);
CreateUploadLog(UploadLogPath, "Failed to connect to content manager.");
return false;
}
// Create the record based on record type, set default assignee and default bailiff type.
RecordType oRecordType = new RecordType(dbCntMgr, "Revenues - Bailiff");
Record oRecord = new Record(dbCntMgr, oRecordType);
oRecord.Assignee = new Location(dbCntMgr, "Bailiff Returns Pending");
oRecord.SetFieldValue(new FieldDefinition(dbCntMgr, "Bailiff Type"), new UserFieldValue("Liability Order Return"));
// Set the default container, not changed if no result is found.
Record oContainer;
oContainer = new Record(dbCntMgr, "014/1065/0973/55198");
// Search via property ID and "80" for Revenues Property File
TrimMainObjectSearch search = new TrimMainObjectSearch(dbCntMgr, BaseObjectTypes.Record);
TrimSearchClause trimSearchClause = new TrimSearchClause(dbCntMgr, BaseObjectTypes.Record, new FieldDefinition(dbCntMgr, "Property ID"));
trimSearchClause.SetCriteriaFromString(pathArgs[2].Substring(2));
search.AddSearchClause(trimSearchClause);
trimSearchClause = new TrimSearchClause(dbCntMgr, BaseObjectTypes.Record, SearchClauseIds.RecordType);
trimSearchClause.SetCriteriaFromString("80");
search.AddSearchClause(trimSearchClause);
// Sets the container to found record if any are found.
foreach (Record record in search)
{
//oContainer = new Record(dbCntMgr, record.Uri);
oContainer = record;
}
// Once container is finalised, set record container to located container.
oRecord.Container = oContainer;
// Set the title to name
oRecord.Title = pathArgs[3];
// Set the input document.
InputDocument oInputDocument = new InputDocument();
oInputDocument.SetAsFile(path);
oRecord.SetDocument(oInputDocument, false, false, "Created Via Bailiff Content Manager Uploader service.");
// Save if valid, print error if not.
if (oRecord.Verify(false))
{
oRecord.Save();
CreateUploadLog(UploadLogPath, string.Format("File uploaded : {0}", Path.GetFileNameWithoutExtension(path)));
return true;
}
else
{
CreateUploadLog(UploadLogPath, string.Format("Upload of {0} file attempt did not meet validation criteria. Not uploaded.", Path.GetFileNameWithoutExtension(path)));
return false;
}
}
}
catch (Exception ex)
{
bailiffEventLogger.WriteEntry(ex.Message, EventLogEntryType.Error);
return false;
}
}
/// <summary>
/// Deletes file when successfully uploaded.
/// </summary>
/// <param name="path"></param>
void DeleteFile(string path)
{
try
{
string pathFileNoExt = Path.GetFileNameWithoutExtension(path);
// If file exists, delete.
if (File.Exists(path))
{
File.Delete(path);
CreateUploadLog(UploadLogPath, string.Format("File deleted from Upload folder : {0}", pathFileNoExt));
}
else
{
CreateUploadLog(UploadLogPath, string.Format("Error deleting file from upload folder : {0}", pathFileNoExt));
}
}
catch (Exception ex)
{
bailiffEventLogger.WriteEntry(ex.Message, EventLogEntryType.Warning);
CreateUploadLog(UploadLogPath, ex.Message);
}
}
/// <summary>
/// Moves non uploaded files (failed to upload) to an error location as specified in the app config.
/// </summary>
/// <param name="path"></param>
void MoveFileToError(string path)
{
try
{
string pathFileNoExt = Path.GetFileName(path);
// If directory and file exist, attempt move.
if(Directory.Exists(UploadErrorLocation))
{
if(File.Exists(path))
{
if(File.Exists(Path.Combine(UploadErrorLocation, pathFileNoExt)))
{
File.Delete(Path.Combine(UploadErrorLocation, pathFileNoExt));
}
File.Move(path, Path.Combine(UploadErrorLocation, pathFileNoExt));
} else
{
CreateUploadLog(UploadLogPath, "Could not move non-uploaded file to error location");
}
} else
{
CreateUploadLog(UploadLogPath, "Could not move non-uploaded file to error location, does the error folder exist?");
}
}
catch (Exception ex)
{
bailiffEventLogger.WriteEntry("Error while moving file to error location : " + ex.Message, EventLogEntryType.Warning);
CreateUploadLog(UploadLogPath, ex.Message);
}
}
/// <summary>
/// Takes full path of upload log path and a message to add to the upload log. Upload log location is specified in the app config.
/// </summary>
/// <param name="fullPath"></param>
/// <param name="message"></param>
private void CreateUploadLog(string fullPath, string message)
{
try
{
//StreamWriter streamWriter;
// If file does not exist, create.
if (!File.Exists(Path.Combine(fullPath, "UploadLog_" + DateTime.Now.ToString("ddMMyyyy") + ".txt")))
{
using (StreamWriter streamWriter = File.CreateText(Path.Combine(fullPath, "UploadLog_" + DateTime.Now.ToString("ddMMyyyy") + ".txt")))
{
streamWriter.Close();
}
}
// Append text to file.
using (StreamWriter streamWriter = File.AppendText(Path.Combine(fullPath, "UploadLog_" + DateTime.Now.ToString("ddMMyyyy") + ".txt")))
{
streamWriter.WriteLine(string.Format("{0} -- {1}", DateTime.Now.ToString(), message));
streamWriter.Close();
}
}
catch (Exception ex)
{
bailiffEventLogger.WriteEntry(ex.Message, EventLogEntryType.Warning);
}
}
/// <summary>
/// Attempts to access the file, returns true if file is locked, false if file is not locked.
/// </summary>
/// <param name="file"></param>
/// <returns></returns>
private bool IsFileLocked(FileInfo file)
{
try
{
using(FileStream stream = file.Open(FileMode.Open, FileAccess.Read, FileShare.None))
{
stream.Close();
}
}
catch(IOException)
{
return true;
}
return false;
}
}
}
So, I think there are a few areas that might be causing the leak:
- The event registration/raising - From a few searches and a good hour or two of experimentation, I have a suspicion that it's the event raising and delegation that is causing the memory leak. Something to do with not unregistering or disposing of the event once it's done?
- The 3rd Party SDK that is performing the interaction/uploading contains the leak - This might be possible, if replies believe this is the cause, then I will pursue this issue with the maintainers of the SDK. I'm somewhat confident that this not the cause of the leak, as the SDK contains a debug mode that can be enabled via a method that outputs any non-disposed objects to the event log, having tried that, it did not show any objects not being disposed.
- The upload log file writing - Could the process of writing to the file with upload events etc be the cause? I don't have much confidence in this being the memory leak, as
usingstatements are being used for the streamwriter, but maybe?
What are your thoughts?
