Windows Service, FileSystemWatcher + Polling [modified]

[C# Programming]

Hello,

I am developing a windows service that will process
xml files as they come in.
I have a FileSystemWatcher that is watching a folder
and processing the files when the fileSystemWatcher1_Created
event is raised.
I created a FileProcessor class that handles processing a file,
it splits the incoming xml file into many different files and
then pushes data to a database using sqlBulkCopy, etc.

So right now my code looks like this...
private void fileSystemWatcher1_Created(object sender, FileSystemEventArgs e)
{
Logger.AddInformation(string.Format("File {0} {1}.", e.Name, e.ChangeType));

FileProcessor fileProcessor = new FileProcessor(e.Name);
bool retValue = fileProcessor .ProcessFile();

Logger.AddInformation(string.Format("Processed File {0} {1}.", e.Name, retValue ? "Successfully" : "Unsuccessfully"));
}

But I have read that the FileSystemWatcher can be unreliable in some
situations so I wanted to build in a backup mechanism that polls
and checks the folder periodically for files that may have been missed,
or were already in the folder when the service was started, etc.

So my goal was to have some kind of object that maintains a queue
of files that need to be processed.
Then when the fileSystemWatcher1_Created event is raised instead of
processing the file at that time I would just add the file to the queue.
And also when the polling if I encountered a new file I would add it to the
queue as well (making sure that file was not already processed or already
on the queue).

Problem is I'm not really sure how to implement this.
I'm imagining I may need another Thread or Threads.
I have some code that creates another thread and uses a timer so that
could potentially be my backup polling mechanism running every x seconds
scanning folder and adding files to the queue.

But I still don't fully understand how to make it work.
I'm feeling like something would need to manage the queue and keep
processing the files until it's empty, and not sure if that should be
happening on the main windows service thread or a separate thread, etc.

I'm trying to keep the design as simple as possible and hopefully limit it
to just 1 windows service.
I just want to process the files 1 by 1 anyway, so I'm not looking to spawn
a new thread for every file or anything like that.

Can anyone point me in the right direction?
Maybe slap together some skeleton code that would help me understand.
Or are there any good articles or examples of doing something like this or
a design pattern I should look into?

Thanks!modified on Tuesday, March 2, 2010 1:29 PM