Introduction
The worker service template in .NET is the project template that is used for long running and resource intensive tasks best regulated to run headless(no UI) and in the background. .NET’s implementation of the background service gives use access to cross platform targeting and also since it uses the generic host mechanism, it gives us access to commonly used things like the dependency injection chain, logging, and configuration that we may already be used to. The .NET FileSystemWatcher is a component used to watch a directory for change notifications from the system such as a file being added, updated, or deleted.
The Plan
- We will create a worker service project using the .NET CLI too.
- We will create a service class that utilizes the
FileSystemWatcher to watch a specified directory for changes. - When a file addition occurs, we will consume the file and read the lines from it. We will do this by having a scoped service class that will act as our file consumer. We will use the DI chain’s service provider to implement this class with a limited scope.
What You Will Need
- .NET SDK, I’m using 6.0
- IDE I’m using VS 2022 Community
Setup
Initialize the project using the .NET CLI tool. I'm on Windows and I used PowerShell:
dotnet new worker --name FileWatching
FileConsumerService.cs
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
namespace FileWatching
{
public class FileConsumerService : IFileConsumerService
{
ILogger<fileconsumerservice> _logger;
public FileConsumerService(ILogger<fileconsumerservice> logger)
{
_logger = logger;
}
public async Task ConsumeFile(string pathToFile)
{
if(!File.Exists(pathToFile))
return;
_logger.LogInformation($"Starting read of {pathToFile}");
using (StreamReader sr = File.OpenText(pathToFile))
{
string? s = null;
int counter = 1;
while ((s = await sr.ReadLineAsync()) != null)
{
_logger.LogInformation($"Reading Line {counter}
of the file {pathToFile}");
counter++;
}
}
_logger.LogInformation($"Completed read of {pathToFile}");
}
}
}
The first class we’ll start with is our consumer class - this service’s responsibility is to simply take the path to the newly added file open it, and consume it line by line. In this example, I use some CSV files. This would be where you would usually read the data and maybe do something else with it like insert it into a database. Take note that the main ConsumeFile function is done asynchronously. This lets us do this longer running process on another thread and be able to quickly return back to the class that uses this service. As you will see, this will handle the issue of having a lot of files dropped consecutively.
MyFileWatcher.cs
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
namespace FileWatching
{
public class MyFileWatcher : IMyFileWatcher
{
private string _directoryName = Path.Join
(Environment.CurrentDirectory, "files");
private string _fileFilter = "*.*";
FileSystemWatcher _fileSystemWatcher;
ILogger<myfilewatcher> _logger;
IServiceProvider _serviceProvider;
public MyFileWatcher(ILogger<myfilewatcher> logger,
IServiceProvider serviceProvider)
{
_logger = logger;
if(!Directory.Exists(_directoryName))
Directory.CreateDirectory(_directoryName);
_fileSystemWatcher = new FileSystemWatcher(_directoryName, _fileFilter);
_serviceProvider = serviceProvider;
}
public void Start()
{
_fileSystemWatcher.NotifyFilter = NotifyFilters.Attributes
| NotifyFilters.CreationTime
| NotifyFilters.DirectoryName
| NotifyFilters.FileName
| NotifyFilters.LastAccess
| NotifyFilters.LastWrite
| NotifyFilters.Security
| NotifyFilters.Size;
_fileSystemWatcher.Changed += _fileSystemWatcher_Changed;
_fileSystemWatcher.Created += _fileSystemWatcher_Created;
_fileSystemWatcher.Deleted += _fileSystemWatcher_Deleted;
_fileSystemWatcher.Renamed += _fileSystemWatcher_Renamed;
_fileSystemWatcher.Error += _fileSystemWatcher_Error;
_fileSystemWatcher.EnableRaisingEvents = true;
_fileSystemWatcher.IncludeSubdirectories = true;
_logger.LogInformation($"File Watching has started
for directory {_directoryName}");
}
private void _fileSystemWatcher_Error(object sender, ErrorEventArgs e)
{
_logger.LogInformation($"File error event {e.GetException().Message}");
}
private void _fileSystemWatcher_Renamed(object sender, RenamedEventArgs e)
{
_logger.LogInformation($"File rename event for file {e.FullPath}");
}
private void _fileSystemWatcher_Deleted(object sender, FileSystemEventArgs e)
{
_logger.LogInformation($"File deleted event for file {e.FullPath}");
}
private void _fileSystemWatcher_Changed(object sender, FileSystemEventArgs e)
{
}
private void _fileSystemWatcher_Created(object sender, FileSystemEventArgs e)
{
using (var scope = _serviceProvider.CreateScope())
{
var consumerService = scope.ServiceProvider.GetRequiredService
<ifileconsumerservice>();
Task.Run(() => consumerService.ConsumeFile(e.FullPath));
}
}
}
}
This is the workhorse of the application. The class implements the FileSystemWatcher component and describes:
- the directory that we want to watch
- what type of files we want to look out for
- what properties about those files we are interested in
- what notification events we want to act upon
- what we want to do when those events occur
private string _directoryName =
Path.Join(Environment.CurrentDirectory, "files");
Simple. This property is the directory that we want to watch. You can change this to whatever directory you want the FileSystemWatcher component to observe.
private string _fileFilter = "*.*";
Another simple one. This line pretty much describes the filter for the files we want if we only wanted text files, we would’ve used “*.txt” . Adjust this as needed.
public MyFileWatcher(ILogger<myfilewatcher> logger, IServiceProvider serviceProvider)
{
_logger = logger;
if(!Directory.Exists(_directoryName))
Directory.CreateDirectory(_directoryName);
_fileSystemWatcher = new FileSystemWatcher(_directoryName, _fileFilter);
_serviceProvider = serviceProvider;
}
This is the constructor of this class. We are using the GenericHost’s DI mechanism to inject a logger and the DI’s serviceProvider. One thing we do is check for the existence of our directory and if it’s not there, we create it. We initialize the FileSystemWatcher using the constructor that takes the directory we want to watch and the filter of the files we want to watch. This class will be used as a singleton for our application. This means that only one instance of this will exist for the application’s life cycle. So we do our directory check once, but as you will see, we will use the serviceProvider to create a scoped instance of our consumer class for every new file created.
_fileSystemWatcher.NotifyFilter = NotifyFilters.Attributes
| NotifyFilters.CreationTime
| NotifyFilters.DirectoryName
| NotifyFilters.FileName
| NotifyFilters.LastAccess
| NotifyFilters.LastWrite
| NotifyFilters.Security
| NotifyFilters.Size;
In our start method, the first thing we do is set the notify filters. These filters are the properties of the files we want to be notified about. We are telling the FileSystemWatcher that when any of these properties change, we want our notification events to fire. Modify this set to the properties that you are interested in observing changes about.
_fileSystemWatcher.Changed += _fileSystemWatcher_Changed;
_fileSystemWatcher.Created += _fileSystemWatcher_Created;
_fileSystemWatcher.Deleted += _fileSystemWatcher_Deleted;
_fileSystemWatcher.Renamed += _fileSystemWatcher_Renamed;
_fileSystemWatcher.Error += _fileSystemWatcher_Error;
These are the events of the files we want to watch. When one of these actions occur on our files, we want to declare what we have to happen.
_fileSystemWatcher.EnableRaisingEvents = true;
_fileSystemWatcher.IncludeSubdirectories = true;
_logger.LogInformation($"File Watching has started for directory
{_directoryName}");
Finally, the EnableRaisingEvents property is set to “true”. This is simply the “on” switch for the FileSystemWatcher. If we want the watcher to stop, but the program to continue, we would set this property to false. IncludeSubDirectories is pretty straightforward, if true, it will watch subdirectories in your declared directory, if false, it will just do the base directory. We also log our watcher class being started successfully.
private void _fileSystemWatcher_Created(object sender, FileSystemEventArgs e)
{
using (var scope = _serviceProvider.CreateScope())
{
var consumerService =
scope.ServiceProvider.GetRequiredService<ifileconsumerservice>();
Task.Run(() => consumerService.ConsumeFile(e.FullPath));
}
}
The final thing to pay attention to in this class is the event when a file is created. When a file is created, we use the serviceProvider to create a scope of our service. We then retrieve a scoped instance of our consumer service, then we fire the task off to ConsumeFile. We pass the path of our file using the FileSystemEventArgs object. The lifetime of the consumer service is only within this scope. This lets us use the GenericHost’s built in DI mechanism to handle the allocation of our consumerService in an efficient and correct manner.
Worker.cs
namespace FileWatching;
public class Worker : BackgroundService
{
private readonly ILogger<worker> _logger;
private readonly IMyFileWatcher _watcher;
public Worker(ILogger<worker> logger,IMyFileWatcher watcher)
{
_logger = logger;
_watcher = watcher;
}
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
_watcher.Start();
while (!stoppingToken.IsCancellationRequested)
{
}
}
}
The worker class is the class that actually runs in the background. It implements BackgroundService class which has the necessary things we need to have a program fire off and run in the background until getting a signal to stop. First, we inject instances of the logger and of our FileWatcher service. We override the base class’s ExecuteAsync method, which fires off when the host has a worker service declared. We simply call our FileWatcher service’s “Start” method. Next, we have a loop that simply flows until we get a cancellation request, for example, at the console during debug which is done by hitting Ctrl+C command or when this is deployed as a windows service is done by telling the service to STOP.
Program.cs
using FileWatching;
IHost host = Host.CreateDefaultBuilder(args)
.ConfigureServices(services =>
{
services.AddHostedService<worker>();
services.AddSingleton<imyfilewatcher,myfilewatcher>();
services.AddScoped<ifileconsumerservice,fileconsumerservice>();
})
.Build();
await host.RunAsync();
This class sets everything up. We use our generichost’s ConfigureServices function to setup the injection of the Worker class by using the AddHostedService function. Next, we add our filewatcher service as a singleton using the AddSingleton function. Then we tell our service injector that when we need an instance of the FileConsumer service, we would do it as a scoped instance.
When we run this, the application will create a “files” directory in the running directory.
Now, if you drop some files into the “files” directory, you should see the results of the events firing off, we see the “created” event firing off and we see the consumer class’s function logging the read off lines from the file. I have included some example files to use for this.
Other Helpful Information
History
- 16th October, 2022: Initial version
