Multithreaded Task Execution on a Timer

Backstory: Every time one of our APIs is accessed, we record a log of the request and the response. That means for each one, the headers and the body of the message. It’s very useful for debugging issues with our own APIs as well as issues with vendors interfacing with us.

The data for each row can get big. Like, really big. A call to our Winning Numbers API could return a payload that’s a 1.5MB in size. There really isn’t too much I can do about that, besides paging, but the data may not be paged on the page that is consuming this API call. Of course, when I built the logging system, I didn’t think I would be recording 1.5MB records into this temporary database… until I started getting Win32Exception timeouts. Writing to a separate temp database. On SSDs. In a RAID configuration.

Ouch.

I needed something to check to see if I should even keep the logs. There’s a property on the vendors table to tell the system how many days to keep these logs for. They’re automagically flushed at the end of each day when there’s little load. Of course, having the ability for the system to record logs for a day is nice to have, so I don’t want to remove it, but I want to be able to shut it off. Because I currently can’t, having a 0 or a 1 in that field will do almost the same thing. So, I have to have something in my code to say ‘if that value is 0, then skip adding a log entirely’.

The next problem was, implementing that was fine, but I didn’t want to hit that table to lookup the value every time a log was added. I didn’t want to read it on the cctor of the class, so I decided reading the values every 30 seconds or so would be perfect – I could turn it on or off at will, to record some logs when problems are occurring and then turn it back off so I can analyze a portion without slowing things down too much.

The technique is basically setting up an object that will hold my values (in this case, a Dictionary) that will hold the vendor ID and the # of days to keep records; a lock object, since these are APIs and the lifecycle initializes the class, runs the method and destroys the class; a timer to execute the function on a schedule.

The following class allows me to achieve this. See below on how I used it. Remember, you must lock every time it is used or you may get a dirty read at best or worse yet NullReferenceException. (I almost said “at worst, ” but backtracked because I’m sure with some ingenuity something far worse could happen.)

““

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Timers;

namespace MCS.Lottery.Infrastructure.Logic
{

///
/// Used to safely execute an action every so often.
///

///
public class MultithreadedAccessTaskExecuter
{

///
/// The object used in multiple threads.
///

public TObject Object { get; set; }

///
/// The lock object used when accessing from inside and outside of this class.
/// Use this in a lock statement if accessing from anywhere.
///

public object LockObject { get; } = new object();

///
/// The action to take when executing. Be sure to lock on the lock object, the first parameter,
/// before doing anything with the lock object.
///

private Action ActionToExecute { get; set; }

///
/// Repeats the action every N milliseconds.
/// If null, the action will not be repeated at an interval.
///

public double? ExecuteEveryMilliseconds
{
get { return timer.Interval; }
set
{
if (value == null)
{
timer.Enabled = false;
}
else
{
timer.Interval = value.Value;
timer.Enabled = true;
}
}
}

///
///
///

public void Execute()
{
ActionToExecute();
}

private Timer timer = new Timer();

public MultithreadedAccessTaskExecuter(double? ms, Action whenExecuted)
{
if (whenExecuted == null)
throw new ArgumentNullException(nameof(whenExecuted));
ExecuteEveryMilliseconds = ms;
ActionToExecute = whenExecuted;
timer.Elapsed += Timer_Elapsed;
}

private void Timer_Elapsed(object sender, ElapsedEventArgs e)
{
Execute();
}
}
}

““

How I used it:

““

private static Action waAction = new Action(() =>
{
lock (WalletApplicationExpiries.LockObject)
{
if (WalletApplicationExpiries.Object == null)
WalletApplicationExpiries.Object = new Dictionary();
using (var db = Data.DataFactory.LotteryDB())
{
var walletapps = db.WalletApplications.Select(w => new { w.ID, w.DaysToKeepAPIRecordsFor }).ToArray();
foreach (var w in walletapps)
{
WalletApplicationExpiries.Object[w.ID] = w.DaysToKeepAPIRecordsFor;
}
}
}
});

public static MultithreadedAccessTaskExecuter<Dictionary> WalletApplicationExpiries = new MultithreadedAccessTaskExecuter<Dictionary>(30000, waAction);

/////////////

public static void TrackPerformance(APIPerformanceTrackingEntry entry)
{
if (entry == null) return; // do nothing.

short wakeepdays = 1;
if (entry.WalletApplicationID.HasValue)
{
lock (WalletApplicationExpiries.LockObject)
{
wakeepdays = WalletApplicationExpiries.Object[entry.WalletApplicationID.Value];
}
}

if (wakeepdays > 0)
{
….

““

Advertisement