You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We encounter AccessViolationExceptions in SQLiteDistributedLock.UpdateExpiration. Mainly in debug mode, but we also encouter problems with the release version, were hangfire storage gets corrupted which might be caused by this exception.
It looks like two threads are accessing the same SQLiteConnection instance at the same time. First one is the heartbeat timer from SQLiteDistributedLock the second one in our case is RecurringJobScheduler tryiing to Release another DistributedLock.
Thread 29892 and thread 34292 use the same connection instance.
I think this can always happen, because the connection is shared between the consumer of the lock an the lock itself.
Can be reproduced with the following test:
[Fact]
//this will result in an access violation exception caused by multithreaded usage of the same connection
//(distributed lock heartbeat is trying to update expiry on owners connection).
public void Use_Connection_When_Heartbeat_Fires()
{
UseConnection(database =>
{
using var slock = SQLiteDistributedLock.Acquire("resource1", TimeSpan.FromSeconds(10), database, new SQLiteStorageOptions
{
DistributedLockLifetime = TimeSpan.FromSeconds(5) //heartbeat every second
});
var start = DateTimeOffset.Now;
//generate some pressure on connection to increase probability of race condition with heartbeat timer thread
while (DateTimeOffset.Now - start < TimeSpan.FromMinutes(1))
{
database.Database.Insert(new JobParameter()
{
ExpireAt = start.AddSeconds(15).UtcDateTime,
JobId = 13,
Name = "MyParameter",
Value = "MyValue"
});
}
});
}
Using SQLiteOpenFlags.FullMutex is a workaround for the problem, but I think that should not be the final solution.
Uh oh!
There was an error while loading. Please reload this page.
Version 0.4.2, .net 8:
We encounter AccessViolationExceptions in SQLiteDistributedLock.UpdateExpiration. Mainly in debug mode, but we also encouter problems with the release version, were hangfire storage gets corrupted which might be caused by this exception.
It looks like two threads are accessing the same SQLiteConnection instance at the same time. First one is the heartbeat timer from SQLiteDistributedLock the second one in our case is RecurringJobScheduler tryiing to Release another DistributedLock.
Thread 29892 and thread 34292 use the same connection instance.
I think this can always happen, because the connection is shared between the consumer of the lock an the lock itself.
Can be reproduced with the following test:
Using SQLiteOpenFlags.FullMutex is a workaround for the problem, but I think that should not be the final solution.
The text was updated successfully, but these errors were encountered: