Messaging pacts - WithContentAsync - factory is invoked before provider state has been set #459
Comments
|
You can see here that the call is awaited immediately upon being invoked: And then it sets the resultant value as a sync factory internally. We know that's definitely worked because otherwise it throws an exception because the internal factory value is null. I suspect this is more a problem with the example than the code itself, because firstly it's capturing variables inside lambdas, which is always fraught with danger, and secondly it's awaiting The only other thing I can think of is because the variable is a dynamic so perhaps it's not awaiting properly, but I doubt that given it still sets the internal state properly. |
|
Thanks @adamrodger, that makes some sense. I wasn't expecting the factory to be awaited immediately. The way public async Task WithContentAsync(Func<Task<dynamic>> factory)
{
this.factory = () => factory().GetAwaiter().GetResult;
}or alternatively, declare MessageScenarioBuilder.factory as |
|
I'm not sure that's the case. Given the method is called and awaited immediately then the value must be available at that point: public async Task WithContentAsync(Func<Task<dynamic>> factory)
{
dynamic value = await factory();
// the value is now known because the task must have resolved already
this.factory = () => value; // this just re-wraps the value that already exists so it can be retrieved sync later on
}So there's no way for the supplied factory method to somehow not have completed yet, because execution isn't passed back to the user code until the factory method completes due to the await point. The problem is in your example - you aren't awaiting the setup part yourself, and thus the code proceeds to the verify part before the setup part completes. That's a fault with the example code, not with PactNet. verifier.MessagingProvider("Message Producer", defaultSettings)
.WithProviderMessages(scenarios =>
{
scenarios.Add("some events", scenarioBuilder =>
{
////////// THIS IS NOT AWAITED ////////////
return scenarioBuilder.WithContentAsync(async () =>
{
var events = new List<MyMessage>
{
new MyMessage
{
MessageText = "Hello World"
}
};
Assert.That(isProviderStatesCalled, Is.True);
await Task.CompletedTask;
return events;
});
});
}).WithFileSource(new FileInfo(Path.Join(pactDir, "Message Consumer-Message Producer.json")))
.WithProviderStateUrl(new Uri(new Uri(pactProviderServiceUri), "/provider-states"))
.Verify(); |
@adamrodger that's the point. The method is called and awaited too soon - before the call has been made to the provider-states endpoint. |
|
Ah ok I see the confusion - the API isn't intended to be a callback that's executed for each interaction during the verification. It's a one-time setup to create some canned responses which are analogous to what the service would publish at runtime. So that means those factory methods are invoked once during the verifier setup and the responses kept effectively in a dictionary of magic string to canned response. A callback feature is potentially possible, but it's not a feature that exists in PactNet currently. So yeah, this isn't a defect in the existing functionality, it's more a misunderstanding of how it's intended to work. |
|
This still doesn't explain why the POST to the provider state url is made before the factory runs in a synchronous scenario, but runs after in an async scenario. Perhaps the provider state configuration should be removed entirely from messaging pacts, as otherwise it is very confusing and of no use when WithContentAsync is used. Thanks very much for your help in explaining how this all works though. |
|
What's required to move this forward - is there a proposed change, documentation update etc.? |
|
Hi @mefellows Is it possible to update the documentation to show a working example of a messaging pact provider test using a state provider and This is my latest attempt. It is still failing because the POST to the provider state endpoint is not being executed until after the verification attempt. As previously, altering the code to use App.csproj <Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>net7.0</TargetFramework>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
</PropertyGroup>
<ItemGroup>
<FrameworkReference Include="Microsoft.AspNetCore.App" />
</ItemGroup>
<ItemGroup>
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.7.1" />
<PackageReference Include="NUnit" Version="3.13.3" />
<PackageReference Include="NUnit3TestAdapter" Version="4.5.0" />
<PackageReference Include="PactNet" Version="4.5.0" />
</ItemGroup>
</Project>program.cs using Newtonsoft.Json.Serialization;
using Newtonsoft.Json;
using NUnit.Framework;
using PactNet;
using PactNet.Matchers;
using PactNet.Verifier;
using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Hosting;
using System.Net;
using Microsoft.AspNetCore.Http;
// create pact
var pactDir = Path.Join("..", "..", "..", "pacts");
var v3 = Pact.V3("Message Consumer", "Message Producer", new PactConfig
{
PactDir = pactDir,
DefaultJsonSettings = new JsonSerializerSettings
{
ContractResolver = new CamelCasePropertyNamesContractResolver()
}
});
var messagePact = v3.WithMessageInteractions();
messagePact.ExpectsToReceive("some events")
.Given("events exist")
.WithJsonContent(Match.MinType(new { MessageText = "Hello World" }, 1))
.Verify<ICollection<MyMessage>>(events => Assert.That(events, Is.Not.Empty));
// verify pact
var messageSender = new MyMessageSender();
// configure provider states handler
const string pactProviderServiceUri = "http://127.0.0.1:9001";
var builder = WebApplication.CreateBuilder(new WebApplicationOptions());
builder.WebHost.UseUrls(pactProviderServiceUri);
await using var app = builder.Build();
app.MapPost("/provider-states", async context =>
{
Console.WriteLine("Adding event");
messageSender.Messages.Add(new MyMessage
{
MessageText = "Hello World"
});
context.Response.StatusCode = (int)HttpStatusCode.OK;
await context.Response.WriteAsync(string.Empty);
});
await app.StartAsync();
using (var verifier = new PactVerifier(new PactVerifierConfig
{
LogLevel = PactLogLevel.Debug
}))
{
var defaultSettings = new JsonSerializerSettings
{
ContractResolver = new CamelCasePropertyNamesContractResolver(),
DefaultValueHandling = DefaultValueHandling.Ignore,
NullValueHandling = NullValueHandling.Ignore,
Formatting = Formatting.Indented
};
try
{
verifier.MessagingProvider("Message Producer", defaultSettings)
.WithProviderMessages(scenarios =>
{
scenarios.Add("some events", async scenarioBuilder =>
{
await scenarioBuilder.WithContentAsync(async () =>
{
var events = await messageSender.GetMessagesAsync();
Assert.That(events.Any(), Is.True);
return events;
});
});
}).WithFileSource(new FileInfo(Path.Join(pactDir, "Message Consumer-Message Producer.json")))
.WithProviderStateUrl(new Uri(new Uri(pactProviderServiceUri), "/provider-states"))
.Verify();
}
catch (Exception ex)
{
Console.WriteLine($"Verification failed {ex}");
}
}
await app.StopAsync();
public class MyMessageSender
{
public List<MyMessage> Messages { get; } = new();
public async Task<IList<MyMessage>> GetMessagesAsync()
{
await Task.Delay(50);
return Messages.ToList();
}
}
public class MyMessage
{
public string MessageText { get; init; }
} |
@adamrodger that's the way the underlying Provider States FFI works. For each interaction, it first calls the provider state endpoint given to it, and then executes the test. I'm still a little confused as some of the .NET jargon is getting in the way of my complete comprehension, but if I understand correctly, you're saying it only calls this states endpoint once and stores the response?
I think just having some basic documentation explaining how it works would be a starting point. Then we can work on examples etc. |
|
The best way I can think to explain this is to look at the sync version first: The entire point is that you provide some canned message, and that canned message is registered intereally so that when the message is requested by the FFI (using the special magic string which indicates which message to return) then we have a canned message to respond with. You can see that here: So that is "a callback" in terms of the FFI - it makes a request with a special ID and the messaging provider is expected to respond with a message that matches that ID, and it does. The linked code above is to the Sometimes you can't create that canned message synchronously. For example, you may wish to load it from a JSON file, or off some network resource, or out of a database, etc. Thus, there is also an async version so that you can perform some async calls when preparing the canned data ( Once the canned data is prepared and registered, after that it doesn't matter whether it was prepared sync or async - it's just a canned message to return when the FFI sends a request for it. The confusion here is that the OP expected that this generation happens every time the FFI requests the message, but that's not how it works. It's prepared once and then cached for subsequent invocations (e.g. if verifying more than one pact, where there are multiple consumers of the same message, so they all get she same thing). If we changed it to work that way then you could get really strange situations - for example if you're loading the message from a file then you could lock the file inside the factory method, and then the first pact would pass fine and each subsequent pact would fail to verify (and may even deadlock completely, depending on what you're doing). Or you could get different pacts that think they're verifying the same message actually getting different results because something changed in the meantime. That creates obvious footguns, so that's why we only generate it once on registration and then cache if for future invocations when the FFI requests it. So I'm not sure what really needs to happen here. I think the issue just needs closing. It's working as expected. |
That isn't what I meant in my original post. It was about when the post to the provider states endpoint is made. It happens after the canned messages have been generated so it isn't really of any use. Using the synchronous set up the post to the provider states endpoint is called before the message set up code. |
We still have to invoke the factory with sync-over-async at request time and this puts a burden on the user not to use this improperly, but if we want to delay invocation of the factory until after provider states have been configured then this is the first step. This is also obviously a breaking API change and requires a major version bump. `WithAsyncContent` makes way more sense because it's the content factory that's async, not the call to the builder as `WithContentAsync` would imply. This also means the return type changes to void instead of Task. See #459
When verifying a messaging pact scenario using WithContentAsync, the factory Func is invoked before a POST to /provider-states has completed.
To reproduce, create a dotnet 7 console app. Add a FrameworkReference to Microsoft.AspNetCore.App in the .csproj file:
add the following package refs:
Edit program.cs:
On running the app, the assertion
fails because the POST to provider-states has not been made.
Changing to WithContent works:
The text was updated successfully, but these errors were encountered: