8000 State machine unit testing · Issue #687 · nasa/fpp · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

State machine unit testing #687

8000
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
zimri-leisher opened this issue May 2, 2025 · 7 comments
Open

State machine unit testing #687

zimri-leisher opened this issue May 2, 2025 · 7 comments
Assignees
@bocchino
Labels
proposed feature A proposed new feature

Comments

@zimri-leisher
Copy link
zimri-leisher commented May 2, 2025 •
edited
Loading

The state machines could have good unit testing support, but at the moment they are difficult to unit test.
I'd like:

  1. ASSERT_SIGNALS__SIZE(signal count)
  2. ASSERT_SIGNAL_(signal idx, signal type, signal value if it has one)
  3. ASSERT_STATE_TRANSITION__SIZE(transition count)
  4. ASSERT_STATE_(current expected state)
  5. ASSERT_ACTION_<sm_name>_<action_name>_SIZE(count of times action was raised)
  6. ASSERT_ACTION__(index of action instance, action value if it has one)

Without at least some of these it is very difficult to unit test functions that raise signals. I would have to manually parse the component's message queue
@bocchino bocchino added the proposed feature A proposed new feature label May 13, 2025
@bocchino
Copy link
Collaborator

I'm not exactly sure what is being proposed. The way we currently test state machines is reflected here:

https://github.com/nasa/fprime/tree/devel/FppTest/state_machine/internal_instance

The testing uses some helper classes here:

https://github.com/nasa/fprime/tree/devel/FppTest/state_machine/internal/harness

Would it be sufficient to document these helper classes so users can use them to write tests? If not, can you provide a detailed example that shows what you would like to do that's not being done here?

@bocchino
Copy link
Collaborator
bocchino commented May 20, 2025 •
edited
Loading

Are you requesting an autocoded state machine tester, similar to what we have for the component tester? Or are you looking for some kind of introspection into behavior history of a state machine that's instantiated inside a component? In the first case, I can envision how that would work. In the second case, I'm not so sure. It sounds like you may want test-only instrumentation of the component flight code.

@zimri-leisher
Copy link
Author
zimri-leisher commented May 20, 2025 •
edited
Loading

An autocoded state machine tester like the ASSERTions in the GTestBase/TesterBase AC files, adding utilities that let me check in component unit tests whether expected SM signals are raised in component port implementations or similar. So if I'm understanding you right, the first case is what I'm asking for.

I would imagine the implementation would be similar to the current impl for ASSERT_EVENTS_XYZ_SIZE or whatever. A new History class in the TesterBase and some new AC functions in GTestBase.

@bocchino
Copy link
Collaborator
bocchino commented May 20, 2025 •
edited
Loading

An autocoded state machine tester like the ASSERTions in the GTestBase/TesterBase AC files, adding utilities that let me check in component unit tests whether expected SM signals are raised in component port implementations or similar.

This sounds like the second case to me. It sounds like you want the component test code to record information about the internal behavior of the component implementation, e.g., how many times the component handlers have sent a certain signal to a state machine instance. To do this I think we would need to instrument the component code with trace or debug information.

The first case would provide a way to instantiate a state machine outside of a component and test its behavior, similarly to what's being done in FppTest.

< 8000 /div>
@bocchino
Copy link
Collaborator
bocchino commented May 20, 2025 •
edited
Loading

I would imagine the implementation would be similar to the current impl for ASSERT_EVENTS_XYZ_SIZE or whatever. A new History class in the TesterBase and some new AC functions in GTestBase.

But those history classes are recording external behavior of the component (how it is sending data on its ports). It sounds like you are looking for introspection into the internals of the component implementation. E.g., sending a signal in a component handler to a component state machine does not in itself cause any externally visible behavior like sending data on a port.

@bocchino
Copy link
Collaborator

@garthwatney Can you comment on this issue?

@zimri-leisher
Copy link
Author

Ah I see what you're saying. Yes, it seems like there would need to be instrumentation in the component for this. However, I'd be satisfied with an approach which just lets us check what's currently in the component's queue. If we could assert that a particular signal is in the work queue of the component, then that would be fine for my purposes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@bocchino bocchino

Labels
proposed feature A proposed new feature
Projects
F´ Development
Status: CCB
Milestone
No milestone
Development

No branches or pull requests
2 participants
@bocchino @zimri-leisher
0