Description
Issue / Inconsistency
Currently, the structure / verbiage of Voight Kampff steps appears to be a bit inconsistent. I noticed this as I was looking into implementing a few improvements re: language and skill settings.
The rhetoric is varied, as is the order of the phrasing (subject, verbs, descriptions, etc.). This is true between step types (given vs when vs then), but also true within a given type of step.
Given the opportunity for this testing suite to expand and be a common interface between the Mycroft project and community developers, I think that developing a consistent standard re: step phasing structure would be helpful.
Proposed Solution
Given Steps
As I think about the Given steps and future uses, two specific uses come to mind.
- Setting global settings (eg: language, timezone, device location)
- Setting skill-specific settings
This section seems to be largely "setting the scene", ie: setting variables. As such, I would recommend we follow a familiar structure: the phase equivalent of x = y (square brackets added for clarity of pattern; would not be in actual test structure).
- "Given the user's [language] is [English["
- "Given the user's [location] is [France]"
- "Given the skill's [SETTING_NAME] is [SETTING_VALUE]"
I think this structure provides a very consistent template / pattern for developers to follow, and allows us to easily expand Given testing options in the future while following the same general structure.
When
When steps already follow a similar pattern to Given, using "the user", not "a / an user" and leading with the subject of the phrase. No changes necessary.
- "When the user says [PHRASE]"
Then
The final type of step, Then steps, also provide an opportunity for standardization. Given Then steps are (currently) largely text / dialog-related, stndardization could follow a relatively simple pattern:
- Then
- Subject:
['mycroft', '[SKILL_NAME]'] - Action / Conditional:
['should reply with'] - Indirect Subject:
['[EXACT_STRING]', 'dialog from file [FILE]', 'anything', 'nothing']
Finally, I believe the last block in the codebase, dedicated to continued user interaction with the system (eg: "Then the user replies with [STRING]"), should be recategorized to a When step type. This aligns with the Behave documentation:
Given we put the system in a known state before the user (or external system) starts interacting with the system (in the When steps). Avoid talking about user interaction in givens.
When we take key actions the user (or external system) performs. This is the interaction with your system which should (or perhaps should not) cause some state to change.
Then we observe outcomes.
As this is an additional interaction, it deserves the When step type. I think this can be concisely communicated as When = user interaction; Then = system response. This will allow us to streamline the codebase and
Action
I am happy to make these changes in the underlying codebase / provide documentation to support it. Wanted to get the opinion of others on this topic, though.