Rework CLI UI/UX #26

ragnarlonn · 2016-12-22T10:29:10Z

Imported from https://trello.com/c/wRUf1Gbf/101-rework-cli-ui-ux

https://zapier.com/engine/hydrate/875802/.eJwVjEEOwiAQAP-y5x4syLLhEX6BLBQswaIpm5im6d_F68xkTiitC7eYfFnAzXdETdZOkEt6Lb7xlsABi3Bct9SkwwRxTbH6mg5wZA3d1EDvJsN6OT7__jGq-uX92cGdEHkfazBkAusQtFUZZ1RIWfFCCNf1A22HKgk:1cK0cU:wREbIdBbmX3oQe_hoxXtsh7YuRI/

ragnarlonn · 2016-12-22T10:55:58Z

Emily Ekberg
@davidrosen23 On it

Provide percentiles for trend metrics
Dec 1 at 10:41 PM

Emily Ekberg
@davidrosen23 On it

Provide percentiles for trend metrics
Dec 1 at 10:41 PM - Reply - Add Link as Attachment - Delete

Emily Ekberg
completed Simplified CLI invocations on this cardDec 1 at 10:39 PM

David Rosen
@uppfinnarn - rather than having up to 6 decimals on metrics output in stdout, can we reduce to e.g. 2 and then add e.g. 95th percentile metric?
Dec 1 at 2:28 PM - Reply - Delete

Emily Ekberg
completed Thresholds on this cardNov 30 at 6:20 PM

Emily Ekberg
completed Progress bar on this cardNov 29 at 1:24 PM

Emily Ekberg
completed Banner + test info on this cardNov 29 at 1:24 PM

Emily Ekberg
completed Groups + checks on this cardNov 29 at 1:24 PM

Emily Ekberg
completed Metrics on this cardNov 29 at 1:24 PM

Emily Ekberg
added Checklist to this cardNov 29 at 1:22 PM

Emily Ekberg
Working on the basics for this now, waiting for Maja to come up with a color scheme, but it's no rush - lots of other stuff to do first
Nov 25 at 5:44 PM - Reply - Delete

Pepe Cano Ruiz
@uppfinnarn Now, I understand the purpose of the UI.

The new group output looks better IMO.
Nov 12 at 5:49 PM - Reply - Delete

Emily Ekberg
moved this card from Backlog to In progressNov 11 at 6:05 PM

Emily Ekberg
I tried to make some changes to the group output, how's this? Also includes a nested group.
Nov 11 at 5:45 PM - Reply - Delete

Emily Ekberg
attached Screen Shot 2016-11-11 at 17.39.27.png to this card
Nov 11 at 5:44 PM - Reply

Emily Ekberg
Whoops, I didn't even think about making it work on a white background… maybe someone who has an eye for color could help me pick some? ( @majaloadimpact?)

We also can't really do much along the lines of real-time updates in a console, which is exactly why the web UI exists - but if we can get this UI working properly, we could flip --run back to --paused (start tests immediately) and default to quitting at the end of a test, as the same information will be presented at the end of a test!

The web UI also helps ensure that the API is actually functional and usable to work with, since I'll have to work with it myself.
Nov 11 at 5:43 PM - Reply - Delete

Pepe Cano Ruiz
@uppfinnarn The structure (config/running/assertions/metrics) makes a lot of sense to me.

All the details comes to my mind:

As David says; you have to check the colours.
Welcome to speedboat! -> We could make it better
if I am not wrong, you will remove web ui (we don't support) .
Normally, test duration will be from 1 to X minutes:
a) Ideally, if possible, we could show some information (current running assertion or ....)
b) Otherwise, let's say something while running

Could you represent a nested grouping?
I think, the nested group will be nested with indentation using the yellow color and same icon. I like the icon, I don't see a need for a header because it's obvious (the user has written the group name in the test).
The 100% - 100 x 0 is redundant. The simple tick + green makes it obvious, I wouldn't show this info.
When it fails 95% - 100 5 could be shown in the same line. If you don't show the pass case (see above point). You could show it like --> 5% 10/200
Counter metrics (http_reqs and errors) could be slightly separated and different than the other metrics.
errors counter metric could be replaced to be shown as summary of all the assertions in the test; the summary should be shown below the last assertion.
Example: (content could be different)

3 assertions of 5 passed, 2 failed. (in red) --> From QUnit
5 tests completed. (in red) --> From Mocha
Nov 10 at 9:27 PM (edited) - Reply - Delete

David Rosen
See https://cl.ly/0f3r363y0e1W. My default terminal is white background, the gray coloring does not work out...
With that said, I'm unaware of terminal output color scheme "patterns". Perhaps @majaloadimpact can help here?

@uppfinnarn - The "front page" and "some article" sections, are those groups? If so, can we make that more clear by at least adding a header? Also, in the case of nested groups in scripts, would this output also reflect that?

Lastly, is there a general need for "min" as an output metric? Would it not be more valuable to instead report e.g. 95th percentile?
Nov 10 at 7:47 PM - Reply - Add Link as Attachment - Delete

Emily Ekberg
output_demo.py is my interactive sample, I'd love to hear any feedback you have.

I'm particularly iffy about what information to present at the top and how to do it, along with the metrics at the bottom - the fields are supposed to be in order, but I couldn't think of any good way to present them.

For machine-readable output, I'm thinking of having the option to output YAML or JSON rather than human-readable output, I don't think we can have both be the same.
Nov 10 at 4:39 PM - Reply - Delete

Emily Ekberg
attached output_demo.py to this card
Nov 10 at 4:35 PM - Reply

Pepe Cano Ruiz
@uppfinnarn I saw it, at first glance, it looked very nice.

I didn't review deeply the information being presented. Let me know if you have any specific question.
Nov 10 at 4:15 PM - Reply - Delete

Emily Ekberg
I wonder if @ppcano has any opinions on this, I'm no UI designer, so I'm sure someone else could do better.
Nov 10 at 4:08 PM - Reply - Delete

Emily Ekberg
added Maja Vojnoska to this cardNov 7 at 11:13 AM

Emily Ekberg
Screen Shot 2016-11-04 at 16.45.57.png

Some kind of idea I threw together, very much unfinished.
Nov 7 at 11:13 AM - Reply - Delete

Emily Ekberg
attached Screen Shot 2016-11-04 at 16.45.57.png to this card
Nov 7 at 11:12 AM - Replycompleted Simplified CLI invocations on this cardDec 1 at 10:39 PM