8000 Allow for a "live" role to manage video streams · Issue #980 · icpctools/icpctools · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Allow for a "live" role to manage video streams #980

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
ubergeek42 opened this issue May 27, 2024 · 4 comments
Open

Allow for a "live" role to manage video streams #980

ubergeek42 opened this issue May 27, 2024 · 4 comments

Comments

@ubergeek42
Copy link
Collaborator

At NAC, something happened with one of the webcam streams on on the live cds. The webcam stream was still functional on the main cds in coachview (so it wasn't an issue with the stream itself on the team machine). I clicked reset in the cds admin for the stream, and then live was able to connect successfully.

It'd be good for live to have access to the stream management page on the cds, then they could reset the streams themselves without having to ask us. I don't think there's any policy reason for them not to have access.

image

@deboer-tim
Copy link
Collaborator

There is no video state stored in the CDS, so I would suggest looking at the logs to see why it wasn't connecting. It could be that the camera was refusing connection to this client but not others, or kept the stream open but stopped sending data and some configuration change or fix (lower connection error timeout?) would make this work for all clients without human intervention.

Conversely, it's not hard to provide a different user role and give Live access to this page, but it is due to historic reasons that we usually run multiple CDSs. If we keep them separate, this change implies that Live will reset their CDS but someone else must still continue to monitor and reset other CDSs (e.g. for coach view). Or if we decide to run one CDS, do we really want to give Live the option to reset coachview? Either way, better to fix the underlying problem.

@ubergeek42
Copy link
Collaborator Author

It seemed like "max" for some connections showed a few hundred, and current was 48 the second time we looked and found it being failed. But then it reproduced a few minutes later after just running a few curl's.

I think maybe it's because both the main cds (via coachview) and the live cds were both connecting to the team machine feed at the same time? Or maybe we needed to fiddle with eager/lazy/lazy close; I don't know the difference but the live cds was set to lazy.

The logs didn't seem to have anything useful, and it seemed like it was closing the connections when the clients went away at least some of the time. Maybe live was pulling too many streams at once? Or somehow not closing their connections properly?

@deboer-tim
Copy link
Collaborator

A max of a few hundred means there was a time when there were that many clients at once, which does imply either someone isn't closing connections properly, there was a networking issue, or someone was (hopefully unintentionally) DoSing or testing connection limits. If the same problem happened with a few curls that wasn't the root problem though.

So you could have cameras that don't support/lock with more than one client, which means you'd have to route all video traffic through one CDS (or disable it on others). One interesting thing to confirm is if you hit the problem, disconnect all clients (i.e. 'current' is '0' on the web page) and reconnect, does it work? If so it could indicate camera just stops sending bytes and we need a lower connection timeout to at least auto-reset in this case.

@ubergeek42
Copy link
Collaborator Author

I was doing something like curl http://cds/api/contests/nac/stream/66 | xxd | head to test, and running that a few times did seem to raise "current" connections from 0 up to 5. I'm not sure how quickly the connections should be cleaned up? I'd have expected when curl quits for the connection to disappear from the cds. i.e. I don't think the number should have gone over 1...

We did save the cds logs onto the external disk, so we can get them from Sam/Troy. The stream(s) in question were 66/67 (team 15), it was the team who was doing well and live was I think almost constantly connected/pulling that stream (along with lots of others).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants
0