Ensure IO is properly closed when importing NewPipe subscriptions #4346
Conversation
|
|
||
| user.watched += db.query_all("SELECT url FROM streams", as: String) | ||
| .map(&.lchop("https://www.youtube.com/watch?v=")) | ||
| temp = File.tempfile(".db") do |tempfile| |
There was a problem hiding this comment.
Would we maybe want to randomly generate part of the file file's name? (ex: .db-{UUID}) I imagine this might cause some issues if invidious starts to use Concurrency
There was a problem hiding this comment.
Yes, that might be a huge problem! The ideal option would be ti use in-memory databases and use sqlite3_serialize, but that requires to make a PR to crystal-sqlite3
There was a problem hiding this comment.
Yes, that would be even better! I imagine there might need to be a PR made here as well (if serialization is supported by other databases): crystal-db
There was a problem hiding this comment.
In memory-databases are already supported in crystal-sqlite3 like so:
DB.open "sqlite3::memory" do | db |
db.exec "create table contacts (name text, age integer)"
db.exec "insert into contacts values (?, ?)", "John Doe", 30
endThe only thing that needs to be added is a way to actually access an in-memory database.
There's no need for a PR directly imo, an issue would be enough.
There was a problem hiding this comment.
8c15384 to
67f98cb
Compare
fcca23d to
67b362e
Compare
67b362e to
702982a
Compare
702982a to
77de46f
Compare
77de46f to
2d1437f
Compare
2d1437f to
acfd07e
Compare
acfd07e to
bba1769
Compare
|
Thank you for your work here! This should help make Invidious more stable on large instances 🎉 |
File.open accepts a second argument which will close the initial stream (
IO::Memory.new(body)) when the File stream closes.I also made it so that the code is a bit more safe with closing connections. ex: if an error occurs. This may also fix some potential memory leaks