8000 Async replication improvements · Issue #886 · rockstor/rockstor-core · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Async replication improvements #886

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
schakrava opened this issue Sep 14, 2015 · 3 comments
Closed

Async replication improvements #886

schakrava opened this issue Sep 14, 2015 · 3 comments
Assignees
@schakrava
Labels
bug Critical
Milestone
Tuolumne

Comments

@schakrava
Copy link
Member

Appliance to Appliance asynchronous replication of Shares stopped working after 4.x kernel updates.

My personal replication setup is also in a broken state. :(
@schakrava schakrava added the bug label Sep 14, 2015
@schakrava schakrava self-assigned this Sep 14, 2015
@schakrava schakrava added this to the Toulumne milestone Sep 14, 2015
@iFloris
Copy link
iFloris commented Sep 14, 2015

Bummer, but that is the real dogfooding :) So kudos for that.

@schakrava
Copy link
Member Author

http://forum.rockstor.com/t/rockstor-3-8-8-on-vsphere-6-0-u1-storage-issue/547

schakrava added a commit to schakrava/rockstor-core that referenced this issue Nov 2, 2015
@schakrava
update kernel to 4.2.5 rockstor#886
adeb481
schakrava added a commit to schakrava/rockstor-core that referenced this issue Nov 2, 2015
@schakrava
get service names from the db. rockstor#886
7b41872
schakrava added a commit to schakrava/rockstor-core that referenced this issue Nov 2, 2015
@schakrava
if there is no designated mgmt interface, default to the first. rocks…
16f5087 
…tor#886
schakrava added a commit to schakrava/rockstor-core that referenced this issue Nov 2, 2015
@schakrava
update the ip attribute of current appliance if it changed. rockstor#886
019f7d5
schakrava added a commit to schakrava/rockstor-core that referenced this issue Nov 5, 2015
@schakrava
properly differentiate between ids and name regex. rockstor#886
7aaf8c1
schakrava added a commit to schakrava/rockstor-core that referenced this issue Nov 5, 2015
@schakrava
update error message check. rockstor#886
4018a1a
schakrava added a commit to schakrava/rockstor-core that referenced this issue Nov 5, 2015
@schakrava
properly deserialize datetime strings. rockstor#886
f8783d6
schakrava added a commit to schakrava/rockstor-core that referenced this issue Nov 5, 2015
@schakrava
make exception logging a bit cleaner. rockstor#886
1a9016d
schakrava added a commit to schakrava/rockstor-core that referenced this issue Nov 5, 2015
@schakrava
update the snapshots location. rockstor#886
6651f6c
schakrava added a commit to schakrava/rockstor-core that referenced this issue Nov 5, 2015
@schakrava
properly serialize datetime objects before making for api calls. rock…
3b7cd68 
…stor#886
schakrava added a commit to schakrava/rockstor-core that referenced this issue Nov 5, 2015
@schakrava
replication interface fetch should always pick one with an ip address. …
a4e1d6a 
…rockstor#886
schakrava added a commit to schakrava/rockstor-core that referenced this issue Nov 5, 2015
@schakrava
Allow deletes of snapshots created outside of web-ui. rockstor#886
0936663
schakrava added a commit to schakrava/rockstor-core that referenced this issue Nov 5, 2015
@schakrava
simplify network interface and uuid gathering of current appliance. r…
c121d01 
…ockstor#886
schakrava added a commit to schakrava/rockstor-core that referenced this issue Nov 5, 2015
@schakrava
make delete of special snapshots idempotent. rockstor#886
dfebdb7
schakrava added a commit to schakrava/rockstor-core that referenced this issue Nov 5, 2015
@schakrava
more cleanup of exception logging. rockstor#886
5468910
schakrava added a commit to schakrava/rockstor-core that referenced this issue Nov 6, 2015
@schakrava
improve logic to include snapshots of snapshots in the list rockstor#886
d3b7398
schakrava added a commit to schakrava/rockstor-core that referenced this issue Nov 6, 2015
@schakrava
make receive paths same as normal snapshot paths. rockstor#886
6cb8f87
schakrava added a commit to schakrava/rockstor-core that referenced this issue Nov 6, 2015
@schakrava
exception logging improvement. rockstor#886
08652b2
schakrava added a commit to schakrava/rockstor-core that referenced this issue Nov 6, 2015
@schakrava
improve sleep time calculation and comments. rockstor#886
5114387
schakrava added a commit to schakrava/rockstor-core that referenced this issue Nov 6, 2015
@schakrava
simplify sender appliance validation and validate connectivity. rocks…
682eb82 
…tor#886
schakrava added a commit to schakrava/rockstor-core that referenced this issue Nov 6, 2015
@schakrava
minor correction of blocking timeout from 600 to 60 seconds. rockstor…
64568bb 
…#886
schakrava added a commit to schakrava/rockstor-core that referenced this issue Nov 6, 2015
@schakrava
improve sleep time diff more accurately. Also don't let q processing
6c320cd 
block for more than 1000 messages at a time. rockstor#886
schakrava added a commit to schakrava/rockstor-core that referenced this issue Nov 6, 2015
@schakrava
remove hard coded value for num snaps to retain by sender. rockstor#886
567e589
schakrava added a commit to schakrava/rockstor-core that referenced this issue Nov 7, 2015
@schakrava
fix replica get by id. rockstor#886
1ffb890
schakrava added a commit to schakrava/rockstor-core that referenced this issue Dec 7, 2015
@schakrava
Coordinate the start of a send-diff with the receiver. rockstor#886
7338508
schakrava added a commit to schakrava/rockstor-core that referenced this issue Dec 7, 2015
@schakrava
if parent snapshots match between sender and receiver, check that it
fb17ce0 
exists in the system. rockstor#886
schakrava added a commit to schakrava/rockstor-core that referenced this issue Dec 7, 2015
@schakrava
add regext support in deleting old snapshots to delete only replication
cf0cdc7 
related ones. rockstor#886
schakrava added a commit to schakrava/rockstor-core that referenced this issue Dec 7, 2015
@schakrava
simplify the sender initiation and throw appropriate exceptions. rock…
a8ff7d3 
…stor#886
schakrava added a commit to schakrava/rockstor-core that referenced this issue Dec 8, 2015
@schakrava
simplify pollling and have one reactor for the broker process. rockst…
5b1db03 
…or#886
schakrava added a commit to schakrava/rockstor-core that referenced this issue Dec 8, 2015
@schakrava
Dynamically switch to sending full snapshot backup. rockstor#886
acd820c 
Sometimes the admin may want to trigger a full replication without
having to delete and recreate a new replication task. Or sometimes, full
replication may be necessary if snapshots are accidentally deleted on
either side or go out of sync in some other way.
schakrava added a commit to schakrava/rockstor-core that referenced this issue Dec 8, 2015
@schakrava
Add retry logic to replication task triggering script. rockstor#886
0d83a54
schakrava added a commit to schakrava/rockstor-core that referenced this issue Dec 8, 2015
@schakrava
refresh share state along with snapshot state. rockstor#886
3079cf8
schakrava added a commit to schakrava/rockstor-core that referenced this issue Dec 8, 2015
@schakrava
Make polling and waiting for replies uniform. rockstor#886
769c2c2 
When retries are possible and appropriate, poll for upto 6 seconds and
up to 10 retries, so up to 60 seconds for each message. But once the
snapshot data is being transmitted, we can't(yet) have a robust retry
mechanism, so in these cases poll for 60 seconds once.
schakrava added a commit to schakrava/rockstor-core that referenced this issue Dec 8, 2015
@schakrava
set ipc endpoint in settings. rockstor#886
3a3827e
schakrava added a commit to schakrava/rockstor-core that referenced this issue Dec 8, 2015
@schakrava
correctly set the ipc endpoint. rockstor#886
08598aa
schakrava added a commit to schakrava/rockstor-core that referenced this issue Dec 8, 2015
@schakrava
Throw receiver-init-error if there's a pre-existing/live receiver. ro…
f4306fe 
…ckstor#886
schakrava added a commit to schakrava/rockstor-core that referenced this issue Dec 8, 2015
@schakrava
Abort sender if remote end vanishes. rockstor#886
594fc9b
schakrava added a commit to schakrava/rockstor-core that referenced this issue Dec 8, 2015
@schakrava
make debug log messages consistent. rockstor#886
d08898a
schakrava added a commit to schakrava/rockstor-core that referenced this issue Dec 8, 2015
@schakrava
Rename files and remote deprecated(pub/sub) files. rockstor#886
7eca916 
This commit marks the official switch over to the new implementation.
Old scheduler.py, sender.py and receiver.py are deleted.
now, listener_brokery.py replaced the old scheduler and new
sender/receiver .py files replaced the old ones.
schakrava added a commit to schakrava/rockstor-core that referenced this issue Dec 8, 2015
@schakrava
cleanup settings, imports and variables. rockstor#886
0365f9f
schakrava added a commit to schakrava/rockstor-core that referenced this issue Dec 9, 2015
@schakrava
Add configure option for Replication service. rockstor#886
3668ce6 
User must pick a network interface and can customize the listener port.
schakrava added a commit to schakrava/rockstor-core that referenced this issue Dec 9, 2015
@schakrava
get replication address,port to use from the service config. rockstor…
1344500 
…#886
schakrava added a commit to schakrava/rockstor-core that referenced this issue Dec 9, 2015
@schakrava
update replication task add/edit for backend changes. rockstor#886
fc18089
schakrava added a commit to schakrava/rockstor-core that referenced this issue Dec 9, 2015
@schakrava
If no successful replica trail is found, start a Full-Send instead of
0eb5417 
erroring. rockstor#886
schakrava added a commit to schakrava/rockstor-core that referenced this issue Dec 9, 2015
@schakrava
pickup remote listener port from the task config. rockstor#886
7649bdd
schakrava added a commit to schakrava/rockstor-core that referenced this issue Dec 9, 2015
@schakrava
Update the trail status if receiver is unreachable. rockstor#886
3a0d608
schakrava added a commit to schakrava/rockstor-core that referenced this issue Dec 9, 2015
@schakrava
exclude transient Replication related Shares from api response. rocks…
311762b 
…tor#886
schakrava added a commit to schakrava/rockstor-core that referenced this issue Dec 9, 2015
@schakrava
add comment about transient full-send shares. rockstor#886
4d28015
schakrava added a commit to schakrava/rockstor-core that referenced this issue Dec 9, 2015
@schakrava
Log data transfer and rate updates. rockstor#886
1692c87
schakrava added a commit to schakrava/rockstor-core that referenced this issue Dec 10, 2015
@schakrava
setup socket listener for service status update. rockstor#886
90848a5
schakrava added a commit to schakrava/rockstor-core that referenced this issue Dec 10, 2015
@schakrava
truncate send/receive task trails periodically. rockstor#886
ec15fc9
@schakrava
Copy link
Member Author

Closed by #994 #1002 #1010 #1016 #1047

schakrava added a commit that referenced this issue Dec 11, 2015
@schakrava
update the default port. #886
5014f01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@schakrava schakrava

Labels
bug Critical < 3CC2 /div>
Projects
None yet
Milestone
Tuolumne
Development

No branches or pull requests
2 participants
@iFloris @schakrava
0