8000 Is it possible to load data from Source to Dest between servers? · Issue #11 · square/ETL · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Is it possible to load data from Source to Dest between servers? #11

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
shut000 opened this issue Dec 10, 2014 · 4 comments
Open

Is it possible to load data from Source to Dest between servers? #11

shut000 opened this issue Dec 10, 2014 · 4 comments

Comments

@shut000
Copy link
shut000 commented Dec 10, 2014

Is it possible to load data from Source to Dest between servers? Like what ETL usually does. Thanks!
And I don't know where to add questions, just put it here...
@jeffreyiacono
Copy link
Collaborator

hmm, you'd probably need to expand the ETLs class's ability to hold more than one connection (source_connection / destination_connection) so you could pull from one server and send to another. Let me know if that makes sense and answers your question.

@shut000
Copy link
Author
shut000 commented Feb 4, 2015

thank you! I tried another one (activewarehouse-etl), it's more complex and support multiple servers. I'm new in Ruby and trying to break it down...

@Emerson
Copy link
Emerson commented Jul 10, 2015

interesyed in this functionality as well. Wondering what the API might look like.

@jeffreyiacono
Copy link
Collaborator

@Emerson @shut000 was just giving this a look over and have an idea on what could work here: right now we have ETL.connection that's used to speak with a single db via #query. This works fine when doing data processing within the same server / db, but totally see your use case. My thought: add a ETL.read_connection and ETL.write_connectionthat you can set and then interact with through two new methods#readand#writewhich would use either{read,write}_connection#query`. One requirement is we'd need to hold the read data in memory and then prepare it to be written, but that shouldn't been too bad!

Happy to take a pass at doing the above, let me know if it's still something you're interested in!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@jeffreyiacono @Emerson @shut000
0