Consider publishing tailer separately #199
Labels
enhancement
This is considered a feature request, not currently guaranteed by the code or design today
process
Issues related to process and CI
Comments
|
Do you use all of tailer and watcher? I'm not really a fan of them at the
moment, but if you can tell me what you like then we can figure out
something.
I expected someone to want to do this, so I'm glad you asked.
…On Fri, 21 Dec 2018 at 06:48, Matthew Crenshaw ***@***.***> wrote:
Hi,
I'm currently using a chunk of mtail as a replacement for hpcloud/tail
<https://github.com/hpcloud/tail>. I was previously pulling
google/mtail/[tailer,watcher,logline], but with the latest changes on
master this isn't possible anymore. I agree with the move to internal and
I can always maintain my own copy and make updates but it would be very
useful to have the tail implementation in mtail exposed as a standalone
library. It's outside of the scope of mtail to expose a library, but maybe
useful to the golang community as a whole :)
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#199>, or mute the thread
<https://github.com/notifications/unsubscribe-auth/AC5b-wagTDNbivFdSmdvvFZOLpkxak5oks5u6-mdgaJpZM4Zczvc>
.
|
|
My usage is very roughly like: func (t *Tail) Start() error {
t.Lock()
defer t.Unlock()
t.lines := make(chan *logline.LogLine)
t.watcher := watcher.NewLogWatcher(pollDuration, !poll)
if err != nil {
defer close(t.lines)
return err
}
var opts []func(*tailer.Tailer) error
if t.FromBeginning {
opts = append(opts, tailer.OneShot)
}
t.tailer, err := tailer.New(t.lines, afero.NewOsFs(), t.watcher, opts...)
if err != nil {
defer t.watcher.Close()
defer close(t.lines)
return err
}
t.wg.Add(1)
go t.processLines()
return t.tailNew()
}
func (t *Tail) Stop() error {
t.Lock()
defer t.Unlock()
err := t.tailer.Close()
t.wg.Wait()
return err
}
// never errors but that's ok for now
func (t *Tail) tailFiles() error {
for _, filepath := t.Files {
// snip globbing stuff i'm doing myself
for file := range globbed {
if _, ok := t.files[file]; ok {
continue
}
err := t.tailer.TailPath(file)
if err != nil {
t.log.Error(err)
continue
}
t.log.Debug("start tailing %s", file)
t.files[file] = true
}
}
return nil
}
func (t *Tail) processLines() {
defer t.wg.Done()
for line = range t.lines {
// do stuff with line.Line depending on line.Filename
}
for _, file := range t.files {
t.log.Debug("stop tailing %s", file)
}
}Other methods add/remove files to be tailed on the fly |
|
Ok so you use the same core functions as mail does. I think I'm happy to
move it out of internal and let you depend on it directly.
I don't want to require use of fsnotify though, so the watcher will change,
and I don't want to force glog on anyone so that'll change too.
I don't think I'm comfortable with the obligation of making it a public API
but I'm sure it'll work out.
…On Sat., 22 Dec. 2018, 06:31 Matthew Crenshaw, ***@***.***> wrote:
My usage is very roughly like:
func (t *Tail) Start() error {
t.Lock()
defer t.Unlock()
t.lines := make(chan *logline.LogLine)
t.watcher := watcher.NewLogWatcher(pollDuration, !poll)
if err != nil {
defer close(t.lines)
return err
}
var opts []func(*tailer.Tailer) error
if t.FromBeginning {
opts = append(opts, tailer.OneShot)
}
t.tailer, err := tailer.New(t.lines, afero.NewOsFs(), t.watcher, opts...)
if err != nil {
defer t.watcher.Close()
defer close(t.lines)
return err
}
t.wg.Add(1)
go t.processLines()
return t.tailNew()
}
func (t *Tail) Stop() error {
t.Lock()
defer t.Unlock()
err := t.tailer.Close()
t.wg.Wait()
return err
}
// never errors but that's ok for nowfunc (t *Tail) tailFiles() error {
for _, filepath := t.Files {
// snip globbing stuff i'm doing myself
for file := range globbed {
if _, ok := t.files[file]; ok {
continue
}
err := t.tailer.TailPath(file)
if err != nil {
t.log.Error(err)
continue
}
t.log.Debug("start tailing %s", file)
t.files[file] = true
}
}
return nil
}
func (t *Tail) processLines() {
defer t.wg.Done()
for line = range t.lines {
// do stuff with line.Line depending on line.Filename
}
for _, file := range t.files {
t.log.Debug("stop tailing %s", file)
}
}
Other methods add/remove files to be tailed on the fly
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#199 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AC5b-64vEvkRT1F-VsQ97ox_1fREKCzyks5u7TcPgaJpZM4Zczvc>
.
|
|
Hi @jaqx0r, I started work on separating out the lib here: https://github.com/sgtsquiggs/mtail Other than removing unrelated mtail code: I removed the dependency on glog. It should still work with glog though 😄 Please let me know if I'm missing anything license-wise. I need to add a README. |
|
As another use-case, I've been dying for a simple-evcorr replacement. I grew up on perl, so it's hard for me to bad-mouth it; but in 2019, using perl is not cute anymore. I love the simple-evcorr idea (and I use it exclusively on my syslog servers for alerting), but it's very primitive and archaic, and mtail has a lot of the core pieces of a replacement. |
|
I think the tailer API needs some more work before I'm willing to move it out of
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hi,
I'm currently using a chunk of mtail as a replacement for hpcloud/tail. I was previously pulling google/mtail/[tailer,watcher,logline], but with the latest changes on master this isn't possible anymore. I agree with the move to
internaland I can always maintain my own copy and make updates but it would be very useful to have the tail implementation in mtail exposed as a standalone library. It's outside of the scope of mtail to expose a library, but maybe useful to the golang community as a whole :)The text was updated successfully, but these errors were encountered: