8000 Not parsing correctly with dump1090-fa · Issue #6 · kosho/flight-track · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Not parsing correctly with dump1090-fa #6

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
donslice opened this issue Apr 26, 2018 · 3 comments
Open

Not parsing correctly with dump1090-fa #6

donslice opened this issue Apr 26, 2018 · 3 comments

Comments

@donslice
Copy link

I've gotten this set up, pointing to my dump1090-fa instance. Unfortunately, it does not have data.json, but does have aircraft.json. The included logstash parser does not parse this correctly. Was wondering if I could get some help?

aircraft.json output attached.

aircraft.txt
@donslice
Copy link
Author

Also, some parsing errors occur with "altitude":"ground", as "ground" is not an integer.

[2018-04-26T23:15:36,651][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"flight-track-2018", :_type=>"doc", :_routing=>nil}, #LogStash::Event:0x1f62838b], :response=>{"index"=>{"_index"=>"flight-track-2018", "_type"=>"doc", "_id"=>"e8E9BGMBEEpYgM88nMFH", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse [aircraft.altitude]", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"For input string: "ground""}}}}}

@donslice
Copy link
Author
donslice commented Apr 27, 2018 •
edited
Loading

Ok... After messing with it some more, I changed the codec from "json" to "json_lines" and the data began parsing mostly correct. I still have some warns, though.

Example:

[2018-04-27T04:12:44,004][WARN ][logstash.codecs.jsonlines] JSON parse error, original data now in message field {:error=>#<LogStash::Json::ParserError: Unexpected close marker '}': expected ']' (for root starting at [Source: (String)"}"; line: 1, column: 0]),
at [Source: (String)"}"; line: 1, column: 2]>, :data=>"}"},
at [Source: (String)" ]"; line: 1, column: 4]>, :data=>" ]"},
[2018-04-27T04:12:44,004][WARN ][logstash.codecs.jsonlines] JSON parse error, original data now in message field {:error=>#<LogStash::Json::ParserError: Unexpected close marker ']': expected '}' (for root starting at [Source: (String)" ]"; line: 1, column: 0]),
[2018-04-27T04:12:43,995][WARN ][logstash.codecs.jsonlines] JSON parse error, original data now in message field {:error=>#<LogStash::Json::ParserError: incompatible json object type=java.lang.String , only hash map or arrays are supported>, :data=>" "aircraft" : ["}

@SomKen
Copy link
SomKen commented Jan 20, 2019 •
edited
Loading

@donslice Sorry for the @ then a delete...

https://github.com/SomKen/flight-track

I've created a fork to fix the issues with this. It's a total hack, due to using the exec input in logstash to run curl and pipe into jq, but it got the job done.

If someone who knows how to configure logstash wants to make a PR on my fork, I'll be more than happy to merge. Based on @kosho 's creds, I'm sure he'll be more than able to fix. (notice me senpai)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@donslice @SomKen
0