8000 Missing columns (?) resulting in: Error casting $COLUMN () to $TYPE · Issue #6319 · osquery/osquery · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Missing columns (?) resulting in: Error casting $COLUMN () to $TYPE #6319

Closed
nyanshak opened this issue Mar 19, 2020 · 6 comments
Closed

Missing columns (?) resulting in: Error casting $COLUMN () to $TYPE #6319

nyanshak opened this issue Mar 19, 2020 · 6 comments
Labels
bug Linux macOS triage Issue needs to be verified, reproduced and prioritized

Comments

@nyanshak
Copy link
Contributor

Bug report

What operating system and version are you using?

Reproduced using Mac and Linux, and

What version of osquery are you using?

tested 4.2.0 and a few earlier versions

What steps did you take to reproduce the issue?

Run osqueryd in foreground with specific scheduled queries, with --verbose flag on.

What did you expect to see?

No errors related to query execution

What did you see instead?

I saw t 8000 his for multiple tables, but this is the pattern:

Error casting $COLUMN () to $TYPE

I'll try to list the tables below...

Querying interface_details on Linux:

I0303 12:49:21.993166 192253952 dynamic_table_row.cpp:114] Error casting enabled () to INTEGER
I0303 12:49:21.993263 192253952 dynamic_table_row.cpp:114] Error casting physical_adapter () to INTEGER
I0303 12:49:21.993315 192253952 dynamic_table_row.cpp:114] Error casting speed () to INTEGER
I0303 12:49:21.993362 192253952 dynamic_table_row.cpp:114] Error casting dhcp_enabled () to INTEGER

Table usb_devices (macOS):

I0303 10:43:10.491322 192253952 scheduler.cpp:96] Executing scheduled query pack_somepack_usb-devices: select usb.* from usb_devices as usb;
I0303 10:43:10.494263 192253952 dynamic_table_row.cpp:114] Error casting usb_address () to INTEGER

Table interface_details (macOS):

I0303 12:49:21.990605 192253952 scheduler.cpp:96] Executing scheduled query pack_somepack_network-interface-changes: select interface, mac, type, mtu, ierrors, oerrors, idrops, odrops, last_change, description, manufacturer, connection_id, connection_status, enabled, physical_adapter, speed,
 dhcp_enabled, dhcp_lease_expires, dhcp_lease_obtained, dhcp_server, dns_domain, dns_domain_suffix_search_order, dns_host_name, dns_server_search_order from interface_details;
I0303 12:49:21.993166 192253952 dynamic_table_row.cpp:114] Error casting enabled () to INTEGER
I0303 12:49:21.993263 192253952 dynamic_table_row.cpp:114] Error casting physical_adapter () to INTEGER
I0303 12:49:21.993315 192253952 dynamic_table_row.cpp:114] Error casting speed () to INTEGER
I0303 12:49:21.993362 192253952 dynamic_table_row.cpp:114] Error casting dhcp_enabled () to INTEGER

Tables hash, process_events, users, groups (macOS):

I0303 10:49:29.274178 192253952 scheduler.cpp:96] Executing scheduled query pack_somepack_audit-process-events: select h.md5, p.path, p.cmdline, p.cwd, p.gid, p.egid, p.uid, p.euid, p.pid, p.parent, p.time, u.username, eu.username as e_username, g.groupname, eg.groupname as e_groupname from
hash as h, process_events as p, users as u, groups as g join users as eu on p.euid=eu.uid join groups as eg on p.egid=eg.gid where h.path = p.path and u.uid = p.uid and g.gid = p.gid and p.cmdline NOT LIKE "%docker-credentials%" group by h.md5;
I0303 10:49:29.317302 192253952 user_groups.mm:56] Error with OpenDirectory results: An invalid attribute type was provided.
I0303 10:51:07.342941 192253952 user_groups.mm:56] Error with OpenDirectory results: An invalid attribute type was provided.
I0303 10:51:07.751210 192253952 user_groups.mm:56] Error with OpenDirectory results: An invalid attribute type was provided.
I0303 10:51:08.365211 192253952 user_groups.mm:56] Error with OpenDirectory results: An invalid attribute type was provided.
I0303 10:51:08.775454 192253952 user_groups.mm:56] Error with OpenDirectory results: An invalid attribute type was provided.
I0303 10:51:08.775894 192253952 dynamic_table_row.cpp:123] Error casting parent () to BIGINT
I0303 10:51:09.383039 192253952 user_groups.mm:56] Error with OpenDirectory results: An invalid attribute type was provided.
I0303 10:51:09.790145 192253952 user_groups.mm:56] Error with OpenDirectory results: An invalid attribute type was provided.
I0303 10:51:09.790694 192253952 dynamic_table_row.cpp:123] Error casting parent () to BIGINT
I0303 10:51:10.409380 192253952 user_groups.mm:56] Error with OpenDirectory results: An invalid attribute type was provided.
I0303 10:51:10.813109 192253952 user_groups.mm:56] Error with OpenDirectory results: An invalid attribute type was provided.

Table process_open_sockets (macOS):

I0303 12:51:56.315275 192253952 scheduler.cpp:96] Executing scheduled query pack_somepack_netstat: select pos.socket from process_open_sockets as pos;
I0303 12:51:56.329641 192253952 dynamic_table_row.cpp:123] Error casting socket (17500881079293304539) to BIGINT

Table docker_containers, docker_container_stats (macOS):

I0303 13:56:40.996515 192253952 scheduler.cpp:96] Executing scheduled query pack_somepack_docker-container-stats: select * from docker_containers as dc join docker_container_stats as dcs on dc.id=dcs.id;
I0303 13:56:42.775250 192253952 dynamic_table_row.cpp:114] Error casting pids (^Q) to INTEGER

Tables users, account_policy_data (macOS):

I0303 13:57:56.043328 192253952 scheduler.cpp:96] Executing scheduled query pack_somepack_failed-local-logins: select * from users join account_policy_data using (uid) where failed_login_count > 0;
I0303 13:57:56.226645 192253952 dynamic_table_row.cpp:123] Error casting failed_login_count () to BIGINT
@nyanshak
Copy link
Contributor Author

For additional context - I am still receiving results from these, so I think maybe just some of the columns are missing?

From #general in the osquery slack:

Stefano Bonicatti:trailofbits: 16:04
That happens when a table do not return any value for that column, which can be either "ok", in the sense that there's effectively no data to be returned (although, it would be better to explicitly set it), or can be a bug.
So it really depends.

seph:kolide: 16:12
The times I’ve looked at it, it’s been platform skew. Some columns don’t mean anything on a platform. And the platform table implementation returns nothing, which causes the cast error.

Stefano Bonicatti:trailofbits: 16:16
yeah but shouldn't those columns be under an extended_schema then? (edited)
16:17
or again explicitly set to empty, if they actually are?

seph:kolide: 16:18
Yes, those would be good ways to fix it.
16:19
Downside to extended schema, is that you need to run platform specific queries.
16:19
Anyhow, I think it’s bug when we have cast errors. The times I’ve investigated it, it’s been platform skew. We should fix them.

Stefano Bonicatti:trailofbits: 16:20
I see, makes sense!
My intention was, even though they only appear with --verbose, to have them highlight an actual issue if present and don't leave cases where they can be ignored. (edited)

seph:kolide: 16:20
I agree with you completely.

@nyanshak
Copy link
Contributor Author

Similar previous issue: #2832

@mike-myers-tob mike-myers-tob added bug Linux macOS triage Issue needs to be verified, reproduced and prioritized labels May 24, 2020
@directionless
Copy link
Member

This was partially fixed in #6371, usb_devices on macOS no longer has this issue. Unfortunately, process_open_sockets still does

@directionless
Copy link
Member

The process_open_sockets appears to be because macOS represents the underlying socket as a unsigned 64bit number, and is returning some sockets with values that high. While sqlite's integers are signed 64bit. I'm not sure what the right way out is, probably not using floating point in sqlite ;)

directionless added a commit to directionless/osquery that referenced this issue Jul 7, 2020
@directionless
Fix process_open_sockets type error on darwin
352eba9 
On darwin, `soi_so` is an unsigned 64bit number. sqlite, however, is
signed. Current behavior is to emit a warning, and drop the column.
It's not clear we have a good fix for the type mismatch, so comment and
drop it in the virtual table instead.

Relates to: osquery#6319
@directionless directionless mentioned this issue Jul 7, 2020
Merged
directionless added a commit to directionless/osquery that referenced this issue Jul 25, 2021
@directionless
Fix process_open_sockets type error on darwin
b35d2d9 
On darwin, `soi_so` is an unsigned 64bit number. sqlite, however, is
signed. Current behavior is to emit a warning, and drop the column.
It's not clear we have a good fix for the type mismatch, so comment and
drop it in the virtual table instead.

Relates to: osquery#6319
@noahtalerman noahtalerman mentioned this issue Feb 26, 2024
Closed
@zwass
Copy link
Member
zwass commented Mar 25, 2024

This was fixed by #6371 and #6546, so happily it's been resolved for years now :)

@zwass zwass closed this as completed Mar 25, 2024
@noahtalerman
Copy link

FYI @dandavison

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Linux macOS triage Issue needs to be verified, reproduced and prioritized
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@directionless @zwass @nyanshak @mike-myers-tob @noahtalerman
0