Shimcache Table #6463

puffyCid · 2020-05-25T00:52:38Z

This PR adds a shimcache table to osquery for Windows systems.
Shimcache (Application Compatibility Cache, also sometimes called "AppcompatCache") contains entries of executed applications on Windows systems (it may also contain a list of executable's enumerated by Explorer when browsing the file system). Shimcache entries contain:

Path of file executed
Last modified time of the file
An execution flag on the file (this flag does not exist on Windows 10 systems)

Additional info can be found here: https://medium.com/@bromiley/windows-wednesday-shim-cache-1997ba8b13e7

This PR enumerates all the ControlSets on a Windows system and parses the Shimcache entries.
Its been tested on Windows 8.1 and Windows 10 (Creators update)
It should also work on Windows 10 (Pre-Creators) but I do not have VM to test on.
The PR also makes a few small changes to Userassist in regards timestamp conversion
Let me know if there are any issues

+-------+------------------------------------------------------------------------------------------------------------------------------+--------------------------------------+----------------+
| entry | path                                                                                                                         | DATETIME(modified_time, 'unixepoch') | execution_flag |
+-------+------------------------------------------------------------------------------------------------------------------------------+--------------------------------------+----------------+
| 1     | C:\Users\bob\AppData\Local\Temp\B7893AFE-DD60-43ED-AB8A-E0C7CEBF3BF1\dismhost.exe                                            | 2019-03-19 04:46:23                  |                |
| 2     | C:\WINDOWS\SoftwareDistribution\Download\Install\AM_Delta_Patch_1.315.1346.0.exe                                             | 2020-05-24 18:53:51                  |                |
| 3     | SIGN.MEDIA=103148F8 osqueryd.exe                                                                                             | 2020-05-24 18:37:35                  |                |
| 4     | SIGN.MEDIA=1031C8F8 osqueryd.exe                                                                                             | 2020-05-24 18:06:12                  |                |
| 5     | SIGN.MEDIA=1031C8F8 AppCompatCacheParser.exe                                                                                 | 2019-12-11 04:03:48                  |                |
| 6     | SIGN.MEDIA=E4A0A1F0 osqueryd.exe                                                                                             | 2020-05-24 01:01:14                  |                |
| 7     | SIGN.MEDIA=E8F4DD42 Microsoft ATA Center Setup.exe                                                                           | 2017-06-29 07:50:49                  |                |
| 8     | C:\Users\bob\Desktop\osqueryd.exe                                                                                            | 2020-05-24 18:37:35                  |                |
| 9     | C:\WINDOWS\SoftwareDistribution\Download\Install\AM_Delta_Patch_1.315.1290.0.exe                                             | 2020-05-24 17:49:46                  |                |
| 10    | C:\Users\bob\Projects\osquery\build\plugins\remote\enroll\tests\RelWithDebInfo\plugins_remote_enroll_tlsenrolltests-test.exe | 2020-05-24 22:24:16                  |                |
+-------+------------------------------------------------------------------------------------------------------------------------------+--------------------------------------+----------------+```

osquery/tables/system/windows/shimcache.cpp

puffyCid · 2020-06-13T00:38:20Z

@theopolis Thanks for the comments and review
All comments should be addressed, let me know if there are any other issues

theopolis

Thanks for iterating on this!

osquery/tables/system/windows/shimcache.cpp

osquery/tables/system/windows/userassist.h

specs/windows/shimcache.table

osquery/tables/system/windows/shimcache.cpp

puffyCid · 2020-07-03T02:34:07Z

@theopolis All comments should be addressed, let me know if there are any other issues

theopolis

This looks great!

One last consideration for consistency. In a lot of other tables we represent "tri-states" in columns as either {-1, 0, 1} or {NULL, 0, 1}. My recommendation is to follow that pattern for execution_flag in the schema.

So concretely:

Change

specs/windows/shimcache.table

theopolis · 2020-07-12T00:57:32Z

Looks good!

theopolis · 2020-07-12T03:10:25Z

Hi @puffyCid, I did some work to refactor+minify the code. I'd like to look at recent PRs, specifically around Windows filetime parsing, and see if we should move some of the utility functions here, elsewhere.

theopolis · 2020-07-12T21:56:56Z

Actually, we can do code shuffle in another PR.

Last question about this table: can we drop the entry column? Having it will make table comparisons difficult?

puffyCid · 2020-07-12T22:25:42Z

Actually, we can do code shuffle in another PR.

Last question about this table: can we drop the entry column? Having it will make table comparisons difficult?

I would prefer if the entry column stayed.
The entry column can help show/clarify the order of how files were executed for example

+-------+------------------------------------------------------------------------------------------------------+---------------+----------------+
| entry | path                                                                                                 | modified_time | execution_flag |
+-------+------------------------------------------------------------------------------------------------------+---------------+----------------+
| 1     | C:\WINDOWS\system32\sc.exe                                                                           | 1552970734    | -1             |
| 2     | C:\WINDOWS\system32\Windows.WARP.JITService.exe                                                      | 1552970651    | -1             |
| 3     | C:\WINDOWS\system32\browser_broker.exe                                                               | 1571637135    | -1             |
| 4     | C:\WINDOWS\SystemApps\Microsoft.MicrosoftEdge_8wekyb3d8bbwe\Application                              | 1592019201    | -1             |
| 5     | C:\WINDOWS\system32\ApplicationFrameHost.exe                                                         | 1552970663    | -1             |
| 6     | C:\WINDOWS\system32\net1.exe                                                                         | 1552970734    | -1             |
| 7     | C:\WINDOWS\system32\net.exe                                                                          | 1552970734    | -1             |
| 8     | C:\WINDOWS\system32\PING.EXE                                                                         | 1552970734    | -1             |
| 9     | C:\WINDOWS\system32\ipconfig.exe                                                                     | 1552970734    | -1             |
| 10    | C:\Program Files\WindowsApps\Microsoft.WindowsTerminal_1.0.1811.0_x64__8wekyb3d8bbwe\OpenConsole.exe | 1593651272    | -1             |
...

The above shimcache entries show that the user launched Windows Terminal, executed ipconfig.exe, ping.exe, net.exe in that order (execution order is read bottom to top, newest shimcache entries are at the top)

but if it will cause issues i can remove it if necessary.
@theopolis what do you mean by making table comparisons difficult?

theopolis · 2020-07-13T01:57:04Z

Ok, we can leave it. The problem I referenced is related to "differential" queries where osquery tries to calculate set differences between scheduled queries. If someone removes a row, for example the first sc.exe that will have a ripple effect on all other rows in shimcache because their entries will offset. If this happens osquery's differential will detect that all rows have changed, vs, detecting that the first entry is now missing.

theopolis added virtual tables Windows labels Jun 6, 2020

theopolis requested changes Jun 6, 2020

View reviewed changes

theopolis requested changes Jun 29, 2020

View reviewed changes

theopolis requested changes Jul 6, 2020

View reviewed changes

specs/windows/shimcache.table Outdated Show resolved Hide resolved

theopolis force-pushed the appcompat branch from 63cfe6c to 396b616 Compare July 12, 2020 01:03

initial support for shimcache

b0e56eb

theopolis force-pushed the appcompat branch 2 times, most recently from 5584a9b to 040d472 Compare July 12, 2020 03:00

refactor parseRegistry

89e9284

theopolis force-pushed the appcompat branch from 040d472 to 89e9284 Compare July 12, 2020 16:20

theopolis approved these changes Jul 13, 2020

View reviewed changes

theopolis merged commit 336e6b0 into osquery:master Jul 13, 2020

puffyCid deleted the appcompat branch July 13, 2020 02:48

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Shimcache Table #6463

Shimcache Table #6463

Shimcache Table #6463

Shimcache Table #6463

Conversation

Choose a reason for hiding this comment

Choose a reason for hiding this comment