src

General Build Script

Index

General Build Script

Purpose

The build.py script was created to compile, test and validate the available modules and its code readiness. This tool was meant to developers to speed up the code readiness checking and development and to be able to include the usage of the tool as part of automation processes and checks.

Dependencies

There are several modules needed in order to the tool to work correctly.

cppcheck
valgrind
lcov
gcov
astyle
scan-build-12

Compile Wazuh

In order to run unit tests on a specific wazuh target, the project needs to be built with the DEBUG and TEST options as shown below:

make TARGET=server|agent DEBUG=1 TEST=1

How to use the tool

Once the code is compiled and built successfully the following line should be executed:

usage: python3 build.py [-h] [-r READYTOREVIEW]
[-rc READYTOREVIEWANDCLEAN] [-m MAKE]
[-t TESTS] [-c COVERAGE] [-v VALGRIND] [--clean CLEAN] [--cppcheck CPPCHECK] [--asan ASAN] [--scheck SCHECK]
[--sformat SFORMAT] [--scanbuild SCANBUILD] [--deleteLogs DELETELOGS] [--target TARGET]

Optional arguments:

Argument	Description
`-h`, `--help`	Show the help message and exit
`-r`, `--readytoreview`	Run all the quality checks needed to create a PR. Example: `python3 build.py -r <wazuh_modules/syscollector
`-d`, `--readytoreviewandclean`	Run all the quality checks needed to create a PR. Example: `python3 build.py -d <wazuh_modules/syscollector
`-m`, `--make`	Compile the lib. Example: `python3 build.py -m <wazuh_modules/syscollector
`-t`, `--tests`	Run tests (should be configured with TEST=on). Example: `python3 build.py -t <wazuh_modules/syscollector
`-c`, `--coverage`	Collect tests coverage and generates report. Example: `python3 build.py -c <wazuh_modules/syscollector
`-v`, `--valgrind`	Run valgrind on tests. Example: `python3 build.py -v <wazuh_modules/syscollector
`--clean`	Clean the lib. Example: `python3 build.py --clean <wazuh_modules/syscollector
`--cppcheck`	Run cppcheck on the code. Example: `python3 build.py --cppcheck <wazuh_modules/syscollector
`--asan`	Run ASAN on the code. Example: `python3 build.py --asan <wazuh_modules/syscollector
`--scheck`	Run AStyle on the code for checking purposes. Example: `python3 build.py --scheck <wazuh_modules/syscollector
`--sformat`	Run AStyle on the code formatting the needed files. Example: `python3 build.py --sformat <wazuh_modules/syscollector
`--scanbuild`	Run scan-build on the code. Example: `python3 build.py --scanbuild <agent\|server\|winagent>`
`--target`	This is an add-on to the ready-to-review test because this could add specific tests, by default the target is the agent. Example: `python3 build.py -r --target <agent\|server\|winagent>`
`--deleteLogs`	Clean log results to some module. Example: `python3 build.py --deleteLogs <wazuh_modules/syscollector

Ready to review checks:

Runs cppcheck on <wazuh_modules/syscollector|shared_modules/dbsync|shared_modules/rsync|shared_modules/utils|data_provider|syscheckd> folder.
Compiles <wazuh_modules/syscollector|shared_modules/dbsync|shared_modules/rsync|shared_modules/utils|data_provider|syscheckd>.
Runs <wazuh_modules/syscollector|shared_modules/dbsync|shared_modules/rsync|shared_modules/utils|data_provider|syscheckd> UTs.
Runs valgrind on <wazuh_modules/syscollector|shared_modules/dbsync|shared_modules/rsync|shared_modules/utils|data_provider|syscheckd> UTs.
Runs code coverage on <wazuh_modules/syscollector|shared_modules/dbsync|shared_modules/rsync|shared_modules/utils|data_provider|syscheckd> tests and generates coverage reports.
Runs AStyle on <wazuh_modules/syscollector|shared_modules/dbsync|shared_modules/rsync|shared_modules/utils|data_provider|syscheckd>.
Runs ASAN on <wazuh_modules/syscollector|shared_modules/dbsync|shared_modules/rsync|shared_modules/utils|data_provider|syscheckd>.
Runs Test tool for Windows on .
Runs check output from test tool on .

If all the checks passed it returns 0 and prints a "[RTR: PASSED]", otherwise it stops the execution of the checking on the first failure, prints the info related to the failure and returns and error code. We can use a flag like target to run tests on Windows this runs unit tests so this works only for syscheckd at the moment.

Output Example executing the RTR tool with dbsync module:

#> python3 build.py -r shared_modules/dbsync
<shared_modules/dbsync>=================== Running RTR checks  ===================<shared_modules/dbsync>
<shared_modules/dbsync>=================== Running cppcheck    ===================<shared_modules/dbsync>
[Cppcheck: PASSED]
[Cleanfolder : PASSED]
<shared_modules/dbsync>=================== Running CMake Conf  ===================<shared_modules/dbsync>
[ConfigureCMake: PASSED]
<shared_modules/dbsync>=================== Compiling library   ===================<shared_modules/dbsync>
shared_modules/dbsync > [make: PASSED]
<shared_modules/dbsync>=================== Running Tests       ===================<shared_modules/dbsync>
[dbsyncPipelineFactory_unit_test: PASSED]
[dbengine_unit_test: PASSED]
[dbsync_unit_test: PASSED]
[fim_integration_test: PASSED]
[sqlite_unit_test: PASSED]
<shared_modules/dbsync>=================== Running Valgrind    ===================<shared_modules/dbsync>
[dbsyncPipelineFactory_unit_test : PASSED]
[dbengine_unit_test : PASSED]
[dbsync_unit_test : PASSED]
[fim_integration_test : PASSED]
[sqlite_unit_test : PASSED]
<shared_modules/dbsync>=================== Running Coverage    ===================<shared_modules/dbsync>
[lcov info: GENERATED]
[genhtml info: GENERATED]
Report: /root/repos/wazuh/src/shared_modules/dbsync/coverage_report/index.html
[Lines Coverage 95.1%: PASSED]
[Functions Coverage 98.0%: PASSED]
<shared_modules/dbsync>=================== Running AStyle      ===================<shared_modules/dbsync>
[Cleanfolder : PASSED]
<shared_modules/dbsync>[AStyle Check: PASSED]<shared_modules/dbsync>
<shared_modules/dbsync>=================== Running ASAN        ===================<shared_modules/dbsync>
[CleanInternals: PASSED]
[MakeTarget: PASSED]
[Cleanfolder : PASSED]
<shared_modules/dbsync>=================== Running CMake Conf  ===================<shared_modules/dbsync>
[ConfigureCMake: PASSED]
<shared_modules/dbsync>=================== Compiling library   ===================<shared_modules/dbsync>
shared_modules/dbsync > [make: PASSED]
[Cleanfolder : PASSED]
<TESTTOOL>=================== Running TEST TOOL   ===================<TESTTOOL>
/root/repos/wazuh/src/shared_modules/dbsync/build/bin/dbsync_test_tool -c config.json -a snapshotsUpdate/insertData.json,snapshotsUpdate/updateWithSnapshot.json -o ./output
[TestTool: PASSED]
<TESTTOOL>=================== Running TEST TOOL   ===================<TESTTOOL>
/root/repos/wazuh/src/shared_modules/dbsync/build/bin/dbsync_test_tool -c config.json -a InsertionUpdateDeleteSelect/inputSyncRowInsert.json,InsertionUpdateDeleteSelect/inputSyncRowModified.json,InsertionUpdateDeleteSelect/deleteRows.json,InsertionUpdateDeleteSelect/inputSelectRows.json -o ./output
[TestTool: PASSED]
<TESTTOOL>=================== Running TEST TOOL   ===================<TESTTOOL>
/root/repos/wazuh/src/shared_modules/dbsync/build/bin/dbsync_test_tool -c config.json -a txnOperation/createTxn.json,txnOperation/inputSyncRowInsertTxn.json,txnOperation/pksGetDeletedRows.json,txnOperation/inputSyncRowModifiedTxn.json,txnOperation/closeTxn.json -o ./output
[TestTool: PASSED]
<TESTTOOL>=================== Running TEST TOOL   ===================<TESTTOOL>
/root/repos/wazuh/src/shared_modules/dbsync/build/bin/dbsync_test_tool -c config.json -a txnOperation/createTxn.json,txnOperation/inputSyncRowInsertTxn.json,txnOperation/fullyGetDeletedRows.json,txnOperation/inputSyncRowModifiedTxn.json,txnOperation/closeTxn.json -o ./output
[TestTool: PASSED]
<TESTTOOL>=================== Running TEST TOOL   ===================<TESTTOOL>
/root/repos/wazuh/src/shared_modules/dbsync/build/bin/dbsync_test_tool -c config.json -a triggerActions/insertDataProcesses.json,triggerActions/insertDataSocket.json,triggerActions/addTableRelationship.json,triggerActions/deleteRows.json -o ./output
[TestTool: PASSED]
<TESTTOOL>=================== Running TEST TOOL   ===================<TESTTOOL>
/root/repos/wazuh/src/shared_modules/dbsync/build/bin/dbsync_test_tool -c config.json -a triggerActions/insertDataProcesses.json,triggerActions/insertDataSocket.json,triggerActions/addTableRelationship.json,triggerActions/deleteRows.json -o ./output
[TestTool: PASSED]
<shared_modules/dbsync>[ASAN: PASSED]<shared_modules/dbsync>
<shared_modules/dbsync>[RTR: PASSED]<shared_modules/dbsync>

Output Example executing the RTR tool with syscheck module and target winagent:

#> python3 build.py -r syscheckd --target winagent
 <syscheckd>=============== Running RTR checks  ===============<syscheckd>
 <syscheckd>=============== Running cppcheck    ===============<syscheckd>
 [Cppcheck: PASSED]
 <winagent>=============== Running Make Deps   ===============<winagent>
 [MakeDeps: PASSED]
 <winagent>=============== Running Make project ==============<winagent>
 [MakeTarget: PASSED]
 <syscheckd>=============== Running Tests       ===============<syscheckd>
 [fim_db_interface_test.exe: PASSED]
 [fileitem_unit_test.exe: PASSED]
 [registryvalue_unit_test.exe: PASSED]
 [fimdb_unit_test.exe: PASSED]
 [fim_registry_interface_test.exe: PASSED]
 [fim_file_interface_test.exe: PASSED]
 [registrykey_unit_test.exe: PASSED]
 <syscheckd>[All tests: PASSED]<syscheckd>
 [Cleanfolder : PASSED]
 <syscheckd>=============== Running CMake Conf  ===============<syscheckd>
 [ConfigureCMake: PASSED]
 <syscheckd>=============== Running AStyle      ===============<syscheckd>
 [Cleanfolder : PASSED]
 [AStyle Check: PASSED]
 [CleanAll: PASSED]
 [CleanExternals: PASSED]
 <agent>=============== Running Make Deps   ===============<agent>
 [MakeDeps: PASSED]
 <agent>=============== Running Make project ==============<agent>
 [MakeTarget: PASSED]
 [Cleanfolder : PASSED]
 <syscheckd>=============== Running ASAN        ===============<syscheckd>
 [CleanInternals: PASSED]
 <agent>=============== Running Make project ==============<agent>
 [MakeTarget: PASSED]
 [Cleanfolder : PASSED]
 <syscheckd>=============== Running CMake Conf  ===============<syscheckd>
 [ConfigureCMake: PASSED]
 <syscheckd>=============== Compiling library   ===============<syscheckd>
 [make: PASSED]
 [Cleanfolder : PASSED]
 <TESTTOOL>=============== Running TEST TOOL   ===============<TESTTOOL>
 /home/francorivero/Desktop/Wazuh_repositories/vagrant/wazuh/src/syscheckd/build/bin/fimdb_test_tool -c config.json -a FimDBTransaction/StartTransaction.json,FimDBTransaction/SyncTxnRows_1.json,FimDBTransaction/GetDeletedRows.json,FimDBTransaction/CountFiles.json,FimDBTransaction/StartTransaction.json,FimDBTransaction/SyncTxnRows_2.json,FimDBTransaction/GetDeletedRows.json,FimDBTransaction/CountFiles.json -o ./output/fileTransaction
 <TESTTOOL>=============== Running TEST TOOL   ===============<TESTTOOL>
 /home/francorivero/Desktop/Wazuh_repositories/vagrant/wazuh/src/syscheckd/build/bin/fimdb_test_tool -c config.json -a atomicFileOperations/SyncRow_1.json,atomicFileOperations/SyncRow_2.json,atomicFileOperations/CountFiles.json,atomicFileOperations/SyncRow_3.json,atomicFileOperations/DeleteFile.json,atomicFileOperations/CountFiles.json,atomicFileOperations/GetFile.json -o ./output/AtomicOperations
 [ASAN: PASSED]
 <syscheckd>=============== Running TEST TOOL for Windows =====<syscheckd>
 [CleanAll: PASSED]
 [CleanExternals: PASSED]
 <winagent>=============== Running Make Deps   ===============<winagent>
 [MakeDeps: PASSED]
 <winagent>=============== Running Make project ==============<winagent>
 [MakeTarget: PASSED]
 <TESTTOOL>=============== Running TEST TOOL   ===============<TESTTOOL>
 WINEPATH="/usr/i686-w64-mingw32/lib;/home/francorivero/Desktop/Wazuh_repositories/vagrant/wazuh/src"                            WINEARCH=win64 /usr/bin/wine /home/francorivero/Desktop/Wazuh_repositories/vagrant/wazuh/src/syscheckd/build/bin/fimdb_test_tool.exe -c configWindows.json -a FimDBTransaction/StartTransactionRegistryKey.json,FimDBTransaction/SyncTxnRowsRegistryKey_1.json,FimDBTransaction/GetDeletedRows.json,FimDBTransaction/StartTransactionRegistryKey.json,FimDBTransaction/SyncTxnRowsRegistryKey_2.json,FimDBTransaction/GetDeletedRows.json -o ./output/registryKeyTransaction
 <TESTTOOL>=============== Running TEST TOOL   ===============<TESTTOOL>
 WINEPATH="/usr/i686-w64-mingw32/lib;/home/francorivero/Desktop/Wazuh_repositories/vagrant/wazuh/src"                            WINEARCH=win64 /usr/bin/wine /home/francorivero/Desktop/Wazuh_repositories/vagrant/wazuh/src/syscheckd/build/bin/fimdb_test_tool.exe -c configWindows.json -a FimDBTransaction/StartTransactionRegistryData.json,FimDBTransaction/SyncTxnRowsRegistryData_1.json,FimDBTransaction/GetDeletedRows.json,FimDBTransaction/StartTransactionRegistryData.json,FimDBTransaction/SyncTxnRowsRegistryData_2.json,FimDBTransaction/GetDeletedRows.json -o ./output/registryDataTransaction
 [TEST TOOL for Windows: PASSED]
 [TestTool check: PASSED]
 <syscheckd>[RTR: PASSED]<syscheckd>

Address sanitizer checks:

Clean previous builds <wazuh_modules/syscollector|shared_modules/dbsync|shared_modules/rsync|shared_modules/utils|data_provider|syscheckd> folder.
Compiles with address sanitizers flags<wazuh_modules/syscollector|shared_modules/dbsync|shared_modules/rsync|shared_modules/utils|data_provider|syscheckd>.
Runs smoke tests <wazuh_modules/syscollector|shared_modules/dbsync|shared_modules/rsync|shared_modules/utils|data_provider|syscheckd>
Runs valgrind on <wazuh_modules/syscollector|shared_modules/dbsync|shared_modules/rsync|shared_modules/utils|data_provider|syscheckd> UTs. If all the checks passed it returns 0 and prints a "[ASAN: PASSED]", otherwise it stops the execution of the checking on the first failure, prints the info related to the failure and returns and error code.

Output Example executing the ASAN tests with dbsync module:

#> python3 build.py --asan shared_modules/dbsync
<shared_modules/dbsync>=================== Running ASAN        ===================<shared_modules/dbsync>
[CleanInternals: PASSED]
[MakeTarget: PASSED]
[Cleanfolder : PASSED]
<shared_modules/dbsync>=================== Running CMake Conf  ===================<shared_modules/dbsync>
[ConfigureCMake: PASSED]
<shared_modules/dbsync>=================== Compiling library   ===================<shared_modules/dbsync>
shared_modules/dbsync > [make: PASSED]
[Cleanfolder : PASSED]
<TESTTOOL>=================== Running TEST TOOL   ===================<TESTTOOL>
/root/repos/wazuh/src/shared_modules/dbsync/build/bin/dbsync_test_tool -c config.json -a snapshotsUpdate/insertData.json,snapshotsUpdate/updateWithSnapshot.json -o ./output
[TestTool: PASSED]
<TESTTOOL>=================== Running TEST TOOL   ===================<TESTTOOL>
/root/repos/wazuh/src/shared_modules/dbsync/build/bin/dbsync_test_tool -c config.json -a InsertionUpdateDeleteSelect/inputSyncRowInsert.json,InsertionUpdateDeleteSelect/inputSyncRowModified.json,InsertionUpdateDeleteSelect/deleteRows.json,InsertionUpdateDeleteSelect/inputSelectRows.json -o ./output
[TestTool: PASSED]
<TESTTOOL>=================== Running TEST TOOL   ===================<TESTTOOL>
/root/repos/wazuh/src/shared_modules/dbsync/build/bin/dbsync_test_tool -c config.json -a txnOperation/createTxn.json,txnOperation/inputSyncRowInsertTxn.json,txnOperation/pksGetDeletedRows.json,txnOperation/inputSyncRowModifiedTxn.json,txnOperation/closeTxn.json -o ./output
[TestTool: PASSED]
<TESTTOOL>=================== Running TEST TOOL   ===================<TESTTOOL>
/root/repos/wazuh/src/shared_modules/dbsync/build/bin/dbsync_test_tool -c config.json -a txnOperation/createTxn.json,txnOperation/inputSyncRowInsertTxn.json,txnOperation/fullyGetDeletedRows.json,txnOperation/inputSyncRowModifiedTxn.json,txnOperation/closeTxn.json -o ./output
[TestTool: PASSED]
<TESTTOOL>=================== Running TEST TOOL   ===================<TESTTOOL>
/root/repos/wazuh/src/shared_modules/dbsync/build/bin/dbsync_test_tool -c config.json -a triggerActions/insertDataProcesses.json,triggerActions/insertDataSocket.json,triggerActions/addTableRelationship.json,triggerActions/deleteRows.json -o ./output
[TestTool: PASSED]
<TESTTOOL>=================== Running TEST TOOL   ===================<TESTTOOL>
/root/repos/wazuh/src/shared_modules/dbsync/build/bin/dbsync_test_tool -c config.json -a triggerActions/insertDataProcesses.json,triggerActions/insertDataSocket.json,triggerActions/addTableRelationship.json,triggerActions/deleteRows.json -o ./output
[TestTool: PASSED]
<shared_modules/dbsync>[ASAN: PASSED]<shared_modules/dbsync>

Scan-build analysis agent/server:

Clean previous compiled binaries.
Remove externals libraries.
Download externals libraries based on the specified target.
Run scan-build on the specified target. If all the checks passed it returns 0 and prints a "[SCANBUILD: PASSED]", otherwise it stops the execution checking on the first failure, prints the info related to the failure and returns an error code.

Scan-build analysis winagent:

Clean previous compiled binaries.
Remove externals libraries.
Download externals libraries for windows.
Compile winagent.
Clean internals.
Run scan-build for winagent specifying the cc and cxx compiler and the target. Steps 4-5 were added to fix an issue found running scan-build on winagent using pre-compiled externals libraries. If all the checks passed it returns 0 and prints a "[SCANBUILD: PASSED]", otherwise it stops the execution of the checking on the first failure, prints the info related to the failure and returns an error code. Output Example executing the SCANBUILD analysis with agent target:

#> python3 build.py --scanbuild agent
<agent>=================== Running Scanbuild   ===================<agent>
[CleanAll: PASSED]
[CleanExternals: PASSED]
[MakeDeps: PASSED]
[ScanBuild: PASSED]
<agent>[SCANBUILD: PASSED]<agent>

Output Example executing the SCANBUILD analysis with winagent target:

#> python3 build.py --scanbuild winagent
<winagent>=================== Running Scanbuild   ===================<winagent>
[CleanAll: PASSED]
[CleanExternals: PASSED]
[MakeDeps: PASSED]
[MakeTarget: PASSED]
[CleanInternals: PASSED]
[ScanBuild: PASSED]
<winagent>[SCANBUILD: PASSED]<winagent>

Name		Name	Last commit message	Last commit date
parent directory ..
active-response		active-response
addagent		addagent
agentlessd		agentlessd
analysisd		analysisd
ci		ci
client-agent		client-agent
config		config
data_provider		data_provider
error_messages		error_messages
external		external
generated_headers		generated_headers
headers		headers
init		init
logcollector		logcollector
monitord		monitord
os_auth		os_auth
os_crypto		os_crypto
os_csyslogd		os_csyslogd
os_dbd		os_dbd
os_execd		os_execd
os_integrator		os_integrator
os_maild		os_maild
os_net		os_net
os_regex		os_regex
os_xml		os_xml
os_zlib		os_zlib
remoted		remoted
reportd		reportd
rootcheck		rootcheck
selinux		selinux
shared		shared
shared_modules		shared_modules
syscheckd		syscheckd
unit_tests		unit_tests
util		util
wazuh_db		wazuh_db
wazuh_modules		wazuh_modules
win32		win32
CMakeLists.txt		CMakeLists.txt
Doxyfile		Doxyfile
Makefile		Makefile
REVISION		REVISION
Readme.md		Readme.md
VERSION		VERSION
build.py		build.py

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

src

src

Readme.md

General Build Script

Index

Purpose

Dependencies

Compile Wazuh

How to use the tool

Optional arguments:

Files

src

Directory actions

More options

Directory actions

More options

Latest commit

History

src

Folders and files

parent directory

Readme.md

General Build Script

Index

Purpose

Dependencies

Compile Wazuh

How to use the tool

Optional arguments: