This Python script queries an Active Directory (AD) server for user information, exports the results to a CSV file, and uploads the file to Cribl Cloud as a lookup table. It supports both creating new lookup objects and updating existing ones, followed by committing and deploying changes to a specified Cribl worker group.
- AD Query: Retrieves user attributes (
sAMAccountName,displayName,mail,department,title) from an AD server via LDAP. - CSV Export: Saves query results to a CSV file.
- Cribl Cloud Integration: Authenticates with Cribl Cloud, uploads the CSV, manages lookup objects, and deploys changes.
- Flexible Configuration: Supports configuration via a
config.inifile or command-line arguments. - User-Friendly AD Credentials: Accepts AD user input in UPN (
joe@mycompany.com), NetBIOS (MYDOMAIN\joe), or plain username (joe) formats. - Robust Error Handling: Provides clear error messages for configuration issues and API failures.
- Python: Version 3.7 or higher.
- Dependencies: Install required libraries listed in
requirements.txt. - Active Directory Access: Access to an AD server with LDAP enabled (port 389 or 636 for LDAPS).
- Cribl Cloud Account: Credentials (client ID and secret) for Cribl Cloud API access.
- Network Access: Ability to connect to the AD server and Cribl Cloud APIs (
https://login.cribl.cloud,https://app.cribl.cloud).
-
Clone or Download the Script:
- Save
main.py,requirements.txt, and an optionalconfig.inito a directory.
- Save
-
Set Up a Virtual Environment (recommended):
python -m venv venv source venv/bin/activate # On Windows: venv\Scripts\activate
-
Install Dependencies:
pip install -r requirements.txt
-
Create a Configuration File (optional):
- Copy the example
config.inibelow to a file namedconfig.iniin the same directory asmain.py.
- Copy the example
The script supports configuration via a config.ini file or command-line arguments. Command-line arguments override config.ini values.
[cribl]
client_id = your_client_id
client_secret = your_client_secret
organization_id = your_org_id
lookup_filename = users.csv
target_worker_group = default
ad_server = ldap://ad.mycompany.com
ad_user = joe@mycompany.com
ad_password = your_password
ad_domain = mycompany.com
# ad_search_domain = child.mycompany.com # Optional, only if search base differsclient_id,client_secret: Cribl Cloud API credentials (obtain from Cribl Cloud admin).organization_id: Your Cribl Cloud organization ID.lookup_filename: Name of the CSV file to generate and upload (e.g.,users.csv).target_worker_group: Cribl worker group to upload the lookup to (e.g.,default).ad_server: AD server address (e.g.,ldap://ad.mycompany.comorldaps://ad.mycompany.com:636).ad_user: AD user for authentication, in one of:- UPN format:
joe@mycompany.com - NetBIOS format:
MYDOMAIN\joeorMYDOMAIN/joe - Plain username:
joe(requiresad_domain)
- UPN format:
ad_password: Password for the AD user.ad_domain: AD domain for authentication and search base (e.g.,mycompany.comorMYDOMAIN).ad_search_domain: Optional, use if the search base domain differs fromad_domain(e.g.,child.mycompany.com).
-
Avoid storing
ad_passwordor Cribl credentials in plain text. Consider using environment variables or a secrets manager. -
Example using environment variables:
import os ad_password = os.getenv("AD_PASSWORD", config["ad_password"])
Run the script via the command line, specifying configuration via config.ini or arguments.
python main.pypython main.py \
--ad-server ldap://ad.mycompany.com \
--ad-user joe@mycompany.com \
--ad-password secret \
--ad-domain mycompany.com \
--client-id your_client_id \
--client-secret your_client_secret \
--organization-id your_org_id \
--lookup-filename users.csv \
--target-group defaultpython main.py \
--ad-server ldap://ad.mycompany.com \
--ad-user MYDOMAIN\joe \
--ad-password secret \
--ad-domain mycompany.com \
--ad-search-domain child.mycompany.com \
--lookup-filename users.csv- Parse Arguments and Config: Loads settings from
config.iniand overrides with command-line arguments. - Query AD: Connects to the AD server, queries user attributes, and exports to a CSV file.
- Cribl Cloud Authentication: Obtains a bearer token using Cribl client credentials.
- Upload CSV: Uploads the CSV to the specified Cribl worker group.
- Manage Lookup: Creates a new lookup object or updates an existing one.
- Commit and Deploy: Commits changes and deploys them to Cribl Cloud.
To test the script:
-
Set Up a Test AD Environment:
- Use a Windows Server trial with Active Directory Domain Services (AD DS) in a VM (e.g., VirtualBox).
- Populate with test users using PowerShell or tools like Albus Bit’s AD Test Data Generator.
-
Create a Test
config.ini:[cribl] ad_server = ldap://192.168.56.10 ad_user = Administrator@testlab.local ad_password = Password123! ad_domain = testlab.local lookup_filename = test_users.csv
-
Run with Dummy Cribl Credentials:
- For AD-only testing, comment out Cribl-related code in
main()(lines afterquery_ad_users). - Run:
python main.py - Verify that
test_users.csvcontains the expected user data.
- For AD-only testing, comment out Cribl-related code in
-
Test Cribl Integration:
- Use valid Cribl Cloud credentials and ensure network access to
https://app.cribl.cloud. - Check Cribl Cloud UI to confirm the lookup table is created/updated.
- Use valid Cribl Cloud credentials and ensure network access to
-
AD Connection Errors:
- Verify
ad_serveris reachable (pingortelnet ad.mycompany.com 389). - Ensure
ad_userhas permission to query AD. - Check
ad_domainmatches the AD domain (e.g.,mycompany.com). - For LDAPS, use
ldaps://and port 636, and ensure the server certificate is trusted.
- Verify
-
Invalid User Format:
- Ensure
ad_useris valid (e.g.,joe@mycompany.com,MYDOMAIN\joe, orjoewithad_domain). - Check error messages for guidance (e.g.,
Configuration error: AD domain must be specified when using plain username: joe).
- Ensure
-
Cribl API Errors:
- Verify
client_id,client_secret, andorganization_id. - Ensure network access to Cribl Cloud APIs.
- Check response messages for HTTP status codes or API errors.
- Verify
-
CSV Issues:
- Ensure
lookup_filenameends with.csv. - Verify write permissions in the script’s directory.
- Ensure
- Credentials: Store
ad_password,client_id, andclient_secretsecurely (e.g., environment variables, AWS Secrets Manager). - LDAPS: Prefer
ldaps://overldap://to encrypt AD communication. - File Permissions: Restrict access to
config.iniand the generated CSV file. - API Tokens: Handle Cribl bearer tokens securely and avoid logging them.
- Additional AD Attributes: Modify the
attributeslist inquery_ad_usersto include other AD fields (e.g.,telephoneNumber,lastLogon). - Logging: Add Python’s
loggingmodule for detailed debug output. - Scheduling: Use a scheduler like
cronorWindows Task Schedulerto run the script periodically. - Multi-Domain Support: Extend
ad_search_domainto support querying multiple domains or organizational units.
This script is provided as-is for educational and testing purposes. Ensure compliance with your organization’s policies and Cribl’s terms of service.
For issues or feature requests, please contact ssimmons@cribl.io.