This is the implementation for AutoProfiler on the synthetic dataset (SynthPAI).
/agentcontains the implementation of LLM agent for the attack./configcontains the API keys for various LLM providers./datasetcontains the orginal user history (/synthpai) as well as the inference results of each LLM on each user./functionscontains the function callings for retriever./promptscontains all prompts for each agent./utilcontains scripts about data loader and useful functions.
We use the dataset and ground truth from the paper: A Synthetic Dataset for Personal Attribute Inference.
Simply run the script for inference attack:
python main.py -m [llm_model] -u [user]This will load three agents (i.e., Retriever, Profiler, and Summarizer) to complete the task.
The results are stored at /dataset/{llm_model}/.
Note that in implementation we combine Strategist and Extractor as Profiler.