This is a fork of the original wrench repository created to present my personal fix for the installation of the wrench benchmark. The installation using pip install always resulted in missing packages, and the environment.yml file (that create a conda environment) had many problems with version conflicts among the packages. To fix that, a new environment.yml file was created with the correct versions of the required packages (dependencies) to create the conda wrench environment.
This repository is entirely based on the original wrench repository; the only modifications are this README.md file and the new environment.yml file. For information about how to use this benchmark, visit the original repository; there are many useful examples.
-
Make sure to have conda installed.
-
Clone this repository.
-
Navigate to the folder of the cloned repository on your computer.
-
Run the following command to create the conda environment:
CPU enviroment:
$ conda env create -f environment.yml
GPU enviroment:
$ conda env create -f environment_gpu.yml
-
Start the environment:
$ conda activate wrench
-
Now, inside the repository folder, you can run your Python scripts using the wrench benchmark.
P.S.: There was no problem with the conda environment creation process using windows WSL ubuntu. But, ubuntu OS and pop OS may require the installation of some libraries used by certain dependencies. That was my experience with the environment creation process.
PS. 2: Windows is NOT an option for using wrench. According to reddit users, the package faiss-gpu (a dependency of wrench) requires Linux. To use it on Windows, you must use windows WSL.
PS. 3: If on ubuntu you have some problem related to libssl.so.10 or libcrypto.so.10 follow the steps described in the section Installing ubuntu libs
PS. 4: If you have any problem with lib transformers uninstall with the command
$ pip uninstall transformersand reinstall directly from the official git repository with the command bellow
$ pip install git+https://github.com/huggingface/transformersSome libs may be required to run wrench using ubuntu, if you have problems with libssl.so.10 or libcrypto.so.10 try the steps below to install the librarys:
libssl.so.10:
- Install the libssl:
$ sudo apt-get update
$ sudo apt install libssl-dev- Now rename the libary
$ cd /lib/x86_64-linux-gnu
$ sudo ln -s libssl.so libssl.so.10libcrypto.so.10:
- Install the libcrypto:
$ sudo apt-get update
$ sudo apt install libcrypto-dev- Now rename the libary
$ cd /lib/x86_64-linux-gnu
$ sudo ln -s libcrypto.so libcrypto.so.10from huggingface_hub import snapshot_download
path = "path to local dir"
snapshot_download(repo_id="jieyuz2/WRENCH", repo_type="dataset", local_dir=path)Remember to cite the original benchmark work when you use this repository.
@inproceedings{
zhang2021wrench,
title={{WRENCH}: A Comprehensive Benchmark for Weak Supervision},
author={Jieyu Zhang and Yue Yu and Yinghao Li and Yujing Wang and Yaming Yang and Mao Yang and Alexander Ratner},
booktitle={Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track},
year={2021},
url={https://openreview.net/forum?id=Q9SKS5k8io}
}