Code for 2023 ACL Findings Paper Sentence Ordering with a Coherence Verifier
- python==3.8.10
- torch==1.13.0
- transformers==4.24.0
- dgl==0.9.1
- BERSON: https://github.com/hwxcby/BERSON
- B-TSort: https://github.com/shrimai/Topological-Sort-for-Sentence-Ordering/
For the AAN and NIPS data, please contact the authors of Sentence Ordering and Coherence Modeling using Recurrent Neural Networks.
The SIND dataset can be downloaded from the Visual Storytelling website.
And the ROCStory dataset can be download from here.
- Construct the instance with gradual permutation. Replace the specific name such as 'roc' with 'dataset'
python get_pairwise_hier_dataset.py dataset
- Train the model.
sh run_score.sh
If you already have the baseline B-TSort and BERSON model and a CoVer model, you can run run_rerank.sh in Topo_CoVer or run_{dataset}_coherence.sh in BERSON_CoVer.
Besides, we saved the pairwise scores generated by B-TSort in Topo_CoVer/pairwise_score, so that you can directly do the reranking process without having a B-TSort model.
We provide the pretrained coherence model
You can download from here: https://drive.google.com/drive/folders/1gHqH3inelArIDPhUIu8XjOAbXcaZSKok?usp=sharing
If you are intereseted in this work, please cite the following paper.
@inproceedings{jia-etal-2023-sentence,
title = "Sentence Ordering with a Coherence Verifier",
author = "Jia, Sainan and
Song, Wei and
Gong, Jiefu and
Wang, Shijin and
Liu, Ting",
booktitle = "Findings of the Association for Computational Linguistics: ACL 2023",
month = jul,
year = "2023",
address = "Toronto, Canada",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2023.findings-acl.592",
doi = "10.18653/v1/2023.findings-acl.592",
pages = "9301--9314",
abstract = "This paper presents a novel sentence ordering method by plugging a coherence verifier (CoVer) into pair-wise ranking-based and sequence generation-based methods. It does not change the model parameters of the baseline, and only verifies the coherence of candidate (partial) orders produced by the baseline and reranks them in beam search. We also propose a coherence model as CoVer with a novel graph formulation and a novel data construction strategy for contrastive pre-training independently of the sentence ordering task. Experimental results on four benchmarks demonstrate the effectiveness of our method with topological sorting-based and pointer network-based methods as the baselines. Detailed analyses illustrate how CoVer improves the baselines and confirm the importance of its graph formulation and training strategy. Our code is available at https://github.com/SN-Jia/SO{\_}with{\_}CoVer.",
}