Highlights
- Pro
Lists (1)
Stars
SWIM-IR is a Synthetic Wikipedia-based Multilingual Information Retrieval training set with 28 million query-passage pairs spanning 33 languages, generated using PaLM 2 and summarize-then-ask promp…
TyDi QA contains 200k human-annotated question-answer pairs in 11 Typologically Diverse languages, written without seeing the answer and without the use of translation, and is designed for the trai…
M2D2: A Massively Multi-domain Language Modeling Dataset (EMNLP 2022) by Machel Reid, Victor Zhong, Suchin Gururangan, Luke Zettlemoyer
PyTorch repository for ICLR 2022 paper (GSAM) which improves generalization (e.g. +3.8% top-1 accuracy on ImageNet with ViT-B/32)
Free Lunch: Robust Cross-Lingual Transfer via Model Checkpoint Averaging
PyTorch implementation of FIM and empirical FIM
Collection of algorithms for approximating Fisher Information Matrix for Natural Gradient (and second order method in general)
Data and code for ACL 2023 paper XSemPLR: Cross-Lingual Semantic Parsing in Multiple Natural Languages and Meaning Representations
The official implementation of “Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training”
Gantry streamlines running Python experiments in Beaker by managing containers and boilerplate for you
[EMNLP22] Improving Sharpness-Aware Minimization with Fisher Mask for Better Generalization on Language Models
Code for "SAM as an Optimal Relaxation of Bayes", ICLR 2023.
Code for "Train Flat, Then Compress" paper @ EMNLP 2022
This repository includes code to reproduce the tables in "Loss Landscapes are All You Need: Neural Network Generalization Can Be Explained Without the Implicit Bias of Gradient Descent"
A simple library for querying the URIEL typological database.
Pytorch implementation of JointBERT: "BERT for Joint Intent Classification and Slot Filling"
allennlp-light is a port of AllenNLP's core modules and nn portions into a standalone package with minimum dependencies
A modern look at the relationship between sharpness and generalization [ICML 2023]
Simple implementation of flat minima methods (SAM, fisher penalty) for Huggingface trainer.
Matplotlib styles for scientific plotting
The geometry of multilingual language model representations (EMNLP 2022).
Obtain Word Alignments using Pretrained Lan 31CE guage Models (e.g., mBERT)