Publications
Peer-reviewed papers, preprints, and theses from the group — download all as a BibTeX file.
-
Lost in Simulation: LLM-Simulated Users are Unreliable Proxies for Human Users in Agentic Evaluations
Agentic benchmarks increasingly rely on LLM-simulated users to scalably evaluate agent performance, yet the robustness, validity, and fairness of this approach remain unexamined. Through a user study with participants across the United States, India, Kenya, and Nigeria, we investigate whether LLM-simulated users serve as reliable proxies for real human users in evaluating agents on τ-Bench retail tasks. We find that user simulation lacks robustness, with agent success rates varying up to 9 percentage points across different user LLMs. Furthermore, evaluations using simulated users exhibit systematic miscalibration, underestimating agent performance on challenging tasks and overestimating it on moderately difficult ones. African American Vernacular English (AAVE) speakers experience consistently worse success rates and calibration errors than Standard American English (SAE) speakers, with disparities compounding significantly with age. We also find simulated users to be a differentially effective proxy for different populations, performing worst for AAVE and Indian English speakers. Additionally, simulated users introduce conversational artifacts and surface different failure patterns than human users. These findings demonstrate that current evaluation practices risk misrepresenting agent capabilities across diverse user populations and may obscure real-world deployment challenges.
@inproceedings{lostinsim:acl26, author = {Preethi Seshadri and Samuel Cahyawijaya and Ayomide Odumakinde and Sameer Singh and Seraphina Goldfarb-Tarrant}, title = { {Lost in Simulation: LLM-Simulated Users are Unreliable Proxies for Human Users in Agentic Evaluations} }, booktitle = {Association for Computational Linguistics (ACL)}, year = {2026} } -
Parallel Token Prediction for Language Models
Autoregressive decoding in language models is inherently slow, generating only one token per forward pass. We propose Parallel Token Prediction (PTP), a general-purpose framework for predicting multiple tokens in a single model call. PTP moves the source of randomness from post-hoc sampling to random input variables, making future tokens deterministic functions of those inputs and thus jointly predictable in a single forward pass. We prove that a single PTP call can represent arbitrary dependencies between tokens. PTP is trained by distilling an existing model or through inverse autoregressive training without a teacher. Experimentally, PTP achieves a 2.4x speedup on a diverse-task speculative decoding benchmark. We provide code and checkpoints at https://github.com/mandt-lab/ptp.
@inproceedings{ptp:iclr26, author = {Felix Draxler and Justus Will and Farrin Marouf Sofian and Theofanis Karaletsos and Sameer Singh and Stephan Mandt}, title = { {Parallel Token Prediction for Language Models} }, booktitle = {International Conference on Learning Representations (ICLR)}, year = {2026} } -
Entropy-Aligned Decoding of LMs for Better Writing and Reasoning
Language models (LMs) are trained on billions of tokens in an attempt to recover the true language distribution. Still, vanilla random sampling from LMs yields low quality generations. Decoding algorithms attempt to restrict the LM distribution to a set of high-probability continuations, but rely on greedy heuristics that introduce myopic distortions, yielding sentences that are homogeneous, repetitive and incoherent. In this paper, we introduce EPIC, a hyperparameter-free decoding approach that incorporates the entropy of future trajectories into LM decoding. EPIC explicitly regulates the amount of uncertainty expressed at every step of generation, aligning the sampling distribution's entropy to the aleatoric (data) uncertainty. Through Entropy-Aware Lazy Gumbel-Max sampling, EPIC manages to be exact, while also being efficient, requiring only a sublinear number of entropy evaluations per step. Unlike current baselines, EPIC yields sampling distributions that are empirically well-aligned with the entropy of the underlying data distribution. Across creative writing and summarization tasks, EPIC consistently improves LM-as-judge preference win-rates over widely used decoding strategies. These preference gains are complemented by automatic metrics, showing that EPIC produces more diverse generations and more faithful summaries. We also evaluate EPIC on mathematical reasoning, where it outperforms all baselines.
@techreport{epic:arxiv26, author = {Kareem Ahmed and Sameer Singh}, title = { {Entropy-Aligned Decoding of LMs for Better Writing and Reasoning} }, institution = {arXiv preprint}, year = {2026} }
-
Characterizing Mamba's Selective Memory using Auto-Encoders
State space models (SSMs) are a promising alternative to transformers for language modeling because they use fixed memory during inference. However, this fixed memory usage requires some information loss in the hidden state when processing long sequences. While prior work has studied the sequence length at which this information loss occurs, it does not characterize the types of information SSM language models (LMs) tend to forget. In this paper, we address this knowledge gap by identifying the types of tokens (e.g., parts of speech, named entities) and sequences (e.g., code, math problems) that are more frequently forgotten by SSM LMs. We achieve this by training an auto-encoder to reconstruct sequences from the SSM's hidden state, and measure information loss by comparing inputs with their reconstructions. We perform experiments using the Mamba family of SSM LMs (130M-1.4B) on sequences ranging from 4-256 tokens. Our results show significantly higher rates of information loss on math-related tokens (e.g., numbers, variables), mentions of organization entities, and alternative dialects to Standard American English. We then examine the frequency that these tokens appear in Mamba's pretraining data and find that less prevalent tokens tend to be the ones Mamba is most likely to forget. By identifying these patterns, our work provides clear direction for future research to develop methods that better control Mamba's ability to retain important information.
@inproceedings{mamba:aacl25, author = {Tamanna Hossain and Robert L. Logan IV and Chandrasekhara Ganesh Jagadeesan and Sameer Singh and Joel R. Tetreault and Alejandro Jaimes}, title = { {Characterizing Mamba's Selective Memory using Auto-Encoders} }, booktitle = {International Joint Conference on Natural Language Processing and the Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (IJCNLP-AACL)}, doi = {10.18653/v1/2025.ijcnlp-long.109}, pages = {2007-2022}, year = {2025} } -
Small Changes, Large Consequences: Analyzing the Allocational Fairness of LLMs in Hiring Contexts
Large language models (LLMs) are increasingly being deployed in high-stakes applications like hiring, yet their potential for unfair decision-making remains understudied in generative and retrieval settings. In this work, we examine the allocational fairness of LLM-based hiring systems through two tasks that reflect actual HR usage: resume summarization and applicant ranking. By constructing a synthetic resume dataset with controlled perturbations and curating job postings, we investigate whether model behavior differs across demographic groups. Our findings reveal that generated summaries exhibit meaningful differences more frequently for race than for gender perturbations. Models also display non-uniform retrieval selection patterns across demographic groups and exhibit high ranking sensitivity to both gender and race perturbations. Surprisingly, retrieval models can show comparable sensitivity to both demographic and non-demographic changes, suggesting that fairness issues may stem from broader model brittleness. Overall, our results indicate that LLM-based hiring systems, especially in the retrieval stage, can exhibit notable biases that lead to discriminatory outcomes in real-world contexts.
@inproceedings{hiringfairness:aacl25, author = {Preethi Seshadri and Hongyu Chen and Sameer Singh and Seraphina Goldfarb-Tarrant}, title = { {Small Changes, Large Consequences: Analyzing the Allocational Fairness of LLMs in Hiring Contexts} }, booktitle = {International Joint Conference on Natural Language Processing and the Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (IJCNLP-AACL)}, doi = {10.18653/v1/2025.ijcnlp-long.143}, pages = {2645-2665}, year = {2025} } -
Nudging: Inference-time Alignment of LLMs via Guided Decoding
In the Top 25 Papers at ACLLarge language models (LLMs) require alignment to effectively and safely follow user instructions. This process necessitates training an aligned version for every base model, resulting in significant computational overhead. In this work, we propose NUDGING, a simple, training-free algorithm that aligns any base model at inference time using a small aligned model. NUDGING is motivated by recent findings that alignment primarily alters the model's behavior on a small subset of stylistic tokens (e.g., discourse markers). We find that base models are significantly more uncertain when generating these tokens. Building on this insight, NUDGING employs a small aligned model to generate nudging tokens to guide the base model's output during decoding when the base model's uncertainty is high, with only a minor additional inference overhead. We evaluate NUDGING across 3 model families on a diverse range of open-instruction tasks. Without any training, nudging a large base model with a 7x-14x smaller aligned model achieves zero-shot performance comparable to, and sometimes surpassing, that of large aligned models. By operating at the token level, NUDGING enables off-the-shelf collaboration between model families. For instance, nudging Gemma-2-27b with Llama-27b-chat outperforms Llama-2-70b-chat on various tasks. Overall, our work offers a modular and cost-efficient solution to LLM alignment. Our code and demo are available at: https://fywalter.github.io/nudging/.
@inproceedings{nudging:acl25, author = {Yu Fei and Yasaman Razeghi and Sameer Singh}, title = { {Nudging: Inference-time Alignment of LLMs via Guided Decoding} }, booktitle = {Association for Computational Linguistics (ACL)}, doi = {10.18653/v1/2025.acl-long.623}, pages = {12702-12739}, year = {2025} } -
TurtleBench: A Visual Programming Benchmark in Turtle Geometry
Humans have the ability to reason about geometric patterns in images and scenes from a young age. However, developing large multimodal models (LMMs) capable of similar reasoning remains a challenge, highlighting the need for robust evaluation methods to assess these capabilities. We introduce TurtleBench, a benchmark designed to evaluate LMMs' capacity to interpret geometric patterns—given visual examples, textual instructions, or both—and generate precise code outputs. Inspired by turtle geometry, a notion used to teach children foundational coding and geometric concepts, TurtleBench features tasks with patterned shapes that have underlying algorithmic logic. Our evaluation reveals that leading LMMs struggle significantly with these tasks, with GPT-4V achieving only 19% accuracy on the simplest tasks and few-shot prompting only marginally improves their performance (<2%). TurtleBench highlights the gap between human and AI performance in intuitive and visual geometrical understanding, setting the stage for future research in this area and stands as one of the few benchmarks to evaluate the integration of visual understanding and code generation capabilities in LMMs, setting the stage for future research.
@inproceedings{turtlebench:naacl25, author = {Sina Rismanchian and Yasaman Razeghi and Sameer Singh and Shayan Doroudi}, title = { {TurtleBench: A Visual Programming Benchmark in Turtle Geometry} }, booktitle = {Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)}, doi = {10.18653/v1/2025.naacl-long.607}, pages = {12170-12188}, year = {2025} } -
Semantic Probabilistic Control of Language Models
Semantic control entails steering LM generations towards satisfying subtle non-lexical constraints, e.g., toxicity, sentiment, or politeness, attributes that can be captured by a sequence-level verifier. It can thus be viewed as sampling from the LM distribution conditioned on the target attribute, a computationally intractable problem due to the non-decomposable nature of the verifier. Existing approaches to LM control either only deal with syntactic constraints which cannot capture the aforementioned attributes, or rely on sampling to explore the conditional LM distribution, an ineffective estimator for low-probability events. In this work, we leverage a verifier's gradient information to efficiently reason over all generations that satisfy the target attribute, enabling precise steering of LM generations by reweighing the next-token distribution. Starting from an initial sample, we create a local LM distribution favoring semantically similar sentences. This approximation enables the tractable computation of an expected sentence embedding. We use this expected embedding, informed by the verifier's evaluation at the initial sample, to estimate the probability of satisfying the constraint, which directly informs the update to the next-token distribution. We evaluated the effectiveness of our approach in controlling the toxicity, sentiment, and topic-adherence of LMs yielding generations satisfying the constraint with high probability (>95%) without degrading their quality.
@techreport{scone:arxiv25, author = {Kareem Ahmed and Catarina Belem and Padhraic Smyth and Sameer Singh}, title = { {Semantic Probabilistic Control of Language Models} }, institution = {arXiv preprint}, year = {2025} } -
Leveraging In-Context Learning for Language Model Agents
In-context learning (ICL) with dynamically selected demonstrations combines the flexibility of prompting large language models (LLMs) with the ability to leverage training data to improve performance. While ICL has been highly successful for prediction and generation tasks, leveraging it for agentic tasks that require sequential decision making is challenging -- one must think not only about how to annotate long trajectories at scale and how to select demonstrations, but also what constitutes demonstrations, and when and where to show them. To address this, we first propose an algorithm that leverages an LLM with retries along with demonstrations to automatically and efficiently annotate agentic tasks with solution trajectories. We then show that set-selection of trajectories of similar tasks as demonstrations significantly improves performance, reliability, robustness, and efficiency of LLM agents. However, trajectory demonstrations have a large inference cost overhead. We show that this can be mitigated by using small trajectory snippets at every step instead of an additional trajectory. We find that demonstrations obtained from larger models (in the annotation phase) also improve smaller models, and that ICL agents can even rival costlier trained agents. Thus, our results reveal that ICL, with careful use, can be very powerful for agentic tasks as well.
@inproceedings{iclagents:mtillm25, author = {Shivanshu Gupta and Sameer Singh and Ashish Sabharwal and Tushar Khot and Ben Bogin}, title = { {Leveraging In-Context Learning for Language Model Agents} }, booktitle = {NeurIPS Workshop on Multi-Turn Interactions in Large Language Models (MTI-LLM)}, year = {2025} }
-
Are Models Biased on Text without Gender-related Language?
Gender bias research has been pivotal in revealing undesirable behaviors in large language models, exposing serious gender stereotypes associated with occupations, and emotions. A key observation in prior work is that models reinforce stereotypes as a consequence of the gendered correlations that are present in the training data. In this paper, we focus on bias where the effect from training data is unclear, and instead address the question: Do language models still exhibit gender bias in non-stereotypical settings? To do so, we introduce UnStereoEval (USE), a novel framework tailored for investigating gender bias in stereotype-free scenarios. USE defines a sentence-level score based on pretraining data statistics to determine if the sentence contain minimal word-gender associations. To systematically benchmark the fairness of popular language models in stereotype-free scenarios, we utilize USE to automatically generate benchmarks without any gender-related language. By leveraging USE's sentence-level score, we also repurpose prior gender bias benchmarks (Winobias and Winogender) for non-stereotypical evaluation. Surprisingly, we find low fairness across all 28 tested models. Concretely, models demonstrate fair behavior in only 9%-41% of stereotype-free sentences, suggesting that bias does not solely stem from the presence of gender-related words. These results raise important questions about where underlying model biases come from and highlight the need for more systematic and comprehensive bias evaluation. We release the full dataset and code at ucinlp.github.io/unstereo-eval.
@inproceedings{diagnosing:iclr24, author = {Catarina Belem and Preethi Seshadri and Yasaman Razeghi and Sameer Singh}, title = { {Are Models Biased on Text without Gender-related Language?} }, booktitle = {International Conference on Learning Representations}, doi = {https://openreview.net/forum?id=w1JanwReU6}, year = {2024} } -
Benchmark Data Repositories for Better Benchmarking
In machine learning research, it is common to evaluate algorithms via their performance on standard benchmark datasets. While a growing body of work establishes guidelines for—and levies criticisms at—data and benchmarking practices in machine learning, comparatively less attention has been paid to the data repositories where these datasets are stored, documented, and shared. In this paper, we analyze the landscape of these benchmark data repositories and the role they can play in improving benchmarking. This role includes addressing issues with both datasets themselves (e.g., representational harms, construct validity) and the manner in which evaluation is carried out using such datasets (e.g., overemphasis on a few datasets and metrics, lack of reproducibility). To this end, we identify and discuss a set of considerations surrounding the design and use of benchmark data repositories, with a focus on improving benchmarking practices in machine learning.
@inproceedings{benchmark:neurips24, author = {Rachel Longjohn and Markelle Kelly and Sameer Singh and Padhraic Smyth}, title = { {Benchmark Data Repositories for Better Benchmarking} }, booktitle = {Neural Information Processing Systems (NeurIPS)}, doi = {10.18653/v1/2024.nips-2744}, pages = {86435-86457}, year = {2024} } -
PlotTwist: Multimodal Models Don't Comprehend Simple Chart Details
Recent advances in multimodal models show remarkable performance in real-world benchmarks for chart and figure understanding like ChartQA that involve interpreting trends, comparing data points, and extracting insights from visuals.In this paper, we investigate the extent to which these models truly comprehend the underlying information in charts by posing direct, elementary questions about simple features such as axes ranges and values to examine their fundamental visual understanding abilities in the context of charts.Our questions are applied to two sets of figures: synthetic and real-world.The empirical evaluation of 5 popular multimodal models on our dataset reveals shortfalls in understanding charts and figures, contrary to what their performance on complex benchmarks might suggest.For instance, Gemini Pro Vision only achieves 57.9% accuracy on our elementary set of questions on real-world plots, while other popular multimodal models showed similar or less performance.This work highlights an important limitation of current multimodal models, and cautions against overly optimistic interpretations of their abilities based on results of canonical evaluations.
@inproceedings{plottwist:emnlp24, author = {Yasaman Razeghi and Ishita Dasgupta and Fangyu Liu and Vinay Venkatesh Ramasesh and Sameer Singh}, title = { {PlotTwist: Multimodal Models Don't Comprehend Simple Chart Details} }, booktitle = {Empirical Methods in Natural Language Processing (EMNLP)}, doi = {10.18653/v1/2024.findings-emnlp.342}, pages = {5922-5937}, year = {2024} } -
Perceptions of Linguistic Uncertainty by Language Models and Humans
*Uncertainty expressions* such as 'probably' or 'highly unlikely' are pervasive in human language. While prior work has established that there is population-level agreement in terms of how humans quantitatively interpret these expressions, there has been little inquiry into the abilities of language models in the same context. In this paper, we investigate how language models map linguistic expressions of uncertainty to numerical responses. Our approach assesses whether language models can employ theory of mind in this setting: understanding the uncertainty of another agent about a particular statement, independently of the model's own certainty about that statement. We find that 7 out of 10 models are able to map uncertainty expressions to probabilistic responses in a human-like manner. However, we observe systematically different behavior depending on whether a statement is actually true or false. This sensitivity indicates that language models are substantially more susceptible to bias based on their prior knowledge (as compared to humans). These findings raise important questions and have broad implications for human-AI and AI-AI communication.
@inproceedings{linguncertain:emnlp24, author = {Catarina Belem and Markelle Kelly and Mark Steyvers and Sameer Singh and Padhraic Smyth}, title = { {Perceptions of Linguistic Uncertainty by Language Models and Humans} }, booktitle = {Empirical Methods in Natural Language Processing (EMNLP)}, doi = {10.18653/v1/2024.emnlp-main.483}, pages = {8467-8502}, year = {2024} } -
Downstream Trade-offs of a Family of Text Watermarks
Watermarking involves implanting an imperceptible signal into generated text that can later be detected via statistical tests. A prominent family of watermarking strategies for LLMs embeds this signal by upsampling a (pseudorandomly-chosen) subset of tokens at every generation step. However, such signals alter the model's output distribution and can have unintended effects on its downstream performance. In this work, we evaluate the performance of LLMs watermarked using three different strategies over a diverse suite of tasks including those cast as k-class classification (CLS), multiple choice question answering (MCQ), short-form generation (e.g., open-ended question answering) and long-form generation (e.g., translation) tasks. We find that watermarks (under realistic hyperparameters) can cause significant drops in LLMs' effective utility across all tasks. We observe drops of 10 to 20% in CLS tasks in the average case, which shoot up to 100% in the worst case. We notice degradations of about 7% in MCQ tasks, 10-15% in short-form generation, and 5-15% in long-form generation tasks. Our findings highlight the trade-offs that users should be cognizant of when using watermarked models.
@inproceedings{watermark:emnlp24, author = {Anirudh Ajith and Sameer Singh and Danish Pruthi}, title = { {Downstream Trade-offs of a Family of Text Watermarks} }, booktitle = {Empirical Methods in Natural Language Processing (EMNLP)}, doi = {10.18653/v1/2024.findings-emnlp.821}, pages = {14039-14053}, year = {2024} } -
Measuring and Improving Attentiveness to Partial Inputs with Counterfactuals
The inevitable appearance of spurious correlations in training datasets hurts the generalization of NLP models on unseen data. Previous work has found that datasets with paired inputs are prone to correlations between a specific part of the input (e.g., the hypothesis in NLI) and the label; consequently, models trained only on those outperform chance. Are these correlations picked up by models trained on the full input data? To address this question, we propose a new evaluation method, Counterfactual Attentiveness Test (CAT). CAT uses counterfactuals by replacing part of the input with its counterpart from a different example (subject to some restrictions), expecting an attentive model to change its prediction. Using CAT, we systematically investigate established supervised and in-context learning models on ten datasets spanning four tasks: natural language inference, reading comprehension, paraphrase detection, and visual & language reasoning. CAT reveals that reliance on such correlations is mainly data-dependent. Surprisingly, we find that GPT3 becomes less attentive with an increased number of demonstrations, while its accuracy on the test data improves. Our results demonstrate that augmenting training or demonstration data with counterfactuals is effective in improving models' attentiveness. We show that models' attentiveness measured by CAT reveals different conclusions from solely measuring correlations in data.
@inproceedings{partial:emnlp24, author = {Yanai Elazar and Bhargavi Paranjape and Hao Peng and Sarah Wiegreffe and Khyathi Chandu and Vivek Srikumar and Sameer Singh and Noah A. Smith}, title = { {Measuring and Improving Attentiveness to Partial Inputs with Counterfactuals} }, booktitle = {Empirical Methods in Natural Language Processing (EMNLP)}, doi = {10.18653/v1/2024.findings-emnlp.205}, pages = {3603-3623}, year = {2024} } -
Skill Set Optimization: Reinforcing Language Model Behavior via Transferable Skills
@inproceedings{skillsetopt:icml24, author = {Kolby Nottingham and Bodhisattwa Prasad Majumder and Bhavana Dalvi Mishra and Sameer Singh and Peter Clark and Roy Fox}, title = { {Skill Set Optimization: Reinforcing Language Model Behavior via Transferable Skills} }, booktitle = {International Conference on Machine Learning (ICML)}, year = {2024} } -
MisgenderMender: A Community-Informed Approach to Interventions for Misgendering
Content Warning: This paper contains examples of misgendering and erasure that could be offensive and potentially triggering.Misgendering, the act of incorrectly addressing someone's gender, inflicts serious harm and is pervasive in everyday technologies, yet there is a notable lack of research to combat it. We are the first to address this lack of research into interventions for misgendering by conducting a survey of gender-diverse individuals in the US to understand perspectives about automated interventions for text-based misgendering. Based on survey insights on the prevalence of misgendering, desired solutions, and associated concerns, we introduce a misgendering interventions task and evaluation dataset, MisgenderMender. We define the task with two sub-tasks: (i) detecting misgendering, followed by (ii) correcting misgendering where misgendering is present, in domains where editing is appropriate. MisgenderMender comprises 3790 instances of social media content and LLM-generations about non-cisgender public figures, annotated for the presence of misgendering, with additional annotations for correcting misgendering in LLM-generated text. Using this dataset, we set initial benchmarks by evaluating existing NLP systems and highlighting challenges for future models to address. We release the full dataset, code, and demo at https://tamannahossainkay.github.io/misgendermender/
@inproceedings{misgendermender:naacl24, author = {Tamanna Hossain and Sunipa Dev and Sameer Singh}, title = { {MisgenderMender: A Community-Informed Approach to Interventions for Misgendering} }, booktitle = {Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)}, doi = {10.18653/v1/2024.naacl-long.419}, pages = {7538-7558}, year = {2024} } -
The Bias Amplification Paradox in Text-to-Image Generation
Bias amplification is a phenomenon in which models exacerbate biases or stereotypes present in the training data. In this paper, we study bias amplification in the text-to-image domain using Stable Diffusion by comparing gender ratios in training vs. generated images. We find that the model appears to amplify gender-occupation biases found in the training data (LAION) considerably. However, we discover that amplification can be largely attributed to discrepancies between training captions and model prompts. For example, an inherent difference is that captions from the training data often contain explicit gender information while our prompts do not, which leads to a distribution shift and consequently inflates bias measures. Once we account for distributional differences between texts used for training and generation when evaluating amplification, we observe that amplification decreases drastically. Our findings illustrate the challenges of comparing biases in models and their training data, as well as evaluation more broadly, and highlight how confounding factors can impact analyses.
@inproceedings{biasampl:naacl24, author = {Preethi Seshadri and Sameer Singh and Yanai Elazar}, title = { {The Bias Amplification Paradox in Text-to-Image Generation} }, booktitle = {Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)}, doi = {10.18653/v1/2024.naacl-long.353}, pages = {6367-6384}, year = {2024} } -
EchoPrompt: Instructing the Model to Rephrase Queries for Improved In-context Learning
Language models are achieving impressive performance on various tasks by aggressively adopting inference-time prompting techniques,such as zero-shot and few-shot prompting. In this work, we introduce EchoPrompt, a simple yet effective approach that prompts the model to rephrase its queries before answering them. EchoPrompt is tailored for four scenarios, including standard and chain-of-thought prompting, in both zero-shot and few-shot settings. Experimental results show that EchoPrompt yields substantial improvementsacross all these settings for four families of causal language models. These improvements are observed across various numerical reasoning (e.g., GSM8K, SVAMP), reading comprehension (e.g., DROP), and logical reasoning (e.g., Coin flipping) tasks. On average, EchoPrompt improves the Zero-shot-CoT performance of code-davinci-002 by 5% in numerical tasks and 13% in reading comprehension tasks. Our empirical results indicate that EchoPrompt is an effective technique that enhances in-context learning performance.
@inproceedings{echoprompt:naacl24, author = {Raja Sekhar Reddy Mekala and Yasaman Razeghi and Sameer Singh}, title = { {EchoPrompt: Instructing the Model to Rephrase Queries for Improved In-context Learning} }, booktitle = {Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)}, doi = {10.18653/v1/2024.naacl-short.35}, pages = {399-432}, year = {2024} } -
Selective Perception: Learning Concise State Descriptions for Language Model Actors
The latest large language models (LMs) support increasingly longer contexts. While this trend permits using substantial amounts of text with SOTA LMs, requiring these large LMs to process potentially redundant or irrelevant data needlessly increases inference time and cost. To remedy this problem, we propose BLINDER, a method that leverages a small finetuned LM to sample the minimal set of input features that maximizes the performance of a downstream LM. BLINDER trains an LM with a value head to estimate the likelihood of optimal outputs from a downstream LM given an input. We evaluate BLINDER on embodied decision making tasks with notoriously verbose state descriptions: NetHack and robot planning. BLINDER reduces the length of LM actor input by 87% and 99% while improving task success rates by 158% and 54% on NetHack and robot planning respectively which represents substantial inference cost savings while actually increasing performance.
@inproceedings{selperception:naacl24, author = {Kolby Nottingham and Yasaman Razeghi and Kyungmin Kim and JB Lanier and Pierre Baldi and Roy Fox and Sameer Singh}, title = { {Selective Perception: Learning Concise State Descriptions for Language Model Actors} }, booktitle = {Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)}, doi = {10.18653/v1/2024.naacl-short.29}, pages = {327-341}, year = {2024} } -
What's In My Big Data?
@inproceedings{wimbd:iclr24, author = {Yanai Elazar and Akshita Bhagia and Ian Helgi Magnusson and Abhilasha Ravichander and Dustin Schwenk and Alane Suhr and Evan Pete Walsh and Dirk Groeneveld and Luca Soldaini and Sameer Singh and Hannaneh Hajishirzi and Noah A. Smith and Jesse Dodge}, title = { {What's In My Big Data?} }, booktitle = {International Conference on Learning Representations (ICLR)}, year = {2024} }
-
To Adapt or to Annotate: Challenges and Interventions for Domain Adaptation in Open-Domain Question Answering
Recent advances in open-domain question answering (ODQA) have demonstrated impressive accuracy on general-purpose domains like Wikipedia. While some work has been investigating how well ODQA models perform when tested for out-of-domain (OOD) generalization, these studies have been conducted only under conservative shifts in data distribution and typically focus on a single component (i.e., retriever or reader) rather than an end-to-end system. This work proposes a more realistic end-to-end domain shift evaluation setting covering five diverse domains. We not only find that end-to-end models fail to generalize but that high retrieval scores often still yield poor answer prediction accuracy. To address these failures, we investigate several interventions, in the form of data augmentations, for improving model adaption and use our evaluation set to elucidate the relationship between the efficacy of an intervention scheme and the particular type of dataset shifts we consider. We propose a generalizability test that estimates the type of shift in a target dataset without training a model in the target domain and that the type of shift is predictive of which data augmentation schemes will be effective for domain adaption. Overall, we find that these interventions increase end-to-end performance by up to ~24 points.
@inproceedings{adaptqa:acl23, author = {Dheeru Dua and Emma Strubell and Sameer Singh and Pat Verga}, title = { {To Adapt or to Annotate: Challenges and Interventions for Domain Adaptation in Open-Domain Question Answering} }, booktitle = {Association for Computational Linguistics (ACL)}, doi = {10.18653/v1/2023.acl-long.807}, pages = {14429–14446}, year = {2023} } -
MISGENDERED: Limits of Large Language Models in Understanding Pronouns
Content Warning: This paper contains examples of misgendering and erasure that could be offensive and potentially triggering.Gender bias in language technologies has been widely studied, but research has mostly been restricted to a binary paradigm of gender. It is essential also to consider non-binary gender identities, as excluding them can cause further harm to an already marginalized group. In this paper, we comprehensively evaluate popular language models for their ability to correctly use English gender-neutral pronouns (e.g., singular they, them) and neo-pronouns (e.g., ze, xe, thon) that are used by individuals whose gender identity is not represented by binary pronouns. We introduce Misgendered, a framework for evaluating large language models’ ability to correctly use preferred pronouns, consisting of (i) instances declaring an individual’s pronoun, followed by a sentence with a missing pronoun, and (ii) an experimental setup for evaluating masked and auto-regressive language models using a unified method. When prompted out-of-the-box, language models perform poorly at correctly predicting neo-pronouns (averaging 7.6% accuracy) and gender-neutral pronouns (averaging 31.0% accuracy). This inability to generalize results from a lack of representation of non-binary pronouns in training data and memorized associations. Few-shot adaptation with explicit examples in the prompt improves the performance but plateaus at only 45.4% for neo-pronouns. We release the full dataset, code, and demo at https://tamannahossainkay.github.io/misgendered/.
@inproceedings{misgendered:acl23, author = {Tamanna Hossain and Sunipa Dev and Sameer Singh}, title = { {MISGENDERED: Limits of Large Language Models in Understanding Pronouns} }, booktitle = {Association for Computational Linguistics (ACL)}, doi = {10.18653/v1/2023.acl-long.293}, pages = {5352–5367}, year = {2023} } -
Do Embodied Agents Dream of Pixelated Sheep: Embodied Decision Making using Language Guided World Modelling
@inproceedings{deckard:icml23, author = {Kolby Nottingham and Prithviraj Ammanabrolu and Alane Suhr and Yejin Choi and Hannaneh Hajishirzi and Sameer Singh and Roy Fox}, title = { {Do Embodied Agents Dream of Pixelated Sheep: Embodied Decision Making using Language Guided World Modelling} }, booktitle = {International Conference on Machine Learning (ICML)}, year = {2023} } -
Towards Factual and Informative Review Generation for Explainable Recommendation
@inproceedings{recomm:aaai23, author = {Zhouhang Xie and Sameer Singh and Julian McAuley and Bodhisattwa P. Majumder}, title = { {Towards Factual and Informative Review Generation for Explainable Recommendation} }, booktitle = {AAAI Conference on Artificial Intelligence (AAAI)}, year = {2023} } -
Maestro: A Gamified Platform for Teaching AI Robustness
@inproceedings{maestro:eaai23, author = {Margarita Geleta and Jiacen Xu and Manikanta Loya and Junlin Wang and Sameer Singh and Zhou Li and Sergio Gago Masague}, title = { {Maestro: A Gamified Platform for Teaching AI Robustness} }, booktitle = {AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI)}, year = {2023} } -
Post Hoc Explanations of Language Models Can Improve Language Models
@inproceedings{posthoc:neurips23, author = {Satyapriya Krishna and Jiaqi Ma and Dylan Slack and Asma Ghandeharioun and Sameer Singh and Himabindu Lakkaraju}, title = { {Post Hoc Explanations of Language Models Can Improve Language Models} }, booktitle = {Neural Information Processing Systems (NeurIPS)}, year = {2023} } -
Coverage-based Example Selection for In-Context Learning
@inproceedings{demselect:emnlp23, author = {Shivanshu Gupta and Matt Gardner and Sameer Singh}, title = { {Coverage-based Example Selection for In-Context Learning} }, booktitle = {Empirical Methods in Natural Language Processing (EMNLP)}, year = {2023} } -
Evaluating the generalisability of neural rumour verification models
Research on automated social media rumour verification, the task of identifying the veracity of questionable information circulating on social media, has yielded neural models achieving high performance, with accuracy scores that often exceed 90%. However, none of these studies focus on the real-world generalisability of the proposed approaches, that is whether the models perform well on datasets other than those on which they were initially trained and tested. In this work we aim to fill this gap by assessing the generalisability of top performing neural rumour verification models covering a range of different architectures from the perspectives of both topic and temporal robustness. For a more complete evaluation of generalisability, we collect and release COVID-RV, a novel dataset of Twitter conversations revolving around COVID-19 rumours. Unlike other existing COVID-19 datasets, our COVID-RV contains conversations around rumours that follow the format of prominent rumour verification benchmarks, while being different from them in terms of topic and time scale, thus allowing better assessment of the temporal robustness of the models. We evaluate model performance on COVID-RV and three popular rumour verification datasets to understand limitations and advantages of different model architectures, training datasets and evaluation scenarios. We find a dramatic drop in performance when testing models on a different dataset from that used for training. Further, we evaluate the ability of models to generalise in a few-shot learning setup, as well as when word embeddings are updated with the vocabulary of a new, unseen rumour. Drawing upon our experiments we discuss challenges and make recommendations for future research directions in addressing this important problem.
@article{rumors:ipm23, author = {Elena Kochkina and Tamanna Hossain and Robert L. Logan IV and Miguel Arana-Catania and Rob Procter and Arkaitz Zubiaga and Sameer Singh and Yulan He and Maria Liakata}, title = { {Evaluating the generalisability of neural rumour verification models} }, journal = {Information Processing and Management}, doi = {10.1016/j.ipm.2022.103116}, year = {2023} } -
Explaining machine learning models with interactive natural language conversations using TalkToModel
Practitioners increasingly use machine learning (ML) models, yet models have become more complex and harder to understand. To understand complex models, researchers have proposed techniques to explain model predictions. However, practitioners struggle to use explainability methods because they do not know which explanation to choose and how to interpret the explanation. Here we address the challenge of using explainability methods by proposing TalkToModel: an interactive dialogue system that explains ML models through natural language conversations. TalkToModel consists of three components: an adaptive dialogue engine that interprets natural language and generates meaningful responses; an execution component that constructs the explanations used in the conversation; and a conversational interface. In real-world evaluations, 73% of healthcare workers agreed they would use TalkToModel over existing systems for understanding a disease prediction model, and 85% of ML professionals agreed TalkToModel was easier to use, demonstrating that TalkToModel is highly effective for model explainability.
@article{talktomodel:ni23, author = {Dylan Slack and Satyapriya Krishna and Himabindu Lakkaraju and Sameer Singh}, title = { {Explaining machine learning models with interactive natural language conversations using TalkToModel} }, journal = {Nature Machine Intelligence}, doi = {10.1038/s42256-023-00692-8}, year = {2023} }
-
Successive Prompting for Decomposing Complex Questions
@inproceedings{decompqa:emnlp22, author = {Dheeru Dua and Shivanshu Gupta and Sameer Singh and Matt Gardner}, title = { {Successive Prompting for Decomposing Complex Questions} }, booktitle = {Empirical Methods in Natural Language Processing (EMNLP)}, year = {2022} } -
Continued Pretraining for Better Zero- and Few-Shot Promptability
@inproceedings{pretraining:emnlp22, author = {Zhaofeng Wu and Robert L. Logan IV and Pete Walsh and Akshita Bhagia and Dirk Groeneveld and Sameer Singh and Iz Beltagy}, title = { {Continued Pretraining for Better Zero- and Few-Shot Promptability} }, booktitle = {Empirical Methods in Natural Language Processing (EMNLP)}, year = {2022} } -
Impact of Pretraining Term Frequencies on Few-Shot Numerical Reasoning
@inproceedings{impact:femnlp22, author = {Yasaman Razeghi and Robert L. Logan IV and Matt Gardner and Sameer Singh}, title = { {Impact of Pretraining Term Frequencies on Few-Shot Numerical Reasoning} }, booktitle = {Findings of the Association for Computational Linguistics: EMNLP (EMNLP Findings)}, year = {2022} } -
Structurally Diverse Sampling for Sample-Efficient Training and Comprehensive Evaluation
@inproceedings{structdiversity:femnlp22, author = {Shivanshu Gupta and Sameer Singh and Matt Gardner}, title = { {Structurally Diverse Sampling for Sample-Efficient Training and Comprehensive Evaluation} }, booktitle = {Findings of the Association for Computational Linguistics: EMNLP (EMNLP Findings)}, year = {2022} } -
Unobserved Local Structures Make Compositional Generalization Hard
@inproceedings{compgenhardness:emnlp22, author = {Ben Bogin and Shivanshu Gupta and Jonathan Berant}, title = { {Unobserved Local Structures Make Compositional Generalization Hard} }, booktitle = {Empirical Methods in Natural Language Processing (EMNLP)}, year = {2022} } -
Learning to Query Internet Text for Informing Reinforcement Learning Agents
Extended abstractGeneralization to out of distribution tasks in reinforcement learning is a challenging problem. One successful approach improves generalization by conditioning policies on task or environment descriptions that provide information about the current transition or reward functions. Previously, these descriptions were often expressed as generated or crowd sourced text. In this work, we begin to tackle the problem of extracting useful information from natural language found in the wild (e.g. internet forums, documentation, and wikis). These natural, pre-existing sources are especially challenging, noisy, and large and present novel challenges compared to previous approaches. We propose to address these challenges by training reinforcement learning agents to learn to query these sources as a human would, and we experiment with how and when an agent should query. To address the how, we demonstrate that pretrained QA models perform well at executing zero-shot queries in our target domain. Using information retrieved by a QA model, we train an agent to learn when it should execute queries. We show that our method correctly learns to execute queries to maximize reward in a reinforcement learning setting.
@inproceedings{queryrl:rldm22, author = {Kolby Nottingham and Alekhya Pyla and Sameer Singh and Roy Fox}, title = { {Learning to Query Internet Text for Informing Reinforcement Learning Agents} }, booktitle = {Reinforcement Learning and Decision Making (RLDM)}, year = {2022} } -
FRUIT: Faithfully Reflecting Updated Information in Text
Best Task Paper AwardTextual knowledge bases such as Wikipedia require considerable effort to keep up to date and consistent. While automated writing assistants could potentially ease this burden, the problem of suggesting edits grounded in external knowledge has been under-explored. In this paper, we introduce the novel generation task of *faithfully reflecting updated information in text* (FRUIT) where the goal is to update an existing article given new evidence. We release the FRUIT-WIKI dataset, a collection of over 170K distantly supervised data produced from pairs of Wikipedia snapshots, along with our data generation pipeline and a gold evaluation set of 914 instances whose edits are guaranteed to be supported by the evidence. We provide benchmark results for popular generation systems as well as EDIT5 – a T5-based approach tailored to editing we introduce that establishes the state of the art. Our analysis shows that developing models that can update articles faithfully requires new capabilities for neural generation models, and opens doors to many new applications.
@inproceedings{fruit:naacl22, author = {Robert L. Logan IV and Alexandre Passos and Sameer Singh and Ming-Wei Chang}, title = { {FRUIT: Faithfully Reflecting Updated Information in Text} }, booktitle = {Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)}, doi = {10.18653/v1/2022.naacl-main.269}, pages = {3670–3686}, year = {2022} } -
Prompt Waywardness: The Curious Case of Discretized Interpretation of Continuous Prompts
Fine-tuning continuous prompts for target tasks has recently emerged as a compact alternative to full model fine-tuning. Motivated by these promising results, we investigate the feasibility of extracting a discrete (textual) interpretation of continuous prompts that is faithful to the problem they solve. In practice, we observe a “wayward” behavior between the task solved by continuous prompts and their nearest neighbor discrete projections: We can find continuous prompts that solve a task while being projected to an arbitrary text (e.g., definition of a different or even a contradictory task), while being within a very small (2%) margin of the best continuous prompt of the same size for the task. We provide intuitions behind this odd and surprising behavior, as well as extensive empirical analyses quantifying the effect of various parameters. For instance, for larger model sizes we observe higher waywardness, i.e, we can find prompts that more closely map to any arbitrary text with a smaller drop in accuracy. These findings have important implications relating to the difficulty of faithfully interpreting continuous prompts and their generalization across models and tasks, providing guidance for future progress in prompting language models.
@inproceedings{wayward:naacl22, author = {Daniel Khashabi and Xinxi Lyu and Sewon Min and Lianhui Qin and Kyle Richardson and Sean Welleck and Hannaneh Hajishirzi and Tushar Khot and Ashish Sabharwal and Sameer Singh and Yejin Choi}, title = { {Prompt Waywardness: The Curious Case of Discretized Interpretation of Continuous Prompts} }, booktitle = {Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)}, pages = {3631-3643}, year = {2022} } -
ReCLIP: A Strong Zero-Shot Baseline for Referring Expression Comprehension
Training a referring expression comprehension (ReC) model for a new visual domain requires collecting referring expressions, and potentially corresponding bounding boxes, for images in the domain. While large-scale pre-trained models are useful for image classification across domains, it remains unclear if they can be applied in a zero-shot manner to more complex tasks like ReC. We present ReCLIP, a simple but strong zero-shot baseline that repurposes CLIP, a state-of-the-art large-scale model, for ReC. Motivated by the close connection between ReC and CLIP’s contrastive pre-training objective, the first component of ReCLIP is a region-scoring method that isolates object proposals via cropping and blurring, and passes them to CLIP. However, through controlled experiments on a synthetic dataset, we find that CLIP is largely incapable of performing spatial reasoning off-the-shelf. We reduce the gap between zero-shot baselines from prior work and supervised models by as much as 29% on RefCOCOg, and on RefGTA (video game imagery), ReCLIP’s relative improvement over supervised ReC models trained on real images is 8%.
@inproceedings{reclip:acl22, author = {Sanjay Subramanian and William Merrill and Trevor Darrell and Matt Gardner and Sameer Singh and Anna Rohrbach}, title = { {ReCLIP: A Strong Zero-Shot Baseline for Referring Expression Comprehension } }, booktitle = {Association for Computational Linguistics (ACL)}, doi = {10.18653/v1/2022.acl-long.357}, pages = {5198-5215}, year = {2022} } -
Combining Feature and Instance Attribution to Detect Artifacts
Training the deep neural networks that dominate NLP requires large datasets. These are often collected automatically or via crowdsourcing, and may exhibit systematic biases or annotation artifacts. By the latter we mean spurious correlations between inputs and outputs that do not represent a generally held causal relationship between features and classes; models that exploit such correlations may appear to perform a given task well, but fail on out of sample data. In this paper we evaluate use of different attribution methods for aiding identification of training data artifacts. We propose new hybrid approaches that combine saliency maps (which highlight "important" input features) with instance attribution methods (which retrieve training samples "influential" to a given prediction). We show that this proposed training-feature attribution can be used to efficiently uncover artifacts in training data when a challenging validation set is available. We also carry out a small user study to evaluate whether these methods are useful to NLP researchers in practice, with promising results.
@inproceedings{tfa:facl22, author = {Pouya Pezeshkpour and Sarthak Jain and Sameer Singh and Byron Wallace}, title = { {Combining Feature and Instance Attribution to Detect Artifacts} }, booktitle = {Findings of the Association for Computational Linguistics (ACL Findings)}, doi = {10.18653/v1/2022.findings-acl.153}, pages = {1934–1946}, year = {2022} } -
Cutting Down on Prompts and Parameters: Simple Few-Shot Learning with Language Models
Also presented at the Neurips workshop on Efficient Natural Language and Speech Processing (ENLSP)Prompting language models (LMs) with training examples and task descriptions has been seen as critical to recent successes in few-shot learning. In this work, we show that finetuning LMs in the few-shot setting can considerably reduce the need for prompt engineering. In fact, one can use null prompts, prompts that contain neither task-specific templates nor training examples, and achieve competitive accuracy to manually-tuned prompts across a wide range of tasks. While finetuning LMs does introduce new parameters for each downstream task, we show that this memory overhead can be substantially reduced: finetuning only the bias terms can achieve comparable or better accuracy than standard finetuning while only updating 0.1% of the parameters. All in all, we recommend finetuning LMs for few-shot learning as it is more accurate, robust to different prompts, and can be made nearly as efficient as using frozen LMs.
@inproceedings{cutting:facl22, author = {Robert L. Logan IV and Ivana Balažević and Eric Wallace and Fabio Petroni and Sameer Singh and Sebastian Riedel}, title = { {Cutting Down on Prompts and Parameters: Simple Few-Shot Learning with Language Models} }, booktitle = {Findings of the Association for Computational Linguistics (ACL Findings)}, doi = {10.18653/v1/2022.findings-acl.222}, pages = {2824–2835}, year = {2022} } -
BottleFit: Learning Compressed Representations in Deep Neural Networks for Effective and Efficient Split Computing
Although mission-critical applications require the use of deep neural networks (DNNs), their continuous execution at mobile devices results in a significant increase in energy consumption. While edge offloading can decrease energy consumption, erratic patterns in channel quality, network and edge server load can lead to severe disruption of the system’s key operations. An alternative approach, called split computing, generates compressed representations within the model (called "bottlenecks"), to reduce bandwidth usage and energy consumption. Prior work has proposed approaches that introduce additional layers, to the detriment of energy consumption and latency. For this reason, we propose a new framework called BottleFit, which, in addition to targeted DNN architecture modifications, includes a novel training strategy to achieve high accuracy even with strong compression rates. We apply BottleFit on cutting-edge DNN models in image classification, and show that BottleFit achieves 77.1% data compression with up to 0.6% accuracy loss on ImageNet dataset, while state of the art such as SPINN loses up to 6% in accuracy. We experimentally measure the power consumption and latency of an image classification application running on an NVIDIA Jetson Nano board (GPU-based) and a Raspberry PI board (GPU-less). We show that BottleFit decreases power consumption and latency respectively by up to 49% and 89% with respect to (w.r.t.) local computing and by 37% and 55% w.r.t. edge offloading. We also compare BottleFit with state-of-the-art autoencoders-based approaches, and show that (i) BottleFit reduces power consumption and execution time respectively by up to 54% and 44% on the Jetson and 40% and 62% on Raspberry PI; (ii) the size of the head model executed on the mobile device is 83 times smaller. We publish the code repository for reproducibility of the results in this study.
@inproceedings{bottlefit:wowmom22, author = {Yoshitomo Matsubara and Davide Callegaro and Sameer Singh and Marco Levorato and Francesco Restuccia}, title = { {BottleFit: Learning Compressed Representations in Deep Neural Networks for Effective and Efficient Split Computing} }, booktitle = {IEEE International Symposium on a World of Wireless, Mobile and Multimedia Networks (WoWMoM)}, doi = {10.1109/WoWMoM54355.2022.00032}, pages = {337-346}, year = {2022} } -
Snoopy: An Online Interface for Exploring the Effect of Pretraining Term Frequencies on Few-Shot LM Performance
@inproceedings{snoopy:emnlp22, author = {Yasaman Razeghi and Raja Sekhar Reddy Mekala and Robert L. Logan IV and Matt Gardner and Sameer Singh}, title = { {Snoopy: An Online Interface for Exploring the Effect of Pretraining Term Frequencies on Few-Shot LM Performance} }, booktitle = {Demo at the Empirical Methods in Natural Language Processing (EMNLP)}, year = {2022} } -
PYLON: A PyTorch Framework for Learning with Constraints
Also presented as a Demo paper at Neurips 2021.AAAI ProceedingsAAAI PDF (shorter)Neurips ProceedingsNeurIPS PDF (longer)WebsiteCodeVideoAbstractBibTeXDeep learning excels at learning low-level task information from large amounts of data, but struggles with learning high-level domain knowledge, which can often be directly and succinctly expressed. In this work, we introduce Pylon, a neuro-symbolic training framework that builds on PyTorch to augment procedurally trained neural networks with declaratively specified knowledge. Pylon allows users to programmatically specify constraints as PyTorch functions, and compiles them into a differentiable loss, thus training predictive models that fit the data whilst satisfying the specified constraints. Pylon includes both exact as well as approximate compilers to efficiently compute the loss, employing fuzzy logic, sampling methods, and circuits, ensuring scalability even to complex models and constraints. A guiding principle in designing Pylon has been the ease with which any existing deep learning codebase can be extended to learn from constraints using only a few lines: a function expressing the constraint and a single line of code to compile it into a loss. We include case studies from natural language processing, computer vision, logical games, and knowledge graphs, that can be interactively trained, and highlights Pylon's usage.
@inproceedings{pylon:aaai22, author = {Kareem Ahmed and Tao Li and Thy Ton and Quan Guo and Kai-Wei Chang and Parisa Kordjamshidi and Vivek Srikumar and Guy Van den Broeck and Sameer Singh}, title = { {PYLON: A PyTorch Framework for Learning with Constraints} }, booktitle = {Demo at the AAAI Conference on Artificial Intelligence (AAAI)}, doi = {10.1609/aaai.v36i11.21711}, pages = {13152-13154}, year = {2022} } -
An Empirical Comparison of Machine Learning Methods for Text-based Sentiment Analysis of Online Consumer Reviews
@article{sentiment:ijrm22, author = {Huwail J.Alantari and Imran S.Currim and Yiting Deng and Sameer Singh}, title = { {An Empirical Comparison of Machine Learning Methods for Text-based Sentiment Analysis of Online Consumer Reviews} }, journal = {International Journal of Research in Marketing}, volume = {39}, number = {1}, doi = {10.1016/j.ijresmar.2021.10.011}, pages = {1-19}, year = {2022} } -
Quantifying Social Biases Using Templates is Unreliable
@inproceedings{templates:tsrml22, author = {Preethi Seshadri and Pouya Pezeshkpour and Sameer Singh}, title = { {Quantifying Social Biases Using Templates is Unreliable} }, booktitle = {TSRML Workshop @ NeurIPS}, year = {2022} } -
TalkToModel: Explaining Machine Learning Models with Interactive Natural Language Conversations
Honorable mention for Outstanding Paper award
@inproceedings{talktomodel:tsrml22, author = {Dylan Slack and Satyapriya Krishna and Himabindu Lakkaraju and Sameer Singh}, title = { {TalkToModel: Explaining Machine Learning Models with Interactive Natural Language Conversations } }, booktitle = {TSRML Workshop @ NeurIPS}, year = {2022} } -
Rethinking Explainability as a Dialogue: A Practitioner's Perspective
@inproceedings{rethinking:hcai22, author = {Himabindu Lakkaraju and Dylan Slack and Yuxin Chen and Chenhao Tan and Sameer Singh}, title = { {Rethinking Explainability as a Dialogue: A Practitioner's Perspective } }, booktitle = {HCAI Workshop @ NeurIPS}, year = {2022} } -
SAFER: Data-Efficient and Safe Reinforcement Learning via Skill Acquisition
@inproceedings{safer:darl22, author = {Dylan Slack and Yinlam Chow and Bo Dai and Nevan Wichers}, title = { {SAFER: Data-Efficient and Safe Reinforcement Learning via Skill Acquisition } }, booktitle = {DARL Workshop @ ICML}, year = {2022} }
-
COVR: A Test-Bed for Visually Grounded Compositional Generalization with Real Images
@inproceedings{covr:emnlp21, author = {Ben Bogin and Shivanshu Gupta and Jonathan Berant and Matt Gardner}, title = { {COVR: A Test-Bed for Visually Grounded Compositional Generalization with Real Images} }, booktitle = {Empirical Methods in Natural Language Processing (EMNLP)}, year = {2021} } -
Counterfactual Explanations Can Be Manipulated
@inproceedings{manipcfs:neurips21, author = {Dylan Slack and Sophie Hilgard and Himabindu Lakkaraju and Sameer Singh}, title = { {Counterfactual Explanations Can Be Manipulated} }, booktitle = {Neural Information Processing Systems (NeurIPS)}, year = {2021} } -
Reliable Post hoc Explanations Modeling Uncertainty in Explainability
@inproceedings{bayeslimeshap:neurips21, author = {Dylan Slack and Sophie Hilgard and Sameer Singh and Himabindu Lakkaraju}, title = { {Reliable Post hoc Explanations Modeling Uncertainty in Explainability} }, booktitle = {Neural Information Processing Systems (NeurIPS)}, year = {2021} } -
Generative Context Pair Selection for Multi-hop Question Answering
@inproceedings{genqa:emnlp21, author = {Dheeru Dua and Cicero Nogueira dos Santos and Patrick Ng and Ben Athiwaratkun and Bing Xiang and Matt Gardner and Sameer Singh}, title = { {Generative Context Pair Selection for Multi-hop Question Answering} }, booktitle = {Empirical Methods in Natural Language Processing (EMNLP)}, year = {2021} } -
Entity-Based Knowledge Conflicts in Question Answering
@inproceedings{qaconflicts:emnlp21, author = {Shayne Longpre and Kartik Perisetla and Anthony Chen and Nikhil Ramesh and Chris DuBois and Sameer Singh}, title = { {Entity-Based Knowledge Conflicts in Question Answering} }, booktitle = {Empirical Methods in Natural Language Processing (EMNLP)}, year = {2021} } -
Learning with Instance Bundles for Reading Comprehension
When training most modern reading comprehension models, all the questions associated with a context are treated as being independent from each other. However, closely related questions and their corresponding answers are not independent, and leveraging these relationships could provide a strong supervision signal to a model. Drawing on ideas from contrastive estimation, we introduce several new supervision techniques that compare question-answer scores across multiple related instances. Specifically, we normalize these scores across various neighborhoods of closely contrasting questions and/or answers, adding another cross entropy loss term that is used in addition to traditional maximum likelihood estimation. Our techniques require bundles of related question-answer pairs, which we can either mine from within existing data or create using various automated heuristics. We empirically demonstrate the effectiveness of training with instance bundles on two datasets -- HotpotQA and ROPES -- showing up to 11% absolute gains in accuracy.
@inproceedings{bundles:emnlp21, author = {Dheeru Dua and Pradeep Dasigi and Sameer Singh and Matt Gardner}, title = { {Learning with Instance Bundles for Reading Comprehension} }, booktitle = {Empirical Methods in Natural Language Processing (EMNLP)}, year = {2021} } -
Competency Problems: On Finding and Removing Artifacts in Language Data
@inproceedings{competency:emnlp21, author = {Matt Gardner and William Merrill and Jesse Dodge and Matthew Peters and Alexis Ross and Sameer Singh and Noah A. Smith}, title = { {Competency Problems: On Finding and Removing Artifacts in Language Data} }, booktitle = {Empirical Methods in Natural Language Processing (EMNLP)}, year = {2021} } -
Paired Examples as Indirect Supervision in Latent Decision Models
@inproceedings{pairednmn:emnlp21, author = {Nitish Gupta and Sameer Singh and Matt Gardner and Dan Roth}, title = { {Paired Examples as Indirect Supervision in Latent Decision Models} }, booktitle = {Empirical Methods in Natural Language Processing (EMNLP)}, year = {2021} } -
Calibrate Before Use: Improving Few-shot Performance of Language Models
@inproceedings{poisoning:icml21, author = {Tony Z. Zhao and Eric Wallace and Shi Feng and Dan Klein and Sameer Singh}, title = { {Calibrate Before Use: Improving Few-shot Performance of Language Models} }, booktitle = {International Conference on Machine Learning (ICML)}, year = {2021} } -
Evaluating Entity Disambiguation and the Role of Popularity in Retrieval-Based NLP
@inproceedings{amber:acl21, author = {Anthony Chen and Pallavi Gudipati and Shayne Longpre and Xiao Ling and Sameer Singh}, title = { {Evaluating Entity Disambiguation and the Role of Popularity in Retrieval-Based NLP} }, booktitle = {Association for Computational Linguistics (ACL)}, doi = {10.18653/v1/2021.acl-long.345}, year = {2021} } -
Benchmarking Scalable Methods for Streaming Cross Document Coreference
@inproceedings{streamingcdcr:acl21, author = {Robert L. Logan IV and Andrew McCallum and Sameer Singh and Dan Bikel}, title = { {Benchmarking Scalable Methods for Streaming Cross Document Coreference} }, booktitle = {Association for Computational Linguistics (ACL)}, doi = {10.18653/v1/2021.acl-long.364}, year = {2021} } -
Enforcing Consistency in Weakly Supervised Semantic Parsing
@inproceedings{spconsistency:acl21, author = {Nitish Gupta and Sameer Singh and Matt Gardner}, title = { {Enforcing Consistency in Weakly Supervised Semantic Parsing} }, booktitle = {Association for Computational Linguistics (ACL)}, doi = {10.18653/v1/2021.acl-short.22}, year = {2021} } -
An Empirical Comparison of Instance Attribution Methods for NLP
Widespread adoption of deep pretrained (masked) neural language models has motivated a pressing need for approaches for interpreting network outputs and for facilitating model debugging. Instance attribution methods constitute one means of accomplishing these goals by retrieving training instances that (may have) led to a particular prediction. Influence functions (IF) provide machinery for doing this by quantifying the effect that perturbing individual train instances would have on a specific test prediction. However, even approximating the IF is computationally expensive, to a degree that may be prohibitive in many cases. Might simpler approaches (e.g., retrieving train instance most similar to a given test point) perform comparably? In this work we evaluate the degree to which different potential instance attribution agree with respect to the importance of training samples. We find that simple retrieval methods yield training instances that differ from those identified via gradient-based methods (such as the IF), but that nonetheless exhibit desirable characteristics similar to more complex attribution methods.
@inproceedings{emp-instance:naacl21, author = {Pouya Pezeshkpour and Sarthak Jain and Byron Wallace and Sameer Singh}, title = { {An Empirical Comparison of Instance Attribution Methods for NLP} }, booktitle = {Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)}, year = {2021} } -
Concealed Data Poisoning Attacks on NLP Models
Adversarial attacks alter NLP model predictions by perturbing test-time inputs. However, it is much less understood whether, and how, predictions can be manipulated with small, concealed changes to the training data. In this work, we develop a new data poisoning attack that allows an adversary to control model predictions whenever a desired trigger phrase is present in the input. For instance, we insert 50 poison examples into a sentiment model’s training set that causes the model to frequently predict Positive whenever the input contains “James Bond”. Crucially, we craft these poison examples using a gradient-based procedure so that they do not mention the trigger phrase. We also apply our poison attack to language modeling (“Apple iPhone” triggers negative generations) and machine translation (“iced coffee” mistranslated as “hot coffee”). We conclude by proposing three defenses that can mitigate our attack at some cost in prediction accuracy or extra human annotation.
@inproceedings{poisoning:naacl21, author = {Eric Wallace and Tony Z. Zhao and Shi Feng and Sameer Singh}, title = { {Concealed Data Poisoning Attacks on NLP Models} }, booktitle = {Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)}, year = {2021} } -
Improved Consistency Regularization for GANs
Recent work has increased the performance of Generative Adversarial Networks (GANs) by enforcing a consistency cost on the discriminator. We improve on this technique in several ways. We first show that consistency regularization can introduce artifacts into the GAN samples and explain how to fix this issue. We then propose several modifications to the consistency regularization procedure designed to improve its performance. We carry out extensive experiments quantifying the benefit of our improvements. For unconditional image synthesis on CIFAR-10 and CelebA, our modifications yield the best known FID scores on various GAN architectures. For conditional image synthesis on CIFAR-10, we improve the state-of-the-art FID score from 11.48 to 9.21. Finally, on ImageNet-2012, we apply our technique to the original BigGAN model and improve the FID from 6.66 to 5.38, which is the best score at that model size.
@inproceedings{icrgan:aaai21, author = {Zhengli Zhao and Sameer Singh and Honglak Lee and Zizhao Zhang and Augustus Odena and Han Zhang}, title = { {Improved Consistency Regularization for GANs} }, booktitle = {AAAI Conference on Artificial Intelligence (AAAI)}, year = {2021} } -
PARSINLU: A Suite of Language Understanding Challenges for Persian
Despite the progress made in recent years in addressing natural language understanding (NLU) challenges, the majority of this progress remains to be concentrated on resource-rich languages like English. This work focuses on Persian language, one of the widely spoken languages in the world, and yet there are few NLU datasets available for this rich language. The availability of high-quality evaluation datasets is a necessity for reliable assessment of the progress on different NLU tasks and domains. We introduce PARSINLU, the first benchmark in Persian language that includes a range of high-level tasks — Reading Comprehension, Textual Entailment, etc. These datasets are collected in a multitude of ways, often involving manual annotations by native speakers. This results in over 14.5k new instances across 6 distinct NLU tasks. Besides, we present the first results on state-of-the-art monolingual and multi-lingual pre-trained language-models on this benchmark and compare them with human performance, which provides valuable insights into our ability to tackle natural language understanding challenges in Persian. We hope PARSINLU fosters further research and advances in Persian language understanding.
@article{parsinlu:tacl21, author = {Daniel Khashabi and Arman Cohan and Siamak Shakeri and Pedram Hosseini and Pouya Pezeshkpour and Malihe Alikhani and Moin Aminnaseri and Marzieh Bitaab and Faeze Brahman and Sarik Ghazarian and Mozhdeh Gheini and Arman Kabiri and Rabeeh Karimi Mahabadi and Omid Memarrast and Ahmadreza Mosallanezhad and Erfan Noury and Shahab Raji and Mohammad Sadegh Rasooli and Sepideh Sadeghi and Erfan Sadeqi Azer and Niloofar Safi Samghabadi and Mahsa Shafaei and Saber Sheybani and Ali Tazarv and Yadollah Yaghoobzadeh}, title = { {PARSINLU: A Suite of Language Understanding Challenges for Persian} }, journal = {Transactions of the Association for Computational Linguistics (TACL)}, year = {2021} } -
Climatology and Evolution of the Antarctic Peninsula Föhn Wind‐Induced Melt Regime From 1979–2018
@article{fohn:jgr21, author = {Matthew K Laffin and Charles Zender and Sameer Singh and J. Van Wessem and C. J. P. P. Smeets and C. H. Reijmer}, title = { {Climatology and Evolution of the Antarctic Peninsula Föhn Wind‐Induced Melt Regime From 1979–2018} }, journal = {Journal of Geophysical Research: Atmospheres}, volume = {126}, number = {4}, doi = {10.1029/2020JD033682}, year = {2021} } -
Cutting Down on Prompts and Parameters: Simple Few-Shot Learning with Language Models
Best Poster Award
@inproceedings{nullprompts:effnlp21, author = {Robert L. Logan IV and Ivana Balažević and Eric Wallace and Fabio Petroni and Sameer Singh and Sebastian Riedel}, title = { {Cutting Down on Prompts and Parameters: Simple Few-Shot Learning with Language Models} }, booktitle = {NeurIPS Workshop on Efficient Natural Language and Speech Processing (ENLSP)}, year = {2021} } -
Modular Framework for Visuomotor Language Grounding
@inproceedings{modulargl:embodied21, author = {Kolby Nottingham and Litian Liang and Daeyun Shin and Charless C. Fowlkes and Roy Fox and Sameer Singh}, title = { {Modular Framework for Visuomotor Language Grounding} }, booktitle = {Embodied AI Workshop at CVPR}, year = {2021} } -
Deriving Behavioral Tests from Common Sense Knowledge Graphs
Although NLP models have demonstrated “superhuman” performance on common sense reasoning tasks, it is unclear whether these models truly have common sense knowledge. Constructing evaluation datasets to test this knowledge is expensive due to the manual effort involved, and is also limited in scope. Meanwhile, common sense knowledge graphs (CSKGs) aim for a wide coverage of structured common sense knowledge, but can not be directly used for testing purposes. In this work, we introduce a semi-automated approach that leverages CSKGs to construct out-of-domain evaluation sets for NLP tasks that are more scalable than purely manual approaches. Using this procedure, we create test cases from two popular CSKGs—ConceptNet and ATOMIC—to test the common sense reasoning capability of models trained for natural language inference (NLI) and question answering (QA). These tests reveal interesting differences in failure modes of these models; models trained on NLI tend to perform better on tests of ontological knowledge, e.g. ’is a’ and ’used for’ relations, failing on tests that require understanding ’desires’, ’needs’, and ’wants’, while QA models perform better on tests that involve ’wants’, and ’desires’.
@inproceedings{cskgtests:cskg21, author = {Yasaman Razeghi and Robert L. Logan IV and Sameer Singh}, title = { {Deriving Behavioral Tests from Common Sense Knowledge Graphs} }, booktitle = {AAAI Workshop on Common Sense Knowledge Graphs (CSKGs)}, year = {2021} } -
What Models Know About Their Attackers: Deriving Attacker Information From Latent Representations
Adversarial attacks curated against NLP models are increasingly becoming practical threats. Although various methods have been developed to detect adversarial attacks, securing learning-based NLP systems in practice would require more than identifying and evading perturbed instances. To address these issues, we propose a new set of adversary identification tasks, Attacker Attribute Classification via Textual Analysis (AACTA), that attempts to obtain more detailed information about the attackers from adversarial texts. Specifically, given a piece of adversarial text, we hope to accomplish tasks such as localizing perturbed tokens, identifying the attacker’s access level to the target model, determining the evasion mechanism imposed, and specifying the perturbation type employed by the attacking algorithm. Our contributions are as follows: we formalize the task of classifying attacker attributes, and create a benchmark on various target models from sentiment classification and abuse detection domains. We show that signals from BERT models and target models can be used to train classifiers that reveal the properties of the attacking algorithms. We demonstrate that adversarial attacks leave interpretable traces in both feature spaces of pre-trained language models and target models, making AACTA a promising direction towards more trustworthy NLP systems.
@inproceedings{advdetect:bbox21, author = {Zhouhang Xie and Jonathan Brophy and Adam Noack and Wencong You and Kalyani Asthana and Carter Perkins and Sabrina Reis and Zayd Hammoudeh and Daniel Lowd and Sameer Singh}, title = { {What Models Know About Their Attackers: Deriving Attacker Information From Latent Representations} }, booktitle = {EMNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackBoxNLP)}, year = {2021} }
-
AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts
The remarkable success of pretrained language models has motivated the study of what kinds of knowledge these models learn during pretraining. Reformulating tasks as fill-in-the-blanks problems (e.g., cloze tests) is a natural approach for gauging such knowledge, however, its usage is limited by the manual effort and guesswork required to write suitable prompts. To address this, we develop AutoPrompt, an automated method to create prompts for a diverse set of tasks, based on a gradient-guided search. Using AutoPrompt, we show that masked language models (MLMs) have an inherent capability to perform sentiment analysis and natural language inference without additional parameters or finetuning, sometimes achieving performance on par with recent state-of-the-art supervised models. We also show that our prompts elicit more accurate factual knowledge from MLMs than the manually created prompts on the LAMA benchmark, and that MLMs can be used as relation extractors more effectively than supervised relation extraction models. These results demonstrate that automatically generated prompts are a viable parameter-free alternative to existing probing methods, and as pretrained LMs become more sophisticated and capable, potentially a replacement for finetuning.
@inproceedings{autoprompt:emnlp20, author = {Taylor Shin and Yasaman Razeghi and Robert L. Logan IV and Eric Wallace and Sameer Singh}, title = { {AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts } }, booktitle = {Empirical Methods in Natural Language Processing (EMNLP)}, pages = {4222–4235}, year = {2020} } -
MOCHA: A Dataset for Training and Evaluating Generative Reading Comprehension Metrics
Posing reading comprehension as a generation problem provides a great deal of flexibility, allowing for open-ended questions with few restrictions on possible answers. However, progress is impeded by existing generation metrics, which rely on token overlap and are agnostic to the nuances of reading comprehension. To address this, we introduce a benchmark for training and evaluating generative reading comprehension metrics: MOdeling Correctness with Human Annotations. MOCHA contains 40K human judgement scores on model outputs from 6 diverse question answering datasets and an additional set of minimal pairs for evaluation. Using MOCHA, we train a Learned Evaluation metric for Reading Comprehension, LERC, to mimic human judgement scores. LERC outperforms baseline metrics by 10 to 36 absolute Pearson points on held-out annotations. When we evaluate robustness on minimal pairs, LERC achieves 80% accuracy, outperforming baselines by 14 to 26 absolute percentage points while leaving significant room for improvement. MOCHA presents a challenging problem for developing accurate and robust generative reading comprehension metrics.
@inproceedings{mocha:emnlp20, author = {Anthony Chen and Gabriel Stanovsky and Sameer Singh and Matt Gardner}, title = { {MOCHA: A Dataset for Training and Evaluating Generative Reading Comprehension Metrics} }, booktitle = {Empirical Methods in Natural Language Processing (EMNLP)}, pages = {6521–6532}, year = {2020} } -
Gradient-based Analysis of NLP Models is Manipulable
@inproceedings{facade:femnlp20, author = {Junlin Wang and Jens Tuyls and Eric Wallace and Sameer Singh}, title = { {Gradient-based Analysis of NLP Models is Manipulable} }, booktitle = {Findings of the Association for Computational Linguistics: EMNLP (EMNLP Findings)}, pages = {247–258}, year = {2020} } -
Evaluating Models’ Local Decision Boundaries via Contrast Sets
@inproceedings{contrast:femnlp20, author = {Matt Gardner and Yoav Artzi and Victoria Basmov and Jonathan Berant and Ben Bogin and Sihao Chen and Pradeep Dasigi and Dheeru Dua and Yanai Elazar and Ananth Gottumukkala and Nitish Gupta and Hannaneh Hajishirzi and Gabriel Ilharco and Daniel Khashabi and Kevin Lin and Jiangming Liu and Nelson F. Liu and Phoebe Mulcaire and Qiang Ning and Sameer Singh and Noah A. Smith and Sanjay Subramanian and Reut Tsarfaty and Eric Wallace and Ally Zhang and Ben Zhou}, title = { {Evaluating Models’ Local Decision Boundaries via Contrast Sets} }, booktitle = {Findings of the Association for Computational Linguistics: EMNLP (EMNLP Findings)}, pages = {1307–1323}, year = {2020} } -
MedICaT: A Dataset of Medical Images, Captions, and Textual References
@inproceedings{medicat:femnlp20, author = {Sanjay Subramanian and Lucy Lu Wang and Ben Bogin and Sachin Mehta and Madeleine van Zuylen and Sravanthi Parasa and Sameer Singh and Matt Gardner and Hannaneh Hajishirzi}, title = { {MedICaT: A Dataset of Medical Images, Captions, and Textual References} }, booktitle = {Findings of the Association for Computational Linguistics: EMNLP (EMNLP Findings)}, pages = {2112–2120}, year = {2020} } -
Beyond Accuracy: Behavioral Testing of NLP models with CheckList
Best Paper AwardAlthough measuring held-out accuracy has been the primary approach to evaluate generalization, it often overestimates the performance of NLP models, while alternative approaches for evaluating models either focus on individual tasks or on specific behaviors. Inspired by principles of behavioral testing in software engineering, we introduce CheckList, a task-agnostic methodology for testing NLP models. CheckList includes a matrix of general linguistic capabilities and test types that facilitate comprehensive test ideation, as well as a software tool to generate a large and diverse number of test cases quickly. We illustrate the utility of CheckList with tests for three tasks, identifying critical failures in both commercial and state-of-art models. In a user study, a team responsible for a commercial sentiment analysis model found new and actionable bugs in an extensively tested model. In another user study, NLP practitioners with CheckList created twice as many tests, and found almost three times as many bugs as users without it.
@inproceedings{checklist:acl20, author = {Marco Tulio Ribeiro and Tongshuang Wu and Carlos Guestrin and Sameer Singh}, title = { {Beyond Accuracy: Behavioral Testing of NLP models with CheckList} }, booktitle = {Association for Computational Linguistics (ACL)}, pages = {4902-4912}, year = {2020} } -
On Importance Sampling-Based Evaluation of Latent Language Models
Language models that use additional latent structures (e.g., syntax trees, coreference chains, knowledge graph links) provide several advantages over traditional language models. However, likelihood-based evaluation of these models is often intractable as it requires marginalizing over the latent space. Existing works avoid this issue by using importance sampling. Although this approach has asymptotic guarantees, analysis is rarely conducted on the effect of decisions such as sample size and choice of proposal distribution on the reported estimates. In this paper, we carry out this analysis for three models: RNNG, EntityNLM, and KGLM. In addition, we elucidate subtle differences in how importance sampling is applied in these works that can have substantial effects on the final estimates, as well as provide theoretical results which reinforce the validity of this technique.
@inproceedings{impsample:acl20, author = {Robert L. Logan IV and Matt Gardner and Sameer Singh}, title = { {On Importance Sampling-Based Evaluation of Latent Language Models} }, booktitle = {Association for Computational Linguistics (ACL)}, pages = {2171-2176}, year = {2020} } -
Obtaining Faithful Interpretations from Compositional Neural Networks
Neural module networks (NMNs) are a popular approach for modeling compositionality: they achieve high accuracy when applied to problems in language and vision, while reflecting the compositional structure of the problem in the network architecture. However, prior work implicitly assumed that the structure of the network modules, describing the abstract reasoning process, provides a faithful explanation of the model’s reasoning; that is, that all modules perform their intended behaviour. In this work, we propose and conduct a systematic evaluation of the intermediate outputs of NMNs on NLVR2 and DROP, two datasets which require composing multiple reasoning steps. We find that the intermediate outputs differ from the expected output, illustrating that the network structure does not provide a faithful explanation of model behaviour. To remedy that, we train the model with auxiliary supervision and propose particular choices for module architecture that yield much better faithfulness, at a minimal cost to accuracy.
@inproceedings{nmninterpret:acl20, author = {Sanjay Subramanian and Ben Bogin and Nitish Gupta and Tomer Wolfson and Sameer Singh and Jonathan Berant and Matt Gardner}, title = { {Obtaining Faithful Interpretations from Compositional Neural Networks} }, booktitle = {Association for Computational Linguistics (ACL)}, pages = {5594-5608}, year = {2020} } -
Benefits of Intermediate Annotations in Reading Comprehension
Complex compositional reading comprehension datasets require performing latent sequential decisions that are learned via supervision from the final answer. A large combinatorial space of possible decision paths that result in the same answer, compounded by the lack of intermediate supervision to help choose the right path, makes the learning particularly hard for this task. In this work, we study the benefits of collecting intermediate reasoning supervision along with the answer during data collection. We find that these intermediate annotations can provide two-fold benefits. First, we observe that for any collection budget, spending a fraction of it on intermediate annotations results in improved model performance, for two complex compositional datasets: DROP and Quoref. Second, these annotations encourage the model to learn the correct latent reasoning steps, helping combat some of the biases introduced during the data collection process.
@inproceedings{intannot:acl20, author = {Dheeru Dua and Sameer Singh and Matt Gardner}, title = { {Benefits of Intermediate Annotations in Reading Comprehension} }, booktitle = {Association for Computational Linguistics (ACL)}, pages = {5627-5634}, year = {2020} } -
Dynamic Sampling Strategies for Multi-Task Reading Comprehension
Building general reading comprehension systems, capable of solving multiple datasets at the same time, is a recent aspirational goal in the research community. Prior work has focused on model architecture or generalization to held out datasets, and largely passed over the particulars of the multi-task learning set up. We show that a simple dynamic sampling strategy, selecting instances for training proportional to the multi-task model’s current performance on a dataset relative to its single task performance, gives substantive gains over prior multi-task sampling strategies, mitigating the catastrophic forgetting that is common in multi-task learning. We also demonstrate that allowing instances of different tasks to be interleaved as much as possible between each epoch and batch has a clear benefit in multitask performance over forcing task homogeneity at the epoch or batch level. Our final model shows greatly increased performance over the best model on ORB, a recently-released multitask reading comprehension benchmark.
@inproceedings{dynsample:acl20, author = {Ananth Gottumukkala and Dheeru Dua and Sameer Singh and Matt Gardner}, title = { {Dynamic Sampling Strategies for Multi-Task Reading Comprehension} }, booktitle = {Association for Computational Linguistics (ACL)}, pages = {920-924}, year = {2020} } -
Revisiting Evaluation of Knowledge Base Completion Models
Runner-up for Best Paper AwardRepresenting knowledge graphs (KGs) by learning embeddings for entities and relations has led to accurate models for existing KG completion benchmarks. However, due to the open-world assumption of existing KGs, evaluation of KG completion uses ranking metrics and triple classification with negative samples, and is thus unable to directly assess models on the goals of the task: completion. In this paper, we first study the shortcomings of these evaluation metrics. Specifically, we demonstrate that these metrics (1) are unreliable for estimating how calibrated the models are, (2) make strong assumptions that are often violated, and 3) do not sufficiently, and consistently, differentiate embedding methods from each other, or from simpler approaches. To address these issues, we gather a semi-complete KG referred as YAGO3-TC, using a random subgraph from the test and validation data of YAGO3-10, which enables us to compute accurate triple classification accuracy on this data. Conducting thorough experiments on existing models, we provide new insights and directions for the KG completion research. Along with the dataset and the open source implementation of the models, we also provide a leaderboard for knowledge graph completion that consists of a hidden, and growing, test set, available at https://pouyapez.github.io/yago3-tc/.
@inproceedings{kbeval:akbc20, author = {Pouya Pezeshkpour and Yifan Tian and Sameer Singh}, title = { {Revisiting Evaluation of Knowledge Base Completion Models} }, booktitle = {Automated Knowledge Base Construction (AKBC)}, year = {2020} } -
Building a Better Lie Detector with BERT: The Difference Between Truth and Lies
@inproceedings{bertdecept:ijcnn20, author = {Dan Barsever and Sameer Singh and Emre Neftci}, title = { {Building a Better Lie Detector with BERT: The Difference Between Truth and Lies} }, booktitle = {International Joint Conference on Neural Networks (IJCNN)}, year = {2020} } -
Neural Module Networks for Reasoning over Text
Answering compositional questions that require multiple steps of reasoning against text is challenging, especially when they involve discrete, symbolic operations. Neural module networks (NMNs) learn to parse such questions as executable programs composed of learnable modules, performing well on synthetic visual QA domains. However, we find that it is challenging to learn these models for non-synthetic questions on open-domain text, where a model needs to deal with the diversity of natural language and perform a broader range of reasoning. We extend NMNs by: (a) introducing modules that reason over a paragraph of text, performing symbolic reasoning (such as arithmetic, sorting, counting) over numbers and dates in a probabilistic and differentiable manner; and (b) proposing an unsupervised auxiliary loss to help extract arguments associated with the events in text. Additionally, we show that a limited amount of heuristically-obtained question program and intermediate module output supervision provides sufficient inductive bias for accurate learning. Our proposed model significantly outperforms state-of-the-art models on a subset of the DROP dataset that poses a variety of reasoning challenges that are covered by our modules.
@inproceedings{nmn:iclr20, author = {Nitish Gupta and Kevin Lin and Dan Roth and Sameer Singh and Matt Gardner}, title = { {Neural Module Networks for Reasoning over Text} }, booktitle = {International Conference on Learning Representations (ICLR)}, year = {2020} } -
Explain Your Move: Understanding Agent Actions Using Specific and Relevant Feature Attribution
As deep reinforcement learning (RL) is applied to more tasks, there is a need to visualize and understand the behavior of learned agents. Saliency maps explain agent behavior by highlighting the features of the input state that are most relevant for the agent in taking an action. Existing perturbation-based approaches to compute saliency often highlight regions of the input that are not relevant to the action taken by the agent. Our proposed approach, SARFA (Specific and Relevant Feature Attribution), generates more focused saliency maps by balancing two aspects (specificity and relevance) that capture different desiderata of saliency. The first captures the impact of perturbation on the relative expected reward of the action to be explained. The second downweighs irrelevant features that alter the relative expected rewards of actions other than the action to be explained. We compare SARFA with existing approaches on agents trained to play board games (Chess and Go) and Atari games (Breakout, Pong and Space Invaders). We show through illustrative examples (Chess, Atari, Go), human studies (Chess), and automated evaluation methods (Chess) that SARFA generates saliency maps that are more interpretable for humans than existing approaches. For the code release and demo videos, see: https://nikaashpuri.github.io/sarfa-saliency/.
@inproceedings{salrl:iclr20, author = {Piyush Gupta and Nikaash Puri and Sukriti Verma and Dhruv Kayastha and Shripad Deshmukh and Balaji Krishnamurthy and Sameer Singh}, title = { {Explain Your Move: Understanding Agent Actions Using Specific and Relevant Feature Attribution} }, booktitle = {International Conference on Learning Representations (ICLR)}, year = {2020} } -
Minecraft as a Platform for Project-Based Learning in AI
Undergraduate courses that focus on open-ended, projectbased learning teach students how to define concrete goals, transfer conceptual understanding of algorithms to code, and evaluate/analyze/present their solution. However, AI, along with machine learning, is getting increasingly varied in terms of both the approaches and applications, making it challenging to design project courses that span a sufficiently wide spectrum of AI. For these reasons, existing AI project courses are restricted to a narrow set of approaches (e.g. only reinforcement learning) or applications (e.g. only computer vision).
In this paper, we propose to use Minecraft as the platform for teaching AI via project-based learning. Minecraft is an open-world sandbox game with elements of exploration, resource gathering, crafting, construction, and combat, and is supported by the Malmo library that provides a programmatic interface to the player observations and actions at various levels of granularity. In Minecraft, students can design projects to use approaches like search-based AI, reinforcement learning, supervised learning, and constraint satisfaction, on data types like text, audio, images, and tabular data. We describe our experience with an open-ended, undergraduate AI projects course using Minecraft that includes 82 different projects, covering themes that ranged from navigation, instruction following, object detection, combat, and music/image generation.@inproceedings{malmo:eaai20, author = {Sameer Singh}, title = { {Minecraft as a Platform for Project-Based Learning in AI} }, booktitle = {AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI)}, doi = {10.1609/aaai.v34i09.7070}, pages = {13504-13505}, year = {2020} } -
Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methods
As machine learning black boxes are increasingly being deployed in domains such as healthcare and criminal justice, there is growing emphasis on building tools and techniques for explaining these black boxes in an interpretable manner. Such explanations are being leveraged by domain experts to diagnose systematic errors and underlying biases of black boxes. In this paper, we demonstrate that post hoc explanations techniques that rely on input perturbations, such as LIME and SHAP, are not reliable. Specifically, we propose a novel scaffolding technique that effectively hides the biases of any given classifier by allowing an adversarial entity to craft an arbitrary desired explanation. Our approach can be used to scaffold any biased classifier in such a way that its predictions on the input data distribution still remain biased, but the post hoc explanations of the scaffolded classifier look innocuous. Using extensive evaluation with multiple real-world datasets (including COMPAS), we demonstrate how extremely biased (racist) classifiers crafted by our framework can easily fool popular explanation techniques such as LIME and SHAP into generating innocuous explanations which do not reflect the underlying biases.
@inproceedings{advlime:aies20, author = {Dylan Slack and Sophie Hilgard and Emily Jia and Sameer Singh and Himabindu Lakkaraju}, title = { {Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methods} }, booktitle = {AAAI/ACM Conference on AI, Ethics, and Society (AIES)}, doi = {10.1145/3375627.3375830}, pages = {180-186}, year = {2020} } -
Head Network Distillation: Splitting Distilled Deep Neural Networks for Resource-Constrained Edge Computing Systems
@article{headnet:ieee20, author = {Yoshitomo Matsubara and Davide Callegaro and Sabur Baidya and Marco Levorato and Sameer Singh}, title = { {Head Network Distillation: Splitting Distilled Deep Neural Networks for Resource-Constrained Edge Computing Systems} }, journal = {IEEE Access}, volume = {126}, number = {4}, doi = {10.1109/ACCESS.2020.3039714}, year = {2020} } -
On the Utility of Active Instance Selection for Few-Shot Learning
@inproceedings{activefew:hamlets20, author = {Pouya Pezeshkpour and Zhengli Zhao and Sameer Singh}, title = { {On the Utility of Active Instance Selection for Few-Shot Learning} }, booktitle = {NeurIPS Workshop on Human And Model in the Loop Evaluation and Training Strategies (HAMLETS)}, year = {2020} } -
COVIDLies: Detecting COVID-19 Misinformation on Social Media
Best Paper AwardThe ongoing pandemic has heightened the need for developing tools to flag COVID-19-related misinformation on the internet, specifically on social media such as Twitter. However, due to novel language and the rapid change of information, existing misinformation detection datasets are not effective for evaluating systems designed to detect misinformation on this topic. Misinformation detection can be divided into two sub-tasks: (i) retrieval of misconceptions relevant to posts being checked for veracity, and (ii) stance detection to identify whether the posts Agree, Disagree, or express No Stance towards the retrieved misconceptions. To facilitate research on this task, we release COVIDLies (https://ucinlp.github.io/covid19), a dataset of 6761 expert-annotated tweets to evaluate the performance of misinformation detection systems on 86 different pieces of COVID-19 related misinformation. We evaluate existing NLP systems on this dataset, providing initial benchmarks and identifying key challenges for future models to improve upon.
@inproceedings{covidlies:nlpcovid20, author = {Tamanna Hossain and Robert L. Logan IV and Arjuna Ugarte and Yoshitomo Matsubara and Sean Young and Sameer Singh}, title = { {COVIDLies: Detecting COVID-19 Misinformation on Social Media} }, booktitle = {EMNLP NLP Covid19 Workshop}, doi = {10.18653/v1/2020.nlpcovid19-2.11}, year = {2020} } -
Tweeki: Linking Named Entities on Twitter to a Knowledge Graph
To identify what entities are being talked about in tweets, we need to automatically link named entities that appear in tweets to structured KBs like WikiData. Existing approaches often struggle with such short, noisy texts, or their complex design and reliance on supervision make them brittle, difficult to use and maintain, and lose significance over time. Further, there is a lack of a large, linked corpus of tweets to aid researchers, along with lack of gold dataset to evaluate the accuracy of entity linking. In this paper, we introduce (1) Tweeki, an unsupervised, modular entity linking system for Twitter, (2) TweekiData, a large, automatically-annotated corpus of Tweets linked to entities in WikiData, and (3) TweekiGold, a gold dataset for entity linking evaluation. Through comprehensive analysis, we show that Tweeki is comparable to the performance of recent state-of-the-art entity linkers models, the dataset is of high quality, and a use case of how the dataset can be used to improve downstream tasks in social media analysis (geolocation prediction).
@inproceedings{tweeki:wnut20, author = {Bahareh Harandizadeh and Sameer Singh}, title = { {Tweeki: Linking Named Entities on Twitter to a Knowledge Graph} }, booktitle = {EMNLP Workshop on Noisy, User-generated Text (W-NUT)}, doi = {10.18653/v1/2020.wnut-1.29}, year = {2020} } -
Citations Beyond Self Citations: Identifying Authors, Affiliations, and Nationalities in Scientific Papers
The question of the utility of the blind peer-review system is fundamental to scientific research. Some studies investigate exactly how “blind” the papers are in the double-blind review system by manually or automatically identifying the true authors, mainly suggesting the number of self-citations in the submitted manuscripts as the primary signal for identity. However, related work on the automated approaches are limited by the sizes of their datasets and the restricted experimental setup, thus they lack practical insights into the blind review process. In this work, we train models that identify the authors, their affiliations, and their nationalities through real-world, large-scale experiments on the Microsoft Academic Graph, including the cold start scenario. Our models are accurate; we identify at least one of authors, affiliations, and nationalities of held-out papers with 40.3%, 47.9% and 86.0% accuracy respectively, from the top-10 guesses of our models. However, through insights from the model, we demonstrate that these entities are identifiable with a small number of guesses primarily by using a combination of self-citations, social, and common citations. Moreover, our further analysis on the results leads to interesting findings, such as that prominent affiliations are easily identifiable (e.g. 93.8% of test papers written by Microsoft are identified with top-10 guesses). The experimental results show, against conventional belief, that the self-citations are no more informative than looking at the common citations, thus suggesting that removing self-citations is not sufficient for authors to maintain their anonymity.
@inproceedings{deblind:wosp20, author = {Yoshitomo Matsubara and Sameer Singh}, title = { {Citations Beyond Self Citations: Identifying Authors, Affiliations, and Nationalities in Scientific Papers} }, booktitle = {Workshop on Mining Scientific Publications (WOSP)}, year = {2020} } -
Data Importance-Based Active Learning for Limited Labels
@inproceedings{ibal:vl320, author = {Pouya Pezeshkpour and Zhengli Zhao and Sameer Singh}, title = { {Data Importance-Based Active Learning for Limited Labels} }, booktitle = {CVPR Workshop on Visual Learning with Limited Labels (VL3)}, year = {2020} }
-
Universal Adversarial Triggers for Attacking and Analyzing NLP
Adversarial examples highlight model vulnerabilities and are useful for evaluation and interpretation. We define universal adversarial triggers: input-agnostic sequences of tokens that trigger a model to produce a specific prediction when concatenated to any input from a dataset. We propose a gradient-guided search over tokens which finds short trigger sequences (e.g., one word for classification and four words for language modeling) that successfully trigger the target prediction. For example, triggers cause SNLI entailment accuracy to drop from 89.94% to 0.55%, 72% of “why” questions in SQuAD to be answered “to kill american people”, and the GPT-2 language model to spew racist output even when conditioned on non-racial contexts. Furthermore, although the triggers are optimized using white-box access to a specific model, they transfer to other models for all tasks we consider. Finally, since triggers are input-agnostic, they provide an analysis of global model behavior. For instance, they confirm that SNLI models exploit dataset biases and help to diagnose heuristics learned by reading comprehension models.
@inproceedings{trigger:emnlp19, author = {Eric Wallace and Shi Feng and Nikhil Kandpal and Matt Gardner and Sameer Singh}, title = { {Universal Adversarial Triggers for Attacking and Analyzing NLP} }, booktitle = {Empirical Methods in Natural Language Processing (EMNLP)}, doi = {10.18653/v1/D19-1221}, pages = {2153-2162}, year = {2019} } -
Do NLP Models Know Numbers? Probing Numeracy in Embeddings
@inproceedings{numeracy:emnlp19, author = {Eric Wallace and Yizhong Wang and Sujian Li and Sameer Singh and Matt Gardner}, title = { {Do NLP Models Know Numbers? Probing Numeracy in Embeddings} }, booktitle = {Empirical Methods in Natural Language Processing (EMNLP)}, doi = {10.18653/v1/D19-1534}, pages = {5307-5315}, year = {2019} } -
Knowledge Enhanced Contextual Word Representations
@inproceedings{knobert:emnlp19, author = {Matthew E. Peters and Mark Neumann and Robert L. Logan IV and Roy Schwartz and Vidur Joshi and Sameer Singh and Noah A. Smith}, title = { {Knowledge Enhanced Contextual Word Representations} }, booktitle = {Empirical Methods in Natural Language Processing (EMNLP)}, doi = {10.18653/v1/D19-1005}, pages = {43-54}, year = {2019} } -
Barack's Wife Hillary: Using Knowledge Graphs for Fact-Aware Language Modeling
Modeling human language requires the ability to not only generate fluent text but also encode factual knowledge. However, traditional language models are only capable of remembering facts seen at training time, and often have difficulty recalling them. To address this, we introduce the knowledge graph language model (KGLM), a neural language model with mechanisms for selecting and copying facts from a knowledge graph that are relevant to the context. These mechanisms enable the model to render information it has never seen before, as well as generate out-of-vocabulary tokens. We also introduce the Linked WikiText-2 dataset, a corpus of annotated text aligned to the Wikidata knowledge graph whose contents (roughly) match the popular WikiText-2 benchmark. In experiments, we demonstrate that the KGLM achieves significantly better performance than a strong baseline language model. We additionally compare different language model’s ability to complete sentences requiring factual knowledge, showing that the KGLM outperforms even very large language models in generating facts.
@inproceedings{kglm:acl19, author = {Robert L. Logan IV and Nelson F. Liu and Matthew E. Peters and Matt Gardner and Sameer Singh}, title = { {Barack's Wife Hillary: Using Knowledge Graphs for Fact-Aware Language Modeling} }, booktitle = {Association for Computational Linguistics (ACL)}, doi = {10.18653/v1/P19-1598}, pages = {5962-5971}, year = {2019} } -
Are Red Roses Red? Evaluating Consistency of Question-Answering Models
@inproceedings{impl:acl19, author = {Marco Tulio Ribeiro and Carlos Guestrin and Sameer Singh}, title = { {Are Red Roses Red? Evaluating Consistency of Question-Answering Models} }, booktitle = {Association for Computational Linguistics (ACL)}, doi = {10.18653/v1/P19-1621}, pages = {6174-6184}, year = {2019} } -
Compositional Questions Do Not Necessitate Multi-hop Reasoning
@inproceedings{mhop:acl19, author = {Sewon Min and Eric Wallace and Sameer Singh and Matt Gardner and Hannaneh Hajishirzi and Luke Zettlemoyer}, title = { {Compositional Questions Do Not Necessitate Multi-hop Reasoning} }, booktitle = {Association for Computational Linguistics (ACL)}, doi = {10.18653/v1/P19-1416}, pages = {4249-4257}, year = {2019} } -
DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs
Reading comprehension has recently seen rapid progress, with systems matching humans on the most popular datasets for the task. However, a large body of work has highlighted the brittleness of these systems, showing that there is much work left to be done. We introduce a new reading comprehension benchmark, DROP, which requires Discrete Reasoning Over the content of Paragraphs. In this crowdsourced, adversarially-created, 55k-question benchmark, a system must resolve references in a question, perhaps to multiple input positions, and perform discrete operations over them (such as addition, counting, or sorting). These operations require a much more comprehensive understanding of the content of paragraphs, as they remove the paraphrase-and-entity-typing shortcuts available in prior datasets. We apply state-of-the-art methods from both the reading comprehension and semantic parsing literatures on this dataset and show that the best systems only achieve 38.4% F1 on our generalized accuracy metric, while expert human performance is 96%. We additionally present a new model that combines reading comprehension methods with simple numerical reasoning to achieve 51% F1.
@inproceedings{drop:naacl19, author = {Dheeru Dua and Yizhong Wang and Pradeep Dasigi and Gabriel Stanovsky and Sameer Singh and Matt Gardner}, title = { {DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs} }, booktitle = {Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)}, doi = {10.18653/v1/N19-1246}, pages = {2368-2378}, year = {2019} } -
Investigating Robustness and Interpretability of Link Prediction via Adversarial Modifications
Representing entities and relations in an embedding space is a well-studied approach for machine learning on relational data. Existing approaches, however, primarily focus on improving accuracy and overlook other aspects such as robustness and interpretability. In this paper, we propose adversarial modifications for link prediction models: identifying the fact to add into or remove from the knowledge graph that changes the prediction for a target fact after the model is retrained. Using these single modifications of the graph, we identify the most influential fact for a predicted link and evaluate the sensitivity of the model to the addition of fake facts. We introduce an efficient approach to estimate the effect of such modifications by approximating the change in the embeddings when the knowledge graph changes. To avoid the combinatorial search over all possible facts, we train a network to decode embeddings to their corresponding graph components, allowing the use of gradient-based optimization to identify the adversarial modification. We use these techniques to evaluate the robustness of link prediction models (by measuring sensitivity to additional facts), study interpretability through the facts most responsible for predictions (by identifying the most influential neighbors), and detect incorrect facts in the knowledge base.
@inproceedings{criage:naacl19, author = {Pouya Pezeshkpour and Yifan Tian and Sameer Singh}, title = { {Investigating Robustness and Interpretability of Link Prediction via Adversarial Modifications} }, booktitle = {Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)}, doi = {10.18653/v1/N19-1337}, pages = {3336-3347}, year = {2019} } -
GenderQuant: Quantifying Mention-Level Genderedness
Language is gendered if the context surrounding a mention is suggestive of a particular binary gender for that mention. Detecting the different ways in which language is gendered is an important task since gendered language can bias NLP models (such as for coreference resolution). This task is challenging since genderedness is often expressed in subtle ways. Existing approaches need considerable annotation efforts for each language, domain, and author, and often require handcrafted lexicons and features. Additionally, these approaches do not provide a quantifiable measure of how gendered the text is, nor are they applicable at the fine-grained mention level.
In this paper, we use existing NLP pipelines to automatically annotate gender of mentions in the text. On corpora labeled using this method, we train a supervised classifier to predict the gender of any mention from its context and evaluate it on unseen text. The model confidence for a mention's gender can be used as a proxy to indicate the level of genderedness of the context. We test this gendered language detector on movie summaries, movie reviews, news articles, and fiction novels, achieving an AUC-ROC of up to 0.71, and observe that the model predictions agree with human judgments collected for this task. We also provide examples of detected gendered sentences from aforementioned domains.@inproceedings{gender:naacl19, author = {Ananya Ananya and Nitya Parthasarthi and Sameer Singh}, title = { {GenderQuant: Quantifying Mention-Level Genderedness} }, booktitle = {Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)}, doi = {10.18653/v1/N19-1303}, pages = {2959-2969}, year = {2019} } -
PoMo: Generating Entity-Specific Post-Modifiers in Context
We introduce entity post-modifier generation as an instance of a collaborative writing task. Given a sentence about a target entity, the task is to automatically generate a post-modifier phrase that provides contextually relevant information about the entity. For example, for the sentence, "Barack Obama, _______, supported the #MeToo movement.", the phrase "a father of two girls" is a contextually relevant post-modifier. To this end, we build PoMo, a post-modifier dataset created automatically from news articles reflecting a journalistic need for incorporating entity information that is relevant to a particular news event. PoMo consists of more than 231K sentences with post-modifiers and associated facts extracted from Wikidata for around 57K unique entities. We use crowdsourcing to show that modeling contextual relevance is necessary for accurate post-modifier generation.
We adapt a number of existing generation approaches as baselines for this dataset. Our results show there is large room for improvement in terms of both identifying relevant facts to include (knowing which claims are relevant gives a >20% improvement in BLEU score), and generating appropriate post-modifier text for the context (providing relevant claims is not sufficient for accurate generation). We conduct an error analysis that suggests promising directions for future research.@inproceedings{pomo:naacl19, author = {Jun Seok Kang and Robert L. Logan IV and Zewei Chu and Yang Chen and Dheeru Dua and Kevin Gimpel and Sameer Singh and Niranjan Balasubramanian}, title = { {PoMo: Generating Entity-Specific Post-Modifiers in Context} }, booktitle = {Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)}, doi = {10.18653/v1/N19-1089}, pages = {826-838}, year = {2019} } -
AllenNLP Interpret: A Framework for Explaining Predictions of NLP Models
Best Demonstration Paper Award.Neural NLP models are increasingly accurate but are imperfect and opaque---they break in counterintuitive ways and leave end users puzzled at their behavior. Model interpretation methods ameliorate this opacity by providing explanations for specific model predictions. Unfortunately, existing interpretation codebases make it difficult to apply these methods to new models and tasks, which hinders adoption for practitioners and burdens interpretability researchers. We introduce AllenNLP Interpret, a flexible framework for interpreting NLP models. The toolkit provides interpretation primitives (e.g., input gradients) for any AllenNLP model and task, a suite of built-in interpretation methods, and a library of front-end visualization components. We demonstrate the toolkit's flexibility and utility by implementing live demos for five interpretation methods (e.g., saliency maps and adversarial attacks) on a variety of models and tasks (e.g., masked language modeling using BERT and reading comprehension using BiDAF). These demos, alongside our code and tutorials, are available at https://allennlp.org/interpret.
@inproceedings{interpret:emnlp19, author = {Eric Wallace and Jens Tuyls and Junlin Wang and Sanjay Subramanian and Matt Gardner and Sameer Singh}, title = { {AllenNLP Interpret: A Framework for Explaining Predictions of NLP Models} }, booktitle = {Demo at the Empirical Methods in Natural Language Processing (EMNLP)}, doi = {10.18653/v1/D19-3002}, pages = {7-12}, year = {2019} } -
Detecting Conversation Topics in Primary Care Office Visits from Transcripts of Patient-Provider Interactions
@article{convtopics:jamia19, author = {Jihyun Park and Dimitrios Kotzias and Patty Kuo and Robert L. Logan IV and Kritzia Merced and Sameer Singh and Michael Tanana and Efi Karra-Taniskidou and Jennifer Elston Lafata and David C. Atkins and Ming Tai-Seale and Zac E Imel and Padhraic Smyth}, title = { {Detecting Conversation Topics in Primary Care Office Visits from Transcripts of Patient-Provider Interactions} }, journal = {Journal of the American Medical Informatics Association}, volume = {26}, number = {12}, doi = {10.1093/jamia/ocz140}, pages = {1493-1504}, year = {2019} } -
Comment on Semantic Based Adversarial Examples Fool Face Recognition
@misc{review:synced19, author = {Sameer Singh}, title = { {Comment on Semantic Based Adversarial Examples Fool Face Recognition} }, editor = {Synced Review}, month = {August}, url = {https://syncedreview.com/2019/08/09/semantic-based-adversarial-examples-fool-face-recognition/}, year = {2019} } -
Distilled Split Deep Neural Networks for Edge-Assisted Real-Time Systems
@inproceedings{distill:hottopics19, author = {Yoshitomo Matsubara and Sabur Baidya and Davide Callegaro and Marco Levorato and Sameer Singh}, title = { {Distilled Split Deep Neural Networks for Edge-Assisted Real-Time Systems} }, booktitle = {Mobicom Workshop on Hot Topics in Video Analytics and Intelligent Edges}, year = {2019} } -
Evaluating Question Answering Evaluation
Best Paper Award.
@inproceedings{evalqa:mrqa19, author = {Anthony Chen and Gabriel Stanovsky and Sameer Singh and Matt Gardner}, title = { {Evaluating Question Answering Evaluation} }, booktitle = {Workshop on Machine Reading and Question Answering (MRQA)}, year = {2019} } -
ORB: An Open Reading Benchmark for Comprehensive Evaluation of Machine Reading Comprehension
@inproceedings{orb:mrqa19, author = {Dheeru Dua and Ananth Gottumukkala and Alon Talmor and Sameer Singh and Matt Gardner}, title = { {ORB: An Open Reading Benchmark for Comprehensive Evaluation of Machine Reading Comprehension} }, booktitle = {Workshop on Machine Reading and Question Answering (MRQA)}, year = {2019} } -
Analyzing Compositionality of Visual Question Answering
@inproceedings{compvqa:vigil19, author = {Sanjay Subramanian and Sameer Singh and Matt Gardner}, title = { {Analyzing Compositionality of Visual Question Answering} }, booktitle = {NeurIPS Workshop on Visually Grounded Interaction and Language (ViGIL)}, year = {2019} } -
Improving Differentially Private Models with Active Learning
@inproceedings{dpal:priml19, author = {Zhengli Zhao and Nicolas Papernot and Sameer Singh and Neoklis Polyzotis and Augustus Odena}, title = { {Improving Differentially Private Models with Active Learning} }, booktitle = {NeurIPS Workshop on Privacy in Machine Learning (PriML)}, year = {2019} }
-
From Reinforcement Learning to Deep Reinforcement Learning: An Overview
@incollection{deeprl:chap18, author = {Forest Agostinelli and Guillaume Hocquet and Sameer Singh and Pierre Baldi}, title = { {From Reinforcement Learning to Deep Reinforcement Learning: An Overview} }, booktitle = {Braverman Readings in Machine Learning: Key Ideas from Inception to Current State, Springer Press}, pages = {298-328}, year = {2018} } -
Embedding Multimodal Relational Data for Knowledge Base Completion
Representing entities and relations in an embedding space is a well-studied approach for machine learning on relational data. Existing approaches, however, primarily focus on simple link structure between a finite set of entities, ignoring the variety of data types that are often used in knowledge bases, such as text, images, and numerical values. In this paper, we propose multimodal knowledge base embeddings (MKBE) that use different neural encoders for this variety of observed data, and combine them with existing relational models to learn embeddings of the entities and multimodal data. Further, using these learned embedings and different neural decoders, we introduce a novel multimodal imputation model to generate missing multimodal values, like text and images, from information in the knowledge base. We enrich existing relational datasets to create two novel benchmarks that contain additional information such as textual descriptions and images of the original entities. We demonstrate that our models utilize this additional information effectively to provide more accurate link prediction, achieving state-of-the-art results with a considerable gap of 5-7% over existing methods. Further, we evaluate the quality of our generated multimodal values via a user study.
@inproceedings{mmkb:emnlp18, author = {Pouya Pezeshkpour and Liyan Chen and Sameer Singh}, title = { {Embedding Multimodal Relational Data for Knowledge Base Completion} }, booktitle = {Empirical Methods in Natural Language Processing (EMNLP)}, doi = {10.18653/v1/D18-1359}, pages = {3208-3218}, year = {2018} } -
Interpretation of Natural Language Rules in Conversational Machine Reading
Most work in machine reading focuses on question answering problems where the answer is directly expressed in the text to read. However, many real-world question answering problems require the reading of text not because it contains the literal answer, but because it contains a recipe to derive an answer together with the reader's background knowledge. One example is the task of interpreting regulations to answer "Can I...?" or "Do I have to...?" questions such as "I am working in Canada. Do I have to carry on paying UK National Insurance?" after reading a UK government website about this topic. This task requires both the interpretation of rules and the application of background knowledge. It is further complicated due to the fact that, in practice, most questions are underspecified, and a human assistant will regularly have to ask clarification questions such as "How long have you been working abroad?" when the answer cannot be directly derived from the question and text. In this paper, we formalise this task and develop a crowd-sourcing strategy to collect 32k task instances based on real-world rules and crowd-generated questions and scenarios. We analyse the challenges of this task and assess its difficulty by evaluating the performance of rule-based and machine-learning baselines. We observe promising results when no background knowledge is necessary, and substantial room for improvement whenever background knowledge is needed.
@inproceedings{quarc:emnlp18, author = {Marzieh Saeidi and Max Bartolo and Patrick Lewis and Sameer Singh and Tim Rocktaschel and Mike Sheldon and Guillaume Bouchard and Sebastian Riedel}, title = { {Interpretation of Natural Language Rules in Conversational Machine Reading} }, booktitle = {Empirical Methods in Natural Language Processing (EMNLP)}, doi = {10.18653/v1/D18-1233}, pages = {2087-2097}, year = {2018} } -
Semantically Equivalent Adversarial Rules for Debugging NLP models
Honorable Mention for Best Paper.Complex machine learning models for NLP are often brittle, making different predictions for input instances that are extremely similar semantically. To automatically detect this behavior for individual instances, we present semantically equivalent adversaries (SEAs) - semantic-preserving perturbations that induce changes in the model’s predictions. We generalize these adversaries into semantically equivalent adversarial rules (SEARs) - simple, universal replacement rules that induce adversaries on many instances. We demonstrate the usefulness and flexibility of SEAs and SEARs by detecting bugs in black-box state-of-the-art models for three domains: machine comprehension, visual question-answering, and sentiment analysis. Via user studies, we demonstrate that we generate high-quality local adversaries for more instances than humans, and that SEARs induce four times as many mistakes as the bugs discovered by human experts. SEARs are also actionable: retraining models using data augmentation significantly reduces bugs, while maintaining accuracy.
@inproceedings{sears:acl18, author = {Marco Tulio Ribeiro and Sameer Singh and Carlos Guestrin}, title = { {Semantically Equivalent Adversarial Rules for Debugging NLP models} }, booktitle = {Association for Computational Linguistics (ACL)}, doi = {10.18653/v1/P18-1079}, pages = {856-865}, year = {2018} } -
Generating Natural Adversarial Examples
Due to their complex nature, it is hard to characterize the ways in which machine learning models can misbehave or be exploited when deployed. Recent work on adversarial examples, i.e. inputs with minor perturbations that result in substantially different model predictions, is helpful in evaluating the robustness of these models by exposing the adversarial scenarios where they fail. However, these malicious perturbations are often unnatural, not semantically meaningful, and not applicable to complicated domains such as language. In this paper, we propose a framework to generate natural and legible adversarial examples that lie on the data manifold, by searching in semantic space of dense and continuous data representation, utilizing the recent advances in generative adversarial networks. We present generated adversaries to demonstrate the potential of the proposed approach for black-box classifiers for a wide range of applications such as image classification, textual entailment, and machine translation. We include experiments to show that the generated adversaries are natural, legible to humans, and useful in evaluating and analyzing black-box classifiers.
@inproceedings{natadv:iclr18, author = {Zhengli Zhao and Dheeru Dua and Sameer Singh}, title = { {Generating Natural Adversarial Examples} }, booktitle = {International Conference on Learning Representations (ICLR)}, year = {2018} } -
Combining Symbolic Expressions and Black-box Function Evaluations for Training Neural Programs
Neural programming involves training neural networks to learn programs, mathematics, or logic from data. Previous works have failed to achieve good generalization performance, especially on problems and programs with high complexity or on large domains. This is because they mostly rely either on black-box function evaluations that do not capture the structure of the program, or on detailed execution traces that are expensive to obtain, and hence the training data has poor coverage of the domain under consideration. We present a novel framework that utilizes black-box function evaluations, in conjunction with symbolic expressions that define relationships between the given functions. We employ tree LSTMs to incorporate the structure of the symbolic expression trees. We use tree encoding for numbers present in function evaluation data, based on their decimal representation. We present an evaluation benchmark for this task to demonstrate our proposed model combines symbolic reasoning and function evaluation in a fruitful manner, obtaining high accuracies in our experiments. Our framework generalizes significantly better to expressions of higher depth and is able to fill partial equations with valid completions.
@inproceedings{funeval:iclr18, author = {Forough Arabshahi and Sameer Singh and Animashree Anandkumar}, title = { {Combining Symbolic Expressions and Black-box Function Evaluations for Training Neural Programs} }, booktitle = {International Conference on Learning Representations (ICLR)}, year = {2018} } -
Anchors: High-Precision Model-Agnostic Explanations
We introduce a novel model-agnostic system that explains the behavior of complex models with high-precision rules called anchors, representing local, “sufficient” conditions for predictions. We propose an algorithm to efficiently compute these explanations for any black-box model with high-probability guarantees. We demonstrate the flexibility of anchors by explaining a myriad of different models for different domains and tasks. In a user study, we show that anchors enable users to predict how a model would behave on unseen instances with less effort and higher precision, as compared to existing linear explanations or no explanations.
@inproceedings{anchors:aaai18, author = {Marco Tulio Ribeiro and Sameer Singh and Carlos Guestrin}, title = { {Anchors: High-Precision Model-Agnostic Explanations} }, booktitle = {AAAI Conference on Artificial Intelligence (AAAI)}, pages = {1527-1535}, year = {2018} } -
A Framework of Rapid Regional Tsunami Damage Recognition from Post-event TerraSAR-X Imagery Using Deep Neural Networks
Near real-time building damage mapping is an indispensable prerequisite for governments to make decisions for disaster relief. With high-resolution synthetic aperture radar (SAR) systems, such as TerraSAR-X, the provision of such products in a fast and effective way becomes possible. In this letter, a deep learning-based framework for rapid regional tsunami damage recognition using post-event SAR imagery is proposed. To perform such a rapid damage mapping, a series of tile-based image split analysis is employed to generate the data set. Next, a selection algorithm with the SqueezeNet network is developed to swiftly distinguish between built-up (BU) and nonbuilt-up regions. Finally, a recognition algorithm with a modified wide residual network is developed to classify the BU regions into wash away, collapsed, and slightly damaged regions. Experiments performed on the TerraSAR-X data from the 2011 Tohoku earthquake and tsunami in Japan show a BU region extraction accuracy of 80.4% and a damage-level recognition accuracy of 74.8%, respectively. Our framework takes around 2 h to train on a new region, and only several minutes for prediction.
@article{tsunami:geosense18, author = {Yanbing Bai and Chang Gao and Sameer Singh and Magaly Koch and Bruno Adriano and Erick Mas and Shunichi Koshimura}, title = { {A Framework of Rapid Regional Tsunami Damage Recognition from Post-event TerraSAR-X Imagery Using Deep Neural Networks} }, journal = {IEEE Geoscience and Remote Sensing Letters}, volume = {15}, number = {1}, doi = {10.1109/LGRS.2017.2772349}, pages = {43-47}, year = {2018} } -
Towards Solving Differential Equations through Neural Programming
@inproceedings{diffeqeval:nampi18, author = {Forough Arabshahi and Sameer Singh and Animashree Anandkumar}, title = { {Towards Solving Differential Equations through Neural Programming} }, booktitle = {ICML Workshop on Neural Abstract Machines and Program Induction (NAMPI)}, year = {2018} }
-
Entity Linking via Joint Encoding of Types, Descriptions, and Context
For accurate entity linking, we need to capture various information aspects of an entity, such as its description in a KB, contexts in which it is mentioned, and structured knowledge. Additionally, a linking system should work on texts from different domains without requiring domain-specific training data or hand-engineered features.
In this work we present a neural, modular entity linking system that learns a unified dense representation for each entity using multiple sources of information, such as its description, contexts around its mentions, and its fine-grained types. We show that the resulting entity linking system is effective at combining these sources, and performs competitively, sometimes out-performing current state-of-the-art systems across datasets, without requiring any domain-specific training data or hand-engineered features. We also show that our model can effectively "embed" entities that are new to the KB, and is able to link its mentions accurately.@inproceedings{neuralel:emnlp17, author = {Nitish Gupta and Sameer Singh and Dan Roth}, title = { {Entity Linking via Joint Encoding of Types, Descriptions, and Context} }, booktitle = {Empirical Methods in Natural Language Processing (EMNLP)}, month = {September}, doi = {10.18653/v1/D17-1284}, pages = {2681-2690}, year = {2017} } -
Intelligent Data Filtering in Constrained IoT Systems
The expansion of complex autonomous sensing and control mechanisms in the Internet-of-Things systems clashes with constraints on computation and wireless communication resources. In this paper, we propose a framework to address this conflict for applications in which resolution using a centralized architecture with a general-purpose compression of observations is not appropriate. Three approaches for distributing observation detection workload between sensing and processing devices are considered for sensor systems within wireless islands. Each of the approaches is formulated for the shared configuration of a sensor-edge system, in which the network structure, observation monitoring problem, and machine learning-based detector implementing it are not modified. For every approach, a high-level strategy for realization of the detector for different assumptions on the relation between its complexity and the system's constraints is considered. In each case, the potential for the constraints' satisfaction is shown to exist and be exploitable via division, approximation, and delegation of the detector's workload to the sensing devices off the edge processor. We present examples of applications that benefit from the proposed approaches.
@inproceedings{semcompress:asilomar17, author = {Igor Burago and Davide Callegaro and Marco Levorato and Sameer Singh}, title = { {Intelligent Data Filtering in Constrained IoT Systems} }, booktitle = {Asilomar Conference on Signals, Systems, and Computers}, year = {2017} } -
Semantic Compression for Edge-Assisted Systems
@inproceedings{semcompress:ita17, author = {Igor Burago and Marco Levorato and Sameer Singh}, title = { {Semantic Compression for Edge-Assisted Systems} }, booktitle = {Information Theory and Applications (ITA) Workshop}, month = {February}, year = {2017} } -
Generating Natural Adversarial Examples
Amazon Best Poster Award at the Southern California Machine Learning Symposium.Shorter version of the paper at ICLR 2018.Due to their complex nature, it is hard to characterize the ways in which machine learning models can misbehave or be exploited when deployed. Recent work on adversarial examples, i.e. inputs with minor perturbations that result in substantially different model predictions, is helpful in evaluating the robustness of these models by exposing the adversarial scenarios where they fail. However, these malicious perturbations are often unnatural, not semantically meaningful, and not applicable to complicated domains such as language. In this paper, we propose a framework to generate natural and legible adversarial examples by searching in semantic space of dense and continuous data representation, utilizing the recent advances in generative adversarial networks. We present generated adversaries to demonstrate the potential of the proposed approach for black-box classifiers in a wide range of applications such as image classification, textual entailment, and machine translation. We include experiments to show that the generated adversaries are natural, legible to humans, and useful in evaluating and analyzing black-box classifiers.
@inproceedings{natadv:mldecept17, author = {Zhengli Zhao and Dheeru Dua and Sameer Singh}, title = { {Generating Natural Adversarial Examples} }, booktitle = {NeurIPS Workshop on Machine Deception}, year = {2017} } -
How Biased Are We? Automated Detection of Gendered Language
Also presented at the NeurIPS 2017 Workshop for Women in Machine Learning (WiML).
@inproceedings{gender:winlp17, author = {Ananya Ananya and Sameer Singh}, title = { {How Biased Are We? Automated Detection of Gendered Language} }, booktitle = {ACL Workshop on Women and Underrepresented Minorities in NLP (WiNLP)}, month = {August}, year = {2017} } -
Compact Factorization of Matrices Using Generalized Round-Rank
@inproceedings{grank:southcal17, author = {Pouya Pezeshkpour and Carlos Guestrin and Sameer Singh}, title = { {Compact Factorization of Matrices Using Generalized Round-Rank} }, booktitle = {Southern California Machine Learning Symposium}, year = {2017} } -
Embedding Multimodal Relational Data
@inproceedings{mmkbe:akbc17, author = {Pouya Pezeshkpour and Liyan Chen and Sameer Singh}, title = { {Embedding Multimodal Relational Data} }, booktitle = {Workshop on Automated Knowledge Base Construction (AKBC)}, year = {2017} } -
Multimodal Attribute Extraction
@inproceedings{maed:akbc17, author = {Robert L. Logan IV and Samuel Humeau and Sameer Singh}, title = { {Multimodal Attribute Extraction} }, booktitle = {Workshop on Automated Knowledge Base Construction (AKBC)}, year = {2017} } -
Relational Learning and Feature Extraction by Querying over Heterogeneous Information Networks
Many real world systems need to operate on heterogeneous information networks that consist of numerous interacting components of different types. Examples include systems that perform data analysis on biological information networks; social networks; and information extraction systems processing unstructured data to convert raw text to knowledge graphs. Many previous works describe specialized approaches to perform specific types of analysis, mining and learning on such networks. In this work, we propose a unified framework consisting of a data model -a graph with a first order schema along with a declarative language for constructing, querying and manipulating such networks in ways that facilitate relational and structured machine learning. In particular, we provide an initial prototype for a relational and graph traversal query language where queries are directly used as relational features for structured machine learning models. Feature extraction is performed by making declarative graph traversal queries. Learning and inference models can directly operate on this relational representation and augment it with new data and knowledge that, in turn, is integrated seamlessly into the relational structure to support new predictions. We demonstrate this system's capabilities by showcasing tasks in natural language processing and computational biology domains.
@inproceedings{saul:starai17, author = {Parisa Kordjamshidi and Sameer Singh and Daniel Khashabi and Christos Christodoulopoulos and Mark Summons and Saurabh Sinha and Dan Roth}, title = { {Relational Learning and Feature Extraction by Querying over Heterogeneous Information Networks} }, booktitle = {International Workshop on Statistical Relational AI (StarAI)}, month = {July}, year = {2017} }
-
Better call Saul: Flexible Programming for Learning and Inference in NLP
@inproceedings{saul:coling16, author = {Parisa Kordjamshidi and Daniel Khashabi and Christos Christodoulopoulos and Bhargav Mangipudi and Sameer Singh and Dan Roth}, title = { {Better call Saul: Flexible Programming for Learning and Inference in NLP} }, booktitle = {International Conference on Computational Linguistics (COLING)}, month = {December}, pages = {3030-3040}, year = {2016} } -
Connotation Frames: A Data-Driven Investigation
@inproceedings{connot:acl16, author = {Hannah Rashkin and Sameer Singh and Yejin Choi}, title = { {Connotation Frames: A Data-Driven Investigation} }, booktitle = {Association for Computational Linguistics (ACL)}, month = {August}, doi = {10.18653/v1/P16-1030}, pages = {311-321}, year = {2016} } -
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Audience Appreciation AwardAlso presented at the CHI 2016 Workshop on Human-Centred Machine Learning (HCML).
@inproceedings{lime:kdd16, author = {Marco Tulio Ribeiro and Sameer Singh and Carlos Guestrin}, title = { {"Why Should I Trust You?": Explaining the Predictions of Any Classifier} }, booktitle = {Knowledge Discovery and Data Mining (KDD)}, month = {August}, doi = {10.1145/2939672.2939778}, pages = {1135-1144}, year = {2016} } -
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Demonstration of the KDD 2016 paper.
@inproceedings{lime:naacl16, author = {Marco Tulio Ribeiro and Sameer Singh and Carlos Guestrin}, title = { {"Why Should I Trust You?": Explaining the Predictions of Any Classifier} }, booktitle = {Demo at the Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)}, month = {June}, year = {2016} } -
Introduction to Local Interpretable Model-Agnostic Explanations (LIME)
@misc{lime:oreilly16, author = {Marco Tulio Ribeiro and Sameer Singh and Carlos Guestrin}, title = { {Introduction to Local Interpretable Model-Agnostic Explanations (LIME)} }, editor = {O'Reilly Media}, month = {August}, url = {https://www.oreilly.com/learning/introduction-to-local-interpretable-model-agnostic-explanations-lime}, year = {2016} } -
Programs as Black-Box Explanations
Recent work in model-agnostic explanations of black-box machine learning has demonstrated that interpretability of complex models does not have to come at the cost of accuracy or model flexibility. However, it is not clear what kind of explanations, such as linear models, decision trees, and rule lists, are the appropriate family to consider, and different tasks and models may benefit from different kinds of explanations. Instead of picking a single family of representations, in this work we propose to use "programs" as model-agnostic explanations. We show that small programs can be expressive yet intuitive as explanations, and generalize over a number of existing interpretable families. We propose a prototype program induction method based on simulated annealing that approximates the local behavior of black-box classifiers around a specific prediction using random perturbations. Finally, we present preliminary application on small datasets and show that the generated explanations are intuitive and accurate for a number of classifiers.
@inproceedings{prog:nipsws16, author = {Sameer Singh and Marco Tulio Ribeiro and Carlos Guestrin}, title = { {Programs as Black-Box Explanations} }, booktitle = {NeurIPS Workshop on Interpretable Machine Learning in Complex Systems}, month = {November}, year = {2016} } -
Nothing Else Matters: Model-Agnostic Explanations By Identifying Prediction Invariance
At the core of interpretable machine learning is the question of whether humans are able to make accurate predictions about a model's behavior. Assumed in this question are three properties of the interpretable output: coverage, precision, and effort. Coverage refers to how often humans think they can predict the model's behavior, precision to how accurate humans are in those predictions, and effort is either the up-front effort required in interpreting the model, or the effort required to make predictions about a model's behavior.
In this work, we propose anchor-LIME (aLIME), a model-agnostic technique that produces high-precision rule-based explanations for which the coverage boundaries are very clear. We compare aLIME to linear LIME with simulated experiments, and demonstrate the flexibility of aLIME with qualitative examples from a variety of domains and tasks.@inproceedings{anchor:nipsws16, author = {Marco Tulio Ribeiro and Sameer Singh and Carlos Guestrin}, title = { {Nothing Else Matters: Model-Agnostic Explanations By Identifying Prediction Invariance} }, booktitle = {NeurIPS Workshop on Interpretable Machine Learning in Complex Systems}, month = {November}, year = {2016} } -
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Shorter version of the paper presented at KDD 2016.
@inproceedings{lime:hcml16, author = {Marco Tulio Ribeiro and Sameer Singh and Carlos Guestrin}, title = { {"Why Should I Trust You?": Explaining the Predictions of Any Classifier} }, booktitle = {CHI Workshop on Human-Centred Machine Learning (HCML)}, month = {May}, year = {2016} } -
Model-Agnostic Interpretability of Machine Learning
Best Paper Award
@inproceedings{lime:whi16, author = {Marco Tulio Ribeiro and Sameer Singh and Carlos Guestrin}, title = { {Model-Agnostic Interpretability of Machine Learning} }, booktitle = {ICML Workshop on Human Interpretability in Machine Learning (WHI)}, month = {June}, year = {2016} } -
Creating Interactive and Visual Educational Resources for AI
Teaching artificial intelligence is effective if the experience is a visual and interactive one, with educational materials that utilize combinations of various content types such as text, math, and code into an integrated experience. Unfortunately, easy-to-use tools for creating such pedagogical resources are not available to the educators, resulting in most courses being taught using a disconnected set of static materials, which is not only ineffective for learning AI, but further, requires repeated and redundant effort for the instructor. In this paper, we introduce Moro, a software tool for easily creating and presenting AI-friendly teaching materials. Moro notebooks integrate content of different types (text, math, code, images), allow real-time interactions via modifiable and executable code blocks, and are viewable in browsers both as long-form pages and as presentations. Creating notebooks is easy and intuitive; the creation tool is also in-browser, is WYSIWYG for quick iterations of editing, and supports a variety of shortcuts and customizations for efficiency. We present three deployed case studies of Moro that widely differ from each other, demonstrating its utility in a variety of scenarios such as in-class teaching and conference tutorials.
@inproceedings{moro:eaai16, author = {Sameer Singh and Sebastian Riedel}, title = { {Creating Interactive and Visual Educational Resources for {AI}} }, booktitle = {AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI)}, year = {2016} }