no code implementations • 1 Mar 2023 • Adam Davies, Jize Jiang, ChengXiang Zhai
Our framework, CALM (Competence-based Analysis of Language Models), establishes the first quantitative measure of LLM competence, which we study by damaging models' internal representations of various linguistic properties in the course of performing various tasks using causal probing and evaluating models' alignment under these interventions with a given causal model.
1 code implementation • 21 Dec 2022 • Jianhao Yuan, Francesco Pinto, Adam Davies, Philip Torr
Neural image classifiers are known to undergo severe performance degradation when exposed to inputs that are sampled from environmental conditions that differ from their training data.