Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Models

1 code implementation • 11 Jan 2024 • Asma Ghandeharioun, Avi Caciularu, Adam Pearce, Lucas Dixon, Mor Geva

We introduce a framework called Patchscopes and show how it can be used to answer a wide range of questions about an LLM's computation.

Acquisition of Chess Knowledge in AlphaZero

no code implementations • 17 Nov 2021 • Thomas McGrath, Andrei Kapishnikov, Nenad Tomašev, Adam Pearce, Demis Hassabis, Been Kim, Ulrich Paquet, Vladimir Kramnik

In this work we provide evidence that human knowledge is acquired by the AlphaZero neural network as it trains on the game of chess.

Game of Chess

An Interpretability Illusion for BERT

no code implementations • 14 Apr 2021 • Tolga Bolukbasi, Adam Pearce, Ann Yuan, Andy Coenen, Emily Reif, Fernanda Viégas, Martin Wattenberg

We describe an "interpretability illusion" that arises when analyzing the BERT model.

Impacts of Social Distancing Policies on Mobility and COVID-19 Case Growth in the US

no code implementations • 21 Apr 2020 • Gregory A. Wellenius, Swapnil Vispute, Valeria Espinosa, Alex Fabrikant, Thomas C. Tsai, Jonathan Hennessy, Andrew Dai, Brian Williams, Krishna Gadepalli, Adam Boulanger, Adam Pearce, Chaitanya Kamath, Arran Schlosberg, Catherine Bendebury, Chinmoy Mandayam, Charlotte Stanton, Shailesh Bavadekar, Christopher Pluntke, Damien Desfontaines, Benjamin Jacobson, Zan Armstrong, Bryant Gipson, Royce Wilson, Andrew Widdowson, Katherine Chou, Andrew Oplinger, Tomer Shekel, Ashish K. Jha, Evgeniy Gabrilovich

Social distancing remains an important strategy to combat the COVID-19 pandemic in the United States.

Visualizing and Measuring the Geometry of BERT

2 code implementations • NeurIPS 2019 • Andy Coenen, Emily Reif, Ann Yuan, Been Kim, Adam Pearce, Fernanda Viégas, Martin Wattenberg

Transformer architectures show significant promise for natural language processing.

Word Embeddings