Literature DB >> 35422527

Semantic projection recovers rich human knowledge of multiple object features from word embeddings.

Gabriel Grand1,2, Idan Asher Blank3,4, Francisco Pereira5, Evelina Fedorenko6,7.   

Abstract

How is knowledge about word meaning represented in the mental lexicon? Current computational models infer word meanings from lexical co-occurrence patterns. They learn to represent words as vectors in a multidimensional space, wherein words that are used in more similar linguistic contexts-that is, are more semantically related-are located closer together. However, whereas inter-word proximity captures only overall relatedness, human judgements are highly context dependent. For example, dolphins and alligators are similar in size but differ in dangerousness. Here, we use a domain-general method to extract context-dependent relationships from word embeddings: 'semantic projection' of word-vectors onto lines that represent features such as size (the line connecting the words 'small' and 'big') or danger ('safe' to 'dangerous'), analogous to 'mental scales'. This method recovers human judgements across various object categories and properties. Thus, the geometry of word embeddings explicitly represents a wealth of context-dependent world knowledge.
© 2022. The Author(s), under exclusive licence to Springer Nature Limited.

Entities:  

Mesh:

Year:  2022        PMID: 35422527     DOI: 10.1038/s41562-022-01316-8

Source DB:  PubMed          Journal:  Nat Hum Behav        ISSN: 2397-3374


  39 in total

1.  Parts and boundaries.

Authors:  R Jackendoff
Journal:  Cognition       Date:  1991-12

2.  The nature and measurement of meaning.

Authors:  C E OSGOOD
Journal:  Psychol Bull       Date:  1952-05       Impact factor: 17.737

3.  Prediction During Natural Language Comprehension.

Authors:  Roel M Willems; Stefan L Frank; Annabel D Nijhof; Peter Hagoort; Antal van den Bosch
Journal:  Cereb Cortex       Date:  2015-04-22       Impact factor: 5.357

4.  Semantics derived automatically from language corpora contain human-like biases.

Authors:  Aylin Caliskan; Joanna J Bryson; Arvind Narayanan
Journal:  Science       Date:  2017-04-14       Impact factor: 47.728

5.  A comparative evaluation of off-the-shelf distributed semantic representations for modelling behavioural data.

Authors:  Francisco Pereira; Samuel Gershman; Samuel Ritter; Matthew Botvinick
Journal:  Cogn Neuropsychol       Date:  2016 May-Jun       Impact factor: 2.468

6.  Parallelograms revisited: Exploring the limitations of vector space models for simple analogies.

Authors:  Joshua C Peterson; Dawn Chen; Thomas L Griffiths
Journal:  Cognition       Date:  2020-08-31

7.  'Clap your hands' or 'take your hands'? One-year-olds distinguish between frequent and infrequent multiword phrases.

Authors:  Barbora Skarabela; Mitsuhiko Ota; Rosie O'Connor; Inbal Arnon
Journal:  Cognition       Date:  2021-02-09

8.  The effect of word predictability on reading time is logarithmic.

Authors:  Nathaniel J Smith; Roger Levy
Journal:  Cognition       Date:  2013-06-06

9.  The Emergence of Richly Organized Semantic Knowledge from Simple Statistics: A Synthetic Review.

Authors:  Layla Unger; Anna V Fisher
Journal:  Dev Rev       Date:  2021-03-03

10.  Structured Semantic Knowledge Can Emerge Automatically from Predicting Word Sequences in Child-Directed Speech.

Authors:  Philip A Huebner; Jon A Willits
Journal:  Front Psychol       Date:  2018-02-22
View more
  2 in total

Review 1.  Beyond the Benchmarks: Toward Human-Like Lexical Representations.

Authors:  Suzanne Stevenson; Paola Merlo
Journal:  Front Artif Intell       Date:  2022-05-24

2.  Context Matters: Recovering Human Semantic Structure from Machine Learning Analysis of Large-Scale Text Corpora.

Authors:  Marius Cătălin Iordan; Tyler Giallanza; Cameron T Ellis; Nicole M Beckage; Jonathan D Cohen
Journal:  Cogn Sci       Date:  2022-02
  2 in total

北京卡尤迪生物科技股份有限公司 © 2022-2023.