<, [David Mimno](https://mimno.infosci.cornell.edu), [Lillian Lee](https://www.cs.cornell.edu/home/llee/)
NAACL-HLT 2018 ([paper link](paper.pdf))
---
<img src="images/merged.png" alt="Concreteness Examples"/ style="width:auto;">
*Examples of concrete words and topics from three datasets. Our algorithm is able to automatically compute concreteness scores based on the geometry of visual and textual representations.*
---
In this work we give a method for quantifying how "concrete" a given
word/topic is within a specific dataset; concrete concepts are those
associated with more consistent images. In addition to correlating
with human judgment, we demonstrate that concreteness scores are
highly correlated with retrieval performance. In general, the more
concrete a words/topic is, the easier it is for machine learning
algorithms to recognize.
#### Update!
We released a new version of the paper on arXiv (the new version is
linked above) which fixes a small typo in Equation 3.
## Dataset and Code
### Wikipedia
A README of the wikipedia dataset is available
[here](wiki_README.txt). The full 4.9GB dataset can be downloaded
[here](https://drive.google.com/open?id=1Lkt4OOMeyYSvm2w_UfQxYZrGcEIEsKNL).
The images.txt file for the wikipedia set is [here](https://drive.google.com/file/d/1Tj_zDYNdVATBoufIn7wIDbp95iPB7Vb4/view?usp=sharing).
Full details of dataset construction are available in the paper. Image
data for 549K images are provided as pre-extracted features from
[ResNet50](https://arxiv.org/abs/1512.03385). While raw images are
unavailable because image copyright varies from image-to-image on
wikipedia, Wikipedia urls are included and can be re-scraped if
needed, though some of the images may have been removed from the commons.
If you have questions about this data, or encounter bugs, please
e-mail jhessel@cs.cornell.edu.
### British Library
A README of the british library dataset is available
[here](bl_README.txt). The full 38GB dataset can be downloaded
[here](https://drive.google.com/open?id=1UDPynI4X62jwf7tqSXY3dtc3HWk6FjPh).
Full details of the dataset construction are available in the paper.
The 405K images are originally from the [British
Library;](https://data.bl.uk/digbks/) they were released to the
[public
domain](https://creativecommons.org/publicdomain/mark/1.0/). Image
data is provided as jpgs; each is resized so that the largest dimension
is at most 600 pixels.
If you have questions about this data, or encounter bugs, please
e-mail jhessel@cs.cornell.edu.
### Code
<a href= https://victorssilva.com/>Victor Silva</a> implemented the <a href=https://github.com/victorssilva/concreteness>concreteness algorithm</a> described in section 3 of our paper for word types!
Here's a function that computes the concretness of <a href=https://gist.github.com/jmhessel/db0e5e1885a2cdf99742dec8a0167377>topics,</a> which is similar to computing the concreteness of word types, except the computation accounts for continuous topic weights (see section 3.2 of the paper).
## Citation
```
@inproceedings{hessel2018concreteness,
title={Quantifying the visual concreteness of words and topics in multimodal datasets},
author={Hessel, Jack and Mimno, David and Lee, Lillian},
booktitle={NAACL},
year={2018}
}
```
<sub><sup>
Images on this website are reproduced in accordance with the following
licensing information. The original authors of the images do not
necessarily endorse this work or the opinions expressed in it.
Copyright information is given left-to-right.
</sub></sup>
<sub><sup>
Wikipedia: "hockey" -- Dan4th Nicholas (CC BY 2.0); Leech44 (CC BY
3.0); Usasoccer2010 (CC BY-SA 3.0). All "software" images derive from
software licensed under GNU GPL v3; screenshot used under fair
use. "Australia" --Toby Hudson (CC BY-SA 3.0); public domain; public
domain. "linguistics" -- Suasysar (public domain); USDA (public
domain); Rama (CC BY-SA 2.0 FR).
</sub></sup>
<sub><sup>
MSCOCO: "ben" -- Michael Garnett (CC BY-NC-ND 2.0); Giovanni Tufo (CC
BY-NC-ND 2.0); Edinho Souto (CC BY-NC-SA 2.0). "magnets" -- Gabriel
Pires (CC BY-NC-SA 2.0); the_blue (CC BY-NC-SA 2.0); Dennis S. Hurd
(CC BY-NC-ND 2.0) "weird" -- waferboard (CC BY 2.0); jaci XIII (CC
BY-NC-SA 2.0); Ozzy Delaney (CC BY 2.0). "caption" -- Robert G. Daigle
(CC BY 2.0) TJ Gehling (CC BY-NC-ND 2.0); amhuxham (CC BY-NC-SA 2.0)
</sub></sup>
<sub><sup>
Flickr: "cheetah" -- Nathan Rupert (CC BY-NC-ND 2.0); Karen Dorsett (CC BY-ND
2.0); Vearl Brown (CC BY-NC 2.0). "bodybuilder" -- Kent (SSCusp) (CC
BY-NC-ND 2.0); Frank Carroll (CC BY-NC-SA 2.0); Kent (SSCusp) (CC
BY-NC-ND 2.0). "geotagged" -- tpholland (CC BY 2.0); s2art (CC
BY-NC-SA 2.0); Joe Penniston (CC BY-NC-ND 2.0). "canon" --
ajagendorf25 (CC BY 2.0); Tom Lin (CC BY-NC-ND 2.0); Leo Reynolds (CC
BY-NC-SA 2.0).
</sub></sup>
]]>