Jack Hessel's Homepage
Research Scientist @ AI2
CV (as of 9/2021).
I am on github, twitter, google scholar, and semantic scholar.
As of July 2021, I am a Research Scientist at AI2. Previously, I earned a PhD in Computer Science at Cornell University.
I work mostly on natural language processing, machine learning, and computer vision. If you're looking for me, I look something like this (facial hair subject to change):
Publications (in reverse chronological order)
MERLOT: Multimodal Neural Script Knowledge Models
Rowan Zellers*, Ximing Lu*, Jack Hessel*, Youngjae Yu, Jae Sung Park, Jize Cao, Ali Farhadi, and Yejin Choi.
NeurIPS 2021; Oral presentation selection
code, project page
CLIPScore: A Reference-free Evaluation Metric for Image Captioning
Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi
How effective is BERT without word ordering? Implications for language understanding and data privacy.
Jack Hessel and Alexandra Schofield
Does my multimodal model learn cross-modal interactions? It’s harder to tell than you might think!
Jack Hessel and Lillian Lee
Beyond Instructional Videos: Probing for More Diverse Visual-Textual Grounding on YouTube
Jack Hessel, Zhenhai Zhu, Bo Pang, and Radu Soricut
Domain-Specific Lexical Grounding in Noisy Visual-Textual Documents
Gregory Yauney, Jack Hessel, and David Mimno
Learning from Multimodal Web Data
Cornell University 2020
Unsupervised Discovery of Multimodal Links in Multi-image, Multi-sentence Documents
Jack Hessel, Lillian Lee, and David Mimno
A Case Study on Combining ASR and Visual Features for Generating Instructional Video Captions
Jack Hessel, Bo Pang, Zhenhai Zhu, and Radu Soricut
Something's Brewing! Early Prediction of Controversy-causing Posts from Discussion Features
Jack Hessel and Lillian Lee
Quantifying the Visual Concreteness of Words and Topics in Multimodal Datasets
Jack Hessel, David Mimno, and Lillian Lee
- Cats and Captions vs. Creators and the Clock: Comparing Multimodal Content to Context in Predicting Relative Popularity
Jack Hessel, Lillian Lee, David Mimno
- Aligning Images and Text in a Digital Library
Jack Hessel and David Mimno
Computer Vision in Digital Humanities Workshop at DH 2017 (extended abstract)
Science, AskScience and BadScience: On the Coexistience of Highly Related Communities
Jack Hessel, Chenhao Tan, and Lillian Lee
- What do Vegans do in their Spare Time? Latent Interest Detection in Multi-Community Networks
Jack Hessel, Alexandra Schofield, Lillian Lee, David Mimno
Workshop on Networks in the Social and Information Sciences at NeurIPS 2015
Image Representations and New Domains in Neural Image Captioning
Jack Hessel, Nicolas Savva, and Kimberly J. Wilber
Workshop on Vision and Language at EMNLP 2015
- Using Reproductive Altruism to Evolve Multicellularity in Digital Organisms
Jack Hessel and Sherri Goings
Things I've Taught
- I was a visiting faculty member at Carleton College in Spring, 2019; I taught two classes: Natural Language Processing and Discrete Math.
INFO/CS 4300 Language and Information, Cornell University, SP 2016
CS4786 Machine Learning for Data Science, Cornell University, SP 2015 for which I received an outstanding TA award!
CS4620 Introduction to Computer Graphics, Cornell University, FA 2014
- Introduction to Computer Science/Graph Theory, Carleton College, FA 2011
Service/Guest Lectures/Other Activites
- Program Committees: ACL 2016, 2017, 2018*, 2019, 2020*, 2021*; NAACL 2018, 2019*, 2021; EMNLP 2017, 2018*, 2019*, 2020; EACL 2017; AAAI 2017; ICWSM 2018; CoNLL 2019*, 2020, 2021; ICML 2020*. For the starred conferences, I was acknowledged as a top reviewer!
- Workshop Program Committees: Black in AI @ NeurIPS 2017, 2018, 2019, 2020; Student Research Workshop @ NAACL 2018, EACL 2021, ACL 2020, 2021 ; Workshop on Noisy User-generated Text (W-NUT) @ EMNLP 2018, 2019, 2020.
- Two guest lectures (and one "magic" trick) for Spring 2016/2017 CS4300 about my favorite machine learning algorithm: the truncated SVD!
- Spoke at Carleton College's Networks Reading Group where I discussed the role of altruism on Kickstarter.
- Spring Cornell CS4740 Guest Lecture on Topic Models with Xanda Schofield
- Helped with Cornell's annual Expand Your Horizons conference, where some friends and I taught a class on programming (2015, 2016)
- I wrote a TreeLSTM in tensorflow 2; this is a neural network whose topology changes based on each input example.
- I wrote a factorization machine layer in pytorch; for speed reasons, the forward and backward passes are written in cython.
- I implemented Monroe et al.'s "Fightin' Words" algorithm for robustly comparing word frequencies in two corpora. This implementation has been used in several publications, e.g., this and this
- As part of an NSF REU,
I contributed to the implementation of a really (really) fast SVM solver in Kilian Weinberger's lab at Washington University, St. Louis. [now at Cornell!] see Parallel Support Vector Machines in Practice by Tyree, S., Gardner, J. R., Weinberger, K. Q., Agrawal, K., & Tran, J. (2014).
- "A Comparative Analysis of Popular Phylogenetic Reconstruction Algorithms." Undergraduate thesis project/best paper award at MICS 2014: joint work with Evan Albright, Nao Hiranuma, Cody Wang, and Sherri Goings.
I grew up in beautiful Portola Valley, California. I earned a B.A. from Carleton College in 2014, studying computer science and mathematics/statistics. During my time in Northfield, I played ice hockey, hosted a radio show, and was lucky enough to meet and work with a lot of great people. I even returned to Carleton briefly in 2019, this time, as a visiting faculty member! I'm a die hard San Jose Sharks fan, avid consumer (and very occasionally a producer) of electronic music, and, most of the time, a vegetarian (I am no longer a vegan!). During graduate school at Cornell, I was a member of Stewart Little Coop, a cooperative living community of 15 people, I played ice hockey in the Ithaca Hockey Association (and, during summer internships in CA, in the San Jose Adult Hockey League).