12/7/2017
(After getting an overview of deep learning from the videos in CS231N and CS224N, my goal was to get my hands dirty with existing deep learning models. Here I give an overview of some exploratory experiments I ran to see how "data provenance" ideas can be applied to deep learning models, specifically focusing on models that perform Question Answering.)
The idea explored here: Can data provenance (or a similar concept) help explain why a deep learning model produces a particular output for a given input?
Since deep learning models are thought to be black-box functions, I was interested in seeing whether provenance could give insights on how deep learning models work.
In data-oriented workflows, the provenance of an output element is the subset of input elements that contributed to the output element.
Here we explore whether a similar concept can be defined for deep learning models. The concept we propose is "importance."
Let Q be the original question that the model answered with answer A. Let Q' be an edited version of question Q that omits word W. The model answers Q' with answer A'. If A is correct and A' is incorrect (or vice versa), then word W is important to question Q.
Intuitively, if omitting a word in the question caused the model's answer to change from correct to incorrect (or vice versa), the word is deemed "important."
The deep learning model used in the experiments was trained on the SQuAD dataset. (SQuAD is described in the Appendix in more detail.)
This model is a simplified version of the model presented in Bidirectional Attention Flow for Machine Comprehension (ICLR 2017), with similar performance to the original implementation.
The code used for the model is from https://github.com/yolandawww/QASystem.
Note that these questions have relatively few words deemed "important."
Also consider that these questions from SQuAD are answered in the context of a particular paragraph from Wikipedia. Thus, if the subject (e.g., Tesla in Q3) of the paragraph is already known, then the Question Answering system can answer correctly even if the word "Tesla" is omitted.
Data Augmentation: By omitting certain words in the training data, we may find new questions that are well-posed and yet the system cannot answer correctly. By adding these new questions to our training set, we may improve the accuracy of our trained model.
Exploration: By finding words in the question that influence the model's answer, we can gain an intuition on how the model currently works, suggesting possible ways to improve the model.
For now, I will shift my focus away from SQuAD and onto word problems. There is a recently released dataset from DeepMind on algebra word problems. Since this dataset is newer than SQuAD, explorations of this dataset may be more fruitful.
SQuAD (Stanford Question Answering Dataset) is a dataset that tests a machine's reading comprehension.
The questions and answers in the dataset are based on context paragraphs from Wikipedia.
After reading a paragraph from Wikipedia, the machine is asked a question about the paragraph.
Context paragraph (from Wikipedia article Super Bowl 50)
Super Bowl 50 was an American football game to determine the champion of the National Football League (NFL) for the 2015 season. The American Football Conference (AFC) champion Denver Broncos defeated the National Football Conference (NFC) champion Carolina Panthers 24-10 to earn their third Super Bowl title. The game was played on February 7, 2016, at Levi's Stadium in the San Francisco Bay Area at Santa Clara, California. As this was the 50th Super Bowl, the league emphasized the "golden anniversary" with various gold-themed initiatives, as well as temporarily suspending the tradition of naming each Super Bowl game with Roman numerals (under which the game would have been known as "Super Bowl L"), so that the logo could prominently feature the Arabic numerals 50.
Question: Which NFL team represented the AFC at Super Bowl 50?
Answer: Denver Broncos
For the SQuAD dataset, the correct answer is always a subspan of the context paragraph.
Deep learning models today perform worse than a human, but substantially better than the logistic regression baseline.
Performance Comparison (F1 Scores are listed below)