TY - GEN
T1 - Dynamic word recommendation to obtain diverse crowdsourced paraphrases of user utterances
AU - Yaghoub-Zadeh-Fard, Mohammad Ali
AU - Benatallah, Boualem
AU - Casati, Fabio
AU - Barukh, Moshe Chai
AU - Zamanirad, Shayan
PY - 2020/3/17
Y1 - 2020/3/17
N2 - Building task-oriented bots requires mapping a user utterance to an intent with its associated entities to serve the request. Doing so is not easy since it requires large quantities of high-quality and diverse training data to learn how to map all possible variations of utterances with the same intent. Crowdsourcing may be an effective, inexpensive, and scalable technique for collecting such large datasets. However, the diversity of the results suffers from the priming effect (i.e. workers are more likely to use the words in the sentence we are asking to paraphrase). In this paper, we leverage priming as an opportunity rather than a threat: we dynamically generate word suggestions to motivate crowd workers towards producing diverse utterances. The key challenge is to make suggestions that can improve diversity without resulting in semantically invalid paraphrases. To achieve this, we propose a probabilistic model that generates continuously improved versions of word suggestions that balance diversity and semantic relevance. Our experiments show that the proposed approach improves the diversity of crowdsourced paraphrases.
AB - Building task-oriented bots requires mapping a user utterance to an intent with its associated entities to serve the request. Doing so is not easy since it requires large quantities of high-quality and diverse training data to learn how to map all possible variations of utterances with the same intent. Crowdsourcing may be an effective, inexpensive, and scalable technique for collecting such large datasets. However, the diversity of the results suffers from the priming effect (i.e. workers are more likely to use the words in the sentence we are asking to paraphrase). In this paper, we leverage priming as an opportunity rather than a threat: we dynamically generate word suggestions to motivate crowd workers towards producing diverse utterances. The key challenge is to make suggestions that can improve diversity without resulting in semantically invalid paraphrases. To achieve this, we propose a probabilistic model that generates continuously improved versions of word suggestions that balance diversity and semantic relevance. Our experiments show that the proposed approach improves the diversity of crowdsourced paraphrases.
KW - bots
KW - crowdsourcing
KW - paraphrasing
UR - http://www.scopus.com/inward/record.url?scp=85082447880&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85082447880&partnerID=8YFLogxK
U2 - 10.1145/3377325.3377486
DO - 10.1145/3377325.3377486
M3 - Conference contribution
AN - SCOPUS:85082447880
T3 - International Conference on Intelligent User Interfaces, Proceedings IUI
SP - 55
EP - 66
BT - Proceedings of the 25th International Conference on Intelligent User Interfaces, IUI 2020
PB - Association for Computing Machinery
T2 - 25th ACM International Conference on Intelligent User Interfaces, IUI 2020
Y2 - 17 March 2020 through 20 March 2020
ER -