Publications

For the past two decades, Professor Deb Roy and his research group at the Media Lab have conducted research that has led to more than 175 peer-reviewed publications in human-machine system design, computational social science, machine learning, natural language processing, and data visualization.

2024
Kessler, Daniel and Dimitrakopoulou, Dimitra and Roy, Deb (October 08, 2024) Hearing Personal Experiences Improves Social Evaluations Compared to Personal Opinions, Especially for Polarized Parties Preprint available at SSRN: https://ssrn.com/abstract=4978495 or http://dx.doi.org.ezproxy.canberra.edu.au/10.2139/ssrn.4978495

Peer-Reviewed Paper

Research

View Publication
View Publication
Margaret Hughes, Brandon C. Roy, Deb Roy. 2024. In Pursuit of Constructive Communication: Designing Tools to Support Development of Constructive Communication Metrics. DIS '24 Companion: Companion Publication of the 2024 ACM Designing Interactive Systems Conference Pages 121-124. https://doi-org.ezproxy.canberra.edu.au/10.1145/3656156.3663720

Peer-Reviewed Paper

Research

View Publication
View Publication
Hang Jiang, Nanxi Liu, Cassandra Overney, Rukun Zhang, Artemisia Luk, Diyi Yang, Deb Roy, and Jad Kabbara. 2024. Lost in Translation: Investigating Systematic Discrepancies between Parallel English and Chinese Names of American Chinese Restaurants. Paper presented at both 10th International Conference on Computational Social Science (IC2S2 2024) and 15th Annual Conference on New Directions in Analyzing Text as Data (TADA 2023).

Peer-Reviewed Paper

View Publication
View Publication
Hang Jiang, Doug Beeferman, William Brannon, Andrew Heyward, and Deb Roy. 2024. Bridging Dictionary: AI-Generated Dictionary of Partisan Language Use. In Companion Publication of the 2024 Conference on Computer Supported Cooperative Work and Social Computing (CSCW ’24 Companion).

Peer-Reviewed Paper

View Publication
View Publication
William Brannon, Doug Beeferman, Hang Jiang, Andrew Heyward, and Deb Roy. 2024. AudienceView: AI-Assisted Interpretation of Audience Feedback in Journalism. In Companion Publication of the 2024 Conference on Computer Supported Cooperative Work and Social Computing (CSCW ’24 Companion). AudienceView: AI-Assisted Interpretation of Audience Feedback in Journalism. In Companion Publication of the 2024 Conference on Computer Supported Cooperative Work and Social Computing (CSCW ’24 Companion).

Peer-Reviewed Paper

View Publication
View Publication
Hang Jiang, Xiajie Zhang, Robert Mahari, Daniel Kessler, Eric Ma, Tal August, Irene Li, Alex Pentland, Yoon Kim, Deb Roy, and Jad Kabbara. 2024. Leveraging Large Language Models for Learning Complex Legal Concepts through Storytelling. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7194–7219, Bangkok, Thailand. Association for Computational Linguistics.

Peer-Reviewed Paper

View Publication
View Publication
Wonjune Kang and Deb Roy. Prompting Large Language Models with Audio for General-Purpose Speech Summarization.  Proceedings of Interspeech, 2024, pp. 1955-1959.

Peer-Reviewed Paper

View Publication
View Publication
Wonjune Kang*, Margaret A. Hughes*, and Deb Roy. Anonymization of Voices in Spaces for Civic Dialogue: Measuring Impact on Empathy, Trust, and Feeling Heard. Proceedings of the ACM on Human-Computer Interaction 8, CSCW2. 2024. *Equal Contribution

Peer-Reviewed Paper

View Publication
View Publication
Hang Jiang, Xiajie Zhang, Xubo Cao, Cynthia Breazeal, Deb Roy, and Jad Kabbara. 2024. PersonaLLM: Investigating the Ability of Large Language Models to Express Personality Traits. In Findings of the Association for Computational Linguistics: NAACL 2024, pages 3605–3627, Mexico City, Mexico. Association for Computational Linguistics.

Peer-Reviewed Paper

View Publication
View Publication
Saldías, B., (July 2024). Designing Child-Centered Content Exposure and Moderation. Presented as a poster at the 10th International Conference on Computational Social Science (IC2S2), Philadelphia, USA.

Other Publication

View Publication
View Publication
Poole-Dayan, E. and Roy, D., (July 2024). Sandbagging in LLMs: Targeted Underperformance for Uneducated and ESL Users. Presented as a poster at the 10th International Conference on Computational Social Science (IC2S2), Philadelphia, USA.

Other Publication

Hang Jiang, Doug Beeferman, Weiquan Mao, and Deb Roy. 2024. Topic Detection and Tracking with Time-Aware Document Embeddings. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 16293–16303, Torino, Italia. ELRA and ICCL.

Peer-Reviewed Paper

View Publication
View Publication
Matthew Nay. (2024).  Decentralized Social Networking Protocol (DSNP) and User Empowerment: An Analysis of Online Identity Ownership, Data Privacy, and Comparative Assessment. M.Sc. Thesis, Massachusetts Institute of Technology.

Masters Thesis

View Publication
View Publication
Longpre, S, Mahari, R., Obeng-Marnu, N., Brannon, W., South, T., Kabbara J., and Pentland, S. 2024. “Data Authenticity, Consent, and Provenance for AI Are All Broken: What Will It Take to Fix Them?” An MIT Exploration of Generative AI, March. https://doi-org.ezproxy.canberra.edu.au/10.21428/e4baedd9.a650f77d.

Peer-Reviewed Paper

View Publication
View Publication
Katherine Cramer, Deb Roy. Using “Fora” to Put Public Thought In Front of Policy Makers. Conference on Democratic “Frontsliding,” 2024.

Other Publication

View Publication
View Publication
Cassandra Overney, Belén Saldías, Dimitra Dimitrakopoulou, and Deb Roy. 2024. SenseMate: An Accessible and Beginner-Friendly Human-AI Platform for Qualitative Data Analysis. In Proceedings of the 29th International Conference on Intelligent User Interfaces (IUI '24). Association for Computing Machinery, New York, NY, USA, 922–939. https://doi-org.ezproxy.canberra.edu.au/10.1145/3640543.3645194

Peer-Reviewed Paper

View Publication
View Publication
William Brannon, Wonjune Kang, Suyash Fulay, Hang Jiang, Brandon Roy, Deb Roy, and Jad Kabbara. 2024. ConGraT: Self-Supervised Contrastive Pretraining for Joint Graph and Text Embeddings. In Proceedings of TextGraphs-17: Graph-based Methods for Natural Language Processing, pages 19–39, Bangkok, Thailand. Association for Computational Linguistics.

Peer-Reviewed Paper

View Publication
View Publication
2023
Wonjune Kang, Mark Hasegawa-Johnson, and Deb Roy. End-to-End Zero-Shot Voice Conversion with Location-Variable Convolutions. Proceedings of Interspeech, 2023, pp. 2303-2307.

Peer-Reviewed Paper

View Publication
View Publication
Tiffany Chen, Cassandra Lee, Jessica R Mindel, Neska Elhaouij, and Rosalind Picard. 2023. Closer Worlds: Using Generative AI to Facilitate Intimate Conversations. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems (CHI EA '23). Association for Computing Machinery, New York, NY, USA, Article 68, 1–15. https://doi-org.ezproxy.canberra.edu.au/10.1145/3544549.3585651

Peer-Reviewed Paper

View Publication
View Publication
Longpre, S., Mahari, R., Muennighoff, N., Chen, A., Perisetla, K., Brannon, W., Kabbara, J., Villa, L., Hooker, S., “The Data Provenance Project,” in Proceedings of the 40th International Conference on Machine Learning ’23, GenLaw Workshop, PMLR, 2023. Available: https://genlaw.org/CameraReady/20.pdf

Peer-Reviewed Paper

View Publication
View Publication
Gillani, N., Beeferman, D., Vega-Pourheydarian, C., Overney, C., Van Hentenryck, P., & Roy, D. (2023). Redrawing Attendance Boundaries to Promote Racial and Ethnic Diversity in Elementary Schools. Educational Researcher. https://doi-org.ezproxy.canberra.edu.au/10.3102/0013189X231170858

Peer-Reviewed Paper

View Publication
View Publication
Suyash Fulay, Nabeel Gillani, and Deb Roy. (2023). Divergences in Following Patterns between Influential Twitter Users and Their Audiences across Dimensions of Identity. International AAAI Conference on Web and Social Media (ICWSM’23).

Peer-Reviewed Paper

View Publication
View Publication
Kessler, D.T. A Basis for Narrative Reframing Agents. CHI 2023 Extended Abstracts, April 23, 2023, Hamburg, Germany. ACM CHI Conference on Human Factors in Computing Systems. International Workshop on the Inclusive Design of Conversational User Interfaces Across Modalities and Mobilities.

Peer-Reviewed Paper

View Publication
View Publication
Prashanth Vijayaraghavan and Deb Roy. (2023). M-SENSE: Modeling Narrative Structure in Short Personal Narratives Using Protagonist's Mental Representations. In Proceedings of the 37th AAAI Conference on Artificial Intelligence (AAAI-23).

Peer-Reviewed Paper

View Publication
View Publication
N. Gillani*, D. Beeferman*, C. Overney, C. Vega-Pourheydarian, D. Roy (2023). All a-board: sharing educational data science research with school districts. In Proceedings of the Tenth ACM Conference on Learning @ Scale (L@S '23).

Peer-Reviewed Paper

View Publication
View Publication
Hughes, M., DImitrakopoulou, D., McDowell, C. & Roy, D. (2022). Digital Civic Sensemaking: Computer-Supported Participatory Sensemaking of Nuanced, Experience-Based Dialogue. Proceedings of the 56th Hawaii International Conference on System Sciences, https://hdl.handle.net/10125/102927

Peer-Reviewed Paper

View Publication
View Publication
2022
Dimitrakopoulou, D., & Theodorou, P. (2022). Affective Facilitation: A Framework for Surfacing Lived Experiences and Harnessing Creativity in Focus Group Research. International Journal of Qualitative Methods, 21.

Peer-Reviewed Paper

View Publication
View Publication
Hang Jiang, Doug Beeferman, Brandon Roy, and Deb Roy. 2022. CommunityLM: Probing Partisan Worldviews from Language Models. In Proceedings of the 29th International Conference on Computational Linguistics (COLING), pages 6818–6826, Gyeongju, Republic of Korea. International Committee on Computational Linguistics.

Peer-Reviewed Paper

View Publication
View Publication
Yining Hua*, Hang Jiang*, Shixu Lin, Jie Yang, Joseph M Plasek, David W Bates, Li Zhou. Using Twitter data to understand public perceptions of approved versus off-label use for COVID-19-related medications. Journal of the American Medical Informatics Association, Volume 29, Issue 10, October 2022, Pages 1668–1678, https://doi-org.ezproxy.canberra.edu.au/10.1093/jamia/ocac114

Peer-Reviewed Paper

View Publication
View Publication
Hope Schroeder, Doug Beeferman, Deb Roy. Comparing open-ended community dialogue with local news. Computation + Journalism Symposium, 2022.

Other Publication

View Publication
View Publication
Juliana Nazare, Anneli Woolf, Ivan Sysoev, Sarah Ballinger, Martin Saveski, Marianna Walker, Deb Roy. Technology-assisted coaching can increase engagement with learning technology at home and caregivers’ awareness of it. Computers & Education, Volume 188. 2022.

Peer-Reviewed Paper

View Publication
View Publication
Wonjune Kang. (2022). Speaker Anonymization using End-to-End Zero-Shot Voice Conversion. M.Sc. Thesis, Massachusetts Institute of Technology.

Masters Thesis

View Publication
View Publication
Dimitra Dimitrakopoulou & Seth C. Lewis (2022): The Generative Dialogue Framework and the Pursuit of Better Listening by Journalists: A Design-Centered Approach for More Constructive Conversations with Audiences. Digital Journalism, DOI:10.1080/21670811.2022.2075415

Peer-Reviewed Paper

View Publication
View Publication
Eric Chu. (2022). Learning human beliefs with language models. Ph.D. Thesis, Massachusetts Institute of Technology.

PhD Thesis

View Publication
View Publication
Sysoev, I., Gray, J.H., Fine, S., Makini, S.P., Roy, D. (2022). Child-driven, machine-guided: Automatic scaffolding of constructionist-inspired early literacy play. Computers and Education, 182, 104434

Peer-Reviewed Paper

View Publication
View Publication
Martin Saveski*, Nabeel Gillani*, Ann Yuan, Prashanth Vijayaraghavan, Deb Roy. Perspective-taking to Reduce Affective Polarization on Social Media. ICWSM'22: International AAAI Conference on Web and Social Media. 2022.

Peer-Reviewed Paper

View Publication
View Publication
Hang Jiang*, Yining Hua*, Doug Beeferman, Deb Roy. Annotating the Tweebank Corpus on Named Entity Recognition and Building NLP Models for Social Media Analysis. In Proceedings of the 13th Language Resources and Evaluation Conference (LREC). 2022. *Equal Contribution.

Peer-Reviewed Paper

View Publication
View Publication
2021
Dimitra Dimitrakopoulou. Designing Generative Dialogue Spaces to Enhance Focus Group Research: A Case Study in the Context of COVID-19 Vaccination. International Journal of Qualitative Methods. Vol 20, pp. 1-14. DOI:10.1177/16094069211066704. 2021.

Peer-Reviewed Paper

View Publication
View Publication
Margaret Hughes, Deb Roy. Keeper: A Synchronous Online Conversation Environment Informed by In-Person Facilitation Practices. In Proceedings of ACM CHI Conference on Human Factors in Computing Systems (CHI). ACM, 2021.

Peer-Reviewed Paper

View Publication
View Publication
Doug Beeferman, Hang Jiang. Topic-time Heatmaps for Human-in-the-loop Topic Detection and Tracking. In DaSH@KDD ’21: 3rd Workshop on Data Science with Humans in the Loop: KDD. 2021.

Peer-Reviewed Paper

View Publication
View Publication
Saveski, M., Roy, B., Roy, D. (2021). The Structure of Toxic Conversations on Twitter. In Proceedings of the Web Conference 2021 (WWW'21).

Peer-Reviewed Paper

View Publication
View Publication
Hang Jiang*, Sairam Gurajada*, Qiuhao Lu, Sumit Neelam, Lucian Popa, Prithviraj Sen, Yunyao Li, and Alexander Gray. 2021. LNN-EL: A Neuro-Symbolic Approach to Short-text Entity Linking. In Proceedings of the 59th Association for Computational Linguistics (ACL). 2021. *Equal Contribution.

Peer-Reviewed Paper

View Publication
View Publication
Martin Saveski, Doug Beeferman, David McClure, Deb Roy. Engaging Politically Diverse Audiences on Social Media. ICWSM'22: International AAAI Conference on Web and Social Media. 2021.

Peer-Reviewed Paper

View Publication
View Publication
Margaret Hughes and Emily S. Lin. 2021. From Alinsky to Zoom: Understanding Relational, Constituency-Based Organizing in 2021. In Companion Publication of the 2021 Conference on Computer Supported Cooperative Work and Social Computing (CSCW '21). Association for Computing Machinery, New York, NY, USA, 335–337.

Other Publication

View Publication
View Publication
Anwar, M. S., Saveski, M., Roy, D. (2021). Balanced Influence Maximization in the Presence of Homophily. In Proceedings of the International Conference on Web Search and Data Mining (WSDM'21).

Peer-Reviewed Paper

View Publication
View Publication
Sysoev, I., Gray, J. H., Fine, S., & Roy, D. Designing building blocks for open-ended early literacy software. International Journal of Child-Computer Interaction, Volume 28.

Peer-Reviewed Paper

View Publication
View Publication
N. Gillani, R. Eynon, C. Chiabaut, K. Finkel (2021). Unpacking the “Black Box” of AI in K12 education. Educational Technology & Society.

Peer-Reviewed Paper

View Publication
View Publication
Nabeel Gillani, Eric Chu, Doug Beeferman, Rebecca Eynon, Deb Roy. Parents’ Online School Reviews Reflect Several Racial and Socioeconomic Disparities in K–12 Education. AERA Open, 7(1): 1-16. 2021.

Peer-Reviewed Paper

View Publication
View Publication
Juliana Nazare. (2021). Technology-assisted coaching: A system for children's literacy learning. Ph.D. Thesis, Massachusetts Institute of Technology.

PhD Thesis

View Publication
View Publication
Nabeel Gillani. (2021). Designing for a new "ZIP code destiny." Ph.D. Thesis, Massachusetts Institute of Technology.

PhD Thesis

View Publication
View Publication
2020
Woolf, Anneli Rane, and Ageliki Nicolopoulou. The Two-Lens Approach: A holistic theoretical framework for studying the form, content, and context of children’s narratives. Narrative Inquiry 31.1 (2021): 191-213.

Peer-Reviewed Paper

View Publication
View Publication
Belen Saldias F., Deb Roy. Exploring aspects of similarity between spoken personal narratives by disentangling them into narrative clause types. In Proceedings of the 2020 ACL Workshop on Narrative Understanding, Storylines, and Events (NUSE). ACL, 2020.

Peer-Reviewed Paper

View Publication
View Publication
Wonjune Kang, Brandon C. Roy, and Wesley Chow. Multimodal Speaker Diarization of Real-World Meetings using D-Vectors with Spatial Features. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), pp. 6509-6513. IEEE, 2020.

Peer-Reviewed Paper

View Publication
View Publication
Margaret A. Hughes and Deb Roy. 2020. Keeper: An Online Synchronous Conversation Environment Informed by In-Person Facilitation Practices. In Conference Companion Publication of the 2020 on Computer Supported Cooperative Work and Social Computing (CSCW '20 Companion). Association for Computing Machinery, New York, NY, USA, 275–279. DOI: https://doi-org.ezproxy.canberra.edu.au/10.1145/3406865.3418344.

Peer-Reviewed Paper

View Publication
View Publication
Margaret Hughes. (2020). KEEPER: Online conversation support scaffolding modeled after ancient and modern social technologies. M.Sc. Thesis, Massachusetts Institute of Technology.

Masters Thesis

View Publication
View Publication
Bridgit Claire Mendler. (2020). OurStory | Dispute System Design Technology for Stakeholder Inclusion. M.Sc. Thesis, Massachusetts Institute of Technology.

Masters Thesis

View Publication
View Publication
Martin Saveski. (2020). Polarization and Toxicity in Political Discourse Online. Ph.D. Thesis, Massachusetts Institute of Technology.

PhD Thesis

View Publication
View Publication
Anneli Rane Woolf. (2020). Discovering the Meaning Behind the Story: Creating a System for Documenting and Supporting Children’s Narrative Development. Ph.D. Thesis, Massachusetts Institute of Technology.

PhD Thesis

View Publication
View Publication
Ivan Sysoev. (2020). Digital Expressive Media for Supporting Early Literacy through Child-Driven, Scaffolded Play. Ph.D. Thesis, Massachusetts Institute of Technology.

PhD Thesis

View Publication
View Publication
Nazmus Saquib. (2020). Embodied mathematics by interactive sketching. Ph.D. Thesis, Massachusetts Institute of Technology.

PhD Thesis

View Publication
View Publication
William Brannon (2020). Mapping U.S. Talk Radio: A Textual Survey at Scale. M.Sc. Thesis, Massachusetts Institute of Technology.

Masters Thesis

View Publication
View Publication
2019
Eric Chu*, Peter J. Liu*. MeanSum: A Neural Model for Unsupervised Multi-Document Abstractive Summarization. International Conference on Machine Learning. 2019. *Equal Contribution.

Peer-Reviewed Paper

View Publication
View Publication
Doug Beeferman, William Brannon, and Deb Roy. RadioTalk: A large-scale corpus of talk radio transcripts. 20th Conference of the International Speech Communication Association (INTERSPEECH 2019). 2019.

Peer-Reviewed Paper

View Publication
View Publication
Maggie Hughes, Deb Roy. (2019). RadioSilenced: A Semi-Serendipitous Hyper Local Talk Radio Exploration Tool In Proceedings of the 22nd Annual ACM Conference on Computer-Supported Cooperative Work and Social Computing. ACM, 2019. Austin, TX, USA.

Peer-Reviewed Paper

View Publication
View Publication
Shayne O’Brien. (2019). Unsupervised summarization of public talk radio. M.Sc. Thesis, Massachusetts Institute of Technology.

Masters Thesis

View Publication
View Publication
Marc Exposito Gomez. (2019). The LVN mixtapes : using augmented audiotapes for story sharing. M.Sc. Thesis, Massachusetts Institute of Technology.

Masters Thesis

View Publication
View Publication
Lauren Nicole Fratamico. (2019). Drivers of healthy online conversations about loneliness and depression. M.Sc. Thesis, Massachusetts Institute of Technology.

Masters Thesis

View Publication
View Publication
Belen Saldias F., Rosalind Picard. Tweet Moodifier: Towards giving emotional awareness to Twitter users. In Proceedings of the 8th International Conference on Affective Computing and Intelligent Interaction (ACII 2019). Cambridge, UK.

Peer-Reviewed Paper

View Publication
View Publication
Anneli Hershman*, Juliana Nazare*, Marianna Walker*, Heather Pierce, James Gray, Ivan Sysoev, Martin Saveski, Sneha Makini, Eric Chu, Marc Exposito, and Deb Roy. (2019). Extending the SLP into the Home: Examining Coach-Supported Parent-Child Engagement With an Early Literacy App. Peer-reviewed talk presented at the November 2019 ASHA Convention. *Equal contribution.

Other Publication

Rahul Bhargava*, Anna Chung*, Neil S. Gaikwad*, Alexis Hope*, Dennis Jen*, Jasmin Rubinovitz*, Belén Saldías-Fuentes*, and Ethan Zuckerman*. Gobo: A System for Exploring User Control of Invisible Algorithms in Social Media. In Proceedings of the 22nd Annual ACM Conference on Computer-Supported Cooperative Work and Social Computing. ACM, 2019. Austin, TX, USA. *Equal Contribution.

Peer-Reviewed Paper

View Publication
View Publication
2018
Anneli Hershman*, Juliana Nazare*, Ivan Sysoev, Lauren Fratamico, Juanita Buitrago, Mina Soltangheis, Sneha Makini, Eric Chu, and Deb Roy. (2018). Family Learning Coach: Engaging Families in Children’s Early Literacy Learning with Computer-Supported Tools. In Proceedings of the 26th International Conference on Computers in Education. Philippines, pp. 637-646. *Equal Contribution.

Peer-Reviewed Paper

View Publication
View Publication
Eric Chu*, Prashanth Vijayaraghavan*, and Deb Roy. Learning Personas from Dialogue with Attentive Memory Networks. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. 2018. *Equal Contribution.

Peer-Reviewed Paper

View Publication
View Publication
Nazmus Saquib, Deb Roy. (2018). Children-Centered Sensing in Early Childhood Classrooms. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems (CHI EA ‘18). New York, NY, USA, Paper LBW055, 6 pages. DOI: https://doi-org.ezproxy.canberra.edu.au/10.1145/3170427.3188559.

Peer-Reviewed Paper

View Publication
View Publication
Juliana Nazare*, Anneli Hershman*, Ivan Sysoev, Lauren Fratamico, Juanita Buitrago, Mina Soltangheis, Sneha Makini, Eric Chu, and Deb Roy. (2018). Child-Coach-Parent Network for Early Literacy Learning. In Proceedings of the Thirteenth International Conference of Learning Sciences. London, United Kingdom, pp. 1409-1410. *Equal Contribution.

Peer-Reviewed Paper

View Publication
View Publication
Anneli Hershman*, Juliana Nazare*, Jie Qi*, Deb Roy, and Mitchel Resnick. (2018). Light It Up: Using Paper Circuitry to Enhance Low-Fidelity Paper Prototypes for Children. In Proceedings of the 17th ACM Conference on Interaction Design and Children. ACM, Trondheim, Norway, pp. 365-372. *Equal Contribution.

Peer-Reviewed Paper

View Publication
View Publication
Nabeel Gillani*, Ann Yuan*, Martin Saveski, Soroush Vosoughi, and Deb Roy. (2018). Me, My Echo Chamber, and I: Introspection on Social Media Polarization. In WWW 2018: The 2018 Web Conference, April 23–27, 2018, Lyon France. *Equal Contribution.

Peer-Reviewed Paper

View Publication
View Publication
Sneha Priscilla Makini. (2018). PictureBlocks: Constructing and Deconstructing Picture-Driven Literacy Development. M.Sc. Thesis, Massachusetts Institute of Technology.

Masters Thesis

View Publication
View Publication
2017
Juliana Nazare*, Anneli Hershman*, Ivan Sysoev, and Deb Roy. (2017). Bilingual SpeechBlocks: Investigating How Bilingual Children Tinker with Words in English and Spanish. In Proceedings of the Annual Symposium on Computer-Human Interaction in Play (CHI PLAY ‘17). ACM, Amsterdam, Netherlands, pp. 183-193. *Equal Contribution.

Peer-Reviewed Paper

View Publication
View Publication
Eric Chu and Deb Roy, Audio-Visual Sentiment Analysis for Learning Emotional Arcs in Movies , 2017 IEEE International Conference on Data Mining (ICDM), New Orleans, LA, 2017, pp. 829-834.

Peer-Reviewed Paper

View Publication
View Publication
Soroush Vosoughi*, Prashanth Vijayaraghavan*, Ann Yuan, and Deb Roy. (2017). Mapping Twitter Conversation Landscapes. In Proceedings of the 11th International AAAI Conference on Weblogs and Social Media (ICWSM 2017). Montreal, Canada. *Equal Contribution.

Peer-Reviewed Paper

View Publication
View Publication
Iris Chin, Matthew S. Goodwin, Soroush Vosoughi, Deb Roy, and Letita R. Naigles. (2017). Dense home-based recordings reveal typical and atypical development of tense/aspect in a child with delayed language development. Journal of Child Language (2017): 1-34.

Peer-Reviewed Paper

View Publication
View Publication
Prashanth Vijayaraghavan, Soroush Vosoughi, Ann Yuan, and Deb Roy. (2017). TweetVista: An AI-Powered Interactive Tool for Exploring Conversations on Twitter. In Proceedings of the 22nd International Conference on Intelligent User Interfaces Companion, pp. 145-148. ACM, 2017.

Peer-Reviewed Paper

View Publication
View Publication
Misha Sra, Prashanth Vijayaraghavan, Ognjen Rudovic, Pattie Maes, and Deb Roy. (2017). DeepSpace: Mood-Based Image Texture Generation for Virtual Reality from Music. IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 10 pages.

Peer-Reviewed Paper

View Publication
View Publication
Soroush Vosoughi, Mostafa ‘Neo’ Mohsenvand, and Deb Roy. (2017). Rumor Gauge: Predicting the Veracity of Rumors on Twitter. ACM Transactions on Knowledge Discovery from Data (TKDD), 36 pages.

Peer-Reviewed Paper

View Publication
View Publication
Ivan Sysoev, Anneli Hershman, Susan Fine, Claire Traweek, and Deb Roy. (2017). SpeechBlocks: A Constructionist Early Literacy App. Proceedings of the 2017 Conference on Interaction Design and Children, 10 pages.

Peer-Reviewed Paper

View Publication
View Publication
Prashanth Vijayaraghavan*, Soroush Vosoughi*, and Deb Roy. (2017). Twitter Demographic Classification Using Deep Multi-modal Multi-task Learning. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, 6 pages. *Equal Contribution

Peer-Reviewed Paper

View Publication
View Publication
Ann Yuan*, Nabeel Gillani* (2017). Championing a spirit of genuine inquiry to combat social fragmentation. Invited paper at “Re-examining the meaning of education in an uncertain world”, Oxford Comparative and International Education, Oxford, UK. *Equal Contribution.

Other Publication

View Publication
View Publication
Anneli Hershman. (2017). They’re having fun… but are they learning? Blogpost for Media Lab Learning Initiative, cross-published on Medium.

Other Publication

View Publication
View Publication
2016
Ivan Sysoev, Anneli Hershman, Susan Fine, Deb Roy, Mina Soltangheis, and Brianne Fitzpatrick. (2016). Exploring SpeechBlocks: Piloting a Constructionist Literacy App with Preschool Children. Presented at the 2016 American Speech-Language-Hearing Association (ASHA) Conference. Philadelphia, PA.

Peer-Reviewed Paper

View Publication
View Publication
Anneli Hershman, Ivan Sysoev, and Deb Roy. (2016). SpeechBlocks: Using Literacy Apps as Building Blocks to Analyze Play. Presented at the Conference for Digital Media and Learning (DML). Irvine, CA.

Peer-Reviewed Paper

View Publication
View Publication
Prashanth Vijayaraghavan, Ivan Sysoev, Soroush Vosoughi and Deb Roy. (2016). DeepStance at SemEval-2016 Task 6: Detecting Stance in Tweets Using Character and Word-Level CNNs. In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016). San Diego, California.

Peer-Reviewed Paper

View Publication
View Publication
Soroush Vosough and Deb Roy. (2016). A Semi-automatic Method for Efficient Detection of Stories on Social Media. In Proceedings of the 10th International AAAI Conference on Weblogs and Social Media (ICWSM 2016). Cologne, Germany.

Peer-Reviewed Paper

View Publication
View Publication
Soroush Vosough*, Prashanth Vijayaraghavan* and Deb Roy. (2016). Tweet2Vec: Learning Tweet Embeddings using Character-level CNN-LSTM Encoder-Decoder. Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2016). Pisa, Italy. *Equal Contribution.

Peer-Reviewed Paper

View Publication
View Publication
Martin Saveski, Sophie Chou, and Deb Roy. (2016). Tracking the Yak: An Empirical Study of Yik Yak. In proceedings of 10th International Conference on Web Search and Data Mining (ICWSM’16). Cologne, Germany.

Peer-Reviewed Paper

View Publication
View Publication
Prashanth Vijayaraghavan*, Soroush Vosoughi*, and Deb Roy. (2016). Automatic Detection and Categorization of Election-Related Tweets. In Proceedings of the 10th International AAAI Conference on Weblogs and Social Media (ICWSM 2016). Cologne, Germany. *Equal Contribution.

Peer-Reviewed Paper

View Publication
View Publication
David Alvarez-Melis and Martin Saveski. (2016). Topic Modeling in Twitter: Aggregating Tweets by Conversations. In proceedings of 10th International Conference on Web Search and Data Mining (ICWSM’16). Cologne, Germany.

Peer-Reviewed Paper

View Publication
View Publication
Pau Perng-Hwa Kung. (2016). Detecting and Analyzing Bursty Events on Twitter. M.Sc. Thesis, Massachusetts Institute of Technology.

Masters Thesis

View Publication
View Publication
Sophie Chou. (2016). Reading Between the (Party) Lines: How Political News is Seen and Shared. M.Sc. Thesis, Massachusetts Institute of Technology.

Masters Thesis

View Publication
View Publication
Prashanth Vijayaraghavan. (2016). Automatic Identification of Representative Content on Twitter. M.Sc. Thesis, Massachusetts Institute of Technology.

Masters Thesis

View Publication
View Publication
2015
Soroush Vosoughi, Helen Zhou, and Deb Roy. (2015). Digital Stylometry: Linking Profiles Across Social Networks. In proceedings of the 7th International Conference on Social Informatics (SocInfo 2015). Beijing, China.

Peer-Reviewed Paper

View Publication
View Publication
Soroush Vosoughi and Deb Roy. (2015). A Human-Machine Collaborative System for Identifying Rumors on Twitter. In proceedings of the IEEE ICDM 2015 workshop on Event Analytics using Social Media Data (EASM). Atlantic City, New Jersey.

Peer-Reviewed Paper

View Publication
View Publication
Soroush Vosoughi, Helen Zhou, and Deb Roy. (2015). Enhanced Twitter Sentiment Classification Using Contextual Information. In proceedings of the EMNLP 2015 workshop on Computational Approaches to Subjectivity, Sentiment & Social Media Analysis (WASSA). Lisboa, Portugal.

Peer-Reviewed Paper

View Publication
View Publication
Soroush Vosoughi. (2015). Automatic Detection and Verification of Rumors on Twitter. Ph.D. Thesis, Massachusetts Institute of Technology.

PhD Thesis

View Publication
View Publication
Daniel C. Dennett and Deb Roy. (2015). Our Transparent Future. Scientific America, March 2015.

Other Publication

View Publication
View Publication

Cognitive Machines

All publications below are associated with the Professor Deb Roy's Cognitive Machines group at the Media Lab.

2015
Brandon C. Roy, Michael C. Frank, Philip DeCamp, Matthew Miller and Deb Roy. (2015). Predicting the Birth of a Spoken Word. Proceedings of the National Academy of Sciences of the United States of America (PNAS).

Peer-Reviewed Paper

View Publication
View Publication
2014
Brandon C. Roy, Soroush Vosoughi, and Deb Roy. (2014). Grounding language models in spatiotemporal context. Proceedings of Interspeech 2014. Singapore.

Peer-Reviewed Paper

View Publication
View Publication
Soroush Vosoughi. (2014). Improving automatic speech recognition through head pose driven visual grounding. Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems (CHI ’14). ACM, New York, NY, USA, 3235-3238.

Peer-Reviewed Paper

View Publication
View Publication
Iris Chin, Soroush Vosoughi, Mathew Goodwin, Deb Roy and Letitia Naigles. (2014). How the Speechome Recorder can change our understanding of developmental trajectories. Poster presented at the 13th International Congress for the Study of Child Language. Amsterdam, The Netherlands.

Peer-Reviewed Paper

View Publication
View Publication
Thomas Kollar, Stefanie Tellex, Deb Roy, and Nicholas Roy. (2014). Grounding verbs of motion in natural language commands to robots. Experimental robotics. Springer Berlin Heidelberg.

Peer-Reviewed Paper

View Publication
View Publication
2013
Iris Chin, Soroush Vosoughi, Mathew Goodwin, Deb Roy and Letitia Naigles. (2013). Dense Data Collection Through the Speechome Recorder Better Reveals Developmental Trajectories. In the Extended Abstract of the International Meeting for Autism Research (IMFAR) 2013. San Sebastián, Spain.

Peer-Reviewed Paper

View Publication
View Publication
Brandon C. Roy. (2013). The Birth of a Word. Ph.D. Thesis, Massachusetts Institute of Technology.

PhD Thesis

View Publication
View Publication
Jeff Orkin. (2013). Collective Artificial Intelligence: Simulated Role-Playing from Crowdsourced Data. Ph.D. Thesis, Massachusetts Institute of Technology.

PhD Thesis

View Publication
View Publication
2012
Soroush Vosoughi, Matthew S. Goodwin, Bill Washabaugh, and Deb Roy. (2012). A portable audio/video recorder for longitudinal study of child development. In Proceedings of the 14th ACM International Conference on Multimodal Interaction (ICMI ’12). ACM, New York, NY, USA, 193-200.

Peer-Reviewed Paper

View Publication
View Publication
Jeff Orkin and Deb Roy. (2012). Understanding Speech in Interactive Narratives with Crowdsourced Data. Proceedings of the Artificial Intelligence and Interactive Digital Entertainment Conference (AIIDE).

Peer-Reviewed Paper

View Publication
View Publication
Rony Kubat. (2012). Will They Buy? Ph.D. Thesis, Massachusetts Institute of Technology.

PhD Thesis

View Publication
View Publication
Philip DeCamp. (2012). Data Visualization in the First Person. Ph.D. Thesis, Massachusetts Institute of Technology.

PhD Thesis

View Publication
View Publication
2011
Matthew Miller. (2011). Semantic Spaces: Behavior, Language and Word Learning in the Human Speechome Corpus. M.Sc. in Media Arts and Sciences Thesis.

Masters Thesis

View Publication
View Publication
George Macaulay Shaw. (2011). A Taxonomy of Situated Language in Natural Contexts. M.Sc. in Media Arts and Sciences Thesis.

Masters Thesis

View Publication
View Publication
Naigles, L. R., Chin, I., Vosoughi, S., Goodwin, M., & Roy, D. (2011). Final Report for 3R01DC007428 – 04S1 Collaborative research supplement and infrastructure and research equipment for advancement of science .

Other Publication

View Publication
View Publication
2010
Jeff Orkin, Tynan Smith, and Deb Roy. (2010). Behavior Compilation for AI in Games. Proceedings of the Artificial Intelligence and Interactive Digital Entertainment Conference (AIIDE).

Peer-Reviewed Paper

View Publication
View Publication
Stefanie Tellex, Thomas Kollar, George Shaw, Nicholas Roy and Deb Roy. (2010). Grounding Spatial Language for Video Search. Proceedings of the Twelfth International Conference on Multimodal Interfaces (ICMI). Beijing, China.

Peer-Reviewed Paper

View Publication
View Publication
Best Student Paper Award Philip DeCamp, George Shaw, Rony Kubat and Deb Roy. (2010). An Immersive System for Browsing and Visualizing Surveillance Video. Proceedings of ACM Multimedia 2010. Florence, Italy.

Peer-Reviewed Paper

View Publication
View Publication
Albert Huang, Stefanie Tellex, Abraham Bachrach, Thomas Kollar, Deb Roy, and Nick Roy. (2010). Natural Language Command of an Autonomous Micro-Air Vehicle. Proceedings of the International Conference on Intelligent Robots and Systems (IROS). Taipei, Taiwan.

Peer-Reviewed Paper

View Publication
View Publication
Meredith Meyer, Philip DeCamp, Bridgette Hard, Dare Baldwin and Deb Roy. (2010). Assessing Behavioral and Computational Approaches to Naturalistic Action Segmentation. Proceedings of the 32nd Annual Cognitive Science Conference. Portland, Oregon.

Peer-Reviewed Paper

View Publication
View Publication
Brandon C. Roy*, Soroush Vosoughi*, and Deb Roy. (2010). Automatic Estimation of Transcription Accuracy and Difficulty. Proceedings of Interspeech 2010. Makuhari, Japan. *Equal Contribution.

Peer-Reviewed Paper

View Publication
View Publication
Hilke Reckman, Jeff Orkin and Deb Roy. (2010). Learning meanings of words and constructions, grounded in a virtual game. Proceedings of the 10th Conference on Natural Language Processing (KONVENS). Saarbrücken, Germany.

Peer-Reviewed Paper

View Publication
View Publication
Soroush Vosoughi, Brandon C. Roy, Michael C. Frank, and Deb Roy. (2010). Contributions of Prosodic and Distributional Features of Caregivers’ Speech in Early Word Learning. Proceedings of the 32nd Annual Cognitive Science Conference. Portland, Oregon.

Peer-Reviewed Paper

View Publication
View Publication
Jeff Orkin, Tynan Smith, Hilke Reckman, and Deb Roy. (2010). Semi-Automatic Task Recognition for Interactive Narratives with EAT & RUN. Proceedings of the 3rd Intelligent Narrative Technologies Workshop at the 5th International Conference on Foundations of Digital Games (FDG), Monterey, CA.

Peer-Reviewed Paper

View Publication
View Publication
Kleovoulos Tsourides. (2010). Visually Grounded Virtual Accelerometers: A Longitudinal Video Investigation of Dyadic Bodily Dynamics around the time of Word Acquisition. M.Sc. in Media Arts and Sciences Thesis.

Masters Thesis

View Publication
View Publication
Soroush Vosoughi. (2010). Interactions of caregiver speech and early word learning in the Speechome Corpus: Computational Explorations. M.Sc. Thesis, Massachusetts Institute of Technology.

Masters Thesis

View Publication
View Publication
Sophia Yuditskaya. (2010). Automatic Vocal Recognition of a Child’s Perceived Emotional State within the Speechome Corpus. M.Sc. in Media Arts and Sciences Thesis.

Masters Thesis

View Publication
View Publication
Sheng-Ying Pao. (2010). Connected Strangers: Manipulating Social Perceptions to Study Trust. M.Sc. in Media Arts and Sciences Thesis.

Masters Thesis

View Publication
View Publication
Stefanie Tellex. (2010). Natural Language and Spatial Reasoning. Ph.D. Thesis, Massachusetts Institute of Technology.

PhD Thesis

View Publication
View Publication
2009
Stefanie Tellex and Deb Roy. (2009). Grounding Spatial Prepositions for Video Search. Proceedings of the Eleventh International Conference on Multimodal Interfaces (ICMI-2009). Cambridge, MA.

Peer-Reviewed Paper

View Publication
View Publication
Deb Roy. (2009). New Horizons in the Study of Child Language Acquisition. Proceedings of Interspeech 2009. Brighton, England.

Peer-Reviewed Paper

View Publication
View Publication
Brandon C. Roy and Deb Roy. (2009). Fast transcription of unstructured audio recordings. Proceedings of Interspeech 2009. Brighton, England.

Peer-Reviewed Paper

View Publication
View Publication
Rony Kubat, Daniel Mirman and Deb Roy. (2009). Semantic context effects on color categorization. Proceedings of the 31st Annual Cognitive Science Society Meeting.

Peer-Reviewed Paper

View Publication
View Publication
Brandon C. Roy, Michael C. Frank and Deb Roy. (2009). Exploring word learning in a high-density longitudinal corpus. Proceedings of the 31st Annual Meeting of the Cognitive Science Society.

Peer-Reviewed Paper

View Publication
View Publication
Philip DeCamp and Deb Roy. (2009). A Human-Machine Collaborative Approach to Tracking Human Movement in Multi-Camera Video. Proceedings of the 2009 International Conference on Content-based Image and Video Retrieval (CIVR).

Peer-Reviewed Paper

View Publication
View Publication
Stefanie Tellex and Deb Roy. (2009). Towards Surveillance Video Search by Natural Language Query. Proceedings of the ACM International Conference on Image and Video Retrieval.

Peer-Reviewed Paper

View Publication
View Publication
Jeff Orkin and Deb Roy. (2009). Automatic Learning and Generation of Social Behavior from Collective Human Gameplay. Proceedings of the 8th International Conference on Autonomous Agents and Multiagent Systems (AAMAS).

Peer-Reviewed Paper

View Publication
View Publication
Kai-yuh Hsiao, and Soroush Vosoughi. (2009). The Object Schema Model and Situational Context .

Peer-Reviewed Paper

View Publication
View Publication
David Lazer, Alex Sandy Pentland, Lada Adamic, Sinan Aral, Albert Laszlo Barabasi, Devon Brewer, Nicholas Christakis, Noshir Contractor, James Fowler, Myron Gutmann, Tony Jebara, Gary King, Michael Macy, Deb Roy, and Marshall Van Alstyne. (2009). Life in the network: the coming age of computational social science. Science (New York, NY) 323, no. 5915.

Peer-Reviewed Paper

View Publication
View Publication
2008
Deb Roy. (2008). A Mechanistic Model of Three Facets of Meaning. Symbols, Embodiment, and Meaning, de Vega, Glenberg, and Graesser, eds.

Peer-Reviewed Paper

View Publication
View Publication
Michael Fleischman and Deb Roy. (2008). Grounded Language Modeling for Automatic Speech Recognition of Sports Video. HLT/NAACL. Columbus, OH.

Peer-Reviewed Paper

View Publication
View Publication
Kai-yuh Hsiao, Stefanie Tellex, Soroush Vosoughi, Rony Kubat, and Deb Roy. (2008). Object Schemas for Grounding Language in a Responsive Robot. Connection Science 20, 4 (Dec.2008), 253-276.

Peer-Reviewed Paper

View Publication
View Publication
Kai-yuh Hsiao, Soroush Vosoughi, Stefanie Tellex, Rony Kubat, and Deb Roy. (2008). Object Schemas for Responsive Robotic Language Use. Proceedings of the 3rd ACM/IEEE International Conference on Human-Robot Interaction.

Peer-Reviewed Paper

View Publication
View Publication
Michael Ben Fleischman. (2008). Grounding Language in Events. Ph.D. Thesis, Massachusetts Institute of Technology.

PhD Thesis

View Publication
View Publication
2007
Jeff Orkin and Deb Roy. (2007). The Restaurant Game: Learning Social Behavior and Language from Thousands of Players Online. Journal of Game Development, 3(1), 39-60.

Peer-Reviewed Paper

View Publication
View Publication
Rony Kubat, Philip DeCamp, Brandon Roy, and Deb Roy. (2007). TotalRecall: Visualization and Semi-Automatic Annotation of Very Large Audio-Visual Corpora. Ninth International Conference on Multimodal Interfaces (ICMI 2007).

Peer-Reviewed Paper

View Publication
View Publication
Michael Fleischman and Deb Roy. (2007). Unsupervised Content-Based Indexing of Sports Video Retrieval. 9th ACM Workshop on Multimedia Information Retrieval (MIR). Augsburg, Germany.

Peer-Reviewed Paper

View Publication
View Publication
Michael Fleischman, Brandon Roy, and Deb Roy. (2007). Temporal Feature Induction for Baseball Highlight Classification. ACM Multimedia Conference. Augsburg, Germany.

Peer-Reviewed Paper

View Publication
View Publication
Philip DeCamp. (2007). HeadLock: Wide-Range Head Pose Estimation for Low Resolution Video. M.Sc. in Media Arts and Sciences Thesis.

Masters Thesis

View Publication
View Publication
Philipp Robbel. (2007). Exploiting Object Dynamics for Recognition and Control. M.Sc. in Media Arts and Sciences Thesis.

Masters Thesis

View Publication
View Publication
Brandon Roy. (2007). Human-Machine Collaboration for Rapid Speech Transcription. M.Sc. in Media Arts and Sciences Thesis.

Masters Thesis

View Publication
View Publication
Jeff Orkin. (2007). Learning Plan Networks in Conversational Video Games. M.Sc. in Media Arts and Sciences Thesis.

Masters Thesis

View Publication
View Publication
Kai-yuh Hsiao. (2007). Embodied Object Schemas for Grounding Language Use. Ph.D. Thesis, Massachusetts Institute of Technology.

PhD Thesis

View Publication
View Publication
2006
Michael Fleischman, Philip DeCamp, and Deb Roy. (2006). Mining Temporal Patterns of Movement for Video Content Classification. Proceedings of the 8th ACM SIGMM International Workshop on Multimedia Information Retrieval.

Peer-Reviewed Paper

View Publication
View Publication
Deb Roy, Rupal Patel, Philip DeCamp, Rony Kubat, Michael Fleischman, Brandon Roy, Nikolaos Mavridis, Stefanie Tellex, Alexia Salata, Jethran Guinness, Michael Levit, Peter Gorniak. (2006). The Human Speechome Project. Proceedings of the 28th Annual Cognitive Science Conference.

Peer-Reviewed Paper

View Publication
View Publication
Peter Gorniak and Deb Roy. (2006). Perceived Affordances as a Substrate for Linguistic Concepts. Twenty-eighth Annual Meeting of the Cognitive Science Society, 6 pages.

Peer-Reviewed Paper

View Publication
View Publication
Peter Gorniak, Jeff Orkin, and Deb Roy. (2006). Speech, Space and Purpose: Situated Language Understanding in Computer Games. Twenty-eighth Annual Meeting of the Cognitive Science Society Workshop on Computer Games.

Peer-Reviewed Paper

View Publication
View Publication
Nikolaos Mavridis and Deb Roy. (2006). Grounded Situation Models for Robots: Where Words and Percepts Meet. Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

Peer-Reviewed Paper

View Publication
View Publication
Dong Zhang, Daniel Gatica-Perez, Deb Roy, Samy Bengio. (2006). Modeling Interactions from Email Communication. IEEE International Conference on Multimedia & Expo (ICME).

Peer-Reviewed Paper

View Publication
View Publication
Stefanie Tellex and Deb Roy. (2006). Spatial Routines for a Simulated Speech-Controlled Vehicle. Proceedings of Human Robot Interaction Conference 2006 (HRI-2006).

Peer-Reviewed Paper

View Publication
View Publication
2005
Philip DeCamp, Amber Frid-Jimenez, Jethran Guiness and Deb Roy. (2005). Gist Icons: Seeing Meaning in Large Bodies of Literature. IEEE Info Visualization 2005 Conference.

Peer-Reviewed Paper

View Publication
View Publication
Kai-yuh Hsiao and Deb Roy. (2005). A Habit System for an Interactive Robot. AAAI Fall Symposium 2005: From Reactive to Anticipatory Cognitive Embodied Systems.

Peer-Reviewed Paper

View Publication
View Publication
Peter Gorniak and Roy (2005). Probabilistic Grounding of Situated Speech using Plan Recognition and Reference Resolution. Seventh International Conference on Multimodal Interfaces (ICMI 2005). Best Paper Award.

Peer-Reviewed Paper

View Publication
View Publication
Michael Fleischman and Deb Roy. (2005). Intentional Context in Situated Language Learning. Ninth Conference on Computational Natural Language Learning.

Peer-Reviewed Paper

View Publication
View Publication
Nick Mavridis and Deb Roy. (2005). Grounded Situation Models for Robots: Bridging language, Perception, and Action. AAAI-05 Workshop on Modular Construction of Human-Like Intelligence.

Peer-Reviewed Paper

View Publication
View Publication
Deb Roy. (2005). Semiotic Schemas: A Framework for Grounding Language in Action and Perception. Artificial Intelligence, 167(1-2):170-205.

Peer-Reviewed Paper

View Publication
View Publication
Michael Fleischman and Deb Roy. (2005). Why are verbs harder to learner than nouns? Initial insights from a computational model of situated word learning. 27th Annual Meeting of the Cognitive Science Society.

Peer-Reviewed Paper

View Publication
View Publication
Deb Roy. (2005). Grounding words in perception and action: computational insights. Trends in Cognitive Science, 9(8), 389-396.

Peer-Reviewed Paper

View Publication
View Publication
Kai-yuh Hsiao, Peter Gorniak, and Deb Roy. (2005). NetP: A Network API for Building Heterogeneous Modular Intelligent Systems. Proceedings of AAAI 2005 Workshop in Modular Construction of Human-Like Intelligence.

Peer-Reviewed Paper

View Publication
View Publication
Peter Gorniak and Deb Roy. (2005). Speaking with your Sidekick: Understanding Situated Speech in Computer Role Playing Games. Proceedings of Artificial Intelligence and Interactive Digital Entertainment, 2005.

Peer-Reviewed Paper

View Publication
View Publication
Deb Roy and Niloy Mukherjee. (2005). Towards Situated Speech Understanding: Visual Context Priming of Language Models. Computer Speech and Language, 19(2), pages 227-248.

Peer-Reviewed Paper

View Publication
View Publication
Deb Roy and Ehud Reiter. (2005). Connecting Language to the World. Artificial Intelligence, 167(1-2): 1-12.

Peer-Reviewed Paper

View Publication
View Publication
Anna V. Fisher, Michael Fleischman, Deb Roy, and Vladimir M. Sloutsky. (2005). Effects of Category Labels on Induction and Visual Processing: Support or Interference? Twenty-seventh Annual Meeting of the Cognitive Science Society, 6 pages.

Peer-Reviewed Paper

View Publication
View Publication
Deb Roy. (2005). Grounding language in the world: Schema theory meets semiotics. Special Issue of Artificial Intelligence Journal: Connecting Language to the World.

Peer-Reviewed Paper

View Publication
View Publication
Andre Ribeiro. (2005). Graph Dynamics: Learning and Representation. M.Sc. in Media Arts and Sciences Thesis.

Masters Thesis

View Publication
View Publication
Charles Kehoe. (2005). Indexical Grounding for a Mobile Robot. M.Eng. EECS Thesis.

Masters Thesis

View Publication
View Publication
Peter Gorniak. (2005). The Affordance-Based Concept. Ph.D. Thesis, Massachusetts Institute of Technology.

PhD Thesis

View Publication
View Publication
2004
Joshua Juster and Deb Roy. (2004). Elvis: Situated Speech and Gesture Understanding for a Robotic Chandelier. Proc. Int. Conf. Multimodal Interfaces.

Peer-Reviewed Paper

View Publication
View Publication
Deb Roy, Yair Ghitza, Jeff Bartelma, and Charlie Kehoe. (2004). Visual Memory Augmentation: Using Eye Gaze as an Attention Filter. Proceedings of the IEEE International Symposium on Wearable Computers.

Peer-Reviewed Paper

View Publication
View Publication
Deb Roy, Kai-Yuh Hsiao, and Nikolaos Mavridis. (2004). Mental Imagery for a Conversational Robot. IEEE Transactions on Systems, Man, and Cybernetics, Part B, 34(3), 1374-1383.

Peer-Reviewed Paper

View Publication
View Publication
Peter Gorniak and Deb Roy. (2004). Grounded Semantic Composition for Visual Scenes. Journal of Artificial Intelligence Research, Volume 21, pages 429-470.

Peer-Reviewed Paper

View Publication
View Publication
Deb Roy. (2004). 10x: Human-machine Symbiosis. BT Technology Journal, 22(4): 121-124.

Peer-Reviewed Paper

View Publication
View Publication
Rosalind W Picard, Seymour Papert, Walter Bender, Bruce Blumberg, Cynthia Breazeal, David Cavallo, Tod Machover, Mitchel Resnick, Deb Roy, Carol Strohecker (2004). Affective learning—a manifesto. BT Technology Journal, 22(4): 253-269.

Peer-Reviewed Paper

View Publication
View Publication
Peter Gorniak, Deb Roy. (2004). BISHOP|BLENDER: Spatially Grounded Language Understanding in 3D Modelling Software. In Proceedings of the NAACL, 2 pages.

Peer-Reviewed Paper

View Publication
View Publication
Rupal Patel, Sam Pilato, and Deb Roy. (2004). Beyond Linear Syntax: An Image-Oriented Communication Aid. Journal of Assistive Technology Outcomes and Benefits, 1(1): 57-66

Peer-Reviewed Paper

View Publication
View Publication
Jeff Orkin, Deb Roy. (2004). Capturing and generating social behavior with the restaurant game. Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems.

Peer-Reviewed Paper

View Publication
View Publication
2003
Niloy Mukherjee and Deb Roy. (2003). A Visual Context-Aware Multimodal System for Spoken Language Processing. Proc. Eurospeech, 6 pages.

Peer-Reviewed Paper

View Publication
View Publication
Peter Gorniak and Deb Roy. (2003). Augmenting User Interfaces with Adaptive Speech Commands. In Proceedings of the International Conference for Multimodal Interfaces.

Peer-Reviewed Paper

View Publication
View Publication
Peter Gorniak and Deb Roy. (2003). A Visually Grounded Natural Language Interface for Reference to Spatial Scenes. In Proceedings of the International Conference for Multimodal Interfaces.

Peer-Reviewed Paper

View Publication
View Publication
Brian Whitman, Deb Roy, Barry Vercoe. (2003). Learning Word Meanings and Descriptive Parameter Spaces from Music. In Proceedings of the HLT-NAACL03 workshop on Learning Word Meaning from Non-Linguistic Data.

Peer-Reviewed Paper

View Publication
View Publication
Kai-yuh Hsiao, Nikolaos Mavridis, Deb Roy. Coupling Perception and Simulation: Steps Towards Conversational Robotics. (2003). The Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems .

Peer-Reviewed Paper

View Publication
View Publication
Deb Roy, Kai-yuh Hsiao, Nikolaos Mavridis. (2003). Conversational Robots: Building Blocks for Grounding Word Meanings. In Proceedings of the HLT-NAACL03 Workshop on Learning Word Meaning from Non-Linguistic Data.

Peer-Reviewed Paper

View Publication
View Publication
Deb Roy. (2003). Grounded Spoken Language Acquisition: Experiments in Word Learning. IEEE Transactions on Multimedia, 5(2): 197-209.

Peer-Reviewed Paper

View Publication
View Publication
Niloy Mukherjee. (2003). Spontenous Speech Recognition Using Visual Context-Aware Language Models. M.Sc. in Media Arts and Sciences Thesis.

Masters Thesis

View Publication
View Publication
Sheel Sanjay Dhande. (2003). A Computational Model to Connect Gestalt Perception and Natural Language. M.Sc. in Media Arts and Sciences Thesis.

Masters Thesis

View Publication
View Publication
2002
Deb Roy, Peter Gorniak, Niloy Mukherjee, and Josh Juster. (2002). A Trainable Spoken Language Understanding System for Visual Object Selection. In Proceedings of the International Conference of Spoken Language Processing.

Peer-Reviewed Paper

View Publication
View Publication
Deb Roy. (2002). A Trainable Visually-Grounded Spoken Language Generation System. In Proceedings of the International Conference of Spoken Language Processing.

Peer-Reviewed Paper

View Publication
View Publication
Deb Roy. (2002). Learning Words and Syntax for a Visual Description Task. Computer Speech and Language.

Peer-Reviewed Paper

View Publication
View Publication
Deb Roy, Kai-Yuh Hsiao, Peter Gorniak, and Niloy Mukherjee. (2002). Grounding natural spoken language semantics in visual perception and motor control. AAAI Technical Report FS-02-03.

Peer-Reviewed Paper

View Publication
View Publication
Deb Roy and Alex Pentland. (2002). Learning Words from Sights and Sounds: A Computational Model. Cognitive Science, 26(1), 113-146.

Peer-Reviewed Paper

View Publication
View Publication
Ewa Dominowska, Deb Roy and Rupal Patel. (2002). An Adaptive Context-Sensitive Communication Aid. Proceedings for the 17th Annual International Conference “Technology and Persons with Disabilities”.

Peer-Reviewed Paper

View Publication
View Publication
Deb Roy. (2002). A System that Learns to Describe Objects in Visual Scenes. Proc. Seventh International Conference on Spoken Language Processing, 4 pages.

Peer-Reviewed Paper

View Publication
View Publication
Ewa Dominowska. (2002). A Communication Aid with Context-Aware Vocabulary Prediction. M.Eng. EECS Thesis.

Masters Thesis

View Publication
View Publication
Norimasa Yoshida. (2002). Automatic Utterance Segmentation in Spontaneous Speech. M.Eng. EECS Thesis.

Masters Thesis

View Publication
View Publication
2001
Deb Roy. (2001). Situation-Aware Spoken Language Processing. Royal Institute of Acoustics Workshop on Innovation in Speech Processing, Stratford-upon-Avon, England.

Peer-Reviewed Paper

View Publication
View Publication
Ben Yoder. (2001). Spontaneous Speech Recognition Using Hidden Markov Models. M.Eng. EECS Thesis.

Masters Thesis

View Publication
View Publication
2000
Deb Roy. (2000). Integration of Speech and Vision using Mutual Information. Int. Conf. Acoustics, Speech and Signal Processing.

Peer-Reviewed Paper

View Publication
View Publication
Deb Roy. (2000). Grounded Speech Communication. Proceedings of the International Conference on Spoken Language Processing, 5 pages.

Peer-Reviewed Paper

View Publication
View Publication
Deb Roy. (2000). A computational model of word learning from multimodal sensory input. Proceedings of the International Conference of Cognitive Modeling, 8 pages.

Peer-Reviewed Paper

View Publication
View Publication
1999
Deb Roy, Bernt Schiele, and Alex Pentland. (1999). Learning Audio-Visual Associations using Mutual Information. International Conference on Computer Vision, Workshop on Integrating Speech and Image Understanding.

Peer-Reviewed Paper

View Publication
View Publication
Deb Roy. (1999). Learning Words from Sights and Sounds: A Computational Model. Ph.D. in Media Arts and Sciences, MIT.

PhD Thesis

View Publication
View Publication
1998
Deb Roy and Alex Pentland. (1998). A Phoneme Probability Display for Individuals with Hearing Disabilities. Proceedings of the ACM Conference On Assistive Technologies, 4 pages.

Peer-Reviewed Paper

View Publication
View Publication
Deb Roy and Alex Pentland. (1998). Learning words from natural audio-visual input. Fifth International Conference on Spoken Language Processing (ICSLP), 5 pages.

Peer-Reviewed Paper

View Publication
View Publication
Rupal Patel and Deb Roy. (1998). Teachable interfaces for individuals with dysarthric speech and severe physical disabilities. Proceedings of the AAAI Workshop on Integrating Artificial Intelligence and Assistive Technology, 8 pages.

Peer-Reviewed Paper

View Publication
View Publication
Deb Roy and Alex Pentland. (1998). Word Learning in a Multimodal Environment , Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, 4 pages.

Peer-Reviewed Paper

View Publication
View Publication
1997
Deb Roy and Carl Malamud. (1997). Integration of a large text and audio corpus using speaker identification. Proceedings of the AAAI Spring Symposium on the Intelligent Integration and Use of Text, Image, Video and Audio Corpora.

Peer-Reviewed Paper

View Publication
View Publication
Deb Roy and Alex Pentland. (1997). Multimodal adaptive interfaces. MIT Media Laboratory Perceptual Computing Technical Report No. 438.

Peer-Reviewed Paper

View Publication
View Publication
Deb Roy and Carl Malamud. (1997). Speaker identification based test to audio alignment for an audio retrieval system. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, Munich, Vol. 2, pp. 1099-1103.

Peer-Reviewed Paper

View Publication
View Publication
Deb Roy. (1997). Speaker indexing using neural network clustering of vowel spectra. International Journal of Speech Technology, 1(2): 143-149.

Peer-Reviewed Paper

View Publication
View Publication
Deb Roy, Nitin Sawhney, Chris Schmandt, and Alex Pentland. (1997). Wearable audio computing: A survey of interaction techniques. Perceptual Computing Technical Report No. 434, 9 pages.

Peer-Reviewed Paper

View Publication
View Publication
1996
Chris Schmandt and Deb Roy. (1996). Using acoustic structure in a hand-held audio playback device. IBM Systems Journal, 35: (3-4).

Peer-Reviewed Paper

View Publication
View Publication
Deb Roy, Alex Pentland. (1996). Automatic spoken affect classification and analysis. Proc. Second International Conference on Automatic Face and Gesture Recognition, 6 pages.

Peer-Reviewed Paper

View Publication
View Publication
Deb Roy and Chris Schmandt. (1996). NewsComm: A Hand-Held Interface for Interactive Access to Structured Audio. Proceedings of the ACM Conference on Computer Human Interaction, 9 pages.

Peer-Reviewed Paper

View Publication
View Publication
1995
Deb Roy. (1995). NewsComm: A Hand-Held Device for Interactive Access to Structured Audio. M.Sc. in Media Arts and Sciences, MIT.

Masters Thesis

View Publication
View Publication
1991
B. N. Chodirker, D. Roy, C. R. Greenberg, M. Cheang, J. A. Evans, & M. H. Reed. (1991). Computer assisted analysis of hand radiographs in infantile hypophosphatasia carriers. Pediatric Radiology, 21(3): 216-219.

Peer-Reviewed Paper

View Publication
View Publication