Dinesh Babu Jayagopi

NIRF
IQAC

Associate Professor & Faculty-in-charge Internships & Placements

jdinesh@iiitb.ac.in

Education : Ph.D. (École Polytechnique Fédérale de Lausanne)

Dr. Dinesh Babu Jayagopi is currently an Associate Professor at IIIT Bangalore, where he heads the Multimodal Perception Lab. His research interests are in Audio-Visual Signal Processing, Machine Learning, and Social Computing. He obtained his doctorate from Ecole Polytechnic Federale Lausanne (EPFL), Switzerland, beginning of 2011. He received the Outstanding paper award in the International Conference on Multimodal Interaction (ICMI), 2012, Idiap Best PhD student research award for the year 2009. More recently, his PhD student's work was nominated for Best student paper in ICMI 2016 Tokyo, Japan. Another work received Best student paper award at MedPRAI 2018. He also received Department of Science and Technology (DST) Young Scientist Start up Grant in 2016. In the past, he has successfully collaborated and executed sponsored research projects with CAIR, DRDO (Defence Research and Development Organization) and NI Systems. He was a visiting professor at University of Lausanne in summer 2019

Research Interests
Selected Publications
Teaching
Research & Consulting

Audio Visual Signal, Processing, Applied Machine Learning, Machine Perception, Applied Signal Processing, Social Computing, Human Robot Interaction, Multimedia Analysis, Behavior Analysis

Book Chapter :

O. Aran, D. Gatica Perez, and D. Jayagopi . Understanding small groups , Social Signal Processing, Cambridge University Press 2017. Editors J. Burgoon A. Vinciarelli, M. Pantic and N. Magnenat Thalmann. [Collaboration with EPFL]

Journal

E. Kleinlogel, L. Renier, M. Schmid Mast, D. Jayagopi and K. Shubham. From Low Invasiveness to High Control: How Artificial Intelligence Allows to Generate a Large Pool of Standardized Corpora at a Lesser Cost, accepted for publication in METHODS article, Front. Comput. Sci., Sec. Human-Media Interaction, Volume 5 – 2023
A. Sharma and D. Jayagopi. Full-Page Handwriting Recognition and Automated Essay Scoring for in-the-wild Essays, published in Multimedia Tools and Applications, March 2023 (Impact Factor: 2.6). According to Google Scholar metrics, this venue is the best in “Multimedia” category.
C. Thomas and D. Jayagopi. Predicting Presentation Skill of a Speaker Using Automatic Speaker and Audience Measurement, published in IEEE Transactions on Learning Technologies 2022 (Impact Factor: 3.7).
C. Thomas and D. Jayagopi. Automatic Prediction of Presentation Style and Student Engagement from Videos, accepted in Computers and Education: Artificial Intelligence Journal 2022 (Impact factor of 8.5). According to Google Scholar metrics, this venue is the best in “Educational Technology” category.
A. Sharma and D. Jayagopi. Towards efficient unconstrained handwriting recognition using Dilated Temporal Convolution Network, accepted for publication in Expert Systems with Applications, Feb 2021 (Impact Factor: 6.9).
Manjunath K E, Raghavan, S. K., K S Rao, D. Jayagopi, Ramasubramanian V Approaches for multilingual phone recognition in code-switched and non-code-switched scenarios using Indian languages, published in ACM Transactions on Asian and Low-resource Language Information Processing, 2021(Impact Factor: 1.4)
P. Rao, M. Agnihotri, D. Jayagopi. Improving Asynchronous Interview Interaction with Follow-up Question Generation, accepted for publication in The International Journal of Interactive Multimedia and Artificial Intelligence (IJIMAI), 2020 (Impact Factor: 2.56).
A. Sharma and D. Jayagopi. Towards efficient unconstrained handwriting recognition using Dilated Temporal Convolution Network, accepted for publication in Expert Systems with Applications, Feb 2021 (Impact Factor: 5.4).
Manjunath K E, D. Jayagopi, K S Rao, Ramasubramanian V Articulatory-feature-based methods for performance improvement of Multilingual Phone Recognition Systems using Indian languages, published in S?dhan? 45 (1), 1-14, 2020(Impact Factor: 0.77).
S. Rasipuram and D. Jayagopi. Automatic multimodal assessment of soft skills in social interactions: a review , published in Springer Multimedia Tools and Applications 2020 (Impact Factor: 2.6).
Manjunath K E, D. Jayagopi, K S Rao, Ramasubramanian V Development and Analysis of Multilingual Phone Recognition Systems using Indian Languages, accepted for publication in Springer International Journal of Speech Technology (Impact Factor: 0.9).
S. Rasipuram and D. Jayagopi. Automatic assessment of communication skill in interview-based interactions, accepted for publication in Springer Multimedia Tools and Applications 2018 (Impact Factor: 1.5).
D. Sanches-Cortes, O. Aran, D. Jayagopi, M. Schmid-Mast, and D. Gatica-Perez. Emergent leaders through looking and speaking: from audio-visual data to multimodal recognition, accepted for publication in Springer Journal on Multimodal User Interfaces, 2012.
D. Jayagopi, T. Kim, Alex (Sandy) Pentland, and D. Gatica-Perez. Privacy-sensitive recognition of group conversational context with sociometers, Springer Multimedia Systems, 2012, Volume 18, Number 1, Pages 3-14.
D. Jayagopi, and D. Gatica-Perez. Mining group nonverbal conversational patterns using probabilistic topic models, IEEE Transactions on Multimedia, 2010, 12(8), 790-802.
D. Jayagopi, H. Hung, C. Yeo, and D. Gatica-Perez. Modeling dominance in group conversations from nonverbal activity cues, IEEE Transactions on Audio, Speech and Language Processing, 2009,17(3), 501-513.

Conference

Kotlo Budde, Ravi Kiran Reddy, et al. "One shot learning in StyleALAE: Preserving facial identity during semantic modification." Proceedings of the Thirteenth Indian Conference on Computer Vision, Graphics and Image Processing. 2022.
SS Mishra, K. Shubham, and DB Jayagopi. "A Hybrid Rigid and Non-Rigid Motion Approximation for Generating Realistic Listening Behavior Videos." Proceedings of the Thirteenth Indian Conference on Computer Vision, Graphics and Image Processing. 2022.
K Shubham, LNN Venkatesan, DB Jayagopi, R Tumuluri. Multimodal Embodied Conversational Agents: A discussion of architectures, frameworks and modules for commercial applications, Proceedings of IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR), 2022.
K Shubham, A Mukherjee, DB Jayagopi. Review of realistic behavior and appearance generation in embodied conversational agents: A comparison between traditional and modern approaches, Proceedings of International Conference on Multimodal Interaction(ICMI), 2022.
J Berlin, D Pandian, SS Rajagopalan, DB Jayagopi. Detecting A Child’s Stimming Behaviours for Autism Spectrum Disorder Diagnosis using Rgbpose-Slowfast Network, published in the International Conference on Image Processing (ICIP), 2022.
G Gopan, N Sinha, D Babu. Inter-channel Covariance Matrices Based Analyses of EEG Baselines, published in the 30th European Signal Processing Conference (EUSIPCO) 2022.
PSB Rao, DB Jayagopi, M Cherubini. Explainability in Automated Training and Feedback Systems, published in the ACM CHI’22 Workshop on Human-Centered Explainable AI (HCXAI), May 12–14, 2022, New Orleans, LA, USA .
C Thomas, S Purvaj, DB Jayagopi. Student Engagement from Video using Unsupervised Domain Adaptation, published in the Proceedings of IMPROVE 2022.
Jayagopi D. Multimodal Analysis and Synthesis for Conversational Research, In Companion Publication of the 2021 International Conference on Multimodal Interaction (pp. 400-401).
Krishna S, Janmesh U, Jayagopi D. GAN Based Indian Sign Language Synthesis, accepted to the proceedings of 12th Indian Conference on Computer Vision, Graphics and Image Processing (ICVGIP), Jodhpur, India, Dec 2021.
Krishna S, Vignesh V, Jayagopi D. Sign Language Animation Through 3D Pose Lifting, accepted to Crossmodal Social Animation (XSAnim) workshop, In Proceedings of the 2021 International Conference on Computer Vision (ICCV), Montreal
Agnihotri, M., Pooja Rao, S. B., Jayagopi, D. B., Hebbar, S., Rasipuram, S., Maitra, A., & Sengupta, S. Towards Generating Topic-Driven and Affective Responses to Assist Mental Wellness. In Pattern Recognition, ICPR International Workshops and Challenges: Virtual Event, January 10?15, 2021, Proceedings, Part II (pp. 129-143). Collaboration with Accenture Labs
Shubham, K.,G. Venkatesh, R. Sachdev, A. Akshi, Jayagopi, D. B.and S.Gopalakrishnan.Learning a Deep Reinforcement Learning Policy Over the Latent Space of a Pre-trained GAN for Semantic Age Manipulation, In Proceedings of the 2021 International Joint Conference on Neural Networks (IJCNN).
Krishna, S., Jindal, A. R., & Jayagopi, D. Virtual Indian Sign Language Interpreter., In Proceedings of the 2020 4th International Conference on Vision, Image and Signal Processing (pp. 1-5)
K. Shubham, E. Kleinlogel, A. Butera, MS Mast and D. Jayagopi Conventional and Non-conventional Job Interviewing Methods: A Comparative Study in Two Countries, published in International Conference on Multimodal Interaction(ICMI), 2020.
S. Rasipuram, BN Sai, D. Jayagopi and A. Mitra. A Using Deep 3D Features and an LSTM Based Sequence Model for Automatic Pain Detection in the Wild, International Workshop on Automated Assessment for Pain, 15th IEEE International Conference on Automatic Face and Gesture, 2020.
P. Rao, M. Agnihotri, D. Jayagopi. Incorporating Automatic Follow-up Question Generation for Asynchronous Interviews, published in ECAI IntelliLang Workshop, 2020.
S. Rasipuram and D. Jayagopi. A Comprehensive Evaluation of Audio-Visual Behavior in Various Modes of Interviews in the Wild, accepted for oral presentation at the 12th International Conference on PErvasive Technologies Related to Assistive Environments (PETRA), June 2019, Rhodes, Greece.
C. Thomas, D. Gatica-Perez and D. Jayagopi. BookTubing Across Regions: Examining Differences based on Nonverbal and Verbal Cues, accepted for publication in ACM International Conference on Interactive Experiences for Television and Online Video (TVX), June 2019, Manchester, UK. Collaboration with IDIAP, EPFL
S. Raju, N. Duvvuru and D. Jayagopi. Improving Adversarial Images Using Activation Maps, accepted for publication in 8th Joint International Information Technology and Artificial Intelligence Conference (IEEE ITAIC 2019), May 2019, Chongqing,China.
A. Sharma, D. Jayagopi. Handwritten Essay Grading on Mobiles using MDLSTM Model and Word Embeddings, published in the 11th Indian Conference on Computer Vision, Graphics and Image Processing (ICVGIP), Hyderabad, India, Dec 2018.
C. Nagraj, H. Jain, D. Jayagopi. Rash Driving Detection using a Frontal View Camera in Cars, published in the 11th Indian Conference on Computer Vision, Graphics and Image Processing (ICVGIP), Hyderabad, India, Dec 2018. Collaboration with Samsung R&D
D. Gatica-Perez, D. Cortez, T. Minh-Do, K. Otsuka, D. Jayagopi. Vlogging Over Time: Longitudinal Impressions and Behavior in YouTube, accepted in the 17th International Conference on Mobile and Ubiquitous Multimedia (MuM), Cairo, Egypt, Nov 2018.
N. Nair, C. Thomas and D. Jayagopi. Human Activity Recognition Using Temporal Convolutional Network., published in the 5th International Workshop on Sensor-based Activity Recognition and Interaction (iWOAR) 2018, Berlin Sep 2018.
K E Manjunath, K S Rao, D. Jayagopi, V. Ramasubramanian Indian languages ASR: A multilingual phone recognition framework with IPA based common phone-set, predicted articulatory features and feature fusion., published in the Proceedings of Interspeech, Sep 2018, Hyderabad.
A. Gupta and D. Jayagopi. Unsupervised Speaker Cue Usage Detection in Public Speaking Videos, accepted in the 1st International Workshop on Vision for Interaction and Behaviour undErstanding (VIBE 2018), BMVC, New Castle, Sep 2018.
A. Sharma and D. Jayagopi. Automated Grading of Handwritten Essays , published in 16th International Conference on Frontiers in Handwriting Recognition (ICFHR-2018), to be held August 5-8, 2018 in Niagara Falls, USA.
S. Praneeth, U. Agrawal and D. Jayagopi. Human Weapon-Activity Recognition in Surveillance Videos Using Structural-RNN, published in the 2nd Mediterranean Conference on Pattern Recognition and Artificial Intelligence (MedPRAI?2018), to be held in Rabat, Morocco, on March 27-28, 2018. Best student paper award!!
A. Surendranath and D. Jayagopi. Curriculum Learning for Depth Estimation With Deep Convolutional Neural Networks, published in the 2nd Mediterranean Conference on Pattern Recognition and Artificial Intelligence (MedPRAI?2018), to be held in Rabat, Morocco, on March 27-28, 2018.
P. Sudheendra and D. Jayagopi. Genre Linked Automated Assessment and feedback of Photographs based on Visual Aesthetics, published in the International Conference on Image Processing Tools and Applications (IPTA), Montreal, Nov 2017.
P. Rao, S. Rasipuram, R. Das, D. Jayagopi. Automatic Assessment of Communication Skill in Non-Conventional Interview Settings: A Comparative Study, published in International Conference on Multimodal Interaction(ICMI), Glasgow, Scotland, Nov 2017 Pooja R gets Travel Grant!!.
C.Thomas, D. Jayagopi. Predicting Student Engagement in Classrooms Using Facial Behavioral Cues, published in ICMI MIE Workshop Glasgow, Scotland, Nov 2017.
S.K. Nambiar, R. Das, S. Rasipuram, D. Jayagopi. Automatic Generation of Actionable Feedback Towards Improving Social Competency in Job Interviews., published in ICMI MIE Workshop Glasgow, Scotland, Nov 2017.
S. Rasipuram, R. Das, P. Rao, D. Jayagopi. Online Peer-to-peer Discussions: A Platform for Automatic Assessment of Communication Skill, published in Context Based Affect Recognition Workshop, ACII, San Antonio, Texas, USA, Oct 2017.
K. E. Manjunath, K.S. Rao and D. Jayagopi. Development of Multilingual Phone Recognition System for Indian Languages IEEE SPICES Kochi, Aug 2017.
M.K. Prabhu and D. Jayagopi. MERS ? A Multimodal Real Time Emotion Recognition System published in the Second International Workshop on Pattern Recognition (IWPR) Singapore, May 2017.
P. Dhananjay, D. Varshneya and D. Jayagopi. Restaurant Attribute classification using Deep Learning published in the Proceedings of INDICON, Bangalore, Dec 2016.
S. Rasipuram, P. Rao, and D. Jayagopi. Automatic Prediction of Fluency in Interface-based Interviews published in the Proceedings of INDICON, Bangalore, Dec 2016.
M. Madhyastha and D. Jayagopi. A Low Cost Personalised Robot Language Tutor with Perceptual and Interaction Capabilities published in the Proceedings of INDICON, Bangalore, Dec 2016.
S. Rasipuram, P. Rao, D. Jayagopi. Asynchronous Video Interviews vs Face-to-Face Interviews For Communication Skill Measurement: A Systematic Study, accepted in International Conference on Multimodal Interaction(ICMI), Tokyo, Japan, Nov 2016. Best Student Paper Nomination.
G. GopanK, N. Sinha, D. Jayagopi. Statistical Feature Analysis for EEG Baseline Classification : Eyes Open vs Eyes Closed, IEEE TENCON Nov 2016, Singapore.
P. Venkatesh, D. Jayagopi. Automatic Expression Recognition and Expertise Prediction in Bharatnatyam, published in the International Conference on Advances in Computing, Communications and Informatics (ICACCI), Sep 2016, LNMIIT Jaipur.
P. Venkatesh, D. Jayagopi. Automatic Bharatnatyam Dance Posture Recognition and Expertise Prediction using Depth Cameras, published in the SAI Intelligent Systems Conference 2016, London, UK. (IntelliSys 2016).

Graduate/Doctoral level

AI 836 - Advanced Visual Recognition - Aug to Dec 2023
AI 825 - Visual Recognition - Jan to May 2023
AI 825 - Visual Recognition - Jan to May 2022
AI 836 - Advanced Visual Recognition - Aug to Dec 2021
AI 825 - Visual Recognition - Jan to May 2021
AI 836 - Advanced Visual Recognition - Aug to Dec 2020
AI 825 - Visual Recognition - Jan to May 2020
SP 825 – Visual Recognition – Jan to May 2019
DS/NCS 826 – Advanced Visual Recognition – Aug to Dec 2018
DS/NCS/ESD 866 – Advanced Machine Perception – Jan to May 2018
DS/NCS/ESD 866 – Advanced Machine Perception – Aug to Dec 2016

Graduate

AI 511 - Machine Learning - Aug to Dec 2023
AI 511 - Machine Learning - Aug to Dec 2022
AI 511 - Machine Learning - Aug to Dec 2021
CS/DS 612 – Machine Learning – Jan to May 2018
DS/NCS/ESD 863 – Machine Perception – Jun to Jul 2017 [Offered at Samsung – Executive MTech Elective]
DS/NCS/ESD 863 – Machine Perception – Jan to May 2017
DS/NCS/ESD 863 – Machine Perception – Jun to Jul 2016 [Offered at Samsung – Executive MTech Elective]
DS/NCS/ESD 863 – Machine Perception – Jan to May 2016
DS/NCS/ESD 863 – Machine Perception – Aug to Dec 2015
CS/NCE 854 – Digital Image Processing – Jan to May 2015 with Prof. Sinha
CS/NCE 855 – Designing Gaming Simulations – Aug to Dec 2014 with Prof. Sebastiaan Meyer (KTH Sweden)
CS/NCE 854 – Digital Image Processing – Aug to Dec 2014 with Prof. Sinha

Undergraduate

EC304 – Digital Signal Processing – Aug to Dec 2018
EC304 – Digital Signal Processing – Aug to Dec 2017
ES105 – Signal Processing – Aug to Dec 2016 with Prof. Sinha
ES105 – Signal Processing – Aug to Dec 2015 with Prof. Sinha
ES104 – Signals and Systems – Jan to May 2015 with Prof. Sinha
ES105 – Signal Processing – Aug to Dec 2014
ES101 – Basic Electronics – Jan to May 2014 with Prof. Jyotsna

PGD Artificial Intelligence and Machine Learning

Statistical ML

PGD Data Analytics

Predictive Analytics I

Coordinated Multimodal Sensing for 6G Applications, 6G Flagship Finland
Accenture Research Grant – Incorporating Multimodal Nonverbal cues in Dialog
BEL Project - Design and Development of Deep Learning based traffic analytics modules on infrastructure based cameras
SERB(DST) – Automatic prediction of communication skill in self-presentation interviews, video conference based interviews and group discussions
Mphasis Center of Excellence Funding – Accessibility for disabled – Indian Sign Language Synthesis, targeting the mute and deaf community
CAIR, DRDO – Human Behavior Analysis using Indoor Camera
NI R&D – Robust and fast Multi-object tracking and FPGA
SIEMENS – Open Platform for Smart Campus
IIITB Start up Grant — Automatic Analysis of communicative behavior in job-hiring scenario

Associate Professor & Faculty-in-charge Internships & Placements

Institute’s address:

Office Hours:

Telephone: