CS Other Presentations

Department of Computer Science - University of Cyprus

Besides Colloquiums, the Department of Computer Science at the University of Cyprus also holds Other Presentations (Research Seminars, PhD Defenses, Short Term Courses, Demonstrations, etc.). These presentations are given by scientists who aim to present preliminary results of their research work and/or other technical material. Other Presentations serve as a forum for educating Computer Science students and related announcements are disseminated to the Department of Computer Science (i.e., the csall list):

RSS Directions

Presentations Coordinator: Demetris Zeinalipour

PhD Defense: Multimodal Generation of Realistic Human Bodies, Nefeli Andreou (University of Cyprus, Cyprus), Monday, July 1, 2024, 14:00-15:00 EET.

The Department of Computer Science at the University of Cyprus cordially invites you to the PhD Defense entitled:

Multimodal Generation of Realistic Human Bodies

Abstract:
The quest for realism in crafting digital humans using deep learning is pivotal across various domains, spanning animation, virtual reality, and artificial intelligence. This thesis delves into the complexities of generating realistic human bodies in motion using intuitive control signals, such as natural language. While significant advancements have been made, existing techniques have limited capacity in terms of realism and multimodality. To this end, we leverage the recent advancements in deep learning techniques to model and evaluate the realism of digital bodies generated using intuitive control signals. First, we observe that the representation of human bodies is a crucial component influencing the realism of generated motions. Although motion modeling has been extensively studied, there is still a lack of a standardized, concise, and efficient method to represent moving bodies. To address this, we propose a novel pose representation based on dual quaternions and prove experimentally that this formulation is more efficient for deep learning frameworks and leads to more realistic generated motion. Second, we tackle the challenge of generating realistic motion while allowing for intuitive control such as natural language. We observe that modern text-to-motion (T2M) methods often face a trade-off between model expressiveness and text-motion alignment. In contrast, the proposed T2M model, LEAD, harnesses the semantic structure of large language models to overcome these barriers. Leveraging LEAD's capabilities, we introduce the novel task of motion textual inversion, where the objective is to produce personalized motion sequences from text. Finally, having established techniques to model and generate human bodies, we emphasize the importance of proper evaluation methods to enhance the realism of generated humans bodies. We observe that despite the advances in text-to-image (T2I) generation, existing models still fail to generate accurate and realistic human bodies. We argue that the lack of proper evaluation techniques acts as a barrier against model improvements. To this end, we build a novel learnable metric (BodyMetric) that receives an image and corresponding text prompt and outputs a realism score for the human body. Key to BodyMetric is the use of a weak 3D body prior. To train the metric we design a highly curated multimodal dataset of humans with corresponding text prompts, 3D body reconstructions, and body realism scores, namely BodyRealism. In summary, this thesis explores the multifaceted process of developing lifelike digital human bodies with intuitive control. This holistic approach aims to close the loop between the different components in the generation of realistic human bodies with intuitive control, encouraging continuous progress through modeling, generation, evaluation, and refinement.

Short Bio:
Nefeli is a Marie Skłodowska-Curie Early Stage Researcher as part of the ITN-CLIPE project, and a PhD candidate at the University of Cyprus, under the supervision of Professor Yiorgos Chrysanthou. Her research interests span the fields of Computer Vision and Graphics with emphasis on Digital Humans and Multimodal Learning. During her PhD she interned at Amazon Research in Tübingen. Additionally, she has been a Visiting Researcher at École Polytechnique in Paris, under the supervision of Vicky Kalogeiton and Marie-Paule Cani. She collaborated closely with the Max-Planck Institute for Intelligent Systems under the guidance of Victoria Fernández-Abrevaya and Michael Black. Prior to her doctoral studies, she has obtained an MSc in Data Science and BSc in Mathematics from the University of Bath.

	Other Presentations Web: https://www.cs.ucy.ac.cy/colloquium/presentations.php
	Colloquia Web: https://www.cs.ucy.ac.cy/colloquium/
	Calendar: https://www.cs.ucy.ac.cy/colloquium/schedule/cs.ucy.pres.2024.andreou.ics