Immersive Learning System
This project aims to enhance asynchronous e-learning by exploring technologies and approaches that improve learner engagement, interaction, and the overall learning experience. The work focuses on developing more immersive, interactive, and learner-centered environments to address the limitations of conventional asynchronous e-learning systems and improve the quality of online education.
AI for multimodal and multisensor situational awareness
This research project focuses on Artificial Intelligence for Situational Awareness, particularly in defence environments. This work focuses on computer vision and multimodal AI for understanding visual information, with research interests including military object detection, image understanding and summarization, and vision-language models. It also aims to develop reliable AI-based methods for extracting and interpreting relevant information from complex visual scenes to support situational awareness and decision-making.
Enhancing User Experience of Cultural Heritage using Immersive Technology
Cultural Heritage (CH) is an identity of any group (community or tribe). It provides a sense of bindings and belongingness amongst a group and allows them to have a better understanding of their previous generations and traditional evolutionary history. Cultural Heritage also promotes tourism and acts as a means of cultural exchange between groups. However, traditional preservation methods (museum, live tour with written description) of CH have failed to attract individuals, especially the younger generation. Moreover, restricted access to real-world heritage inhibits users' complete understanding of it. These lead to a lack of proper appreciation of CH amongst observers. These motivated researchers to digitize the Cultural Heritage to preserve them as well as to attract individuals. The digitization of CH can be achieved using archiving technologies and immersive technologies (Virtual Reality (VR) and Augmented Reality (AR)). The users can get the sense of being immersed in the CH environment with informative content using immersive technologies. In a VR based CH system, the user should feel like being at a site without being physically present there. In order to generate that feeling, the virtual environment should be as realistic as possible. The user should also be able to navigate through the environment naturally. Allowing the user to interact with the virtual content would enhance the user’s sense of belongingness to the environment. The expectations and capabilities vary from user to user, which necessitates the VR based system to be personalized tailored to individual needs. In an AR assisted CH system, the information presented to the user should be easily perceivable. Moreover, it should not hamper the real-world experience of the user. In order to augment the virtual content in a real environment, there should be a fine-tuned blending of virtual and real environments. The user should also be able to interact with those virtual contents naturally, which enhances the user experience. Like VR systems AR systems also necessitates personalization with the user needs. We have considered three types of users, viz. Live Tour Users, Virtual Tour Users and Mobile Tour Users. Live Tour users being present in-situ, experience CH using smart-glass. Multimedia information is overlaid in the real-life view of the users. Virtual Tour users experience the CH remotely using Head Mounted Display (HMD). The Mobile Tour users, on the other hand, use their handheld mobile devices (smartphones, tablets) to get detailed information about any CH entity. Our main focus is to make our CH system to give a user the experience of sensing realism and allow the users to interact with the virtual world using controllers or without using controllers (body gestures).
Interactive Visualization and Smart Warning
The most influential challenge in meaningful visualization is to represent large data sets that satisfy specific user needs. In Human-computer Interaction (HCI), particularly in visualization, there are challenges to designing an adequately simple system concerning better interaction and user experience. The key intuition to design goals in HCI to explore and understand the data by a non-expert user. A good data visualization also concerns the design decisions such as abstraction of data as per user's cognitive ability/load and clustering of data into groups as per useful goals. We are trying to apply the visualization technique in education settings, particularly in ICT based classroom, to visualize the real-time status of the students. The visualization techniques will help the teacher to explore details about the students' engagement with the teaching-learning. Whether and how many of the students are engaged with the teaching-learning process. The augmented information will help the instructor to decide further corrective actions if required. Currently, we are emphasizing to develop classroom visualization techniques using cluster-based visualization through a limited size display device such as a smartphone/tablet. We are motivated to research this because it is challenging for a teacher to visualize real-time status. The status means which and/or how many in-class students are engaged/understanding, particularly when the classroom is large. We hope that this will significantly improve the teaching-learning process. We are also working on designing smart warnings for the classroom uses. The warnings/alert or the notifications should be sent in a smartphone/tablet, including peripheral smart-band. The affective computing team of our lab is working to build a sensitive classroom where the students' engagement and understanding level will be identified automatically by using the data from digital backchannel and embedded sensors of smartphones/tablets. Once these have been identified, it is required to alert the students' who are not engaged. It is also necessary to warn the teacher if most of the students are not engaged/not understanding what the teacher is teaching. Systematic alert generation is challenging. The major challenges are • When and what type of alert should be sent to a particular student(s) without disturbing the other students. • How frequently the teacher should be alerted without interrupting the flow of the lecture.
Affective Interaction & Sensitive Classroom
Affective computing, sometimes known as emotional computing, is one of the most recent and hottest research area in the field of Human-Computer Interaction (HCI). Researchers are trying to incorporate affect and emotion in HCI to make the interaction “sympatheticâ€, “empathic†or simply “affectiveâ€. The obvious ambition is to improve usability and user experience by making the systems more natural and responsive to the goals and expectations of the user. The first and foremost step for designing an affective interactive system is to recognize the affective state of the user. This may be followed by the design and implementation of appropriate interface and interaction that complement and/or change the users’ affective/emotional state. We, the affective computing team of UCCN Lab of IITG are working both on identifying affective state of the users as well as designing affective interactive systems. For identifying affective state of the user, we mainly emphasize to consider the users of small handheld and touchscreen devices including mobile phones and tablets, as explosion of these devices are clearly observable all around us. At the same time, we are developing a sensitive classroom system where the emotional states of the students will be taken care of to make the teaching-learning process more efficient. Based on the emotional and mental states of the students, teaching learning strategies will dynamically be changed to enhance the students’ interest and engagement and hence, the quality of learning. In our ICT based sensitive classroom system, the students will be warned if they were not engaged in the classroom during lecture. They will also be motivated for engagement through some smart module of the system. The instructor will also be alerted if maximum of the students are not engaged or unable to understand what the teacher is teaching, so that s/he can take corrective actions (if required). The learning level and understanding level of the students will also be periodically measured. All these will be done automatically and systematically by the classroom system itself. Once completed, the sensitive classroom will help to enhance the teaching learning method by a strikingly large extent.
Attention Modeling for Web Page Users
We are currently living in what can arguably be called the "information age". One of the main characteristics of the age is to allow individuals to have instant access to information. Such information can be multi-faceted ranging from latest stock-market data to information related to the health-care facilities available in one's locality. The technology that emerged over time to be one of the primary sources for the individuals to access information is the World Wide Web (WWW). The two major actors in the WWW-based information dissemination process are (a) the information provider and (b) the information user. The "interface" between these two is a "webpage" displayed on a web browser such as Mozilla Firefox, Internet Explorer etc. Evidently, it is of utmost importance that the information provider design the webpage in a way such that the information user finds it easy to locate the information; in other words, a "usable" interface. If a webpage is badly designed, the user may lose interest in locating information contained in the web page. Consequently, the very objective of an information provider to put up web pages will be lost. This may have both economic as well as social consequence. For example, if the user loses interest in viewing the home page of a company, the company may be losing potential customers in spite of having good products. On the other hand, if the user is not interested to view a government web page containing information about various schemes available for the rural people (an e-governance activity), the loser will be the intended segment (rural population) of the society. Keeping this in mind, we aim to model web page usability in terms of "visibility". Visibility is a measure of attention the user is expected to give to particular object/regions on the web page. The visibility model will rank different objects/regions on the page in terms of their visibility to a user. From this ranking, a designer can find out if the most important information on the webpage has the highest visibility. It is also envisaged to produce a visual output of the web page, where objects/regions are marked with different colors according to their visibility. The visual output is expected to make the job of the designer easier. At present, web pages are designed using either of the several available tools such as Microsoft FrontPage, Microsoft Publisher, Dream Weaver etc., none of which checks for the usability of the designed page. A semi-automatic design process can be envisaged as consisting of a tool to check automatically the source program of a web page (output of the existing tools) for any potential visibility problems. The system then draws the designer's attention to these problems in a usable form, so that the designer can understand and rectify the problems.
Computational Model of Aesthetics for Improved Interface Design
Good interface design requires knowledge about people: how they see, understand and think. Another important aspect of good design is how information is visually presented to the user or the aesthetics of the interface. Aesthetically pleasing interfaces improve usability, which implies the improvement in productivity. An interface is composed of the rectangular webpage objects which contain image, text. The positions of these rectangles in the webpage have a direct relation with the aesthetics. Again, the features associated with the text and image affect interface aesthetics. The specific objectives of our work are • Design of computational models for aesthetics prediction of webpage elements: text, image • Design of computational model for aesthetics prediction based on the positions (geometric property) of webpage elements in the layout. • Design of computational model for aesthetics prediction of a whole webpage by combining the previous two objectives. • Use of model for design of aesthetically pleasing webpages. Figure 2 shows an example of IIT Guwahati webpage and its wireframe model, text component and image component. Till date, we developed a computational model for wireframe structure. We also developed a model for images present in an interface. Lastly we developed a computational model for text elements. Currently we are working on integrating these three models to develop a computational model for whole interface.