Institute of Medical Intelligence and XR

Funding Source: Hong Kong RGC Theme-based Research Scheme

Project Period: 01 Jan 2023 - 31 Dec 2027

 

Technological innovation has presented new and promising ways to improve medical diagnosis, treatment, education, and healthcare service with ever-increasing rigor, subtlety, insight, and precision. Among these advanced technologies, artificial intelligence (AI) and extended reality (XR) are growing fast and making huge transformations in medicine and healthcare. XR is a term referring to all real-andvirtual combined environments and human-machine interactions generated by computer technology and wearables. Integrating AI and XR research can open many new possibilities towards achieving and delivering precision medicine for next-generation healthcare. However, there remain challenges to applying AI/XR to medical image computing and computer-assisted intervention in real-world clinical applications. The objective of this proposed project is to build a world-class institute for medical intelligence and XR in Hong Kong by developing cutting-edge techniques aimed at overcoming these challenges and facilitating “one-stop” medicine and healthcare services, covering screening, diagnosis, treatment, management, and prognosis.

Multi-Modality Vision Perception Under Adverse Weather Conditions

Funding Source: Hong Kong RGC General Research Fund

Project Period: 01 Jan 2025 - 31 Dec 2027

 

Our research project tackles the critical issue of vision perception in adverse weather conditions head-on. By creating a rich multimodal dataset, developing dynamic fusion strategies, employing knowledge distillation techniques on large-scale models, and advancing open vocabulary 3D scene understanding, we aim to significantly advance the field of vision perception in adverse weather, ultimately enhancing safety and reliability in real-world scenarios.

Development and Application of the Visual Intelligence Technologies for the Large-scale Urban Traffic Monitoring and Control

Funding Source: Hong Kong Innovation and Technology Fund, under Mainland-Hong Kong Joint Funding Scheme

Project Period: 01 Nov 2023 - 31 Oct 2025

 

The traffic environment in modern cities is becoming increasingly complex, leading to frequent traffic accidents. The existing transport system for vehicle operation monitoring, safety precautions, and collaborative management has limitations of low accuracy, low real-time performance, and systemic deficiency. This project develops a visual intelligence-based large-scale urban traffic safety monitoring and control system to overcome key technical challenges in 2D and 3D vision-based intelligent understanding and privacy protection. The system will enable early warning and management of dangerous situations, reducing unsafe traffic events and ensuring safety in large-scale urban traffic systems.

Research and Development of Intelligent Preoperative Planning System for Tibial Plateau Fractures

Funding Source: Hong Kong Innovation and Technology Fund, under Guangdong-Hong Kong Technology Cooperation Funding Scheme

Project Period: 01 Aug 2023 - 31 Jul 2025

 

Tibial plateau is an important weight-bearing area of the human body, and its fracture is often accompanied by a high incidence of complications and poor prognosis, which is the most challenging injury. Doctors have strong subjectivity in the CT image interpretation of tibial plateau fractures, and the lack of quantitative standards in fracture evaluation makes it difficult to effectively plan the reduction operations. This project plans to combine the technology, medical support and industrial accumulation of Guangdong and Hong Kong to develop an intelligent preoperative planning system for tibial plateau fracture surgery to assist doctors to conduct fracture reduction accurately and efficiently.

Development of a Sign Language Communication and Education Platform based on AI and VR

Funding Source: Hong Kong Innovation and Technology Fund, under Mainland-Hong Kong Joint Funding Scheme

Project Period: 01 Nov 2022 - 30 Oct 2024

 

In this project, we aim to develop a sign language communication and education platform based on artificial intelligence and virtual reality, which can assist the communication between deaf and normal people, as well as providing online sign language education for learners. The proposed platform can enhance the communication between deaf and normal people, and promote the popularization of sign language.

Research on Artificial Intelligence for Multimodal Image-guided Glioma Surgery

Funding Source: Hong Kong Innovation and Technology Fund, under Mainland-Hong Kong Joint Funding Scheme

Project Period: 01 Aug 2022 - 31 Jul 2024

 

Glioma is the most common and fatal brain tumor, and surgical resection is its main treatment. Traditional surgery has low resection rate and high incidence of new neurologic deficits. To address this, this project aims to develop multimodal image-guided augmented reality (AR) surgery navigation technology, and in particular, overcomes the key technical bottleneck of segmentation of glioma and brain functional areas in the navigation system. By enhancing AR-based preoperative planning and intraoperative navigation using the multimodal image analysis techniques, this project will greatly improve the accuracy of glioma surgery, reduce the new functional area defects, optimize the surgical process, reduce the difficulty of surgery, and promote the application and industrialization of intelligent surgery navigation.

An GNN-based Virtual Screening Toolkit for Early Drug Discovery

Funding Source: Hong Kong Innovation and Technology Fund, under ITSP (Seed Project)

Project period: 01 Jan 2022 - 30 Jun 2024

 

This project aims to develop a new virtual screening toolkit based on graph neural networks to accelerate early drug discovery intelligently. It is well known that structure determines activity. This project will use graphs to represent chiral compounds and protein targets, automatically learn their high-dimensional representations, and predict binding sites, poses, and activities. To quickly start screening tasks for new targets, we will use collaborative filtering and meta-learning, which can train a robust model with only a few experimental data. We will further implement the black-box optimization method to rationally schedule the experiments of activity detection to find the preclinical drug with the best binding activity as soon as possible. The research achievements of this project will help improve the efficiency of drug discovery and reduce the overall cost of the drug development process.

Development of Lightweight, Intelligent, and Remote Ultrasound Robotic System

Funding Source: Hong Kong Innovation and Technology Fund, under Guangdong-Hong Kong Technology Cooperation Funding Scheme

Project Period: 01 Jun 2022 - 31 May 2024

 

With the maturity of technologies such as human-computer interaction and internet transmission, remote ultrasonic robots have been developed. However, such a technology is complex and full of challenges. Therefore, this project intends to combine the research and development accumulation of Hong Kong and Shenzhen to develop a lightweight, intelligent, and remote ultrasound robotic system based on the cross-integration of robots, ultrasound imaging, and artificial intelligence in multiple fields.

Developing an AI-based Toolkit for High-content High-Throughput Cellular Image Analysis

Funding Source: Hong Kong Innovation and Technology Fund, under ITSP (Seed Project)

Project Period: 01 Oct 2021 - 31 Mar 2023

 

High-content cellular image analysis technology has been the basis for phenotypic screening (PS) in early drug discovery. Compared with target-based screening methods, interest in PS continuously increase recently due to an analysis from the FDA that PS strongly contributes to the discovery of first-in-class drugs. The power of PS to address poorly understood diseases is clear, such as rare diseases and emergent infectious diseases. The development of automated cellular systems allows rapid visualization of large groups of cells in parallel. However, high-throughput PS has been limited because manually analyzing those data is error-prone and time-consuming due to operational complexity and the immense size of data.

In this project, we aim to develop an advanced AI-based toolkit to automate cellular image analysis based on our team’s original medical image processing and leading machine learning technologies, which have been widely proven to effectively and robustly extract the features of cellular images.

Research on Automatic Diagnosis and Therapeutic Evaluation of Retinopathy of Prematurity based on Deep Attention Network and Longitudinal Learning

Funding Source: Hong Kong Innovation and Technology Fund, under Guangdong-Hong Kong Technology Cooperation Funding Scheme

Project Period: 01 Dec 2020 - 30 Nov 2022

 

The retinopathy of prematurity (ROP) is a retinal vascular proliferative blindness that occurs in preterm or low-weight baby, accounting for approximately 19% of the blindness in children worldwide. Standardized screening, timely diagnosis, and treatment are effective ways to reduce ROP blindness. However, there is a huge shortage of ROP professional doctors, and the manual diagnosis by doctors is usually time-consuming, error-prone and subjective. In this project, we aim to develop an automatic diagnosis and therapeutic evaluation system for ROP by implementing and integrating a variety of new technologies and utilizing the large-scale dataset. The proposed system can reduce the effort of doctors and promote the research and industrial development of artificial intelligence and intelligent healthcare in Hong Kong and Shenzhen.