Hi there! We’re Shlomo Berkovsky from Macquarie University (Sydney, Australia) and Shun Takeuchi from Fujitsu Research. In this post, we’d like to introduce to you the work of Fujitsu Macquarie AI Research Lab, which is part of the broader Fujitsu’s Small Research Lab (SRL) program.
Fujitsu Macquarie AI Research Lab is one of the recent additions to the SRL program that officially started in February 2024 and this is the first instantiation of the SRL in Asia-Pacific (outside of Japan) and in the Southern Hemisphere alike. The aims of the Lab’ are ambitious and include the development of a personalised digital coaching platform using human sensing and generative AI technologies, which is anticipated to be integrated into the Fujitsu Kozuchi suite later on. Considering a few innovative ideas this Lab will combine, we’d like to take this opportunity and introduce Macquarie University and the research areas targeted by this Lab to the readers of Fujitsu Tech Blog.
Macquarie University
Macquarie is a young and agile public university in Sydney, Australia. Established in 1964 only, currently Macquarie is home to more than 40,000 students and the university is well positioned to break the traditional boundaries of academic research, working actively with industry and government partners. Macquarie offers modern facilities for study and research, with all the university’s research areas ranked at or above world level. According to Times Higher Education, Macquarie is currently ranked within top-1% of universities globally, leading Australia in several disciplines. You’d certainly appreciate the vibes of the green and spacious nature of the Wallumattagal campus of Macquarie.
Two research centres at Macquarie actively contribute to the work of the AI Lab: the Centre for Health Informatics of the Australian Institute of Health Innovation and the Centre for Applied AI of the School of Computing. Let us introduce these centres.
Centre for Health Informatics
The Centre for Health Informatics (CHI) explores the application of digital technologies in healthcare with a focus on AI and its implications for transforming models of care and delivering personalised healthcare. The centre leads many large-scale projects in digital health and established the Australian Alliance for AI in Healthcare. As the longest-running digital health research centre in Australia, CHI is a partner to major research institutions and governments, and has numerous national and international industry partnerships. The centre generates high-impact research and has expertise in the development and implementation of state-of-the-art AI methods.
Centre for Applied Artificial Intelligence
The Centre for Applied Artificial Intelligence is a globally recognised leader in AI and Process Automation research. With a commitment to improving business operations, the centre initiated over 40 national and international projects, offering innovative solutions in Big Data understanding and its application in process automation, augmentation, and improvement. The mission of the centre is to foster a comprehensive understanding of AI and its potential, driven by the belief that significant advancements are best achieved through collaborative and pioneering work.
Key research fields
The 2 centres bring to the AI Lab a unique blend of AI technologies, academic expertise areas, and practical skills related to the Lab’s activities. Collectively, these allow us to engage in collaborative research and translation activities, to ensure we meet the ambitious goal of the Lab: to develop a personalised digital coaching platform using human sensing and generative AI technologies. As some of these areas are highly innovative or involve original application of new technologies, we will briefly describe several key technologies and research areas targeted by the Lab.
Persuasive technologies
Digital coaching is about effectively coaching users, to support and improve their reasoning or behaviour. This aligns with the overarching goals of the persuasive technologies research, which is about developing AI technologies that affect humans’ thinking or behaviour [1]. The area relies on and implements in the digital world many established offline theories in psychology, education, communication, and design. However, the notable advantage of the technologies over human coaches that should be highlighted is their ability to monitor human behaviour and interactions, and offer scalable support anywhere and anytime.
Personalised technologies
While the tireless support that the digital coaches can offer is fabulous, different uses have different strengths and weaknesses. As such, the coaching should also exploit personalised technologies, capable of providing support in an adaptive manner, to leverage individual’s strength and most effective their weaknesses. Personalisation research is widespread these days and can be seen in numerous online applications, such as commerce, social media, learning, entertainment, and more, aiming to increase sales, engagement, interactions, and so on [2]. In our case, we blend personalisation into the digital coaching in order to tailor the coaching intervention (for example, content, language, modality, or timing) to the user and improve its effectiveness.
User modelling
To be able to effective and accurately personalise coaching, the technology needs to gain an understanding of every individual’s strength and weaknesses to address. This is addressed by the user modelling research area, which, as the name suggests studies various ways and methods to understand and characterise human users interacting with digital systems [3]. The modelling task is typically implemented by capturing and mining user’s activities, behaviours, preferences, or interactions, to derive the user information that will fuel personalisation. In case of the AI Lab, in addition to all the above, we deploy the Fujitsu’s Vision AI technologies that relies on image- and video-based recognition and interpretation of human activities.
Large Language Models
Once the models of human thinking and behaviour are established, personalised coaching interventions need to be generated. This is, however, a tedious tasks requiring a high level of expertise, especially considering the broad spectrum of use cases and application domains, in which the coaching solution will be deployed. To this end, we consider the application of Generative AI and particularly Large Language Models (LLMs) for the generation of coaching content. Trained on a vast and diverse corpora of documents, these technologies can cater for the requirement of various domains and support future generalisation of the platform. Furthermore, multi-modal General AI can enrich the interventions being generated with images and videos.
Knowledge Graphs
We aim to enhance the generation of coaching content by integrating semantics into the process and implementing Knowledge Graph Augmented Generation. Knowledge Graphs (KGs) are robust semantic tools that represent real-world entities and relationships between them in a machine-interpretable graph structure. This structure can help Generative AI mechanisms to uncover relationships between entities that were not absent from the training data Generative AI models, no matter how large and diverse it may be. In addition, KGs can represent specific non-public knowledge, for instance enterprise data, that will further improve coaching content generation [4].
Lastly, it is worth to bring up the envisaged generic nature of the coaching platform being developed. Our aim is to make is as generalisable as possible and ultimately integrate it into the Fujitsu Kozuchi suite. However, how can this be materialised, given that different domains may have their own requirements and peculiarities? To increase the generalisation potential, we intend to develop the platform in a bottom-up manner from several domain-specific instantiations of personalised coaching.
Enhancing human activity through AI-powered coaching
The first 2 domains have already been identified and they are workplace safety and patient rehabilitation. Both involve humans repeating similar activities numerous time, with no to limited supervision and feedback. These offer a fertile ground for the deployment of AI-driven coaches, which may guide humans toward a safer and healthier behaviour, even when real-life human coaches may not be readily available to support them. We have recently developed a video QA technology allowing to understand human behaviour from video footage and generate answers to user’s queries. It has been presented as a Fujitsu KG-enhanced Retrieval-Augmented Generation (RAG) technology [5]. Fujitsu KG-enhanced RAG will be introduced on the Tech Blog soon.
We are currently collaborating with professionals and domain experts to understand the requirements or such digital coaches and specific aspects, in which they may excel. Once the AI coaches are developed, we will evaluate them in practical studies with experts and end-users alike. Finally, upon developing and evaluating a larger number of AI coaches, we will analyse the findings of the individual case studies, to make them as generalisable as practical.
As you can see, the Fujitsu Macquarie AI Research Lab has embarked on an ambitious and innovative journey of collaborative research. Establishing industry collaboration geared for impact is a key priority for Macquarie and partnerships like this create outstanding opportunities for the university researchers and students. The collaboration is important for Fujitsu as well, as this has the potential to unlock access to talent and the potential of collaborating with the international academic players and solving global challenges. We are all looking forward to seeing the strong impact of the outcomes of the Fujitsu Macquarie AI Research Lab’s work.
Key team members
 |
 |
 |
 |
 |
|---|---|---|---|---|
Professor |
Professor |
Research Fellow |
Research Fellow |
Honorary A/Professor (Fujitsu Research) |
References
- Yu, K., Berkovsky, S., & Conway, D., et al., “Do I trust a machine? Differences in user trust based on system performance,” in Human and machine learning, 2018.
- Beheshti, A., Moraveji-Hashemi, V., Yakhchi, S., et al., “Personality2vec: Enabling the analysis of behavioral disorders in social networks,” in WSDM, 2020.
- Berkovsky, S., & Coiera, E., “Moving beyond algorithmic accuracy to improving user interaction with clinical AI,” in PLOS Digital Health, 2023.
- Beheshti, A., Yang, J., Sheng, Q.-Z., et al., “ProcessGPT: Transforming business process management with generative artificial intelligence,” in ICWS, 2023.
- Fujitsu Limited, “Fujitsu to provide the world’s first enterprise-wide generative AI framework technology to meet changing needs of companies,” Fujitsu Press Releases, 2024.
