Hello! I am Abhra Chaudhuri, a Senior Researcher in the Computing Research Group at Fujitsu Research of Europe. I study the behaviour of deep neural networks under various kinds of distribution shifts.
In May, Vienna was the host to the prestigious 12th International Conference on Learning Representations (ICLR), considered to be the world’s premier annual deep learning event. An exciting new technology development from Fujitsu Research of Europe was selected for the proceedings, presented by Abhra Chaudhuri and CEO Serban Georgescu. Entitled “Learning Conditional Invariances through Non-Commutativity”, the team presented our research project on data-efficient adaptation to new environments, in collaboration with the University of Exeter, and Anjan Dutta from the University of Surrey.
The ICLR is renowned for presenting and publishing cutting-edge research on all aspects of deep learning used in the fields of artificial intelligence, statistics and data science, as well as important application areas such as machine vision, computational biology, speech recognition, text understanding, gaming, and robotics.
Researchers from Fujitsu, in collaboration with the University of Exeter and the University of Surrey, have developed a technology that allows one to leverage information from different sources say drawings, cartoons, cliparts, etc., to meet the needs of an AI system meant to work in a new environment like natural photographs. This becomes beneficial in scenarios where it may be difficult to acquire lots of data in the new environment (say, specific types of photographs).
In this technique, the unique feature values of images from the new environment are extracted and learned, along with those shared with the sources. The complete set of these unique and common feature values are used for discrimination. This way, insights from both the sources as well as the new environment can be learned to make appropriate decisions.
Specifically, the work establishes that non-commutatively mapping source environment samples to the representation space of a new target environment can efficiently learn conditional invariances, satisfying the sample-complexity needs for generalization on the target with samples from the source. In 3 domain adaptation benchmarks (PACS, Office-Home, DomainNet) with multiple kinds of image formats, such as art, cartoons, and photographs, the model, trained with labelled samples of some reference format, can readily adapt to every new input format, with only a few unlabeled samples of the new format during training. The accuracy improvement was around 2% compared to the conventional technology.
In the future, this technology is expected to improve the accuracy of models working with pathological images with various views.
Developer Comments
In our project, we are developing algorithms that are robust to arbitrary distribution shifts in low resource domains like medical images, as well as domains like graphs where distribution shifts are not as extensively studied. This technology is an initial step in characterizing the features that should and should not be discarded when adapting a machine learning model to shifting distributions, presenting a simple and efficient approach to learn the desired feature set. We hope to contribute to society by further developing this technology and applying it to a variety of fields like healthcare and large-scale graph AI.
Related links
https://www.fujitsu.com/global/about/resources/news/press-releases/2024/0509-01.html
