Ever looked at your lunch and wondered just how many calories there are on your plate? Soon, you’ll be able to use your phone and instantaneously retrieve that information — and more — with the help of a new mobile application that’s currently being developed by the NUS-Tsinghua Extreme Search Centre (NExT++).
NExT++ is a joint research center established by the National University of Singapore, Tsinghua University of China, and the University of Southampton, UK. Funded by the National Research Foundation of Singapore, NExT++ aims to research and develop new technology solutions in emerging fields like data analytics, image recognition, and more.
BS in Computer Science in Real-Time Interactive Simulation graduate Irene Tan and BS in Computer Science and Game Design graduate Lim Sing Gee, both of whom started out as NExT++ interns, recently went on to secure full-time positions under the Wellness Team. Currently, they’re helping to develop DietLens, an app that uses artificial intelligence (AI) to help in the prevention and management of diabetes. The team is led by Dr. Ming Zhaoyan, who has been training Irene and Sing Gee in the areas of machine learning since their internship days.
The knowledge I’ve gained from DigiPen provides me with the ability to pick up new things quickly and apply that knowledge in the most efficient manner I can think of”
DietLens is a lifestyle app that incorporates an AI assistant to promote good eating habits while motivating users to cultivate a healthier, more active lifestyle. The app has a food recognition system that allows users to snap a photo of their food. It then automatically scans, analyses, and records the nutrient composition — such as carbohydrates, protein, and fat — of the food. DietLens also generates a weekly or monthly report about the user’s diet, as well as diet change recommendations when necessary. In addition to helping users to easily track their eating habits, the app can also be used by medical professionals as a way to monitor their patients, particularly those at a high risk of developing diabetes. The app comes with an in-built step counter to track walking activity and calories burned for a more holistic approach to lifestyle management.
The idea of developing a healthcare app like DietLens was one of Irene’s main motivations for joining NExT++. According to the International Diabetes Federation, the number of patients with diabetes — type 2 diabetes in particular — is expected to increase by 55% by 2035. This is where lifestyle management apps can play a big part in motivating users for good. “Knowing that I will be able to contribute to the betterment of someone’s health, which in turn benefits society, is meaningful and makes me want to be a part of this project,” Irene says.
While working on DietLens, Irene is in charge of database management. Her day-to-day work involves maintaining the database of food nutrition information. She has to frequently back up and extract useful information from each dataset, taking into account feedback and user input to continuously improve the app. “One of the main challenges I face is the fact that different datasets have different formats, so I have to remove food names that are duplicated in another language,” Irene says. “I also have to be able to extract additional data like brands and cooking methods out of the datasets, which can be tricky. I am currently learning natural language processing in my free time in order to solve this issue more efficiently and effectively.” Additionally, Irene also does front-end and back-end web programming.
In his own day-to-day work, Sing Gee is in charge of evaluating and improving the app’s image recognition capabilities. This function allows the app to automatically detect the type of food users are eating after they take a photo of it. To do so, Sing Gee relies on a type of machine learning known as deep learning. Sing Gee helps to train the deep learning model by preparing multiple images of a particular food. He then labels the images and lets the machine process them. By doing this, the machine will start to recognize common characteristics in each of the images. As the machine processes more and more pictures of a particular food — say, chicken rice — it starts becoming better at identifying what constitutes chicken rice. It learns to focus on important details such as the food’s color and texture, while at the same time discarding less useful information such as the table color or other image background details.
“I decided to join NExT++ full-time after my internship because of the many learning opportunities and the availability of mentorship,” Sing Gee says. “Additionally, I believe that deep learning will be something that can be applied to multiple fields in the future to enhance the current way of doing things.”
In the course of their work, Irene and Sing Gee also had to familiarize themselves with new programming languages such as Go and Python. Despite the learning curve, they both managed to pick up the new languages without difficulty, crediting their solid foundations in programming to DigiPen’s robust curriculum. “The knowledge I’ve gained from DigiPen provides me with the ability to pick up new things quickly and apply that knowledge in the most efficient manner I can think of,” says Irene, with Sing Gee echoing similar sentiments.
This willingness to learn and go the extra mile is what makes DigiPen (Singapore) graduates stand out for Dr. Ming: “The DigiPen graduates always put the assigned task as their first priority and are dedicated to completing their projects with quality. They learn fast and grow with the project. More importantly, they’re able to see the big picture and future of the project, which makes their working attitude one that’s proactive rather than just reactive.”