The CITRIS principal investigator and assistant professor of computer science and engineering combines vibration sensors with the Internet of Things to develop nonintrusive, efficient and effective smart devices for health applications and more.
“I like research on the Internet of Things because it solves problems in people’s lives,” said Shijia Pan, an assistant professor of computer science and engineering at the University of California, Merced, and a principal investigator (PI) at the Center for Information Technology Research in the Interest of Society and the Banatao Institute (CITRIS).
“You can ask — ‘What do I wish to have to make our lives easier?’ — and you can build one yourself.”
Pan joined the UC Merced Department of Computer Science and Engineering in 2019, and she currently serves as a faculty member in the electrical engineering and computer science (EECS) graduate program. She leads the Pervasive Autonomous Networked Systems (PANS) Lab, through which she and a team of students work to make smart devices more ubiquitous and autonomous through efficient, noninvasive sensing using physical vibrations.
Pan received her bachelor of engineering degree in computer science from the University of Science and Technology of China. She pursued a doctorate at Carnegie Mellon University’s Silicon Valley campus at Moffett Field, where she performed cyberphysical systems research at the intersection of electrical and computer engineering and civil engineering.
Her enthusiasm for vibration-based sensing in particular was piqued in graduate school. One day, Pan and her collaborators from the civil engineering department were measuring ambient vibration, or vibrations from the surrounding environment, to help monitor the condition of the building — a process known as structural health monitoring. The sensor picked up a series of short, impulsive signals, which turned out to be a person walking on the other side of the wall.
“That’s when we started thinking about putting vibration sensors on the floor for human sensing,” she said.
“You can actually tell a lot of information about what’s going on in a building without cameras and microphones.”
Tracking well-being in older adults
Pan’s interest in monitoring human activity with vibration sensors has since become a prominent part of her research portfolio. In 2020, she received a CITRIS Seed Award, alongside lead investigator Dr. JoAnn Seibles, an associate physician at UC Davis Health, and Wan Du, a fellow UC Merced assistant professor of electrical engineering and computer science, to create a system to measure and predict fall risks among older adults.
Du and Pan collaborated on a networked system of smart wristbands and vibration sensors installed on floor tiles to estimate older adults’ fall risks based on metrics such as the rhythm of their gait and the lengths of their strides, using the vibrations emitted from their footsteps. The data from the wristbands complements the information gleaned from the floor sensors and enables seamless monitoring.
Pan’s efforts to enhance the lives of older adults and the people who care for them were bolstered again by CITRIS in 2022. With support from another CITRIS Seed Award, Pan began working with lead PI Alyssa Weakley, an assistant professor of neurology at UC Davis, and UC Davis computer scientist and associate professor Hao-Chuan Wang, to develop features for Weakley’s Interactive Care, or I-Care, system. The web-based platform helps connect long-distance caregivers to loved ones with mild cognitive impairment, such as early-stage Alzheimer’s disease.
Weakley was introduced to Pan and her research through a co-worker in the Healthy Aging in a Digital World initiative, a collaboration between CITRIS Health and UC Davis Health.
“My colleague thought that my research with older adults and remote caregivers would fit really well with the work that Shijia is doing with vibration sensors,” Weakley said.
Pan and her team are incorporating ambient vibration sensing into the platform to track different aspects of patients’ well-being, such as their medication intake and the routes they take for their daily activities. One of their primary endeavors is a plug-and-play sensor that can detect vibrations throughout a room when connected to an outlet. Another project is an inexpensive, battery-free “kinetic augment” attachment for prescription pill bottles to help detect whether a patient has taken their medications for the day.
“When you put down or pick up the pill bottle, you’ll create a very unique vibration signal that the room sensors will pick up,” Pan explained.
Weakley, Pan and team have been testing their I-Care upgrades in a home health simulation suite at the UC Davis Betty Irene Moore School of Nursing. The training space is equipped with cameras and microphones that help the researchers calibrate the sensors without placing intrusive equipment anywhere near an actual patient.
Detecting ‘cracks’ in back muscles
In a striking parallel to the conversation that resulted in her initial foray into structural sensing, a casual discussion in Pan’s lab recently sparked another new research direction, one that has sown the seeds for a business venture for two of her undergraduate students.
Several years ago, Pan and her research assistants were in the laboratory typing on their computers, when one student complained about their back pain from so much sitting, much to the dismay — and agreement — of everyone else in the room.
After talking with a doctor from UC Davis, Pan learned that the type of back pain she and her students were experiencing was not solely a product of poor posture straining their muscles; it can also be caused by the contraction of connective tissue called fascia, which often tighten around musculature due to repetitive movements or prolonged sitting.
Pan’s team wondered if they could use their existing expertise in sensors to learn more about their own muscles and, perhaps, uncover a new strategy for relieving back pain.
“We are already experts in vibrations — we can tell if there’s a crack in the wall by just looking at the frequency domain of the vibration signals,” Pan said. “So can we tell whether there’s a ‘crack’ in our muscles?
“When we saw that companies sell posture correctors online that vibrate when it’s time to sit up, we thought, oh, that’s actually a good vibration source. Why don’t we capture the way that the vibrations propagate through muscles to tell muscle condition?”
The researchers built a prototype of what ultimately became the wearable platform PosTrue, a name that PANS Lab members Shreya Shriram and Asiyah Awais also adopted for their startup. Shriram, a 2023 UC Merced graduate who had worked with Pan since 2021, serves as the company’s CEO, and Awais, a 2023 UC Berkeley alum who was introduced to the lab as part of the first CITRIS Workforce Innovation cohort, is chief technology officer. Pan and her doctoral student serve as the venture’s research advisors.
And in part due to Pan’s guidance, PosTrue is gaining momentum. In 2022, the research behind the product won the best poster award at the International Conference on Information Processing in Sensor Networks, and last spring, the startup was announced as a member of the CITRIS Foundry innovation incubator cohort.
Exploring health applications, from millimeter-scale to state-level
Outside of these CITRIS-supported projects, Pan and her team continue to harness the power of IoT sensing systems at extreme scales, particularly in the realm of human health.
In 2021, some ongoing tooth pain and a subsequent visit to the dentist led Pan to explore ways to use piezoelectric sensing, which can detect small changes in pressure and force, to diagnose misaligned bites. Her efforts have culminated in a design for TeethVib (aka IOTeeth), a smart retainer to identify and monitor tooth alignment issues that can cause jaw pain and tooth sensitivity.
Their latest progress on the project was published last month in a top-tier journal for pervasive, anytime-anywhere computing, the Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT).
As part of a major University of California effort to catalyze innovations to combat the effects of climate change, Pan also recently received a grant of more than $600,000 from the UC Merced Office of Research and Economic Development to facilitate highly detailed, AI-enabled air quality monitoring and low-cost, low-power sensor networks to track wildfire emissions that negatively affect climate and health.
The sensors will be placed in Central Valley communities that currently lack air quality monitoring infrastructure and have limited access to smoke hazard forecasts.
“I’m impressed by Shijia’s ability to understand the power of what she’s developing and explain it in a way that’s so easy to understand,” said Weakley. “She’s using very advanced methods for detecting behavior.
“I hope she will be a colleague beyond our current project. I’m very fortunate that I got to know her through CITRIS, and I think we have a very bright future ahead of us in collaborating over the long term.”