Tangible interaction · Mixed-reality learning · 2020–2023
Smart
Chemistry Lab
A sensor-enabled chemistry lab that lets students manipulate physical apparatus while Unity reveals hazardous and otherwise difficult-to-observe reactions.
01 · Background
Chemistry becomes difficult to learn when the most important phenomena are unsafe, expensive, or invisible.
Middle-school chemistry teaching often limits students to demonstrations or tightly prescribed procedures. Reactions such as sodium in water can generate gas, heat, or explosions, making open-ended classroom exploration impractical.
Conventional virtual laboratories remove those risks, but they also remove the physical work of holding, moving, pouring, and selecting apparatus. They can simulate a reaction without preserving the embodied skills of an experiment.
- Role
- Research and prototyping team
- Context
- Master's research project
- Technology
- Unity, C#, 3D printing, sensors, speech
- Output
- Functional mixed-reality prototype
02 · Problem
How might a virtual chemistry lab preserve hands-on experimentation, guide students through mistakes, and safely reveal reactions they cannot normally observe?
03 · Effect preview
Physical actions and virtual reactions unfold together.
04 · Key decisions & challenges
Keep the beaker physical
A 3D-printed smart beaker uses orientation, proximity, touch, and pressure sensing to recognize actions such as picking up, approaching, selecting, and pouring. The physical vessel keeps laboratory handling central to the experience.
Fuse speech and sensing at the decision layer
Sensor intent and spoken intent are matched before the system advances. A mismatch becomes a teachable moment: the interface explains the possible consequence and asks whether the student wants to continue or correct the action.
Support exploration as well as the correct path
The sodium-and-water scenario includes both prescribed steps and deliberate trial-and-error. Students can explore different quantities and observe otherwise unsafe outcomes without consuming real reagents.
Persistent challenge. Speech recognition made the interaction more natural, but its reliability decreased in noisy laboratory environments. The physical dimensions of the apparatus and conflicts between multiple sensor streams also required iterative calibration.
05 · Results
The prototype completed the full sodium-and-water learning sequence—from guided setup to pouring, reaction exploration, and gas testing.
Functional testing covered smart-device interaction, speech commands, multimodal fusion, correct experiment completion, and trial-and-error exploration. Every documented test case produced the expected result.
The Unity implementation synchronized physical actions with virtual animation, particles, video, and instructional feedback. The result was a working cross-platform prototype rather than a measured classroom learning study.
