grasp it


grasp it combines physical interaction with digital augmentation, making learning in science more engaging, effective, and fun!

Grasp it is a hands-on learning tool developed to enable rich insights into scientific phenomena and structural behaviours. It is the result of an ongoing collaboration between me and Dr. Gregory Quinn, born from a frustration with conventional teaching methods of structural behaviour. As one of his former students I joined the endeavour with a particular interest in developing the interaction between the physical assemblies and their digital counterparts.

grasp it website

A set of parts assemble structural understanding

We developed a set of 3d-printed parts modelled after the diagrammatic symbols conventionally used in structural teaching. The parts click together to build a wide variety of assemblies such as frames, trusses, pendulums and many more. Interaction through pushing and pulling results in an instant haptic and visual feedback with exaggerated deflections helping to explain the core concepts of structural behaviour.

Augmentation reveals the hidden properties of the structures

A projector unit can be extended out, enabling the augmentation of otherwise invisible information on top of the physical assemblies. Insights like the flow of forces are projected onto the board and react to the user’s interactions through a trackable controller.

Creating a responsive connection between physical interaction and digital simulation

My work focused on the transfer of the wand's input into our virtual environment. A digital twin is kept in sync with the physical structure. I developed a system that models the structural behaviour of the parts and their connections with each other. Through iterative refinement of the underlaying finite element solver we managed to create a responsive connection between the physical interaction and digital simulation.

The first prototype of the solver was built in Processing using particles comprised of position and momentum in the two-dimensional coordinate space. The interaction between the particles is modelled through a set of constraints that form the finite elements of the simulation (such as springs holding the particles togethers or anchors keeping them fixed in place).

Iteration of the finite element solver

While this allowed us to model simple systems the need to express more sophisticated structural relationships inspired the next solver iteration built in Unity3D. The six degree of freedom solver allows us to simulate deformation more closely aligned with structural behaviour of our physical parts.

Digital interaction as an extension of the physical

Guided by the idea to think of digital interaction as an extension of the physical, I developed a set of virtual parts, that, like their real counterparts, can be assembled and placed on a virtual board alongside images, text, and other media.