Anatomy Education Tools
The research team has been working on developing digital, augmented reality tools for anatomy learning: ARnatomy and FlexAR. Students learn osteology and the muscular system of the canine pelvic limb and human thoracic limb moving physical bones in front of a camera attached to smart devices that provide various anatomical information.
ARnatomy
Integration of the traditional materials (bones) and augmented reality using mobile devices keeps the core quality of an embodied experience of using bones, and builds multimedia information around the bones in computational environment. We created a system that can recognize a variety of 3D printed bones while a user holds and moves a bone in front of the camera of a mobile device or behind the camera. Once recognized the bones are populated with virtual text labels that move on the screen to match the video camera feed of the bones. The labels are clear and effective at pointing out regions of interest. In addition, we created an additional mode that allows the user to see the recognized bone in the context of the entire skeleton.
The system is separated into 3 main components. These elements receive data from the mobile device’s camera. This data is provided to the Object Recognition and Tracking module. The spatial data is approximated and fed to the Unity3D Game engine in the Graphic User Interface step. Inside of the Unity3D application will be a collection of components that define the content of expected and recognized objects. All bones and learning content are stored and kept track of at this level. This collection of data describes and acts on the 3D scene that is presented to the user composite with the video feed from the camera.
Seo, J. H., Storey, J., Chavez, J., Reyna, D., Suh, J., & Pine, M. (2014). ARnatomy: tangible AR app for learning gross anatomy. ACM SIGGRAPH 2014 Posters (SIGGRAPH ’14). New York, NY, USA.
Seo, J. H. (2015). One ARnatomy. Augmented World Expo 2015. Santa Clara, CA. USA.
FlexAR
We focus on demonstrating the flexion and extension of various muscle groups as the result of moving a physical skeletal model. In addition we wanted to explore the different AR interface styles to see how they support different learning styles. The styles we explored were wearable, tablet, and computer. Users of our prototype manipulate a physical skeletal model affixed with augmented reality (AR) targets.
An AR-enabled device records this interaction and projects a digital 3D model consisting of the bones and major muscles of the arm over the physical model. Users are then able to examine both gross anatomy as well as muscle flexion and extension. The user can also interact through a graphical user interface to highlight and display additional information on individual muscles. Flex-AR was built using the Unity game engine with Qualcomm’s Vuforia plugin, a mobile AR library, to handle the capturing and tracking of our augmented reality targets. For FlexAR, we use 4 targets: 1 to determine the basic position of the arm and the others to control the rotation of the shoulder, elbow, and wrist joints of the 3D model. The assets for the 3D overlay were developed in Maya using our physical arm model.
Saenz, M., Strunk, J., Maset, K., Malone, E., & Seo, J. H. (2015). See the Flex: Investigating Various Display Settings for Different Study Conditions. SIGGRAPH 2015 Posters (SIGGRAPH ’15). ACM, New York, NY, USA.
Comments are closed.