Interactive VR Performance: VIVE

April 6, 2018

Michael Bruner has worked with dancers from Texas A&M Dance Program.

 

 

Soft Interaction Lab at AHFE 2017

July 21, 2017

MS VIZ student, Megan Cook presented her research project as well as our Anatomy Builder study at AHFE 2017 (Applied Human Factors and Ergonomics) international conference yesterday. Go Megan!

Congratulations to all co-authors!

Brian Michael Smith, Erica Renee Malone, Dr. Michelle Pine, Steven Leal, Dr. Zhikun Bai (Anatomy Builder) and Annie Sungkajun (for Megan’s project).

Annie presented at EVA London 2017

July 10, 2017

Annie is back from EVA (Electronic Visualization and the Arts) London 2017.

She presented three projects from our lab.

Thanks Annie for representing us!

 

Soft Interaction Lab at SIGGRAPH 2016

August 11, 2016

Andrew Jackbson presented his research project, PulmonaReality at SIGGRAPH Poster 2016.

Way to go, Andrew!

VIZKIDS_YAP_2016

July 17, 2016

Soft Interaction Lab ran a YAP: Virtual Technology program last week (July 11-15).

More Photos

Anatomy Education Tools

July 10, 2016

The research team has been working on developing digital, augmented reality tools for anatomy learning: ARnatomy and FlexAR. Students learn osteology and the muscular system of the canine pelvic limb and human thoracic limb moving physical bones in front of a camera attached to smart devices that provide various anatomical information.

ARnatomy

Integration of the traditional materials (bones) and augmented reality using mobile devices keeps the core quality of an embodied experience of using bones, and builds multimedia information around the bones in computational environment. We created a system that can recognize a variety of 3D printed bones while a user holds and moves a bone in front of the camera of a mobile device or behind the camera. Once recognized the bones are populated with virtual text labels that move on the screen to match the video camera feed of the bones. The labels are clear and effective at pointing out regions of interest. In addition, we created an additional mode that allows the user to see the recognized bone in the context of the entire skeleton.

The system is separated into 3 main components. These elements receive data from the mobile device’s camera. This data is provided to the Object Recognition and Tracking module. The spatial data is approximated and fed to the Unity3D Game engine in the Graphic User Interface step. Inside of the Unity3D application will be a collection of components that define the content of expected and recognized objects. All bones and learning content are stored and kept track of at this level. This collection of data describes and acts on the 3D scene that is presented to the user composite with the video feed from the camera.

Seo, J. H., Storey, J., Chavez, J., Reyna, D., Suh, J., & Pine, M. (2014). ARnatomy: tangible AR app for learning gross anatomy. ACM SIGGRAPH 2014 Posters (SIGGRAPH ’14). New York, NY, USA.

Seo, J. H. (2015). One ARnatomy. Augmented World Expo 2015. Santa Clara, CA. USA.

FlexAR

We focus on demonstrating the flexion and extension of various muscle groups as the result of moving a physical skeletal model. In addition we wanted to explore the different AR interface styles to see how they support different learning styles. The styles we explored were wearable, tablet, and computer. Users of our prototype manipulate a physical skeletal model affixed with augmented reality (AR) targets.

An AR-enabled device records this interaction and projects a digital 3D model consisting of the bones and major muscles of the arm over the physical model. Users are then able to examine both gross anatomy as well as muscle flexion and extension. The user can also interact through a graphical user interface to highlight and display additional information on individual muscles. Flex-AR was built using the Unity game engine with Qualcomm’s Vuforia plugin, a mobile AR library, to handle the capturing and tracking of our augmented reality targets. For FlexAR, we use 4 targets: 1 to determine the basic position of the arm and the others to control the rotation of the shoulder, elbow, and wrist joints of the 3D model. The assets for the 3D overlay were developed in Maya using our physical arm model.

Saenz, M., Strunk, J., Maset, K., Malone, E., & Seo, J. H. (2015). See the Flex: Investigating Various Display Settings for Different Study Conditions. SIGGRAPH 2015 Posters (SIGGRAPH ’15). ACM, New York, NY, USA.