Ask or search…
Comment on page

MRTK Integration

NRSDK 1.9.3 & MRTK 2.8.0 Integration with Sample Projects


This github repo (https://github.com/nreal-ai/NRSDK-MRTK) contains extension that adds compatibility for NRSDK, including hand tracking and controller support, to Microsoft's open source Mixed Reality Toolkit (MRTK) for Unity.
It also contains two open-source sample apps from Microsoft's Mixed Reality Design Labs - Hand Tracking Hub (originally as MRTK examples hub and Surfaces (https://github.com/microsoft/MRDL_Unity_Surfaces), which demonstrates NRSDK's hand tracking algorithmic capabilities with MRTK's input system.
Required Environments:
  • Unity 2020.3.X
  • MRTK 2.8.0
  • NRSDK 1.9.3

Examples and Scene Settings

This repo includes two projects, each containing several pre-configured scenes:
  • Assets/HandTrackingDemo
    • HandTrackingHub (Manager scene)
    • HandTrackingHubMainMenu
    • Clipping
    • ColorPicker
    • Dock
    • HandInteraction
    • HandMenu
    • HandMenuLayout
    • Joystick
    • NonNativeKeyboard
    • PressableButton
    • ScrollingObjectCollection
  • Assets/Surfaces
    • StartupScene (Manager scene)
    • Bubble
    • Ephemeral
    • Flock
    • Goop
    • Growbies
    • Lava
    • Lighting
    • Volume
    • Xylophone
These scenes do not require additional configuration and serve as a blueprint for XREAL's MRTK integration, while HandTrackingHub and StartupScene work as manager scene (configured via MRTK's Scene System) of each project respectively.


This following MRTK Features are supported:
Supported Features in MRTK
  • Hand Tracking
  • Hand Meshing
  • Hand Skeleton / Gestures
  • XREAL Phone Controller / Dev Kit Controller
  • Gaze Input (aka Eye Tracking)
Unsupported Features in MRTK
  • Planes Detection
  • Native Keyboard
  • Voice Input (Using Google speech to text)
  • Scene Meshing