👋
NRSDK(Old)
v1.10.2
v2.2.1v2.1.0v1.10.2
v1.10.2
v2.2.1v2.1.0v1.10.2
  • NRSDK Fundamentals
    • NRSDK Overview
    • XREAL Devices
      • XREAL Glasses
      • Controller
      • Compatibility
    • Getting Started with NRSDK
    • Sample Code
    • Tutorials
    • Release Note
      • NRSDK 1.10.2
      • NRSDK 1.9.5
      • NRSDK 1.9.3
      • NRSDK 1.9.1
      • NRSDK 1.8.0
      • NRSDK 1.7.0
      • NRSDK 1.6.0
  • Development
    • Input and Camera
      • NRInput
      • Interact with Unity UI (Tutorial)
      • Customize Controller UI
      • NRCameraRig
    • Hand Tracking
    • Image Tracking
    • Plane Detection (Tutorial)
      • Overview
      • Import the package
      • Detect planes in the real world
      • Perform a hit test against detected planes
      • Add a car
      • Add gems
      • Wrap up
    • Depth Mesh
      • Meshing Manager Overview
      • Use Meshes in the Editor
      • Tutorial: Mesh Collision
    • Tools
      • First Person View
      • Emulator
      • Notification popup
      • Reset Camera
    • MRTK Integration
  • Advanced
    • Access RGB Camera
    • NRSDK Coordinate Systems
  • XR Streaming
  • Frequently Asked Questions
Powered by GitBook
On this page
  • Overview
  • Examples and Scene Settings
  • Features

Was this helpful?

  1. Development

MRTK Integration

NRSDK 1.9.3 & MRTK 2.8.0 Integration with Sample Projects

PreviousReset CameraNextAccess RGB Camera

Last updated 1 year ago

Was this helpful?

Overview

This github repo (https://github.com/nreal-ai/NRSDK-MRTK) contains extension that adds compatibility for NRSDK, including hand tracking and controller support, to Microsoft's open source Mixed Reality Toolkit (MRTK) for Unity.

It also contains two open-source sample apps from Microsoft's Mixed Reality Design Labs - Hand Tracking Hub (originally as MRTK examples hub and Surfaces (https://github.com/microsoft/MRDL_Unity_Surfaces), which demonstrates NRSDK's hand tracking algorithmic capabilities with MRTK's input system.

Required Environments:

  • Unity 2020.3.X

  • MRTK 2.8.0

  • NRSDK 1.9.3

Examples and Scene Settings

This repo includes two projects, each containing several pre-configured scenes:

  • Assets/HandTrackingDemo

    • HandTrackingHub (Manager scene)

    • HandTrackingHubMainMenu

    • Clipping

    • ColorPicker

    • Dock

    • HandInteraction

    • HandMenu

    • HandMenuLayout

    • Joystick

    • NonNativeKeyboard

    • PressableButton

    • ScrollingObjectCollection

  • Assets/Surfaces

    • StartupScene (Manager scene)

    • Bubble

    • Ephemeral

    • Flock

    • Goop

    • Growbies

    • Lava

    • Lighting

    • Volume

    • Xylophone

These scenes do not require additional configuration and serve as a blueprint for XREAL's MRTK integration, while HandTrackingHub and StartupScene work as manager scene (configured via MRTK's Scene System) of each project respectively.

Features

This following MRTK Features are supported:

Supported Features in MRTK

  • Hand Tracking

  • Hand Meshing

  • Hand Skeleton / Gestures

  • XREAL Phone Controller / Dev Kit Controller

  • Gaze Input (aka Eye Tracking)

Unsupported Features in MRTK

  • Planes Detection

  • Native Keyboard

  • Voice Input (Using Google speech to text)

  • Scene Meshing