👋
NRSDK(Old)
v1.10.2
v2.2.1v2.1.0v1.10.2
v1.10.2
v2.2.1v2.1.0v1.10.2
  • NRSDK Fundamentals
    • NRSDK Overview
    • XREAL Devices
      • XREAL Glasses
      • Controller
      • Compatibility
    • Getting Started with NRSDK
    • Sample Code
    • Tutorials
    • Release Note
      • NRSDK 1.10.2
      • NRSDK 1.9.5
      • NRSDK 1.9.3
      • NRSDK 1.9.1
      • NRSDK 1.8.0
      • NRSDK 1.7.0
      • NRSDK 1.6.0
  • Development
    • Input and Camera
      • NRInput
      • Interact with Unity UI (Tutorial)
      • Customize Controller UI
      • NRCameraRig
    • Hand Tracking
    • Image Tracking
    • Plane Detection (Tutorial)
      • Overview
      • Import the package
      • Detect planes in the real world
      • Perform a hit test against detected planes
      • Add a car
      • Add gems
      • Wrap up
    • Depth Mesh
      • Meshing Manager Overview
      • Use Meshes in the Editor
      • Tutorial: Mesh Collision
    • Tools
      • First Person View
      • Emulator
      • Notification popup
      • Reset Camera
    • MRTK Integration
  • Advanced
    • Access RGB Camera
    • NRSDK Coordinate Systems
  • XR Streaming
  • Frequently Asked Questions
Powered by GitBook
On this page
  • Overview
  • Examples and Scene Settings
  • Features

Was this helpful?

  1. Development

MRTK Integration

NRSDK 1.9.3 & MRTK 2.8.0 Integration with Sample Projects

PreviousReset CameraNextAccess RGB Camera

Last updated 1 year ago

Was this helpful?

Overview

This github repo () contains extension that adds compatibility for NRSDK, including hand tracking and controller support, to Microsoft's open source Mixed Reality Toolkit () for Unity.

Required Environments:

  • Unity 2020.3.X

  • MRTK 2.8.0

  • NRSDK 1.9.3

Examples and Scene Settings

This repo includes two projects, each containing several pre-configured scenes:

  • Assets/HandTrackingDemo

    • HandTrackingHub (Manager scene)

    • HandTrackingHubMainMenu

    • Clipping

    • ColorPicker

    • Dock

    • HandInteraction

    • HandMenu

    • HandMenuLayout

    • Joystick

    • NonNativeKeyboard

    • PressableButton

    • ScrollingObjectCollection

  • Assets/Surfaces

    • StartupScene (Manager scene)

    • Bubble

    • Ephemeral

    • Flock

    • Goop

    • Growbies

    • Lava

    • Lighting

    • Volume

    • Xylophone

These scenes do not require additional configuration and serve as a blueprint for XREAL's MRTK integration, while HandTrackingHub and StartupScene work as manager scene (configured via MRTK's Scene System) of each project respectively.

Features

This following MRTK Features are supported:

Supported Features in MRTK

  • Hand Tracking

  • Hand Meshing

  • Hand Skeleton / Gestures

  • XREAL Phone Controller / Dev Kit Controller

  • Gaze Input (aka Eye Tracking)

Unsupported Features in MRTK

  • Planes Detection

  • Native Keyboard

  • Voice Input (Using Google speech to text)

  • Scene Meshing

It also contains two open-source sample apps from Microsoft's Mixed Reality Design Labs - Hand Tracking Hub (originally as MRTK examples hub and Surfaces (), which demonstrates NRSDK's hand tracking algorithmic capabilities with MRTK's input system.

https://github.com/microsoft/MRDL_Unity_Surfaces
https://github.com/nreal-ai/NRSDK-MRTK
MRTK