# Hand Tracking

\*For guidance on the latest version, refer to the GitHub page.

In a HoloKit application, there are several ways to provide input for the user. For example, in MOFA series, we use Apple Watch to provide input. Hand tracking is the fundamental way of input that HoloKit SDK provides with no extra device needed. The hand tracking algorithm can track the positions of 21 landmarks for each hand. With hand tracking, you can implement some basic AR interactions.

<figure><img src="/files/1SeaGd5xOz8BwabDZN9B" alt=""><figcaption><p>The algorithm can track 21 landmarks of the hand</p></figcaption></figure>

Please notice that, in order to use hand tracking algorithm, you must have an iPhone with LiDAR sensor. The iPhone models which support LiDAR sensor are iPhone 12 Pro, iPhone 12 Pro Max, iPhone 13 Pro, iPhone 13 Pro Max, iPhone 14 Pro and iPhone 14 Pro Max. Without a LiDAR enabled iPhone, you can still use the other functionalities of HoloKit SDK, only the hand tracking is disabled.

This section is a tutorial on how to implement the hand tracking sample project. You can directly import this sample project by clicking the import button as shown below.

<figure><img src="/files/OJxsYuDta6RLnaoxPxOw" alt=""><figcaption><p>Click the import button to import the hand tracking sample</p></figcaption></figure>

The rest of this section will demonstrate how to implement the hand tracking sample step by step.

In the last section, we implemented the sterescopic rendering sample. The sterescopic rendering sample is in fact the basic setup of all HoloKit projects. Therefore, we should continue from the last sample scene.

## Setup HoloKit Hand Tracker

To enable hand tracking, we only need to drag the `HoloKitHandTracker` prefab into the scene. You can find the prefab at `Packages->HoloKit SDK->Assets->Prefabs`.

<figure><img src="/files/yYOAL8Ibr1lh948jiCa7" alt=""><figcaption><p>Add HoloKitHandTracker prefab into the scene</p></figcaption></figure>

## Add AROcclusionManager

In order to run the hand tracking algorithm, we need to turn on the LiDAR sensor. In Unity, `AROcclusionManager` is the component to control the LiDAR sensor. Thus, we add `AROcclusionManager` component under the `HoloKitCamera` object.

<figure><img src="/files/EvWCMYrjUDjJC1WG03jV" alt=""><figcaption><p>Add AROcclusionManager to turn on LiDAR sensor</p></figcaption></figure>

You might notice some configurations in the above image. First, in `EnvironmentDepthMode` field, we chose `Fastest`. Running LiDAR sensor is very computationally expensive and would make the iPhone overheat fast.  We chose `Fastest` to save the consumed energy. Second, we disabled `TemporalSmoothing` field. Themporal smoothing can smooth the occlusion edge, but it has no impact on the hand tracking algorithm. Third, we chose `NoOcclusion` in `OcclusionPreferenceMode` field. This is optional, we chose `NoOcclusion` to prevent the landmarks occluded by user's hands.

## Build The Project

Now you are ready to go! Build the project onto your iPhone. Enter the StAR mode and insert your iPhone into a HoloKit. Raise your hand in front of you and you should see 21 landmarks indicating your hand's realtime position.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.holokit.io/creators/unity-sdk/hand-tracking.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.