Advanced Use of Hand Tracking
Some common use cases
*For guidance on the latest version, refer to the GitHub page.
In the previous section, we demonstrated how to add the HoloKitHandTracker
prefab into the scene and successfully tracked user's hands. However, we haven't done any coding yet such as turning hand tracker on and off at runtime and getting the position of a specific landmark position. In this section we will show some common use cases of the HoloKitHandTracker
. Same as before, you can directly import this sample through the Unity Package Manager.
The rest of this section will be a step by step guide on how to implement the advanced hand tracking sample. We will start on top of the complete version of the basic hand tracking tutorial project.
Prefab HoloKitHandTracker
Configuration
HoloKitHandTracker
ConfigurationWe first have a look at the inspector of HoloKitHandTracker
prefab.
As you can see, there are 3 public properties.
Property Enabled
indicates whether we want to turn on the hand tracker. If we set Enabled
to true, the hand tracker will be turned on and trying to detect user's hands at every frame. If we set Enabled
to false, no hand will be detected. We can dynamically set this property at runtime to save some computational power because the hand tracking algorithm is very computionally expensive.
Property MaxHandCount
indicates the maximum number of hands to be tracked by the algorithm. There are only two options for this property, which is MaxHandCount.OneHand
and MaxHandCount.BothHands
. This setting is up to your app's need. When you only need one hand to be detected at a time, you can set this property to MaxHandCount.OneHand
to save computational power.
Property DebugMode
controls the visibility of detected hands. If we set DebugMode
to true, when a hand is detected, there are 21 landmarks with different colors indicating the position of each hand joint. However, in a real HoloKit app, we may not want to show those colorful landmarks. We can set DebugMode
to false to make those landmarks invisible.
Get The Position of a Specific Landmark
The most common use case of HoloKitHandTracker
might be making a game object interacting with the position of a specific hand landmark, such as the point of the index finger.
In this sample project, we will attach a sphere onto the user's index finger. We first add a sphere into the scene and create a new script called IndexFingerSphere
. We set the size of the sphere to 0.1 and attach the script onto it. The code of the script is shown below. Basically, in every frame, we first check whether there is at least one hand detected. If there is one, we get the first detected hand (there could be two) and set the sphere to the position of its index finger.
Please notice that, although the hand tracking algorithm can detect user's both hands, it cannot tell the handedness. It doesn't know which hand is left and which hand is right. The index of the list property HoloKitHandTracker.Hands
only indicates the detection order, where HoloKitHandTracker.Hands[0]
is the first hand being detected in the frame, and HoloKitHandTracker.Hands[1]
is the second one.
You might have noticed in the current project, there in only one IndexFingerSphere
object. When two hands are detected, the IndexFingerSphere
will jump between the two index fingers. Let's change the IndexFingerSphere
script and add another sphere object to fit this need.
Now when two hands are detected at the same time, there will be two spheres attached to each index finger respectively.
Last updated