Realistic Hand Interaction (Beta)

Contents

• Realistic Hand Interaction (Beta)
  • Introduction
  • How to Use
  • See Also

Introduction

The VIVE Wave™ plugin provides the Realistic Hand Interaction feature within its Essence packages.

The imported prefabs and samples are located at Assets/Wave/Essence/Hand/Interaction.

Both left and right hand grabber prefabs come with a hand mesh, hand grabber and custom grab pose.

The Realistic Hand Interaction feature consists of two main components: HandGrabInteractor and HandGrabInteractable.

The HandGrabInteractor updates hand tracking data and allows grabbing at gameobjects with the HandGrabInteractable component.

The HandGrabInteractable component allows gameobjects to be grabbable. Currently, only gameobjects with at least one 3D Collider component are supported.

Also refer to Realistic Hand Interaction API.

How to Use

The Realistic Hand Interaction contains common parts and Wave-Dependent parts. You can grab an object refering to the Common Use part.

If you would like to align the hand grab pose, refer to the Wave Dependent part.

Common Use

Only gameobjects with the HandGrabInteractor component are capable of grabbing gameobjects with the HandGrabInteractable component.

1. Add Hand Manager to your scene.

   

2. Add HandGrabInteractor and modify the script options if needed

   

   • Handedness : Set the hand for hand tracking.
   • Grab Distance : Set the grabbing distance.
   • Grab Event : Send event when beginning and ending grab.

3. Add HandGrabInteractable to gameobject with collider or its children have.

   • The gameObject with collider.

     

   • The gameObject without collider but its children have.

     

   • Rigidbody : The rigidbody of this object.

   • Is Grabbable : Allow oneself to be grabbed.

   • Finger Requirement : Define which fingers are required for grabbing.

   • Grab Poses Settings : Define the hand gestures for grabbing.

   • Grabbed : Send event when the object is grabbed and released.

   • Constraint Movement (Optional) : Constraint the object’s movement and rotation.

4. Set the Finger Requirement on the HandGrabInteractable of the gameObject, and then build and run to test.

   • Ignored: This finger is not needed for grabbing.
   • Optional: This finger can be used for grabbing.
   • Required: This finger is required for grabbing.

Throw the gameobject (Optional)

If you want objects to experience inertia when thrown, you can add this component.

1. Add component PhysicsInteractable to the gameobject with HandGrabInteractable.

2. Add OnBeginInteractabled and OnEndInteractabled of the PhysicsInteractable to the Grabbed Event of the HandGrabInteractable.

   

3. Set ForceMultiplier to determine the force applied when throwing.

4. After setting the PhysicsInteractable:

Constraint Interaction (Optional)

For certain special objects, they may have limits on their movement or rotation, so constraint interaction is needed.

For movement restriction, you can use OneGrabMoveConstraint.

1. Add OneGrabMoveConstraint and set the object that needs to be constrained.

   

2. Set whether to constraint each axis and the movement limits.

3. Drag OneGrabMoveConstraint to HandGrabInteractable’s Constraint Movement; this will constraint its movement when grabbing.

For rotation restriction, you can use OneGrabRotateConstraint.

1. Add OneGrabRotateConstraint and set the object that needs to be constrained and the rotation pivot.

   

2. Set constraint axis and the rotation limits.

3. Drag OneGrabRotateConstraint to HandGrabInteractable’s Constraint Movement; this will constraint its rotation when grabbing.

Wave Dependent

The VIVE Wave™ plugin provides default hand models. (See Wave Hand)

When grabbing a gameObject, the hand mesh may clip with the gameObject. To achieve a more realistic grabbing effect, you can customize the grab pose in the Unity Editor during Play Mode.

Once the customization is completed, the grab pose will be saved in the Grab Pose Settings of the gameObject. As a result, the gameObject will be grabbed according to the edited grab gesture during grabbing.

Here demonstrates the before-and-after comparison of setting the Grabpose, where the 1st animation as before and the 2nd animation as after results:

Here are two methods to customize grab pose: Custom Grab Pose and Record Grab Pose.

Custom Grab Pose

1. Add CustomGrabPose to the gameObjects with HandMeshRenderer and disable Enable Collider of the HandMeshRenderer.

   

2. Disable “EnableDirectPreview” from Wave > DirectPreview > EnableDirectPreview.

   

3. In Unity Editor, enter Play Mode and set the position of the hand model and the gameObject at the same location.

   

4. Modify the bending value of each fingers of the CustomGrabPose and adjust the position of the hand model or the gameobject until it matches your expectation.

   

5. Click Save HandGrab Pose of the CustomGrabPose before stopping the play mode.

6. Check the Grab Poses settings of the HandGrabInteractable on the gameobject. It should updated.

   

Record Grab Pose

This feature depends on DirectPreview.

1. Enable “EnableDirectPreview” from Wave > DirectPreview > EnableDirectPreview.

   

2. Disable Enable Collider of the HandMeshRenderer.

   

3. Start Streaming Server from Wave > DirectPreview > ControlPanel.

   

4. Ensure the VIVE XR Elite is connected to your PC and launch the VIVE Business Streaming app on the HMD.

5. Enter play mode in the Unity Editor and see the preview on the HMD. Then, modify the position of the gameObject to a location where the hand can easily grab it.

6. Click the Game view in the Unity Editor then you’ll need to use the keyboard to save the GrabPose. Modify the position of the hand model to match the grabbing position, then press Enter to save the GrabPose.

   

7. Check the Grab Poses settings of the HandGrabInteractable on the gameobject. It should updated.

   

Tip

Fine-tune the grab pose

• You can click the Preview Grab Pose of the HandGrabInteractable under the play mode.
• Expand all the game objects in the Hand model and adjust their rotation values until they match your expectation.
• Finally, click Update Grab Pose of the HandGrabInteractable to update the grab pose before stopping play mode.

Tip

Duplicate to other hand

• You can click the “+” button of Grab Poses settings to create a duplicate.
• Click “IsLeft” to toggle between left and right hand grabbing.
• Click the Mirror Pose “Align X axis”, “Align Y axis”, and “Align Z axis” will project the GrabPose onto the left-right, vertical, and front-back directions.

Physic Hand model

To achieve a more realistic effect, you enable Enable Collider of the HandMeshRenderer. If you enable Collider, the HandMeshRenderer will initialize a fixed-size collider (matching our default hand model), allowing it to interact physically with other 3D objects.

See Also

1. Wave Hand
2. DirectPreview
3. Realistic Hand Interaction API