# Gestures and interaction

> Tap objects, model parts and individual copies; drive the scene from gestures with shared values; build custom drags; and combine scenes with buttons, scroll views and overlays.

## Taps

Every shape, `<Mesh>`, `<Model>` (per part) and `<Instances>` (per copy) can be tapped. A tap is one finger lifted within 10 points and half a second of where it went down. The **nearest** object under the finger that listens gets it.

### Shapes

```tsx title="TapShape.tsx"
import { Scene, Sphere } from "@apricot/react";
import { useState } from "react";

export function TapShape() {
  const [on, setOn] = useState(false);
  return (
    <Scene style={{ flex: 1 }}>
      <Sphere material="light" color={on ? "#ffcc66" : "#444444"} onPress={() => setOn((v) => !v)} />
    </Scene>
  );
}
```

`onPress` receives a `PressHit`:

| Field | Meaning |
| --- | --- |
| `point` | World-space point that was hit (a three.js `Vector3`) |
| `distance` | Distance from the camera |
| `mesh` | The mesh that was hit |
| `material`, `materialIndex` | The material of the face that was hit |
| `instanceId` | Which copy, for instanced meshes |

### Model parts

`onPartPress` names the part that was tapped, using your model kit's names. See [Loading models](https://matthiasbrandolin.dev/apricot/guides/models#taps-on-parts).

```tsx title="TapDoor.tsx"
import { Scene } from "@apricot/react";
import { defineModelKit, Model } from "@apricot/react/model";
import { useState } from "react";

const car = defineModelKit(require("./car.glb"), {
  doorLeft: { select: "door_left", hinge: "front", axis: "y", range: [0, -65] },
});

export function TapDoor() {
  const [open, setOpen] = useState(false);
  return (
    <Scene style={{ flex: 1 }} look="studio">
      <Model kit={car} joints={{ doorLeft: open ? 1 : 0 }} onPartPress={({ part }) => part === "doorLeft" && setOpen((o) => !o)} />
    </Scene>
  );
}
```

### Copies

`<Instances onPress>` tells you which copy was tapped: `{ index, item, hit }`. See [Instances](https://matthiasbrandolin.dev/apricot/guides/instances).

## Gestures with shared values

For drags, swipes, pinches and flings, use react-native-gesture-handler around the scene and write **shared values** that the scene reads. Nothing re-renders, and the gesture runs on the UI thread.

```tsx title="DragToTilt.tsx"
import { Box, Group, Scene } from "@apricot/react";
import { Gesture, GestureDetector } from "react-native-gesture-handler";
import { useDerivedValue, useSharedValue, withSpring } from "react-native-reanimated";

export function DragToTilt() {
  const x = useSharedValue(0);
  const y = useSharedValue(0);
  const drag = Gesture.Pan()
    .onChange((e) => {
      x.value += e.changeY * 0.01;
      y.value += e.changeX * 0.01;
    })
    .onEnd(() => {
      x.value = withSpring(0);
      y.value = withSpring(0);
    });
  const rotation = useDerivedValue<readonly [number, number, number]>(() => [x.value, y.value, 0]);

  return (
    <GestureDetector gesture={drag}>
      <Scene style={{ flex: 1 }} look="studio">
        <Group rotation={rotation}>
          <Box size={[2, 1.2, 0.2]} radius={0.08} material="aluminium" />
        </Group>
      </Scene>
    </GestureDetector>
  );
}
```

Ready-made versions: [swipe-to-spin](https://matthiasbrandolin.dev/apricot/recipes/swipe-to-spin) (fling with decay), [card-tilt](https://matthiasbrandolin.dev/apricot/recipes/card-tilt) (drag plus the phone's tilt sensor), [drive-joystick](https://matthiasbrandolin.dev/apricot/recipes/drive-joystick) (a thumbstick).

> [!NOTE]
> Taps keep working inside a `GestureDetector`: the scene observes touches without taking them, so gesture handlers around it still receive their events.

## Orbit the camera

`<CameraRig orbit>` gives drag-to-turn and pinch-to-zoom without writing a gesture. See [Camera](https://matthiasbrandolin.dev/apricot/guides/camera#orbit-drag-pinch-and-wheel).

## Custom drags inside the scene: `useSceneTouches`

To build your own drag interaction that receives the scene's fingers directly (as `<CameraRig orbit>` does), use `useSceneTouches` in a component inside the scene. While it is active, the scene claims touches that start on it.

```tsx title="DragToSpin.tsx"
import { Box, Group, Scene, type SceneObject, useScene, useSceneTouches } from "@apricot/react";
import { useRef } from "react";

function Spinner() {
  const scene = useScene();
  const group = useRef<SceneObject>(null);
  const last = useRef<number | null>(null);

  useSceneTouches({
    touch: (points) => {
      const x = points[0].x;
      if (last.current !== null && group.current) {
        group.current.rotation.y += (x - last.current) * 0.01;
        scene.invalidate();
      }
      last.current = x;
    },
    release: () => {
      last.current = null;
      return 0;
    },
  });

  return (
    <Group ref={group}>
      <Box size={1.4} />
    </Group>
  );
}

export function DragToSpin() {
  return (
    <Scene style={{ flex: 1 }}>
      <Spinner />
    </Scene>
  );
}
```

A listener has `touch(points, time)` (every start and move: fingers in screen points), `release(time)` (all fingers lifted; return how far they moved, in points, so taps can be told apart from drags) and optionally `wheel(deltaY)` on the web. Pass `false` as the second argument to stop listening.

## Buttons and UI over a scene

Views placed over a scene keep their own touches: buttons, sheets and toolbars work as usual. Lay out the scene with `StyleSheet.absoluteFill` and put UI on top, then frame the subject around it with `<CameraRig inset>`.

## Overlays that let touches through

A scene drawn over a screen should not block it: `transparent` plus `pointerEvents="none"` sends every touch to the views below.

```tsx title="CoinShower.tsx"
import { Coin } from "@apricot/assets";
import { Pile, Scene } from "@apricot/react";
import { StyleSheet } from "react-native";

export function CoinShower() {
  return (
    <Scene transparent pointerEvents="none" style={StyleSheet.absoluteFill} camera={{ position: [0, 0, 20], fov: 30 }}>
      <Pile count={30} duration={3}>
        <Coin radius={0.5} />
      </Pile>
    </Scene>
  );
}
```

## Related

- [Animation and shared values](https://matthiasbrandolin.dev/apricot/concepts/animation).
- [Camera](https://matthiasbrandolin.dev/apricot/guides/camera).
- [Hooks reference](https://matthiasbrandolin.dev/apricot/reference/hooks): `useSceneTouches`.