Picking and Labels EXTENSION

How a Trillium view finds what is under the pointer, how it reports it, and how labels stay over 3D points and hide behind what is in front of them. The examples use the globe view from Using Trillium.

Hover and selection

The view's store holds what is under the pointer and what was last clicked. Bind them as you would any store field.

<div class="card" data-show="$globe.selected">
    <h3 data-bind="$globe.selected.entity.place"></h3>
    <p>Magnitude <span data-bind="$globe.selected.entity.mag"></span></p>
</div>
<p class="hint" data-bind="$globe.hovered.entity.place"></p>

selected is set by a click that did not drag, and a click on empty space clears it. hovered follows the pointer. Both hold { binding, id, entity }. binding is the binding's name, id is the entity's id (or the field key sets), and entity is a plain copy made when it changed. A copy does not move with the entity, so look up the live one by its id when you need its current position.

wildflower.component('quake-card', {
    subscribe: ['globe'],
    watch: {
        'store:globe.selected'(sel) {
            if (sel && sel.binding === 'quakes') this.loadDetails(sel.id);
        }
    },
    loadDetails(id) { /* ... */ }
});

hovered is a computed, and it looks for the entity under the pointer only while something reads it. A page that never binds or watches it does no hover work. When something does, the search runs every frame the pointer is over the canvas, so that a moving entity passing under a still pointer is found. Over 16,000 satellites and a few thousand markers, one search takes about a millisecond.

In this example the markup shows the city under the pointer and the one last clicked, and the globe stops turning while the pointer is over a city. The occluder keeps cities on the far side from being picked through the globe.

                        <div data-component="pick-demo">
    <div id="pick-stage" style="height: 240px"></div>
    <p class="small mt-2 mb-1">
        Pointer:
        <strong data-bind="$globe.hovered.entity.name"></strong>
    </p>
    <p class="small" data-show="$globe.selected">
        Selected:
        <strong data-bind="$globe.selected.entity.name"></strong>,
        <span data-bind="$globe.selected.entity.lat"></span>°,
        <span data-bind="$globe.selected.entity.lon"></span>°
    </p>
</div>
                    

                        const cities = [
    { id: 'tokyo', name: 'Tokyo', lat: 35.7, lon: 139.7 },
    { id: 'delhi', name: 'Delhi', lat: 28.6, lon: 77.2 },
    { id: 'cairo', name: 'Cairo', lat: 30.0, lon: 31.2 },
    { id: 'london', name: 'London', lat: 51.5, lon: -0.1 },
    { id: 'lagos', name: 'Lagos', lat: 6.5, lon: 3.4 },
    { id: 'nyc', name: 'New York', lat: 40.7, lon: -74.0 },
    { id: 'mexico', name: 'Mexico City', lat: 19.4, lon: -99.1 },
    { id: 'rio', name: 'Rio de Janeiro', lat: -22.9, lon: -43.2 },
    { id: 'sydney', name: 'Sydney', lat: -33.9, lon: 151.2 }
];
for (const c of cities) {                  // latitude and longitude to x, y, z
    const lat = c.lat * Math.PI / 180, lon = c.lon * Math.PI / 180;
    c.x = Math.cos(lat) * Math.sin(lon);
    c.y = Math.sin(lat);
    c.z = Math.cos(lat) * Math.cos(lon);
}

const container = document.getElementById('pick-stage');
const renderer = new THREE.WebGLRenderer({ antialias: true, alpha: true });
container.appendChild(renderer.domElement);
const scene = new THREE.Scene();
const camera = new THREE.PerspectiveCamera(45, 1, 0.1, 100);
camera.position.set(0, 0.5, 2.7);
camera.lookAt(0, 0, 0);

const spin = new THREE.Group();             // the globe and its cities turn together
scene.add(spin);
spin.add(new THREE.Mesh(
    new THREE.SphereGeometry(1, 48, 32),
    new THREE.MeshBasicMaterial({ color: 0x1e3a5f })
));
spin.add(new THREE.Mesh(
    new THREE.SphereGeometry(1.002, 24, 12),
    new THREE.MeshBasicMaterial({ color: 0x4f7cac, wireframe: true })
));
const dots = new THREE.InstancedMesh(
    new THREE.SphereGeometry(0.04, 12, 8),
    new THREE.MeshBasicMaterial({ color: 0xff7a59 }),
    cities.length
);
spin.add(dots);

const globe = wildflower.three.view('globe', {
    renderer, scene, camera, fit: container,
    occluder: new THREE.Sphere(new THREE.Vector3(), 0.999),
    before: () => { if (!globe.hovered) spin.rotation.y += 0.004; }
});
globe.instanced(() => cities, dots, { name: 'cities', position: ['x', 'y', 'z'] });

wildflower.component('pick-demo', {
    destroy() {
        wildflower.unregister('globe');
        renderer.dispose();
    }
});
                    

Live Preview

Picking from your own handlers

globe.pick(event) returns the entity under a pointer event, or null.

renderer.domElement.addEventListener('dblclick', (e) => {
    const hit = globe.pick(e);
    if (hit) flyTo(hit.point);
});

pick() returns { entity, binding, name, point, hits }. entity is your own row or pool entity, not a copy. point is where the pointer met the entity, in world space. hits lists everything under the pointer, top first, and globe.hits(event) returns that list alone. A binding with pickable: false is left out, and so is a mesh that is hidden.

Picking casts rays against each binding's geometry on the CPU, so it never sees what a vertex shader does. A mesh whose shader moves or stretches its vertices is picked at its unchanged shape, which misses what is on screen. Give such a binding pickable: false, and pick through a binding drawn at its real shape.

Which hit is on top

Hits are ordered the way three.js shows them. A material that writes depth hides what is behind it, so those hits are ordered by distance, nearest first. Markers whose material writes no depth, such as transparent icons, are drawn over anything they are in front of, and among themselves in renderOrder. So the list starts with the markers in front of the nearest depth-writing hit, higher renderOrder first, then the depth-writing hits, then any markers behind them. The order does not depend on the order you created the bindings in.

quakeMesh.renderOrder = 1;
volcanoMesh.renderOrder = 2;   // a volcano among its own earthquakes stays on top, and is picked

Picking buffer bindings

Instanced bindings are picked by casting a ray from the pointer. Tiny points are hard to hit that way, so a buffer binding with entity is picked on screen, within pickRadius pixels of the pointer (6 by default). entity(i) returns what the pick reports for record i, and pickFilter(i, x, y, z) can reject records.

Labels

globe.follow(element, getEntity) keeps an element over an entity every frame, and returns a function that stops it. The element is hidden when getEntity returns null or a point that is not a number, when the point is behind the camera, or when it is behind the occluder. An element removed from the page is dropped.

The view writes the element's style.transform, a translate to the point measured from the canvas's top-left, and its style.display. So position the element absolutely at the canvas's top-left, and put any offset of your own, such as centring the label on the point, on an element inside it.

<div id="tag-iss" class="tag"><span class="tag-text">ISS</span></div>

.tag { position: absolute; left: 0; top: 0; pointer-events: none; }
.tag-text { display: inline-block; transform: translate(-50%, -120%); }

const stop = globe.follow(document.getElementById('tag-iss'), () => wildflower.getStore('stations').iss);

getEntity returns an object with x, y and z in world space. Pass a binding as the third argument to read the binding's position fields and apply its mesh's transform instead. For labels you place yourself, globe.projectPoint(x, y, z) gives a point's screen position, and globe.occluded(x, y, z) says whether it is hidden.

Here each city has a label that follows it as the globe turns. The cities sit in a turning group, so each follow() passes the binding, and the view applies the mesh's transform. A label disappears when its city goes behind the globe, which is the occluder at work.

                        <div data-component="label-demo">
    <div id="label-stage" style="height: 260px"></div>
</div>
                    

                        const cities = [
    { name: 'Tokyo', lat: 35.7, lon: 139.7 },
    { name: 'Cairo', lat: 30.0, lon: 31.2 },
    { name: 'London', lat: 51.5, lon: -0.1 },
    { name: 'New York', lat: 40.7, lon: -74.0 },
    { name: 'Lima', lat: -12.0, lon: -77.0 },
    { name: 'Sydney', lat: -33.9, lon: 151.2 }
];
for (const c of cities) {
    const lat = c.lat * Math.PI / 180, lon = c.lon * Math.PI / 180;
    c.x = Math.cos(lat) * Math.sin(lon);
    c.y = Math.sin(lat);
    c.z = Math.cos(lat) * Math.cos(lon);
}

const container = document.getElementById('label-stage');
const renderer = new THREE.WebGLRenderer({ antialias: true, alpha: true });
container.appendChild(renderer.domElement);
const scene = new THREE.Scene();
const camera = new THREE.PerspectiveCamera(45, 1, 0.1, 100);
camera.position.set(0, 0.5, 2.9);
camera.lookAt(0, 0, 0);

const spin = new THREE.Group();
scene.add(spin);
spin.add(new THREE.Mesh(
    new THREE.SphereGeometry(1, 48, 32),
    new THREE.MeshBasicMaterial({ color: 0x1e3a5f })
));
spin.add(new THREE.Mesh(
    new THREE.SphereGeometry(1.002, 24, 12),
    new THREE.MeshBasicMaterial({ color: 0x4f7cac, wireframe: true })
));
const dots = new THREE.InstancedMesh(
    new THREE.SphereGeometry(0.03, 12, 8),
    new THREE.MeshBasicMaterial({ color: 0xff7a59 }),
    cities.length
);
spin.add(dots);

const world = wildflower.three.view('world', {
    renderer, scene, camera, fit: container,
    occluder: new THREE.Sphere(new THREE.Vector3(), 0.999),
    before: () => { spin.rotation.y += 0.004; }
});
const places = world.instanced(() => cities, dots, { position: ['x', 'y', 'z'] });

for (const c of cities) {
    const tag = document.createElement('div');
    tag.className = 'city-tag';
    tag.innerHTML = '<span class="city-tag-text"></span>';
    tag.firstChild.textContent = c.name;
    container.appendChild(tag);
    world.follow(tag, () => c, places);
}

wildflower.component('label-demo', {
    destroy() {
        wildflower.unregister('world');
        renderer.dispose();
    }
});
                    

                        #label-stage { position: relative; overflow: hidden; }
.city-tag { position: absolute; left: 0; top: 0; pointer-events: none; }
.city-tag-text {
    display: inline-block;
    transform: translate(-50%, -140%);
    padding: 1px 6px;
    border-radius: 4px;
    background: rgba(0, 0, 0, 0.65);
    color: #fff;
    font-size: 12px;
    white-space: nowrap;
}
                    

Live Preview

The occluder

The occluder option is what hides things, in world space. follow() hides an element whose point is behind it, and picking skips entities behind it, so a click on the near side of a globe never selects a marker on the far side.

const globe = wildflower.three.view('globe', {
    renderer, scene, camera, fit: container,
    occluder: new THREE.Sphere(new THREE.Vector3(), 0.999)   // the Earth
});

Choose the form by the shape of what hides things:

A three.js Sphere or Box3 is tested exactly with a few arithmetic operations, fast enough for hundreds of labels every frame. For a globe, a radius a little under the globe's keeps markers on its surface visible.
A mesh, or a list of meshes, is tested against its real shape by casting a ray to each point. Use it for anything the simple shapes do not describe, such as a rotated box, terrain or a model. It costs more per point, so it suits a few labels, and picking, which tests only the entities near the pointer. Meshes the view draws into and hidden objects never hide anything.
A function (x, y, z) => boolean is your own test.

occluder: [ship, station]                                     // real shapes
occluder: new THREE.Box3().setFromObject(crate)               // an axis-aligned box
occluder: (x, y, z) => y < terrainHeight(x, z)               // your own test

Previous Using Trillium

Next DevTools