Three.js: "select tool" How to detect the intersection of a 2D square and 3D objects

Question

Basically what i want to create:

I have a 3D map with objects, i want to select all objects that are in the 2D box x1,y1 to x2,y2 on my screen.

Any ideas how this has to be done, because i'm clueless on how to start.

Thanks in advance!

prevX and prevY is coordinate of mouse down:

function onDocumentMouseUp(event) {
  event.preventDefault();

  var x = (event.clientX / window.innerWidth) * 2 - 1;
  var y = -(event.clientY / window.innerHeight) * 2 + 1;

  var width = (x - prevX); //* window.innerWidth;
  var height = (y - prevY); //* window.innerHeight;
  var dx = prevX; //* window.innerWidth;
  var dy = prevY; //* window.innerHeight;

  console.log(
    dx + ',' + 
    dy + "," + 
    (dx + width) + "," + 
    (dy + height) + 
    ", width=" + width + 
    ", height=" + height
  );
  var topLeftCorner3D = new THREE.Vector3(dx, dy, 1).unproject(
    camera);
  var topRightCorner3D = new THREE.Vector3(dx + width, dy, 1)
    .unproject(camera);
  var bottomLeftCorner3D = new THREE.Vector3(dx, dy + height,
    1).unproject(camera);
  var bottomRightCorner3D = new THREE.Vector3(dx + width, dy +
    height, 1).unproject(camera);

  var topPlane = new THREE.Plane();
  var rightPlane = new THREE.Plane();
  var bottomPlane = new THREE.Plane();
  var leftPlane = new THREE.Plane();

  topPlane.setFromCoplanarPoints(camera.position,
    topLeftCorner3D, topRightCorner3D);
  rightPlane.setFromCoplanarPoints(camera.position,
    topRightCorner3D, bottomRightCorner3D);
  bottomPlane.setFromCoplanarPoints(camera.position,
    bottomRightCorner3D, bottomLeftCorner3D);
  leftPlane.setFromCoplanarPoints(camera.position,
    bottomLeftCorner3D, topLeftCorner3D);

  //var frustum = new THREE.Frustum( topPlane, bottomPlane, leftPlane, rightPlane, nearPlane, farPlane);

  function isObjectInFrustum(object3D) {
    var sphere = object3D.geometry.boundingSphere;
    var center = sphere.center;
    var negRadius = -sphere.radius;

    if (topPlane.distanceToPoint(center) < negRadius) { return false; }
    if (bottomPlane.distanceToPoint(center) < negRadius) { return false; }
    if (rightPlane.distanceToPoint(center) < negRadius) { return false; }
    if (leftPlane.distanceToPoint(center) < negRadius) { return false; }

    return true;
  }
  var matches = [];
  for (var i = 0; i < window.objects.length; i++) {

    if (isObjectInFrustum(window.objects[i])) {
      window.objects[i].material = window.selectedMaterial;
    }
  }
}

Answer 1

Intersecting a box in the screen space is equivalent to intersecting a pyramid (perspective) or a cube (orthogonal view) the 3D space. I think you should define a THREE.Frustum based on your 2D box.

For perspective camera:

1. convert the screen-space box corner coordinates to 3D vectors (a vector from the camera position to the given point)

var topLeftCorner3D = new THREE.Vector3( topLeftCorner2D.x, topLeftCorner2D.y, 1 ).unproject( camera );

2. construct 4 planes based on these points (one plane for one box side). The third point of the plane is the camera position.

topPlane.setFromCoplanarPoints (camera.position, topLeftCorner3D , topRightCorner3D ) rightPlane.setFromCoplanarPoints (camera.position, topRightCorner3D , bottomRightCorner3D ) bottomPlane.setFromCoplanarPoints (camera.position,bottomRightCorner3D , bottomLeftCorner3D ) leftPlane.setFromCoplanarPoints (camera.position, bottomLeftCorner3D , topLeftCorner3D )

3. Construct a Frustum based ot these planes, adding the camera near and far planes to them.

var frustum = new THREE.Frustum( topPlane, bottomPlane, leftPlane, rightPlane, nearPlane, farPlane);

4. Intersect the frustum with the objects

frustum.intersectsBox(object.geometry.boundingBox)

or

frustum.intersectsSphere(object.geometry.boundingSphere)

An alternative to using the Frustum object itself: you can skip the near and far planes, you only have to check whether the object is in the space bordered with your 4 planes. You can write a function like Frustum.intersectSphere() method with only 4 planes:

function isObjectInFrustum(object3D) {
    var sphere = object3D.geometry.boundingSphere;
    var center = sphere.center;
    var negRadius = - sphere.radius;

    if ( topPlane.distanceToPoint( center )< negRadius ) return false;
    if ( bottomPlane.distanceToPoint( center )< negRadius ) return false;
    if ( rightPlane.distanceToPoint( center )< negRadius ) return false;
    if ( leftPlane.distanceToPoint( center )< negRadius ) return false;

    return true;  
}

Answer 2

You can use the bounding boxes of the objects. THREE.Geometry contains a boundingBox property. This is null by default, you should compute it by calling computeBoundingBox() directly.

scene.traverse( function ( node ) {
    if ( node.geometry )
         node.geometry.computeBoundingBox();
} );

Once you have the bounding boxes, the bounding box 2D coordinates are (if 'y' is the UP axis):

mesh.geometry.boundingBox.min.x
mesh.geometry.boundingBox.min.z
mesh.geometry.boundingBox.max.x
mesh.geometry.boundingBox.max.z

See http://threejs.org/docs/#Reference/Core/Geometry

(however, if you need the exact result for meshes that have non-rectangular shape, you need further computations)

Answer 3

For a more accurate method than subfrustum selection see marquee selection with three js. It works by projecting the 8 corners of bounding boxes onto screen space and then intersecting with screen rectangle. See also here for a 3rd faster method (written in ClojureScript) that projects 3D bounding sphere onto bounding circle in screen space, as well as implementations of the first two methods. See also related stack overflow question that tries to use GPU to perform screen projection and picking with readpixels (I haven't looked into this method since I assume readpixels would stall the rendering pipeline too much, but let me know if I am wrong).

I made a working example of subfrustum selection at http://jsbin.com/tamoce/3/ , but it is too inaccurate. The important part is:

          var rx1 = ( x1 / window.innerWidth ) * 2 - 1;
          var rx2 = ( x2 / window.innerWidth ) * 2 - 1;
          var ry1 = -( y1 / window.innerHeight ) * 2 + 1;
          var ry2 = -( y2 / window.innerHeight ) * 2 + 1;

          var projectionMatrix = new THREE.Matrix4();
          projectionMatrix.makeFrustum( rx1, rx2, ry1, ry2, camera.near, camera.far );

          camera.updateMatrixWorld();
          camera.matrixWorldInverse.getInverse( camera.matrixWorld );

          var viewProjectionMatrix = new THREE.Matrix4();
          viewProjectionMatrix.multiplyMatrices( projectionMatrix, camera.matrixWorldInverse );

          var frustum = new THREE.Frustum();
          frustum.setFromMatrix( viewProjectionMatrix );