ARKit by Example — Part 3: Adding geometry and physics fun

In the last article we used ARKit to detect horizontal planes in the real world, then visualized those planes. In this article we are now going to start adding virtual content to our AR experience and start interacting with the planes that were detected.

By the end of this article we will be able to drop cubes into the world, apply realistic physics to the cubes so they interact with one another and also create mini shock waves to make the cubes fly around a bit.

Here is a video showing the app in action, you can see how first we capture the horizontal planes, then we add some 3D cubes to interact with the scene and then finally cause some mini explosions to make the cubes jump around:

As always, you can follow along with the code here: https://github.com/markdaws/arkit-by-example/tree/part3

Hit Testing

As you saw in the first tutorial we can insert virtual 3D content at any X,Y,Z position and it will render and track in the real world. Now that we have plane detection we want to add content that interacts with those planes. This will make the app look like there are object on top of your table, chairs, floor etc.

In this app, when the user single taps on the screen, we perform a hit test, this involves taking the 2D screen coordinates and firing a Ray from the camera origin through the 2D screen point (which has a 3D position on the projection plane) and into the scene. If the ray intersects with any plane we get a hit result, we then take the 3D coordinate where the ray and plane intersected and place our content at that 3D position.

The code for this is pretty simple, ARSCNView contains a hitTest method, you pass in the screen coordinates and it takes care of projecting a ray in 3D through that point from the camera origin and returning results:

- (void)handleTapFrom: (UITapGestureRecognizer *)recognizer {
  // Take the screen space tap coordinates and pass them to the
  // hitTest method on the ARSCNView instance
  CGPoint tapPoint = [recognizer locationInView:self.sceneView];
  NSArray<ARHitTestResult *> *result = [self.sceneView   hitTest:tapPoint types:ARHitTestResultTypeExistingPlaneUsingExtent];
  // If the intersection ray passes through any plane geometry they
  // will be returned, with the planes ordered by distance 
  // from the camera
  if (result.count == 0) {
    return;
  }
  // If there are multiple hits, just pick the closest plane
  ARHitTestResult * hitResult = [result firstObject];
  [self insertGeometry:hitResult];
}

Given a ARHitTestResult, we can get the world coordinate where the ray/plane intersection took place and place some virtual content at that location. For this article we will just insert a simple cube, later we will make the objects look more realistic:

- (void)insertGeometry:(ARHitTestResult *)hitResult {
float dimension = 0.1;
SCNBox *cube = [SCNBox boxWithWidth:dimension 
                             height:dimension 
                             length:dimension 
                      chamferRadius:0];
SCNNode *node = [SCNNode nodeWithGeometry:cube];
// The physicsBody tells SceneKit this geometry should be
// manipulated by the physics engine
node.physicsBody = [SCNPhysicsBody         
                     bodyWithType:SCNPhysicsBodyTypeDynamic 
                            shape:nil];
node.physicsBody.mass = 2.0;
node.physicsBody.categoryBitMask = CollisionCategoryCube;
// We insert the geometry slightly above the point the user tapped
// so that it drops onto the plane using the physics engine
float insertionYOffset = 0.5;
node.position = SCNVector3Make(
  hitResult.worldTransform.columns[3].x,
  hitResult.worldTransform.columns[3].y + insertionYOffset,
  hitResult.worldTransform.columns[3].z
);
// Add the cube to the scene
[self.sceneView.scene.rootNode addChildNode:node];
// Add the cube to an internal list for book-keeping
[self.boxes addObject:node];
}

Adding some physics

AR is suppose to augment the real world, so in order to make our objects feel a bit more realistic, we will add some physics to give then a feeling of weight.

As you can see in the code above we give each cube a physicsBody that indicates the to SceneKit physics engine that this geometry should be controlled by the physics engine. We then also give each plane that ARKit detects a physicsBody also so that the cubes can interact with the planes (see the Plane.m class in the github repo for more exact details).

Stopping plane detection

Once we have mapped our world and have a number of planes we don’t want ARKit to keep giving us new planes and potentially updating existing planes, since this may affect geometry we have already added to the world.

In this app if the user holds down two fingers for a second then we hide all of the planes and turn off Plane detection. To do that you update the planeDetection property of the ARSession configuration and re-run the session. By default the session will maintain the same coordinate system and any anchors that were found:

// Get our existing session configuration
ARWorldTrackingSessionConfiguration *configuration = (ARWorldTrackingSessionConfiguration *)self.sceneView.session.configuration;
// Turn off future plane detection and updating
configuration.planeDetection = ARPlaneDetectionNone;
// Re-run the session for the changes to take effect
[self.sceneView.session runWithConfiguration:configuration];

Next

In the next article we will take a small step back and look at robustifying some of the code we already wrote adding some UI controls to enable/disable features. We will also play around with lighting and textures to make the inserted geometry seem more realistic.