<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Mark Dawson on Medium]]></title>
        <description><![CDATA[Stories by Mark Dawson on Medium]]></description>
        <link>https://medium.com/@dawson.mark?source=rss-ea8c786721a8------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*cdaZgw_H5z9maNEpcHyGTA.jpeg</url>
            <title>Stories by Mark Dawson on Medium</title>
            <link>https://medium.com/@dawson.mark?source=rss-ea8c786721a8------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 02:28:44 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@dawson.mark/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com" rel="hub"/>
        <item>
            <title><![CDATA[ARKit by Example — Part 4: Realism - Lighting & PBR]]></title>
            <link>https://medium.com/mark-dawson/arkit-by-example-part-4-realism-lighting-pbr-b9a0bedb013e?source=rss-ea8c786721a8------2</link>
            <guid isPermaLink="false">https://medium.com/p/b9a0bedb013e</guid>
            <category><![CDATA[virtual-reality]]></category>
            <dc:creator><![CDATA[Mark Dawson]]></dc:creator>
            <pubDate>Sun, 18 Jun 2017 05:37:00 GMT</pubDate>
            <atom:updated>2017-06-18T05:37:00.247Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1018/1*o848QWJYJxzVJRjbsMnxaQ.png" /><figcaption>Virtual cubes on a counter top</figcaption></figure><p>In this article we are going to insert more realistic looking virtual content in to the scene. We can achieve this by using more detailed models using a technique called Physically Based Rendering (PBR) and also a more accurate representation of lighting in the scene.</p><p>To see the updates, check out the video below, instead of just plane solid boring cubes we now have added some PBR based materials that give us a much more realistic object that seems to fit into the real world, with variable lighting and reflections.</p><iframe src="https://cdn.embedly.com/widgets/media.html?url=https%3A%2F%2Ffd.xuwubk.eu.org%3A443%2Fhttp%2Fwww.youtube.com%2Fwatch%3Fv%3DrNFQl7I4T6Y&amp;src=https%3A%2F%2Ffd.xuwubk.eu.org%3A443%2Fhttps%2Fwww.youtube.com%2Fembed%2FrNFQl7I4T6Y&amp;type=text%2Fhtml&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/34ad13b4d832db6ca2914ddc4de84aed/href">https://medium.com/media/34ad13b4d832db6ca2914ddc4de84aed/href</a></iframe><p>If you haven’t read the other articles in the series, you can find a list <a href="https://blog.markdaws.net/apple-arkit-by-example-ef1c8578fb59">here</a>.</p><h3>Scene Lighting</h3><p>One of the main aims of Augmented Reality is to mix virtual content with the real world. Sometimes the content we add may be stylized and not look “real” but other times we want to insert content that looks and feels like it is part of the actual space we are interacting with.</p><h4>Intensity</h4><p>In order to achieve a high level of realism, lighting the scene is very important. Trying to model the real-world lighting as closely as possible in the virtual scene will make content you insert feel more real.</p><p>For example, if you are in a dimly lit room and insert a 3D model which is lit using a bright light it’s going to look totally out of place and vice versa, a dimly lit 3D object in a bright room is going to feel out of place.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1019/1*EHRiizG-VtzhmIL9RUMbSQ.png" /><figcaption>A brightly lit cube in a dark environment — feels totally out of place</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RMolADbOF-bxmmbDKYb6IQ.png" /><figcaption>A dark cube in a brightly lit location (also the direction of the light is off)</figcaption></figure><p>So let’s start from the beginning and build up to higher and higher levels of realism. First, if your virtual scene has no lights, then just as in the real world all of the content will be black, there is no light to reflect of the object surfaces. If we turn off the lights in our scene and insert some cubes you will see the following result:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VZf42yoYgl8sChkAKyX8vg.png" /><figcaption>Two virtual cubes on a real world surface, without any lighting in the scene</figcaption></figure><p>Now we need to add some lights to our scene, in 3D graphics there are various different kinds of lights you can add to a scene:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/719/1*8BSG48yy39Y7KqrMZGw5Pw.png" /><figcaption>Examples of different lighting modes — SceneKit</figcaption></figure><p><strong>Ambient</strong> — simulates an equal amount of light hitting the object from all directions. As you can see since light hits the object equally from every direction there are no shadows.</p><p><strong>Directional</strong> — directional light has a direction but no source location, just imagine an infinite place emitting light from the surface.</p><p><strong>Omni </strong>— also known as a spot light. This is a light that has direction (like directional) but also a position. This is useful if you want to perform calculations like how intense the light is based on the distance of the geometry from the light source.</p><p><strong>Spot </strong>— spot light is like omni, but as well as direction and position, a spot light falls off in intensity in a cone shape, just like a spot light on your desk.</p><p>There are some other types of lights but we don’t need to use those right now, for more info you can read the SceneKit documentation for <a href="https://developer.apple.com/documentation/scenekit/scnlight/light_types">SCNLight</a>.</p><h4>autoenablesDefaultLighting</h4><p>SceneKit SCNView has a property called <a href="https://developer.apple.com/documentation/scenekit/scnscenerenderer/1523812-autoenablesdefaultlighting">autoenablesDefaultLighting</a> if you set this to true, SceneKit will add an Onmi directional light to the scene, located from the position of the camera and pointing in the direction of the camera, this is a good starting point and it is enabled by default on your project (that is what has been lighting all the cubes in the previous articles in this <a href="https://blog.markdaws.net/apple-arkit-by-example-ef1c8578fb59">series</a>) until you add your own light sources. A couple of problems with this easy settings are:</p><ul><li>The intensity of the light is always 1000 which means “normal” so again placing content in different lighting situations will not look right.</li><li>The light has a changing direction, so as you walk around an object it will always look like the light is coming from your point of view (like you are holding a torch infront of you) which isn’t the case normally, most scenes have static lighting so your model will look unnatural as you move around.</li></ul><p>We are going to turn this off for our app:</p><pre>self.sceneView.autoenablesDefaultLighting = NO;</pre><h4>automaticallyUpdatesLighting</h4><p>ARKit’s ARSCNView has a property called <a href="https://developer.apple.com/documentation/arkit/arscnview/2887446-automaticallyupdateslighting?language=objc">automaticallyUpdatesLighting</a> which the documentation says will automatically add lights to the scene based on the estimated light intensity. Sounds great, but as far as I can tell it does nothing, setting this in various combinations with other properties didn’t seem to do anything, not sure if it is a bug in this release of the SDK or if I am doing something wrong (more likely), but it doesn’t matter since we can get estimated lighting another way which we will do now.</p><h4>lightEstimationEnabled</h4><p>The ARSessionConfiguration class has a l<a href="https://developer.apple.com/documentation/arkit/arsessionconfiguration/2867268-lightestimationenabled?language=objc">ightEstimationEnabled</a> property, setting this to true, inside every captured ARFrame, we will get a <a href="https://developer.apple.com/documentation/arkit/arframe/2878306-lightestimate?language=objc">lightEstimate</a> value that we can use to render the scene.</p><p>With this information we can take the lightEstimate every frame and modify the intensity of the lights in our scene to mimic the ambient light intensity of the real world scene which helps the too bright/ too dim issue mentioned above.</p><h3>Lighting</h3><p>First let’s add a light to the scene, we will add a spot light that is pointing directly down and insert it into the scene a few meters above the origin. This roughly simulates the environment I am in making the videos in my house where I have spot lights in the ceiling. To add a spot light you:</p><pre>- (void)insertSpotLight:(SCNVector3)position {<br>  SCNLight *spotLight = [SCNLight light];<br>  spotLight.type = SCNLightTypeSpot;<br>  spotLight.spotInnerAngle = 45;<br>  spotLight.spotOuterAngle = 45;</pre><pre>  SCNNode *spotNode = [SCNNode new];<br>  spotNode.light = spotLight;<br>  spotNode.position = position;</pre><pre>  // By default the stop light points directly down the negative<br>  // z-axis, we want to shine it down so rotate 90deg around the <br>  // x-axis to point it down<br>  spotNode.eulerAngles = SCNVector3Make(-M_PI / 2, 0, 0);<br>  [self.sceneView.scene.rootNode addChildNode: spotNode];<br>}</pre><p>As well as a spot light we also add an ambient light, this is because in the real world there are usually multiple light sources and light bouncing off wall and other physical objects that provide light to all sides of an object. The process is similar to above, I’ll omit it from here. When we do this we can now insert a piece of geometry and have it feel more like it is actually part of the scene.</p><h3>Light Estimation</h3><p>Finally, we mentioned lightEstimation, ARKit can analyze the scene and <a href="https://developer.apple.com/documentation/arkit/arlightestimate?language=objc">estimate</a> the ambient light intensity. It will returns a value, 1000 representing “neutral” values below that are darker and values above are brighter. To enable light estimation, you need to set the <a href="https://developer.apple.com/documentation/arkit/arsessionconfiguration/2867268-lightestimationenabled?language=objc">lightEstimationEnabled</a> property to true in your scene configuration:</p><pre>configuration.lightEstimationEnabled = YES;</pre><p>Once you do this, you can then implement the following method in the ARSCNViewDelegate protocol and modify the intensity of the spot light and ambient light we added to the scene:</p><pre>- (void)renderer:(id &lt;SCNSceneRenderer&gt;)renderer updateAtTime:(NSTimeInterval)time {</pre><pre>  ARLightEstimate *estimate = self.sceneView.session.currentFrame.lightEstimate;</pre><pre>  if (!estimate) {<br>    return;<br>  }</pre><pre>  // TODO: Put this on the screen<br>  NSLog(@&quot;light estimate: %f&quot;, estimate.ambientIntensity);</pre><pre>  // Here you can now change the .intensity property of your lights<br>  // so they respond to the real world environment<br>}</pre><p>Now this is very cool, watch the video below, you can see how when I dim down the lights in my house the virtual cube also gets darker as the virtual lights are dimmed! Then it gets brighter as the lights get brighter.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Ffd.xuwubk.eu.org%3A443%2Fhttps%2Fwww.youtube.com%2Fembed%2F7Kk6iVr5ULo%3Ffeature%3Doembed&amp;url=https%3A%2F%2Ffd.xuwubk.eu.org%3A443%2Fhttps%2Fwww.youtube.com%2Fwatch%3Fv%3D7Kk6iVr5ULo&amp;image=https%3A%2F%2Ffd.xuwubk.eu.org%3A443%2Fhttps%2Fi.ytimg.com%2Fvi%2F7Kk6iVr5ULo%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/2b4e22cf3a81db4c51881b7e9185469d/href">https://medium.com/media/2b4e22cf3a81db4c51881b7e9185469d/href</a></iframe><p>The takeaway from this is that getting the lighting to match the real world is tricky, but we have a useful blunt tool in the form of lighting estimation that can help us with some realism. I think the general guideline here is that ideally you make sure your users are using your app in a well lit environment that has consistent lighting that can be modeled easily.</p><h3>Physically Based Rendering</h3><p>Ok, so we have the concept of basic lighting, we will throw 99% of that away :) Instead of trying to add lights to our scene and handle the complexity, we are going to keep the lighting estimation functionality but texture our geometry using a technique called <a href="https://www.marmoset.co/posts/basic-theory-of-physically-based-rendering/">Physically Based Rendering</a>.</p><p>Some examples of objects rendered using this technique are shown below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/971/1*jYY9kszw-HoZvHpJm48mpQ.png" /><figcaption>Image from <a href="https://www.marmoset.co/posts/tag/pbr/page/5/">https://www.marmoset.co/posts/tag/pbr/page/5/</a></figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/698/1*AGEgWXzAO6KK1Wmqu618rA.png" /><figcaption>Image from: <a href="https://medium.com/@avihay/amazing-physically-based-rendering-using-the-new-ios-10-scenekit-2489e43f7021">https://medium.com/@avihay/amazing-physically-based-rendering-using-the-new-ios-10-scenekit-2489e43f7021</a></figcaption></figure><p>For an interesting SceneKit article on this, check out this article:</p><p><a href="https://medium.com/@avihay/amazing-physically-based-rendering-using-the-new-ios-10-scenekit-2489e43f7021">Amazing Physically Based Rendering Using the New iOS 10 SceneKit</a></p><p>I’m not going to try to explain all the details of this process in this article because there are many excellent resources, but the basic concept is that when you texture your object you provide information that includes:</p><p><strong>Albedo</strong> — this is the base color of the model. It maps to the diffuse component of your material, it is the material texture without any baked in lighting or shadow information.</p><p><strong>Roughness </strong>— describes how rough the material will be, the rougher surfaces show dimmer reflections, smoother materials show brighter specular reflections.</p><p><strong>Metalness</strong> — a rough equivalent to how shiny a material will be.</p><p>See <a href="https://www.marmoset.co/posts/tag/pbr/page/5/">https://www.marmoset.co/posts/tag/pbr/page/5/</a> for a much more detailed explanation.</p><p>For our purposes we just want to render our cubes and planes with more realism, for that I grabbed some textures from <a href="http://freepbr.com/">http://freepbr.com/</a> and rendered the materials using them:</p><pre>mat = [SCNMaterial new];<br>mat.lightingModelName = SCNLightingModelPhysicallyBased;<br>mat.diffuse.contents = [UIImage imageNamed:@&quot;wood-albedo.png&quot;];<br>mat.roughness.contents = [UIImage imageNamed:@&quot;wood-roughness.png&quot;];<br>mat.metalness.contents = [UIImage imageNamed:@&quot;wood-metal.png&quot;];<br>mat.normal.contents = [UIImage imageNamed:@&quot;wood-normal.png&quot;];</pre><p>You need to set the lightingModelName of the material to SCNLightingModelPhysicallyBased and set the various material types.</p><p>The final important part is that you have to tell your SCNScene you are using PBR lighting, when you do this the light source for the scene actually comes from an image you specify, for example I used this image:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/695/1*rLncYnZmC7YeZVgJ6UnSJQ.png" /><figcaption>An example environment map, from: <a href="https://medium.com/@avihay/amazing-physically-based-rendering-using-the-new-ios-10-scenekit-2489e43f7021">https://medium.com/@avihay/amazing-physically-based-rendering-using-the-new-ios-10-scenekit-2489e43f7021</a></figcaption></figure><p>So the lighting of your geometry is taken from this image, think of the image being projected all around the geometry as a background then SceneKit is using this background to figure out how the geometry is being lit.</p><pre>UIImage *env = [UIImage imageNamed: @&quot;spherical.jpg&quot;];<br>self.sceneView.scene.lightingEnvironment.contents = env;</pre><p>The final part is taking the light estimation value we get from ARKit and applying it to the intensity of this environment image. ARKit returns a value of 1000 to represent neutral lighting, so less that that is darker and more is brighter. The lighting environment value takes a value of 1.0 for neutral, so we need to scale the value we get from ARKit:</p><pre>CGFloat intensity = estimate.ambientIntensity / 1000.0;<br>self.sceneView.scene.lightingEnvironment.intensity = intensity;</pre><h3><strong>UI Improvements</strong></h3><p>I changed the UI so now if you press and hold with a single finger on a plane, it will change the material, same for the cubes, press and hold to change the material of the cube. Press and hold with two fingers to cause an explosion.</p><p>I also added a toggle to stop plane detection once you are happy with the planes you have found, and a settings screen to turn on/off various debug items.</p><p>As always, you can find the code for this project here: <a href="https://github.com/markdaws/arkit-by-example">https://github.com/markdaws/arkit-by-example</a></p><h3>Next</h3><p>So far we wrote the app assuming the happy path, that nothing will go wrong, but in the real world especially with tracking we know that is not always the case. There are a number of scenarios we need to handle when it comes to ARKit to make our app more robust, in the next article we will take a step back and handle the error and degradation cases.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b9a0bedb013e" width="1" height="1" alt=""><hr><p><a href="https://medium.com/mark-dawson/arkit-by-example-part-4-realism-lighting-pbr-b9a0bedb013e">ARKit by Example — Part 4: Realism - Lighting &amp; PBR</a> was originally published in <a href="https://medium.com/mark-dawson">Mark Dawson</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[ARKit by Example — Part 3: Adding geometry and physics fun]]></title>
            <link>https://medium.com/mark-dawson/arkit-by-example-part-3-adding-geometry-and-physics-fun-8dd2b90037f2?source=rss-ea8c786721a8------2</link>
            <guid isPermaLink="false">https://medium.com/p/8dd2b90037f2</guid>
            <category><![CDATA[augmented-reality]]></category>
            <category><![CDATA[scenekit]]></category>
            <category><![CDATA[apple]]></category>
            <category><![CDATA[arkit]]></category>
            <dc:creator><![CDATA[Mark Dawson]]></dc:creator>
            <pubDate>Tue, 13 Jun 2017 23:11:11 GMT</pubDate>
            <atom:updated>2017-06-13T23:11:11.490Z</atom:updated>
            <content:encoded><![CDATA[<p>In the <a href="https://blog.markdaws.net/arkit-by-example-part-2-plane-detection-visualization-10f05876d53">last article</a> we used ARKit to detect horizontal planes in the real world, then visualized those planes. In this article we are now going to start adding virtual content to our AR experience and start interacting with the planes that were detected.</p><p>By the end of this article we will be able to drop cubes into the world, apply realistic physics to the cubes so they interact with one another and also create mini shock waves to make the cubes fly around a bit.</p><p>Here is a video showing the app in action, you can see how first we capture the horizontal planes, then we add some 3D cubes to interact with the scene and then finally cause some mini explosions to make the cubes jump around:</p><iframe src="https://cdn.embedly.com/widgets/media.html?url=https%3A%2F%2Ffd.xuwubk.eu.org%3A443%2Fhttp%2Fwww.youtube.com%2Fwatch%3Fv%3DVsk9erdCvdk&amp;src=https%3A%2F%2Ffd.xuwubk.eu.org%3A443%2Fhttps%2Fwww.youtube.com%2Fembed%2FVsk9erdCvdk&amp;type=text%2Fhtml&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/f1afc00022987c2fa435b85bd480506e/href">https://medium.com/media/f1afc00022987c2fa435b85bd480506e/href</a></iframe><p>As always, you can follow along with the code here: <a href="https://github.com/markdaws/arkit-by-example/tree/part3">https://github.com/markdaws/arkit-by-example/tree/part3</a></p><h3>Hit Testing</h3><p>As you saw in the first tutorial we can insert virtual 3D content at any X,Y,Z position and it will render and track in the real world. Now that we have plane detection we want to add content that interacts with those planes. This will make the app look like there are object on top of your table, chairs, floor etc.</p><p>In this app, when the user single taps on the screen, we perform a hit test, this involves taking the 2D screen coordinates and firing a Ray from the camera origin through the 2D screen point (which has a 3D position on the projection plane) and into the scene. If the ray intersects with any plane we get a hit result, we then take the 3D coordinate where the ray and plane intersected and place our content at that 3D position.</p><p>The code for this is pretty simple, ARSCNView contains a hitTest method, you pass in the screen coordinates and it takes care of projecting a ray in 3D through that point from the camera origin and returning results:</p><pre>- (void)handleTapFrom: (UITapGestureRecognizer *)recognizer {</pre><pre>  // Take the screen space tap coordinates and pass them to the<br>  // hitTest method on the ARSCNView instance<br>  CGPoint tapPoint = [recognizer locationInView:self.sceneView];</pre><pre>  NSArray&lt;ARHitTestResult *&gt; *result = [self.sceneView   hitTest:tapPoint types:ARHitTestResultTypeExistingPlaneUsingExtent];</pre><pre>  // If the intersection ray passes through any plane geometry they<br>  // will be returned, with the planes ordered by distance <br>  // from the camera<br>  if (result.count == 0) {<br>    return;<br>  }</pre><pre>  // If there are multiple hits, just pick the closest plane<br>  ARHitTestResult * hitResult = [result firstObject];<br>  [self insertGeometry:hitResult];<br>}</pre><p>Given a ARHitTestResult, we can get the world coordinate where the ray/plane intersection took place and place some virtual content at that location. For this article we will just insert a simple cube, later we will make the objects look more realistic:</p><pre>- (void)insertGeometry:(ARHitTestResult *)hitResult {</pre><pre>float dimension = 0.1;<br>SCNBox *cube = [SCNBox boxWithWidth:dimension <br>                             height:dimension <br>                             length:dimension <br>                      chamferRadius:0];</pre><pre>SCNNode *node = [SCNNode nodeWithGeometry:cube];</pre><pre>// The physicsBody tells SceneKit this geometry should be<br>// manipulated by the physics engine<br>node.physicsBody = [SCNPhysicsBody         <br>                     bodyWithType:SCNPhysicsBodyTypeDynamic <br>                            shape:nil];</pre><pre>node.physicsBody.mass = 2.0;<br>node.physicsBody.categoryBitMask = CollisionCategoryCube;</pre><pre>// We insert the geometry slightly above the point the user tapped<br>// so that it drops onto the plane using the physics engine</pre><pre>float insertionYOffset = 0.5;<br>node.position = SCNVector3Make(<br>  hitResult.worldTransform.columns[3].x,<br>  hitResult.worldTransform.columns[3].y + insertionYOffset,<br>  hitResult.worldTransform.columns[3].z<br>);</pre><pre>// Add the cube to the scene<br>[self.sceneView.scene.rootNode addChildNode:node];</pre><pre>// Add the cube to an internal list for book-keeping<br>[self.boxes addObject:node];</pre><pre>}</pre><h3>Adding some physics</h3><p>AR is suppose to augment the real world, so in order to make our objects feel a bit more realistic, we will add some physics to give then a feeling of weight.</p><p>As you can see in the code above we give each cube a physicsBody that indicates the to SceneKit physics engine that this geometry should be controlled by the physics engine. We then also give each plane that ARKit detects a physicsBody also so that the cubes can interact with the planes (see the <a href="https://github.com/markdaws/arkit-by-example/blob/part3/arkit-by-example/Plane.m">Plane.m</a> class in the github repo for more exact details).</p><h3>Stopping plane detection</h3><p>Once we have mapped our world and have a number of planes we don’t want ARKit to keep giving us new planes and potentially updating existing planes, since this may affect geometry we have already added to the world.</p><p>In this app if the user holds down two fingers for a second then we hide all of the planes and turn off Plane detection. To do that you update the planeDetection property of the ARSession configuration and re-run the session. By default the session will maintain the same coordinate system and any anchors that were found:</p><pre>// Get our existing session configuration<br>ARWorldTrackingSessionConfiguration *configuration = (ARWorldTrackingSessionConfiguration *)self.sceneView.session.configuration;</pre><pre>// Turn off future plane detection and updating<br>configuration.planeDetection = ARPlaneDetectionNone;</pre><pre>// Re-run the session for the changes to take effect<br>[self.sceneView.session runWithConfiguration:configuration];</pre><h3>Next</h3><p>In the next article we will take a small step back and look at robustifying some of the code we already wrote adding some UI controls to enable/disable features. We will also play around with lighting and textures to make the inserted geometry seem more realistic.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8dd2b90037f2" width="1" height="1" alt=""><hr><p><a href="https://medium.com/mark-dawson/arkit-by-example-part-3-adding-geometry-and-physics-fun-8dd2b90037f2">ARKit by Example — Part 3: Adding geometry and physics fun</a> was originally published in <a href="https://medium.com/mark-dawson">Mark Dawson</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[ARKit By Example — Part 2: Plane Detection + Visualization]]></title>
            <link>https://medium.com/mark-dawson/arkit-by-example-part-2-plane-detection-visualization-10f05876d53?source=rss-ea8c786721a8------2</link>
            <guid isPermaLink="false">https://medium.com/p/10f05876d53</guid>
            <category><![CDATA[apple]]></category>
            <category><![CDATA[arkit]]></category>
            <category><![CDATA[augmented-reality]]></category>
            <category><![CDATA[virtual-reality]]></category>
            <dc:creator><![CDATA[Mark Dawson]]></dc:creator>
            <pubDate>Sun, 11 Jun 2017 07:27:30 GMT</pubDate>
            <atom:updated>2017-06-11T07:27:30.847Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KgucYRSsQ0Rvna0uGoQPRw.png" /><figcaption>ARKit — plane detected and mapped to floor</figcaption></figure><p>In our first <a href="https://blog.markdaws.net/arkit-by-example-part1-7830677ef84d">hello world</a> ARKit app we setup our project and rendered a single virtual 3D cube which would render in the real world and track as you moved around.</p><p>In this article, we will look at extracting 3D geometry from the real world and visualizing it. Detecting geometry is very important for Augmented Reality apps because if you want to be able to feel like you are interacting with the real world you need to know that the user clicked on a table top, or is looking at the floor, or another surface to get life-like 3D interactions. Once we have the plane detection completed in this article, in a future article we will use them to place virtual objects in the real world.</p><p>ARKit can detect horizontal planes (I suspect in the future ARKit will detect more complex 3D geometry but we will probably have to wait for a depth sensing camera for that, iPhone8 maybe…). Once we detect a plane, we will visualize it to show the scale and orientation of the plane. To see it all in action, watch the video below:</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Ffd.xuwubk.eu.org%3A443%2Fhttps%2Fwww.youtube.com%2Fembed%2F9ZnG9wrVxtM%3Ffeature%3Doembed&amp;url=https%3A%2F%2Ffd.xuwubk.eu.org%3A443%2Fhttp%2Fwww.youtube.com%2Fwatch%3Fv%3D9ZnG9wrVxtM&amp;image=https%3A%2F%2Ffd.xuwubk.eu.org%3A443%2Fhttps%2Fi.ytimg.com%2Fvi%2F9ZnG9wrVxtM%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/d2350ef37332d1351c270f9fe7d9d06a/href">https://medium.com/media/d2350ef37332d1351c270f9fe7d9d06a/href</a></iframe><p><strong>NOTE: It is assumed you are following along from the code located here: </strong><a href="https://github.com/markdaws/arkit-by-example/tree/part2">https://github.com/markdaws/arkit-by-example/tree/part2</a></p><h3>Computer Vision Concepts</h3><p>Before we dive in to the code, it is useful to know at a high level what’s happening beneath the covers in ARKit, because this technology is not perfect and there are certain situations where it will break down and affect the performance of your app.</p><p>The aim of Augmented Reality is to be able to insert virtual content in to the real world at specific points and have that virtual content track as you move around the real world. With ARKit the basic process for this involves reading video frames from the iOS device camera, for each frame the image is processed and feature points are extracted. Features can be many things, but you want to try to detect interesting features in the image that you can track across multiple frames. A feature may be the corner of an object or the edge of a textured piece of fabric etc. There are many ways to generate these features, you can read more on that across the web (e.g. search for SIFT) but for our purpose it’s enough to know that there are many features extracted per image which can be uniquely identified.</p><p>Once you have the features for an image, you can then track the features across multiple frames, as the user moves around the world, you can take these corresponding points and estimate 3D pose information, such as the current camera position and the positions of the features. As the user moves around more and we get more and more features, these 3D pose estimations improve.</p><p>For plane detection, once you have a number of feature points in 3D you can then try to fit planes to those points and find the best match in terms of scale, orientation and position. ARKit is constantly analyzing the 3D feature points and reporting all the planes it finds back to us in the code.</p><p>Below is a screenshot from my phone looking at the arm of my sofa. As you can see the fabric has good texture, plenty of interesting and unique features that can be tracked, each cross is a unique feature found by ARKit.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WY5UmghrTfumK6PEDn-hWw.png" /><figcaption>ARKit Feature Point Extraction — Sofa Fabric</figcaption></figure><p>This next picture is one I took of my fridge door, notice how there are not many points:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bvNKXemL0gGOPMlQ3H0zeQ.png" /><figcaption>ARKit Feature Extraction — poor matches on fridge door reflection</figcaption></figure><p>This is important because for ARKit to detect features you have to be looking at content that has lots of interesting features to detect. Things that would cause poor feature extraction are:</p><ol><li><strong>Poor lighting</strong> — not enough light or too much light with shiny specular highlights. Try and avoid environments with poor lighting.</li><li><strong>Lack of texture</strong> — if you point your camera at a white wall there is really nothing unique to extract, ARKit won’t be able to find or track you. Try to avoid looking at areas of solid colors, shiny surfaces etc.</li><li><strong>Fast movement </strong>— this is subjective for ARKit, normally if you are only using images to detect and estimate 3D poses, if you move the camera too fast you will end up with blurry images that will cause tracking to fail. However ARKit uses something called Visual-Inertial Odometry so as well as the image information ARKit uses the devices motion sensors to estimate where the user has turned. This makes ARKit very robust in terms of tracking.</li></ol><p>In another article we will test out different environments to see how the tracking performs.</p><h3>Adding Debug Visualizations</h3><p>Before we start, it’s useful to add some debugging information to the application, namely rendering the world origin reported from ARKit and then also rendering the feature points ARKit has detected, that will be useful to let you know that you are in an area that is tracking well or not. To do this we can turn on the debug options on our ARSCNView instance:</p><pre>self.sceneView.debugOptions = <br>  ARSCNDebugOptionShowWorldOrigin | <br>  ARSCNDebugOptionShowFeaturePoints;</pre><h3>Detecting plane geometry</h3><p>In ARKit you can specify that you want to detect horizontal planes by setting the <a href="https://developer.apple.com/documentation/arkit/arworldtrackingsessionconfiguration/2867263-planedetection?language=objc">planeDetection</a> property on your session configuration object. This value can be set to either ARPlaneDetectionHorizontal or ARPlaneDetectionNone.</p><p>Once you set that property, you will start getting callbacks to delegate methods for the <a href="https://developer.apple.com/documentation/arkit/arscnviewdelegate">ARSCNViewDelegate</a> protocol. There are a number of methods there, the first we will use is:</p><pre>/**<br>Called when a new node has been mapped to the given anchor.</pre><pre>@param renderer The renderer that will render the scene.<br>@param node The node that maps to the anchor.<br>@param anchor The added anchor.<br>*/<br>- (void)renderer:(id &lt;SCNSceneRenderer&gt;)renderer<br>      didAddNode:(SCNNode *)node<br>       forAnchor:(ARAnchor *)anchor {<br>}</pre><p>This method gets called each time ARKit detects what it considers to be a new plane. We get two pieces of information, node and anchor. The SCNNode instance is a SceneKit node that ARKit has created, it has some properties set like the orientation and position, then we get an anchor instance, this tells use more information about the particular anchor that has been found, such as the size and center of the plane.</p><p>The anchor instance is actually an ARPlaneAnchor type, from that we can get the <a href="https://developer.apple.com/documentation/arkit/arplaneanchor/2882055-extent?language=objc">extent</a> and center information for the plane.</p><h3>Rendering the plane</h3><p>With the above information we can now draw a SceneKit 3D plane in the virtual world. To do this we create a <a href="https://github.com/markdaws/arkit-by-example/blob/part2/arkit-by-example/Plane.m">Plane class</a> that inherits from SCNNode. In the constructor method we create the plane and size it accordingly:</p><pre>// Create the 3D plane geometry with the dimensions reported<br>// by ARKit in the ARPlaneAnchor instance<br>self.planeGeometry = [SCNPlane planeWithWidth:anchor.extent.x height:anchor.extent.z];</pre><pre>SCNNode *planeNode = [SCNNode nodeWithGeometry:self.planeGeometry];</pre><pre>// Move the plane to the position reported by ARKit<br>planeNode.position = SCNVector3Make(anchor.center.x, 0, anchor.center.z);</pre><pre>// Planes in SceneKit are vertical by default so we need to rotate<br>// 90 degrees to match planes in ARKit<br>planeNode.transform = SCNMatrix4MakeRotation(-M_PI / 2.0, 1.0, 0.0, 0.0);</pre><pre>// We add the new node to ourself since we inherited from SCNNode<br>[self addChildNode:planeNode];</pre><p>Now that we have our Plane class, back in the ARSCNViewDelegate callback method, we can create our new plane when ARKit reports a new Anchor:</p><pre>- (void)renderer:(id &lt;SCNSceneRenderer&gt;)renderer <br>      didAddNode:(SCNNode *)node <br>       forAnchor:(ARAnchor *)anchor {<br>  if (![anchor isKindOfClass:[ARPlaneAnchor class]]) {<br>    return;<br>  }</pre><pre>  Plane *plane = [[Plane alloc] initWithAnchor: (ARPlaneAnchor *)anchor];</pre><pre>  [node addChildNode:plane];<br>}</pre><p><strong>NOTE:</strong> In the actual code, I also set a grid material on the SCNPlane geometry, to make the visualization look nicer, I have omitted that code here for brevity.</p><h3>Updating the plane</h3><p>If you run the above code, as you walk around you will see new planes rendering in the virtual world, however the planes do not grow properly as you move around. ARKit is constantly analyzing the scene and as it finds that a Plane is bigger/smaller than it thought it updates the planes extent values. So we also need to update the Plane SceneKit is already rendering.</p><p>We get this updated information from another ARSCNViewDelegate method:</p><pre>- (void)renderer:(id &lt;SCNSceneRenderer&gt;)renderer <br>   didUpdateNode:(SCNNode *)node <br>       forAnchor:(ARAnchor *)anchor {</pre><pre>  // See if this is a plane we are currently rendering<br>  Plane *plane = [self.planes objectForKey:anchor.identifier];<br>  if (plane == nil) {<br>    return;<br>  }</pre><pre>  [plane update:(ARPlaneAnchor *)anchor];<br>}</pre><p>Inside the update method of our Plane class, we then update the width and height of the plane:</p><pre>- (void)update:(ARPlaneAnchor *)anchor {</pre><pre>  self.planeGeometry.width = anchor.extent.x;<br>  self.planeGeometry.height = anchor.extent.z;</pre><pre>  // When the plane is first created it&#39;s center is 0,0,0 and <br>  // the nodes transform contains the translation parameters. <br>  // As the plane is updated the planes translation remains the <br>  // same but it&#39;s center is updated so we need to update the 3D<br>  // geometry position<br>  self.position = SCNVector3Make(anchor.center.x, 0, anchor.center.z);</pre><pre>}</pre><p>Now that we have got the plane rendering and updating, we can take a look in the app. I added a Tron style grid texture to the SCNPlane geometry, I have omitted it here but you can look in the source code.</p><h3>Extraction Results</h3><p>Below are some screen shots of the planes detected from the video above as I walked around part of my house:</p><p>This is an island in my kitchen, ARKit did a good job finding the extents and orientating the plane correctly to match the raised surface</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9wAJaYgZ7nBsj9Oap3iuIA.png" /></figure><p>Here is a shot looking at the floor, as you can see as you move around ARKit is constantly surfacing new planes, this is interesting because if you are developing an app, the user will first have to move around in a space before they can place content, it will be important to provide good visual clues to the user when the geometry is good enough to use.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dP6AvQJ52rcuFPyt2QEmyA.png" /></figure><p>Below is the same scene as above, but a few seconds later, ARKit has merged all of the above planes in to a single plane. Note, that in the ARSCNViewDelegate callbacks you have to handle the case where ARKit has deleted a ARPlaneAnchor instance as planes are merged.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*51G5dcUbKXe__op2mLBzTA.png" /></figure><p>This in interesting because I am standing above this floor, about 12 to 15 feet above it, in poor lighting conditions and ARKit still managed to extract a plane at that distance, impressive!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HAKUQCP9a7hAyjK55UNXeg.png" /></figure><p>Here is a plane extracted on top of a small wall next to the stairs. Notice how the plane extends past the edge of the actual surface.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1016/1*h1Mcnnn46HHVWLytd6xpIg.png" /></figure><h3>Recognition takeaways</h3><p>Here are some points I found from plane detection:</p><ol><li>Don’t expect a plane to align perfectly to a surface, as you can see from the video, planes are detected but the orientation might be off, so if you are developing an AR app that wants to get the geometry really accurate for a good effect you might be disappointed.</li><li>Edges are not great, as well as alignment the actual extend of the planes is sometimes too small or large, don’t try to make an app that requires perfect alignment</li><li>Tracking is really robust and fast. As you can see as I walk around the planes track to the real world really well, I’m moving the camera pretty fast and it still looks great</li><li>I was impressed by the feature extraction, even in low light and from a distance of about 12–15 feet away ARKit still extracted some planes.</li></ol><h3>Next</h3><p>In the next article we will use the planes to start placing 3D objects in the real world and also look more into robustification of the app.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=10f05876d53" width="1" height="1" alt=""><hr><p><a href="https://medium.com/mark-dawson/arkit-by-example-part-2-plane-detection-visualization-10f05876d53">ARKit By Example — Part 2: Plane Detection + Visualization</a> was originally published in <a href="https://medium.com/mark-dawson">Mark Dawson</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Apple ARKit by Example]]></title>
            <link>https://medium.com/mark-dawson/apple-arkit-by-example-ef1c8578fb59?source=rss-ea8c786721a8------2</link>
            <guid isPermaLink="false">https://medium.com/p/ef1c8578fb59</guid>
            <category><![CDATA[augmented-reality]]></category>
            <category><![CDATA[ios]]></category>
            <category><![CDATA[arkit]]></category>
            <category><![CDATA[apple]]></category>
            <dc:creator><![CDATA[Mark Dawson]]></dc:creator>
            <pubDate>Sun, 11 Jun 2017 06:34:33 GMT</pubDate>
            <atom:updated>2017-06-18T05:37:50.659Z</atom:updated>
            <content:encoded><![CDATA[<p>With the beta release of ARKit by Apple, I will be writing a series of articles working through a real world example integrating ARKit in iOS. Every time I publish a new part to this series I will update the links below.</p><p>As of the time of writing, you will need XCode Beta 9, and iOS11 beta in order to run and build these projects.</p><h3>Sample Code</h3><p>All sample code can be found here: <a href="https://github.com/markdaws/arkit-by-example">https://github.com/markdaws/arkit-by-example</a> The master branch is set to the most updated code, but there is a branch for each version of this article so you can follow along with a specific article.</p><h3>Articles</h3><p><a href="https://blog.markdaws.net/arkit-by-example-part1-7830677ef84d">Part 1 — Getting setup, draw a cube in virtual reality</a></p><p><a href="https://blog.markdaws.net/arkit-by-example-part-2-plane-detection-visualization-10f05876d53">Part 2 — Plane Detection and Visualization</a></p><p><a href="https://blog.markdaws.net/arkit-by-example-part-3-adding-geometry-and-physics-fun-8dd2b90037f2">Part 3 — Adding geometry and physics fun</a></p><p><a href="https://blog.markdaws.net/arkit-by-example-part-4-realism-lighting-pbr-b9a0bedb013e">Part 4 — Physically Based Rendering</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ef1c8578fb59" width="1" height="1" alt=""><hr><p><a href="https://medium.com/mark-dawson/apple-arkit-by-example-ef1c8578fb59">Apple ARKit by Example</a> was originally published in <a href="https://medium.com/mark-dawson">Mark Dawson</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Apple ARKit by Example]]></title>
            <link>https://medium.com/@dawson.mark/apple-arkit-by-example-12cbc42e0cda?source=rss-ea8c786721a8------2</link>
            <guid isPermaLink="false">https://medium.com/p/12cbc42e0cda</guid>
            <category><![CDATA[arkit]]></category>
            <category><![CDATA[augmented-reality]]></category>
            <category><![CDATA[ios]]></category>
            <dc:creator><![CDATA[Mark Dawson]]></dc:creator>
            <pubDate>Fri, 09 Jun 2017 00:14:37 GMT</pubDate>
            <atom:updated>2017-06-09T00:14:37.170Z</atom:updated>
            <content:encoded><![CDATA[<p>With the beta release of ARKit by Apple, I will be writing a series of articles working through a real world example integrating ARKit in iOS. Every time I publish a new part to this series I will update the links below.</p><p>As of the time of writing, you will need XCode Beta 9, and iOS11 beta in order to run and build these projects.</p><h3>Sample Code</h3><p>All sample code can be found here: <a href="https://github.com/markdaws/arkit-by-example">https://github.com/markdaws/arkit-by-example</a> The master branch is set to the most updated code, but there is a branch for each version of this article so you can follow along with a specific article.</p><h3>Articles</h3><p><a href="https://blog.markdaws.net/arkit-by-example-part1-7830677ef84d">Part 1 — Getting setup, draw a cube in virtual reality</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=12cbc42e0cda" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[ARKit by Example — Part 1: AR Cube]]></title>
            <link>https://medium.com/mark-dawson/arkit-by-example-part1-7830677ef84d?source=rss-ea8c786721a8------2</link>
            <guid isPermaLink="false">https://medium.com/p/7830677ef84d</guid>
            <category><![CDATA[3d-graphics]]></category>
            <category><![CDATA[augmented-reality]]></category>
            <category><![CDATA[ios]]></category>
            <category><![CDATA[arkit]]></category>
            <dc:creator><![CDATA[Mark Dawson]]></dc:creator>
            <pubDate>Fri, 09 Jun 2017 00:13:10 GMT</pubDate>
            <atom:updated>2017-06-09T19:14:25.081Z</atom:updated>
            <content:encoded><![CDATA[<p>This first article is going to create a really simple hello world AR app using ARKit. By the end we will be able to position a 3D cube inside the augmented world and move around it with our iOS device.</p><p>To render 3D content inside ARKit, we are going to use SceneKit: <a href="https://developer.apple.com/scenekit/">https://developer.apple.com/scenekit/</a> This is a framework for rendering 3D graphics on an iOS device. If you are familiar with basic 3D concepts it will be pretty straightforward.</p><p>If you can’t wait until the end of the article, here is a video of the app in action. As you can see using ARKit we can place virtual objects in the real world and have them stick in place as we move the camera around.</p><p>Even though this is a very simple app, we will keep building more and more features through these series of articles, including geometry detection, 3D physics and lots more fun things.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Ffd.xuwubk.eu.org%3A443%2Fhttps%2Fwww.youtube.com%2Fembed%2FcHuitQ0WAX0%3Ffeature%3Doembed&amp;url=https%3A%2F%2Ffd.xuwubk.eu.org%3A443%2Fhttp%2Fwww.youtube.com%2Fwatch%3Fv%3DcHuitQ0WAX0&amp;image=https%3A%2F%2Ffd.xuwubk.eu.org%3A443%2Fhttps%2Fi.ytimg.com%2Fvi%2FcHuitQ0WAX0%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/9adfc73e5fa5b71fb65742e7024f206c/href">https://medium.com/media/9adfc73e5fa5b71fb65742e7024f206c/href</a></iframe><h3>Requirements</h3><p>As of the time of writing you need to have an iOS device with an A9/A10 processor in it to support ARKit. This means an iPhone 6S or better, or an iPad 2017 or better.</p><p>For the sfotware, you need to install:</p><ul><li>iOS 11 Beta: <a href="https://developer.apple.com/download/">https://developer.apple.com/download/</a></li><li>XCode 9 Beta: <a href="https://developer.apple.com/download/">https://developer.apple.com/download/</a></li></ul><p>Once you have those installed you are ready to go.</p><h3>Creating the project</h3><p>First open XCode, choose the ARKit project template:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/745/1*4d-Qovl_HSLbeEEB1MDEdg.png" /><figcaption>XCode 9 Beta - New Project Template Chooser</figcaption></figure><p>Fill out the project details, make sure you choose “Scene Kit” as the Content Technology option. By default is may say “Sprite Kit” which is used for 2D rendering, but we want “Scene Kit” which is used for rendering 3D content.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/431/1*YxNXFEY8IHUVz_Hde7GXeA.png" /></figure><p>Run the project, if everything is working as expected on your device you should see an app that shows the live camera feed and a 3D model of an airplane that is positioned in your physical space.</p><p>Move around and see how the plane stays is the same position in the real world even as you move the camera:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/871/1*Azhl1SRVJD8680S4Qxznjw.png" /></figure><p>The above example in the sample project is actually more advanced than we are going to create, but the whole point here is to learn how to code these project from scratch, so open up the project, and remove all the code from the viewDidLoad method (apart from the super call).</p><h3>ARKit core classes</h3><p><a href="https://developer.apple.com/documentation/arkit/arscnview"><strong>ARSCNView</strong></a> — a helper view that helps augment the real-time camera view with 3D content rendered by SceneKit. There are a couple of things this class does:</p><ul><li>Renders a live video stream from your device camera in the view as a background to your 3D scene</li><li>The 3D coordinate system of ARKit matches the 3D coordinate system of SceneKit, so object rendered in this view will automatically match the augmented ARKit world view</li><li>Automatically moves the virtual SceneKit 3D camera to match the 3D position tracked by ARKit, so there is no extra code needed to hook up the ARKit movement events to map to the SceneKit 3D rendering.</li></ul><p><a href="https://developer.apple.com/documentation/arkit/arsession"><strong>ARSession</strong></a> — every Augmented Reality session requires an ARSession instance. It is responsible for controlling the camera, gathering all of the sensor data from the device etc to build this seamless experience. The ARSCNView instance already has an ARSession instance, you just need to configure it at startup.</p><p><a href="https://developer.apple.com/documentation/arkit/arworldtrackingsessionconfiguration"><strong>ARWorldTrackingSessionConfiguration</strong></a> — this class indicates to the ARSession that we want to use six degrees of freedom for tracking the user in the real world, roll, pitch, yaw and translation in X, Y and Z. This allows us to create AR experiences where you can not only rotate in the same spot to see augmented content, but also move around object in the 3D space. If you don’t need the translation part and the user will stay still as you project augmented content, you can use the ARSessionConfiguration class instead to initialize the ARSession instance.</p><p>For part 1 of this series we only need these classes, there are many more but this is a good starting point. Going back to our project, we can see in the viewWillAppear method that the ARSession instance is initialized, the self.sceneView refers to a ARSCNView instance.</p><pre>- (void)viewWillAppear:(BOOL)animated {<br>  [super viewWillAppear:animated];</pre><pre>  // Create a session configuration<br>  ARWorldTrackingSessionConfiguration *configuration = [ARWorldTrackingSessionConfiguration new];</pre><pre>  // Run the view&#39;s session<br>  [self.sceneView.session runWithConfiguration:configuration];<br>}</pre><h3>Drawing the Cube</h3><p>We are going to draw a 3D Cube using SceneKit. SceneKit has a couple of basic classes, SCNScene is a container for all your 3D content, you can add multiple pieces of 3D geometry to the scene, in various positions, rotations, scales etc.</p><p>To add content to a scene, you first create some Geometry, geometry can be complex shapes, or simple ones like sphere, cube, plane etc. You then wrap the geometry in a scene node and add it to the scene. SceneKit will then traverse the scene graph and render the content.</p><p>To add a scene and draw a cube we would add the following code inside the viewDidLoad method:</p><pre>- (void)viewDidLoad {<br>  [super viewDidLoad];</pre><pre>  // Container to hold all of the 3D geometry  <br>  SCNScene *scene = [SCNScene new];</pre><pre>  // The 3D cube geometry we want to draw<br>  SCNBox *boxGeometry = [SCNBox <br>                          boxWithWidth:0.1 <br>                          height:0.1 <br>                          length:0.1 <br>                          chamferRadius:0.0];</pre><pre>  // The node that wraps the geometry so we can add it to the scene<br>  SCNNode *boxNode = [SCNNode nodeWithGeometry:boxGeometry];</pre><pre>  // Position the box just in front of the camera<br>  boxNode.position = SCNVector3Make(0, 0, -0.5);</pre><pre>  // rootNode is a special node, it is the starting point of all<br>  // the items in the 3D scene<br>  [scene.rootNode addChildNode: boxNode];</pre><pre>  // Set the scene to the view<br>  self.sceneView.scene = scene;<br>}</pre><p>The coordinates in ARKit roughly correspond to meters, so in this case we are creating a 10x10x10 centimeter box.</p><p>The coordinate system for ARKit and SceneKit looks like the following:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/416/1*fOeokerm3049ltZhG6e-vQ.png" /></figure><p>As you can see in the code code above the position the camera -0.5 units infront of the camera, since the camera faces in the negative Z direction.</p><p>When the ARSession starts up the calculated camera position is initially set as X=0, Y=0, Z=0.</p><p>If you now run the sample, you should see a small 3D cube floating in space that maintains it’s position when you move around, you should be able to walk all the way around, look under, above it.</p><p>One quick tweak we will want to make is to add some default lighting in the 3D scene so that we can see the sides of the cubes, we can add some more advanced lighting later but for now we can just set the autoenablesDefaultLighting on the SCNScene instance:</p><pre>self.sceneView.autoenablesDefaultLighting = YES;</pre><h3>Next Article</h3><p>In the next article we will start making our app a bit more interesting, adding some more complex objects, detecting planes in the scene and interacting with geometry in the scene, stay tuned …</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7830677ef84d" width="1" height="1" alt=""><hr><p><a href="https://medium.com/mark-dawson/arkit-by-example-part1-7830677ef84d">ARKit by Example — Part 1: AR Cube</a> was originally published in <a href="https://medium.com/mark-dawson">Mark Dawson</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Apple ARKit — Augmented Reality in iOS]]></title>
            <link>https://medium.com/mark-dawson/apple-arkit-augmented-reality-in-ios-e762e33b900b?source=rss-ea8c786721a8------2</link>
            <guid isPermaLink="false">https://medium.com/p/e762e33b900b</guid>
            <category><![CDATA[augmented-reality]]></category>
            <category><![CDATA[arkit]]></category>
            <category><![CDATA[ios]]></category>
            <dc:creator><![CDATA[Mark Dawson]]></dc:creator>
            <pubDate>Wed, 07 Jun 2017 07:31:31 GMT</pubDate>
            <atom:updated>2017-06-07T07:31:31.975Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/945/1*Pm383swnJOvjAo0bq-fbTQ.png" /></figure><h3>Overview</h3><p>Apple just released a beta of ARKit for iOS11 Beta. If you didn’t watch WWDC 2017, you can see an example of ARKit in action here: <a href="https://www.youtube.com/watch?v=S14AVwaBF-Y">https://www.youtube.com/watch?v=S14AVwaBF-Y</a></p><p>This is Apple’s first step into the world of Augmented Reality, it’s really exciting in that you can build AR apps for a large part of the Apple ecosystem.</p><p>You can find a download button for the SDK and the release notes here: <a href="https://developer.apple.com/arkit/">https://developer.apple.com/arkit/</a></p><h3>Requirements</h3><p>ARKit requires iOS11+ to run, as of the time of writing you can download iOS11 Beta from the Apple developer downloads page: <a href="https://developer.apple.com/download/">https://developer.apple.com/download/</a></p><p>For the hardware, the release notes say it is limited to A9/A10 processors, so that means you need an iPhone SE, iPhone6S, iPhone 6S Plus, or newer to run the apps, or on the iPad and iPad 2017 model.</p><p>As soon as I have had chance to poke around and kick the tires I will post some more articles and tutorials.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e762e33b900b" width="1" height="1" alt=""><hr><p><a href="https://medium.com/mark-dawson/apple-arkit-augmented-reality-in-ios-e762e33b900b">Apple ARKit — Augmented Reality in iOS</a> was originally published in <a href="https://medium.com/mark-dawson">Mark Dawson</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[golang — version injection]]></title>
            <link>https://medium.com/mark-dawson/golang-version-injection-dae0c1b78c08?source=rss-ea8c786721a8------2</link>
            <guid isPermaLink="false">https://medium.com/p/dae0c1b78c08</guid>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[golang]]></category>
            <category><![CDATA[versioning]]></category>
            <dc:creator><![CDATA[Mark Dawson]]></dc:creator>
            <pubDate>Mon, 02 Jan 2017 21:09:03 GMT</pubDate>
            <atom:updated>2017-01-02T21:09:03.885Z</atom:updated>
            <content:encoded><![CDATA[<p>Sometimes you want to have access to a version number inside your go application, for example you might have a command line app that supports a “-version” option which prints the current app version.</p><p>One way to do this is by using the go linker to inject the value into a variable in your code at compile time.</p><p>In the code below we have a variable called “version” which we simply print to the console:</p><pre>package main</pre><pre>import &quot;fmt&quot;</pre><pre>var version string</pre><pre>func main() {<br>    fmt.Println(version)<br>}</pre><p>To set the “version” variable at build time, you specify the -ldflags option like so:</p><pre>go install -ldflags &quot;-X main.version=12345&quot; cmd/myapp</pre><p>This injects 12345 as the initial value to the “version” variable. You can make this more useful by using a date as your build value, or in some cases maybe you store the version in a file, in some of my projects I have a file called VERSION which contains a string like “1.5.9”, to inject that you can use the cat command to pipe it to the go command:</p><pre>go install -ldflags &quot;-X main.version=`cat ./VERSION`&quot; cmd/myapp</pre><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=dae0c1b78c08" width="1" height="1" alt=""><hr><p><a href="https://medium.com/mark-dawson/golang-version-injection-dae0c1b78c08">golang — version injection</a> was originally published in <a href="https://medium.com/mark-dawson">Mark Dawson</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>