Image Tracker iOS Tutorial
1. Overview
2. iOS Development
2.1 Create Instants
2.2 Start / Stop Tracker
2.3 Use Tracking Information
2.4 Set Target Image
2.5 Add / Replace Target Image
2.6 Change Tracking Mode
3. Reference
3.1 API Reference
3.2 Sample
1. Overview
Start developing MAXST ARSDK Image Tracker on iOS Platform. Refer to Image Tracker Introduction for detailed information.
To specify a target for Image Tracker, You should generate 2D Mapfile of the target using Target Manager. Refer to Target Manager and Recommended Conditions for Target Images.
Once 2D Mapfile is created, proceed to 2. iOS Developmen to continue the tutorial.
Also, refer to Tracker Coordinate System to better understand 3D coordinate system of Image Tracker.
Reference |
---|
Image Tracker Introduction |
Target Manager |
Recommended Conditions for Target Images |
Tracker Coordinate System |
2. iOS Development
Start developing on xCode using Swift. Refer to Requirements & Supports to find out which devices are supported.
ARSDK has to properly integrate on iOS UIViewController. Refer to Life Cycle documents for detail.
2.1 Create Instants
var cameraDevice:MasCameraDevice = MasCameraDevice() var trackingManager:MasTrackerManager = MasTrackerManager()
2.2 Start / Stop Tracker
To start / stop the tracker after loading the map, refer to the following code.
※ To change tracker, destroyTracker() should be called before
@objc func resumeAR() { ... trackingManager.start(.TRACKER_TYPE_IMAGE) } @objc func pauseAR() { trackingManager.stopTracker() ... }
2.3 Use Tracking Information
In the folder where SDK is installed, go to ‘data > SDKSample > Original > ImageTarget' folder, and there is sample target image. Print the image.
If you use the sample code, the following content will be augmented for each image.
- Blocks.jpg: The alpha video is augmented.
- Lego.jpg: The normal video is augmented.
- Glacier.jpg: The cube with the texture is augmented.
To apply tracking results to augmented objects, refer to the following code.
※ startTracker() should be called before
func draw(in view: MTKView) { ... let trackingState:MasTrackingState = trackingManager.updateTrackingState() let result:MasTrackingResult = trackingState.getTrackingResult() let backgroundImage:MasTrackedImage = trackingState.getImage() var backgroundProjectionMatrix:matrix_float4x4 = cameraDevice.getBackgroundPlaneProjectionMatrix() let projectionMatrix:matrix_float4x4 = cameraDevice.getProjectionMatrix() if let cameraQuad = backgroundCameraQuad { cameraQuad.setProjectionMatrix(projectionMatrix: backgroundProjectionMatrix) cameraQuad.draw(commandEncoder: commandEncoder, image: backgroundImage) } let trackingCount:Int32 = result.getCount() if trackingCount > 0 { for i in stride(from: 0, to: trackingCount, by: 1) { let trackable:MasTrackable = result.getTrackable(i) let poseMatrix:matrix_float4x4 = trackable.getPose() if trackable.getName() == "Lego" { if videoCaptureController.getState() == MEDIA_STATE.PLAYING { videoCaptureController.play() videoCaptureController.update() videoPanelRenderer.setProjectionMatrix(projectionMatrix: projectionMatrix) videoPanelRenderer.setPoseMatrix(poseMatrix: poseMatrix) videoPanelRenderer.setTranslation(x: 0.0, y: 0.0, z: 0.0) videoPanelRenderer.setScale(x: 0.26, y: 0.15, z: 1.0) videoPanelRenderer.draw(commandEncoder: commandEncoder, videoTextureId: videoCaptureController.getMetalTextureId()) } } else if trackable.getName() == "Blocks" { if chromakeyVideoCaptureController.getState() == MEDIA_STATE.PLAYING { chromakeyVideoCaptureController.play() chromakeyVideoCaptureController.update() chromakeyVideoPanelRenderer.setProjectionMatrix(projectionMatrix: projectionMatrix) chromakeyVideoPanelRenderer.setPoseMatrix(poseMatrix: poseMatrix) chromakeyVideoPanelRenderer.setTranslation(x: 0.0, y: 0.0, z: 0.0) chromakeyVideoPanelRenderer.setScale(x: 0.26, y: 0.18, z: 1.0) chromakeyVideoPanelRenderer.draw(commandEncoder: commandEncoder, videoTextureId: chromakeyVideoCaptureController.getMetalTextureId()) } } else if trackable.getName() == "Glacier" { textureCube!.setProjectionMatrix(projectionMatrix: projectionMatrix) textureCube!.setPoseMatrix(poseMatrix: poseMatrix) textureCube!.setTranslation(x: 0.0, y: 0.0, z: -0.025) textureCube!.setScale(x: 0.15, y: 0.15, z: 0.05) textureCube!.draw(commandEncoder: commandEncoder) } else { colorCube!.setProjectionMatrix(projectionMatrix: projectionMatrix) colorCube!.setPoseMatrix(poseMatrix: poseMatrix) colorCube!.setTranslation(x: 0.0, y: 0.0, z: -0.075) colorCube!.setScale(x: 0.15, y: 0.15, z: 0.15) colorCube!.draw(commandEncoder: commandEncoder) } } } else { videoCaptureController.pause() chromakeyVideoCaptureController.pause() } ... }
2.4 Set Target Image
By calling function addTrackerData to register the map file and calling function loadTrackerData, the target image can be tracked. To set a target image, refer to the following code.
func startEngine() { ... let blocksTrackerMapPath:String = Bundle.main.path(forResource: "Blocks", ofType: "2dmap", inDirectory: "data/SDKSample")! let glacierTrackerMapPath:String = Bundle.main.path(forResource: "Glacier", ofType: "2dmap", inDirectory: "data/SDKSample")! let legoTrackerMapPath:String = Bundle.main.path(forResource: "Lego", ofType: "2dmap", inDirectory: "data/SDKSample")! trackingManager.start(.TRACKER_TYPE_IMAGE) trackingManager.setTrackingOption(.NORMAL_TRACKING) trackingManager.addTrackerData(blocksTrackerMapPath) trackingManager.addTrackerData(glacierTrackerMapPath) trackingManager.addTrackerData(legoTrackerMapPath) trackingManager.loadTrackerData() }
2.4.1 Train Target Image Instantly
If you want to use a raw image file as an image target without an offline training process via Target Manager, enter a JSON object as the first parameter of addTrackerData().
A sample JSON format is like below.
{ "image":"add_image", "image_path":"ImageTarget/Blocks.png", "image_width":0.26, }
The "image":"add_image" pair should be located at first. The value of "image_path" is an image path and the value of "image_width" is a real width (meter unit) of an image target.
A sample code is like below.
trackingManager.addTrackerData("{\"image\":\"add_image\",\"image_path\":\"ImageTarget/Blocks.png\",\"image_width\":0.26}");
"Image_path" inserts absolute path that has 2D map file.
The instant training permits only jpg and png formats. An image width as a pixel size should be more than 320 and the best is 640.
※ Instant training of an image takes twice as much time as loading a 2dmap.
※ You must call loadTrackerData () after calling addTrackerData ().
2.4.2 Set Target Image Package
If you are using multiple image maps, packaging them in a 2dpkg file will greatly improve map loading speed and recognition speed.
Create a 2dpkg with MapPackager, call addTrackerData() to register the package file, and call loadTrackerData() to track the target image.
func startEngin() ... let packagePath:String = Bundle.main.path(forResource: "package", ofType: "2dpkg", inDirectory: "data/SDKSample")! trackingManager.start(.TRACKER_TYPE_IMAGE) trackingManager.setTrackingOption(.NORMAL_TRACKING) trackingManager.addTrackerData(packagePath) trackingManager.loadTrackerData() ... }
※ Image Package(2dpkg) is available from AR SDK 6.1.0.
※ Only one 2dpkg can be added to Image Tracker.
2.5 Add / Replace Target Image
Create a map file refer to Target Manager
Download the file you created.
Unzip the downloaded file and copy it to the desired path.
2.6 Change Tracking Mode
5 Tracking Modes of Image Tracker:
- NORMAL_TRACKING
- EXTENDED_TRACKING
- MULTI_TRACKING
- JITTER_REDUCTION_ACTIVATION
- JITTER_REDUCTION_DEACTIVATIO
NORMAL_TRACKING: Default Setting. Track one target image.
trackingManager.setTrackingOption(.NORMAL_TRACKING)
- EXTENDED_TRACKING: Track even at far distance from the target image.
trackingManager.setTrackingOption(.EXTENDED_TRACKING)
- MULTI_TRACKING: Recognize and track up to three target images at the same time.
trackingManager.setTrackingOption(.MULTI_TRACKING)
- JITTER_REDUCTION_ACTIVATION: Jitter reduction.
trackingManager.setTrackingOption(.JITTER_REDUCTION_ACTIVATION)
- JITTER_REDUCTION_DEACTIVATION: Disable the jitter reduction option.
trackingManager.setTrackingOption(.JITTER_REDUCTION_DEACTIVATION)
3. References
These are additional references to develop Image Tracker
3.1 API Reference
Following documents explain classes and functions used to run Image Tracker.
3.2 Sample
For information regarding sample build and run of Image Tracker, refer to Sample