FARO: Under the Digital Hood

Hybrid Reality Capture, Powered by Flash Technology and the FARO Focus Core, Premium and Premium Max Laser Scanners.

With the global construction market expected to grow at over 10% annually, new technology will be essential to scan and categorise job site progress accurately and efficiently.

Original Source - FARO Technologies

As outlined in Part I of this three-part series, FARO's patented Hybrid Reality Capture, Powered by Flash Technology. It’s a technology that merges 360° panoramic images with a higher-speed scan to create full-colourised scans in less than 30 seconds with virtually no loss in data accuracy; a transformative tool in this ongoing endeavor. Part II of this three-part series will discuss some of the software that underpins Flash Technology and how it works when used with a FARO Focus Core, Premium and Premium Max Laser Scanners.

It all comes down to proprietary smart upscaling algorithms. The output includes all collected images and points captured by a Ricoh Theta Z1 360° camera and a Focus Core, Premium and Premium Max Laser Scanner, resulting in full-colour scans that look crisper than the same resolution scans with traditional methods.

Released to the market in April 2023, Hybrid Reality Capture, powered by Flash Technology functions as a scan mode for the Focus Scanners accessible through the FARO Stream mobile app.

Also important to note is that Flash is offered for a one-year subscription with Focus Core and Premium users while the Premium Max comes with a perpetual license.

With FARO Stream, freely available on iOS and Android platforms, users can benefit from:

• Control all FARO devices, Focus laser scanners, Orbis mobile scanners and future scanners.
• Pre-register captured scans directly in the field to gain confidence on data completeness and reduce off-site registration failures.
• Collect complementary data as annotations marked in the scan with image or file attachments.
• Upload captured data directly into Sphere XG for immediate project sharing with stakeholders.

With its data accessed through FARO Sphere XG, an SaaS cloud-based information platform, the Focus Scanner Flash scans are ideal for companies and agencies taking large-volume 3D reality capture scans of buildings, infrastructure, corridors, mines and forests as part of their existing conditions, progress monitoring and pre-incident planning needs.

How Does Flash Technology Work? (Upscaling vs. Extrapolation)

At the heart of Flash Technology for the Focus line of products is a process called image upscaling or interpolation. Upscaling, as defined by Upscale.media, begins with a low-resolution image and improves its visual quality at higher levels.

It does so through an algorithm, which interpolates between two frames. Because it does not have access to the complete data set, (meaning every possible data point) the best the software can do is make an estimation based on a mathematical formula. It’s important to note, that this lack of data is not a failure of any kind. Rather, it is a commonplace variable faced in science and engineering where an intermediate value is not known but can be probabilistically determined with a high likelihood of accuracy based on other known data points.

Historically, software engineers might employ interpolation or upscaling as a way to build tools that allow users to enlarge images, which adds pixels. To create a true-to-value enlarged image, resising tools use the data from the original image to interpolate new values, allowing them to add pixels between the original data points. Software engineers might also use interpolation in video games, coding for character movements.

Hybrid Reality Capture works along these same principles, too. The lower-resolution 3D scan is faster to capture resulting in less data. But those gaps in data are augmented with the data captured from 360° panoramic images. This is where image upscaling comes into play. With Flash Technology, if there is a picture point from the camera, where it does not have the 3D point for it and are searching for the neighbors of that point, the software determines if there is a 3D coordinate for it. And then it smartly interpolates between the neighboring points to create a new 3D point for a colourised pixel. If the viewing angle is too flat, then Flash Technology does not create an interpolated point in between.

In the second step, the technology checks if the angle of incidence and the parallax that the two cameras have support that point. If they support it, then the software keeps it and continues generating the image.

In the Know:

• Angle of incidence - A ray of light hits a surface at a point. From that point the line straight up, at 90° to the surface, is called the normal. The angle between the normal and the ray of light is called the angle of incidence. You measure the angle from the normal, which is 0 degrees, to the ray of light.
• Parallax - The apparent displacement or the difference in apparent direction of an object as seen from two different points not on a straight line with the object. The easiest way to visualise this phenomenon is to look at an object and alternate closing your eyes. The image you are looking at will appear to change positions relative to which eye you are looking out of. The object’s position didn’t change; your perception of its location did.

A critical part of what makes Flash Technology for Focus Core, Premium and Premium Max so intuitive and easy to use is that all this formula plugging, all the measurements related to angle of incidence and parallax, which data points to accept and reject, and a breakdown of what is actual data versus what is upscaled data is completed under the hood. From a user’s perspective, all they will see is the fully colourised hybrid panoramic image/3D laser scan composite.

Additional behind-the-scenes benefits include:

• Optimised scan operation - Is a scan functionality that when a user hits the ‘start’ button, the device coordinates what the scanner does and in what sequence to maximise speed efficiency; when the scanner starts rotating, when are the pictures captured, etc. So, for example, the panoramic image is always taken first, then the scan.
• Optimised streaming to mobile devices - Optimised transfer and use of point clouds on mobile devices to make in-field registrations as fast as possible to ensure efficient use of time out in the field.
• Coarse inclinometer reading - Is a way to measure the levelness of a scan. Traditionally this step takes ~10 seconds. But with Flash Technology all it takes is 1 second to obtain the necessary data. While a 1-second inclinometer reading is less accurate than a 10-second reading, the data is accurate enough for 99% of existing and predicted new customer use cases.

What must be emphasised here is that what’s being upscaled is the 3D point cloud, not the image itself. Flash Technology takes the above data and upscales it to 4,000 steps horizontally after it’s processed. This might sound counterintuitive, but it’s where Flash Technology leads in market development. Throughout the entire capturing process, the image from the PanoCam stays at 23 million points per photograph and what is being upscaled is the point cloud in the background.

So, for the user there is no visual impact at all because the picture that they see stays the same. Prior to the upscaling the low-resolution scan would only generate a 3D coordinate every fourth pixel of the photograph. After upscaling a 3D coordinate exists for nearly every pixel of the photograph.

Why is Flash Technology Different and what are its Ideal Use Cases?

For customers who have been in the 3D measurement and laser scanning business for some time, even a cursory review of the Focus brochures with its Flash Technology enhancements should make clear the products’ distinguishing factors. But for less experienced customers or potential new buyers, there could be lingering ambiguity — especially when other laser scanning companies are also augmenting photographic images with 3D scans.

The differences here are several fold: chief among them is that Flash Technology functions as an independent scan mode for the scanner which can be turned on and off as needs dictate.

Another distinguishing factor is colour. Historically, colourised scans were a costly and time-consuming endeavor, especially for large-volume projects requiring hundreds of scans. For a time, this barrier to product adoption was so high that some customers discarded colourised scanning altogether, opting instead to manage without while executing still-satisfactory deliverables. With Flash Technology, because colour scanning takes only one second to capture it is likely Hybrid Reality Capture will re-ignite interest in this ability and make colour scanning an economically and temporally wise decision, arguably for the first time since colourised scanning was developed.

Flash Technology also has the distinction of helping shift the market toward a more visual experience overall as other companies introduce high-resolution photography, videos and 360° image capture of their own.

Bottom line: Hybrid Reality Capture is an entirely different category of image capture, an innovative way to think about the relationship between a panoramic image and a 3D point cloud. In a sense, the process has been reversed. No one else begins with the photographic image, and then upscales the 3D point cloud to augment the data. This is what was essential in unlocking the unprecedented speed of capture.