Oteemo is committed to being at the forefront of emerging technology and providing advanced solutions to our customers in the technology space. One fascinating area of emergence is applying computer vision beyond visible light.

Computers are learning to see, but unlike human eyes, they can go beyond the visual spectrum. To help understand where we’re heading, it’s important to know where we are. In this article, we go over the basics of computer vision and discover how hyperspectral imaging is changing national security.

What is Computer Vision?

Unlike humans, computers do not see the world in terms of colors or objects. A picture displayed on a computer screen is really just a set of pixels with values from 0 to 255. A black-and-white image (figure 2) will have a single color channel and true color images will have 3 channels, one for each of the three primary colors: Red, Blue, and Green (see Figure 1). These values tell the computer screen how much intensity of each color light to show on the display and thus combine to produce the full range of colors a human can see.

Credit: Stanford AI Lab

Converting Image to Pixel bitmap, notice the shoe?

How Computer Vision is Applied in the Real World

Recent advances in computer vision algorithms such as Convolutional Neural Networks utilize these pixel maps to train models to find specific objects within images, label them accurately, and even understand the context of a picture and describe it in natural language. Such systems allow for common use cases like facial recognition in Facebook photos to security detection when people walk into an unauthorized area. The progressing ability of computer vision to understand images in the context of the real world and make predictions about what will happen next is what allows applications like UAVs to avoid objects while flying or for Tesla autopilot to safely drive autonomously. In military applications, it even allows for the identification of lawful targets such as rocket launch locations, even in densely populated areas, and allows for precision targeting as seen during the May 2021 Operation Guardian of the Walls in Israel.

What is Hyperspectral Imaging?

The human eye is limited to emissions only within the visible light spectrum however the electromagnetic spectrum includes everything from radio to infrared to ultraviolet to X-rays and even gamma rays. Objects in the universe emit and reflect radiation across the entire spectrum but we can only perceive a very small portion of it. When we look at a color image, no matter how amazing your 4K television is, it is fundamentally limited to three channels: Red, Green, and Blue. A hyperspectral image (HSI) takes hundreds or even thousands of channels across the EM spectrum including infrared, UV, radio, etc to produce a 3D image with an order of magnitude more information. By detecting the emission and absorption of specific wavelengths of light, you can begin to decipher the specific chemical composition and characteristics of an object without ever having to physically interact with it. This science, known as spectroscopy, is how we know the sun is made up of mostly hydrogen and helium despite never having sent a probe directly into the sun. We are able to see what light, across the EM spectrum, is absorbed and thus can calculate with extreme precision, the chemistry of distant astronomical objects based on our knowledge of each element’s spectral emission pattern. These spectral patterns act as a chemical fingerprint both for individual atoms like hydrogen and helium and also for more complex molecules. When NASA scientists announce using the James Webb Space Telescope (JWST) that they have discovered a planet 40 lightyears away with water in its atmosphere, this is how they are doing it: spectroscopy across a plethora of wavelengths to detect the telltale chemical fingerprints of life-sustaining compounds.

The most significant hurdle in hyperspectral imaging is the overwhelming volume of data that needs to be processed to generate an image. Traditionally, techniques such as Principal Component Analysis (PCA) have been used to condense the data from hundreds or even thousands of dimensions to the three primary color channels viewable to the human eye. 

Researchers often needed to manually classify which spectral emissions corresponded to specific categories, such as “water,” “built-up areas,” or “vegetation,” and subsequently compress the image data. However, this approach is time-consuming and doesn’t scale well, particularly when dealing with extensive datasets. Moreover, it assumes that a predefined classification exists for every spectral emission present in the image, which can limit the capacity for discovering new elements or handling previously unseen data. This is where modern deep learning techniques, such as Convolutional Neural Networks (CNNs), offer a substantial advantage. When trained with labeled data, these algorithms can autonomously learn the representation of data, offering a more scalable and flexible approach. 

Deep learning can also be paired with unsupervised learning techniques, like clustering, to differentiate between known and unknown compounds. The learned features from these methods can then be visualized in a false-color image. This kind of image employs unnatural colors to represent the intensity of certain spectral features, enabling a more detailed view or highlighting specific components. As a result, the application of these modern techniques to hyperspectral imaging can yield far richer insights than traditional methods.

Hyperspectral Vision and National Security

CNNs trained on HSI data can be taught to look for the specific spectral fingerprints of different objects to identify dangers such as explosives, chemical weapons, and rocket fuel. This would allow an orbiting ISR satellite to go beyond just detecting the infrared flash of a ballistic missile launch to analyzing the specific chemical signature of the rocket fuel and begin to make predictions about its performance based on burn rate patterns. HSI can also be used to explicitly detect and differentiate different types of explosives from C4 to gunpowder. By combining AI-powered HSI analysis with traditional X-ray scanners, the TSA can more accurately identify not only which bags have explosives but exactly what kind too. These HSI techniques can also be applied on UAV and satellite platforms to detect buried explosives and identify IEDs (Improvised Explosive Devices) well before the convoy ever leaves the base.

HSI imaging has a distinct advantage for national security applications in that it is passive as compared to radar sensing modalities like Synthetic Aperture Radar (SAR). HSI sensing simply collects what is naturally being reflected back across the EM spectrum without emitting any of its own signals and thus aiding in the reconnaissance of denied areas where stealth is paramount. HSI and SAR can be combined to yield a very powerful suite of capabilities. SAR can be used to map geographic features and penetrate into forests and caves while HSI can detect any specific chemical fingerprints. This would be particularly useful to penetrate deep into the canopies of Central American jungles to find cocaine production or even underground chemical weapons facilities.

Harness the Power of Hyperspectral Imaging

Applying computer vision models to hyperspectral imaging can yield a unique edge to the warfighter to identify buried or invisible threats, such as mapping minefields or buried rocket launchers, without announcing itself to the adversary. Click here to learn more about how Oteemo can help you harness the power of computer vision for national security.