Table of Contents

Editor: Camera Payloads

Description

Within Nominal Editor, there are three key camera payload types. These are the Optical, Infrared, and Event cameras. The technical manual for the cameras, found in the document Payload: Camera, provides a greater explanation of the technical aspects and use cases for each camera type. This guide will focus on the shared camera utilities and how to deploy and configure each camera payload type.

Shared Camera Function Libraries

Regardless of the camera payload type, the following functions exist in the Camera Math Library, to calculate aspects of the simulated cameras:

  • Get Aperture FStops: Calculates the Aperture of a Camera in the F-Stop unit for a given Focal Length and Aperture Diameter.
  • Get DOF: Calculates and returns the Near, Far and Depth of Field (DOF) Distances for a given Focal Length, Circle of Confusion, Focal Distance and Hyperfocal Distance. The DOF planes represent the distance ranges where objects appear acceptably sharp.
  • Get FOV: Calculates the diagonal Field of View (FOV) for a given Pixel Pitch, Focal Length and Resolution
  • Get FOVXY: Calculates the Horizontal and Vertical Field of View (FOV) for a given Pixel Pitch, Focal Length and Resolution
  • Get Hyper Focal Distance: Calculates the Hyper Focal Distance for a given Focal Length, Aperture F-stop and Circle of Confusion and represents the distance that gives the greatest Depth of Field. It is the distance between a camera and the closest object that appears in focus when the lens is set to infinity.

Additionally, the Camera Image Library contains the following utility functions:

  • Create Image Message from File: Based on an absolute path to a supported image file (.png, .jpg, .exr) and a configured width and height of the image, this function generates a Nominal Image Message.
  • Create Image Texture: Based on the configured Width and Height a 2D Unreal Texture will be created that supports the configured Image Format.
  • Save Image Byte Data: Based on a specified file name this function will attempt to save Byte array data of an image to disk based on the specified format. The File Name can also include pathing information, that can be absolute or relative to the Saved Directory.
  • Save Image Struct Data: This function saves the Payload of the Camera Image Message Type to disk.
  • Update Texture from Byte Data: using a referenced Unreal 2D Texture, this function will update the texture to display new image data of the specified image format.

Common Camera Parameters

Each camera payload type shares the following parameters:

  • Field of View (FOV): The angle in degrees that represents the arc that the camera can see.
  • Resolution: The number of pixels in the sensor of a camera in the vertical and horizontal axes.
  • Pixel Pitch: The distance in millimetres from the centre of a pixel and the centre of its neighbouring pixels.
  • Aperture: The diameter of the opening of the lens in millimetres. A larger diameter/opening means a wider FOV.
  • Focal Length: The Distance from the nodal point (where light converges) of the lens to the sensor in millimetres. A longer distance reduces the FOV.
  • Circle Of Confusion: Defines the acceptable level of blur in millimetres, as measured on the sensor.
  • Focusing Distance: Defines the distance from the lens to where objects are in focus in meters.

Deploying an Optical Camera

An example of the optical camera payload in use can be found in the Data/Demo_ImageDownlink demo. In this example, an image captured by the Operational Land Imager (OLI) of the Landsat 8 Digital Twin is transmitted to the ground station and displayed to the user. To achieve this, the demo uses the Camera Image Out Message of the OLI to transmit the image to the ground station.

Deploying an Infrared Camera

An example of the Infrared Camera Payload in use can be found in the Payloads/Demo_CameraPayloads and Payloads/Demo_AircraftObservation demos in the payloads folder. The aircraft observation demo includes an Infrared Camera paired with an Optical Camera, whilst the satellite is pointing at a moving aircraft. The camera payloads example demo shows a spacecraft pointing its Optical Camera Payload at the earth but features an Infrared Camera observing the spacecraft. As the various spacecraft components have been configured for a thermal simulation, the thermal data is visualized in the camera preview window.

The visual colours used by the Infrared camera can be adjusted via the Set Material Function, with Nominal providing the following by default: Black to White, Blue to Red Hues and finally Blue to Red Lerp.

Untitled

Note

As the Infrared Camera uses the Heatmap system, it can be repurposed to visualize other Physical Component properties. For example, replacing Temperature in the heatmap property string with Mass would visualize the mass of each component.

Deploying an Event Camera

Deploying the Event Camera in a Nominal Simulation is the same as the Infrared and Optical Cameras. The Payloads/Demo_CameraPayloads demo spacecraft comes with an Event Camera attached. The Generate Event Data, function is used to generate Event Camera Data. The function will need to be manually called per unreal tick to constantly detect events, with an array of data generated per call. The array contains entries in the following format:

<Timestamp> <Horizontal Pixel> <Vertical Pixel> <BrightnessChange>

The timestamp to use is a string that is passed into the simulation, with the below example showing how the current simulation seconds can be used. The brightness change will be a 0 or 1 to reflect if the brightness value for the specific pixel decreases or increased in brightness. The other parameter of the Generate Event Data function, Polarity Threshold, refers to the required intensity delta that a pixel needs to exceed between calls of the Generate Event Data function.

Untitled