Onboard processing for EO satellites - FarEarth Edge

To extract reliable intelligence onboard your satellite, you need consistent and accurate input imagery. FarEarth Edge runs onboard your satellite to pre-process raw data into radiometrically calibrated, band-aligned, accurately geolocated images – within seconds. FarEarth Edge is specifically designed from scratch and highly optimised for onboard scenarios.

Why Edge?

Edge processing happens onboard the satellite and is especially useful in two scenarios:

• for real-time applications where data is processed, and intelligence extracted before the data is even downlinked

• and to minimise downlink volumes by first filtering and selecting which images are worthwhile

This means a developer creates a scientific algorithm or trains an AI model on the ground, and uploads it to the satellite’s onboard computer. The algorithm is applied, or AI inference is performed on each raw image as soon as it is captured and decoded. The resulting event information or image cutout is downlinked.

It sounds simple enough. But there are two problems people don’t often talk about which keeps these types of projects from maturing beyond the proof-of-concept phase.

• The raw input imagery onboard the satellite does not look like the training data on the ground. Raw imagery onboard is noisy, bands are unaligned, and the reported geolocation is way off.

• In the few cases where real representative data is available on-ground for training, it turns out that the algorithms only cover the ideal acquisition scenarios. AI model training becomes unwieldy and struggles to efficiently converge, given the vast range of raw data messiness.

To extract reliable intelligence onboard your satellite, you need consistent and accurate input imagery. This is where FarEarth Edge comes in.

What does FarEarth Edge do onboard?

FarEarth Edge takes raw data as input and generates orthorectified Level 1C products onboard your satellite in seconds.

You can choose between systematic or precision processing. Systematic processing is faster, while precision processing has more precise pixel geolocations. Both provide the same high-quality radiometric corrections.

For both options, FarEarth uses the boresight alignment, calibration parameters, and a sensor model calculated during calibration as input for processing.

Systematic Edge processing includes:

• image processing with calibrated physical model

• temperature-compensated radiometric correction

• top-of-atmosphere reflectance

• lens distortion correction

• noise filtering

• precision band alignment

• calibrated geolocation using a sensor model and satellite attitude and ephemeris data

Precision Edge processing includes:

• precision refinement using reference imagery

• processing with calibrated physical model

• terrain correction using a DEM

• precision pixel alignment with references

The processing output is a Level 1C image formatted as top-of-atmosphere reflectance compatible with existing Sentinel-2 workflows, with options to include:

• full resolution true or false colour image

• downsampled RGB thumbnail

• detailed STAC product file

• conditional band selection

• satellite viewing angle

• solar incidence angle

• cloud cover percentage

• cloud classification index compatible with Sentinel-2

• thematic classification layer (example; vegetation, water, bare soil, etc.)

• NDVI output

• GeoTIFF image compression

• CE95 geolocation accuracy against reference imagery

Below is an example of a Phisat-2 RGB image over Abilene before and after radiometric processing with FarEarth Edge. Notice the detector non-uniformity in the before image. FarEarth also removes read-out noise and vignetting. (Note: The image below views better on your desktop)

Below is a PhiSat-2 RGB image over Albuquerque before and after precision band alignment with FarEarth Edge. Imagine the difference in the result of an object detection algorithm using the raw versus processed image as input.

The pointing accuracy of smaller satellites is not very precise. This means that the location reported by the satellite is usually quite far off from where the image is really located. Keep in mind that an AI model using the raw input would therefore report a detected object in the wrong location. In our Albuquerque example above, you can see how the road crossing shifts after FarEarth Edge corrects the geolocation – in this case about 400 meters.

Performance

Edge processing is a trade-off between image accuracy and latency. Where this balance lies depends on your specific application. Let’s look at some typical FarEarth Edge onboard results.

Input: 20x20km PhiSat-2 scene with 8 bands and 5m GSD

• Full precision processing with orthorectification and alignment against a reference takes about 20 seconds in total

• If you need lower latency and the alignment is not as important for your application, systematic processing only, will shave the time down to only a few seconds

Be wary of quality

It is easy to create accurately aligned imagery over flat nadir scenes. FarEarth excels in scenarios where images are taken at off-nadir angles and over irregular terrains.

FarEarth verifies image quality offline against full Sentinel-2 reference images.

Be wary of accuracy reports. Some people calculate offsets only against the handful of reference chips used during their processing, which does not give a true reflection of accuracy.

Below is a PhiSat-2 image over Anaheim after precision orthorectification with FarEarth Edge. The PhiSat-2 image is overlaid on a Sentinel-2 backdrop.

Let’s zoom in on a mountainous area.

Now we measure the geometric accuracy of the Anaheim image with FarEarth’s GVerify. The dots below show the individual pixel offsets of our processed PhiSat-2 image as measured against Sentinel-2. Green and light blue dots indicate subpixel accuracy (in this case less than a 5-meters difference).

While FarEarth has its own accuracy verification tools, the details below show an independent assessment with Telespazio’s excellent Karios toolkit (https://github.com/telespazio-tim/karios).

Typical results for FarEarth Edge across various incidence angles and terrains show band alignment with a CE95 of about 5 meters, and absolute pixel geolocation with a CE95 of less than 10 meters.

If you want more detailed quality and performance results, watch for our next blog, where we will discuss our test results on different onboard hardware platforms.

FarEarth Edge deployment

What does the deployment look like in practice?

You will need:

• an ARM64 or AMD64 onboard computer with about 200GB of static storage (if precision alignment is required)

• an optional GPU for CUDA acceleration

Pre-launch, FarEarth will provide you with the following to preload onto your onboard computer:

• a 700MB container image compatible with ARM64 or AMD64

• about 150GB of static ancillaries (which includes DEM, GRI, and other references)

After launch, once FarEarth has calibrated your imager, we will provide you with the following to uplink to your satellite:

• about 100MB of ancillaries including a precise sensor model

• geometric calibration parameters

• radiometric calibration parameters

Over the lifetime of your satellite, FarEarth performs continuous calibration on-ground. It will periodically provide you with updated ancillaries to uplink to your satellite.

How does Edge fit into the bigger FarEarth platform?

FarEarth Edge integrates seamlessly with the rest of the FarEarth platform. FarEarth Cloud is designed for flexibility, precision, and integration on-ground. FarEarth Edge is built for speed onboard your satellite.

FarEarth Edge runs onboard the satellite to pre-process raw data into radiometrically corrected, orthorectified images – within seconds. These corrected images become the input for your onboard applications, vastly improving your real-time results.

FarEarth Cloud, on the other hand, reliably and consistently processes every image you downlink to the highest quality. These precision images are used in your on-ground applications. FarEarth also handles data management, archiving and cataloguing. 

We are there for you from the start of your mission until the end of your satellite’s lifetime.

Before launch, we review your lab data to validate your camera.

During commissioning, FarEarth reports on the stability of your satellite metrics, validates your downlinked data, and calibrates your camera.

For production operations, FarEarth provides you with the onboard calibration parameters and sensor models to uplink to your satellite. You are now ready for production both onboard and on-ground.

But what about training data?

The images processed, archived, and catalogued on-ground by FarEarth Cloud are your training data. Keep in mind that this is a comprehensive set of images from your actual satellite and is representative of the actual imagery FarEarth Edge will provide you onboard the satellite. Calibrated, geolocated, and orthorectified. A pleasure to train, validate and test on. Since our processed products closely aligns with publicly available datasets, existing foundational models can easily be fine-tuned and adapted to exactly match your satellite’s images.

Continuous calibration

Your satellite’s behaviour and the imager’s response will change over time. FarEarth Cloud uses the downlinked images to continuously calibrate your camera. This ensures that you always have the best quality image processing both onboard and on-ground. You can compare images over time.