Distributed Processing Basics

What is Distributed Processing?

A distributed processing system consists of a cluster of several processing ‘Nodes' that can communicate with each other and coordinate computing tasks.

  • In SURE, Distributed Processing (DP) is based on a hierarchical structure where one ‘Master' node communicates with 'Processing Nodes' in different machines within the cluster and assigns them tasks.

  • The Master prepares the Input Images into a central storage location, referred to as Network Image Directory. All Processing Nodes can access this directory, from which they can pull Imagery data, as they are needed.

  • SURE Distributed Processing divides large projects into smaller sub-projects and distributes them to the processing machines available.

It is worth noting that one can also benefit from DP using a single machine that houses the Master Node and a Processing Node (referred to as Local-DP).

Benefits of Distributed Processing

  • Easily scale-up production by adding Processing nodes on demand.

  • Inspect initial results and perform quality assessment within 1 or 2 days.

  • Minimize progress loss in case of machine restart, power outage, or network disruptions.

  • Reduce required disk space.

SURE Distributed Processing Workflow


The Master:

  • Prepares Images in the Network Image Directory.

  • Performs the Analysis step and divides the Project into smaller sub-projects.

  • Assigns and sends the sub-projects to Processing Nodes.


The Processing Nodes:

  • Receive sub-project from Master.

  • Pull and cache Imagery data from the Network Image Directory, as they are needed.

  • Process sub-project and return results to Master.


Once all sub-projects are completed:

  • The Master merges sub-projects and performs final steps (depending on results requested).