Scaling Production with Distributed Processing

See Also

Distributed Processing Basics

Distributed Processing FAQ & Troubleshooting


Before You Start

Checklist

  • All machines to be used in Distributed Processing follow our Hardware Recommendations

  • All machines are in the same network

  • SURE is installed in all machines and a license is activated in each one. Visit Quick Start for details

Once you have confirmed all the prerequisites above are met follow these 2 simple steps:

Setting Up a DP Project

1. Start all Nodes

Open a terminal (Powershell or Command Prompt) and start the SURE-node executable. Use the workspace flag for specifying the working directory:

SURE-Node.exe --workspace path\to\desired\node\directory

Ensure that the selected working directory lies within an SSD RAID with at least 4TB of available storage (See Hardware Recommendations).

The nodes can also be started by simply double-clicking the SURE-Node.exe file in the Windows Explorer. In this case, a working directory will be automatically created in the Default directory, as specified in the Project Wizard window of the SURE GUI.

A Windows pop-up message should appear asking to allow SURE-Node.exe through the Firewall. Click on 'Allow Access’.

If it is the first time you use SURE-Node.exe and this window does not appear please follow the steps described here.

2. Activate DP mode in SURE and start project

GUI:

  1. Start a project on the Master computer (see Quick Start)

  2. Once the Project Wizard window closes check the ‘Activate Distributed Processing’ box in the bottom right corner of SURE’s main project window

CLI: Include the -d or --distributed-processing flag in your script.


Monitoring a DP Project

Once the Analysis step is shown as 'Finished' in the Project Status tab, the Master will start to distribute sub-projects to all processing nodes. At this stage you can monitor the progress of your Distributed Processing project as follows:

  • The Project Status tab will display the Sub-projects that were created. Each sub-project will have a status bar specifying whether it is pending, running, or finished.

  • The Cluster Manager should automatically open in a new window of your default browser. Here you can obtain information on the cluster such as the nodes that are currently connected and the status of the task each node is carrying out. If the window does not open or you accidentally close it, navigate to "localhost:5006" via the address bar of the browser.


Results

Results from each sub-project are available as soon as the sub-project status is set to ‘Finished’ in the Project Status tab. These results are available in: Project_Folder\Results.

Once all sub-projects are finished, the Master node will generate the additional LOD levels in the Mesh results. Once completed, all results are available in the usual Project_Folder\Results folder


Learn More

Distributed Processing Basics

Distributed Processing FAQ & Troubleshooting