Scaling Production with Distributed Processing
This page describes the steps required to set up and monitor a Distributed Processing project. To learn how Distributed Processing works in SURE please visit our Distributed Processing Basics page. For common questions on Distributed Processing and troubleshooting visit our Distributed Processing FAQ & Troubleshooting page.
Before You Start
All machines to be used in Distributed Processing follow our Hardware Recommendations
All machines are in the same network
Once you have confirmed all the prerequisites above are met follow these 2 simple steps:
Setting Up a DP Project
1. Start all Nodes
Open a terminal (Powershell or Command Prompt) and start the SURE-node executable. Use the workspace flag for specifying the working directory:
Ensure that the selected working directory lies within an SSD RAID with at least 4TB of available storage (See Hardware Recommendations).
The nodes can also be started by simply double-clicking the SURE-Node.exe file in the Windows Explorer. In this case, a working directory will be automatically created in the Default directory, as specified in the Project Wizard window of the SURE GUI.
A Windows pop-up message should appear asking to allow SURE-Node.exe through the Firewall. Click on 'Allow Access’.
If it is the first time you use SURE-Node.exe and this window does not appear please follow the steps described here.
2. Activate DP mode in SURE and start project
CLI: Include the
Monitoring a DP Project
Once the Analysis step is shown as 'Finished' in the Project Status tab, the Master will start to distribute sub-projects to all processing nodes. At this stage you can monitor the progress of your Distributed Processing project as follows:
The Project Status tab will display the Sub-projects that were created. Each sub-project will have a status bar specifying whether it is pending, running, or finished.
The Cluster Manager should automatically open in a new window of your default browser. Here you can obtain information on the cluster such as the nodes that are currently connected and the status of the task each node is carrying out. If the window does not open or you accidentally close it, navigate to "localhost:5006" via the address bar of the browser.
Results from each sub-project are available as soon as the sub-project status is set to ‘Finished’ in the Project Status tab. These results are available in: Project_Folder\Results.
Once all sub-projects are finished, the Master node will generate the additional LOD levels in the Mesh results. Once completed, all results are available in the usual Project_Folder\Results folder