In the previous section, we created a HPC cluster using AWS ParallelCluster. This is ideal for when you want to run jobs across multiple-nodes in a batch like fashion. However a typical CFD requirement is to use pre-processing and post-processing software to interactively create meshes and visualize the solution. This has traditionally been why CFD engineers have had powerful workstations with large RAM and a GPU to do this work prior to submiting to a HPC cluster. We can do something similar on AWS and create a high RAM, GPU-enabled machine which can do your pre-processing and post-processing, whilst having access to the same file system that your HPC cluster is on. Using NICE DCV, you can connect seamlessly to this virtual workstation and have a completely cloud-native solution.
The first step is to launch the EC2 console (search for EC2 in the top search bar):
Next, click ‘Launch Instance’
At this point you can decide on the Amazon Machine Image (AMI) that your instance will be based on. This will give you the operating system and certain pre-install packages. We want to use NICE DCV (read more here: ) so lets use their AMI (which has no software charges on AWS - only the price of the instance itself) so we have it pre-installed.
To do this we search for ‘NICE DCV’, and head to the ‘Marketplace’ Tab.
You can see a range of versions for Windows/Linux and various operating systems. For this workshop we’ll use a Linux version (Amazon Linux 2) and because we want to use a GPU we won’t pick the ARM option (even if your cluster was configured to use ARM in Section 3). At present there is no Amazon EC2 Graviton2 (Arm-based) GPU instance, however you could follow the same instructions and pick a non-GPU ARM instance. Please note however that the specific instructions that follows are for a GPU.
Next we can select the instance type. We’re going to select the G4dn.4xlarge (16vCPUs and 64GB RAM) but you could select a larger or smaller one depending on your needs. Make sure you’ve remembered to request for a larger service quota in section II or you may not be able to launch one of these GPUs. To proceed, click ‘Next: Configure Instance Details’:
You can read more about various GPU instances here:
The next step is to pick the Network and Subnet, you can read more about them here, but if you followed the steps in Section 3, we can just pick the network and public subnet (so we can SSH to it from anywhere) that were automatically created by ParallelCluster.
Please note that you are building a dependancy on the ParallelCluster and EC2 instance by using this approach i.e if you try to delete the cluster, it won’t actually delete the VPC and Subnet you created (although it will delete the Master node for the cluster). For production you may wish to therefore create a VPC and Subnet outside of ParallelCluster so they are not so tightly linked on each others creation and deletion.
Another key point is the selection of an IAM role that provides permission to the GPU instance to your S3 bucket. We did this for ParallelCluster using the
s3_read_write_resource = * command.
For this EC2 instance we will attach the role we made in Section 2 so that we have access to S3.
The next section is to select the storage needs. You can increase the root volume from the default 8GB to 50GB just to give you space but we will use the FSx for Lustre volume you created in the previous section so we don’t need much storage on the node itself.
The next step is to configure the security groups, which is essentially limiting the access to your instance. By default with the NICE DCV AMI the port needed for DCV is opened (8443) but we should also create a new one and select SSH, so that we can SSH into the instance. You can pick 0.0.0.0/0 which means open to everyone, but for production you want to limit this.
After optionally creating tags (so you can track usage), you can launch the instance and select the same SSH key that you used for the ParallelCluster in Section 3.
You can now move onto the next section.