microscope

Managing Batch Compute Environments in AWS

You successfully stood up your AWS Batch Compute Environment using CloudFormation. (HORRAY!). After sending some jobs to it you realize you need to increase the Docker data volume /dev/sdb (Assuming you’ve followed along with this great AWS blog entry on building compute nodes for genomic data.) So you add a Launch Template to your CloudFormation template. Awesome! You wire it up in the template like so:

  MyAwesomeLaunchTemplate:
    Type: "AWS::EC2::LaunchTemplate"
    Properties:
      LaunchTemplateData:
        BlockDeviceMappings:
        - DeviceName: '/dev/sdb'
          Ebs:
            VolumeSize: 1200
            VolumeType: gp2

  MyAwesomeBatchComputeEnv:
    Type: AWS::Batch::ComputeEnvironment
    Properties:
      Type: MANAGED
      ServiceRole: !Ref YourAwesomeServiceRoleArn
      ComputeResources:
        LaunchTemplate:
          LaunchTemplateId: !Ref MyAwesomeLaunchTemplate

You update the stack with the new Launch Template and everything should go as planned right? Nope! The volume size has not increased. Why? Because Launch Templates are versioned resources and it will default to 1 if a version is not specified. Meaning you need to specify which version to use. How do we do that? This is where the Fn::GetAtt comes in handy!

Modify the Compute Environment resource to grab the latest version of the Launch Template like so:

  MyAwesomeBatchComputeEnv:
    Type: AWS::Batch::ComputeEnvironment
    Properties:
      Type: MANAGED
      ServiceRole: !Ref YourAwesomeServiceRoleArn
      ComputeResources:
        LaunchTemplate:
          LaunchTemplateId: !Ref MyAwesomeLaunchTemplate
          Version: !GetAtt MyAwesomeLaunchTemplate.LatestVersionNumber

Update the stack again and you will now see the change reflected on any new instances launched in your compute environment.

This is something that we have ran into in the past and we hope this helps you out!

Want to get the latest analysis and open source tools we publish?

It's so easy for experts to put their head down and work without ever sharing lessons learned with the rest of the world. We publish all our best ideas, analysis, and latest open source tools and techniques by email every week.

    We won't send you spam. Unsubscribe at any time.

    Powered By ConvertKit