Use AWS SAM to Create AWS Lambda Layer for Python (Serverless Application Model)

Question

I'm trying to use AWS SAM / Serverless application Model to deploy a combination of Python and native libraries to Lambda as a layer, for use from a Python Lambda function. I'd appreciate some help getting the build and deploy working.

My initial POC was to manually create a test lambda function and package up the native Oracle Instant Client library inside the same lambda zip. This worked fine, so now I'm deploying it using Infrastructure as Code as a layer so multiple functions can reuse the client in the layer.

Here's my SAM template

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31

Globals:
  Function:
    Runtime: python3.8

Resources:
  OracleInstantClientLayer:
    Type: AWS::Serverless::LayerVersion
    Properties:
      ContentUri: LayerResources
      CompatibleRuntimes:
        - python3.8
    Metadata:
      #BuildMethod: makefile
      BuildMethod: python3.8

In the "LayerResources" subdirectory I have

• Makefile
• requirements.txt (which has a single line including cx_Oracle)
• abc.so (bunch of library files compiled for x86 Linux)

When my template specifies "BuildMethod python3.8" the Python libraries are downloaded and put into the ".aws.sam\build\OracleInstantClientLayer\python" folder as expected, but the native library abc.so is not. I have tried "sam build" and "sam build --use-container" with the same results.

I've also tried "BuildMethod makefile" with this makefile. I'm trying to get all the files into the "BuildTest" folder, which probably isn't right but it would be a step forward.

build-OracleInstantClientLayer:
    python -m pip install -r requirements.txt -t "BuildTest"
    cp LayerResources\* BuildTest\

When I "sam build" here's the error

sam build
Building layer 'OracleInstantClientLayer'
Running CustomMakeBuilder:CopySource
Running CustomMakeBuilder:MakeBuild
Current Artifacts Directory : C:\ProjectFolder\.aws-sam\build\OracleInstantClientLayer
python -m pip install -r requirements.txt -t "BuildTest"
Collecting cx_Oracle
  Using cached cx_Oracle-8.3.0-cp38-cp38-win_amd64.whl (219 kB)
Installing collected packages: cx_Oracle
Successfully installed cx_Oracle-8.3.0
cp LayerResources\* BuildTest\

Build Failed
Error: CustomMakeBuilder:MakeBuild - Make Failed: process_begin: CreateProcess(NULL, cp LayerResources* BuildTest, ...) failed.
make (e=2): The system cannot find the file specified.
make: *** [C:\ProjectFolder\LayerResources\Makefile:3: build-OracleInstantClientLayer] Error 2

Can anyone help get this working? I'd prefer to use AWS SAM but I'm also open to creating the layer zip file manually and uploading it using SAM, which is a fallback I haven't tried yet.

Answer 1

My conclusion is AWS SAM is not made to create the layer zip file. Here's the commands I used. The layer files are put into a LayerBuild folder. In the case of Python the need to be in LayerBuild\python, as when it's uploaded to the lambda server it's unzipped to opt (docs here).

7za a -tzip LayerBuild\LambdaLayer.zip .\LayerBuild\*

aws s3 cp --profile %1 LayerBuild\LambdaLayer.zip s3://bucketname/foldername/LambdaLayer.zip

aws lambda publish-layer-version --profile AwsProfileName --layer-name LayerName --description "xyz" --compatible-runtimes python3.9 --content S3Bucket=bucketname,S3Key=lambda/LambdaLayer.zip

When you create your lambda function be sure to set the LD Library Path

Function:
    Type: AWS::Serverless::Function
    Properties:
      Description: Lambda function
      FunctionName: Function
      CodeUri: python-code/
      Role: RoleName
      Handler: LambdaFileName.lambda_handler
      VpcConfig:
        SecurityGroupIds:
          - SG-ID
        SubnetIds:
         - Subnet1-ID
         - Subnet1-ID
      Environment:
        Variables:
          LD_LIBRARY_PATH: '/var/lang/lib:/lib64:/usr/lib64:/var/runtime:/var/runtime/lib:/var/task:/var/task/lib:/opt/lib:/opt/python'
      Layers:
       - Layer URI