Using Layer Version ARNs with AWS Lambda

Ever since Python 2 was finally sunset, I’ve been making the transition to Python 3. Kind of hard to believe that these two code bases are 20 and 14 years old, respectively! Part of this transition meant updating and refactoring AWS Lambda functions and imported modules, such as with this Roxie bot.

I hit a small stumbling point when trying to use the requests module with my Python script that rotates a Rubrik API token into HashiCorp Vault so that downstream tools can use the token for automated tasks. Here’s the GitHub repo.

Digging In Further

Here was the old code:

import botocore.vendored.requests as requests 
requests.post(all_the_things)

When executing the function, an error is returned that indicates a successful import of the module but no attribute named post. This is kind of important when trying to send over an HTTP payload to an API. ?

{
   "errorMessage": "module 'botocore.vendored.requests' has no attribute 'post'",
   "errorType": "AttributeError",
   "stackTrace": [
     "  File \"/var/task/lambda_function.py\", line 33, in lambda_handler\n    response = requests.post(all_the_things)\n"
   ]
 }

It turns out that I’ve been living under a rock. The AWS Developer Blog published information on the Botocore module back in October of 2019. Woops! Unfortunately, most of my initial Google searches stumbled upon years of blogs pushing for the usage of botocore.vendored, making it a tad hard to find the deprecation details below:

In August of last year, we made significant improvements to the internals of Botocore to allow for pluggable HTTP clients. A key part of the internal refactoring was changing the HTTP client library from the requests library to urllib3. As part of this change, we also decided to unvendor our HTTP library. This allows us to support a range of versions of urllib3 instead of requiring us to depend on a specific version. This meant that we no longer used the vendored version of requests in Botocore and we could remove this unused code.
– James Saryerwinnie

Thus, I needed to start using a Layer for the Lambda function. This means importing my own module as a layer or relying on someone else’s. You can see more on Layers towards the bottom of this Python SDK in AWS Lambda post.

Note: Ever wonder why the module is called boto? Wonder no more!

Using Lambda Layer Version ARNs

I’m a bit lazy, so I decided to use Keith’s Layers (Klayers). This is a really nifty open source project containing a collection of AWS Lambda Layers for Python 3.7 and 3.8. Everything is automated and builds occur weekly. Each region has a CSV that contains the ARNs. In my case, I wanted to pull requests version 2.23.0 or higher, which can be found here on the list.

Using the Layer is painless. In the function’s Designer panel select the “Layers” icon below the function itself. From there, choose Add a layer. Since I already have the ARN, I switched the radio button to Provide a layer version ARN and dropped in the value for requests.

Using the Lambda Layer ARN to provide additional module functionality.

From there, I commented out the old botocore import statement and tried using the standard import requests command. Make sure to save and test!

Commenting the old import and testing the new import.

I’m also subscribed to the repo to check on future changes to the Layers and can push out changes with the AWS CLI. To make this easier, I’ve tagged my functions with layer_dependency and a value related to the modules / libraries needed.