Using Jenkins to Trigger AWS Lambda Functions

AWS Lambda is a serverless compute service that allows you to run code in response to events without provisioning or managing servers. Integrating Jenkins with AWS Lambda can help automate workflows, such as deploying code, triggering serverless applications, or running scheduled tasks. In this post, we will explore how to configure Jenkins to trigger AWS Lambda functions, enabling you to build automated workflows that leverage serverless computing.

1. Introduction to AWS Lambda

AWS Lambda enables you to run your code in response to various events, such as changes in data, HTTP requests, or scheduled tasks, without worrying about server management. It is highly scalable, allowing you to handle varying workloads efficiently. With Jenkins, you can automate tasks such as deploying applications, running tests, or triggering workflows that require Lambda functions.

By integrating Jenkins with AWS Lambda, you can build a fully automated pipeline that responds to events in your development and production environments.

2. Prerequisites

Before you begin, ensure you have the following:

1. AWS Account: Access to an AWS account to create and manage Lambda functions and IAM roles.
2. Jenkins Installed: A running instance of Jenkins, either locally or on a server.
3. AWS CLI Installed: Optional but recommended for managing AWS resources from your Jenkins server.
4. IAM Permissions: Ensure you have permission to create Lambda functions and IAM roles.

3. Creating an AWS Lambda Function

1. Log in to the AWS Management Console and navigate to the Lambda service.
2. Click Create Function and choose the option to Author from scratch.
3. Configure the following settings:
   • Function Name: Give your function a descriptive name (e.g., MyLambdaFunction).
   • Runtime: Choose the programming language (e.g., Python, Node.js, Java) for your function.
   • Execution Role: Select an existing role or create a new role with basic Lambda permissions.
4. Click Create Function. You will be redirected to the function's configuration page.
5. Add Code: In the function code section, you can either write your code directly in the console or upload a zip file containing your code and dependencies.
6. Test Your Function: Use the Test feature in the AWS Lambda console to verify that your function works as expected.

4. Setting Up IAM Permissions for Jenkins

To allow Jenkins to invoke AWS Lambda functions, you need to create an IAM user or role with the appropriate permissions.

1. Create an IAM User:
   • In the AWS Management Console, navigate to the IAM service.
   • Click on Users > Add user.
   • Enter a username (e.g., JenkinsUser) and select Programmatic access.
   • On the permissions page, click on Attach existing policies directly.
   • Search for and select the AWSLambda_FullAccess policy or create a custom policy with permission to invoke Lambda functions. The following JSON policy allows invoking a specific Lambda function:
2. Save Access Keys: After creating the user, save the Access Key ID and Secret Access Key, as you will need them to configure Jenkins.

Attach Policies:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "lambda:InvokeFunction",
            "Resource": "arn:aws:lambda:us-west-2:123456789012:function:MyLambdaFunction"
        }
    ]
}

5. Configuring Jenkins for AWS Lambda Integration

5.1 Installing Required Plugins

To integrate Jenkins with AWS services, you may need to install the following plugins:

1. AWS Steps: This plugin allows you to use AWS-related commands within Jenkins Pipeline scripts.
2. AWS Credentials: This plugin allows you to securely manage AWS credentials in Jenkins.

To install the plugins:

1. In Jenkins, navigate to Manage Jenkins > Manage Plugins.
2. Under the Available tab, search for the required plugins and install them.

5.2 Configuring AWS Credentials in Jenkins

1. In Jenkins, navigate to Manage Jenkins > Manage Credentials.
2. Select the appropriate domain (or global) and click Add Credentials.
3. Choose AWS Credentials and enter your Access Key ID and Secret Access Key. Give it an ID (e.g., aws-credentials).
4. Click OK to save the credentials.

6. Creating a Jenkins Job to Trigger a Lambda Function

6.1 Configuring Source Code Management

1. Create a new Freestyle Project in Jenkins.
2. In the Source Code Management section, configure your repository (e.g., Git) from which you will pull the code.

6.2 Adding Build Steps to Trigger Lambda

1. Under the Build section of the Jenkins job configuration, click on Add build step and select Invoke AWS Lambda Function.
2. Fill in the following details:
   • Lambda Function Name: Enter the name of your Lambda function (e.g., MyLambdaFunction).
   • AWS Credentials: Select the credentials you configured earlier (e.g., aws-credentials).
3. (Optional) Add any parameters you want to pass to the Lambda function in the Payload section.

6.3 Testing the Job

1. Save the job configuration and click Build Now.
2. Monitor the build output to ensure that the Lambda function was triggered successfully.
3. Check the AWS Lambda console for execution logs to verify that the function executed as expected.

7. Using Jenkins Pipeline to Trigger AWS Lambda

You can also use a Jenkins Pipeline (Declarative or Scripted) to trigger AWS Lambda functions. Here’s how to do it using a Jenkinsfile:

1. Create a Pipeline Job in Jenkins.
2. Define the Jenkinsfile with the following steps:

Example Jenkinsfile:

pipeline {
    agent any
    environment {
        AWS_ACCESS_KEY_ID = credentials('aws-access-key-id')
        AWS_SECRET_ACCESS_KEY = credentials('aws-secret-access-key')
    }
    stages {
        stage('Trigger Lambda') {
            steps {
                script {
                    def response = sh(script: 'aws lambda invoke --function-name MyLambdaFunction --payload \'{"key":"value"}\' response.json', returnStdout: true)
                    echo "Lambda response: ${response}"
                }
            }
        }
    }
}

In this pipeline:

• AWS credentials are retrieved securely from Jenkins credentials.
• The AWS CLI is used to invoke the specified Lambda function, and the response is stored in a file named response.json.

8. Best Practices for Triggering AWS Lambda Functions with Jenkins

1. Use Environment Variables for Credentials: Store AWS credentials securely in Jenkins and access them via environment variables in your Jenkins jobs or pipelines to avoid hardcoding sensitive information.
2. Monitor Lambda Executions: Use AWS CloudWatch to set up logging and monitoring for your Lambda functions. Set up alarms for errors or performance issues to ensure your integrations run smoothly.
3. Implement Error Handling: Consider adding error handling logic in your Lambda function to gracefully manage exceptions and failures.
4. Test Lambda Functions Independently: Before integrating Lambda with Jenkins, test your functions independently to ensure they work as expected.
5. Keep Lambda Functions Lightweight: Since Lambda functions have execution time limits, keep them focused on a specific task and avoid long-running processes.

Conclusion

Integrating Jenkins with AWS Lambda opens up a world of possibilities for automating serverless workflows in your CI/CD pipelines. By triggering Lambda functions directly from Jenkins, you can streamline various processes, such as deployments, event handling, and automated testing.

In this post, we covered the steps to create an AWS Lambda function, set up IAM permissions, configure Jenkins for AWS Lambda integration, and create Jenkins jobs and pipelines that trigger Lambda functions. With these tools and practices in place, you can effectively harness the power of AWS Lambda in your Jenkins workflows.