How to read files from S3 using Python AWS Lambda

This post explains how to read a file from S3 bucket using Python AWS Lambda function. We will use boto3 apis to read files from S3 bucket.

In this tutorial you will learn how to
  1. Read a file from S3 using Python Lambda Function.
  2. List and read all files from a specific S3 prefix using Python Lambda Function.

Create Lambda Function

  1. Login to AWS account and Navigate to AWS Lambda Service.

  2. Navigate to AWS Lambda function and select Functions

  3. Click on Create function

  4. Select Author from scratch
  5. Enter Below details in Basic information
    • Function name: test_lambda_function
    • Runtime: choose run time as per the python version from output of Step 3
    • Architecture: x86_64
    • Select appropriate role that is having proper S3 bucket permission from Change default execution role

  6. Click on create function

Read a file from S3 using Lambda function

  • Import boto3 and create S3 client
           
    import boto3
    s3_client = boto3.client("s3")
           
  • Define bucket name
           
    S3_BUCKET_NAME = 'BUCKET_NAME'
           
  • Define lambda handler

    Write below code in Lambda function and replace the OBJECT_KEY.

           
    def lambda_handler(event, context):
      object_key = "OBJECT_KEY"  # replace object key
      file_content = s3_client.get_object(
          Bucket=S3_BUCKET, Key=object_key)["Body"].read()
      print(file_content)
           
  • Complete code for reading a S3 file with AWS Lambda Python
     
    import boto3
    
    s3_client = boto3.client("s3")
    S3_BUCKET = 'BUCKET_NAME'
    
    def lambda_handler(event, context):
      object_key = "OBJECT_KEY"  # replace object key
      file_content = s3_client.get_object(
          Bucket=S3_BUCKET, Key=object_key)["Body"].read()
      print(file_content)
          
           

List and read all files from a specific S3 prefix

  • Define bucket name and prefix.
       
    import json
    import boto3
    
    s3_client = boto3.client("s3")
    S3_BUCKET = 'BUCKET_NAME'
    S3_PREFIX = 'BUCKET_PREFIX'
         
  • Write below code in Lambda handler to list and read all the files from a S3 prefix. Replace BUCKET_NAME and BUCKET_PREFIX.

     
      
    def lambda_handler(event, context):
        response = s3_client.list_objects_v2(
            Bucket=S3_BUCKET, Prefix=S3_PREFIX, StartAfter=S3_PREFIX,)
        s3_files = response["Contents"]
        for s3_file in s3_files:
            file_content = json.loads(s3_client.get_object(
                Bucket=S3_BUCKET, Key=s3_file["Key"])["Body"].read())
            print(file_content)
         
  • Complete code snippet to list and read all files.
  •    
    import json
    import boto3
    
    s3_client = boto3.client("s3")
    S3_BUCKET = 'BUCKET_NAME'
    S3_PREFIX = 'BUCKET_PREFIX'
    
    
    def lambda_handler(event, context):
        response = s3_client.list_objects_v2(
            Bucket=S3_BUCKET, Prefix=S3_PREFIX, StartAfter=S3_PREFIX,)
        s3_files = response["Contents"]
        for s3_file in s3_files:
            file_content = json.loads(s3_client.get_object(
                Bucket=S3_BUCKET, Key=s3_file["Key"])["Body"].read())
            print(file_content)
     

Follow US on Twitter:

Category: AWS

Similar Articles