Interacting with AWS S3 in MODLR Processes

[Daniel Murphy]

MODLR Processes support the ability to interact with an AWS S3 (Simple Storage Service) Bucket. To interact with an S3 Bucket, an S3Bucket object must first be initialized with the following parameters

• access_key (string): The access key for AWS S3.
• secret_key (string): The secret key for AWS S3.
• bucket (string): The name of the S3 bucket you want to interact with.
• region (string): The region where your S3 bucket resides

var s3Bucket = datasource.S3Bucket("YOUR_ACCESS_KEY", "YOUR_SECRET_KEY", "YOUR_BUCKET_NAME", "us-west-1");

Once an S3 bucket is initialized there are four methods to interact with it

list(): Returns a list of objects in the bucket, each object in the returned list contains:

• name: Name of the object.
• size: Size of the object in bytes.
• last_modified: Timestamp when the object was last modified.

let allObjects = s3Bucket.list();
allObjects.forEach(object => {
    console.log(`Name: ${object.name}, Size: ${object.size}, Last Modified: ${object.last_modified}`);
});

uploadFile(object_key, uploaded_filename): Uploads a file from the MODLR file system to the S3 bucket.

Parameters:

• object_key(string): The key for the object in the S3 bucket.
• uploaded_filename(string): Path of the file to upload.
  

let objectKey = "destination/folder/filename.txt";
let uploadedFilename = "local/path/to/yourfile.txt";

s3Bucket.uploadFile(objectKey, uploadedFilename);

downloadFile(object_key, output_filename): Downloads a file from the S3 bucket into the MODLR file system.

Parameters:

• object_key(String): The key for the object in the S3 bucket.
• output_filename(String): The path where the file will be saved.

let objectKey = "source/folder/filename.txt";
let outputFilename = "local/path/where/file/should/be/saved.txt";

s3Bucket.downloadFile(objectKey, outputFilename);

generatePresignedURL(object_key, expires_after_minutes): Generates a pre-signed URL for an object. A presigned URL is a URL that grants temporary access to a specific S3 object. More information about pre-signed URLs can be found on the S3 Documentation Page.

Parameters:

• object_key(String): The key for the object in the S3 bucket.
• expires_after_minutes(Number): Duration in minutes after which the generated URL will expire.
  

let objectKey = "path/to/your/object.txt";
let expiresAfterMinutes = 60;
let presignedUrl = s3Bucket.generatePresignedURL(objectKey, expiresAfterMinutes);

console.log(`Pre-signed URL: ${presignedUrl}`);