Getting started with the AWS SDK for Rust

·

11 min read

Getting started with the AWS SDK for Rust

Introduction

It’s no secret that Rust has blown up in popularity in recent years. The Stack Overflow developer survey ranked it as the most loved programming language for the 6th time in a row. AWS has adopted it internally with projects like Firecracker powering AWS Lambda and Fargate and they are using it to scale critical infrastructure services like S3, EC2, and CloudFront.

Rust is a systems programming language that has performance on par with C/C++. It has unique features that make managing memory safe like Go/C#/Java — without using a garbage collector. With AWS announcing an official AWS SDK for Rust, and it gaining popularity with developers, Rust could become an important language for optimizing code where performance is critical. AWS recently added Sustainability to the Well-Architected framework and they are investing in Rust so that customers can build more sustainable energy-efficient solutions in the cloud.

Outside of AWS many other companies (Microsoft, Google, Facebook, Cloudflare, Vercel, Discord, Dropbox) are adopting Rust and it is gaining popularity within the JavaScript ecosystem as well.

Getting started with Rust

The official way to install Rust is through a tool called Rustup. Head over to the install page to get started. Rustup will install the Rust toolchain into the ~/.cargo/bin directory and this is where you will find the cargo rustc and rustup binaries. rustc is the Rust compiler and cargo is the Rust package manager. Open source Rust libraries are published on crates.io and you use cargo to download, install and compile packages. Rust packages are also referred to as “crates”.

Run the Rustup installer, then check that cargo is installed with the following command:

$ cargo --version
cargo 1.59.0 (49d8809dc 2022-02-10)

If you’re interested in a more in-depth introduction to Rust the learn page has links to books, tutorials, and examples. The “Rust book” has a getting started tutorial and is a good place to learn more about the language.

Setting up VS Code

Rust has very good editor support which you can check out on the tools page. If you’re using VS Code I recommend using the following extensions:

Hello World

To create a basic rust program use the cargo new command:

$ cargo new hello_rust
Created binary (application) `hello_rust` package
$ cd hello_rust

This generates a src/main.rs file containing our hello world program, and a Cargo.toml which we can use to install packages from crates.io.

fn main() {
    println!("Hello, world!");
}

src/main.rs

To compile and run the program use the cargo run command:

$ cargo run
Compiling hello_rust v0.1.0 (/mnt/c/dev/jakejscott/blog/hello_rust)
    Finished dev [unoptimized + debuginfo] target(s) in 1.02s
     Running `target/debug/hello_rust`
Hello, world!

If you want to check your program is valid use cargo check — this will perform a full compile but skip the code generation step. To do a full compile use cargo build.

Getting started with the AWS SDK for Rust

Open the Cargo.toml file and edit the section under [dependencies]. Once you hit “save” rust-analyzer will start downloading the crates in the background and VS Code will show a progress indicator in the status bar. If you’ve installed the crates extension it will show ✅ for crates that have the latest version specified. If it shows an ❌ you can hover over it a click to upgrade to the latest version of the crate.

[package]
name = "hello_rust"
version = "0.1.0"
edition = "2021"

[dependencies]
tokio = { version = "1.16.1", features = ["full"] }
aws-config = "0.8.0"
aws-types = "0.8.0"
aws-sdk-s3 = "0.8.0"

Cargo.toml

To ensure cargo has downloaded all the crates run cargo build. The first time you build a Rust program it might take a while because it has to download all the crates and build them. On subsequent builds, it’s much faster 😊

$ cargo build

You might be wondering what each of these crates are for so I will give a brief explanation here:

  • The tokio crate provides an asynchronous runtime for Rust. The AWS SDK is built on top of Tokio which you can read more about it here.
  • The aws-config crate has credential provider implementations.
  • The aws-sdk-* crates are for any AWS services you want to use. In this example we are referencing the aws-sdk-s3 crate which we will use to talk to AWS S3. Each AWS service is published as a separate crate. You can view information about the S3 crate on crates.io which has links to the Github repository and documentation. It also shows the downloads for the last 90 days which are trending up!

How to authenticate with AWS

The simplest way to authenticate using the Rust SDK for AWS is using the default credentials profile. On Linux and MacOS, this file is found at ~/.aws/credentials. On Microsoft Windows it is found at %USERPROFILE%\.aws\credentials.

  1. If the credentials file doesn’t exist, create it.
  2. Add the following to the file, where YOUR-ACCESS-KEY is the value of your access key and YOUR-SECRET-KEY is the value of your secret key:
[default]
aws_access_key_id=YOUR-ACCESS-KEY
aws_secret_access_key=YOUR-SECRET-KEY
region=us-east-2

[my-custom-profile]
aws_access_key_id=YOUR-ACCESS-KEY
aws_secret_access_key=YOUR-SECRET-KEY
region=us-east-2

~/.aws/credentials

Update the code in src/main.rs with the following code.

use aws_sdk_s3::{Client, Error};

#[tokio::main]
async fn main() -> Result<(), Error> {
    // Get default credentials
    let config = aws_config::load_from_env().await;

    // Create an S3 client
    let s3 = Client::new(&config);

    // List the first page of buckets in the account
    let response = s3.list_buckets().send().await?;

    // Check if the response returned any buckets
    if let Some(buckets) = response.buckets() {
        // Print each bucket name out
        for bucket in buckets {
            println!("bucket name: {}", bucket.name().unwrap());
        }
    } else {
        println!("You don't have any buckets!");
    }

    Ok(())
}

src/main.rs

If you want to use a named credentials profile you can use the following code:

use aws_config::profile::ProfileFileCredentialsProvider;
use aws_sdk_s3::{Client, Error};

#[tokio::main]
async fn main() -> Result<(), Error> {
    // The name of the custom credentials profile you want to load
    let profile_name = "my-custom-profile";

    // This credentials provider will load credentials from 
    // ~/.aws/credentials.
    let credentials_provider = ProfileFileCredentialsProvider::builder()
        .profile_name(profile_name)
        .build();

    // Load the credentials
    let config = aws_config::from_env()
        .credentials_provider(credentials_provider)
        .load()
        .await;

    // Create an S3 client
    let s3 = Client::new(&config);

    // List the first page of buckets in the account
    let response = s3.list_buckets().send().await?;

    // Check if the response returned any buckets
    if let Some(buckets) = response.buckets() {
        // Print each bucket name out
        for bucket in buckets {
            println!("bucket name: {}", bucket.name().unwrap());
        }
    } else {
        println!("You don't have any buckets!");
    }

    Ok(())
}

src/main.rs

Basic SDK example using S3

For this example, we are going to create a bucket, upload and download some files to it, and then finally I will show you how you can empty a bucket before deleting it. Remember that bucket names are globally unique, so choose a name that nobody else has used before — I’ve highlighted the line of code with a comment ✨ to show you where to change it. If you’re feeling brave, try to type out all the code by hand. You can check if the code is valid by running cargo build.

Create a bucket 🪣

use aws_sdk_s3::{
    model::{BucketLocationConstraint, CreateBucketConfiguration},
    Client, Error,
};

#[tokio::main]
async fn main() -> Result<(), Error> {
    // Get default credentials
    let config = aws_config::load_from_env().await;

    // Create an S3 client
    let s3 = Client::new(&config);

    // The name of the bucket we want to create, make sure to choose 
    // a unique bucket name!
    let bucket_name = "hello-rust-bucket"; // ✨ CHANGE THE BUCKET NAME ✨
    let bucket_region = "ap-southeast-2";

    println!("Creating {bucket_name} in {bucket_region}");

    // If you want to create a bucket outside of `us-east-1` you have to 
    // specify a location constraint.
    let constraint = BucketLocationConstraint::from(bucket_region);
    let bucket_configuration = CreateBucketConfiguration::builder()
        .location_constraint(constraint)
        .build();

    // Create the bucket
    s3.create_bucket()
        .create_bucket_configuration(bucket_configuration)
        .bucket(bucket_name)
        .send()
        .await?;

    println!("Successfully created {bucket_name} 🪣");

    Ok(())
}

src/main.rs

To run the program type cargo run:

$ cargo run
Compiling hello_rust v0.1.0 (/mnt/c/dev/jakejscott/blog/hello_rust)
    Finished dev [unoptimized + debuginfo] target(s) in 2.84s
     Running `target\debug\hello_rust.exe`
Creating hello-rust-bucket in ap-southeast-2
Successfully created hello-rust-bucket 🪣

If you get an error, perhaps the bucket name you used was already taken, the program will panic and print out the error that happened. In the example below you can see a big ugly error:

$ cargo run
Finished dev [unoptimized + debuginfo] target(s) in 0.10s
     Running `target\debug\hello_rust.exe`
Creating hello-rust-bucket in ap-southeast-2
Error: BucketAlreadyOwnedByYou(BucketAlreadyOwnedByYou { message: Some("Your previous request to create the named bucket succeeded and you already own it.") })
error: process didn't exit successfully: `target\debug\hello_rust.exe` (exit code: 1)

Uploading files to S3

We will use the same bucket that we created before and to keep it simple we can use plain text for the contents of the object rather than uploading a file from disk.

use aws_sdk_s3::{ByteStream, Client, Error};

#[tokio::main]
async fn main() -> Result<(), Error> {
    // Get default credentials
    let config = aws_config::load_from_env().await;

    // Create an S3 client
    let s3 = Client::new(&config);

    // The name of the bucket we want to upload a file to.
    let bucket_name = "hello-rust-bucket";

    // The key of the object in the bucket
    let key = "plaintext.txt";

    // Create a `ByteStream` from a string
    let body = ByteStream::from_static("Hello".as_bytes());

    // Upload the object to S3s
    let result = s3
        .put_object()
        .bucket(bucket_name)
        .key(key)
        .body(body)
        .content_type("text/plain")
        .send()
        .await;

    match result {
        Ok(_) => {
            println!("Successfully uploaded {key} to {bucket_name}");
        }
        Err(err) => {
            eprintln!("Error uploading {key} {err}");
        }
    }

    Ok(())
}

src/main.rs

Test it out by running the program, and feel free to check the file is in the bucket using the AWS console 🙂

$ cargo run
Compiling hello_rust v0.1.0 (/mnt/c/dev/jakejscott/blog/hello_rust)
    Finished dev [unoptimized + debuginfo] target(s) in 2.75s
     Running `target\debug\hello_rust.exe`
Successfully uploaded plaintext.txt to hello-rust-bucket

To upload an actual file, add a testfile.txt in the root of the project, in the same folder as the Cargo.toml file, then change the following code:

use aws_sdk_s3::{ByteStream, Client, Error};
use std::path::Path;

#[tokio::main]
async fn main() -> Result<(), Error> {
    // Get default credentials
    let config = aws_config::load_from_env().await;

    // Create an S3 client
    let s3 = Client::new(&config);

    // The name of the bucket we want to upload a file to.
    let bucket_name = "hello-rust-bucket";

    // The key of the object in the bucket
    let file_name = "testfile.txt";

    // Read the file from disk and convert it to a `ByteStream`.
    // Also note the `unwrap()` will panic if there's an error 
    // reading the file.
    let body = ByteStream::from_path(Path::new(file_name)).await.unwrap(); 

    // Upload the object to S3s
    let result = s3
        .put_object()
        .bucket(bucket_name)
        .key(file_name)
        .body(body)
        .content_type("text/plain")
        .send()
        .await;

    match result {
        Ok(_) => {
            println!("Successfully uploaded {file_name} to {bucket_name}");
        }
        Err(err) => {
            eprintln!("Error uploading {file_name} {err}");
        }
    }

    Ok(())
}

src/main.rs

Run the program:

$ cargo run
Compiling hello_rust v0.1.0 (/mnt/c/dev/jakejscott/blog/hello_rust)
    Finished dev [unoptimized + debuginfo] target(s) in 2.75s
     Running `target\debug\hello_rust.exe`
Successfully uploaded testfile.txt to hello-rust-bucket

Downloading files from S3

In this example, we are going to download the files that we previously uploaded and print out the contents of the file.

use aws_sdk_s3::{Client, Error};

#[tokio::main]
async fn main() -> Result<(), Error> {
    // Get default credentials
    let config = aws_config::load_from_env().await;

    // Create an S3 client
    let s3 = Client::new(&config);

    // The name of the bucket we want to upload a file to.
    let bucket_name = "hello-rust-bucket";

    // List the first 10 keys in the bucket
    let result = s3
        .list_objects_v2()
        .bucket(bucket_name)
        .max_keys(10)
        .send()
        .await?;

    // Loop through each object
    for object in result.contents().unwrap() {
        let key = object.key().unwrap();

        // Download the object from S3
        let object = s3.get_object()
            .bucket(bucket_name).key(key).send().await?;

        // Convert the body into a string
        let data = object.body.collect().await.unwrap().into_bytes();

        // Note that this code assumes that the files are utf8 encoded 
        // plain text format.
        let contents = std::str::from_utf8(&data).unwrap();

        println!("Key: {key}, Contents: {contents}");
    }

    Ok(())
}

src/main.rs

Run the program:

$ cargo run
Compiling hello_rust v0.1.0 (/mnt/c/dev/jakejscott/blog/hello_rust)
    Finished dev [unoptimized + debuginfo] target(s) in 2.64s
     Running `target\debug\hello_rust.exe`
Key: plaintext.txt, Contents: Hello
Key: testfile.txt, Contents: This is a test file

Deleting the bucket

You must “empty” the S3 bucket before you can delete it. So this last example is a little bit more complicated as it shows you how you can paginate through objects returned from S3. We iterate through the keys deleting the objects for each page of results. Once all the files are deleted we can delete the bucket.

use aws_sdk_s3::{Client, Error};

#[tokio::main]
async fn main() -> Result<(), Error> {
    // Get default credentials
    let config = aws_config::load_from_env().await;

    // Create an S3 client
    let s3 = Client::new(&config);

    // The name of the bucket we want to upload a file to.
    let bucket_name = "hello-rust-bucket";

    // Paginate through all the files in the bucket
    let mut token: Option<String> = None;
    loop {
        // List 1 file at a time so that we get a pagination token
        let result = s3
            .list_objects_v2()
            .bucket(bucket_name)
            .max_keys(1)
            .set_continuation_token(token.clone())
            .send()
            .await?;

        for object in result.contents().unwrap() {
            let key = object.key().unwrap();

            // Delete the object from S3
            s3.delete_object()
                .bucket(bucket_name)
                .key(key)
                .send()
                .await?;

            println!("Deleted {key}");
        }

        if let Some(next_token) = result.next_continuation_token() {
            token = Some(next_token.to_string());
        } else {
            break;
        }
    }

    // Now we can delete the bucket
    let delete_result = s3.delete_bucket().bucket(bucket_name).send().await;
    match delete_result {
        Ok(_) => {
            println!("Deleted {bucket_name}");
        }
        Err(err) => {
            eprintln!("Error deleting {bucket_name} {err}");
        }
    }

    Ok(())
}

src/main.rs

Run the program to delete the bucket 🔥

$ cargo run
Compiling hello_rust v0.1.0 (/mnt/c/dev/jakejscott/blog/hello_rust)
    Finished dev [unoptimized + debuginfo] target(s) in 2.97s
     Running `target\debug\hello_rust.exe`
Deleted plaintext.txt
Deleted testfile.txt
Deleted hello-rust-bucket

Next Steps

In this article, we discussed how to get started with the Rust programming language, as well as, how to run basic examples using the AWS SDK for Rust. The goal of this article was to hopefully get some readers interested in Rust — without going into too much detail about the language. In the next blog post, we will learn how to use Rust with DynamoDB and perform some basic CRUD operations. We will also cover more about this how to do error handling in Rust. After that, I’m planning to do another article on using Rust with Lambda and eventually a tutorial on how to build out a real-world example using AWS CDK.

Further reading and references

If you’re wanting to learn more about AWS SDK for Rust here are the links to the official docs:

This blog post was originally posted on Serverless Guru's blog serverlessguru.com/blog/aws-sdk-for-rust-ge..