AWS Bedrock is a fully-managed service that provides LLM interactions and has models from lots of leading organisations. In my opinion, it's a great way to get started with LLMs if you're using AWS since there's no worry about infrastructure scaling.

This post is a short description of how to get AWS Bedrock responding to your Rust code. It's a simple example, but should point the way forward for more complex interactions.

At the time of writing (April 2024), there are no Rust Bedrock examples in the usual AWS Doc SDK Examples Github Repo. The documentation on each crate is also vague, since they omit the actual code examples.

For example, this is on the README of the Rust crates:

// ... make some calls with the client

And the client just takes a Blob as a payload!

Anyway, this is what the Internet is for; people can talk and share their experiences.

Amazon will likely catch up on their documentation and this blog post will be rapidly out-of-date. But for now, let's get started!

Firstly, request access to Bedrock and models. I did this via the AWS Console, since I was hacking around and didn't want to Terraform anything just yet.

I'm also using the eu-central-1 AWS region as Bedrock isn't currently available in all AWS Regions. My closest region eu-west-2 doesn't have Bedrock yet.

The Bedrock SDKs are split into multiple packages for all programming languages. Rust has four.

For this short intro we'll use the crate aws-sdk-bedrockruntime which allows us to submit inference requests, plus the standard aws_config crate. The other Bedrock crates not used here cover provisioning, and using agents.

Once you have access to Bedrock, and have a cargo project with the required dependencies (plus tokio, serde, and serde_json), we can try figuring out how to send a request.

As mentioned earlier, the documentation is vague. We need to create a Client set some parameters, and then send a request. The body is a blob of JSON.

There are some docs on more inference parameters, such as temperature here.

I'll keep it simple though, and just send a simple prompt:

use serde::Serialize;

#[derive(Debug, Serialize)]
#[serde(rename_all = "camelCase")]
struct PromptBody {
    input_text: String,
}

We'll use the Amazon Titan Text Express model. The code then becomes:

use aws_config::BehaviorVersion;
use aws_sdk_bedrockruntime as bedrock;
use bedrock::primitives::Blob;
use std::str;


#[tokio::main]
async fn main() {
    let config = aws_config::load_defaults(BehaviorVersion::latest()).await;
    let client = bedrock::Client::new(&config);

    let prompt = PromptBody {
        input_text: "Hello, are you an LLM?".to_string(),
    };

    let result = client
        .invoke_model()
        .model_id("amazon.titan-text-express-v1")
        .content_type("application/json")
        .body(Blob::new(serde_json::to_string(&prompt).unwrap()))
        .send()
        .await
        .unwrap();

    let output = str::from_utf8(result.body().as_ref()).unwrap();
    println!("{}", output);
}

Running this code with cargo run yields the following JSON.

{
    "inputTextTokenCount": 8,
    "results": [
        {
            "tokenCount": 24,
            "outputText": "\nNo, this model is not an LLM. It is Amazon Titan, a large language model built by AWS.",
            "completionReason": "FINISH"
        }
    ]
}

This model doesn't seem to realise that LLM means large language model.

I guess it's not perfect yet, but it's an excellent start.

You now have a working Rust program that uses AWS Bedrock, and the foundations in place for your own exploration and experimentation!