I recently wrote a lambda in Rust that needed me to decode an SQS message and use it within the lambda. This post will help fellow Rust newbies that are trying to decode an SQS message and use it within a lambda. The heart of this post is simply converting a string to a JSON – so please skip this post if you are an experienced Rust programmer. But if you are like me, a non-programmer that has been trying to use Rust well before fully grokking the language, read on.
Note: The code for this example is in this gist.
Background
I am moving to Rust. I have begun using it in my daily tasks. While Qxf2 loves Python I feel like it is time for us to switch to Rust. The business reasons are beyond this post but what I can say is that as a company we will move to Rust in the coming years. This blog post captures an early attempt to use Rust in a practical setting. I feel extremely apprehensive to write such a post. One, the material here is probably too simple for any experienced Rust developer. Two, I am such a n00b that I am not confident that I have done the right thing. If I am wrong, I will simply hide behind Cunningham’s law.
The SQS message format
The lambda I was playing with receives a message from a SQS queue. That queue in turn, gets its message from another lambda that I have no control over. The message format looks something like this:
{
"Records": [
{
"body": "{\"Message\":\"{\\\"msg\\\": \\\"Some string \\\",
\\\"chat_id\\\": \\\"a channel id\\\",
\\\"user_id\\\":\\\"the user who sent the message\\\"
}
\"}"
}
]
} |
My lambda needs `msg` and `chat_id` from the above message. In a well formed JSON, the data could have been accessed like `Records[0].body.Message.msg`. But in this case, `Records[0].body` is a string that needs to be decoded to a JSON. We need to do the same when decoding the value of `Records[0].body.Message`. If this were a larger project that might evolve into something more complex, we would have to write custom deserializers. But I wanted to avoid writing so much code for such a well defined case and decided to take a much easier if less elegant route.
If you have a different kind of JSON you are decoding, you can follow a similar thought process. The trick is to create a struct for each key you see in the JSON. Then, use the serde_json crate to convert Strings to JSON.
A struct for each key
Let us start by adding one struct for each key you want to decode. This looks something like this:
use serde::{Deserialize, Serialize}; use serde_json; #[derive(Deserialize)] struct SkypeListenerMessage { msg: String, chat_id: String, user_id: String, } #[derive(Deserialize)] struct Message{ #[serde(rename = "Message")] //This is a trick to rename the field in the struct. //It is needed so the Rust compiler does not complain about snake case not being used. message: String, } #[derive(Deserialize)] struct Record { body: String, } #[derive(Deserialize)] struct Event { //This struct is for the event coming into lambda #[serde(rename = "Records")] records: Vec<Record>, } |
The struct we want to be using in our lambda is: `SkypeListenerMessage`. We have the following deserialization path:
`event` -> `Record` -> `body` -> String => `Message` -> String => `SkypeListenerMessage`
Looking at the path above, it is clear we need to convert a String to a JSON of type `Message`. Then, we need to convert another String to a JSON of type SkypeListenerMessage. These conversions are denoted by => in the deserialization path above. To do this, we use the following lies of code:
let body = &event.records[0].body; println!("Body: {}", body); let message_key: Message = serde_json::from_str(body).unwrap(); println!("Message: {}", message_key.message); let message_details: SkypeListenerMessage = serde_json::from_str(&message_key.message).unwrap(); let message = SkypeListenerMessage{ msg: message_details.msg, user_id: message_details.user_id, chat_id: message_details.chat_id, }; println!("Message: {}", message.msg); println!("User ID: {}", message.user_id); println!("Channel: {}", message.chat_id); |
And that is about it! We now have what we want within the lambda handler. We can go ahead and implement our lambda’s logic.
Putting it all together
You can skip this section if you are somewhat comfortable with Rust. However, as a beginner, I have read enough Rust tutorials with snippets that I was not skilled enough to integrate directly into my program. So in the interest of fellow-beginners, I am providing the entire code needed to get this working.
Cargo.toml
This is the Cargo.toml of my project
[package] name = "joke_of_the_day" version = "0.1.0" authors = ["Qxf2 <[email protected]>"] edition = "2018" autobins = false # See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html [dependencies] futures = "0.3" # for our async / await blocks tokio = { version = "1.12.0", features = ["full"] } # for our async runtime serde = { version = "1.0", features = ["derive"] } serde_json = "1.0" lambda_runtime = "0.4.1" # Src: https://aws.amazon.com/blogs/opensource/rust-runtime-for-aws-lambda/ [[bin]] name = "bootstrap" path = "src/main.rs" |
src/main.rs
This is the main.rs file
/* Example of decoding a (poorly crafted?) SQS message using Rust */ use lambda_runtime::{Context, Error}; use serde::{Deserialize, Serialize}; use serde_json; #[derive(Deserialize)] struct SkypeListenerMessage { msg: String, chat_id: String, user_id: String, } #[derive(Deserialize)] struct Message{ #[serde(rename = "Message")] message: String, } #[derive(Deserialize)] struct Record { body: String, } #[derive(Deserialize)] struct Event { #[serde(rename = "Records")] records: Vec<Record>, } #[derive(Serialize)] struct Output {//Done just for this example //You will likely have a different structure here once you write you lambda status: String, } #[tokio::main] async fn main() -> Result<(), Error> { let handler = lambda_runtime::handler_fn(handler); lambda_runtime::run(handler).await?; Ok(()) } fn get_message_details(event: Event) -> SkypeListenerMessage{ let body = &event.records[0].body; println!("Body: {}", body); let message_key: Message = serde_json::from_str(body).unwrap(); println!("Message: {}", message_key.message); let message_details: SkypeListenerMessage = serde_json::from_str(&message_key.message).unwrap(); let message = SkypeListenerMessage{ msg: message_details.msg, user_id: message_details.user_id, chat_id: message_details.chat_id, }; println!("Message: {}", message.msg); println!("User ID: {}", message.user_id); println!("Channel: {}", message.chat_id); return message; } async fn handler(event: Event, _context: Context) -> Result<Output, Error> { let message = get_message_details(event); let status = "This is an example".to_string(); Ok(Output { status }) } |
You can try following the instructions put out by Amazon to compile your code and deploy it to your lambda.
This was a useful exercise for me (a Rust beginner) to try and work with Rust. I have a nagging feeling that I am not thinking about Rust correctly. I suspect that I am thinking like a Python developer when writing Rust. Feel free to comment with how this code should be improved.
I want to find out what conditions produce remarkable software. A few years ago, I chose to work as the first professional tester at a startup. I successfully won credibility for testers and established a world-class team. I have lead the testing for early versions of multiple products. Today, I run Qxf2 Services. Qxf2 provides software testing services for startups. If you are interested in what Qxf2 offers or simply want to talk about testing, you can contact me at: [email protected]. I like testing, math, chess and dogs.
Hi!
Did you all look at Golang?
Hi Eric,
We have incidentally used Golang at a client. But as a group we mostly use Python, JavaScript and (recently) Rust for most of our internal projects. What were you looking for?