{"id":16026,"date":"2022-04-12T03:36:49","date_gmt":"2022-04-12T07:36:49","guid":{"rendered":"https:\/\/qxf2.com\/blog\/?p=16026"},"modified":"2022-04-12T03:36:49","modified_gmt":"2022-04-12T07:36:49","slug":"decode-sqs-messages-lambda-rust","status":"publish","type":"post","link":"https:\/\/qxf2.com\/blog\/decode-sqs-messages-lambda-rust\/","title":{"rendered":"Decode an SQS message within a lambda using Rust"},"content":{"rendered":"

I recently wrote a lambda in Rust that needed me to decode an SQS message and use it within the lambda. This post will help fellow Rust newbies that are trying to decode an SQS message and use it within a lambda. The heart of this post is simply converting a string to a JSON – so please skip this post if you are an experienced Rust programmer. But if you are like me, a non-programmer that has been trying to use Rust well before fully grokking the language, read on. <\/p>\n

Note:<\/strong> The code for this example is in this gist<\/a>.<\/p>\n

\n

Background<\/h3>\n

I am moving to Rust. I have begun using it in my daily tasks. While Qxf2<\/a> loves Python I feel like it is time for us to switch to Rust. The business reasons are beyond this post but what I can say is that as a company we will move to Rust in the coming years. This blog post captures an early attempt to use Rust in a practical setting. I feel extremely apprehensive to write such a post. One, the material here is probably too simple for any experienced Rust<\/del> developer. Two, I am such a n00b that I am not confident that I have done the right thing. If I am wrong, I will simply hide behind Cunningham’s law<\/a>.<\/p>\n

\n

The SQS message format<\/h3>\n

The lambda I was playing with receives a message from a SQS queue. That queue in turn, gets its message from another lambda that I have no control over. The message format looks something like this:<\/p>\n

\r\n{\r\n  \"Records\": [\r\n    {\r\n      \"body\": \"{\\\"Message\\\":\\\"{\\\\\\\"msg\\\\\\\": \\\\\\\"Some string \\\\\\\", \r\n                               \\\\\\\"chat_id\\\\\\\": \\\\\\\"a channel id\\\\\\\", \r\n                               \\\\\\\"user_id\\\\\\\":\\\\\\\"the user who sent the message\\\\\\\"\r\n                              }\r\n             \\\"}\"\r\n    }\r\n  ]\r\n}\r\n<\/pre>\n
My lambda needs `msg` and `chat_id` from the above message. In a well formed JSON, the data could have been accessed like `Records[0].body.Message.msg`. But in this case, `Records[0].body` is a string that needs to be decoded to a JSON. We need to do the same when decoding the value of `Records[0].body.Message`. If this were a larger project that might evolve into something more complex, we would have to write custom deserializers<\/a>. But I wanted to avoid writing so much code for such a well defined case and decided to take a much easier if less elegant route.<\/p>\n
If you have a different kind of JSON you are decoding, you can follow a similar thought process. The trick is to create a struct for each key you see in the JSON. Then, use the serde_json<\/a> crate to convert Strings to JSON.<\/p>\n
\n
A struct for each key<\/h3>\n
Let us start by adding one struct for each key you want to decode. This looks something like this:<\/p>\n
\r\nuse serde::{Deserialize, Serialize};\r\nuse serde_json;\r\n\r\n#[derive(Deserialize)] \r\nstruct SkypeListenerMessage {\r\n    msg: String,\r\n    chat_id: String,\r\n    user_id: String,\r\n}\r\n\r\n#[derive(Deserialize)]\r\nstruct Message{\r\n    #[serde(rename = \"Message\")] \/\/This is a trick to rename the field in the struct. \r\n    \/\/It is needed so the Rust compiler does not complain about snake case not being used.\r\n    message: String,\r\n}\r\n\r\n#[derive(Deserialize)]\r\nstruct Record {\r\n    body: String,\r\n}\r\n\r\n#[derive(Deserialize)]\r\nstruct Event { \/\/This struct is for the event coming into lambda\r\n    #[serde(rename = \"Records\")]\r\n    records: Vec,\r\n}\r\n<\/pre>\n
The struct we want to be using in our lambda is: `SkypeListenerMessage`. We have the following deserialization path:
\n`event` -> `Record` -> `body` -> String => `Message` -> String => `SkypeListenerMessage` <\/p>\n
Looking at the path above, it is clear we need to convert a String to a JSON of type `Message`. Then, we need to convert another String  to a JSON of type SkypeListenerMessage. These conversions are denoted by => in the deserialization path above. To do this, we use the following lies of code:<\/p>\n
\r\n    let body = &event.records[0].body;\r\n    println!(\"Body: {}\", body);\r\n    let message_key: Message = serde_json::from_str(body).unwrap();\r\n    println!(\"Message: {}\", message_key.message);\r\n    let message_details: SkypeListenerMessage = serde_json::from_str(&message_key.message).unwrap();\r\n    let message = SkypeListenerMessage{\r\n        msg: message_details.msg,\r\n        user_id: message_details.user_id,\r\n        chat_id: message_details.chat_id,\r\n    };\r\n    println!(\"Message: {}\", message.msg);\r\n    println!(\"User ID: {}\", message.user_id);\r\n    println!(\"Channel: {}\", message.chat_id);\r\n<\/pre>\n
And that is about it! We now have what we want within the lambda handler. We can go ahead and implement our lambda’s logic.<\/p>\n
\n
Putting it all together<\/h3>\n
You can skip this section if you are somewhat comfortable with Rust. However, as a beginner, I have read enough Rust tutorials with snippets that I was not skilled enough to integrate directly into my program. So in the interest of fellow-beginners, I am providing the entire code needed to get this working. <\/p>\n
Cargo.toml<\/h5>\n
This is the Cargo.toml of my project<\/p>\n
\r\n[package]\r\nname = \"joke_of_the_day\"\r\nversion = \"0.1.0\"\r\nauthors = [\"Qxf2 \"]\r\nedition = \"2018\"\r\nautobins = false\r\n\r\n# See more keys and their definitions at https:\/\/doc.rust-lang.org\/cargo\/reference\/manifest.html\r\n\r\n[dependencies]\r\nfutures = \"0.3\" # for our async \/ await blocks\r\ntokio = { version = \"1.12.0\", features = [\"full\"] } # for our async runtime\r\nserde = { version = \"1.0\", features = [\"derive\"] }\r\nserde_json = \"1.0\"\r\nlambda_runtime = \"0.4.1\"\r\n\r\n# Src: https:\/\/aws.amazon.com\/blogs\/opensource\/rust-runtime-for-aws-lambda\/\r\n[[bin]]\r\nname = \"bootstrap\"\r\npath = \"src\/main.rs\"\r\n<\/pre>\n
src\/main.rs<\/h5>\n
This is the main.rs file<\/p>\n
\r\n\/*\r\nExample of decoding a (poorly crafted?) SQS message using Rust\r\n*\/\r\n\r\nuse lambda_runtime::{Context, Error};\r\nuse serde::{Deserialize, Serialize};\r\nuse serde_json;\r\n\r\n#[derive(Deserialize)]\r\nstruct SkypeListenerMessage {\r\n    msg: String,\r\n    chat_id: String,\r\n    user_id: String,\r\n}\r\n\r\n#[derive(Deserialize)]\r\nstruct Message{\r\n    #[serde(rename = \"Message\")] \r\n    message: String,\r\n}\r\n\r\n#[derive(Deserialize)]\r\nstruct Record {\r\n    body: String,\r\n}\r\n\r\n#[derive(Deserialize)]\r\nstruct Event {\r\n    #[serde(rename = \"Records\")]\r\n    records: Vec,\r\n}\r\n\r\n#[derive(Serialize)]\r\nstruct Output {\/\/Done just for this example\r\n    \/\/You will likely have a different structure here once you write you lambda\r\n    status: String, \r\n}\r\n\r\n#[tokio::main]\r\nasync fn main() -> Result<(), Error> {\r\n    let handler = lambda_runtime::handler_fn(handler);\r\n    lambda_runtime::run(handler).await?;\r\n    Ok(())\r\n}\r\n\r\nfn get_message_details(event: Event) -> SkypeListenerMessage{\r\n    let body = &event.records[0].body;\r\n    println!(\"Body: {}\", body);\r\n    let message_key: Message = serde_json::from_str(body).unwrap();\r\n    println!(\"Message: {}\", message_key.message);\r\n    let message_details: SkypeListenerMessage = serde_json::from_str(&message_key.message).unwrap();\r\n    let message = SkypeListenerMessage{\r\n        msg: message_details.msg,\r\n        user_id: message_details.user_id,\r\n        chat_id: message_details.chat_id,\r\n    };\r\n    println!(\"Message: {}\", message.msg);\r\n    println!(\"User ID: {}\", message.user_id);\r\n    println!(\"Channel: {}\", message.chat_id);\r\n\r\n    return message;\r\n}\r\n\r\nasync fn handler(event: Event, _context: Context) -> Result {\r\n    let message = get_message_details(event);\r\n    let status = \"This is an example\".to_string();\r\n    Ok(Output { status })\r\n}\r\n\r\n<\/pre>\n
You can try following the instructions put out by Amazon<\/a> to compile your code and deploy it to your lambda. <\/p>\n
\n
This was a useful exercise for me (a Rust beginner) to try and work with Rust. I have a nagging feeling that I am not thinking about Rust correctly. I suspect that I am thinking like a Python developer when writing Rust. Feel free to comment with how this code should be improved.<\/p>\n
\n","protected":false},"excerpt":{"rendered":"
I recently wrote a lambda in Rust that needed me to decode an SQS message and use it within the lambda. This post will help fellow Rust newbies that are trying to decode an SQS message and use it within a lambda. The heart of this post is simply converting a string to a JSON – so please skip this […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[177,282,240,306],"tags":[237,307,308],"class_list":["post-16026","post","type-post","status-publish","format-standard","hentry","category-aws","category-aws-lambda","category-aws-sqs","category-rust","tag-amazon-simple-queue-servicesqs","tag-lambda","tag-rust"],"_links":{"self":[{"href":"https:\/\/qxf2.com\/blog\/wp-json\/wp\/v2\/posts\/16026","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/qxf2.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/qxf2.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/qxf2.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/qxf2.com\/blog\/wp-json\/wp\/v2\/comments?post=16026"}],"version-history":[{"count":21,"href":"https:\/\/qxf2.com\/blog\/wp-json\/wp\/v2\/posts\/16026\/revisions"}],"predecessor-version":[{"id":16113,"href":"https:\/\/qxf2.com\/blog\/wp-json\/wp\/v2\/posts\/16026\/revisions\/16113"}],"wp:attachment":[{"href":"https:\/\/qxf2.com\/blog\/wp-json\/wp\/v2\/media?parent=16026"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/qxf2.com\/blog\/wp-json\/wp\/v2\/categories?post=16026"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/qxf2.com\/blog\/wp-json\/wp\/v2\/tags?post=16026"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}