Amazon SQS

Amazon Simple Queue Service (SQS) is a fully managed distributed message queuing service that enables you to decouple and scale microservices, distributed systems, and serverless applications.

Asynchronous workflows have always been the primary use case for SQS. Using queues ensures one component can keep running smoothly without losing data when another component is unavailable or slow.

amazon sqs 572x360 1

How does It Work?

Amazon SQS works on the concept like any other messaging queue with minor enhancements. Producer creates a message and puts it into a queue. We can have multiple producers and add multiple messages to the queue at the same time – you don’t have to worry about the traffic or peaks, SQS handles that for you.

  • Amazon SQS does not support subscription feature for consumer. Consumer is requesting new messages periodically from the queue. You can have multiple consumers, but you must remember that each message can be processed only once. It means that you can have multiple instances of the same consumer, but you can’t read the same message from one queue in two different components.

Handling Data Deduplication

After a consumer processes the message, it must be deleted from the queue. Deleting is important because SQS assumes that processing can fail. To prevent that, after the consumer receives a message, it is hidden from the queue for a defined period and after that, if it is not deleted, the message shows up in the queue again.

Type of Amazon SQS Queue

SQS offers two types of message queues. Standard queues offer maximum throughput, best-effort ordering, and at-least-once delivery. SQS FIFO queues are designed to guarantee that messages are processed exactly once, in the exact order that they are sent.

Features

  • SQS is a distributed queue and it is possible that messages won’t be deleted by a consumer, the message can be received at least once. Once message is processed it must be deleted to prevent deduplication of messages.
  • The message size is limited to 256KB (the body and all attributes).
  • Messages are not retained indefinitely in the queue. By default, messages are deleted after 4 days, but you can extend this time to 14 days.
  • SQS, being a distributed queue, can return messages to the consumers randomly, with no defined order. You can add your own identifier as a part of the message, but the approximate order is probably enough to work with in most cases.
  • “Expect failure but don’t accept it”. Some messages can generate errors every time they are processed – in SQS you can define a special queue “dead letter Q”. It receives messages processed with errors more than n times (where 1⩽n<1000). This is especially useful if you don’t want to lose messages.

[newsletter_form form="1" list="1"]

Disclaimer: I am the author at PLM ECOSYSTEM, focusing on developing digital-thread platforms with capabilities across CAD, CAM, CAE, PLM, ERP, and IT systems to manage the product data lifecycle and connect various industry networks. My opinions may be biased. Articles and thoughts on PLMES represent solely the author's views and not necessarily those of the company. Reviews and mentions do not imply endorsement or recommendations for purchase.