OPTION 1: Set up streaming using SQS

  • Updated on Mar 4, 2025
  • Published on Jan 23, 2023

The most common method for automating S3 ingestion is using notifications on your S3 bucket and directing them to a dedicated Amazon SQS (Simple Queue Service), provided by Hunters. The steps below explain how to create an event notification for the target path (or “prefix,” in AWS terminology) in your S3 bucket where your security data is stored.

⚠️ Attention

If a conflicting event notification exists for your S3 bucket, use OPTION 2: Setup streaming using SNS.

💡Before you start

To complete this process, you'll need the SQS ARN queue value from the previous step.

SQS ARN.png


To configure event notifications:

  1. Log into the AWS Management Console.

  2. In the AWS Management Console search for, and select S3.
    S3.png

  3. From the left-side menu, click Buckets.

  4. Locate the relevant bucket from the list and click to open it.

  5. Open the Properties tab and scroll down to the Event Notifications section.
    image.png

  6. Click Create event notification.
    image.png
    The Create Event Notification page opens.

  7. Fill in the fields as follows:

    • Event Name: Name of the event notification (e.g. Realtime Ingestion Hunters).

    • Prefix: Choose prefix if your S3 bucket consists of multiple data flows (choose the prefix to the data you wish to ingest).

    • Event types: Select the ObjectCreate (All) option.

    • Destination: Select SQS Queue from the list.

    • Specify SQS queue: Select Enter SQS queue ARN from the list.

    • SQS queue ARN: Paste the SQS queue that was set for you by Hunters.

📘Learn more

AWS S3 documentation