--- title: "OPTION 2: Setup streaming using SNS" slug: "option-2-setup-streaming-using-sns" updated: 2025-03-04T16:26:26Z published: 2025-03-04T16:26:26Z canonical: "docs.hunters.ai/option-2-setup-streaming-using-sns" --- > ## Documentation Index > Fetch the complete documentation index at: https://docs.hunters.ai/llms.txt > Use this file to discover all available pages before exploring further. # OPTION 2: Set up streaming using SNS This article will walk you through settings up streaming using SNS. This process is comprised of the following steps: 1. Create an Amazon SNS topic 2. Redirect S3 events to the new SNS topic 3. Subscribe Hunters SQS Queue to the SNS Topic 💡Before you start Before creating the SNS, contact Hunters Support to notify that an SNS subscription will be made and wait for an approval. ****Step 1: Create an Amazon SNS topic**** 1. In the [AWS Management Console](https://us-east-1.console.aws.amazon.com/console/home?region=us-east-1#) search for, and select **SNS**. 2. From the top bar, select the region where your S3 bucket is located. ![image](https://cdn.document360.io/5f63f0e0-3c70-4d00-b623-82493be0bde5/Images/Documentation/sns1%281%29.png) 3. From the left-side menu, select **Topics**. ![image.png](https://cdn.document360.io/5f63f0e0-3c70-4d00-b623-82493be0bde5/Images/Documentation/image%2813%29.png) 4. Now click **Create topic** to open the Create topic screen. 5. In the **Details** section, fill-in the fields as follow: - In the **Type** field, selet **Standard**. - In the **Name** field, fill in a name for the topic. For example: Realtime ingestion 6. In the **Access Policy** section, select **Advanced** to open the JSON editor. 7. In the JSON editor, select **lines 5-27**, and replace them with the following code: ```plaintext    {      "Sid": "s3-publish",      "Effect": "Allow",      "Principal": {        "AWS": "*"      },      "Action": "SNS:Publish",      "Resource": "arn:aws:sns:::",      "Condition": {        "ArnLike": {          "aws:SourceArn": "arn:aws:s3:*:*:"        }      }    } ``` 8. Replace the following variables with actual information: 1. Replace `<region>` with the region in which your S3 bucket is located. 2. Replace `<account>` with your **AWS account ID**. You can find it by clicking your user name from the upper bar. ![image.png](https://cdn.document360.io/5f63f0e0-3c70-4d00-b623-82493be0bde5/Images/Documentation/image%2814%29.png) 3. Replace `<SNS Topic>` with the name of the topic you're creating right now (the same value from step 5). 4. Replace `arn:aws:s3:*:*:<Your S3 bucket>` with the ARN of the relevant bucket. > [!NOTE] > 💡Where can I find the ARN? > > 1. Navigate to **S3 > Buckets**. > 2. Open the relevant bucket. > 3. Click **Properties**. > 4. Copy the ARN from the Bucket Overview section. 9. Once you've replaced all of the placeholders in the JSON code, scroll down and click **Create topic**. ****Step 2: Redirect S3 events to new SNS Topic**** 1. In the [AWS Management Console](https://us-east-1.console.aws.amazon.com/console/home?region=us-east-1#) search for, and select **S3**. ![S3.png](https://cdn.document360.io/5f63f0e0-3c70-4d00-b623-82493be0bde5/Images/Documentation/S3.png) 2. From the left-side menu, click **Buckets**. 3. Locate the relevant bucket from the list and click to open it. 4. Open the **Properties** tab and scroll down to the **Event Notifications** section. ![image.png](https://cdn.document360.io/5f63f0e0-3c70-4d00-b623-82493be0bde5/Images/Documentation/image%287%29.png) 5. Click **Create event notification**. ![image.png](https://cdn.document360.io/5f63f0e0-3c70-4d00-b623-82493be0bde5/Images/Documentation/image%288%29.png) The Create Event Notification page opens. 6. Fill in the fields as follows: - In the **Event Name** field, provide a name for the event notification (e.g. Realtime Ingestion Hunters). - In the **Prefix** field, choose a prefix if your S3 bucket consists of multiple data flows (choose the prefix to the data you wish to ingest). - In the **Event types** section, select the **All object create events** option. - In the **Destination** section, select **SNS topic** and then **Choose from your SNS topics**. - From the **SNS topic** dropdown list, select the SNS topic created in the previous section. ⚠️Attention If you can't find the SNS topic on the list, make sure the SNS topic's region is the same as the bucket region. ****Step 3: Subscribe Hunters SQS Queue to the SNS Topic**** 1. In the [AWS Management Console](https://us-east-1.console.aws.amazon.com/console/home?region=us-east-1#) search for, and select **SNS**. 2. From the left-side menu, click **Topics**. 3. Locate the SNS topic you created in the previous sections of this process and and click to open it. 4. Click **Create subscription** to open the Create subscription screen. ![image.png](https://cdn.document360.io/5f63f0e0-3c70-4d00-b623-82493be0bde5/Images/Documentation/image%2816%29.png) 5. Fill in the field as follows: ![image.png](https://cdn.document360.io/5f63f0e0-3c70-4d00-b623-82493be0bde5/Images/Documentation/image%2818%29.png) - Keep the **Topic ARN** field as is. - Under **Protocol**, select **Amazon SQS**. - In the **Endpoint** field, paste the SQS ARN provided to you on the Hunters platform ![image.png](https://cdn.document360.io/5f63f0e0-3c70-4d00-b623-82493be0bde5/Images/Documentation/image%2817%29.png) - Check the **Enable raw message delivery** checkbox. 6. Click **Create subscription**. You SNS topic ARN will now appear. ![image.png](https://cdn.document360.io/5f63f0e0-3c70-4d00-b623-82493be0bde5/Images/Documentation/image%2819%29.png) ⚠️ Attention Use the technical support ticket created at the start of this process to notify Hunters that a subscription request has been sent. Hunters will then approve the request to complete the process. ****Optional: Configure your other SQS queues to receive messages from SNS topic**** 1. Log into the AWS Management Console. 2. From the home dashboard, choose **SQS**. 3. Search and choose your SQS queue. 4. Navigate to **SNS subscriptions -> Subscribe to Amazon SNS topic** and choose your SNS topic. 5. Navigate to **Access policy** and choose **Edit**. 6. In the access policy, Add a new Statement which allows the new SNS topic to publish messages to your SQS queue. Policy statement example to add to access policy: ```plaintext {      "Sid": "allow SNS to notify",      "Effect": "Allow",      "Principal": {        "Service": "sns.amazonaws.com"      },      "Action": "SQS:SendMessage",      "Resource": "",      "Condition": {        "ArnLike": {          "aws:SourceArn": ""        }      }    } ```