Overview

Before sending data to Hunters SOC, you need to prepare your Cloud and/or On-Premise infrastructure and configure it accordingly so data can be ingested properly.

Refer to the appropriate documentation within this page, based on the Cloud infrastructure you will use in order to ship data to Hunters SOC.

Ingestion from S3

Many security products have a built-in capability to export various logs to AWS S3. For these sources, Hunters SOC supports ingesting the data directly from your S3 buckets. In order to allow Hunters SOC access to your S3 bucket, please follow the steps below to learn how to configure an appropriate ARN role.

Create an IAM Policy

  1. Log into the AWS Management Console.

  2. From the home dashboard, choose Identity & Access Management (IAM).

  3. Choose Policies from the left-hand navigation pane and then click Create Policy.

  4. Click the JSON tab and paste the following document. Note: Make sure to replace bucket and prefix with your actual bucket name and folder path prefix.

    {
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "s3:ListAllMyBuckets",
      "Resource": "*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "s3:ListBucket",
        "s3:GetObject",
        "s3:GetBucketLocation"
       ],
      "Resource": [
         "arn:aws:s3:::BUCKET-NAME-HERE",
         "arn:aws:s3:::BUCKET-NAME-HERE/*"
      ]
    },
    {
      "Effect": "Allow",
      "Action": [
        "kms:Decrypt" 
       ],
      "Resource": "arn:aws:kms:REGION:ACCOUNT_ID:KEY/EXAMPLE_NAME"
    }
  ]
}
    CODE

  5. Give a name to the IAM policy and click Create policy.

Create an IAM Role

  1. Login to AWS console and go to Services > IAM.

  2. Click Create role.

  3. Choose Another AWS account.

  4. In Account ID enter Hunters’ Account ID (as provided to you by Hunters).

  5. Select the Require external ID checkbox. Note: External IDs are always created for you by Hunters, you should not create them manually on your own. For further reading on the purpose of an external ID, see AWS Documentation.

  6. Go to the Hunters SOC portal, select the Data Flows page and click Add Data Flows.

  7. Under Product, select AWS. This should automatically set AWS S3 under Source. Scroll down to Use Cross Account Role and copy the generated External ID.

  8. Final result should look like this (with the AWS account and External ID replaced with the real values):

  9. Click Next: Permissions at the bottom right of the screen. Then, choose the previously created IAM policy and click Next.

  10. Click Next: Tags. If required, add tags to the role. Then, click Next: Review, and grant the role a name.

  11. Click Create role.

  12. Go back to the IAM menu and click Roles. Filter the newly created role and click on it.

  13. Copy the Role ARN. Note: ACCOUNT_ID should match the account you are giving Hunters access permissions to.

  14. If you wish to narrow the role permissions to a specific user provided by Hunters, click the Trust Relationship tab under the role and click Edit trust relationship.

  15. Under Principal -> ARN, replace arn:aws:iam::ACCOUNT_ID:root with the User ARN you were provided by Hunters and click Update Trust Policy. After following these directions and creating all the required resources in your AWS account, go back to the relevant Product wizard in the Hunters SOC platform, where you will be asked to provide your role's ARN.

AWS S3 Streaming

Many security products have a built-in capability to export various logs to S3. For these sources, Hunters supports ingesting data from your S3 buckets.

As a part of our Next Generation Ingestion system, Hunters provides a realtime streaming capability, currently available only for POC and in the future will be available for all customers.

NOTE: In order to initially allow access to your S3 bucket, please follow the steps in the first section of this page (IAM Policy creation wizard).

Once access to your S3 bucket is set, please choose one of the alternatives below and follow the steps to configure event notifications on your S3 bucket and direct them into Hunters' SQS queue. The resulting automated notifications will allow Hunters to ingest data from your S3 bucket in realtime.

Alternative 1: Creating a New Direct S3 Event Notification

This section describes the most common option for automating S3 ingestion using notifications on your S3 bucket and directing them to a dedicated Amazon SQS (Simple Queue Service), provided by Hunters. The steps below explain how to create an event notification for the target path (or “prefix,” in AWS terminology) in your S3 bucket where your security data is stored.

Important: If a conflicting event notification exists for your S3 bucket, use [Alternative 2: Configure an SNS topic](#Alternative 2: Configure an SNS Topic)

Step 1 - Configure Event Notification

  1. Log into the AWS Management Console.

  2. From the home dashboard, choose S3.

  3. Search and choose your S3 bucket.

  4. Navigate to Properties -> Event notifications -> Create event notification

Complete the fields as follows:

  • Event Name: Name of the event notification (e.g. Realtime Ingestion Hunters).

  • Prefix: Choose prefix if your S3 bucket consists of multiple data flows (choose the prefix to the data you wish to ingest).

  • Event types: Select the ObjectCreate (All) option.

  • Destination: Select SQS Queue from the list.

  • Specify SQS queue: Select Enter SQS queue ARN from the list.

  • SQS queue ARN: Paste the SQS queue that was set for you by Hunters.

For more information: AWS S3 documentation.

Alternative 2: Configure an SNS Topic

This section describes how to automate S3 ingestion using Amazon SNS (Simple Notification Service) notifications and directing them to a dedicated Amazon SQS (Simple Queue Service), provided by Hunters. The steps below explain how to configure an SNS topic that will allow publishing S3 event notifications to multiple subscribers in parallel, including Hunters' automated ingestion SQS queue.

Important: This section assumes that there's an existing event notification on your S3 bucket for the desired target path where your security events reside. If no such event notification exists please follow [Alternative 1: Creating a New Direct S3 Event Notification](#Alternative 1: Creating a New Direct S3 Event Notification)

Step 1 - Create an Amazon SNS Topic and Subscription

  1. Log into the AWS Management Console.

  2. From the home dashboard, choose SNS.

  3. Choose the same region where your S3 bucket resides.

  4. Choose Topics from the left-hand navigation pane.

  5. Navigate to Create topic

Complete the fields as follows:

  • Type: Select Standard

  • Event Name: Name of SNS topic (e.g. realtime-ingestion).

  • Access policy Select Advanced and add a new Statement which allows your S3 bucket to Publish to the SNS topic

Policy statement example to add to access policy:

    {
      "Sid": "s3-publish",
      "Effect": "Allow",
      "Principal": {
        "AWS": "*"
      },
      "Action": "SNS:Publish",
      "Resource": "arn:aws:sns:<region>:<account>:<SNS Topic>",
      "Condition": {
        "ArnLike": {
          "aws:SourceArn": "arn:aws:s3:*:*:<Your S3 bucket>"
        }
      }
    }
CODE

Step 2 - Subscribe Hunter's SQS Queue to the SNS Topic

  1. Log into the AWS Management Console.

  2. From the home dashboard, choose SNS.

  3. Choose Topics from the left-hand navigation pane.

  4. Search and choose your SNS topic.

  5. Navigate to Subscriptions -> Create Subscription

Complete the fields as follows:

  • Topic ARN Keep as is (the current SNS topic ARN)

  • Protocol Choose Amazon SQS from the dropdown list

  • Endpoint Insert the SQS ARN provided to you by Hunters

  • Enable raw message delivery Tick the box (important !)

In order to complete the step you must provide Hunters with your SNS topic ARN. Currently there's no existing interface, please pass your SNS topic ARN manually to Hunters' personnel

Step 3: (Optional) Configure your other SQS queues to receive messages from SNS topic

  1. Log into the AWS Management Console.

  2. From the home dashboard, choose SQS.

  3. Search and choose your SQS queue.

  4. Navigate to SNS subscriptions -> Subscribe to Amazon SNS topic and choose your SNS topic

  5. Navigate to Access policy and choose Edit

  • Access policy Add a new Statement which allows the new SNS topic to publish messages to your SQS queue

Policy statement example to add to access policy:

{
      "Sid": "allow SNS to notify",
      "Effect": "Allow",
      "Principal": {
        "Service": "sns.amazonaws.com"
      },
      "Action": "SQS:SendMessage",
      "Resource": "<Existing SQS ARN>",
      "Condition": {
        "ArnLike": {
          "aws:SourceArn": "<SNS topic ARN>"
        }
      }
    }
CODE

Step 4 - Redirect S3 events to new SNS Topic

  1. Log into the AWS Management Console.

  2. From the home dashboard, choose S3.

  3. Search and choose your S3 bucket.

  4. Navigate to Properties -> Event notifications -> Create event notification

Complete the fields as follows:

  • Event Name: Name of the event notification.

  • Prefix: Choose prefix if your S3 bucket consists of multiple data flows (choose the prefix to the data you wish to ingest).

  • Event types: Select the ObjectCreate (All) option.

  • Destination: Select SNS topic from the list.

  • Specify SNS topic: Select Enter SNS topic ARN from the list.

  • SNS topic: Paste the new SNS topic that you created in Step 1.

Microsoft Azure

Many security products have a built-in capability to export various logs to Azure Blob Storage. In order to allow us access to your Azure data, please follow these steps to create access keys.

Storage Accounts

In your Azure portal home screen, click Storage Accounts and choose the account that is to be shared with Hunters.

Connection Strings

Click Access Keys and share with us the two Connection strings available (key1, key2).