Prerequisites
Overview
Before sending data to Hunters SOC, you need to prepare your Cloud and/or On-Premise infrastructure and configure it accordingly so data can be ingested properly.
Refer to the appropriate documentation within this page, based on the Cloud infrastructure you will use in order to ship data to Hunters SOC.
Ingestion from S3
Many security products have a built-in capability to export various logs to AWS S3. For these sources, Hunters SOC supports ingesting the data directly from your S3 buckets. In order to allow Hunters SOC access to your S3 bucket, please follow the steps below to learn how to configure an appropriate ARN role.
Create an IAM Policy
Log into the AWS Management Console.
From the home dashboard, choose Identity & Access Management (IAM).
Choose Policies from the left-hand navigation pane and then click Create Policy.
Click the JSON tab and paste the following document. Note: Make sure to replace bucket and prefix with your actual bucket name and folder path prefix.
{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": "s3:ListAllMyBuckets", "Resource": "*" }, { "Effect": "Allow", "Action": [ "s3:ListBucket", "s3:GetObject", "s3:GetBucketLocation" ], "Resource": [ "arn:aws:s3:::BUCKET-NAME-HERE", "arn:aws:s3:::BUCKET-NAME-HERE/*" ] }, { "Effect": "Allow", "Action": [ "kms:Decrypt" ], "Resource": "arn:aws:kms:REGION:ACCOUNT_ID:KEY/EXAMPLE_NAME" } ] }
CODEGive a name to the IAM policy and click Create policy.
Create an IAM Role
Login to AWS console and go to Services > IAM.
Click Create role.
Choose Another AWS account.
In Account ID enter Hunters’ Account ID (as provided to you by Hunters).
Select the Require external ID checkbox. Note: External IDs are always created for you by Hunters, you should not create them manually on your own. For further reading on the purpose of an external ID, see AWS Documentation.
Go to the Hunters SOC portal, select the Data Flows page and click Add Data Flows.
Under Product, select AWS. This should automatically set AWS S3 under Source. Scroll down to Use Cross Account Role and copy the generated External ID.
Final result should look like this (with the AWS account and External ID replaced with the real values):
Click Next: Permissions at the bottom right of the screen. Then, choose the previously created IAM policy and click Next.
Click Next: Tags. If required, add tags to the role. Then, click Next: Review, and grant the role a name.
Click Create role.
Go back to the IAM menu and click Roles. Filter the newly created role and click on it.
Copy the Role ARN. Note:
ACCOUNT_ID
should match the account you are giving Hunters access permissions to.If you wish to narrow the role permissions to a specific user provided by Hunters, click the Trust Relationship tab under the role and click Edit trust relationship.
Under Principal -> ARN, replace
arn:aws:iam::ACCOUNT_ID:root
with the User ARN you were provided by Hunters and click Update Trust Policy. After following these directions and creating all the required resources in your AWS account, go back to the relevant Product wizard in the Hunters SOC platform, where you will be asked to provide your role's ARN.
AWS S3 Streaming
Many security products have a built-in capability to export various logs to S3. For these sources, Hunters supports ingesting data from your S3 buckets.
As a part of our Next Generation Ingestion system, Hunters provides a realtime streaming capability, currently available only for POC and in the future will be available for all customers.
NOTE: In order to initially allow access to your S3 bucket, please follow the steps in the first section of this page (IAM Policy creation wizard).
Once access to your S3 bucket is set, please choose one of the alternatives below and follow the steps to configure event notifications on your S3 bucket and direct them into Hunters' SQS queue. The resulting automated notifications will allow Hunters to ingest data from your S3 bucket in realtime.
Alternative 1: Creating a New Direct S3 Event Notification
This section describes the most common option for automating S3 ingestion using notifications on your S3 bucket and directing them to a dedicated Amazon SQS (Simple Queue Service), provided by Hunters. The steps below explain how to create an event notification for the target path (or “prefix,” in AWS terminology) in your S3 bucket where your security data is stored.
Important: If a conflicting event notification exists for your S3 bucket, use [Alternative 2: Configure an SNS topic](#Alternative 2: Configure an SNS Topic)
Step 1 - Configure Event Notification
Log into the AWS Management Console.
From the home dashboard, choose S3.
Search and choose your S3 bucket.
Navigate to Properties -> Event notifications -> Create event notification
Complete the fields as follows:
Event Name: Name of the event notification (e.g. Realtime Ingestion Hunters).
Prefix: Choose prefix if your S3 bucket consists of multiple data flows (choose the prefix to the data you wish to ingest).
Event types: Select the ObjectCreate (All) option.
Destination: Select SQS Queue from the list.
Specify SQS queue: Select
Enter SQS queue ARN
from the list.SQS queue ARN: Paste the SQS queue that was set for you by Hunters.
For more information: AWS S3 documentation.
Alternative 2: Configure an SNS Topic
This section describes how to automate S3 ingestion using Amazon SNS (Simple Notification Service) notifications and directing them to a dedicated Amazon SQS (Simple Queue Service), provided by Hunters. The steps below explain how to configure an SNS topic that will allow publishing S3 event notifications to multiple subscribers in parallel, including Hunters' automated ingestion SQS queue.
Important: This section assumes that there's an existing event notification on your S3 bucket for the desired target path where your security events reside. If no such event notification exists please follow [Alternative 1: Creating a New Direct S3 Event Notification](#Alternative 1: Creating a New Direct S3 Event Notification)
Step 1 - Create an Amazon SNS Topic and Subscription
Log into the AWS Management Console.
From the home dashboard, choose SNS.
Choose the same region where your S3 bucket resides.
Choose Topics from the left-hand navigation pane.
Navigate to Create topic
Complete the fields as follows:
Type: Select
Standard
Event Name: Name of SNS topic (e.g. realtime-ingestion).
Access policy Select
Advanced
and add a new Statement which allows your S3 bucket to Publish to the SNS topic
Policy statement example to add to access policy:
{
"Sid": "s3-publish",
"Effect": "Allow",
"Principal": {
"AWS": "*"
},
"Action": "SNS:Publish",
"Resource": "arn:aws:sns:<region>:<account>:<SNS Topic>",
"Condition": {
"ArnLike": {
"aws:SourceArn": "arn:aws:s3:*:*:<Your S3 bucket>"
}
}
}
Step 2 - Subscribe Hunter's SQS Queue to the SNS Topic
Log into the AWS Management Console.
From the home dashboard, choose SNS.
Choose Topics from the left-hand navigation pane.
Search and choose your SNS topic.
Navigate to Subscriptions -> Create Subscription
Complete the fields as follows:
Topic ARN Keep as is (the current SNS topic ARN)
Protocol Choose
Amazon SQS
from the dropdown listEndpoint Insert the SQS ARN provided to you by Hunters
Enable raw message delivery Tick the box (important !)
In order to complete the step you must provide Hunters with your SNS topic ARN. Currently there's no existing interface, please pass your SNS topic ARN manually to Hunters' personnel
Step 3: (Optional) Configure your other SQS queues to receive messages from SNS topic
Log into the AWS Management Console.
From the home dashboard, choose SQS.
Search and choose your SQS queue.
Navigate to SNS subscriptions -> Subscribe to Amazon SNS topic and choose your SNS topic
Navigate to Access policy and choose Edit
Access policy Add a new Statement which allows the new SNS topic to publish messages to your SQS queue
Policy statement example to add to access policy:
{
"Sid": "allow SNS to notify",
"Effect": "Allow",
"Principal": {
"Service": "sns.amazonaws.com"
},
"Action": "SQS:SendMessage",
"Resource": "<Existing SQS ARN>",
"Condition": {
"ArnLike": {
"aws:SourceArn": "<SNS topic ARN>"
}
}
}
Step 4 - Redirect S3 events to new SNS Topic
Log into the AWS Management Console.
From the home dashboard, choose S3.
Search and choose your S3 bucket.
Navigate to Properties -> Event notifications -> Create event notification
Complete the fields as follows:
Event Name: Name of the event notification.
Prefix: Choose prefix if your S3 bucket consists of multiple data flows (choose the prefix to the data you wish to ingest).
Event types: Select the ObjectCreate (All) option.
Destination: Select SNS topic from the list.
Specify SNS topic: Select
Enter SNS topic ARN
from the list.SNS topic: Paste the new SNS topic that you created in Step 1.
Microsoft Azure
Many security products have a built-in capability to export various logs to Azure Blob Storage. In order to allow us access to your Azure data, please follow these steps to create access keys.
Storage Accounts
In your Azure portal home screen, click Storage Accounts and choose the account that is to be shared with Hunters.
Connection Strings
Click Access Keys and share with us the two Connection strings available (key1, key2).
