Overview

Many Hunters integrations support the ability for Hunters to gather information from that product from an intermediary S3 bucket. This process works in three stages:

  1. The third party service outputs the data to a specified S3 bucket.

  2. The data is stored in the S3 bucket.

  3. Hunters retrieves the data from the S3 bucket and ingests it.

In this guide, we'll walk you through the steps needed to configure one of these buckets for Hunters to be able to access it. Information on how to get data from the third-party integration into the bucket can be found on each product's Integrations page in the sidebar.


How To Ingest Data From S3

Many security products have a built-in capability to export various logs to AWS S3. For these sources, Hunters SOC supports ingesting the data directly from your S3 buckets. In order to allow Hunters access to your S3 bucket, we'll first create a policy to define the access, and then establish a role which Hunters can assume to access the bucket.

Creating an AWS Bucket

If you haven't already, create a S3 bucket to store logs in. This can be done via the console, or via the command line.

  1. In the AWS Management Console search for, and select "S3".

  2. Click "Create bucket" to open the bucket creation wizard.

  3. Define a bucket name, and choose a desired region. Configure the rest of the wizard with specifics according to your organization's cloud security policies. Some general recommendations are as follows:

  • Object Ownership - ACLs Disabled

  • Bucket Public Access - Keep as "Block all public access"

  • Bucket Versioning - Enabled

  • Default Encryption - Enabled - Amazon S3 Managed Keys

Note, if enabling encryption on the bucket with a customer managed key, that key will need to be included within the IAM policy in the next steps.


Creating an IAM Policy

Next, we'll create an AWS IAM Policy. This will allow Hunters to access the necessary resources for retrieving data from the bucket. The permissions Hunters requires are as follows:

  • s3:ListAllMyBuckets - Allows Hunters to list all buckets in your AWS account (but not read them)

  • s3:ListBucket - Allows Hunters to list the specific bucket we're defining in the policy

  • s3:GetObject - Allows Hunters to retrieve objects in the specific bucket (logs placed in the bucket)

  • s3:GetBucketLocation - Allows Hunters to determine the AWS Region the bucket is located in

  • kms:Decrypt - Allows Hunters to decrypt the bucket contents, if you are using a customer managed KMS key to encrypt the bucket contents.

To create the policy in the console, follow these steps:

  1. In the AWS Management Console search for, and select "Identity & Access Management (IAM)".

  2. Choose Policies from the left-hand navigation pane and click Create Policy.

  3. Click the JSON tab and paste the following document with the replaced text. Make sure to replace <BUCKET-NAME-HERE> with your actual bucket name.

1428

Note: If your bucket is encrypted using KMS Customer Managed Keys, not Amazon S3 Managed Keys, you will need to use the second JSON document provided. Otherwise - use the first.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "s3:ListAllMyBuckets",
      "Resource": "*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "s3:ListBucket",
        "s3:GetObject",
        "s3:GetBucketLocation"
       ],
      "Resource": [
         "arn:aws:s3:::<BUCKET-NAME-HERE>",
         "arn:aws:s3:::<BUCKET-NAME-HERE>/*"
      ]
    }
  ]
}
JSON

Use the following document if you are using custom KMS keys to encrypt your bucket. Make sure to replace <BUCKET-NAME-HERE>, <REGION>, <ACCOUNT_ID>, and <KEY/EXAMPLE_NAME>.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "s3:ListAllMyBuckets",
      "Resource": "*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "s3:ListBucket",
        "s3:GetObject",
        "s3:GetBucketLocation"
       ],
      "Resource": [
         "arn:aws:s3:::<BUCKET-NAME-HERE>",
         "arn:aws:s3:::<BUCKET-NAME-HERE>/*"
      ]
    },
    {
      "Effect": "Allow",
      "Action": [
        "kms:Decrypt" 
       ],
      "Resource": "arn:aws:kms:<REGION>:<ACCOUNT_ID>:<KEY/EXAMPLE_NAME>"
    }
  ]
}
JSON

4. Pick a name for the IAM policy (example: HuntersBucketAccess) and click "Create policy".

1428

Creating an IAM Role

With our policy created, we'll now create an IAM role that Hunters will assume to access the buckets defined via the policy above.

  1. In the Hunters console, visit the AWS Data Source page to retrieve the principal ARN and external ID. Keep the tab open, as we'll use them in a few steps.

  2. In the AWS Management Console search for, and select "Identity & Access Management (IAM)".

  3. In the side bar, select "Roles" and click "Create role".

  4. In the wizard, select "AWS Account" and then "Another AWS account".

  5. Input the account ID from step 1 - this is the last 12 digits of the Principal ARN. As an example:

If the principal ARN is:

arn:aws:iam::123456789123
CODE

The account ID to paste in is:

123456789123
CODE

6. Check the box for "Require External ID", and paste in the External ID from step 1.

7. Click "Next". In the "Add Permissions" section of the wizard, select the policy that was just created in the previous section (HuntersBucketAccess). Then click "Next".

2062

8. Name the role hunters-assume-role, and write a short description.

9. Click "Create role".

2024

10. Return to the Hunters Data Sources page and select the data source that is feeding this bucket logs.

11. Follow the setup instructions in the wizard for configuring that data source. You may be required to provide the ARN of the role, which is available by selecting the hunters-assume-role from the Roles list in the IAM console.