Connect this data source on your own, using the Hunters platform.
TL;DR
Supported data types | 3rd party detection | Hunters detection | IOC search | Search | Table name | Log format | Collection method |
---|---|---|---|---|---|---|---|
Proofpoint On-Demand Messages | ✅ | ✅ | proofpoint_on_demand_message | NDJSON | API | ||
Proofpoint On-Demand Maillog | proofpoint_on_demand_maillog | NDJSON | API | ||||
Proofpoint Tap Messages Delivered | ✅ | ✅ | proofpoint_tap_messages_delivered | NDJSON | API | ||
Proofpoint Tap Messages Blocked | ✅ | ✅ | proofpoint_tap_messages_blocked | NDJSON | API | ||
Proofpoint Tap Clicks Blocked | ✅ | ✅ | proofpoint_tap_clicks_blocked | NDJSON | API | ||
Proofpoint Tap Clicks Permitted | ✅ | ✅ | proofpoint_tap_clicks_permitted | NDJSON | API | ||
Proofpoint Gateway Filter Logs | ✅ | proofpoint_gateway_filter_logs | Text | S3 | |||
Proofpoint Gateway-Sendmail Logs | proofpoint_gateway_sendmail_logs | Text | S3 |
Overview
This article explains how to integrate your Proofpoint logs into Hunters. Proofpoint has various products and respectful data schemas - TAP, Proofpoint On Demand (PoD), and Proofpoint GateWay (on-premise). Following the guide below will allow Hunters to integrate your Proofpoint logs and ingest them to our database in a predefined schema, and then use these logs in our dedicated hunting mechanism.
Supported data types
📘Self Service Note
Hunters support the ProofPoint on Demand (PoD) Suite in Beta mode. Please reach out to Hunters Support to complete the setup.
Proofpoint On Demand - Proofpoint’s email cloud data services. Consists of raw email data, and includes 2 data types:
proofpoint-on-demand-message
proofpoint-on-demand-maillog
ProofPoint Targeted Attack Protection - Proofpoint’s email cloud protection services, contains alerts data and includes the following data types:
proofpoint-tap-messages-delivered
proofpoint-tap-messages-blocked
proofpoint-tap-clicks-blocked
proofpoint-tap-clicks-permitted
ProofPoint Email Gateway - Proofpoint on Premise server logs. Includes 2 data types:
proofpoint-gateway-filter-logs
proofpoint-gateway-sendmail-logs
Send data to Hunters
Connect Proofpoint On Demand (PoD) logs
Hunters supports the connection of PoD logs (Message and MailLog) using AWS S3 as an intermediary storage.
📘Learn more
Before you dive into the process below, take a look at these general AWS S3 guidelines and consolidate the two processes.
To connect Proofpoint On Demand (PoD) logs:
Log into your Proofpoint On Demand account.
Navigate to Log Management or Log Configuration (this might vary depending on your Proofpoint version).
Look for the Export or Forwarding Options for logs. You should find an option to configure where logs are sent.
Configure the S3 integration:
Provide the S3 Bucket Name and Region.
You may need to input AWS Access Key and Secret Access Key for authentication.
Specify the Log File Format (typically JSON or CSV) that you want the logs to be exported in.
Set the Log Types to "Message" and "MailLog" if available.
Test the connection to ensure that Proofpoint can successfully send logs to your S3 bucket.
Set up S3 log retention and monitoring:
Configure S3 lifecycle policies for automatic log retention and deletion after a specified period.
Optionally, set up S3 event notifications to trigger AWS Lambda functions or other services when new logs arrive.
Once the export is completed and the logs are collected to S3, follow the steps in this section.
Connect Proofpoint Targeted Attack Protection (TAP) logs
Hunters supports the ingestion of these logs using API. To set up the connection, retrieve the TAP Principle and Secret.
To connect Proofpoint TAP logs:
Log into Proofpoint.
Click on the Settings icon and then on Connected Applications.
Click Create New Credentials.
Choose a name and then click Generate.
⚠️ Attention
Save the Principal and Secret values as they will not be available after the window is closed.
Complete the process on the Hunters platform, following this guide.
Proofpoint Email Gateway
To enable Hunters' ingestion of the Email Gateway data for your account, complete the following process:
Set up a collector to gather the data and ship it to an AWS S3 bucket. The data should be in Text format, as exported by the product and as explained below.
Share the bucket information with Hunters Support to establish the connection.
Logs supported by Hunters contain two data types,filter_instance
and sendmail data
. The logs are stored as a text which contains key-value format for both data types, and with expected time format as: %Y-%m-%dT%H:%M:%S.%f%z
.
Expected format
Proofpoint Targeted Attack Protection (TAP)
Logs are expected in JSON format.
Messages Blocked
{"completelyRewritten":false,"headerReplyTo":null,"spamScore":100,"malwareScore":0,"quarantineFolder":"Phish","subject":"NewFaxReceived-12/2/2021","headerFrom":"String","recipient":["String"],"fromAddress":["String"],"messageID":"String","cluster":"String","eventType":"messagesBlocked","threatsInfoMap":[{"threatTime":"2021-12-13T10:24:36.000Z","threatType":"url","threat":"URL","threatUrl":"URL","campaignID":null,"threatStatus":"active","classification":"phish","threatID":"String"}],"id":"String","messageSize":9569,"sender":"String","xmailer":null,"impostorScore":0.0,"messageTime":"2021-12-01T18:21:31.000Z","replyToAddress":[],"eventTime":"2021-12-13T10:40:10.744Z","quarantineRule":"inbound_phish","senderIP":"String","GUID":"String","messageParts":[{"md5":"String","filename":"String","sha256":"String","contentType":"text/html","oContentType":"text/html","disposition":"inline","sandboxStatus":"NOT_SUPPORTED"}],"toAddresses":["String"],"modulesRun":["av","spf","sandbox","dkimv","spam","urldefense"],"ccAddresses":[],"policyRoutes":["default_inbound"],"QID":"String","phishScore":100}
Messages Delivered
{"spamScore":0,"phishScore":0,"threatsInfoMap":[{"threatID":"String","threatStatus":"active","classification":"phish","threatUrl":"URL","threatTime":"2021-12-12T22:31:18.000Z","threat":"URL","campaignID":null,"threatType":"url"}],"messageTime":"2021-12-12T22:16:58.000Z","impostorScore":0.0,"malwareScore":0,"cluster":"String","subject":"String","quarantineFolder":"Audit","quarantineRule":"audit","policyRoutes":["default_inbound"],"modulesRun":["av","zerohour","spf","spam","pdr","urldefense"],"messageSize":24546,"headerFrom":"String","headerReplyTo":null,"fromAddress":["String"],"ccAddresses":[],"replyToAddress":[],"toAddresses":["String"],"xmailer":null,"messageParts":[{"disposition":"inline","sha256":"String","md5":"String","filename":"String","sandboxStatus":null,"oContentType":"text/plain","contentType":"text/plain"}],"completelyRewritten":true,"id":"String","QID":"String","GUID":"String","sender":"String","recipient":["String"],"senderIP":"String","messageID":"String"}
Clicks Permitted
{"url":"String","classification":"spam","click_time":"2021-12-06T08:43:36+00:00","threat_time":"2021-12-08T19:46:14+00:00","user_agent":"String","campaign_id":"","id":"String","click_ip":"String","sender":"String","recipient":"String","sender_ip":"String","guid":"String","threat_id":"String","threat_url":"String","threat_status":"active","message_id":"String","domain":"String"}
Clicks Blocked
{"url": "String","classification": "phish","click_time": "2021-11-29T19:46:21+00:00","threat_time": "2021-11-27T00:53:00+00:00","user_agent": "String","campaign_id": "","id": "String","click_ip": "String","sender": "String","recipient": "String","sender_ip": "String","guid": "String","threat_id": "String","threat_url": "String","threat_status": "active","message_id": "String","domain": }
Proofpoint Email Gateway
Logs are expected in text format. Each data type has multiple combinations of keys that can appear in a specific record. Hunters parses the most important and prevalent keys during the ingestion process.
Filter instance logs
2022-02-20T08:35:00.157994-06:00 myserver123 filter_instance1[21074]: rprt s=3ebnb6gs7y mod=session cmd=disconnect module= rule= action= helo=mydomain.com msgs=1 rcpts=1 routes=allow_relay,firewallsafe, duration=0.085 elapsed=0.176
Sendmail logs
2022-02-20T08:34:59.967502-06:00 myserver123 sendmail[21646]: to=<mail@tempmail.com>, delay=1+16:51:01, xdelay=00:00:00, mailer=esmtp, tls_verify=NONE, tls_version=NONE, cipher=NONE, pri=3285075, relay=tempdomain.com. [165.231.167.17], dsn=4.0.0, stat=Deferred: Connection refused by temdomain.com.