Overview

In this page you’ll find an explanation on integrating your CloudFlare data source to Hunters.

Cloudflare acts as an intermediary between a client and a server, using a reverse proxy to mirror and cache websites. By storing web content for delivery on the closest edge server, it is able to optimize loading times. That also allows it to modify content, such as images and rich text, for better performance

This data source is used in the Hunters Pipeline for detection and investigation regarding the logged activity in the organization's network.

Supported Data Types

  • Cloudflare HTTP - This table holds information about HTTP requests and responses by CloudFlare. For more information on the data schema see here.

  • Cloudflare Firewall - This table is a slimmed-down version of the HTTP requests table, aimed at receiving said logs in a more timely manner (<60 seconds according to Cloudflare). For more information on the data schema see here.

  • Cloudflare DNS - This table holds information about the DNS requests and responses by CloudFlare. For more information on the data schema see here.

Sending Data to Hunters

In order to integrate your Cloudflare logs into Hunters, the logs need to be collected from your network (follow this guide for more details) to a Storage Service (e.g. to an S3 bucket or Azure Blob Storage) shared with Hunters.

Expected formats

Data should be pushed in a NDJSON format.

When prompted by Cloudflare to choose the field names to be exported, please choose to export all fields to the bucket. If there is a storage restriction, please make sure the following groups of fields are exported (all columns within the following groups):

Client, ClientRequest, Edge, Firewall, Origin, OriginResponse, WAF

Do note that supplying a partial subset of the columns might result in content not being fully deployed in your environment.

CloudFlare HTTP Example
{
    "ClientIP": "1.1.1.1",
    "ClientRequestHost": "url.com:7634",
    "ClientRequestMethod": "GET",
    "ClientRequestURI": "String",
    "EdgeEndTimestamp": "2021-12-15T15:34:52Z",
    "EdgeResponseBytes": 3792,
    "EdgeResponseStatus": 504,
    "EdgeStartTimestamp": "2021-12-15T15:33:51Z",
    "RayID": "469a0877e07db07",
    "BotTags": [],
    "CacheCacheStatus": "unknown",
    "CacheResponseBytes": 7325,
    "CacheResponseStatus": 504,
    "CacheTieredFill": false,
    "ClientASN": 5682,
    "ClientCountry": "",
    "ClientDeviceType": "desktop",
    "ClientIPClass": "noRecord",
    "ClientMTLSAuthCertFingerprint": "",
    "ClientMTLSAuthStatus": "unknown",
    "ClientRequestBytes": 3805,
    "ClientRequestPath": "String",
    "ClientRequestProtocol": "HTTP/1.1",
    "ClientRequestReferer": "https://www.X.com/",
    "ClientRequestScheme": "https",
    "ClientRequestSource": "String",
    "ClientRequestUserAgent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/984.25 (KHTML, like Gecko) Chrome/98.0.4589.32 Safari/234.54",
    "ClientSSLCipher": "BDEA-BA34ED-FKE",
    "ClientSSLProtocol": "TLSv1.3",
    "ClientSrcPort": 568302,
    "ClientTCPRTTMs": 89,
    "ClientXRequestedWith": "",
    "EdgeCFConnectingO2O": false,
    "EdgeColoCode": "String",
    "EdgeColoID": 45,
    "EdgePathingOp": "wl",
    "EdgePathingSrc": "macro",
    "EdgePathingStatus": "nr",
    "EdgeRateLimitAction": "",
    "EdgeRateLimitID": 0,
    "EdgeRequestHost": "url.com:7634",
    "EdgeResponseBodyBytes": 5537,
    "EdgeResponseCompressionRatio": 1,
    "EdgeResponseContentType": "text/html",
    "EdgeServerIP": "1.1.1.1",
    "EdgeTimeToFirstByteMs": 60193,
    "FirewallMatchesActions": [],
    "FirewallMatchesRuleIDs": [],
    "FirewallMatchesSources": [],
    "OriginDNSResponseTimeMs": 2,
    "OriginIP": "1.1.1.2",
    "OriginRequestHeaderSendDurationMs": 0,
    "OriginResponseBytes": 0,
    "OriginResponseDurationMs": 47911,
    "OriginResponseHTTPExpires": "",
    "OriginResponseHTTPLastModified": "",
    "OriginResponseHeaderReceiveDurationMs": 60026,
    "OriginResponseStatus": 504,
    "OriginResponseTime": 47911000000,
    "OriginSSLProtocol": "TLSv1.2",
    "OriginTCPHandshakeDurationMs": 32,
    "OriginTLSHandshakeDurationMs": 56,
    "ParentRayID": "00",
    "SecurityLevel": "med",
    "SmartRouteColoID": 0,
    "UpperTierColoID": 0,
    "WAFAction": "unknown",
    "WAFFlags": "0",
    "WAFMatchedVar": "",
    "WAFProfile": "unknown",
    "WAFRuleID": "",
    "WAFRuleMessage": "",
    "WorkerCPUTime": 0,
    "WorkerStatus": "unknown",
    "WorkerSubrequest": false,
    "WorkerSubrequestCount": 0,
    "ZoneID": 3601763,
    "ZoneName": "String"
}
CODE
CloudFlare Firewall Example
{
  "Action": "log",
  "ClientASN": 701,
  "ClientASNDescription": "UUNET",
  "ClientCountry": "us",
  "ClientIP": "174.64.104.224",
  "ClientIPClass": "noRecord",
  "ClientRefererHost": "",
  "ClientRefererPath": "",
  "ClientRefererQuery": "",
  "ClientRefererScheme": "",
  "ClientRequestHost": "www.<client>.com",
  "ClientRequestMethod": "POST",
  "ClientRequestPath": "/autodiscover/autodiscover.xml",
  "ClientRequestProtocol": "HTTP/1.1",
  "ClientRequestQuery": "",
  "ClientRequestScheme": "https",
  "ClientRequestUserAgent": "Microsoft Office/16.0 (Windows NT 10.0; Microsoft Outlook 16.0.12345; Pro)",
  "Datetime": "2022-04-11T16:23:45Z",
  "EdgeColoCode": "EWR",
  "EdgeResponseStatus": 404,
  "Kind": "firewall",
  "MatchIndex": 0,
  "Metadata": {
    "filter": "c6d604cb89143be0a43cefa6fa354e8c",
    "type": "customer"
  },
  "OriginResponseStatus": 404,
  "OriginatorRayID": "00",
  "RayID": "6ea82953cc945c81",
  "RuleID": "249936d33e9c4bf6918f2e75f12f3c46",
  "Source": "firewallrules"
}
CODE
CloudFlare DNS Example
{
  "ColoCode": "ATL",
  "EDNSSubnet": "",
  "EDNSSubnetLength": 0,
  "QueryName": "www.<costumer>.com",
  "QueryType": 65535,
  "ResponseCached": false,
  "ResponseCode": 0,
  "SourceIP": "127.0.0.1",
  "Timestamp": "2022-04-11T23:59:50Z"
}
CODE