Linux

Overview

This article explains how to ingest your on-premise Linux Logs of various types to Hunters. Integrating Linux logs will allow ingestion and querying of the data, as well as leveraging it for various security use cases.

Supported Data Types

Linux Auditd Logs

Table name: linux_auditd

These logs hold alerts from the Linux Auditing system logs, used to monitor system calls, file accesses and more. These files should be located in the /var/log/audit/ folder on your Linux machines.

Linux SSHD Logs

Table name: linux_sshd_logs

sshd, the Secure Shell Daemon, allows remote access to the system. It’s a silent process that listens to all the authentication and login attempts of Linux. Using sshd logs, you can monitor authorized and unauthorized login attempts on your system, which helps in keeping your system secure. The logs should be located under the /var/log/auth.log location.

Linux Mail Logs

Table name: linux_mail_logs

This data is for mail server logs, handy for postfix, smtpd, or email-related services info running on your server. The logs should be located under the /var/log/maillog location.

Linux IPtables traffic logs

Table name: linux_iptables

Linux iptables is a powerful utility for configuring network packet filtering rules in the Linux kernel. By setting up logging rules within iptables, administrators can monitor and record network traffic for analysis and troubleshooting. These logs can provide detailed information about the source and destination IP addresses, ports, protocol types, and the action taken by the firewall (such as ACCEPT or DROP). To enable logging, the -j LOG target is used in iptables rules, directing packets to the system log, which can be viewed using tools like dmesg or by inspecting log files in /var/log. Properly configuring and managing iptables logging helps enhance network security by providing insights into traffic patterns and potential threats.

Send data to Hunters

Shipping Linux logs into Hunters requires collecting the logs from all relevant locations within your on premise, and then shipping them to an S3 bucket.

1. Use the below information to set up Fluentd config.

   • Source config (receiving the syslog):

   <source>
     @type syslog
     port <port>  # Replace <port> with your specific port if different
     bind 0.0.0.0
     <transport tcp> # Replace "tcp" with "udp" if you are sending the logs in udp
     </transport>
     <parse>
       @type regexp
       expression /^<\d+>(?<message>\S+ \d+ \d{2}:\d{2}:\d{2} \S+ [^\[]+\[\d+\]: .*)$/
       time_format %b %d %H:%M:%S
     </parse>
     tag MY-DATA-TYPE # Replace "MY-DATA-TYPE" to a name that describes the datatype like: audit, ssd, mail
   </source>
   

   • Match config (Forward to the S3): use the same one described here. Make sure to match the MY-DATA-TYPE field with the one used in the source example.

2. Once the export is completed and the logs are collected to S3, follow the steps in this section.

Expected format

Linux Audit Logs

Logs are expected in Key Value format.

type=SYSCALL msg=audit(1364481363.243:24287): arch=c000003e syscall=2 hostname=samplehostname success=no exit=-13 a0=7fffd19c5592 a1=0 a2=7fffd19c4b50 a3=a items=1 ppid=2686 pid=3538 auid=500 uid=500 gid=500 euid=500 suid=500 fsuid=500 egid=500 sgid=500 fsgid=500 tty=pts0 ses=1 comm="cat" exe="/bin/cat" subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 key="sshd_config"

type=USER_AUTH msg=audit(1364475353.159:24270): user pid=3280 uid=500 auid=500 ses

Linux SSHD Logs

Logs are expected in text format.

Jul  4 09:02:28 host123 sshd[1870211]: error: kex_exchange_identification: Connection closed by remote host

Aug  4 11:12:28 host124 sshd[1870311]: Invalid user anonymous from 10.100.100.100 port 12345

Linux Mail Logs

Logs are expected in text format.

Oct 26 16:58:42 mail postfix/smtpd[29761]: connect from unknown[11.11.111.123]

Oct 26 16:58:44 mail postgrey[1111]: action=pass, reason=recipient whitelist, client_name=unknown, client_address=22.22.222.222, sender=test.email@test.co.ca, recipient=info@mydomain.com

Jul 15 17:11:21 abc.foo.com sendmail[33333]: d5674rfytg654: from=<feb(z)foo.com>, size=123, class=0, nrcpts=1, msgid=<123456789012.e5TR56T4E3WY(a)abcd.foo.com>, proto=ESMTP, daemon=ABC, relay=jan(a)abcd.foo.com [333.333.3.3]

Linux IPtables traffic logs

Logs are expected in text format.

<6>Mar 19 00:00:05 fw-1 kernel: [46157107.824667] Shorewall:guest-inet:DROP:IN=eth1.4000 OUT=eth0 MAC=1.2.3.4:99:51:83:cb:08:00 SRC=1.2.3.4 DST=1.2.3.4 LEN=1278 TOS=0x00 PREC=0x00 TTL=63 ID=57513 DF PROTO=UDP SPT=59642 DPT=443 LEN=1258