Log Management: Basics

by
Sharing is caring

Views: 12

Introduction to Log Management

Logs are a record of events within a system. These records provide a detailed account of what a system has been doing, capturing a wide range of events such as user logins, file accesses, system errors, network connections, and changes to data or system configurations.

While the specific details may differ based on the type of log, a log entry usually includes the following information:

  • A timestamp of when an event was logged
  • The name of the system or application that generated the log entry
  • The type of event that occurred
  • Additional details about the event, such as the user who initiated the event or the device’s IP address that generated the event

This information is typically stored in a log file, which contains aggregated entries of what occurred at any given time on a system.

Source: TryHackMe

 Logs can answer critical questions about an event, such as:

  • What happened?
  • When did it happen?
  • Where did it happen?
  • Who is responsible?
  • Were their actions successful?
  • What was the result of their action?

Log Investigation Basics

What happened?An adversary was confirmed to have accessed SwiftSpend Financial’s GitLab instance.
When did it happen?Access started at 22:10 on Wednesday, September 8th, 2023.
Where did it happen?The event originated from a device with an IP address of 10.10.133.168 within the VPN Users’ segment (10.10.133.0/24).
Who is responsible?Upon examining the network logs, it was observed that the device, identified by the User-Agent “Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/115.0”, was allocated the IP address 10.10.133.168.
Were they successful?Yes, since an API Key was found to be publicly exposed on the GitLab instance. Moreover, the web proxy logs confirm that the adversary device reached gitlab.swiftspend.finance and maintained access through their uploaded web shell.
What is the result of their action?The adversary achieved remote code execution on gitlab.swiftspend.finance and performed post-exploitation activities.

Log Types

Most common log types:

  • Application Logs: Messages about specific applications, including status, errors, warnings, etc.
  • Audit Logs: Activities related to operational procedures crucial for regulatory compliance.
  • Security Logs: Security events such as logins, permissions changes, firewall activity, etc.
  • Server Logs: Various logs a server generates, including system, event, error, and access logs.
  • System Logs: Kernel activities, system errors, boot sequences, and hardware status.
  • Network Logs: Network traffic, connections, and other network-related events.
  • Database Logs: Activities within a database system, such as queries and updates.
  • Web Server Logs: Requests processed by a web server, including URLs, response codes, etc.

Log Formats

Semi-structured Logs

These logs may contain structured and unstructured data, with predictable components accommodating free-form text.

Syslog:

Syslog Message Format
May 31 12:34:56 WEBSRV-02 CRON[2342593]: (root) CMD ([ -x /etc/init.d/anacron ] && if [ ! -d /run/systemd/system ]; then /usr/sbin/invoke-rc.d anacron start >/dev/null; fi)

Windows Event Logs:

Windows Event Log (EVTX) Format
   ProviderName: Microsoft-Windows-Security-SPP

TimeCreated                      Id LevelDisplayName Message
-----------                      -- ---------------- -------
31/05/2023 17:18:24           16384 Information      Successfully scheduled Software Protection service for re-start
31/05/2023 17:17:53           16394 Information      Offline downlevel migration succeeded.

Structured Logs: 

These logs are following a strict and standardized format and are conducive to parsing and analysis. 

Field Delimited Formats: 

Comma-Separated Values (CSV) and Tab-Separated Values (TSV) are formats often used for tabular data.

CSV Format
"time","user","action","status","ip","uri"
"2023-05-31T12:34:56Z","adversary","GET",200,"34.253.159.159","http://gitlab.swiftspend.finance:80/"

Log Standards