Defensive Logging and Incident Triage
Before you start
Who is this for?
For support and security learners who need to investigate alerts carefully.
Prerequisites
- Security Fundamentals
- Basic Windows or Linux administration
What you'll be able to do
- Identify log sources
- Explain central logging and SIEM concepts
- Interpret alerts and baselines
- Preserve evidence and document a timeline
Practical outcome
Triage a signal while recording facts, uncertainty, and escalation needs.
Logs turn system activity into evidence that helps explain what happened, when, and where. Good logging supports troubleshooting, detection, audit, and incident response. Logs are incomplete views: missing events do not prove an action did not occur, and one event rarely proves malicious intent.
Learning objectives
Section titled “Learning objectives”- Identify useful host, identity, application, and network log sources.
- Explain centralized logging, SIEMs, alerts, baselines, and false positives.
- Triage a signal while preserving evidence and recording a timeline.
Sources and useful context
Section titled “Sources and useful context”Collect only what supports an operational or security purpose, protect it from unauthorized access or alteration, and define retention. Common sources include:
- Identity/authentication: successful and failed sign-ins, account changes, MFA outcomes, privilege changes.
- Operating systems: service/process errors, configuration changes, kernel/system events.
- Applications: authentication, authorization decisions, errors, and important business actions. Avoid logging passwords, tokens, or unnecessary personal data.
- Network controls: firewall decisions, DNS queries, VPN sessions, proxy and flow metadata. Encrypted payloads generally are not visible in flow records.
On Linux, journalctl queries systemd journal records; traditional authentication files and their names vary by distribution. On Windows, Event Viewer and PowerShell Get-WinEvent expose Application, Security, System, and additional provider-specific channels. Audit policy determines which Windows Security events are available.
Get-WinEvent -LogName System -MaxEvents 20 | Select-Object TimeCreated, Id, LevelDisplayName, ProviderName, Messagejournalctl --since "1 hour ago" -p warningThese examples are read-only. Check local retention and access controls before exporting records.
Central collection and alerting
Section titled “Central collection and alerting”Centralized logging collects records from multiple systems so an operator can search across a timeline and retain evidence when a host is unavailable. Use protected transport, access controls, reliable time synchronization, and an explicit retention policy. A Security Information and Event Management (SIEM) platform normalizes, searches, correlates, and alerts on event data; it does not automatically make data complete or conclusions correct.
An alert is a lead to investigate, not a verdict. Baselines describe expected activity for a system, account, or time period. A deviation may be benign (for example, a scheduled deployment) or important. Poorly tuned rules create false positives and alert fatigue; overly narrow rules miss events. Tune rules with documented rationale and measure both noise and coverage.
Triage workflow
Section titled “Triage workflow”- Record the signal: alert name, source, observed time and timezone, affected asset/account, and analyst.
- Validate scope: confirm asset ownership and criticality; determine whether the event is expected maintenance or a known test.
- Build a timeline: correlate identity, endpoint, application, and network events using synchronized timestamps. Record gaps and uncertainty.
- Assess impact and urgency: consider access level, exposure, data/system criticality, and whether activity is continuing. Follow the organization’s escalation path.
- Preserve evidence: retain original records and relevant metadata using approved procedures. Record collection time, source, commands, and any transformations. Do not alter a suspected system casually; coordinate containment with incident leadership.
- Document and hand off: distinguish observed facts from hypotheses, include queries and time ranges, and note actions taken.
Incident response also includes preparation, detection and analysis, containment, eradication, recovery, and learning. Real incidents require the organization’s plan and assigned authority; avoid ad hoc destructive remediation that could destroy evidence or disrupt services. See NIST SP 800-61 Rev. 3 for current incident-response recommendations.
Practical exercise
Section titled “Practical exercise”Using a lab or authorized system, find one failed service event and one authentication event. Write a five-entry timeline with timezone, source, and observed fact. Add one alternative benign explanation and identify what evidence would help distinguish it. Do not include secrets or personal data in your notes.
Triage a signal while recording facts, uncertainty, and escalation needs.
Related: Linux systemd and Logs, Windows Fundamentals and Event Logs, Security Fundamentals.