In this tutorial, we'll learn how to install and use OSSEC HIDS on Ubuntu 24.04 server.
We’ll walk through setting up OSSEC HIDS (Host-Based Intrusion Detection System) on Ubuntu 24.04. OSSEC is a powerful open-source tool that helps monitor system activity, detect unauthorized access, and alert us to suspicious behavior — making it an essential part of any server’s security stack.
What Is OSSEC and Why Do We Use It?
OSSEC HIDS stands for Open Source Security Event Correlator. It analyzes system logs, detects rootkits, file integrity issues, and unauthorized changes to configurations.
Key reasons we use OSSEC HIDS:
- It performs real-time log analysis
- Supports file integrity monitoring (FIM)
- Detects rootkits and unauthorized users
- Sends active response alerts
- It's open-source, lightweight, and supports centralized logging for multiple servers
Prerequisites
Before we start, let’s ensure we have:
- A Ubuntu 24.04 dedicated server or KVM VPS.
- SSH access to the server
- Root or sudo privileges
How to Install and Use OSSEC HIDS on Ubuntu 24.04
Step 1: Update Our Ubuntu 24.04 System
Before we install anything, we make sure our packages are up-to-date:
sudo apt update && sudo apt upgrade -y
This ensures our system has the latest security patches and package versions, reducing potential vulnerabilities.
Step 2: Install Required Dependencies
We install the packages needed to compile OSSEC:
sudo apt update && sudo apt install -y \
build-essential \
gcc \
make \
unzip \
wget \
curl \
zlib1g-dev \
libpcre2-dev \
libevent-dev \
libssl-dev \
libsystemd-dev \
policycoreutils \
libcap-ng-dev
These dependencies allow OSSEC to build from source and support SSL communication securely.
Step 3: Download and Extract OSSEC
We fetch the latest OSSEC server version from the official repository :
cd /tmp
wget https://github.com/ossec/ossec-hids/archive/3.8.0.tar.gz -O ossec.tar.gz
tar -xvzf ossec.tar.gz
cd ossec-hids-3.8.0
Replace 3.8.0 with the latest version if a newer one is available.
Step 4: Start OSSEC Installation
We launch the OSSEC installation script:
sudo ./install.sh
During the installation, we are prompted to answer a few questions. Here's what to choose for a standard host-based installation:
- Language: en
- Installation type: server (or local if monitoring a single system only)
- Enable Email Alerts?: yes (optional but recommended)
- Email to send alerts to: our email address
- SMTP server IP or hostname: use default or custom if available
- Enable integrity checking: yes
- Enable rootkit detection: yes
- Enable active response: yes
- Enable system inventory: yes
Once the installation finishes, OSSEC will be installed under /var/ossec.
Step 5: Start the OSSEC Service
To start OSSEC:
sudo /var/ossec/bin/ossec-control start
To check its status:
sudo /var/ossec/bin/ossec-control status
This confirms the OSSEC daemon is running in the background and actively monitoring.
Step 6: Configure Email Alerts (Optional but Important)
If we opted for email alerts during setup, we should verify the configuration in:
sudo nano /var/ossec/etc/ossec.conf
Look for the <global> and <email_notification> sections and update the <email_to> and <smtp_server> values. This ensures alerts reach us reliably.
If our SMTP server requires authentication, we may need to configure relay tools like msmtp or ssmtp.
Step 7: Monitor Logs and Alerts
OSSEC creates a rich set of logs and alerts. We can review them using:
sudo tail -f /var/ossec/logs/alerts/alerts.log
This is where OSSEC writes all high-level alerts, such as suspicious logins, file changes, and potential threats.
Step 8: Add Custom File Integrity Monitoring
We can monitor specific directories or config files (e.g., /etc, /var/www) by editing:
sudo nano /var/ossec/etc/ossec.conf
Find the <syscheck> section and add:
<directory check_all="yes" realtime="yes">/etc</directory>
<directory check_all="yes" realtime="yes">/var/www</directory>
Then restart OSSEC to apply:
sudo /var/ossec/bin/ossec-control restart
Step 9: Enable Active Response
Active Response helps us block suspicious IPs automatically. OSSEC uses scripts like firewalldrop or host-deny.
To customize active response actions:
sudo nano /var/ossec/etc/ossec.conf
In the <active-response> section, we can enable:
<active-response>
<command>firewalldrop</command>
<location>local</location>
<level>10</level>
</active-response>
Level 10+ will trigger automatic blocking. Always test this carefully to avoid accidental lockouts.
Step 10: View and Manage Rules
OSSEC rules determine how events are classified. To explore them:
cd /var/ossec/rules
Custom rules can be created in:
sudo nano /var/ossec/etc/rules/local_rules.xml
Example: alert on repeated failed logins:
<rule id="100010" level="10">
<if_sid>5710</if_sid>
<match>sshd</match>
<description>Possible brute force attack on SSH</description>
</rule>
Step 11: Autostart OSSEC on Boot
Create a systemd service to ensure OSSEC runs on startup:
sudo nano /etc/systemd/system/ossec.service
Paste the following:
[Unit]
Description=OSSEC Host-based Intrusion Detection System
After=network.target
[Service]
Type=forking
ExecStart=/var/ossec/bin/ossec-control start
ExecStop=/var/ossec/bin/ossec-control stop
RemainAfterExit=yes
[Install]
WantedBy=multi-user.target
Enable and start the service:
sudo systemctl daemon-reexec
sudo systemctl enable ossec
sudo systemctl start ossec
Final Notes
Security Tips:
- Regularly update OSSEC rules and system packages.
- Consider integrating with Wazuh if we want a modern UI and advanced log correlation.
- Always monitor /var/ossec/logs/alerts/alerts.log for real-time issues.
Use Cases:
- Protect public cloud servers (AWS, Azure)
- Detect malicious user behavior
- Audit file changes in critical web apps or Linux config directories
Conclusion
By installing OSSEC HIDS on Ubuntu 24.04, we add a crucial layer of protection to our Linux servers. Unlike network-based firewalls, OSSEC gives us deep visibility into system-level activities, ensuring we detect breaches early and respond quickly.
Let’s remember — host-based intrusion detection is not optional anymore. It’s a foundational security layer we should always deploy on any server we manage.
