Collectors

The Collector is a machine on your network running Rapid7 software that is responsible for gathering log information from endpoints and making it available for InsightIDR analysis. You can send your logs to InsightIDR directly or you can use the Collector.

The Collector workflow has two main advantages over sending logs to InsightIDR directly:

  • Normalization - It transforms log data from multiple diverse sources into a common JSON format and extracts standard information like hostnames, timestamps, error levels, etc. Normalization allows you to run more advanced queries on your endpoint logs and enhance your data visualization.
  • User attribution - It correlates endpoint activity to individual users who were logged into applications on that endpoint at that time. Attribution can give you a more complete image of your security posture since user accounts are the most common targets for sophisticated attacks.

If you decide to use the collector, there can be a delay of up to 5 minutes for endpoint information to show up on InsightIDR. You should consider the ‘Add Log’ workflow if real-time visibility of logs is a critical priority.

Requirements

Before you can install a Collector, you should verify that your system meets the following requirements. [The above section says the machine with installed software is a Collector, but this section makes it sound like the software itself is the Collector.

System requirements

You can install a Collector on a network server or virtual machine that meets the following requirements:

  • Operating system - Linux 64-bit or Windows 64-bit
  • Minimum Hardware - 4 GB RAM and 60 GB disk space
  • 2 CPUs recommended

Only one Collector can be installed per machine on your network. Rapid7 strongly recommends that the machine (physical or virtual) is dedicated to running the Collector.

Account requirements

When setting up the Collector, you should be aware that:

  • InsightIDR ingests data from existing sources in your environment. InsightIDR needs administrator access to these sources.
  • While privileged accounts can be difficult to obtain due to internal controls, it is strongly recommended to obtain a Domain Admin service account for easier configuration and more accurate results in InsightIDR.
  • Treat your Collector(s) as you would any other highly valuable asset. Credentials for event sources will be stored on this device.
  • Credentials are not stored in AWS. Raw unnecessary logs are stripped by the Collector in your environment so that sensitive data, such as personally identifiable information, medical records, etc., is not stored by Rapid7. Employee, organization, and asset names are also obfuscated in AWS.

Bandwidth requirements

Minimum network bandwidth - 100 Mbps network (recommended), 1000Mbps (strongly recommended)

What's Next?