When you are monitoring your AWS infrastructure for security alerts, not all security signals can be remediated automatically. Large or complex environments can generate multiple security flags (repeatedly) that need to be manually investigated in detail. This requires access to sometimes complex metadata to thoroughly investigate which may place a strain on your in-house security skills or may even require you to find the budget for a dedicated security expert.
There are also increased storage requirements and costs when you start collecting security log data and costs associated with processing this data to identify potential security issues.
What is Amazon Detective?
Amazon Detective is a service that helps you analyze and investigate the root cause of security findings or suspicious activity. Using data from AWS Cloudtrail logs, VPC flow logs and AWS GuardDuty data, Amazon Detective will collate, filter the collected security data and using AWS Security Hub, GuardDuty, integrated partner security products, machine learning and statistical analysis investigates the security findings to get to the root cause of the problem or suspicious activity.
Amazon Detective ingests security data automatically, so there are no requirements to organize data, set up and tune queries or script any algorithms to monitor for security events. There are also no upfront cost for collecting the data and there are no additional software subscriptions or software to deploy. You simply pay for the security events you analyze using Detective.
How does Amazon Detective work?
Detective extracts time-based events like API calls, log in attempts and network traffic on data sources to which it applies machine-learning and visualization to create a view of normal resource interactions and behaviours over time.
Detective builds a baseline so it can determine whether things like API calls are typical for the role making the call, or if traffic spikes from a particular instance are out of character.
Guard Duty, Amazon Inspector and Amazon Security Hub are all services providing security alerts and monitoring. Detective enhances these services and also differs in the following ways.
Guard Duty manages threat detection, provides continuous monitoring for unusual or malicious behaviour and protects AWS accounts from things like port scanning, penetration testing and even bitcoin mining. Guard duty activity and log data can be ingested into other security tools and provides a platform for centralized monitoring of AWS accounts at scale.
Amazon Inspector automates network and host-based security analysis and enhances the overall security of AWS by providing application level security assessments.
AWS Security Hub aggregates security data from AWS and external sources to help identify trends and establish a more advanced security posture enabling you to react to a wider range of security threats.
Amazon Detective enables you to investigate security events or potential threats from a wide source of information. Detective collects and integrates terabytes of log data and and transforms it for analysis while providing visualizations to help spot anomalies. This allows you to conduct investigations faster and more effectively.
How to use Amazon Detective.
The first use case for Detective is for Alert Analysis. Detective enables you to triage security alerts and findings as the arise which may lead to averting unnecessary security escalations. Detective can provide some context around reported issues like how normal the traffic looks, what data was accessed, what network activity was like either side of the incident, was a failed API call typical or abnormal. This information helps you quickly determine whether the security incident was a false-positive or whether it was real and you need to initiate mitigation actions.
Incident investigation is another main function of Amazon Detective. Once an incident is identified, Detective can provide some context around the network and resource activity like what API calls were made and what IP address did they originate from. What other network resources communicated with the same IP address which helps formulate a picture of the scope of the incident and the potential impact on your systems and data.
With a potential security incident identified, you can use Amazon Detective to target the threat by hunting out all the activity related to the offending IP address. What instances has the target communicated with over the past year, what other API calls have they made. The historical view and visualization assists with surfacing potential threats.
Amazon Detective key concepts.
Behaviour Graph
A Detective behaviour graph is a linked set of data generated from one or more AWS accounts. Data is ingested and Detective uses machine learning to establish normal behavior of traffic, roles and resources so that anomalies can be identified.
Investigation
Detective enables you to dig into suspicious activity, establish the underlying source or cause of the security finding and then helps determine how to move forward. If the incident proves not to be an issue, like a false-positive Guard Duty alert, you can archive the issue using Detective.
Account Structure
Detective provides the ability to have a management account invite members. The management account owns and uses the behaviour graph to ingest source data and conduct investigations. Members accounts can then be invited to contribute data to the graph.
Profile Panel Visualizations
Profile panels provide a visual representation of data and supporting guidance for findings and are designed to provide an analyst with answers to a specific question around a finding. An individual panel might details things like overall VPC flow volume at the time of the incident.
Scope Time
The scope time is a time window surrounding the security incident, typically starting at the time the suspicious activity was observed and ending when it ceased. The times are editable and govern the data that is displayed on the profile panel visualizations.
What does it cost.
At the time of writing, Detective charges for data ingested into the behaviour graph from CloudTrail, VPC Flow Logs and GuardDuty.
First 1000 GB $2.00/GB
Next 4000 GB $1.00/GB
Next 5000 GB $0.50/GB
Over 10k GB $0.25/GB
There is a fully featured Amazon Detective 30 Day Free trial available.
What do you need to start using Amazon Detective?
An AWS Account. This should be fairly obvious, since Detective monitors your AWS resources.
Amazon Guard Duty. You will need a GuardDuty service that has been running for more that 48 hours to enable Detective. This is required for Detective to determine the volume of log data it will need to ingest.
Permissions Policy - If you are not an administrator, you will need to attach a permissions policy to your IAM principle before you can enable detective. You can also use an existing IAM user or role, or create a specific user/role with the appropriate permissions for initiating Amazon Detective.
You can enable Detective from the Detective Console, the Detective API or from the AWS CLI
What happens next?
Once enabled, Detective runs in ‘training mode’. This allows Detective to establish baselines and normal operational behaviour patterns as a basis for your behaviour graphs.
To enable Detective, you will already have GuardDuty running. You should sync the management account of GuardDuty with Detective.
The default frequency for Cloudwatch notifications within the GuardDuty detectors is 6 hours. This means Detective may not receive alerts for a potential incident for many hours. AWS advise that the Cloudwatch notification frequency should be reduced to 15 minutes when using Amazon Detective. This does not increase the pricing of GuardDuty.
Ensure you enable Detective separately in each region that you have workloads. You can use cloudformation for detective and detective multi-account scripts for this.
Ensure your IAM policy permissions include the permissions required for detective like “detective:Get*” “detective:CreateGraph” etc
See:
https://docs.aws.amazon.com/detective/latest/adminguide/security_iam_id-based-policy-examples.html
Synchronize the management account of Security Hub with Detective.
https://docs.aws.amazon.com/detective/latest/adminguide/detective-prerequisites.html#recommended-service-alignment
Detective is a fairly advanced solution for detecting and assessing suspicious activity and events as they occur or shortly after. One of the diagrams built when you connect your AWS accounts to Hava.io is the Security View diagram.
This interactive diagram will document all the security groups defined in your AWS account and visualizes the open ports detailing the ingress and egress points in the network.
Selecting a security group on the diagram will change the right hand security pane which will display all the settings, metadata and connected resources related to the security group.
This diagram allows your security team to spot vulnerabilities and resolve them before any real traffic is detected by Amazon Detective and the other services detailed above.
To try out the security view, or our AWS Network Diagram Tool, you can trial Hava for free here:
