SANS Cloud Curriculum - 2024 Workshop Series

Aviata Solo Flight Challenge - Chapter 9

Requirements

• AWS Account
• Web Browser

Setup

This workshop requires attendees to deploy the SANS Cloud Security Flight Simulator to their personal AWS account. Ensure you have completed each of the following requirements before starting the workshop:

1. If you do not have one, start by creating an AWS account. Then, sign in to the account with your email address and password.

Launch the SANS Cloud Security Flight Simulator in your AWS account using CloudShell.

Running Costs

Running the Cloud Security Flight Simulator in your AWS Account will cost about $5.00 USD per day.

Make sure to follow the Cleanup instructions to avoid ongoing expenses.

Launch your Flight Simulator

1. Log in to the AWS Console

   

2. Change to the us-east-2 region.

   

3. Open a CloudShell terminal in a new tab.

   

4. Run the following commands to create a new directory called ace135 and change into the directory:

   mkdir ~/ace135
   cd ~/ace135
   

   

5. In the Terminal, run the following commands from inside the ace135 directory to launch the SANS Cloud Security Simulator.

   wget https://aviata.cloud/simulator/ch09/install.sh
   /bin/bash ./install.sh huntWithELK
   

6. The install.sh command will take a few minutes to create a new VPC and launch the Cloud Security Flight Simulator instance. Once it is finished, you will see the following:

   • The environment_summary output contains the information you need to connect to the SANS Cloud Security Flight Simulator.

   Apply complete! Resources: 14 added, 0 changed, 0 destroyed.
   
   Outputs:
   
   environment_summary = <<EOT
     Latest AMI:  ami-0c949285f6116b47d
     Running AMI: ami-0c949285f6116b47d
     Public IP:   18.226.169.134
   
     Local IP:    13.59.111.234
     Allow CIDR:  13.59.111.234/32
   
     Kibana URL:  http://18.226.169.134:5601/
   
   
   EOT
   

Instructions

Connect to SOF-ELK by copying the Kibana URL value from Cloud Shell into a new browser tab or window.

Startup Delay

The SOF-ELK services can take 5-10 minutes to start up fully. Refresh the page if you get a connection error.

The rest of this lab will be centered around log investigation within SOF-ELK. We've preloaded the Google Cloud logs needed for the investigation into the SOF-ELK instance. To access these logs, go to Kibana and perform the following steps:

1. From the expandable menu on the left, go to Discover.
2. On the top left of the Discover view, set your index to gcp-*
   
3. Set the time frame by copying and pasting the following Absolute values, as shown in the screenshot below:
   • Start date: 2024-12-01 22:45:00.000 +00:00
   • End date: 2024-12-02 00:52:00.000 +00:00

Now we can begin our investigation!

Background

Maligator Labs has detected suspicious API activity in their Google Cloud development environment. Initial alerts show failed attempts to create API keys from a Kubernetes service account, suggesting potential container escape or privilege escalation attempts. Further investigation reveals that the user chappie@maligatorlabs.com deployed a malicious container from Docker Hub disguised as aviata-cloud-updater, which was used to automate lateral movement within the environment.

Phase 1: Identify Failed API Key Creation Attempts

Since the initial alerts involved failed attempts to create API keys, let's start by trying to understand the context of these API key creations in order to uncover potential indicators of compromise that we can pivot from.

First, we should identify the account or other attributes associated with API key creation attempts.

What attributes in the logs might be able to be used as pivot points to identify other activity tied to this threat actor?

Tip

Search for

method_name: "google.api.apikeys.v2.ApiKeys.CreateKey"

Expand one of the events for more details.

Answer

Unfortunately, while the raw logs would provide the service account that created the keys, the SOF-ELK parser did not extract that information. We do, however, have an IP address which may be unique enough to identify related activity. Let's use this as our pivot point in the next investigation phase!

source_ip: 34.172.104.252

What is notable about the timestamps combined with the user agent that might tell you something about the threat actor's methods?

Tip

Review the frequency between the timestamps (disregarding duplicates). Determine what the user agent indicates.

Answer

The reoccurring attempts every couple of minutes, combined with a user agent indicating the use of Python indicates that it's likely an automated script performing this activity, rather than an attacker executing the commands directly.

Phase 2: Analyze Network Activity Correlation

Given the IP address uncovered in the previous investigation phase, determine if the any additional malicious actions occurred from the same source. Make sure you clear out existing filters before beginning new searches!

What other method names are associated with this IP address?

Tip

Search for

source_ip: 34.172.104.252

Add method_name as a selected field.

Answer

Additional method names include:

• storage.buckets.list
• Docker-ServeBlob
• Docker-HeadManifest
• Docker-GetManifest

The storage bucket activity seems to be an attempt at enumeration but the ERROR severity tells us these attempts failed. More interesting is the Docker-* activities, indicating container interactions are involved.

What user is associated with this activity and what does the account name indicate?

Tip

Look at the username field.

Answer

The user performing these actions is 1035231799844-compute@developer.gserviceaccount.com, which is the username format used for default service accounts tied to Compute VMs.

What is the repository that the Docker artifacts are tied to?

Tip

Look at the authorization_info fields.

Answer

The authorization_info.permission field indicates that these are artifacts being downloaded from an Artifact Registry repository. The authorization_info.resource points us to the specific repository: projects/aviata-cloud-dev-7fb7bf6d/locations/us-central1/repositories/aviata-cloud-updater. Let's use this information to look at the container and registry activities in the next phase of our investigation.

Phase 3: Container Activity Analysis

Using the information about the aviata-cloud-updater repository that is being interacted with, let's scope out additional interactions with that repository. First, let's figure out who created the repository. Make sure you clear out existing filters before beginning new searches!

What user is associated with the repository creation?

Tip

Search for

method_name: "google.devtools.artifactregistry.v1.ArtifactRegistry.CreateRepository"

Expand one of the events for more details.

Answer

The user performing these actions is chappie@maligatorlabs.com.

What additional Docker operations (outside the ones noted in the previous investigation phase) are associated with this repository?

Tip

Search for

authorization_info.resource: "projects/aviata-cloud-dev-7fb7bf6d/locations/us-central1/repositories/aviata-cloud-updater"

Add method_name as a selected field.

Answer

Additional Docker options include:
- Docker-StartUpload
- Docker-FinishUpload

These are significant as they are DATA_WRITE operations vs. the previously observed DATA_READ operations. This means the user who created the repository subsequently uploaded Docker artifacts.

Phase 4: Investigate Kubernetes Infrastructure

One common way to interact with containers on Google Cloud infrastructure is via Google Kubernetes Engine (GKE). Let's explore this aspect of the incident more.

What container runtime was used for the Docker operations?

Tip

Examine the useragent field.

Answer

The useragent is containerd/1.7.22. containerd is the container runtime used on GKE nodes.

What is the name of the Kubernetes deployment created by the threat actor, and what are the operations performed in Kubernetes?

Tip

Search for

service_name: k8s.io

Add method_name as a selected field.

To identify the deployment name, look for the deployment creation event.

Answer

The following are some of the Kubernetes events observed, specifically creation events: - io.k8s.apps.v1.deployments.create - io.k8s.apps.v1.replicasets.create - io.k8s.core.v1.pods.create

Additional events for updating, patching, and listing are also observed.

By looking at the event for io.k8s.apps.v1.deployments.create, we can see that the resource name is apps/v1/namespaces/default/deployments/aviata-updater. Thus, the deployment is named aviata-updater.

Phase 5: Attack Pattern Analysis

Investigation analysis often requires moving backwards and forwards in time as you find different pivot points. To wrap up, provide a summary of the timeline of events uncovered above to give a full picture of the attacker's actions.

In what order did the above activities occur?

Tip

Review the timestamps for each of the activities identified.

Answer

1. The Artifact Registry repository is created.
2. Docker artifacts are uploaded to the repository.
3. Docker artifacts are downloaded from the repository.
4. Kubernetes infrastructure is deployed.
5. Failed attempts to create API keys occur.
6. Failed attempts to enumerate storage buckets occur.

Conclusion

Based on our findings, we observed the following key takeaways:

• Supply chain attacks can exploit weak configurations in cloud environments.
• Default Kubernetes service accounts pose significant security risks if not properly configured.
• Container registry operations can reveal attacker preparation and payload deployment.
• Understanding user agent strings helps identify attack tools and methods.
• Correlating timestamps across different Google Cloud services is crucial for tracing attack progression.

This lab provided an introduction to several concepts when it comes to cloud security and investigations, such as:

• Methodical analysis of Google Cloud audit logs across multiple services.
• Correlation between container, IAM, and Kubernetes activities.
• The importance of proper service account configuration in GKE.
• Real-world attack patterns in cloud-native environments.
• The value of timeline analysis in tracing lateral movement.

Clean-Up

• Return to the CloudShell environment used previously, or open a terminal in a new tab.

  

• Run the following commands to delete the resources created by Terraform:

  cd ~/ace135
  terraform destroy -auto-approve
  cd
  rm -rf ~/ace135