Lab 1 - Explore the Environment

Welcome to the first lab. This lab will help you become more familiar with the environment.

Prerequisite

You should have the following items,

• Laptop (PC or Mac)
• Firefox browser preferred Click to download link or
• Chrome Web browser Click to download link and Postman click to download link

Time allocation

25 minutes

Access to the environment

Please launch your browser (Firefox or Chrome) to visit the URL https://www.sansapi.com/

You are greeted with a quick flash of a page and redirected to the authentication page on auth.sansapi.com, see the screenshot below.

Since this is your first time visiting this site, please click "Register" to register as a user.

Data Exposure

The information you enter into this system will be exposed to others. Use dummy data only.

Enter your user details, then click on "Register". Remember that this is a lab environment, and your information WILL BE exposed. Using fictional information and dummy records is required unless you want your real personal information exposed.

Data Exposure

The information you enter into this system will be exposed to others. Use dummy data only.

Once the user is created, you can access the website’s main page (see screenshot below). This is an animal shelter website that uses multiple APIs as information sources.

Browser to backend interaction

Let's dive in and observe the frontend client (browser) interacting with our backend APIs.

We will explore the API calls using the browser (Chrome or Firefox).

Step 1. To launch the Developer Tools, use Ctrl + Shift + I or F12 on Windows and Linux. On macOS, use Cmd + Opt + I or F12.

Step 2. Select the "Network" tab at the top of the Developer Tool (Chrome and Firefox use a slightly different look, but both are called “Network”).

Firefox view Chrome view below

Step 3. Once the Network tab is selected, hit the browser's Refresh button to reload the page

This will populate the tab in Developer Tools with all the HTTP requests that the browser has made. Please review the requests by scrolling the tab up and down.

Firefox view Chrome view below

Select one of the API calls to understand these calls better. You will need to scroll up a few lines from the bottom to find the call by the browser to the “animal” endpoint.

Click on the request to “/animal” to review it. Please review the call using the “GET” method. Avoid the "OPTIONS" request a few lines above the "GET" request. Chrome browser view (Firefox browser looks similar with a few visual differences)

From the Developer Tools, we can observe that we started the website visit with https://www.sansapi.com/. This is the URL displayed in your URL bar. Then, some code loaded with the web page triggered the browser to retrieve data from the animal endpoint location of https://api.sansapi.com/api/animal.

Click on the "Response" tab within the Developer Tools. Firefox and Chrome work similarly here, with minor visual differences.

OPTIONS request

Web browsers implement the Same Origin Policy (SOP) to restrict how a document or script loaded from one origin can interact with a resource from another origin. It is designed to prevent malicious web pages from accessing sensitive data loaded from other sites. This feature protects users from potential security vulnerabilities and attacks. In our case, https://www.sansapi.com/ and https://api.sansapi.com/ are different origins.

As our web content from https://www.sansapi.com/ tells the browser to visit https://api.sansapi.com, we want to make sure the site on port 8442 is willing to allow visitors originating from https://www.sansapi.com/ to interact with its content.

We are even sending an Authorization header (we will get to this in a moment). We must “ask” the web server listening on api.sansapi.com for explicit consent. The way to do it is to send an OPTIONS request to the server, asking for permission. To jog your memory, other request types are GET, POST, DELETE...

If the website listening on port 8442 includes the appropriate header in its response, permission is granted to the browser, and the request can proceed. See the example response headers below.

Access-Control-Allow-Credentials: true
Access-Control-Allow-Headers: Authorization,Content-Type
Access-Control-Allow-Methods: GET,HEAD,PUT,PATCH,POST,DELETE,OPTIONS,TRACE,CONNECT
Access-Control-Allow-Origin: https://www.sansapi.com

The API environment

The API environment is

A few key attributes of the environment,

• All Internet-facing components share the web address www.sansapi.com. The components listen on different ports but use the same FQDN (fully qualified domain name).

• Keycloak, our authentication server, supports modern authentication methods like OAuth with JSON Web Tokens (JWTs). When an unauthenticated user tries to access a protected resource, they are redirected to Keycloak for authentication. Keycloak generates a JWT upon successful authentication, which it returns to the user's client application.

• The client includes the JWT in the request headers to access the backend APIs. These requests are routed through Kong, an API gateway that verifies the validity of the JWT using Keycloak's public key. If the token is valid, Kong extracts the user's identity and permissions from the JWT and forwards the request to the appropriate backend API, which can then make authorization decisions based on the user's information. This architecture separates authentication and authorization logic from the backend APIs' core functionality.

• Kong, our API gateway, maintains the separation between external and internal services. It provides essential functions like traffic filtering controls, token validation, HTTP headers manipulation, logging, and other security validations. By managing these aspects, Kong is a secure and efficient entry point for all API requests, ensuring that only authorized and valid traffic reaches the internal services.

• This ecosystem's "static" component serves as the static content web server, hosting the static web pages and images used in the website. In modern cloud-native applications, cloud storage services like Amazon S3 (Simple Storage Service) or Azure Blob Storage can efficiently handle this functionality.

• The 'Bad Gateway' is an intentionally vulnerable API environment designed for learning about common API security flaws through hands-on experience. It serves as a practice target for various attack techniques, safely helping users—such as cybersecurity students and software developers—understand the consequences of insecure API design and implementation. This environment aligns with the 'attack and defend' theme, enabling participants to test their exploiting skills at API vulnerabilities.

• We have also added a bypass conduit on bypass.sansapi.com, allowing access directly to the backend API. This exposure enables unfiltered access to the API endpoints, allowing us to study the APIs directly. It also simulates an application environment that inadvertently provides access to the backend.

The list above is a quick index of the Internet-facing endpoints. The links below are not fully functional. Some APIs require specific paths to trigger certain actions. Clicking the links submits a GET request to the resource. Other HTTP actions might be necessary to trigger specific actions.

Components	Address / Port
Keycloak	https://auth.sansapi.com/
Kong	https://api.sansapi.com/
static	https://www.sansapi.com/
Bad gateway	https://badgw.sansapi.com/
bypass	https://bypass.sansapi.com/

What is REST APIs

A REST API, which stands for Representational State Transfer Application Programming Interface is an architecture style that allows software components to interact with each other. In modern practices, REST is the building foundation for most of the modern applications we use on regular basis.

We will cover some of the highlights here which will help you understand our API environment a bit better,

• In REST, everything is a resource, which can be a document, an image, a collection of other resources, etc. Each resource is identified by a stable URI (Uniform Resource Identifier).

• REST APIs use standard HTTP methods to perform operations on resources:

  • GET to retrieve a resource.
  • POST to create a new resource.
  • PUT to update an existing resource.
  • DELETE to remove a resource.
• Structured response formats - Responses from a REST API typically use standard formats like JSON (JavaScript Object Notation) or XML (eXtensible Markup Language) to transmit data.
• Stateless communication: Each request from a client to a server must contain all the necessary information to understand and complete the request. The server does not store any client context between requests.

REST Tutorial

To explore the REST APIs using the browser (Chrome or Firefox)

Step 1. In the browser, visit https://bypass.sansapi.com/fruits/

Step 2. Make sure to launch the Developer Tools. To launch the Developer Tools, use Ctrl + Shift + I or F12 on Windows and Linux. On macOS, use Cmd + Opt + I or F12.

Step 3. Once the "Network" tab is selected, hit the "Refresh" button for the page.

This refresh will populate the tab in Developer Tools with all the HTTP requests that the browser has made. Please review the request to /fruits. Details of the page in Firefox are shown below. The Chrome menu can be visually different, but all components are in the same place.

Dynamic database

The dataset here is inside a dynamic database and there are multiple users concurrently using the system. You will likely need to adapt your request to the data presented to you. Also, please respect others who are trying to learn as well, do not delete/alter data records you did not create. Thank you.

Step 4. Inspect the request to /fruits in detail - including the request header and response. What we observe here is that the GET method to /fruits yield all the data records in the database completely. This matches up with the expected behavior that GET method to the /fruits resource is retrieving the data.

Step 5. At the same tab, alter the URL to https://bypass.sansapi.com/fruits/1. The number 1 in this request may not exist by the time you generate your request, you may have to select an ID that exists from the last step. Please note that you are again submitting a GET request this time. Observe the request and response. In this case, we are requesting a specific fruit by referring to it's ID number.

Step 6. Next, we want to explore POST which will create a new record. Firefox has the ability to send customized request to the website/API directly from the Developer Tools. Chrome does not have this ability which is why we need Postman to help us craft the exact request we need.

Firefox

To enable the Edit and Resend feature in Firefox Developer Tools, follow these steps:

• First, we need to configure an option in the Firefox browser, type in about:config into the browser URL bar. It may throw up a warning about being cautious on changing the setting, proceed with the configuration. The setting is browser.opaqueResponseBlocking.javascriptValidator and please set that to false (double click on the setting toggles it). See screenshot for details
• Select the recent API request that returned all the fruits. (https://bypass.sansapi.com/fruits)
• In the Network tab, hover over the specific request.
• Right-click on the request to open a new menu.
• From the pop-up menu, choose the Edit and Resend option.

See screenshot for details.

This will pop up a new tab on the LEFT side of the Developer Tools. The tab allows you to edit the details of the HTTP request before submitting it to the server/API. Please edit the request type to POST, see the below screenshot

Then, scroll down to the bottom of the headers where Firefox allows you to enter a new item, you will see the name and value in gray color. Edit the name to Content-Type and value to application/json (remember to copy and paste these) At the bottom where it states Body, type in

{"fruit": "Mango", "taste": "Sweet and tangy"}

Please alter this to a unique fruit of your choice (replace "Mango" with another fruit), as the database will not allow duplicate records. Here is a list of fruits from Wikipedia. https://simple.wikipedia.org/wiki/List_of_fruits Once you made the changes, please click send at the bottom. You can then observe the response tab to ensure that the record is accepted.

Postman

In Chrome, we will need to copy the request and import it to the Postman tool. Please follow these steps:

• Select the recent API request that returned all the fruits. (https://bypass.sansapi.com/fruits)
• In the Network tab, hover over the specific request.
• Right-click on the request to open a new menu.
• From the pop-up menu, choose the copy option which will expand the menu to the right.
• From the expanded menu from copy, select Copy as cURL

Once the request is copied, you can launch Postman. You do not need to login to Postman for the functions we use in the exercises. This means that lightweight mode of Postman works perfectly for us.

In Postman: - Select the New button then select HTTP from the popup. - Select the URL bar to paste in the cURL details

-Please edit the request type to POST. Then, scroll down to the bottom of the headers where Postman allows you to enter a new item. Edit the name to Content-Type and value to application/json (remember to copy and paste these)

• Change the tab to body (next to the headers)
• Select raw as data type and the select JSON (see screenshot below)

• At the blank area for Body, type in

  {"fruit": "Mango", "taste": "Sweet and tangy"}

  Please alter this to a unique fruit of your choice (replace "Mango" with another fruit), as the database will not allow duplicate records. Here is a list of fruits from Wikipedia. https://simple.wikipedia.org/wiki/List_of_fruits

Once you made the changes, please click send at the bottom. You can then observe the response tab (below) to ensure that the record is accepted.

Step 7. We have observed how the POST can work to add a new record to the API set of resources. We can also validate that the record you entered have been incorporated into the API by visiting https://bypass.sansapi.com/fruits/ again to ensure your record is there. You can also subsequently follow up with the https://bypass.sansapi.com/fruits/ ID of your fruit's ID.

Step 8. Updates are done over PUT. Please craft a PUT request to update the fruit you entered.

Answer

For the PUT request to work, you have to first retrieve the ID of the fruit. A single GET request to https://bypass.sansapi.com/fruits/ can yield that information. The request type needs to be switched to PUT and the Content-Type needs to be set to application/json. The URL points to the fruit that will be updated, in this example, we will point to fruit ID 4. https://bypass.sansapi.com/fruits/4 The content is {"fruit": "Mango", "taste": "sour"} See screenshot (Postman) for example

Step 9. The "DELETE" method would work the same. Please craft a delete request against the fruit that you created.