System design

RESTful APIs: Building Blocks of Modern Web Applications

12 min read
#System design

What is a REST API?

A RESTful API (Representational State Transfer) is an style for designing web applications. They allow various clients including browser apps to communicate with services via the REST API. It relies on stateless communication, typically over HTTP, and uses standard HTTP methods such as GET, POST, PUT, DELETE, etc., to interact with resources, often represented in formats like JSON or XML. Each resource is identified by a URL, and the API responds with data relevant to the resource or performs the necessary operation on it.

Here’s a simple example of how a RESTful API could be structured in Node.js using Express:

const express = require('express');
const app = express();
app.use(express.json());

let items = [
  { id: 1, name: 'Item 1' },
  { id: 2, name: 'Item 2' },
];

// GET all items
app.get('/api/items', (req, res) => {
  res.json(items);
});

// GET a single item by ID
app.get('/api/items/:id', (req, res) => {
  const item = items.find(i => i.id === parseInt(req.params.id));
  if (!item) return res.status(404).send('Item not found');
  res.json(item);
});

// POST a new item
app.post('/api/items', (req, res) => {
  const newItem = {
    id: items.length + 1,
    name: req.body.name,
  };
  items.push(newItem);
  res.status(201).json(newItem);
});

// PUT (update) an item
app.put('/api/items/:id', (req, res) => {
  const item = items.find(i => i.id === parseInt(req.params.id));
  if (!item) return res.status(404).send('Item not found');

  item.name = req.body.name;
  res.json(item);
});

// DELETE an item
app.delete('/api/items/:id', (req, res) => {
  const itemIndex = items.findIndex(i => i.id === parseInt(req.params.id));
  if (itemIndex === -1) return res.status(404).send('Item not found');

  items.splice(itemIndex, 1);
  res.status(204).send(); // 204 No Content
});

const port = process.env.PORT || 3000;
app.listen(port, () => console.log(`Server running on port ${port}`));

Key Characteristics of a RESTful API

  • Stateless: Each request from the client to the server must contain all the information needed to understand and process the request. The server does not store any client context between requests. This principle makes REST APIs scalable, reliable, and easier to maintain.
  • Client-Server Separation: The API separates the concerns of the client and the server. The client is responsible for the user interface, and the server handles data storage and business logic.
  • Uniform Interface: A REST API exposes resources through a uniform URL structure and uses standard HTTP methods to perform actions on these resources.

Idempotency

Idempotent methods (e.g., PUT, DELETE) should return the same result when repeated, ensuring safe re-execution of requests without side effects. Idempotent operations ensure that if a client sends the same request multiple times (e.g., due to network issues or retries), the server's state will remain unchanged after the first successful request. If the client issues the request again, the outcome will be identical to the first request, preventing unintended consequences like duplicate resource creation.

Example of Idempotent Methods in REST:

  • GET: Fetching data is inherently idempotent. You can issue a GET request as many times as needed, and the result will be the same every time (as long as the resource hasn't changed).
    • Example: GET /api/users/1 will return the same user each time.
  • PUT: Updates a resource with the given data. If the same PUT request is made multiple times, the resource will only be updated once with the same information.
    • Example: PUT /api/users/1 with the same data will overwrite the user’s information, but making the request multiple times doesn’t create new users or alter the user’s state differently after the first update.
  • DELETE: Removing a resource is also idempotent. Once a resource is deleted, further DELETE requests on that same resource will have no additional effect (they may return a 404 Not Found).
    • Example: DELETE /api/users/1 will remove the user, and sending the same request again will still result in the user being gone.

Non-Idempotent Methods:

  • POST: Typically not idempotent, as POST is used to create resources. Multiple POST requests will create multiple instances of the same resource.
    • Example: POST /api/users with the same data will create a new user each time.

Best practices for rest api design

When designing a RESTful API, several key considerations are important to ensure it is scalable, maintainable, secure, and user-friendly. Here are the main design considerations:

Naming of endspoint:

We shouldn't use verbs in our endpoint paths. Instead, we should use the nouns which represent the entity that the endpoint that we're retrieving or manipulating as the pathname.

Resources (data entities) in REST are identified using nouns in the URL, rather than verbs.

  • Good: /api/items/123
  • Bad: /api/getItemById/123
  • Think of resources like collections or individual entities, such as /users, /orders, /products.

This is because our HTTP request method already has the verb. Having verbs in our API endpoint paths isn’t useful and it makes it unnecessarily long since it doesn’t convey any new information. The chosen verbs could vary by the developer’s whim. For instance, some like ‘get’ and some like ‘retrieve’, so it’s just better to let the HTTP GET verb tell us what and endpoint does.

HTTP Methods and Semantics:

HTTP methods (also known as HTTP verbs) define the action that a client wants to perform on a resource in a REST APIs. These methods are used to interact with resources, typically represented by URLs, following specific semantics. Here’s a detailed look at the commonly used HTTP methods and their intended semantics:

  • GET: Retrieve data.
    • Example GET /users – Retrieve all users.
    • Example GET /users/1 – Retrieve a specific user.
  • POST: Create a new resource.
    • Example POST /users – Create a new user.
  • PUT: Update an existing resource (or create if not exists).
    • Example PUT /users/1 – Update a user’s details.
  • PATCH: Partially update an existing resource.
  • DELETE: Remove a resource.
    • Example DELETE /users/1 – Delete a user.

Use standard HTTP status codes to inform clients of the result of their requests

  • 200 OK: Successful request.
  • 201 Created: Resource successfully created.
  • 204 No Content: Success, but no response body (e.g., after DELETE).
  • 400 Bad Request: Invalid client request (e.g., validation error).
  • 401 Unauthorized: Authentication required or failed.
  • 403 Forbidden: Authentication succeeded, but no permission.
  • 404 Not Found: Resource not found.
  • 500 Internal Server Error: Something went wrong on the server.

Pagination, Filtering, and Sorting

For large datasets, We can use pagination and filtering to avoid loading too much data at once. Filtering and pagination both increase performance by reducing the usage of server resources. As more data accumulates in the database, the more important these features become. Below example gives 50 items of page 2.

  • Example: /api/items?page=2&limit=50.

Implement sorting to allow clients to retrieve specific subsets of data. We can also specify the fields to sort by in the query string. Below example fetches sorted items accoring to price in ascending order.

  • Example: /api/items?category=books&sort=price&direction=asc.

Data Formats and Consistency

JSON is the most common format for REST APIs, but some APIs also support XML, or other formats based on the Accept header. To make sure that when our REST API app responds with JSON that clients interpret it as such, we should set Content-Type in the response header to application/json after the request is made. Many server-side app frameworks set the response header automatically. Some HTTP clients look at the Content-Type response header and parse the data according to that format. Maintain consistent structure in API responses. Typically, responses contain metadata like pagination info, status, and data.

The only exception is if we’re trying to send and receive files between client and server. Then we need to handle file responses and send form data from client to server.

We should also make sure that our endpoints return JSON as a response. Many server-side frameworks have this as a built-in feature.

Let’s take a look at an example API that accepts JSON payloads. This example will use the Express back end framework for Node.js. We can use the body-parser middleware to parse the JSON request body, and then we can call the res.json method with the object that we want to return as the JSON response as follows:

Example response structure:

const express = require('express');
const bodyParser = require('body-parser');

const app = express();

app.use(bodyParser.json());

app.post('/', (req, res) => {
  res.json(req.body);
});

app.listen(3000, () => console.log('server started'));

bodyParser.json() parses the JSON request body string into a JavaScript object and then assigns it to the req.body object. Set the Content-Type header in the response to application/json; charset=utf-8 without any changes. The method above applies to most other back end frameworks.

Error Handling and Responses

To eliminate confusion for API users when an error occurs, we should handle errors gracefully and return HTTP response codes that indicate what kind of error occurred. This gives maintainers of the API enough information to understand the problem that’s occurred. We don’t want errors to bring down our system, so we can leave them unhandled, which means that the API consumer has to handle them.

  • Provide meaningful error messages with proper HTTP status codes.

  • Include error codes and descriptions to help developers diagnose issues.

    • Example:
    {
"status": "error",
"message": "Invalid email format",
"code": "INVALID_EMAIL"
}

Common error HTTP status codes include:

  • 400 Bad Request - This means that client-side input fails validation.
  • 401 Unauthorized - This means the user isn't not authorized to access a resource. It usually returns when the user isn't authenticated.
  • 403 Forbidden - This means the user is authenticated, but it's not allowed to access a resource.
  • 404 Not Found - This indicates that a resource is not found.
  • 500 Internal server error - This is a generic server error. It probably shouldn't be thrown explicitly.
  • 502 Bad Gateway - This indicates an invalid response from an upstream server.
  • 503 Service Unavailable - This indicates that something unexpected happened on server side (It can be anything like server overload, some parts of the system failed, etc.).

Authentication and Authorization

  • Authentication: Use methods like OAuth 2.0, API keys, JWT (JSON Web Tokens) for verifying the identity of the client.
  • Authorization: Ensure users can only access resources they have permission to. Role-based access control (RBAC) or scopes (in OAuth 2.0) can help.
  • Use HTTPS to secure communication between the client and server.

Caching

We can add caching to return data from the local memory cache instead of querying the database to get the data every time we want to retrieve some data that users request. The good thing about caching is that users can get data faster. However, the data that users get may be outdated. This may also lead to issues when debugging in production environments when something goes wrong as we keep seeing old data.

There are many kinds of caching solutions like Redis, in-memory caching, and more. We can change the way data is cached as our needs change.

If you are using caching, you should also include Cache-Control(Cache-Control, ETag, Expires) information in your headers. This will help users effectively use your caching system.

  • Example:
    • Cache-Control: max-age=3600, public – Cache for one hour.

Security Considerations

Most communication between client and server should be private since we often send and receive private information. Therefore, using SSL/TLS for security is a must.

A SSL certificate isn't too difficult to load onto a server and the cost is free or very low. There's no reason not to make our REST APIs communicate over secure channels instead of in the open.

Below are the some security considerations to Implement while designing APIs:

  • Input Validation: Validate all input data to prevent SQL injection, XSS, or other attacks.
  • Rate Limiting: Throttle API requests to protect against DDoS attacks.
  • Data Encryption: Use HTTPS for all API communication.
  • Authentication: Ensure that sensitive data is protected using tokens or API keys, and never expose credentials in URLs.

Versioning the API

We should have different versions of API if we're making any changes to them that may break clients.Versioning of your APIs ensure backward compatibility. This allows you to introduce new features or make breaking changes without affecting existing clients.

  • Methods of versioning:
    • URI versioning: /v1/users
    • Query parameter versioning: /users?version=1
    • Header versioning: Custom Accept header like application/vnd.api.v1+json.

Documentation and Developer Experience

  • Clear and comprehensive API documentation is essential. Use tools like Swagger (OpenAPI) to auto-generate interactive documentation.
  • Provide example requests and responses.
  • Include guidelines on authentication, rate limits, error responses, etc.

Asynchronous and Long-Running Requests

For operations that take a long time (e.g., data processing), consider returning a 202 Accepted response and use asynchronous processing with Webhooks or polling mechanisms.

  • For example, return a URL where the client can check the status of their request.

Conclusion

The most important takeaways for designing high-quality REST APIs is to have consistency by following web standards and conventions. JSON, SSL/TLS, and HTTP status codes are all standard building blocks of the modern web.

Performance is also an important consideration. We can increase it by not returning too much data at once. Also, we can use caching so that we don't have to query for data all the time.

Paths of endpoints should be consistent, we use nouns only since the HTTP methods indicate the action we want to take. Paths of nested resources should come after the path of the parent resource. They should tell us what we’re getting or manipulating without the need to read extra documentation to understand what it’s doing.