1 API Design
This document helps you to get started with best practices while designing Restful API. Focus is on more on best practices than the architecture elements of Restful API development. Assume the reader knows the basics of REST Design Pattern.
1.1 Rest HTTP Mappings [Methods]
Majority of cases Rest endpoints can be mapped as shown in the following tables.
CRUD
|
Description
|
HTTP mapping in REST services
|
Create
|
Create an Entity on the service side
|
POST
|
Read
|
Retrieve an Entity from the service
|
GET
|
Update
|
Implements a persistent change of the Entity on the service
|
PUT / PATCH
|
Delete
|
Remove an Entity from the service
|
DELETE
|
Note: GET verb is idempotent (i.e. if the verb is repeated, the outcome on the service side is identical) while POST is not
1.1.1 Restful Search
HTTP protocol don’t provide a ‘SEARCH’ method. So we need to manage it using http GET [recommended] or http POST.
Http URI length support’s up to 2000 character’s. We can use http GET method with query param’s to set the search criteria. If querystring has any sensitive information, then use SSL which encrypt querystring as well.
GET /searchPNR?firstName=xx, emailID=yy….
Sometime your search criteria can go beyond 10+ key value’s, in those scenario’s it will be good to use http POST and pass the data as part of http request BODY.
This way URI’s will be readable. The difference is the interaction is not changing the state of the resource.
If your search includes sensitive information like password, use POST with URL encoding.
Content-Type: application/x-www-form-urlencoded
1.2 Resource Naming Conventions
ü Use Nouns instead of Verbs [ eg – Use /Orders instead of getOrders ]
ü Use Plural [ Eg - /Orders instead of /Order]
ü Use Concrete Nouns [ Eg - /Orders instead of /Service]
ü Keep resource name short in order to avoid space limitation
ü Action can be implied by http method [eg- GET, PUT, POST or DELETE]
ü Use GET [Idempotent] to read data from server, you should not modify server resource by GET method
ü Use POST [not Idempotent] to create new resource. On successful POST, return http status 201. Certain scenario’s you can return location header the link to the new resource
ü Use Put to update a server resource. On successful operation return 200 or 204, if response body is empty. Similar to POST, PUT changes the state of the server.
ü Use CRUD [create, read, update, delete] as much as possible [eg – createOrder,
getOrder, updateOrder, deleteOrder]
ü Use sub resource to capture the state that results from the execution of the action
[POST /orders/shipped-order{id} ]
Notes – Don’t use GET operation for server side state
1.3 Base URI Design
ü Use Idempotent request’s where ever applicable as the same request can be replayed multiple times without any side effects
ü Use Query parameters to search/get a subset of a collection of resources
[eg - GET /orders? state=shipped ]
ü Use Query parameters to describe sorting rules [GET /orders?sort=date, product
1.4 Contract First API Design
ü Design and document the API before they are being developed
ü Use API Modelling language [RAML] to define the method and resource of a RESTFUL API where ever applicable
ü For larger/complex System, Classify the API’s into logical layer’s including System API’s, Process API’s, Experience API’s.
ü Prefer JSON as standard API data Format, unless there is no choice. Use multipart media types for working with binary data
1.5 Request and Response Headers
1.5.1 Http Headers
ü Use Content-Type, Content-length, Content-encoding & content Language to provide additional metadata of the API request
ü Make Use of HTTP/1.1 header’s [ Use Cache-Control for caching, Authorization for authentication and authorization.
List of http standard header listed below.
Header
|
Description
|
Accept
|
Content-Types that are acceptable for the response, e.g., application/json
|
Accept-Charset
|
Character sets that are acceptable for the response, e.g., utf-8
|
Accept-Encoding
|
List of acceptable encodings, e.g., gzip
|
Accept-Language
|
List of acceptable human languages for response, en-US
|
Content-Type
|
The MIME type of the body of the request (used with POST and PUT requests), e.g., application/json
|
Date
|
The date and time that the message was sent, Mon, 15 Dec 2014 08:15:30 GMT
|
Authorization
|
Authentication credentials for HTTP authentication, e.g., Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ==
|
If-Match
|
Tells the API to only perform the action if the client-supplied entity matches the same entity on the server. This is mainly used in PUT to only update a resource if it has not been modified since the user last updated it, e.g., If-Match: v1
|
Pragma
|
Implementation-specific fields that may have various effects anywhere along the request-response chain (e.g., no-cache)
|
Cache-Control
|
Used to specify directives that must be obeyed by all caching mechanisms along the request-response chain (e.g., no-cache)
|
Connection
|
What type of connection the user-agent would prefer (e.g., keep-alive)
|
ETag
|
An identifier for a specific version of a resource (e.g., ETag: v1)
|
Expires
|
The date/time after which the response is considered stale, e.g., Mon, 15 Dec 2014 09:20:20 GMT
|
Last-Modified
|
The last modified date for the resource, e,g., Mon, 15 Dec 2014 10:10:30 GMT
|
1.5.2 Http Response Code
Use standard http response code’s defined as part of protocol instead of sending success with custom error message as part of response.
Code
|
HTTP Method
|
Response Body
|
Description
|
200 OK
|
GET, PUT, DELETE
|
Resource
|
There are no errors, the request has been successful
|
201 Created
|
POST
|
URI of the resource that has been created
|
The request has been fulfilled and resulted in a new resource being created
|
202 Accepted
|
POST, PUT, DELETE
|
An URI of a resource which represents the processing status
|
The request has been accepted for processing, but the processing has not been completed
|
204 No Content
|
GET, PUT, DELETE
|
N/A
|
There are no errors, the request has been processed and no contact is expected in the body (by design)
|
304 Not Modified
|
conditional GET
|
N/A
|
The resource has not been modified: there is no new data to return
|
400 Bad request
|
GET, POST, PUT, DELETE
|
Error message
|
The request could not be understood by the server due to malformed syntax. The client SHOULD NOT repeat the request without modifications
|
401 Unauthorized
|
GET, POST, PUT, DELETE
|
Error message
|
The request requires user authentication
|
403 Forbidden
|
GET, POST, PUT, DELETE
|
Error message
|
The server understood the request, but is refusing to fulfill it. For example, Authentication failure or invalid Application ID. Authorization will not help and the request SHOULD NOT be repeated
|
404 Not Found
|
GET, POST, PUT, DELETE
|
Error message
|
The server has not found anything matching the request URI
|
405 Method Not Allowed
|
GET, POST, PUT, DELETE
|
Error message
|
The method specified in the request is not allowed for the resource identified by the URI
|
406 Not Acceptable
|
GET, POST, PUT, DELETE
|
Error message
|
The request contains parameters that are not acceptable
|
408 Request Timeout
|
GET, POST, PUT, DELETE
| ||
409 Conflict
|
POST, PUT, DELETE
|
Error message
|
The request could not be completed due to a conflict with the current state of the resource. This code is only allowed in situations where it is expected that the user might be able to resolve the conflict and resubmit the request
|
410 Gone
|
GET, POST, PUT, DELETE
|
Error message
|
Used to indicate that an API endpoint has been turned off. Could be used to deprecate API, for example, to inform the customer that the API will soon stop functioning and to migrate to new version of the API
|
412 Precondition Failed
|
GET, PUT
|
Error message
|
The precondition given in one or more of the request-header fields evaluated to false when it was tested on the server. This response code allows the client to place preconditions on the current resource metainformation (header field data) and thus prevent the requested method from being applied to a resource other than the one intended
|
415 Unsupported Media Type
|
GET, POST, PUT
|
Error Message
|
The server is refusing to service the request because the entity of the request is in a format not supported by the requested resource for the requested method
|
429 Too Many Requests
|
GET, POST, PUT, DELETE
|
Error message
|
Indicates that the user has sent too many requests in a given amount of time ("rate limiting")
|
500 Internal Server Error
|
GET, POST, PUT, DELETE
|
Error message
|
The server encountered an unexpected condition which prevented it from fulfilling the request
|
502 Bad Gateway
|
GET, POST, PUT, DELETE
|
Error message
|
The server, while acting as a gateway or proxy, received an invalid response from the upstream server it accessed in attempting to fulfill the request
|
503 Service Unavailable
|
GET, POST, PUT, DELETE
|
Error message
|
The server is currently unable to handle the request due to a temporary overloading or maintenance of the server. The implication is that this is a temporary condition which will be alleviated after some delay
|
504 Gateway Timeout
|
GET, POST, PUT, DELETE
|
Error message
|
The server, while acting as a gateway or proxy, did not receive a timely response from the upstream server specified by the URI (e.g. HTTP, FTP, LDAP) or some other auxiliary server (e.g. DNS) it needed to access in attempting to complete the request
|
509
|
GET, POST, PUT, DELETE
|
Error Message
|
Bandwidth Limit Exceeded (not included in the W3C standard, implemented as part of the Apache tooling)
|
510 Not Extended
|
GET, POST, PUT, DELETE
|
Error message
|
The policy for accessing the resource has not been met in the request. The server should send back all the information necessary for the client to issue an extended request
|
511
|
GET, POST, PUT, DELETE
|
Error Message
|
Network Authentication Required (not included in the W3C standard)
|
550
|
GET, POST, PUT, DELETE
|
Error message
|
Permission denied (not included in the W3C standard
|
598
|
GET, POST, PUT, DELETE
|
Error Message
|
Network Read Time-Out Error
|
599
|
GET, POST, PUT, DELETE
|
Error Message
|
Network Connect
|
For the completed list of http response code’s, refer below blog
1.6 Exception Handling
It’s very important to handle exception at server side and return appropriate errors code and message back to the client application. It’s strongly recommended to use http defined standard error to convey the status of a request.
1xx: Informational - Communicates transfer protocol-level information
2xx: Success -Indicates that the client’s request was accepted successfully
3xx: Redirection - Indicates that the client must take some additional action in order to
complete their request
4xx: Client Error - This category of error status codes points the finger at clients
5xx: Server Error - The server takes responsibility for these error status codes
1.6.1 Application Error Code
In addition to the standard http response code set in the response header, we can
have an additional application defined error code and error message.
Most of the REST frameworks provides Response Builder utilities to handle rest exceptions
Sample response for a API Validation failure response is shown below.
http error Code: 400 [Bad Request]
{
"errorCode" : 1001,
"errorMessage” : " Mandatory field is missing "
}
1.7 Conditional GET
An ETag (entity tag) is an HTTP response header returned by an HTTP/1.1 compliant web server used to determine change in content at a given URL. eTag can be used for conditional Get, concurrency control as well as optimistic locking.
Conditional GET allows a client to ask a server if a resource has changed using request header If-None-Match with the e-Tag set as value.
eTag changes when a resource has changed. With eTag, we can ask a server to return the resource only if it got changed. With bodiless 304 responses, we can reduce the server computational time as well as the network bandwidth
1.8 Aysnc operation in REST
Use http 202 [ request has been accepted for processing, but the processing has not been completed] for implementing Aysnc operation in Rest.
http 202 return a Location header specifying the url which client can use to monitor the status of the previous request.
As soon as processing is done, server deletes the queue resource and return a 303 code which implies that there is another resource that must be fetched instead of this resource, in our case, the actual blog post resource.
1.9 API Versioning
When we create new flavours [major changes] of an api, it’s a good practise to version the api.
This way we can deprecate older api’s with the new versions and support backward compatibility. User’s will get an advanced notification of the new api’s available and can plan to migrate to the new api’s as per their convenience.
It’s always good to include the version’s in the URI. Sample Rest uri with version provided below.
1.9.1 when not to Version-?
Keeps the URI’s changes to minimal, instead design the api to be extendable. You should avoid versioning unless required. Couple of scenario’s added below.
o Additional data to the response
o Implementation technology or logical layering underneath has been changed
1.10 API Catalogue
It’s as important as API development to document the API in a way which the developer community can understand and consumer the API. API Document should basically contain the API contract including request and response structure, error response structure, header and URL description, http method description etc… It’s good to include sample api request and response as well.
Mule AnyPoint Platform provides the API Portal where we can document the API’s online and can publish the api’s to the developer community in a controlled fashion. If we don’t have a tool, well defined catalogue will do the purpose.
1.11 API Deployments
Traditionally we used to rely to Application servers to deploy the web or web services archive. With heavy application servers many of the core libraries and configurations are owned by application servers instead of API’s.
1.11.1 Micro Containers
With new micro service architecture, we now have micro containers to deploy the API. With micro containers we can embed light weight containers [servlet containers] as one of the application dependency, making the service as well as container’s light weighted. For java developers, choices include embedded Tomcat, jetty as well as Undertow.
2 API Security
API design is not complete unless we have not defined the security aspect of the API.
Who should access the API?
What is the level of access the user has with the API? [read, write or something else]
Below section describes the best practices with respect to API security.
2.1.1 API Authentication and Authorization
There are different methodologies with respect to API authentication and authorization.
When it comes to SOAP security, there are different standardisation, predominantly Oasis Web Service Security. Similarly, for Rest we have Oauth2, Open ID connect ++. Open ID Connect is primarily built on top of Oauth2 with additional support for User Profile Token.
Oauth2 is easy and good fit for securing Rest endpoints. With Oauth2, we rely on access Token [oauth2 Token] to control access to the API’s. Access token can be issued by a security server [security micro service/api] upon validating against an AD or a database.
Authentication as well as authorization for an api can be implemented with the combination of security server as well as api gateway or we can use Identity management platforms available in the market. Mule Any point provides the complete portfolio of services to publish the api to the external world in a secured way. Combination of custom security server with an open source api gateway [Kong or Zuul] is another choice.
For usecases where gateway or remote validation is not feasible, we can have jwt Token by which the api can directly validates the jwt Token using the jwt Validation library bundled along with the api without the intervention of security server or gateway.
3 API Patterns
Below section describes the set of API Patterns as well as Anti Patterns applicable in the context of Restful API Design
3.1 Recommended API Patterns
3.1.1 API Gateway
With micro service approach we develop small services faster and we end up exposing more api’s, each api with a specific well defined functionality. Incorporating security in each of these micro services is a tedious task. Api gateway came to rescues, we can have a api gateway which manages as well as control’s a group of api’s bounded to a domain or a sub domain.
With API gateway, we can validate the access Token with security server, validate the user access against the api ++. Throttling, auditing forms additional functionalities of the Api gateway.
Implementation includes Mule AnyPoint Platform, Netflix Zuul, Azure traffic manager ++
3.1.2 API Routing
To ensure high availability, we need to deploy multiple instances of the micro services. Request from external systems can be routed via api gateway and gateway can do the api load balancing in a configured load balancing strategy [round robin for example]. Gateway can also do the automatic fail over, retry the service also break the circuit [circuit breaker pattern] based on scenarios.
Between micro services with in the same sub domain or context, we can implement client side load balancing where client directly monitors the api’s health and invokes the api which is available at the time of invocation.
For java based apps, Netflix provides ribbon library which provides client side load balancing pattern. For .net applications steeltoe-oss library provides the required frameworks for building REST endpoints following micro service architecture.
3.1.3 Aggregation Strategies
When designing Process API’s [Wrapper API], API layer might need to communicate to System API’s of multiple core systems. In these scenario’s we can apply EIP pattern’s [Integration Pattern’s] avoiding the need for a central service bus. One of the example is Split and Aggregate pattern where we can split the request API to multiple sub request, get the result from each sub request and aggregate the result.
Frameworks like Apache camel, Mule AnyPoint platform supports implementation of most of the EIP Pattern’s and provides ready to use components.
3.2 Anti-Patterns
Patterns to be applied only if it’s add value to the architecture. Anti-patterns describe how not to write a restful API. At the same time, we should try to avoid anti-patterns, sample list provided below
Ø Non cohesive API’s design [API which is trying to do multiple responsibilities, violating the single responsibility]
Ø Invalid usage of http methods -
o exposing data queries using http ‘POST’
o system modifications using http ‘GET’
Ø Ignoring usage of http error codes
o sending 200 response code for success as well as failure scenario’s.
Ø Absence of Rest exception handling
o Throw the technical errors directly to the api consumer
Ø Doing long running batch operation over REST endpoints
o Absence of http response code for manging async operations in REST
Ø API coupling to infrastructure
o API’s which breaks when moved from one data centre to another data centre. [shared drive usage for example]
