Upstream Service - HTTP API for Bus Services
Goals
- Provide mapping of:
- Incoming HTTP requests into outgoing bus messages targeted to the requested bus service
Overview
- Upstream routes the message to the correct service with a regular bus op according to the rules in serviceInfo
Example
- Upstream creates a message for avid.iam with:
- full HTTP info in context.http.request
- serviceType=avid.iam
- Realm and zone are defaulted, unless provided explicitly with realm and region matrix parameters.
- op=createToken
- Headers of the HTTP request goes to context.http.request
- The IAM token which is associated with the request goes to context.identity
- Service using ProxyBAL dispatches the op to the right call:
- Service returns resultSet, errorSet and potentially context.http.response.
- Upstream responds to the client based on whatever is described in the response:
- Including the body, status code and header changes from context.http.response (if available).
HTTP API
URL Template
/apis/$serviceType[;realm=$serviceRealm][;version=$serviceVersion][;region=$regionId][/$path][?$queryParameter%i=queryParameterValue%i]*
Methods
HTTP methods mentioned in the following specifications MUST be supported unless noted otherwise (see below):
- Hypertext Transfer Protocol (HTTP/1.1): Semantics and Content - RFC 7231 (obsoletes RFC 2616)
- PATCH Method for HTTP - RFC 5789
Complete method list supported by the Upstream:
- GET
- POST
- PUT
- DELETE
- PATCH
TRACE method MUST be disabled for security reasons.
CONNECT method is not used in APIs, so there is no immediate need to support it.
Request Mapping
Service Type
Value of the service type MUST be determined as the value of the respective $serviceType path segment in the request URI. This path segment is mandatory.
If value of the $serviceType path segment is not specified or does not correspond to the existing service on the bus then Upstream MUST respond with 404 Not Found.
HTTP Request target | Bus service type |
---|---|
/apis | N/A → 404 Not Found |
/apis/3rd.party/endpoint?query=value | 3rd.party |
/apis/avid.asset.storage | avid.asset.storage |
/apis/avid.iam/principals | avid.iam |
/apis/non-existing-service-type | N/A → 404 Not Found |
Service Realm
Value of the service realm MUST be determined as the value of the respective $serviceRealm matrix parameter in the request URI.
If $serviceRealm matrix parameter is not specified then default service realm MUST be used. Default value of the service realm is global.
If value of the $serviceRealm matrix parameter is not a valid value then Upstream MUST respond with 504 Gateway Time-out.
HTTP Request target | Bus service realm |
---|---|
/apis/avid.assets | global |
/apis/avid.delivery;realm=bac2ea20-2f76-11e4-8c21-0800200c9a66/principals | bac2ea20-2f76-11e4-8c21-0800200c9a66 |
Service Version
Value of the service version MUST be determined as the value of the respective $serviceVersion matrix parameter in the request URI.
If value of the $serviceVersion matrix parameter is not a valid number then Upstream MUST respond with 504 Gateway Time-out.
HTTP Request target | Bus service version |
---|---|
/apis/avid.asset.storage | default version set for the service in the registry |
/apis/avid.iam;version=2/principals | 2 |
/apis/3rd.party;version=abc/endpoint?query=value | N/A → 504 Gateway Time-out |
Region
Bus message MUST be sent to a zone with the identifier determined as the value of the respective $regionId matrix parameter in the request URI.
If $regionId matrix parameter is not specified then message MUST be sent to the local zone.
If value of the $regionId matrix parameter is not a valid value then Upstream MUST respond with 504 Gateway Time-out.
HTTP Request target | Bus Zone |
---|---|
/apis/avid.assets | local zone if service exists in the local zone, otherwise any of the remote zones |
/apis/avid.delivery;region=f9823030-2f77-11e4-8c21-0800200c9a66/principals | f9823030-2f77-11e4-8c21-0800200c9a66 |
/apis/non-existing-service-type | N/A → 504 Gateway Time-out |
Operation
Upstream will look for the operation in the target service with REST description which corresponds to the meta information
of the HTTP request. The name of the operation will be used as a value for op property in the bus message.
Method
Method name MUST be passed in context.http.request.method property of the outgoing bus message as a simple string value.
HTTP Method | Bus Message |
GET |
{
[...],
"context": {
"http": {
"request": {
[...],
"method": "GET"
}
}
[...]
},
[...]
}
|
POST |
{
[...],
"context": {
"http": {
"request": {
[...],
"method": "POST"
}
}
[...]
},
[...]
}
|
Query Parameters
All query parameters passed in the HTTP request target MUST be mapped onto the paramSet object as
key - value pairs where key is a query parameter name and value is a string value of the parameter. If query parameter
is specified several times then value MUST be passed as an array of strings. Query parameters with prefix name
“_avid” IS NOT allowed and wouldn’t be passed to bus.
Query Parameters | Bus Message |
N/A |
No queryParams property:
{
[...],
"paramSet": {
...
},
[...]
}
|
?offset=0&limit=25 |
{
[...],
"paramSet": {
[...]
"offset": "0",
"limit": "25"
[...]
},
[...]
}
|
?key=value1&key=value2 |
{
[...],
"paramSet": {
[...],
"key": [ "value1", "value2" ]
[...]
},
[...]
}
|
?key |
{
[...],
"paramSet": {
[...],
"key": ""
[...]
},
[...]
}
|
?key= |
{
[...],
"paramSet": {
[...],
"key": ""
[...]
},
[...]
}
|
HTTP Message Metadata
Metadata of HTTP request MUST be captured and stored in the bus message context in the form of JSON object residing
under context/http/request path:
For services with Content-Type support
Query parameters are not included into the body anymore, as body may be a picture or video or any other entity in binary format.
Query parameters are now stored in the context of the message. Under http -> request -> parameters
IMPORTANT:
Upstream version 4.7+ performs normalization of all incoming HTTP header names (lowercases them) before putting them
into bus message. According to RFC7230 header field names are
case-insensitive. Making headers lowercase makes service code easier to write and less error-prone since service does
not need to normalize the case - platform is doing that already.
Metadata | Context Path in Bus Message | Data Type | Example |
Request method |
http.request.method | String |
GET /apis/avid.iam/principals?offset=1 HTTP/1.1 [...]
{
[...],
"context": {
"http": {
"request": {
[...],
"method": "GET",
[...]
}
}
},
[...]
}
|
Request target |
http.request.target | String |
GET /apis/avid.iam/principals?offset=1 HTTP/1.1 [...]
{
[...],
"context": {
"http": {
"request": {
[...],
"target": "/apis/avid.iam/principals?offset=1",
[...]
}
}
},
[...]
}
|
Request headers |
http.request.headers | Object |
{
[...],
"context": {
"http": {
"request": {
[...],
"headers": {
"Host": "api.everywhere.avid.com",
"Connection": "keep-alive",
"Accept": "application/json; q=0.9, application/hal+json;q=1",
"User-Agent": "Chrome/36.0.1985.125 Safari/537.36"
},
[...]
}
}
},
[...]
}
Note that upstream version > 4.7.0 will lowercase all header names
{
[...],
"context": {
"http": {
"request": {
[...],
"headers": {
"host": "api.everywhere.avid.com",
"connection": "keep-alive",
"accept": "application/json; q=0.9, application/hal+json;q=1",
"user-agent": "Chrome/36.0.1985.125 Safari/537.36"
},
[...]
}
}
},
[...]
}
|
Client address |
http.request.clientAddress | String |
{
[...],
"context": {
"http": {
"request": {
[...],
"clientAddress": "172.22.34.89"
}
}
},
[...]
}
|
Base URL template |
http.request.baseUrlTemplate | String |
{
[...],
"context": {
"http": {
"request": {
[...],
"baseUrlTemplate": "https://api.us-east-1.everywhere.avid.com/apis{/serviceType}{;version,realm,region}{+path}"
}
}
},
[...]
}
|
Compound Example
HTTP | Bus | ||
Request → |
GET /apis/avid.iam/principals HTTP/1.1 Host: api.everywhere.avid.com Connection: keep-alive Accept: application/json; q=0.9, application/hal+json;q=1 User-Agent: Chrome/36.0.1985.125 Safari/537.36 |
→ |
{
"serviceType": "avid.iam",
"serviceRealm": "global",
"serviceVersion": 0,
"op": [...],
"context": {
"identity": [...],
"http": {
"request": {
"version": "1.1",
"method": "GET",
"target": "/apis/avid.iam/principals",
"headers": {
"Host": "api.everywhere.avid.com",
"Connection": "keep-alive",
"Accept": "application/json; q=0.9, application/hal+json;q=1",
"User-Agent": "Chrome/36.0.1985.125 Safari/537.36"
},
"baseUrlTemplate": "https://api.everywhere.avid.com/apis{/serviceType}{;version,realm,region}{+path}",
"clientAddress": "172.22.34.89"
}
}
},
"paramSet": [...]
}
|
Body
For old style services (Content-Type feature not available)
HTTP request body MUST be mapped onto the paramSet object under body key as a JSON object holding content and encoding.
IMPORTANT: Upstream as a separate service does not support Transfer-encoding: chunked. Most of the developers
work with Upstream via Nginx proxy service and for them Transfer-encoding: chunked will transfer body, but instead
sending data in a series of chunks it would be sent as a whole.
IMPORTANT: HTTP request MUST supply Content-Length header so that the body is read by the Upstream.
Otherwise request body will not be available.
IMPORTANT: Upstream allows to pass XML payloads in requests and responses. To pass XML payload request must
have one of the following content types application/xml, application/+xml*, text/xml, text/+xml*.
New style service with Content-Type support
If the call is addressed to the service(method) with Content-Type support
(containing RequestContentType and/or ResponseContentType annotations on method level)
HTTP request body will be transferred to the service as bytes and deserialization of the body is service’s task.
This allows transferring of any data format, if the service is able to process this type of content.
IMPORTANT Content-Type and Accept headers should correspond to types supported by service, wildcards are allowed.
Encoding
Body encoding is stored under encoding key as a string in the body object. Encoding defines how content is represented in the bus message:
- json - content is passed as plain vanilla JSON. Value is passed as an inline JSON object in the bus message.
This is a default encoding for JSON payloads (application/json, application/*+json and application/json-patch). - string - content is passed as a string. This is a default value for text/plain payloads.
IMPORTANT: This feature is NOT yet supported by the Upstream - base64 - content is Base64 encoded. This is a default value for application/octet-stream payloads.
IMPORTANT: This feature is NOT yet supported by the Upstream.
IMPORTANT: If request body contains XML payload than encoding key in the corresponding body object should have
value string, but this feature is not yet supported by the bus.
NOTE: Encoding of the body data in the bus message is NOT the same as
Content-Encoding header field in HTTP request.
HTTP Request | Bus Messages |
POST /apis/avid.iam/principals HTTP/1.1 Content-Type: application/json; charset=utf-8 Content-Length: 78 [...] { "entity": { "kind": "user", "alias": "user1@google.com" } } |
{
[...],
"paramSet": {
[...],
"body": {
"encoding": "json",
"entity": {
"kind": "user",
"alias": "user1@google.com"
}
}
[...],
},
[...]
}
|
POST /apis/hal.acceptor/giveMeContent HTTP/1.1 Content-Type: application/hal+json; charset=utf-8 Content-Length: 71 [...] { "\_links": { "next": { "href": "/page=2" } } } |
{
[...],
"paramSet": {
[...],
"body": {
"encoding": "json",
"data": {
"\_links": {
"next": {
"href": "/page=2"
}
}
}
}
[...],
},
[...]
}
|
POST /apis/upload.service/text HTTP/1.1 Content-Type: application/octet-stream Content-Length: 5 [...] abcde |
{
[...],
"paramSet": {
[...],
"body": {
"encoding": "base64",
"data": "YWJjZGU="
}
[...],
},
[...]
}
|
POST /apis/service/empty HTTP/1.1 Content-Type: application/json; charset=utf-8 [...] |
{
[...],
"paramSet": {
[...]
},
[...]
}
|
GET /apis/avid.iam/principals HTTP/1.1 [...] |
{
[...],
"paramSet": {
[...]
},
[...]
}
|
POST /apis/avid.iam/principals HTTP/1.1 Content-Type: application/json; charset=utf-8 Content-Length: 2 [...] [] |
400 Bad Request (body content passed in is not a valid JSON) |
Response Mapping
HTTP Message Metadata
Metadata of HTTP response MUST be captured from the bus message context in the form of JSON object residing
under http/response path overriding any default metadata values and assumptions made by the upstream:
Metadata | Context Path in Bus Message | Data Type | Example | Comments |
Response status code | http.response.status | String or Number |
{
[...],
"context": {
"http": {
"response": {
"status": 204,
[...]
}
}
},
[...]
}
HTTP/1.1 204 No Content
[...]
|
Optional field.
Default values:
|
Response headers | http.response.headers | Object |
{
[...],
"context": {
"http": {
"response": {
[...],
"headers": {
"Content-Type": "application/hal+json",
"Content-Encoding": "gzip"
}
}
}
},
[...]
}
HTTP/1.1 [...] [...] Content-Type: application/hal+json Content-Encoding: gzip |
Body
HTTP response body MUST be mapped from the resultSet or errorSet of the bus message:
- If errorSet is present then it MUST be used as a source of response body. Presence of errorSet means there
was (at least one) problem when processing the request. - Else resultSet MUST be used.
For new service with Content-Type support
ErrorSet still should be mapped to errors, but body is sent back as is with addition of Content-Type description
Errors
In case errorSet is present, its first error MUST be mapped to the response body with the rules outlined below:
Bus Message | HTTP Response |
HTTP/1.1 404 Not Found
Content-Type: application/vnd.avid.error+json
...
|
- application/vnd.avid.problem+json is used as a media type for representing errors.
- First error in the errorSet “becomes” the representation of error resource.
- The motivation is to have the simplest and shortest form for the major use case.
- Otherwise client would need to explicitly dive into $.errorSet[0] and think in terms of multiple errors each and every time instead of having instant error access.
- status is always added to the response even if bus message does not carry one.
- API of ACS BAL prior to version 3.2 does not allow to construct error with status. ACS BAL 3.2 and above deprecated this API, but still supports it. If status is NOT present in bus error:
- Upstream 4+ sets status to 500 unless it is overridden by context.http.response.status
- Since code was used in some cases to communicate status code in pre ACS BAL 3.2 API, Upstream 4+ will set status to 404 if code is set to “404” and no context.http.response.status is specified.
- API of ACS BAL prior to version 3.2 does not allow to construct error with status. ACS BAL 3.2 and above deprecated this API, but still supports it. If status is NOT present in bus error:
- exchange is added to the representation to trace back unique HTTP request / response pair.
- details are stripped (still logged to the Upstream main log).
- The motivation behind default behavior is to avoid exposure of sensitive data.
- severity is stripped (still logged to the Upstream main log).
- The motivation behind default behavior is to avoid exposure of data that is of no help to the client.
Only the first error is considered. Others are ignored.
Bus Message | HTTP Response Entity |
{
"serviceType": "avid.service",
"serviceRealm": "global",
"serviceVersion": 1,
"op": "operation",
"context": ...
"errorSet": [
{
"code": "internal/avid.service/global/1/I0001",
...
},
{
"code": "internal/avid.service/global/1/I0002",
...
},
...
]
}
|
{
"code": "internal/avid.service/global/1/I0001",
...
}
|
Mapping of resultSet is based on the following rules:Results
- If there is a key under resultSet/body/data path then respective value MUST be used as a source of response body.
- Else no response body will be provided.
Bus Message | HTTP Response Body |
{
[...],
"resultSet": {
"body": {
"data": {
"key": "value"
}
}
}
}
|
{
"key": "value"
}
|
{
[...],
"resultSet": {
"key": "value"
}
}
|
(no body) |
Body encoding MAY be stored under resultSet/body/encoding key as a string (see Request Mapping > Body > Encoding section). If encoding is not specified then:
- json encoding if resultSet/body/data is a JSON object.
NOTE: Encoding of the body data in the bus message is NOT the same as Content-Encoding header field in HTTP response.
Source value gets decoded according to the encoding rules.
Upstream SHOULD NOT try to validate content negotiation rules - responsibility SHOULD be a responsibility of the service.
Bus Message | HTTP Response Body |
{
[...],
"resultSet": {
"body": {
"data": {
"key": "value"
}
}
}
}
|
{
"key": "value"
}
|
{
[...],
"resultSet": {
"body": {
"data": "string value"
}
}
}
|
string value
|
{
[...],
"resultSet": {
"body": {
"data": 5
}
}
}
|
5
|
{
[...],
"resultSet": {
"body": {
"data": [1, 2, 3, 4, 5]
}
}
}
|
[1, 2, 3, 4, 5]
|
{
[...],
"resultSet": {
"body": {
"data": true
}
}
}
|
true
|
{
[...],
"resultSet": {
"body": {
"data": null
}
}
}
|
(no body) |
{
[...],
"resultSet": {
"body": {
"data": {}
}
}
}
|
(no content)
|
{
[...],
"resultSet": {
"body": {
"encoding": "json",
"data": {}
}
}
}
|
(no content)
|
{
[...],
"resultSet": {
"body": {
"encoding": "base64",
"data": "c3RyaW5nIHZhbHVl"
}
}
}
|
string value
(base64 decoded from data)
|
{
[...],
"resultSet": {
"body": {
"encoding": "json"
}
}
}
|
(no body) |
{
[...],
"resultSet": {
"body": {
"encoding": "string"
}
}
}
|
(no body) |
{
[...],
"resultSet": {
"body": {
"encoding": "base64"
}
}
}
|
(no body) |
{
[...],
"resultSet": {
"body": {
"encoding": "json",
"data": "string value"
}
}
|
500 Internal Server Error (json encoding assumes that data value is a JSON object) |
{
[...],
"resultSet": {
"body": {
"encoding": "string",
"data": {
"key": "value"
}
}
}
}
|
{"key":"value"}
(*string* encoding does automatic conversion of *data* value to a string)
|
{
[...],
"resultSet": {
"body": {
"encoding": "base64",
"data": {
"key": "value"
}
}
}
}
|
*500 Internal Server Error* (*base64* encoding assumes that *data* value is a string) |
Response Metadata Mapping
HTTP | Bus | ||
← Response |
HTTP/1.1 200 OK
Location: httрs://api.everywhere.avid.com/apis/avid.iam/principals/123
Content-Type: application/hal+json
|
← |
|
Retrieving Principals
HTTP | Bus | ||
Request → |
GET /apis/avid.iam/principals?offset=0&limit=25\
&orderBy=name&order=asc&entity=%5B%7Bfield%3A+%22kind%22%2Cvalue%3A+%22user%22%7D%5D HTTP/1.1
[...]
|
→ |
{
"serviceType": "avid.iam",
"serviceRealm": "global",
"serviceVersion": 0,
"op": "findPrincipalById",
"context": {
"identity": [...],
"http": {
"request": {
"version": "1.1",
"method": "GET",
"target": "/apis/avid.iam/principals?offset=0&limit=25\
&orderBy=name&order=asc&entity=%5B%7Bfield%3A+%22kind%22%2Cvalue%3A+%22user%22%7D%5D",
[...]
}
}
},;
"paramSet": {
"offset": "0",
"limit": "25",
"orderBy": "name",
"order": "asc",
"entity": "[{field: \"kind\",value: \"user\"}]"
}
}
|
← Response |
HTTP/1.1 200 OK
[...]
{
"entity": [
{
"field1": "value1",
"field2": "value2"
},
{
"field1": "value3",
"field2": "value4"
}
]
}
|
← |
{
"serviceType": "avid.iam",
"serviceRealm": "global",
"serviceVersion": 0,
"op": "operation",
"resultSet": {
"body": {
"encoding": "json",
"data": {
"entity": [
{
"field1": "value1",
"field2": "value2"
},
{
"field1": "value3",
"field2": "value4"
}
]
}
}
}
}
|
Creating Principal(s)
HTTP | Bus | ||
Request → |
POST /apis/avid.iam/principals HTTP/1.1
Content-Type: application/json; charset=utf-8
Content-Length: 78
[...]
|
→ |
{
"serviceType": "avid.iam",
"serviceRealm": "global",
"serviceVersion": 0,
"op": "operation",
"context": {
"identity": [...],
"http": {
"request": {
"version": "1.1",
"method": "POST",
"target": "/apis/avid.iam/principals",
[...]
}
}
},
"paramSet": {
"body": {
"encoding": "json",
"data": {
"entity": {
"kind": "user",
"alias": "user1@google.com",
[...]
}
}
}
}
}
|
← Response |
HTTP/1.1 201 Created
Location: httрs://api.everywhere.avid.com/apis/avid.iam/principals/123
[...]
|
← |
{
"serviceType": "avid.iam",
"serviceRealm": "global",
"serviceVersion": 0,
"op": "operation",
"context": {
"http": {
"response": {
"status": 201,
"headers": {
"Location": "httрs://api.everywhere.avid.com/apis/avid.iam/principals/123"
}
}
}
},
"resultSet": {
"body": {
"encoding": "json",
"data": {
"entity": {
"kind": "user",
"alias": "user1@google.com",
"created": "20140809T183142-03",
[...]
}
}
}
}
}
|
Retrieving Principal
HTTP | Bus | ||
Request → |
GET /apis/avid.iam/principals/123?regionId=123 HTTP/1.1
[...]
|
→ |
{
"serviceType": "avid.iam",
"serviceRealm": "global",
"serviceVersion": 0,
"op": "operation",
"context": {
"identity": [...],
"http": {
"request": {
"version": "1.1",
"method": "GET",
"target": "/apis/avid.iam/principals/123",
[...]
}
}
"method": "GET"
},
"paramSet": {
"regionId": "123"
}
}
|
← Response |
HTTP/1.1 200 OK
[...]
{
"field1": "value1",
"field2": "value2"
}
|
← |
{
"serviceType": "avid.iam",
"serviceRealm": "global",
"serviceVersion": 0,
"op": "operation",
"resultSet": {
"body": {
"encoding": "json",
"data": {
"field1": "value1",
"field2": "value2"
}
}
}
}
|