Publish/Subscribe Push MEP: Implementation Guidance

Last updated:  JULY 2nd, 2019

Introduction

This document intends to serve as implementation guidance of the Publish/Subscribe Push Message Exchange Pattern [7] using SWIM-TI Yellow Profile Service Bindings [6]. The document is structured as follows:

  • In section 2, we provide a general overview of Publish/Subscribe services in a technology agnostic manner.
  • In section 3, the technology agnostic service design is realised in a concrete implementation using WS-Light and AMQP Messaging Service Bindings [3].

The proposed service design and implementation is intended to provide a basic template for implementers that can be customised or complemented in multiple ways to suit specific needs.

Glossary of Terms

MEP:

Message Exchange Pattern

UUID:

Universally Unique IDentifier

REST:

Representational State Transfer

AMQP:

Advanced Message Queuing Protocol

Technology Agnostic Service Design

Publish/Subscribe Service Design

In its general form a Publish/Subscribe service can be understood as consisting of at least 2 interfaces:

  • A subscription interface: Containing operations that relate to the management of the service subscription (e.g. subscribe, unsubscribe, pause…).
  • A publish interface: Dealing with the actual publication of data to its consumers.

Figure 1: Publish Subscribe Service: Interfaces

Specific details of the implementation of the service subscription interface may vary. For example;

  • The subscribe , unsubscribe and pause operations may be merged into a single overloaded operation that performs either depending on the request payload.
  • The pause subscription operation may altogether not exist if said functionality is not needed.
  • Additional operations can be added to support extended functionality, for instance:
    • A getSubscription/getSubscriptions operation to obtain details of an existing subscription(s),
    • A getTopics operation that lists the possible topics a Consumer can subscribe to,
    • Etc…

What is usually common across implementations is that the Subscribe interface is implemented using a Request/Response pattern [1] as subscribing to the service typically entails some form of approval by the service provider which in turn provides back certain information detailing the means to use to obtain the publications. This interface is provided by the Service Provider and required by the Consumer.

On the other hand, the Publish interface is most commonly implemented using a Fire-and-Forget MEP [7] as it fully leverages the advantages of Publish/Subscribe, minimizing interactions between providers and consumers and preventing unnecessary workload on consumer side, which would otherwise need to perform periodic requests (polling) to get the latest updates. This interface is provided by the Consumer that needs to be able to accept reception of the publish operation and required by the Service Provider who will invoke it.

Particular technical implementations may rely on additional interfaces and operations to manage different aspects of the service like; connection establishment between Consumer and Service, delivery state of messages, heartbeats to communicate the status of each participant, etc... These interfaces are implementation specific and vary from technology to technology and thus are best documented as part of the technology specific service description (see section 3).

Subscription Model

The Subscribe interface needs to provide a model that specifies what defines a subscription. There is no standardized way to do so and different services might have different needs that may justify variation in the subscription models. In this section we provide a simple and extensible subscription model and common alternative variations.

A subscription can be described as depicted in Figure 2.

Figure 2: Subscription class (non-normative).

  • id: Unique identifier of a subscription, (e.g. a UUIDv4).
  • topic: A string that matches under certain predefined rules a topic or set of topics of the service. It may take values in an enumeration list or might be defined using a tree structure and allow the use of wildcard operators.
  • durable: Expresses the durability of the subscription, a subscription is durable if messages are preserved while subscribers are offline. Sometimes can be accompanied by a time-to-live attribute that indicates the expiration time of non-delivered messages.
  • qos: Expresses the reliability Quality of Service of the subscription.
  • status: A description of the state of a subscription, usually taking values in a predefined enumeration list. The possible states must be accompanied by a state diagram indicating the possible transitions and its triggers.

Subscription States and Lifecycle

The subscription model defines the following subscription states:

  • ACTIVE: The subscription has been created and messages are being queued in the subscriber´s queue.
  • PAUSED: The subscription has been created but messages are not being queued in the subscriber’s queue.
  • DELETED: The subscription and the queue have been deleted.

The operations of the Subscribe interface allow the Consumer to manage the state of a subscription, the following state diagram depicts the lifecycle of a subscription and the different state transitions associated with its operations.

Figure 3: Subscription State Diagram

Subscription Filtering

Filtering mechanisms provide a way for consumers to declare interest in a subset of publications, helping reduce unnecessary processing and bandwidth consumption for both provider and consumers.

Subject-based Filtering

In Publish/Subscribe services a topic is a descriptor of the subject of the information exchanged. Topics classify the publications in a number of different subjects, during subscription Consumers can declare interest in a number of topics which allows them to filter the publication received to that particular subset.

Topic Enumerations

An enumeration provides the simplest mechanism to define the list of subjects to which a consumer can subscribe.

The following figure illustrates an example topic enumeration with three possible topics that classify the publications of a service according to their subject.

Figure 4: Example Topic Enumeration

Topic Trees

Topic trees provide a rich mechanism to classify the publications of a service in a structured way.

A topic tree can be defined as follows:

  • Define a reserved character that acts as word separator. E.g.
    • Use the character “/” as a word separator.
  • Define a tree structure of topics using the word separator character. E.g.:
  • /weather/precipitation/LEMD
  • /weather/precipitation/LFPG
  • /weather/wind/LEMD
  • /weather/wind/LFPG
  • Define a number of wildcard characters that can be used to match sub-branches of tree structure. E.g.:
    • “*” as a wildcard that matches any single word of a topic
    • “#” as a wildcard that matches any number of consecutive words of a topic.

With this topic structure the consumers can declare interest to specific data by subscribing to topics using the semantics defined. E.g.:

  • Subscribing to “ /weather/#” declares interest to precipitation and wind data for all airports.
  • Subscribing to “/weather/wind/*” declares interested to wind data for all airports.
  • Subscribing to “/weather/precipitation/LFPG” declares interest to precipitation data for the Paris Charles de Gaulle Airport.
Content-based Filtering

Content-based filtering allows consumers to specify their interest into specific publications based on the runtime evaluation of certain predicates against the content of the publication message. They provide the most flexible and general filtering mechanism but the most difficult to implement and with expensive runtime costs.

For example, a consumer may declare interest in every publication that contains a particular keyword or publications where a particular attribute of the payload is bigger than a concrete value.

Dynamic Behaviour

The following sequence diagram shows the nominal flow of a subscription, subsequent publications and final unsubscription from the service:

Figure 5: Publish/Subscribe sequence diagram

  1. The Consumer performs a subscription request, filling the appropriate request message.
  2. The Service Provider processes the request (payload validation, authentication and authorization processing occur at this moment) and provides the appropriate response message.
  3. When a new message is available it is published to the Service Consumer.
  4. Finally, the Service Consumer unsubscribes from the service which stops further publications.

This nominal flow represents the basic Publish/Subscribe interaction that can be extended in a straight-forward manner to include additional operations like pause and resume subscription. This nominal flow can also be extended to a Brokered Publish/Subscribe Push Message Exchange Pattern, in which case the broker or brokers would appear as intermediaries between Service and Consumer.

Service Implementation

Topic Publish/Subscribe Service Implementation

A Topic-based Publish/Subscribe service can be implemented in the following manner:

  • Using WS-Light Service Binding for the implementation of the Subscribe interface,
  • Using AMQP Messaging Service Binding for the implementation of the Publish interface.

In this section we will provide detailed implementation guidance to realize the generic service design outlined in section 2.1 using the proposed Service Bindings.

Subscription

The Subscribe interface can be implemented using a REST design. The REST resources represent:

  • A collection of topics: /topics/
  • A specific topic: /topics/<topic>
  • The collection of all subscriptions: /subscriptions/
  • A specific subscription: /subscriptions/<subscription_id>

The following table summarizes a mapping between the technology agnostic interface design (to the left) and the REST resource endpoint (to the right) with the corresponding HTTP method.

Design

Implementation

Operation

Resource endpoint

HTTP method

subscribe

/subscriptions/

POST

pause

/subscriptions/<subscription_id>

PUT

resume

/subscriptions/<subscription_id>

PUT

unsubscribe

/subscriptions/<subscription_id>

DELETE

Additional operations to retrieve the list of available topics and the current state of a particular subscription can be included in a simple manner:

Design

Implementation

Operation

Resource endpoint

HTTP method

getTopics

/topics/

GET

getTopic

/topics/<topic>

GET

getSubscription

/subscriptions/<subscription_id>

GET

subscribe

This operation allows a Consumer to request the subscription to a particular Topic of the Service.

Figure 6: subscribe sequence diagram

Description:

  1. The Consumer invokes a “POST” on the /subscriptions/ endpoint.
  2. The Service Provider applies Authentication, Authorization and Data Validation controls and determine the result of the subscription request.
  3. The Service Provider provides a Response with the result of the request including (if successful) the appropriate queue where the messages will be published.

Post-Conditions: If successful, the /<queue> and /subscriptions/<subscription_id> endpoints are created.

Example Request Message

Headers

Accept: application/json, application/xml

Content-Type : application/json

Accept-Encoding: gzip

Body

{“topic”: <topic>,

“qos”: “EXACTLY_ONCE”,

“durable”: true}

Example Response Message

Headers

Status-Code: 201

Reason-Phrase: Created

Content-Type: application/json

Body

{“topic”: <topic>,

"subscription_id": <subscription_id>,

"queue": <queue>,

"subscription_status": "PAUSED",

“qos”: “EXACTLY_ONCE”,

“durable”: true}

Error Codes

The following error codes may be of interest to implementers, covering common failures applicable to this resource invocation. Implementers are free to use a suitable subset of the ones proposed or complement it with additional ones to cover particular implementation needs.

400 Bad Request: Indicated for malformed requests on the part of the client.

401 Unauthorized: Indicated for authentication errors (e.g. the authentication is not provided, the credentials used do not exists, the authentication mechanism used is not valid…).

403 Forbidden: Indicated for authorization errors (e.g. the authenticated credentials are not allowed to perform this operation on the resource).

405 Method Not Allowed: The HTTP method used is not supported by the resource.

406 Not Acceptable: None of the Content-Types provided in the request Accept header are supported.

415 Unsupported Media Type: The Content-Type of the request is not valid for this operation.

500 Internal Server Error: Catch-all error for unexpected internal errors during the processing of the request.

pause [2]

This operation allows a Consumer to pause the publishing of messages for a particular topic subscription.

Figure 7: pause sequence diagram

Description:

Pre-Conditions: The Consumer has successfully subscribed to a particular <topic>.

  1. The Consumer invokes the “PUT” method on the /subscriptions/<subscription_id> endpoint. The body of the message includes a change to the subscription_status to “PAUSED”.
  2. The Service Provider may apply Authentication and Authorization controls to determine the result of the subscription request.
  3. The Service Provider provides a Response with the result of the request including (if successful) the new subscription_status .

Post-Conditions: If successful, the subscription is paused.

Example Request Message

Headers

Accept: application/json, application/xml

Accept-Encoding: gzip

Body

{“topic”: <topic>,

"subscription_status": "PAUSED",

“qos”: “EXACTLY_ONCE”,

“durable”:true}

Example Response Message

Headers

Status-Code: 200

Reason-Phrase: Accepted

Content-Type: application/json

Body

{“topic”: <topic>,

"subscription_id": <subscription_id>,

"queue": <queue>,

"subscription_status": "PAUSED",

“qos”: “EXACTLY_ONCE”,

“durable”:true}

Error Codes

The following error codes may be of interest to implementers, covering common failures applicable to this resource invocation. Implementers are free to use a suitable subset of the ones proposed or complement it with additional ones to cover particular implementation needs.

400 Bad Request: Indicated for malformed requests on the part of the client.

401 Unauthorized: Indicated for authentication errors (e.g. the authentication is not provided, the credentials used do not exists, the authentication mechanism used is not valid…).

403 Forbidden: Indicated for authorization errors (e.g. the authenticated credentials are not allowed to perform this operation on the resource).

404 Not Found: The requested resource cannot be found (e.g. the <subscription_id> does not exist).

405 Method Not Allowed: The HTTP method used is not supported by the resource.

406 Not Acceptable: None of the Content-Types provided in the request Accept header are supported.

415 Unsupported Media Type: The Content-Type of the request is not valid for this operation.

500 Internal Server Error: Catch-all error for unexpected internal errors during the processing of the request.

resume

Due to the stateless implementation of the API, this operation is performed in exactly the same way as the pause changing the subscription_status in the body of the message to “ACTIVE”.

Figure 8: resume sequence diagram

Description:

Pre-Conditions: The Consumer has successfully subscribed to a particular <topic>.

  1. The Consumer invokes the “PUT” method on the /subscriptions/<subscription_id> endpoint. The body of the message includes a change to the subscription_status to “ACTIVE”.
  2. The Service Provider may apply Authentication and Authorization controls to determine the result of the subscription request.
  3. The Service Provider provides a Response with the result of the request including (if successful) the new subscription_status .

Post-Conditions: If successful, the subscription is paused.

Example Request Message

Headers

Accept: application/json, application/xml

Accept-Encoding: gzip

Body

{“topic”: <topic>,

"subscription_status": "ACTIVE",

“qos”: “EXACTLY_ONCE”,

“durable”:true}

Example Response Message

Headers

Status-Code: 200

Reason-Phrase: Accepted

Content-Type: application/json

Body

{“topic”: <topic>,

"subscription_id": <subscription_id>,

"queue": <queue>,

"subscription_status": "ACTIVE",

“qos”: “EXACTLY_ONCE”,

“durable”:true}

Error Codes

The following error codes may be of interest to implementers, covering common failures applicable to this resource invocation. Implementers are free to use a suitable subset of the ones proposed or complement it with additional ones to cover particular implementation needs.

400 Bad Request: Indicated for malformed requests on the part of the client.

401 Unauthorized: Indicated for authentication errors (e.g. the authentication is not provided, the credentials used do not exists, the authentication mechanism used is not valid…).

403 Forbidden: Indicated for authorization errors (e.g. the authenticated credentials are not allowed to perform this operation on the resource).

404 Not Found: The requested resource cannot be found (e.g. the <subscription_id> does not exist).

405 Method Not Allowed: The HTTP method used is not supported by the resource.

406 Not Acceptable: None of the Content-Types provided in the request Accept header are supported.

415 Unsupported Media Type: The Content-Type of the request is not valid for this operation.

500 Internal Server Error: Catch-all error for unexpected internal errors during the processing of the request.

unsubscribe

This operation allows a Consumer to unsubscribe from a particular Topic of the Service.

Figure 9: unsubscribe sequence diagram

Description:

Pre-conditions: The Consumer has successfully subscribed to a particular <topic>.

  1. The Consumer invokes the “DELETE” method on the /subscriptions/<subscription_id> endpoint.
  2. The Service Provider may apply Authentication and Authorization controls to determine the result of the subscription request.
  3. The Service Provider provides a Response with the result of the request.

Post-Conditions: If successful, the <queue> and /subscriptions/<subscription_id> endpoints are removed and the Consumer is unsubscribed.

Example Request Message

Headers

Accept: application/json, application/xml

Accept-Encoding: gzip

Body

None

Example Response Message

Headers

Status-Code: 204

Reason-Phrase: No Content

Body

None

Error Codes

The following error codes may be of interest to implementers, covering common failures applicable to this resource invocation. Implementers are free to use a suitable subset of the ones proposed or complement it with additional ones to cover particular implementation needs.

400 Bad Request: Indicated for malformed requests on the part of the client.

401 Unauthorized: Indicated for authentication errors (e.g. the authentication is not provided, the credentials used do not exists, the authentication mechanism used is not valid…).

403 Forbidden: Indicated for authorization errors (e.g. the authenticated credentials are not allowed to perform this operation on the resource).

404 Not Found: The requested resource cannot be found (e.g. the <subscription_id> does not exist).

405 Method Not Allowed: The HTTP method used is not supported by the resource.

500 Internal Server Error: Catch-all error for unexpected internal errors during the processing of the request.

getSubscription

This operation allows a Consumer to obtain information on a specific subscription.

Figure 10: getSubscription sequence diagram

Description:

Pre-Conditions: The Consumer has successfully subscribed to a particular <topic>.

  1. The Consumer invokes the “GET” method on the /subscriptions/<subscription_id> endpoint.
  2. The Service Provider may apply Authentication and Authorization controls to determine the result of the subscription request.
  3. The Service Provider provides a Response with the information associated to a particular subscription.
Example Request Message

Headers

Accept: application/json, application/xml

Accept-Encoding: gzip

Body

None

Example Response Message

Headers

Status-Code: 200

Reason-Phrase: OK

Content-Type: application/json

Body

{“topic”: <topic>,

"subscription_id": <subscription_id>,

"queue": <queue>,

"subscription_status": "ACTIVE",

“qos”: “EXACTLY_ONCE”,

“durable”:true}

Error Codes

The following error codes may be of interest to implementers, covering common failures applicable to this resource invocation. Implementers are free to use a suitable subset of the ones proposed or complement it with additional ones to cover particular implementation needs.

400 Bad Request: Indicated for malformed requests on the part of the client.

401 Unauthorized: Indicated for authentication errors (e.g. the authentication is not provided, the credentials used do not exists, the authentication mechanism used is not valid…).

403 Forbidden: Indicated for authorization errors (e.g. the authenticated credentials are not allowed to perform this operation on the resource).

404 Not Found: The requested resource cannot be found (e.g. the <subscription_id> does not exist).

405 Method Not Allowed: The HTTP method used is not supported by the resource.

406 Not Acceptable: None of the Content-Types provided in the request Accept header are supported.

415 Unsupported Media Type: The Content-Type of the request is not valid for this operation.

500 Internal Server Error: Catch-all error for unexpected internal errors during the processing of the request.

getTopics

This operation allows a Consumer to request the list of topics available for subscription.

Figure 11: getTopics sequence diagram

Description:

  1. The Consumer invokes the “GET” method on the /topics/ endpoint.
  2. The Service Provider may apply Authentication and Authorization controls to determine the result of the subscription request.
  3. The Service Provider provides a Response with the information of all the topics available for subscription.
Example Request Message

Headers

Accept: application/json, application/xml

Accept-Encoding: gzip

Body

None

Example Request Message

Headers

Status-Code: 200

Reason-Phrase: OK

Content-Type: application/json

Example 1: Topic Enumeration

Body

{“type”: “Enumeration”,

“topics”: [“FLIGHT”, “WEATHER”, “AERONAUTICAL_INFORMATION”] }

Example 2: Topic Tree

Body

{“type”: “Tree”,

“patternMatching”: {“wordSeparator”: “/”,

“singleWordWildcard”: “*”,

“multiWordWildcard”: “#”},

“topics”: [“/weather/precipitation/LEMD”,

“/weather/precipitation/LFPG”,

“/weather/wind/LEMD”,

“/weather/wind/LFPG”]}

Error Codes

The following error codes may be of interest to implementers, covering common failures applicable to this resource invocation. Implementers are free to use a suitable subset of the ones proposed or complement it with additional ones to cover particular implementation needs.

400 Bad Request: Indicated for malformed requests on the part of the client.

401 Unauthorized: Indicated for authentication errors (e.g. the authentication is not provided, the credentials used do not exists, the authentication mechanism used is not valid…).

403 Forbidden: Indicated for authorization errors (e.g. the authenticated credentials are not allowed to perform this operation on the resource).

405 Method Not Allowed: The HTTP method used is not supported by the resource.

406 Not Acceptable: None of the Content-Types provided in the request Accept header are supported.

415 Unsupported Media Type: The Content-Type of the request is not valid for this operation.

500 Internal Server Error: Catch-all error for unexpected internal errors during the processing of the request.

getTopic

This operation allows a Consumer to obtain information on a specific topic.

Figure 12: getTopic sequence diagram

Description:

  1. The Consumer invokes the “GET” method on the /topics/<topic> endpoint.
  2. The Service Provider may apply Authentication and Authorization controls to determine the result of the subscription request.
  3. The Service Provider provides a Response with information on the specific <topic>.
Example Request Message

Headers

Accept: application/json, application/xml

Accept-Encoding: gzip

Body

None

Example Request Message

Headers

Status-Code: 200

Reason-Phrase: OK

Content-Type: application/json

Body

{“topic”: “WEATHER”,

“updateInterval”: “PT4H”,

“timeToLive”: “PT4H”,

“contentType”: “application/xml+iwxxm3.0”,

“contentEncoding”: “gzip”}

Error Codes

The following error codes may be of interest to implementers, covering common failures applicable to this resource invocation. Implementers are free to use a suitable subset of the ones proposed or complement it with additional ones to cover particular implementation needs.

400 Bad Request: Indicated for malformed requests on the part of the client.

401 Unauthorized: Indicated for authentication errors (e.g. the authentication is not provided, the credentials used do not exists, the authentication mechanism used is not valid…).

403 Forbidden: Indicated for authorization errors (e.g. the authenticated credentials are not allowed to perform this operation on the resource).

404 Not Found: The requested resource cannot be found (e.g. the <topic_id> does not exist).

405 Method Not Allowed: The HTTP method used is not supported by the resource.

406 Not Acceptable: None of the Content-Types provided in the request Accept header are supported.

415 Unsupported Media Type: The Content-Type of the request is not valid for this operation.

500 Internal Server Error: Catch-all error for unexpected internal errors during the processing of the request.

Publication

This section defines an implementation of the Publish interface of the agnostic service design using AMQP 1.0. Reference [5] provides an overview of the most important aspects of AMQP 1.0 which are used in the implementation.

Technical Interfaces

The following interfaces provide technical operations to perform connection establishment, control flow of messages and settle their delivery state. From the technology agnostic service design described in section 2 these are technical details of implementation specific to the technology that are abstracted from the high-level service description.

Figure 13: AMQP 1.0 Technical Interfaces

Publish Interface

The publish operation of the agnostic service design corresponds with the transfer performative as depicted below:

Design

Implementation

Operation

Resource endpoint

AMQP 1.0 method

publish

/<queue>

transfer

The following figure depicts this interface and highlights the close correspondence with the technology agnostic Publish interface of section 2.1.

Figure 14: AMQP 1.0 Publish Interface

publish

This operation allows the Service Provider to push publications to a Consumer.

Figure 15: AMQP 1.0 publish operation

Description:

Pre-conditions: The Consumer has subscribed to a <topic> and connected to the <queue> provided in the subscribe response message. There is sufficient link-credit in the consumer���s link endpoint.

  1. The Provider sends a message with the appropriate payload using the “transfer” performative of AMQP 1.0.

Post-Conditions: If successful, the message is received.

Example Message

Message(

properties:

{

"message‐id": <message_id: e.g. int>,

"to": "<consumer_address>",

"content-type": <content-type of payload e.g. application/xml>

"content-encoding": <content-encoding of payload e.g. deflate>

},

application-properties:

{

// Relevant application properties, service dependent. E.g.:

"message-type": "ARRIVAL",

"gufi": "d6fc238a-0bdd-4854-9010-d9c1ca4edc48"

},

application-data:

(

// Body of the payload. Service dependent

)

)

Dynamic Behaviour

The following diagram shows the nominal flow followed by a Consumer that connects to a queue to receive message publications.

Figure 16: AMQP 1.0 Sequence Diagram

  1. The Consumer sends an open performative to create a connection with the Service. The open performative passes as one of its parameters the hostname of the Service.
  2. The Consumer sends a begin performative to start a session with the Service.
  3. The Consumer sends an attach performative to create a directional link from the queue to the Consumer’s endpoint to enable the transfer of messages.
  4. The Consumer sets some available link-credit using the flow performative.
  5. When a new message is available in the queue and while credit is available, the message is pushed from the queue to Consumer’s endpoint using the transfer performative.

Finally, the Consumer may decide to disconnect from the queue.

  1. The Consumer sends a detach with the identifier of the link he attached to queue.
  2. The Consumer sends an end performative to finish the session established.
  3. The Consumer send a close performative to terminate the connection with the Service.

Note: Steps 1 to 3 and 6 to 8 provide a low-level description of the connection and disconnection process at protocol level. AMQP 1.0 clients and libraries may simplify these processes by providing a higher level API that abstracts the different protocol invocations. From the point of view of the Consumer the most important aspect is that he has the initiative to connect to the queue and that the network address of the service and the queue endpoint are used during these steps to establish a connection.

References

  1. AMQP Essentials, Paolo Patierno
  2. OASIS Advanced Message Queuing Protocol Version 1.0, OASIS
  3. Eurocontrol SWIM Technical Infrastructure Yellow Profile Specification, EUROCONTROL
  4. Eurocontrol SWIM Technical Infrastructure Foundation Material, EUROCONTROL
  5. AMQP Messaging Service Binding: Technology Overview, EUROCONTROL
  6. SWIM TI Binding Selection Guidelines, EUROCONTROL
  7. SWIM TI Message Exchange Pattern Identification Guidelines, EUROCONTROL
  8. Authentication and Authorization

Authentication

The Interface Service Bindings used for the implementation proposed in section 3, WS Light and AMQP Messaging, offer equivalent authentication mechanisms:

  1. Mutual authentication with X.509 certificates
  2. Server authentication with X.509 and Client authentication with Username/Password
  3. Server authentication with X.509 and anonymous Client authentication

Of the available authentication mechanism only 1 and 2 are applicable to a Publish/Subscribe service design, as there needs to be a way to authenticate a consumer to perform a subscription.

Option 1 (Mutual authentication with X.509 certificates) utilizes the same technical solution to perform authentication across both Service Bindings, relying on TLS authentication and X.509 certificates. On the other hand it requires an existing Public Key Infrastructure with the overhead associated to its management and the deployment of X.509 certificates on the consumers.

Option 2 (Server authentication with X.509 and Client authentication with Username/Password) relies on HTTP Basic Authorization for the subscription interface and SASL PLAIN for the publication interface to authenticate the user with its username and password. The advantage of a Username/Password authentication of the client is its simplicity and low management burden but it provides a lower security protection against brute-force attacks in comparison to public key certificates.

The choice of one authentication mechanism over the other is a design decision of the service provider who can take into consideration the trade-offs associated with both solutions.

Authorization

Authorization controls should be implemented in the resources exposed by the service. The following are one possible set of authorization controls:

  • Resource “/subscriptions/”: If a pre-existing community of interest exists for the Publish/Subscribe service, the service can verify that the credentials of the authenticated user correspond to an authorized member. Otherwise, the resource can be accessible to every authenticated user.
  • Resource “/subscriptions/<subscription_id>”: The credentials of the authenticated user must correspond with the user that invoked the subscribe operation that resulted in the creation of the endpoint.
  • Resource “/topics/”: If a pre-existing community of interest exists for the Publish/Subscribe service, the service can verify that the credentials of the authenticated user correspond to an authorized member. Otherwise, the resource can be accessible to every authenticated user.
  • Resource “/topics/<topic_id>”: If a pre-existing community of interest exists for the Publish/Subscribe service, the service can verify that the credentials of the authenticated user correspond to an authorized member. Otherwise, the resource can be accessible to every authenticated user.
  • Resource “/<queue>”: The credentials of the user attempting to connect must match the credentials associated to the <subscription_id> that owns the queue.
  1. Technical Messages

The publish interface can be used not only for the exchange of business messages for which the subscriber has declared interest during the subscription but also for technical messages that are related to the status of the subscribe or publish interfaces themselves. In this appendix we propose various technical messages that provide the consumer with a means to obtain updates on the status of the service interfaces.

Heartbeats

Heartbeat messages are periodic messages that allow its recipient to determine that the sender is still alive. The use of heartbeat messages can aid mitigate the intrinsic uncertainty of asynchronous MEPs where a consumer has no way to tell if the producer is still alive and operational.

A heartbeat message typically contains a very minimal amount of information as the reception of the message itself is sufficient to acknowledge that the sender is still able to produce and send the message.

Layers

Heartbeat messages can be implemented at multiple layers of the protocol stack. A heartbeat at a particular layer of the protocol stack informs that the stack is alive up to that layer but doesn’t tell anything of the upper layers of the stack that may have failed but were not involved in the production and delivery of the message.

TCP

At the TCP layer the participants can send TCP keep-alives to ensure the connection remains open and implicitly inform they are still alive.

AMQP 1.0

The AMQP 1.0 Specification proposes the use of empty frames (section 2.4.5 of [2]) to serve the purpose of a heartbeat or keep-alive. AMQP 1.0 clients that support this (e.g. Apache Qpid Proton) allow setting the periodicity of the heartbeat upon connection establishment.

Heartbeats at this layer inform us the AMQP 1.0 connection and link are still operational and that at least the messaging infrastructure of the sending peer is working.

Application

To verify that the application is still alive we can define a simple AMQP 1.0 message that contains some data in its application sections. One option is to use the application-properties section of the AMQP 1.0 message to provide some minimal information that communicates the sending application is alive.

The following example provides such a message, consisting of the following application-properties:

  • “message-type” set to “HEARTBEAT” and
  • “timestamp” set to the UNIX-time of the message delivery.

Because this message contains payload produced by the application it communicates to the receiving peer that the application is still alive.

Heartbeat Message:

Message(

properties:

{

},

application-properties:

{

"message-type": "HEARTBEAT",

"timestamp": "1562075995"

},

application-data:

(

)

)

This message is sent with some pre-defined periodicity, defined on service design. When a subscriber fails to receive the heartbeat message for a number of periods (e.g. more than 3 consecutive messages) he may assume that a problem exists at the application layer of the sending peer.

Subscription State

The subscriber can control the state of his subscriptions through the subscribe interface as described in section 2.1. The Service Provider itself may also want to control the state of a subscription, for example to “PAUSE” it if the queue of the consumer is getting filled.

When doing so the Service Provider may want to use a technical message to communicate the new state of the subscription and let the subscriber know that a state change has occurred.

The following example uses the same JSON structure used in the response messages of the subscribe interface (section 3.1.1).

Subscription State Message:

Message(

properties:

{

"message‐id": <message_id: e.g. int or uuid>,

"to": "<consumer_address>",

"content-type": “application/json”

},

application-properties:

{

"message-type": "SUBSCRIPTION",

},

application-data:

(

{“topic”: <topic>,

"subscription_id": <subscription_id>,

"queue": <queue>,

"subscription_status": "PAUSED",

“qos”: “EXACTLY_ONCE”,

“durable”:true }

)

)

Notes

  1. For some services the subscription process may not be a dynamic process and can be completely set during design time. In this case there would not be an endpoint that can be dynamically accessed to subscribe or unsubscribe, any change to the subscribers would require a new deployment of the service.

  2. To be noted that although the example operation is stateful, the technical implementation is stateless thus fulfilling the REST constraint of service design.