Downstream

    For some applications, capturing registration and modification events from Compound Registration plays an important role. Applications that are downstream consumers of these events have two ways to extract messages from the system:

    • Using a JMS compatible message broker such as ActiveMQ or ActiveMQ Artemis
    • Using the Downstream API (available since version 23.8.0-2306091339)

    To choose between the two downstream modes, set the RegDownstreamMode registry property:

    • RegDownstreamMode=ActiveMQ will enable JMS downstream (default option)
    • RegDownstreamMode=Database messages will be permanently stored in the DOWNSTREAM_MESSAGE table and the message query HTTP API becomes available

    To enable downstream message publishing or change the structure format, set the RegDownstreamPublishEnabled and RegDownstreamFusedImageFormat registry properties:

    Property name Example value Description
    RegDownstreamPublishEnabled true This enables downstream message publishing
    RegDownstreamMode ActiveMQ The active downstream mode. It can be ActiveMQ or Database (case sensitive)
    RegDownstreamFusedImageFormat mol:V3 This option changes the structureWithSalt image format in the message (other examples: mrv, base64:svg)

    These can be provided via the registry.properties file (war deployment) or as environment variables (docker deployment).

    ActiveMQ

    ActiveMQ is a popular open-source, Java based message broker. It is responsible for seamless communication between a producer and one or multiple consumers. In version 21.3.0-2103251330, several new properties were introduced in Compound Registration to support connecting to remote ActiveMQ brokers.

    The message flow between the producer and the consumer is asynchronous. Compound Registration generates, and with the use of the JMS protocol it publishes messages to the broker. From that point, the rest of the delivery process is up to the broker and the consumer.

    Here is a short overview of the messaging architecture:

    ActiveMQ downstream

    1. Compound Registration receives a registration or modification request.
    2. After processing the request the result is saved into the database.
    3. The system collects information about the affected parent compounds and generates downstream messages which are then published to ActiveMQ.
    4. The consumer receives the messages.

    The message format is XML, and it contains all version and lot level compounds that belongs to the parent compound where the registration or modification event happened. At the following link, you can find an example message.

    Any message will disappear from the broker if the consumer successfully received it. It can be useful to send some messages again or synchronize the full compound library. For that the following APIs can be useful:

    • /downstream/publishAll
    • /downstream/publishTrees
    • /downstream/status

    Configuration

    In order to connect to a JMS compatible message broker, the following properties are needed:

    Property name Example value Description
    RegDownstreamJmsPersistent true If enabled, messages are persisted to disk/database so that they will survive a broker restart
    RegDownstreamJmsDestinationName REGISTRATION.DS Destination queue name(s). If you have multiple DS clients, add one for each of those in this format: RegDownstreamJmsDestinationName=REGISTRATION.DS1,REGISTRATION.DS2 and configure the clients to use their dedicated queue
    RegDownstreamJmsBrokerURL tcp://compreg-broker-host:61616 The URL of the message broker
    RegDownstreamJmsUser compreg This will be used by the application to connect to the message broker
    RegDownstreamJmsPassword qu28HDK4uE1Q The password for RegDownstreamJmsUser

    Downstream API

    The Downstream API listens to the same events as it's ActiveMQ counterpart, but there are some differences between the two:

    • It doesn't require additional infrastructure
    • Uses HTTP instead of JMS
    • Messages are stored permanently
    • The client has to keep record and poll for new messages

    Here is a short overview of the messaging architecture:

    API downstream

    1. The highlighted messages, message-1 and message-2 were already present in the system.
    2. Compound Registration receives a registration or modification request.
    3. The system collects information about the affected parent compound and generates the downstream message which is then stored in the database.
    4. The downstream client is polling the Compound Registration API for events and downloads new messages if available.

    Accessing messages:

    GET [http|https]://<host>:<port>/RegistryCxn/rest/downstream/messages?fromId=<fromId>&limit=<limit>

    • fromId:
      • is the last message identifier the client processed
      • the response will exclude the message with the provided identifier
      • the parameter is not required, in the absence the first batch of messages are returned
    • limit:
      • is the number of messages Compound Registration should return
      • the parameter is not required, the default value is 50
      • exceeding 100 the request returns an error
    • The API user must have DOWNSTREAM role

    Notes:

    • The client can also store messages or the last processed identifier
    • Frequent polling can impact system performance, wait a few seconds between requests
    • The client should be able to handle when Compound Registration is not available
    • Messages are permanently stored in Compound Registration, the same message can be read multiple times

    The message format is JSON, and it contains all version and lot level compounds that belongs to the parent compound where the registration or modification event happened. At the following link, you can find an example message.

    Resources