Modelling Message-Oriented-Middleware Brokers using Autoregressive Models for Bottleneck Prediction.
Abstract
Message brokers are the backbone of modern publish subscribe messaging systems. These brokers
can degrade or fail for a variety of reasons. This research specifically looks at the detection,
prediction and mitigation of bottlenecks in brokers. The message-oriented-middleware framework
here uses either a cluster of brokers on a Local Area Network (LAN) or a federation of brokers on a
Wide Area Network (WAN) to route messages, facilitate multicasting and ameliorate demand
surges and geographically related faults. Sensors have been constructed to monitor brokers and
controllers to run the bottleneck detection algorithms. An overlay manager controls broker and
topic pairing. Each topic is assigned a primary and secondary broker. When a failure is predicted,
the overlay manager routes messages from the failing broker by switching topics to its secondary
broker(s). The application for bottleneck forecasting is to allow us to pre-empt a broker failure and
hence reroute messages to other brokers to increase resilience and reliability.
The key contributions of this research are an abstract model of message-oriented-middleware
broker based on the Apache Qpid message broker coupled with the use of analytical autoregressive
exogenous (ARX) models that describes the broker behaviour during bottleneck conditions. The
Apache Qpid message broker is a message broker that implements the Advanced Message Queuing
Protocol (AMQP) for publish-subscribe messaging. ARX models are autoregressive models where
the output depends on the previous output as well as external stimuli. These components are
integrated to produce a generalised technique for calibrating broker performance and detection of
bottlenecks in the broker.
This research show how models were initially constructed using a complete range of input data. As
bottlenecks occur only when the broker is heavily loaded, input data during idle periods can cause
corruption to the model fit. Models were constructed with segmented input data, with each segment
covering the range of one peak period. The segmented input allows the modelling of the broker
behaviour only when it is experiencing a bottleneck. The result of this is a much-improved fit of the
predictive models.
The work here is compared against previous work using Markov-chains for creating predictive
models. The results of both approaches are compared and reported.
Authors
Chew, Zhen BobCollections
- Theses [3711]