Data stream prediction in distributed stream processing environment

With the wide adoption of distributed stream processing, the requirement of guaranteeing QoS has been raised to a new standard. Since node load influences QoS directly, it is a hotspot of research. Through analyzing the relationship between input data stream and load of physical node, this paper abstracted a local node model from traditional distributed stream processing network. Based on this model, a new data stream prediction algorithm grounded on a classic machine learning algorithm — Share Algorithm — is proposed. The new algorithm uses recent data stream as the prediction resource and efficiently accomplishes the prediction of single nodes in the future period. Our experiments show that 73% of predictions can achieve the accuracy more than 90% with some common data traces.