Design and Implementation of the Distributed System Using an Orchestrator Based on the Data Flow Paradigm

The object of this research is distributed systems under the control of an orchestrator based on the data flow control paradigm, as well as micro-service management methods. One of the most problematic places of modern distributed systems is the choice of a method for controlling the logic of the micro-service and the processes of interaction between them. The existing concepts of micro-service orchestration and choreography do not allow to fully use and distribute the load evenly throughout the system, which is primarily due to the heterogeneous nature of the distributed environment.As part of the research, the concept of hybrid orchestration based on the paradigm of data flow control is proposed. This approach allows the orchestrator to be used only to initiate a «wave» of calculations on the micro-service tree, and the micro-services themselves are responsible for the further calculation and dissemination of data. This approach, unlike others, combines the more optimal qualities of orchestration: simple and understandable, at each stage of calculation, system management, coordinated micro-service actions. Also, the use of a specialized hybrid orchestrator eliminated one of the main drawbacks, namely, it reduced the responsibility and the amount of computational burden assigned to the distributed system orchestrator, and computation nodes. As a result of an experiment using a distributed system with an orchestra, based on the data flow control paradigm, a several-fold decrease in the load on one orchestra was achieved. This makes it possible to use micro-controllers such as ESP8266, ESP32, Raspberry Pi as a distributed system. Such micro-controllers can act not only as an orchestrator, but also as data-flow nodes. At the same time, the data flow control paradigm allows to evenly and efficiently distribute the load on the system due to the fact that the input data of the system are presented in the form of a computational graph, where each node is a separate micro-service