Optimizing Neural Network Structures Using Distributive Genetic Algorithms

The tools of Artificial Intelligence are more and more widely used in modeling of tehnological processes. One of the best useable methods of artificial intelligence is the theory of the artificial neural networks and from this kind of networks the most widespread is the feed-forward back-propagation network. This kind of network is able to process large amounts of measurement data and acquire relationship between input and output data automatically. That is why it is preferably applied on such places where these relationships cannot be formulated in closed form.

The biggest difficulty during using these neural networks is that in general the value of several network parameters has to be defined during planning. The most frequent parameters are: the learning rate, the momentum, the number of layers, the number of neurons in different layers, the type of transfer functions, the number of training cycles, etc. Generally, to define these parameters we cannot have any support even in the scientific literature because there are a lot of times opposite expressions and in a few cases the experiences are not in accordance with the guidelines in the scientific literature. Building up a good model based on a neural network is a time consuming activity, a process full of a lot of experiments and deadlocks. As the spreading of the artificial intelligence tools is to be expected in a wider and wider range therefore we must make possible that also for experts inexperienced in this field shall be available this technology. So it seems to be needed to automatize the planning process of the neural networks in a so wide range as possible.

For planning automatization there are no useable algorithms and the development of good algorithms doesn’t seem to be possible in the near future. The solution can be provided by an other - from the artificial intelligence already known - method. These are called Genetic Algorithms. These algorithms are able to do complicated optimization tasks with many variables quickly and efficiently. The scientific literature nowadays is dealing with the combination of genetic algorithms and neural networks as well but these applications are focusing only on accelerating the learning process and not automatic optimizing of the model’s characteristics.

The optimization process based on genetic algorithms is a calculation consuming process that is why it will need a strong computer background. One possible way to eliminate this problem is to use better the current computer capacity. In the most places more computers working together connected in a network generally. The calculation resources utilizations of these machines are rather low and in a given period of day (e.g.: at night) extremely low. If the rest computer calculation capacity could be added together, the resultant calculation performance would be higher in case of connecting enough machines than the performance of the middle class workstation and could mess with the power of a mainframe calculation. The virtual adding of computing power can be realized with the per machine division of the problem (in this case the problems to be solved by genetic algorithms).

The developing of the automatization of neural network model planning can be realized in three steps. The first step is to adapt the genetic algorithms to the optimization of neural network structure. It means the selection of the neural network parameters to be optimized, coding into genes and finding the proper mutation and crossover operators. The second step is to make suitable the genetic algorithm for running on several machines with the utilization of the network traffic and computing power as good as possible. The third step is to develop software and network communication for different operation systems and to develop a server program to supervise the whole system for a selected operation system.