Борис Злотин – PANN: A New Artificial Intelligence Technology. Handbook (страница 2)
4. A trained image neuron can be easily visualized using simple color codes linking the included weights’ levels to the pixels’ brightness or color.
2.3. THE CURIOUS PARADOX OF PANN
The PANN network is simpler because:
1. The Rosenblatt neuron has an activation factor; in other words, the result is processed using a nonlinear logistic (sigmoid) function, an S-curve, etc. This procedure is indispensable, but it complicates the Rosenblatt neuron and makes it nonlinear, which leads to substantial training problems. In contrast, the Progress neuron is strictly linear and does not cause any issues.
2. The Progress neuron has an additional element called a distributor, which is a simple logic device: a demultiplexer. It switches the signal from one input to one of several outputs. In the Rosenblatt neuron, weights are multi-bit memory cells that can store numbers over a wide range, while in PANN, the most superficial cells (triggers) can be used, which can store only the numbers 1 and 0.
3. Unlike classic networks, PANN does not require huge memory and processing power of a computer, so cheap computers can be used, and much less electricity is required.
4. PANN allows you to solve complex problems on a single-layer network.
5. PANN requires tens or even hundreds of times fewer images in the training set.
Thus, it is possible to create full-fledged products based on PANN, using computer equipment that is not very expensive and economical in terms of energy consumption.
Fig. 5. Long and expensive training vs. fast and cheap
2.4. THE MATHEMATICAL BASIS OF RECOGNITION
ON THE PROGRESS NEURON
In 1965, Lotfi Zadeh introduced the concept of “fuzzy sets” and the idea of “fuzzy logic.” To some extent, this served as a clue for our work in developing PANN’s mathematical basis and logic.
In 2009, an exciting discovery was made called the “Marilyn Monroe neuron” or, in other sources, “grandmother’s neuron.” In the human mind, knowledge on specific topics is “divided” into individual neurons and neuron groups, which are connected by associative connections so that excitation can be transmitted from one neuron to another. This knowledge and the accepted paradigm of “one neuron, one image” made building the PANN recognition system possible.
Let’s introduce the “neuron-image” concept –
X is a numerical array (vector) with the following properties:
for X = A, f (A) = N
for X ≠ A, f (A) <N
A is a given value.
N is the dimension of vector X, the number of digits in this vector.
This format, called the Binary Comparison Format (BCF), is a rectangular binary digital matrix in which:
• The number of columns is equal to the length N (the number of digits) of the array.
• The number of rows equals the number of weight levels K selected for the network.
• Each significant digit is denoted by one (1) in the corresponding line, and the absence of a digit is denoted by zero (0).
• Each string corresponds to some significant digit of the numeric array to be written, i.e., in a string marked as “zero,” the digit “1” corresponds to the digit “0” in the original array, and in a string marked as “ninth,” the digit “1” corresponds to the digit 9 in the array.
• In each column of the matrix, one unit corresponds to the value of this figure, and all other values in this column are equal to 0.
• The sum of all units in the array matrix is equal to the length N of the array; for example, for an array of 20 digits, it is 20.
• The total number of zeros and ones in the matrix of each array is equal to the product of the length N of this array and the value of the base of the number system used.
Example: BCF notation of an array of 20 decimal digits [1, 9, 3, 6, 4, 5, 4, 9, 8, 7, 7, 1, 0, 7, 8, 0, 9, 8, 0,2].
Fig. 6. BCF image as a sparse binary matrix
A feature of the PANN network is that the image training of neurons typical of neural networks can be replaced by reformatting files that carry numerical dependencies to the BCF format or simply loading files in this format to the network.
Type X arrays in BCF format are denoted as matrices |X|.
Determining the similarity of particular objects by comparing them plays an enormous role in thinking, making it possible to identify analogies and differences between different objects – beings, objects, processes, ideas, etc. In various branches of science, primarily in the Theory of Similarity, dimensionless similarity coefficients or similarity criteria (Similarity Coefficient or CoS) are used, sometimes called the “measure of similarity,” the “measure of association,” and so on.
Comparison functions in PANN are implemented through mathematical operations on matrices of numeric arrays. Let’s consider the most straightforward comparison algorithm, which uses the vector product of image neuron matrices.
Two arrays are given for comparison in the form of matrices |X1| and |X2|.
|X1| × |X2|T is the vector product of the matrix |X1| on a transposed matrix |X2|. Moreover, the value of this product is proportional to the number of units in |X1| and |X2|.
|X1| × |X2|T = N only if |X1| = |X2|;
|X1| × |X2|T <N if |X1| ≠ |X2|;
|X1| × |X2|T = 0 if none of the pixels of these matrices match.
Consider the relationship:
Here, the CoS (Similarity Coefficient) between the numerical vectors X1 and X2 determines the degree of closeness of these vectors and the images described by these vectors.
Examples:
Fig. 7. Multiplying matrices to compare numerical arrays
Fig. 8. Comparison of decimal numerical arrays |A| and |B|
Classical neural networks only determine which class a recognizable object is most similar to. At the same time, they cannot specify how similar it is. Because of this, recognition is sometimes unstable – there are well-known examples where a change in a pixel in an image was enough to change its recognition. Thus, recognition in classical networks is highly dependent on random noise.
In PANN, the situation is different – the similarity coefficient value very clearly shows how significant the difference between the images is. A similarity difference of one hundredth in the format of 32 × 32 pixels corresponds to a change of about 10 pixels. And this is already enough to distinguish the images from each other confidently. The one-tenth difference indicates a profound difference and high recognition stability – low dependence of recognition on noise.
In contrast to classical neural networks, PANN networks allow you to improve the quality of recognition dramatically by:
• Statistical processing of recognition by classes and by images.
• Combining class-based recognition and image-based recognition. Moreover, combined recognition by classes and images allows us to solve one of the most unpleasant problems that limit the use of neural networks in medicine and many other applications – the problem of transparency and explainability of the network results. We will discuss this in l in the “4.6. Recognition on the PANN Network” section.
The accuracy and reliability of recognition of a classical neural network are determined by testing several dozens, hundreds, or thousands of images and counting the number of correct and incorrect recognitions. This test is very controversial. Due to the opacity of classical networks, recognition is highly dependent on random training features:
• Sometimes, training outcomes are poorly reproduced; the same network trained on the same images will recognize better in some cases than worse in others.
• There are no ways to assess each image’s accuracy and recognition reliability adequately.
• Impact of test image selection. Sometimes, they are selected specifically to ensure the desired result.
Recognition by PANN networks is evaluated by the numerical similarity coefficient of the image under consideration:
1. With any set of individual images loaded to the network.
2. With all classes that this network is trained in.
At the same time, both classes and individual images are ranked according to the degree of similarity, which allows for an accurate assessment of the magnitude of the differences between all the compared classes and, thereby, assessment of the accuracy and reliability of recognition.