Reading the Robot Mind: Deep Convolution Networks Decoded


Decoding of information at successive convolution layers from a deep network trained to recognize Bengali handwriting - PAN Feb 2020

Even as "Explainable AI" is all the rage, coding/transformation fidelity is a critical success factor. Whether you are using frequency bands from Fourier transforms, statistical features of Wavelet decomposition, or various filters in Convolution Networks - researchers must be able to perform the reverse coding/transformation to see if they have retained sufficient information for classification. Without this, they are only guessing via network architecture trial and error. These tools are sorely lacking in Keras on Tensorflow in Python, so I wrote my own. I would like to see these more generalized and made into public libraries.


Question: Who can point me to current work in this area, or can give advice on next steps in my effort?


EXAMPLE: In a deep convolution network trained to recognize Bengali handwriting, this diagram examines the 64 filters after six layers of 2 dimensional convolution. It performs the reverse transformations to show exactly what tiny portion of the handwritten image the filter is looking for.
Decoded filters from a convolution layer inside a deep network trained to recognize handwriting - PAN Feb 2020
EXAMPLE: In a deep convolution network trained to recognize Bengali handwriting, this diagram examines the internal representation of input figure. It performs the reverse transformations to show exactly what info has been retained after successive layers of convolution (as well as other data reduction through strides and pooling).
Decoding of information at successive convolution layers from a deep network trained to recognize Bengali handwriting - PAN Feb 2020


Please see this public Notebook for the details of this work:

https://www.kaggle.com/pnussbaum/grapheme-mind-reader-panv12-nogpu

Comments

Post a Comment

Popular posts from this blog

Image
Reverse Inference Example Software:  This points to example software on GitHub where you can see some of these concepts in action. Here is the link: https://github.com/prof-nussbaum/Reverse_Inference  Reverse_Inference Working backwards through a deep convolutional network, to recreate the input image - and identify areas for improvement. Please see this article for more details on this technique, that I call "Reading the Robot Mind." https://readingtherobotmind.blogspot.com/2021/03/reading-robot-mind-deep-convolution.html What if we could read the robot's mind? It sounds like a silly statement, but if you could read a person's mind, you could "see" what they were thinking of when they mention a classification. If they say "I am seeing a dog" - reading their mind would give you additional details about the dog, or even perhaps, see what they are seeing. This is the same premise for Artificial Intelligence and Machine Learning. I know that deep conv...
Read more
Image
Auto-Associative How To - INCLUDING SOFTWARE CODE This repository includes "how to" examples of auto-associative neural networks. These networks have the ability to encode and decode (CODEC) sample data in the most efficient and low-loss way. The minimization of the loss function is accomplished through the use of gradient descent training algorithms. Example 1 - Auto-Associative How to (using TGS Salt Columns) The first "how-to" example comes from a competition held on the kaggle.com website. WHAT IS THE OVERARCHING GOAL? The competition, as well as details about the data set used for training, can be found here: https://www.kaggle.com/c/tgs-salt-identification-challenge . The purpose of the competition is to see who best can create an AI system that classifies seismic data. There are human experts who can do this classification from the original data, and indeed have done so to create the training and testing data. WHY USE AUTO-ASSOCIATIVE NETWORKS? In many case...
Read more
Auto-Encoder-CODEC This is a continuation of Auto-Encoder or Auto-Associative How To Examples This repository includes "how to" examples of auto-associative neural networks. These are sometimes called Auto-Encoders. These networks have the ability to encode and decode (CODEC) sample data in the most efficient and low-loss way. The minimization of the loss function is accomplished through the use of gradient descent training algorithms. Example 3 - MNIST Digit ConvAutoencoder This "how-to" example comes from a competition held on the kaggle.com website. WHAT IS THE OVERARCHING GOAL? Recognize handwritten digits from the MNIST dataset. Do it with CNN generated "features" or "encoded" version of the input - the center of an autoencoder. The subject matter expert (SME) can recognize digits. They qualitatively rate the amount of data lost due to encoded compression. If the SME can no longer make a classification from a decoded input, they qualitativel...
Read more