One of the major tools of artificial intelligence is the neural net, which models the way the neurons in our heads behave. Although the implementation of a neural net on the computer is mathematically complicated, below I describe an example that closely models the way that implementation works but is easier to understand. This will enable me to answer three questions about neural nets: (1) Do they make unbiased decisions? (2) Can we know the rules a specific neural net uses when making a decision? (3) Do neural nets think?
Suppose you want to organize a group of people to decide whether to grant a mortgage after reading the application. You decide to create a number of teams to read the application. Each team evaluates a different factor affecting the possibility of default; each team member evaluates a different component of that factor. For example, one team might evaluate the finances of the applicant. Another team might evaluate the future financial climate, etc. For the team evaluating the applicant’s finances, one member might evaluate the applicant’s current income. Another team member might evaluate the applicant’s assets, etc. A team then makes a team decision via a yes-or-no vote. These team decisions then become individual votes on a final yes-or-no decision. This process may seem cumbersome, but it closely resembles the structure of a neural net.
An important feature of this voting procedure is that the votes are weighted according to how successful the voters have been in the past in predicting that an applicant avoided defaulting on the mortgage loan. You start by allocating each team member 1,000 votes and each team 1,000 votes. When a loan is in default, those team members who voted to grant the loan lose a few of their votes, while those team members who voted against granting the loan gain a few votes. Conversely, when the loan is not in default those who voted yes gain a few votes, and the naysayers lose a few votes. The same actions are taken for the votes of the teams. In neural net terminology this “feedback” is the mechanism for “reinforcement.”
Now, assume that all the members of the teams have lots of leisure and we can train their network by having them read the applications of thousands of mortgage applications without telling the team members that they were already given a green light (ignoring those that have been rejected) and whose fate – successful completion or default – have been recorded. After they read an application, they make a decision, and the number of votes they have are adjusted. After this training, the system stabilizes and can now be used to evaluate new applications.
In technical terms, the members of the teams are “nodes” (objects that receive or send data) as are the team captains. The number of votes each team member has measures the strength of the “connection” between the team member node and the captain node. And the number of votes the captains have measure the strength of the “connection” between the captain nodes and the overall decision, itself a node. The member nodes constitute one “layer,” the captains a second layer, and the decision a third layer. In practice, neural nets can have various numbers of layers.
Now, to answer the questions I posed at the beginning.
(1) Do they make unbiased decisions?
For the sake of discussion, assume the members of the net above are all unbiased. The choice of the training set (the thousands of mortgage applications) may bias the net’s output. For example, if most of the mortgage applicants were white males, the net may be trained to ignore relevant characteristics of female or black applicants. Then the net may be biased in favor granting mortgages to those applicants or it may be biased in favor of denying their applications.
(2) Can we know the rules a specific neural net uses when making a decision?
An algorithm is a set of rules for solving a problem. For example, in tic-tac-toe, one rule states that the person making the first move can avoid losing by placing a mark in the center box. In the example above I state an algorithm for a mechanism for deciding when to grant a mortgage. But that algorithm tells us nothing about the actual basis for the decision. In that sense, I have designed a “black box” process that provides no insight into the reasoning behind the decision; its decision algorithm is hidden. Neural nets suffer from this problem; we cannot explain their decisions.
(3) Do neural nets think?
When it comes to thinking, there is no there there.
This is a contributed opinion column. Edwin J. Kay is a professor emeritus of computer science and engineering at Lehigh University. The views expressed in this piece are those of its individual author, and should not be interpreted as reflecting the views of this publication. Do you have a perspective to share? Learn more about how we handle guest opinion submissions at themorningcall.com/opinions.
