Programming Discussion Thread

[AnyOldName3]

I'm reading the New Turing Omnibus, as it's something that knowing you've read encourages universities to let you do computer science, and found a chapter on neural nets. I then implemented the psuedocode in Java, and it was going well until I reached the point at which it was theoretically finished. It currently 'learns' how to convert polar coordinates to Cartesian coordinates for a bit, and then starts learning in the wrong direction (i.e. it gets worse than it was earlier) until the error is infinite, then a few cycles later, it starts returning NaN, which it isn't supposed to. Anyway, can anyone see where the issue is?

Code:

import java.util.Random;import java.lang.Math;
//This is broken, but follows the psuedocode perfectly. It is set to exit when something breaks. The error tends to infinity, leading things to break when a correction is attempted. The error problem is either caused by overflow, or division by zero.public class NeuralNet{    final double RATE = 0.1;    double[][] synone = new double[2][30];    double[][] syntwo = new double[30][2];    double[] input = new double[2];    double[] medin = new double[30];    double[] medout = new double[30];    double[] output = new double[2];    double[] error = new double[2];        public void initialise()    {        //set synone to random values between -0.1 and 0.1        for(int i=0; i<30; i++)        {            for(int j=0; j<2; j++)            {                synone[j][i] = new Random().nextDouble();                synone[j][i] = synone[j][i]*0.2 - 0.1;            }        }        //Same for syntwo        for(int i=0; i<2; i++)        {            for(int j=0; j<30; j++)            {                syntwo[j][i] = new Random().nextDouble();                syntwo[j][i] = syntwo[j][i]*0.2 - 0.1;            }        }    }        public void run()    {        for(int i=0; i<=10000; i++)        {            //Generate polar coords on unit circle            double length = 1.0;            double theta = new Random().nextDouble();            theta = theta * 360.0;            //put length and theta into input neurons            input[0] = length;            input[1] = theta;            //Pass these to convert to get back error            convert(input);            //Use back propagation to adjust neural net            backProp();            //Output error E            double errorE = 0.0;            for(int j=0; j<error.length; j++)            {                errorE = errorE + error[j]*error[j];            }            errorE = Math.sqrt(errorE);            System.out.println(errorE);        }    }        public void convert(double[] input)    {        //for each medial neuron        for(int i=0; i<30; i++)        {            //reset medial input            medin[i] = 0;            //for each input neuron            for(int j=0; j<2; j++)            {                //add input times weight to the neuron                medin[i] = medin[i] + synone[j][i]*input[j];            }            //compute tanh of the neuron's value to get it's output            medout[i] = Math.tanh(medin[i]);        }                double[] target = new double[2];        //for each output neuron        for(int i=0; i<2; i++)        {            //reset output            output[i] = 0;            //for each medial neuron            for(int j=0; j<30; j++)            {                //add input times weight to the neuron                output[i] = output[i] + syntwo[j][i] * medout[j];                System.out.println("O"+output[i]+" S2"+syntwo[j][i]+" Mo"+medout[j]);                if(Double.isNaN(output[i]))                    System.exit(0);            }            //compute correct value for this output            //must use correct trig function for x and y            switch(i)            {                case 1: target[i] = input[0] * Math.sin(Math.toRadians(input[1]));                    break;                case 2: target[i] = input[0] * Math.cos(Math.toRadians(input[1]));                    break;                default: break;            }            error[i] = target[i] - output[i];        }    }        public void backProp()    {        //adjust second synaptic layer        //for each output neuron        for(int i=0; i<2; i++)        {            //for each medial neuron            for(int j=0; j<30; j++)            {                //adjust the synaptic multiplier/weight                syntwo[j][i] = syntwo[j][i] + RATE*medout[j]*error[i];            }        }        //Derive the sigmoidal signal        double[] sigma = new double[30];        double[] sigmoid = new double[30];        //for each medial neuron        for(int i=0; i<30; i++)        {            //reset sigma            sigma[i] = 0;            //for each output neuron            for(int j=0; j<2; j++)            {                //add value to sum                sigma[i] = sigma[i] + error[j]*syntwo[i][j];            }            //calculate sigmoid            sigmoid[i] = 1 - medin[i]*medin[i];        }        //adjust the first synaptic layer        //probably for each output neuron, although that doesn't seem obvious        for(int i=0; i<2; i++)        {            //for each medial neuron            for(int j=0; j<30; j++)            {                double delta = RATE*sigmoid[j]*sigma[j]*input[i];                synone[1][j] = synone[i][j] + delta;            }        }    }        public static void main(String[] args)    {        NeuralNet net = new NeuralNet();        net.initialise();        net.run();    }}

---

Okay, code /code tags don't work. have the .java file.

Also, did you know that both .java and .ITMADEMECHANGETHEFILEEXTENSION are blocked from being attached by the forum.