12-29-2005, 11:29 PM
Quote:But how do I do BackPropagation for the hidden layer?
You already got that down, man, it's under 'Calculate HL Error
vspickelen (really likes this)
Array Not Dimensioned Problem
|
12-29-2005, 11:29 PM
Quote:But how do I do BackPropagation for the hidden layer? You already got that down, man, it's under 'Calculate HL Error vspickelen (really likes this)
12-29-2005, 11:45 PM
Hmm... It's still not giving me correct results. Here is the code, followed by output.
[syntax="qbasic"]DEFINT A-Z Declare Function sig(x As Double) As Double Declare Sub DoNet() Declare Sub TrainNet() Declare Sub RunNet() Declare Sub InitNet() Const MAX_NODES = 20 'Maximum number of nuerons in one layer Const NUM_IL = MAX_NODES 'Number of Nodes in Input Layer Const NUM_HL = MAX_NODES 'Number of Nodes in Hidden Layer Const NUM_OL = 10 Const IMG_WIDTH = 12 Const IMG_HEIGHT = 12 Const IMG_TOTAL = IMG_WIDTH * IMG_HEIGHT Const LR = 0.05 Const GAIN = 5 Type Node wt(1 To MAX_NODES) As Double 'Weights act As Double 'Output of this Node th As Double 'Threshold e As Double 'Error End Type Dim Shared hl(1 To NUM_HL) As Node 'Hidden Layer Dim Shared ol(1 To NUM_OL) As Node 'Output Layer 'Digit Images Dim Shared img(0 To 9,1 To 12, 1 To 12) As Byte Dim Shared curImg(1 To IMG_TOTAL) As Byte 'Begin Main 'Load the Images For i = 0 To 9 For y = 1 To 12 For x = 1 To 12 Read img(i,x,y) Next x Next y Next i InitNet Print "First, see how the NN performs before training" RunNet Sleep Cls Print "Now training network" TrainNet Cls Print "Now see how the NN performs after training" RunNet Sleep End SUB DoNet() DIM sum AS single 'Run through nodes in HL FOR i = 1 TO NUM_HL sum = 0 FOR j = 1 TO IMG_TOTAL sum = sum + hl(i).wt(j) * curImg(j) NEXT j hl(i).act = sig(sum - hl(i).th) NEXT i 'Run through nodes in OL FOR i = 1 TO NUM_OL sum = 0 FOR j = 1 TO NUM_HL sum = sum + ol(i).wt(j) * hl(j).act NEXT j ol(i).act = sig(sum - ol(i).th) NEXT i END SUB Sub InitNet() Randomize Timer 'Hidden Layer For i = 1 To NUM_HL For j = 1 To NUM_IL hl(i).wt(j) = (Rnd(1)/4) Next j Next i 'Output Layer For i = 1 To NUM_OL For j = 1 To NUM_HL ol(i).wt(j) = (Rnd(1)/4) Next j Next i End Sub Sub TrainNet() Dim numCycles As Integer Dim current As Integer Dim j As Integer Dim sum As Double Input "How many training cycles", numCycles For i = 1 To numCycles j = 1 current = Int(Rnd(1) * 10) For y = 1 To IMG_WIDTH For x = 1 To IMG_HEIGHT curImg(j) = img(current, x, y) j = j + 1 Next x Next y DoNet 'Calculate OL Error For j = 1 To NUM_OL If j = current Then ol(j).e = (1 - ol(j).act) * ol(j).act * (1 - ol(j).act) Else ol(j).e = (0 - ol(j).act) * ol(j).act * (1 - ol(j).act) End If Next j 'Calculate HL Error For j = 1 To NUM_HL sum = 0 For k = 1 To NUM_OL sum = sum + ol(k).e * ol(k).wt(j) hl(j).e = hl(j).act * (1 - hl(j).act) * sum Next k Next j 'Adjust Weights For j = 1 To NUM_OL For k = 1 To NUM_HL ol(j).wt(k) = ol(j).wt(k) + LR * ol(j).e * hl(k).act ol(j).th = ol(j).th - LR * ol(j).e Next k Next j For j = 1 To NUM_HL For k = 1 To IMG_TOTAL hl(j).wt(k) = hl(j).wt(k) + LR * hl(j).e * curImg(k) hl(j).th = hl(j).th - LR * hl(j).e Next k Next j Print "."; Next i End Sub Sub RunNet() Dim quit As Byte Dim num As Byte Input "Which number do you want to test?", num For y = 1 To IMG_WIDTH For x = 1 To IMG_HEIGHT curImg(j) = img(num, x, y) j = j + 1 Next x Next y Print "You chose this number" For y = 1 To IMG_WIDTH For x = 1 To IMG_HEIGHT If img(num, x, y) = 0 Then Pset(x + 100,y),15 End If Next x Next y DoNet For i = 1 To NUM_OL Print "Value of output node "; Print i - 1 Print ol(i).act Next i End Sub Function sig(x As Double) As Double sig = 1/(1 + Exp(-GAIN * x)) End Function 0: Data 0,0,0,0,0,0,0,0,0,0,0,0 Data 0,0,0,0,1,1,1,1,0,0,0,0 Data 0,0,0,1,0,0,0,0,1,0,0,0 Data 0,0,1,0,0,0,0,0,0,1,0,0 Data 0,0,1,0,0,0,0,0,0,1,0,0 Data 0,0,1,0,0,0,0,0,0,1,0,0 Data 0,0,1,0,0,0,0,0,0,1,0,0 Data 0,0,1,0,0,0,0,0,0,1,0,0 Data 0,0,1,0,0,0,0,0,0,1,0,0 Data 0,0,0,1,0,0,0,0,1,0,0,0 Data 0,0,0,0,1,1,1,1,0,0,0,0 Data 0,0,0,0,0,0,0,0,0,0,0,0 1: Data 0,0,0,0,0,0,0,0,0,0,0,0 Data 0,0,0,0,0,0,1,0,0,0,0,0 Data 0,0,0,0,0,1,1,0,0,0,0,0 Data 0,0,0,0,1,0,1,0,0,0,0,0 Data 0,0,0,0,0,0,1,0,0,0,0,0 Data 0,0,0,0,0,0,1,0,0,0,0,0 Data 0,0,0,0,0,0,1,0,0,0,0,0 Data 0,0,0,0,0,0,1,0,0,0,0,0 Data 0,0,0,0,0,0,1,0,0,0,0,0 Data 0,0,0,0,0,0,1,0,0,0,0,0 Data 0,0,0,0,1,1,1,1,1,0,0,0 Data 0,0,0,0,0,0,0,0,0,0,0,0 2: Data 0,0,0,0,0,0,0,0,0,0,0,0 Data 0,0,0,0,1,1,1,1,0,0,0,0 Data 0,0,0,1,0,0,0,0,1,0,0,0 Data 0,0,1,0,0,0,0,0,0,1,0,0 Data 0,0,0,0,0,0,0,0,0,1,0,0 Data 0,0,0,0,0,0,0,0,0,1,0,0 Data 0,0,0,0,0,0,0,0,1,0,0,0 Data 0,0,0,0,0,0,0,1,0,0,0,0 Data 0,0,0,0,0,0,1,0,0,0,0,0 Data 0,0,0,0,0,1,0,0,0,0,0,0 Data 0,0,0,1,1,1,1,1,1,1,0,0 Data 0,0,0,0,0,0,0,0,0,0,0,0 3: Data 0,0,0,0,0,0,0,0,0,0,0,0 Data 0,0,0,0,1,1,1,1,0,0,0,0 Data 0,0,0,1,0,0,0,0,1,0,0,0 Data 0,0,0,0,0,0,0,0,1,0,0,0 Data 0,0,0,0,0,0,0,0,1,0,0,0 Data 0,0,0,0,0,1,1,1,0,0,0,0 Data 0,0,0,0,0,0,0,0,1,0,0,0 Data 0,0,0,0,0,0,0,0,1,0,0,0 Data 0,0,0,1,0,0,0,0,1,0,0,0 Data 0,0,0,0,1,1,1,1,0,0,0,0 Data 0,0,0,0,0,0,0,0,0,0,0,0 Data 0,0,0,0,0,0,0,0,0,0,0,0 4: Data 0,0,0,0,0,0,0,0,0,0,0,0 Data 0,0,0,0,0,0,0,1,1,0,0,0 Data 0,0,0,0,0,0,1,0,1,0,0,0 Data 0,0,0,0,0,1,0,0,1,0,0,0 Data 0,0,0,0,1,0,0,0,1,0,0,0 Data 0,0,0,1,0,0,0,0,1,0,0,0 Data 0,0,0,1,1,1,1,1,1,1,0,0 Data 0,0,0,0,0,0,0,0,1,0,0,0 Data 0,0,0,0,0,0,0,0,1,0,0,0 Data 0,0,0,0,0,0,0,0,1,0,0,0 Data 0,0,0,0,0,0,0,1,1,1,0,0 Data 0,0,0,0,0,0,0,0,0,0,0,0 5: Data 0,0,0,0,0,0,0,0,0,0,0,0 Data 0,0,0,1,1,1,1,1,1,0,0,0 Data 0,0,0,1,0,0,0,0,0,0,0,0 Data 0,0,0,1,0,0,0,0,0,0,0,0 Data 0,0,0,1,0,0,0,0,0,0,0,0 Data 0,0,0,1,1,1,1,0,0,0,0,0 Data 0,0,0,0,0,0,0,1,0,0,0,0 Data 0,0,0,0,0,0,0,0,1,0,0,0 Data 0,0,0,0,0,0,0,0,1,0,0,0 Data 0,0,0,0,0,0,0,1,0,0,0,0 Data 0,0,0,1,1,1,1,0,0,0,0,0 Data 0,0,0,0,0,0,0,0,0,0,0,0 6: Data 0,0,0,0,0,0,0,0,0,0,0,0 Data 0,0,0,0,1,1,1,1,0,0,0,0 Data 0,0,0,1,0,0,0,0,0,0,0,0 Data 0,0,1,0,0,0,0,0,0,0,0,0 Data 0,0,1,0,0,0,0,0,0,0,0,0 Data 0,0,1,0,0,0,0,0,0,0,0,0 Data 0,0,1,0,1,1,1,1,0,0,0,0 Data 0,0,1,1,0,0,0,0,1,0,0,0 Data 0,0,1,0,0,0,0,0,1,0,0,0 Data 0,0,0,1,0,0,0,0,1,0,0,0 Data 0,0,0,0,1,1,1,1,0,0,0,0 Data 0,0,0,0,0,0,0,0,0,0,0,0 7: Data 0,0,0,0,0,0,0,0,0,0,0,0 Data 0,0,0,1,1,1,1,1,1,1,0,0 Data 0,0,0,0,0,0,0,0,1,0,0,0 Data 0,0,0,0,0,0,0,0,1,0,0,0 Data 0,0,0,0,0,0,0,1,0,0,0,0 Data 0,0,0,0,0,0,0,1,0,0,0,0 Data 0,0,0,0,0,0,1,0,0,0,0,0 Data 0,0,0,0,0,0,1,0,0,0,0,0 Data 0,0,0,0,0,1,0,0,0,0,0,0 Data 0,0,0,0,0,1,0,0,0,0,0,0 Data 0,0,0,0,1,0,0,0,0,0,0,0 Data 0,0,0,0,0,0,0,0,0,0,0,0 8: Data 0,0,0,0,0,0,0,0,0,0,0,0 Data 0,0,0,0,1,1,1,1,0,0,0,0 Data 0,0,0,1,0,0,0,0,1,0,0,0 Data 0,0,0,1,0,0,0,0,1,0,0,0 Data 0,0,0,1,0,0,0,0,1,0,0,0 Data 0,0,0,0,1,1,1,1,0,0,0,0 Data 0,0,0,1,0,0,0,0,1,0,0,0 Data 0,0,0,1,0,0,0,0,1,0,0,0 Data 0,0,0,1,0,0,0,0,1,0,0,0 Data 0,0,0,1,0,0,0,0,1,0,0,0 Data 0,0,0,0,1,1,1,1,0,0,0,0 Data 0,0,0,0,0,0,0,0,0,0,0,0 9: Data 0,0,0,0,0,0,0,0,0,0,0,0 Data 0,0,0,0,1,1,1,1,0,0,0,0 Data 0,0,0,1,0,0,0,0,1,0,0,0 Data 0,0,0,1,0,0,0,0,0,1,0,0 Data 0,0,0,1,0,0,0,0,0,1,0,0 Data 0,0,0,1,0,0,0,0,1,1,0,0 Data 0,0,0,0,1,1,1,1,0,1,0,0 Data 0,0,0,0,0,0,0,0,0,1,0,0 Data 0,0,0,0,0,0,0,0,0,1,0,0 Data 0,0,0,1,0,0,0,0,1,0,0,0 Data 0,0,0,0,1,1,1,1,0,0,0,0 Data 0,0,0,0,0,0,0,0,0,0,0,0[/syntax] Code: First, see how the NN performs before training Any idea why it is still giving results above 1?
quote="Deleter"]judging gameplay, you can adaquately compare quake 4 with pong[/quote]
12-30-2005, 03:26 AM
O.k, last post for today
Sad to say, your code needs more tidying up still. Things that will be quite obvious, once you understand what's goin' on in all those lil'nifty-nested for-loops. The constants are messed up, this will do Code: CONST IMG_WIDTH = 12 Use Code: hl(i).wt(j) = (RND - .5) * .1 While this is fine Code: current = INT(RND * NUM_OL) Code: FOR j = 1 TO NUM_OL I've changed the signs before the bias terms here, guess you can justify the displaced outer-loop exprssions yourself Code: 'Calculate HL Error This innocent one is a classic: Code: INPUT "Which number do you want to test?", num The only thing left then, is to remember that Quote:This GAIN also enters the derivatives of the sig-function. Good night ( snow here )
12-30-2005, 09:29 AM
Hmm... Maybe I'm just a bad programmer. Do you have a working version of my code? Is it actually presenting the correct results? Could you post the code? I view this as a learning experience, so don't worry about spoiling my fun .
quote="Deleter"]judging gameplay, you can adaquately compare quake 4 with pong[/quote]
12-30-2005, 07:52 PM
C'mon Torahteen,
you certainly don't consider yourself a "bad programmer," suspicious perhaps, but probably just running out of patience. I promise: you're so close to having a working program, t'would be a pity if I'd finish in your place. That final satisfaction will be all yours... (and I wanna see my guidance bear fruit.) I uploaded an .exe here, http://home.graffiti.net/vspickelen:graf...orah10.zip about 200 training cycles will suffice. Looks nice eh? Now tell me what problems still remain after my previous post. vspickelen
12-30-2005, 08:30 PM
Why is it that it keeps returning values above 1? The sigmoid functio should never return a value above 1.
quote="Deleter"]judging gameplay, you can adaquately compare quake 4 with pong[/quote]
12-30-2005, 10:01 PM
Quote:Why is it that it keeps returning values above 1? Quote:none of those results are above 1 :???: I didn't notice: Computer programs use so called exponential notation for expressing large and small numbers, e.g. 0.04305563 = 4.305563 * 10^-2 = 4.305563e-002 see? vspickelen
12-31-2005, 03:36 AM
Oh. Silly me :oops:. Anyway, I'm getting an Illegal Operation error when I run my latest version of code. I'm probably not understanding something. It appears to be happening in the DoNet function. Here is my code (I'm making this page really big lol)
[syntax="qbasic"]DEFINT A-Z Declare Function sig(x As Double) As Double Declare Sub DoNet() Declare Sub TrainNet() Declare Sub RunNet() Declare Sub InitNet() Const IMG_WIDTH = 12 Const IMG_HEIGHT = 12 Const NUM_IL = IMG_WIDTH * IMG_HEIGHT 'Number of Nodes in Input Layer Const NUM_HL = 20 'Number of Nodes in Hidden Layer Const NUM_OL = 10 Const LR = 0.1 Const GAIN = 5 Type Node wt(1 To NUM_IL) As Double 'Weights act As Double 'Output of this Node th As Double 'Threshold e As Double 'Error End Type Dim Shared hl(1 To NUM_HL) As Node 'Hidden Layer Dim Shared ol(1 To NUM_OL) As Node 'Output Layer 'Digit Images Dim Shared img(0 To 9,1 To 12, 1 To 12) As Byte Dim Shared curImg(1 To IMG_TOTAL) As Byte 'Begin Main 'Load the Images For i = 0 To 9 For y = 1 To 12 For x = 1 To 12 Read img(i,x,y) Next x Next y Next i InitNet Print "First, see how the NN performs before training" RunNet Sleep Cls Print "Now training network" TrainNet Cls Print "Now see how the NN performs after training" RunNet Sleep End SUB DoNet() DIM sum AS single 'Run through nodes in HL FOR i = 1 TO NUM_HL sum = 0 FOR j = 1 TO IMG_TOTAL sum = sum + hl(i).wt(j) * curImg(j) NEXT j hl(i).act = sig(sum - hl(i).th) NEXT i 'Run through nodes in OL FOR i = 1 TO NUM_OL sum = 0 FOR j = 1 TO NUM_HL sum = sum + ol(i).wt(j) * hl(j).act NEXT j ol(i).act = sig(sum - ol(i).th) NEXT i END SUB Sub InitNet() Randomize Timer 'Hidden Layer For i = 1 To NUM_HL For j = 1 To NUM_IL hl(i).wt(j) = (Rnd - .5) * .1 Next j Next i 'Output Layer For i = 1 To NUM_OL For j = 1 To NUM_HL ol(i).wt(j) = (Rnd - .5) * .1 Next j Next i End Sub Sub TrainNet() Dim numCycles As Integer Dim current As Integer Dim j As Integer Dim sum As Double Input "How many training cycles", numCycles For i = 1 To numCycles j = 1 current = Int(Rnd(1) * 10) For y = 1 To IMG_WIDTH For x = 1 To IMG_HEIGHT curImg(j) = img(current, x, y) j = j + 1 Next x Next y DoNet 'Calculate OL Error For j = 1 To NUM_OL If j = current Then ol(j).e = (1 - ol(j).act) * ol(j).act * (1 - ol(j).act) Else ol(j).e = (0 - ol(j).act) * ol(j).act * (1 - ol(j).act) End If Next j 'Calculate HL Error FOR j = 1 TO NUM_HL sum = 0 FOR k = 1 TO NUM_OL sum = sum + ol(k).e * ol(k).wt(j) hl(j).e = hl(j).act * (1 - hl(j).act) * sum NEXT k NEXT j 'Adjust Weights FOR j = 1 TO NUM_OL FOR k = 1 TO NUM_HL ol(j).wt(k) = ol(j).wt(k) + LR * ol(j).e * hl(k).act ol(j).th = ol(j).th + LR * ol(j).e NEXT k NEXT j FOR j = 1 TO NUM_HL FOR k = 1 TO IMG_TOTAL hl(j).wt(k) = hl(j).wt(k) + LR * hl(j).e * curImg(k) hl(j).th = hl(j).th + LR * hl(j).e NEXT k NEXT j Print "."; Next i End Sub Sub RunNet() Dim quit As Byte Dim num As Byte Dim j As Integer j = 1 Input "Which number do you want to test?", num For y = 1 To IMG_WIDTH For x = 1 To IMG_HEIGHT curImg(j) = img(num, x, y) j = j + 1 Next x Next y DoNet For i = 1 To NUM_OL Print "Value of output node "; Print i - 1 Print ol(i).act Next i End Sub Function sig(x As Double) As Double sig = 1/(1 + Exp(-GAIN * x)) End Function[/syntax] I left out the DATA statements.
quote="Deleter"]judging gameplay, you can adaquately compare quake 4 with pong[/quote]
12-31-2005, 04:55 AM
It's throwing that error because IMG_Total has not been declared.:wink: (i think).
|
« Next Oldest | Next Newest »
|