Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Array Not Dimensioned Problem
#21
Great, now FBIde is causing an illegal error everytime I try to compile it. Man, what is my problem :roll:.
quote="Deleter"]judging gameplay, you can adaquately compare quake 4 with pong[/quote]
Reply
#22
when all else fails compile from command line
Reply
#23
Quote:Man, what is my problem :roll:.
Perhaps being a slovenly reader :wink:?


Since this is Your learning experience: i'll try again.
Code:
'Calculate HL Error
        
        FOR j = 1 TO NUM_HL
            sum = 0
            FOR k = 1 TO NUM_OL
                sum = sum + ol(k).e * ol(k).wt(j)
                hl(j).e = hl(j).act * (1 - hl(j).act) * sum
            NEXT k
        NEXT j
What's the "outer-loop expression" here, and why is it misplaced?
(This isn't about neural networks intricacies - t'is a very basic error.
Once you see it, you'll never make that mistake again.)

Great find in sub RunNet: j = 1
This one belongs in the same innocent category:
Code:
current = INT(RND * NUM_OL)
Code:
'Calculate OL Error
        FOR j = 1 TO NUM_OL
            IF j = current THEN

But all of this will be of no avail until you choose to remember that
Quote:This GAIN also enters the derivatives of the sig-function.
(triggers the obvious question... )

vspickelen
(Aka The Truly Annoying Fart, who holds the believe that one should at least have digested his own code.)
Reply
#24
Three days on and back to work again tomorrow, so it's time to cast off
this amusing thread, which is indeed grossly overstretched already.

All that's left now, is to present the working code you've all been waiting for.
Note how minute the changes!

[syntax="FreeBasic"]DEFINT A-Z
'****************************************************************
'*** Torahteen's backpropagation / propelled by vspickelen ****
'URL's: http://richardbowles.tripod.com/neural/n...eural3.htm
' & http://richardbowles.tripod.com/neural/b...ckprop.htm
'****************************************************************
DECLARE FUNCTION sig(x AS DOUBLE) AS DOUBLE
DECLARE SUB DoNet()
DECLARE SUB TrainNet()
DECLARE SUB RunNet()
DECLARE SUB InitNet()

CONST IMG_WIDTH = 12
CONST IMG_HEIGHT = 12
CONST NUM_IL = IMG_WIDTH * IMG_HEIGHT 'Number of Nodes in Input Layer
CONST NUM_HL = 20 'Number of Nodes in Hidden Layer
CONST NUM_OL = 10
CONST LR = 0.1
CONST GAIN = 5

TYPE Node
wt(1 TO NUM_IL) AS DOUBLE 'Weights
act AS DOUBLE 'Output of this Node
th AS DOUBLE 'Threshold
e AS DOUBLE 'Error
END TYPE

DIM SHARED hl(1 TO NUM_HL) AS Node 'Hidden Layer
DIM SHARED ol(1 TO NUM_OL) AS Node 'Output Layer

'Digit Images
DIM SHARED img(0 TO 9,1 TO 12, 1 TO 12) AS Byte
DIM SHARED curImg(1 TO NUM_IL) AS Byte
'Begin Main
'Load the Images
FOR i = 0 TO 9
FOR y = 1 TO 12
FOR x = 1 TO 12
READ img(i,x,y)
NEXT x
NEXT y
NEXT i


InitNet
PRINT "First, see how the NN performs before training"
RunNet
SLEEP
CLS
PRINT "Now training network"
TrainNet
CLS
PRINT "Now see how the NN performs after training"
RunNet
SLEEP
END

SUB DoNet()
DIM sum AS SINGLE

'Run through nodes in HL
FOR i = 1 TO NUM_HL
sum = 0
FOR j = 1 TO NUM_IL
sum = sum + hl(i).wt(j) * curImg(j)
NEXT j
hl(i).act = sig(sum - hl(i).th)
NEXT i

'Run through nodes in OL
FOR i = 1 TO NUM_OL
sum = 0
FOR j = 1 TO NUM_HL
sum = sum + ol(i).wt(j) * hl(j).act
NEXT j
ol(i).act = sig(sum - ol(i).th)
NEXT i
END SUB

SUB InitNet()
RANDOMIZE TIMER

'Hidden Layer
FOR i = 1 TO NUM_HL
FOR j = 1 TO NUM_IL
hl(i).wt(j) = (RND - .5) * .1
NEXT j
NEXT i

'Output Layer
FOR i = 1 TO NUM_OL
FOR j = 1 TO NUM_HL
ol(i).wt(j) = (RND - .5) * .1
NEXT j
NEXT i
END SUB

SUB TrainNet()
DIM numCycles AS INTEGER
DIM current AS INTEGER
DIM j AS INTEGER
DIM sum AS DOUBLE

INPUT "How many training cycles", numCycles
FOR i = 1 TO numCycles
j = 1

current = INT(RND(1) * 10)

FOR y = 1 TO IMG_WIDTH
FOR x = 1 TO IMG_HEIGHT
curImg(j) = img(current, x, y)
j = j + 1
NEXT x
NEXT y

DoNet

'Calculate OL Error
FOR j = 1 TO NUM_OL
IF j = current + 1 THEN
ol(j).e = (1 - ol(j).act) * GAIN * ol(j).act * (1 - ol(j).act)
ELSE
ol(j).e = (0 - ol(j).act) * GAIN * ol(j).act * (1 - ol(j).act)
END IF
NEXT j

'Calculate HL Error

FOR j = 1 TO NUM_HL
sum = 0
FOR k = 1 TO NUM_OL
sum = sum + ol(k).e * ol(k).wt(j)
NEXT k
hl(j).e = GAIN * hl(j).act * (1 - hl(j).act) * sum
NEXT j

'Adjust Weights
FOR j = 1 TO NUM_OL
FOR k = 1 TO NUM_HL
ol(j).wt(k) = ol(j).wt(k) + LR * ol(j).e * hl(k).act
NEXT k
ol(j).th = ol(j).th + LR * ol(j).e
NEXT j

FOR j = 1 TO NUM_HL
FOR k = 1 TO NUM_IL
hl(j).wt(k) = hl(j).wt(k) + LR * hl(j).e * curImg(k)
NEXT k
hl(j).th = hl(j).th + LR * hl(j).e
NEXT j


PRINT ".";
NEXT i
END SUB

SUB RunNet()
DIM quit AS Byte
DIM num AS Byte
DIM j AS INTEGER

j = 1

INPUT "Which number do you want to test?", num

FOR y = 1 TO IMG_WIDTH
FOR x = 1 TO IMG_HEIGHT
curImg(j) = img(num, x, y)
j = j + 1
NEXT x
NEXT y

DoNet

FOR i = 1 TO NUM_OL
PRINT "Value of output node ";
PRINT i - 1
PRINT ol(i).act
NEXT i
END SUB

FUNCTION sig(x AS DOUBLE) AS DOUBLE
sig = 1/(1 + EXP(-GAIN * x))
END FUNCTION
[/syntax]I left out the DATA statements.

Interested readers may want to learn about the already closed
challenge-with-no-entries that triggered this thread:
http://forum.qbasicnews.com/viewtopic.php?t=11164

And here's a link to some neural networks and cellular automata sources
dug up from my deep & undisturbed QB-vaults, now updated for FreeBasic:
http://home.graffiti.net/vspickelen:graf...index.html

See you next time 'round,
vspickelen
Reply
#25
Thanks spick. I was so darn close. I hate those little bugs. Sorry about the challenge-with-no-entries. I was going to try to enter after I finished this. Thanks for all the help. Hmm... now what should I try to make a neural network for? Tongue Hopefully you won't have to help me anymore.
quote="Deleter"]judging gameplay, you can adaquately compare quake 4 with pong[/quote]
Reply
#26
Quote:I was going to try to enter after I finished this.
So just ignore the shutdown and keep on postin'!

vspickelen
Reply
#27
BTW, the reason why I was having trouble understanding the weights problem, was because I was reading Richards guide to NNs. He has 3 sets of weights, because he doesn't consider the test pattern as an actual layer. He has three layers, input, hidden, and output. He feeds the pattern into the input layer, which has its own set of weights.
quote="Deleter"]judging gameplay, you can adaquately compare quake 4 with pong[/quote]
Reply


Forum Jump:


Users browsing this thread: 1 Guest(s)