globalization

today is/was kinda chill. I ran a 100 by 100 on clemson network, I have to watch it because the threads tend to quit after a bit. I uploaded my code to a repository so if my computer crashes and cloud 9 crashes and clemson crashes then I will still have my code on the internet. After running a small simulation on a computer in utah, or ohio... it`s one of those... I learned that the source of the zeros in my program is two fold. it was found that there was a different type of error message... one that meant that two attempts for the fitness fail...this error message was a fitness with extremely low value. I had just made a method that rounded all the values to the 4th place, so the value was cut off. When the simulation was run, zeros still popped up, but so did the default value that was chaged. So there is potential another zero emitting entity in the system, but as for now Im just happy everything kinda works.

devaluation

Well all the problems seem to be caught to far, but for some reason some individuals no have a fitness of zero... "well whats the problem" you might ask,... well as far as I have seen no individual can have a fitness of zero and the value is usually a sign that the input was syntactically incorrect, and its not. This only happens in some threads and I know this because if I have 100 individuals and 50 threads then there needs to be 2 evaluations, and the number of 0.0 entries is an even number constant with every generation after some point. I looked at the raw data that was begin spit out from the http get request and it match that of the log files. Also I hand ran some individuals and there fitness in a reset machine was actually goodish... not excellent, but not zero. I have also noticed that if a machine has been running for a period then manually running a fitness loops... ie someone is at the door but no one is coming.  just for the sake of sake Im going to rerun and If i get similar results then I will go from there. To ensure that Its not some haywire method over righting finesses zero I will refresh the evaluations.

decolonisation

Well I figured out the reason for half of the zeroPop being zeros, it turns out that when I wrote the code I switched the parameters... oops =1... so that problem was fixed. the reason that it wasn't a problem was that when ran by file the seeded population was .5 instead of .05 so it was half and half. when i would test the Genalgo I would do whatever number I fancied at the time. Well problem fixed and put away. But I now have a different problem. When I run a simulation nearly all the fitness's come back as zero... before you think "well you had a problem with references before, look", This fitness is strait from the horses mouth... ie its printed right after it is returned from the website, my code barely touches it. So I tried running the individual manually and when I do the browser waits and then I get a time out as if the remote evaluation isn't running... you may ask "is it?", yes it its, ye of little faith... the terminal for it is right next to it, waiting for something. When I refresh the browser the terminal stays as is, so this means that the remote eval is not even getting the green light. Ill see whats up, or try another joey, but this is weird.. I have had this problem all day.

variegation

So on Monday, it was discovered that I was using references wrong and altering my programs genetic materiel in a wrong way. So Monday, Tuesday and Wednesday was used to discover and fix that problem. done. I ran a test program and for some reason when I run my program using the UI i made, things go haywire... like half of the population disappears. So today is ensuring that my program reads the popzero file correctly, ensuring that the results for any fitness test is correct, and trying to discover the root cause of the UI malfunction.
I know it is the UI because if I run the file that actually runs everything then everything works fine, but if I run the GenAlgo... which just asks for the parameters then dynamically fills them, I get half the population of Generations zero set to zero.

disambiguation

upon really fine examination of all the parts of the Ga program I found that two major parts of the program where not quite correct. The effects where seen when the numbers that the computer dealt with and the numbers that where presented in the log files where not matching up. It turns out that when the generation was being read to make the next generation the float arrays that where being altered where for the next generation was them selves being altered. ... ugg references...(-U_0-)  Meaning that if a individual had a good fitness, it would share its genetic code, then have that code be altered to a better or worse fitness and the program would not notice. This could have lead to all the default values. I'm going to run the program again and see if It fails again.

misidentification

So there formed a problem with my program, and I didn't catch it because it was in a area that was pretty solid. So my program has the possiblility to die... ie no growth in population fitness. This could happen if the population zero is bad. The problem was that all the runs had the same population zero... or so I thought. I manipulated my popZero class so that its population is not mixed and is in order... this is not a problem, its like when you eat do you start from left to right, up to down bla bla bla... I found that some of the individuals read from the PopZero file spawned  genes from nothing. I saw very closely and it turns out that the scanner reads the line, but sometimes forgets what line its on and reads a gene above or below the line. for example the file is
a b c d e f
g h i j k l
m n o p q
 and you want the g line. the scanner might be like oh  here--- g h i j k l
but it might make a mistake and be like --- g h c d p l
Its hard to read just one line it seams. So this error introduces randomness in the program form the start.
I think its cool... in a ha ha weird way. The thing is now how to fix it. I guess read a line over and over but Ill work it out.

Ps. I was tired of editing the file for the runs so I made a ui that asks for all the params, the runs it itself. and when the run is done it ask if you want a rerun.  Work as smart as you have to


P.s.s
I found the problem, turns out that the way I was saving the float arrays where mixing randomly. I still dont know why, but I changed to strings being passed, Ill see how it goes

explination

So since Monday i have not been making post. This single post will encompass all the events up to this point...
---------------------------------------Monday------------------------------------- (^_^)
Ran simulations for point .6-.8 mutation rates on 20 threads. Also tested out continue class, all done on either imp or hornet... both are an 8 bit so the results could be comparable.
--------------------------------------Tuesday--------------------------------------(n_n)
The continue class has a bug, I knew because I put it there... it had a good property... all I need to do is find a way to save the property but not the bug.
The test results were coming in but all the results from hornet where 0.0 or -0.0. nearly all the results from imp where 1.0E-4. Dr. Remy and I walked through what could be happening and ran a another test with, to test the machine, so the same run was ran on joey, imp and hornet. And I was to manually run the test again on joey.
------------------------------------Wednesday-------------------------------------(-_-)
Results came in and hornet for what-ever reason only returns 0.0s, imp works for about 1 generation and the finesses return a default value. joey was the only one that performed okay. So I ran .6-.8 again on joey, .6 and .7 I used 10 threads but for .8 i used 20... just to see if it was a work load thing.
later that day I came back and all where running. .8 had a weird pattern where everything was okay but the second to last 10 finesses in the entire generation was a time out default, in each generation. I kept it going to see the effects,
----------------------------------Thursday-today----------------------------------(U_U)
for some reason, there is timeout values popping up here and there in every run. in the 20 threads the time out happens in the third run. in the 10 threads time out happens in the fourth run. This means that I will have to do some alterations to the code to allow for a longer wait time, or run  the evaluations with 10 threads which kinda defeats the purpose of a quick evaluation. Anyway Im just running the evals in small batches to see if the results are good

guesstimation

Right now Im running/going to rerun 20 iterations of GA with mutation rate of .6, .7, and .8. There was only one successful run of these so not much data will have to do to waste. If  these run dont snag on anything I dont really know what to expect from the data. From what I have seen I would think that the .6 would have one of the higher averages and maximum finesses. Of the runs that completed, it looks like around 50% mutation rate is a point where fitness peaks and then falls.

This could be speculation, I have not done as many iterations as would have been appropriate, It would be quite interesting to see that final charts for this experiment. Some papers that I have read said that a large population in relation to mutation leaves a generation of better configuration, so much fascination...( /\ ./\) on a roll...anyway, some papers said that a smaller mutation rate would be good with a high population.. high meaning 16 , but this also meant a smaller chance for finding a good solution. With the parameters like 100 population and generation then it could also be that the mutation rate might level off at some point.

synchronization

I was able to finally complete that classes that enables a run to be continued if it is interrupted. Its not much but I was able to successfully run this class that started a run from start to finish and to continue where the run left off. If this program work on large scale all the data that was received before can be put to use, the only thing is that a header file will have to be made for each run, but that wont be much of a problem.

alleviation

Finally the problem might have been caught... might.  Dr.Remy and I ran a sample run with 60 threads and saw that the system average load was increasing to a max point of 8 or so. So the computation was stopped... by me so no prob... and reran the program but with 20 threads instead of 60. The average load didn't go past 5.. This was good. So the prevailing theory is that when the computer was running with 40+ threads, it was working as fast as it could on this one process. But when another program was run, by someone else on campus, the computer didn't could not think of what to do...so it stopped. When 60,70, and 100 threads where used the computer couldn't handle it so it would take longer to complete the task, or just stop altogether. So as of now, I'm going to work on the continuation class, and rerun all the mutations again but using 20 threads.

devastation

The problem still eludes me, but for some reason there is a snag in the program... at least in one of the programs being used. I thought that the problem was the readTimeout I added, turns out that this lead to another problem. So as of now the reason that my computations stop was because of X, a thread would wait for some data that did not come and would halt the program. this was "fixed" by adding a readTimeout, if the connection didn't give a value then it would return some value...0.19283746... this lead to nearly all the outcomes being some default value, either on my programs end or on the evaluation end. So Dr.Remy and I worked through what might be happening while another run was being processed. Not much could be done with out the data from this run....It came in, this run was time based so the data is still valid... I had  a looked at the end results for the computations and it looks like the default value doesn't show up until some point and then is a present throughout the rest of the evolutionary process...(>_<)

deviation

I don`t quite know the exact reason why, but for some reason my program stops running after a while. I know where the problem lies, but it is more of an effect then the reason. So my program sends out threads to have a individual evaluated using different ports. Then when the threads return...all of them... then the program continues. The problem is that when a call is sent to evaluate a individual the call is sent to another program...called lets say eval.... Eval then call another program... lets say called show..., show then returns and eval returns with a number. My problem is that if a snag appears on eval to show, or show to eval, my program just sits idle. I added a connection time out but didn't realize that this did not count for actual time for the connection. I added a readTimeout and Im going to see if this works.
I am also making a class that will be able to work with the bits of data that a unfinished run leaves. As the run goes on each generation is saved along with its fitness. All I need to do is make a class that with the base name given, reads the left over file and continues where that calculation left off. Im running into a snag thou... well not a snag more of a moral memory decision . The file could potentially be very large and it would do to read the file over and over for each need. I don`t know of another easy way that I could read the file so that I know which generation was the last, with out altering the file in questions syntax, without reading the file once, then again to get the values of the last done generation, and then once more to adequately read all the fitness, sort them into the appropriate generation and also find the max and average of each generation. At least not all in one single method or function. But im working on it, as of now im just going to try to run an evaluation just to see if the readTimeout worked.

education

I've been using a nifty little program the past week  and I must say, its very useful. When using a ssh terminal, any program that is running is stopped when the ssh terminal closes. screen allows you to leave the terminal and come back to it later. To use screen,,, type screen... yup, that's it. there's some other commands to get through it like ctrl+a d detaches from the port, ctrl+a c makes a new screen, a screen -ls shows you the name of all the screens and screen -r <name> attaches back to the screen. I'm also using top. It allows me to see who is using what of the computer. Its all little tricks and stuff. Top is useful, because if someone is using that computer with a cpu intensive task then the program to be run might become slow.

actualization

This is just a short fyi post... I was able to make box and whisker plots with the data given from the last run. now all I need to do is call the program, the text file with the data and the name I want to save the chart as. this is taco.png mark two

I dont quite yet know how to change add words and stuff but this is the max values of the run, the x axis is the mutation rate, instead of 1 think of .1 mutation. 

continuation

This morning, I tried my hand with python... i think i like it (n.n). I have made a plot off a run I made yesterday. lookie

I know its not much, but I was able to make a script to do this... Im ecstatic... The script takes in two parameters, for now, the file to be saved and the name of the file. Right now you are looking at taco.png.  I'm now going to tackle making the chart more useful, the file i gave it only had one column and the larger runs have more, so this is going to be the focus of the rest of the day.

Ps, turns out that one of the runs I started last night was still chugging along so this morning I saw and I now have more data for the effects of mutation on the ga evolution.  

assimilation

yesterday wasn't so grand, just some house keeping. The simulation ran and the data was collected just fine. As of now Im trying to learn python so that a script will make a chart of the data instead of using excel. As I was reading up on python I ran some other simulations, I used 2 sets of computers and I think I found a bug of sorts. One of the ball and plate stopped running, so I look ate the computer that was handling the evaluations and found that that computer stopped as well. The problem wasn't with the two so I can assume that it is a problem with the actual code. I think that I know where it is. Like I said...or it think i said it... the when the program is evaluating a generation different threads go and do the computations, then report to one method that reports when all the threads have returned, if one thread is stuck then the whole thing waits. But I don't see how this would happen because the Thread is has a time limit, and if it is met then the thread returns a default value. unless I commented it out hunting down another bug. I`ll check but It seems that is not it. Other then that there is not many while loops in the system, most are for so there would be no reason for a looping system.
I don't know if this is an actual problem, I tried it with the first set and didn't have this problem, It only happened when I tried to run both simulation at the same time. I made sure that there wasn't any overlap in machines so I don't know what could have caused the problem.

vexation

Ug, (-_-)... So this morning I went back to my computer and saw that there was an error, a Java.net.SocketException: connection reset. So I tried again, and got the same thing. It turned out that I was calling computers that were not listening , lesson learned. The bulk of today was just hunting down bugs that seemed to magically appear... more like they just began to be a problem. It went from no floats being stored to no evolutionary growth, to this, and that. I also noticed that the initial population was really bad, like 1.00E-5 bad, this meant that my program wasn't good for really low numbers. the problem was showing up when the elite and fit were being seen, only maybe 2 where classified as good, two a generation does not make. To fix this problem  I made the crossover of the fit come back again after the mutation, just to fill the population up to the population size, after the 3 or fourth this is not even used. Once that problem was finished and bow-Tied I was able to run all three programs... the one that evaluates the fitness, the one that draws out the eval, and the genetic algor... and its running as I type. I hope that I will go well. I read generation log file right now and im seeing some 500+ to 4000+.  here's what the test look like... there's 20 threads running but i only show four, assuming if the first two and last two run then the rest should be okay

I did find something that might be a problem, if only one for a single generation. Some of the test results are like 7000+, in a population who is mostly 1.blaE-4. I ran these manually and the fitness was .0001bla. So if this proves to be a problem, then I guess i could make a little screen that if a number that is really high passes along, a method will rerun the individual and if another number, say plus or minus 25% is a result then the second will be used, or maybe that average of the two...idk right now.

Compulations

Ps. Yesterday was a bit full closer to the end of the day, I wrote down the blog forgot to publish it

Today was...(U_U) ... I found that my program is fickle, sometimes it runs like a dream, and other times it rises high just to crash an burn. But other then the little bugs and such the program works fine...ish. One problem that I had was that the class that has many threads uses one method to check back with...im using indexes and arrays so i dont have to worry about data contamination... This one method will not let the class be finished until all the threads have finished with there computations. Well one thread would get stuck, leaving my program frozen. I added a request Time out and set the gave the result variable a default value of 0.123456789. also  I unknowing suited the program to the first url, and now that i have a second, I have to add some methods that would suit this url. But this one is that actual ball and plate... looky  ( ^.^)

Its cool.  Right now im running the program and hope it went well.

annexation

I was able to successfully alter my program so that only one generation of individuals is stored in run time and the rest are dynamically added to files. So instead of having at least I *N*G floats being stored all that is being stored is I by float. I also found some bugs in my code that made everything serial instead of concurrent. That was fixed but there was a bit of a bug that came with it, but it was nicely squashed. As of now I'm trying to log into the Clemson computers through a ssh terminal. The problem that im having is that i don't know if there's a deadlock in my code or if the terminal is just slow, but its taking along time for a small data call. Ill just wait. (- .-)

Ps. I went to get some water and the terminal is spitting out data, line by line, so ya the terminal is just slow (~...~)