Pyevolve: work on the distributed Genetic Algorithm support
I’m currently working on the distributed GA support for the Pyevolve framework, the approach I’m using to network communication (individuals migration) is the hightweigh and unreliable UDP protocol, due it’s performance (for lan/wan) and broadcasting compatibility.
The distributed GA will have options to create the logical migration topology for the connected nodes (islands of evolution), migration schemes, migration rates and other parameters.
Why not use the processing module to utilize all procs’ available?
Hello, on the next release (0.6) it will have an option to use the multiprocessing module, but “distributed” is different from “multi-processed”, with distributed I mean the evolution on different remote nodes identified as “islands”, with migrations, migration model, etc…
Have you looked at Pyro (http://pyro.sourceforge.net/) for handling distributed computing?
I know Pyro and it’s a very good implementation of remote objects, but the problem is that specifically for GAs I think it will be unsuitable, I’m doing my own implementation, so I can control everything and optimize for GAs and GPs. Thank you for the comment !
From your last comment – I completely understand rolling your own…. However, I’m using the Python 2.6.1 mutiprocessing module for utilizing both multiple cores and multiple machines. Now the module (& Python for that matter) is less than ideal for doing this type of thing but it may be worth looking into.. I’m using both the multiprocessing.Pool and mutliprocessing.managers.*Mangers classes…. You can construct a remote manager with very little effort.
Thanks for the great work, Perone!! You’re a very talented fellow!
Cheers,
–Smitty
Hello Smitty, thanks for your kindly words. I’ve sucessfull implemented muticore processing using the multiprocessing module, it works nice with intensive fitness ! The next step I’m trying to do is to implement a communication between islands using UDP =) I think it will be very fast on internet, but there are problems to solve before hehe.
This is exactly what I am after. Pyevolve is perfect for my needs. Your code is very well written and documented!
Could you post an example GP using the Migration module?
Keep up the awesome work! Thanks.
Hello James ! Thank you for the comment !
The Migration module is still under development, I’ll rewrite the distributed code of Pyevolve to improve it and change some issues, for a while only the multiprocessing is fully functional. I’m working right now on a new interesting feature for GP =)
Sounds very interesting.. Can’t wait 🙂
Hi Perone
I in a very short span of time could understand the implementation of GP through the pyevolve. I started from the TSP and learnt to use for the Spatial Hyper Redundant Robots (SHR). I could resolve at the optimal link angles of the robotic link arms to reach the end effector to the desired target/destination. I am eagerly looking forward to your parallel implementation of the same. In case if you can open up the way to download the SVN version of the Multiprocessing i would be happy to try out the same. – Joseph Winston S
Hello Joseph, your application seems very interesting =) the multiprocessin feature is already available at the SVN repository in sourceforge.net, ina few days I’ll release the 0.6rc1 version of the framework with that feature enabled.
thanks for sharing this. it has been very helpful.
Hi Perone, Congratulations on your release of Pyevolve 0.6rc1! It’s really a fabulous! High extension!
Especially when I tried to implement a distributed GA, I found you had did it!
Originally I want to implement it under TCP/IP protocol because of its reliability. But now it seems that UDP is also feasible in a LAN environment. Do you have any examples to use your migration GA? If yes, I hope you can email me an example to illustrate how to use it. Many thx in advance.
Anyway, well done!
Hello James, the Network/Migration modules are not “production ready” yet, but they are working fine in some testes I’d done. All you need to to is to instantiate a Migration adapter (see Migration.py), then set it topology (setTopology) and then set it into the GSimpleGA class. Unfortunatelly I’ve no example how to use it right now because it should not enter as complete into the 0.6 release, but the code is fully documented and you can take a look at it.
Hello, Christian Perone
I found that PyEvolve is quite a good package. Thank you for your software.
I have a question. Following the example 18, I can find a funtion. How I can save it and reuse the function in another module ?
Thank you