Group items matching

in title, tags, annotations or url

[IPython-user] ipython1 and farm tasking Brian Granger ellisonbg.net@gmail.... Wed Feb 27 16:29:03 CST 2008 Previous message: [IPython-user] ipython1 and farm tasking Next message: [IPython-user] yet another leopard/readline question Messages sorted by: [ date ] [ thread ] [ subject ] [ author ] Alex, First, I would suggest updating your ipython1 install from our svn repository. We are about to push out a major new version and the documentation is _much_ better. Also, there are many new features that will hopefully help you. Here is a simple example (using the latest svn of ipython1): In [1]: from ipython1.kernel import client In [2]: mec = client.MultiEngineClient(('127.0.0.1',10105)) In [3]: tc = client.TaskClient(('127.0.0.1',10113)) In [4]: def fold_package(x): ...: return 2.0*x ...: In [5]: mec.push_function(dict(fold_package=fold_package)) Out[5]: [None, None, None, None] In [6]: tasks = [client.Task("y=fold_package(x)",push={'x':x},pull=('y',)) for x in range(128)] In [7]: task_ids = [tc.run(t) for t in tasks] In [8]: tc.barrier(task_ids) In [9]: task_results = [tc.get_task_result(tid) for tid in task_ids] In [10]: results = [tr.ns.y for tr in task_results] In [11]: print results [0.0, 2.0, 4.0, 6.0, 8.0, 10.0, 12.0, 14.0, 16.0, 18.0, 20.0, 22.0, 24.0, 26.0, 28.0, 30.0, 32.0, 34.0, 36.0, 38.0, 40.0, 42.0, 44.0, 46.0, 48.0, 50.0, 52.0, 54.0, 56.0, 58.0, 60.0, 62.0, 64.0, 66.0, 68.0, 70.0, 72.0, 74.0, 76.0, 78.0, 80.0, 82.0, 84.0, 86.0, 88.0, 90.0, 92.0, 94.0, 96.0, 98.0, 100.0, 102.0, 104.0, 106.0, 108.0, 110.0, 112.0, 114.0, 116.0, 118.0, 120.0, 122.0, 124.0, 126.0, 128.0, 130.0, 132.0, 134.0, 136.0, 138.0, 140.0, 142.0, 144.0, 146.0, 148.0, 150.0, 152.0, 154.0, 156.0, 158.0, 160.0, 162.0, 164.0, 166.0, 168.0, 170.0, 172.0, 174.0, 176.0, 178.0, 180.0, 182.0, 184.0, 186.0, 188.0, 190.0, 192.0, 194.0, 196.0, 198.0, 200.0, 202.0, 204.0, 206.0, 208.0, 210.0, 212.0, 214.0, 216.0, 218.0, 220.0, 222.0, 224.0, 226.0, 228.0, 230.0, 232.0, 234.0, 236.0, 238.0, 240.0, 242.0, 244.0, 246.0, 248.0, 250.0, 252.0, 254.0] Or if you don't need load balancing: # This sends the fold_package function for you! results = mec.map(fold_package, range(128)) Let us know if you run into other problems. Cheers, Brian

Note that it's more rebust to run methods on the public IPython api.I.e. do ip = ipshell.api and then ip.magic('px import os')You can explore the api interactively by playing with _ip object.