Group items tagged

[IPython-user] ipython1 and farm tasking Brian Granger ellisonbg.net@gmail.... Wed Feb 27 16:29:03 CST 2008 Previous message: [IPython-user] ipython1 and farm tasking Next message: [IPython-user] yet another leopard/readline question Messages sorted by: [ date ] [ thread ] [ subject ] [ author ] Alex, First, I would suggest updating your ipython1 install from our svn repository. We are about to push out a major new version and the documentation is _much_ better. Also, there are many new features that will hopefully help you. Here is a simple example (using the latest svn of ipython1): In [1]: from ipython1.kernel import client In [2]: mec = client.MultiEngineClient(('127.0.0.1',10105)) In [3]: tc = client.TaskClient(('127.0.0.1',10113)) In [4]: def fold_package(x): ...: return 2.0*x ...: In [5]: mec.push_function(dict(fold_package=fold_package)) Out[5]: [None, None, None, None] In [6]: tasks = [client.Task("y=fold_package(x)",push={'x':x},pull=('y',)) for x in range(128)] In [7]: task_ids = [tc.run(t) for t in tasks] In [8]: tc.barrier(task_ids) In [9]: task_results = [tc.get_task_result(tid) for tid in task_ids] In [10]: results = [tr.ns.y for tr in task_results] In [11]: print results [0.0, 2.0, 4.0, 6.0, 8.0, 10.0, 12.0, 14.0, 16.0, 18.0, 20.0, 22.0, 24.0, 26.0, 28.0, 30.0, 32.0, 34.0, 36.0, 38.0, 40.0, 42.0, 44.0, 46.0, 48.0, 50.0, 52.0, 54.0, 56.0, 58.0, 60.0, 62.0, 64.0, 66.0, 68.0, 70.0, 72.0, 74.0, 76.0, 78.0, 80.0, 82.0, 84.0, 86.0, 88.0, 90.0, 92.0, 94.0, 96.0, 98.0, 100.0, 102.0, 104.0, 106.0, 108.0, 110.0, 112.0, 114.0, 116.0, 118.0, 120.0, 122.0, 124.0, 126.0, 128.0, 130.0, 132.0, 134.0, 136.0, 138.0, 140.0, 142.0, 144.0, 146.0, 148.0, 150.0, 152.0, 154.0, 156.0, 158.0, 160.0, 162.0, 164.0, 166.0, 168.0, 170.0, 172.0, 174.0, 176.0, 178.0, 180.0, 182.0, 184.0, 186.0, 188.0, 190.0, 192.0, 194.0, 196.0, 198.0, 200.0, 202.0, 204.0, 206.0, 208.0, 210.0, 212.0, 214.0, 216.0, 218.0, 220.0, 222.0, 224.0, 226.0, 228.0, 230.0, 232.0, 234.0, 236.0, 238.0, 240.0, 242.0, 244.0, 246.0, 248.0, 250.0, 252.0, 254.0] Or if you don't need load balancing: # This sends the fold_package function for you! results = mec.map(fold_package, range(128)) Let us know if you run into other problems. Cheers, Brian

You can use Microsoft Script Control. If you have the win32 extensions of python, you can use python in place of vb in this control -open the VBA script editor - In menus/Tools/References add Microsoft Script Control -Make a new module and declare a new MsScriptControl.ScriptControl Global sc as new MsScriptControl.ScriptControl -Initialize the language attibute with python - Note that you and users of your document must have python and its win32 extensions installed. Activestate python distribustion include it. You can put sc.language="python" in the routine Workbook_Open() Now you can import python modules using ExecuteStatement method of the control in vba and have results from python functions with eval method. One interesting thing is that you can pass an object to the control with AddObject method and have python manipulate it. And so on..

Global sc As New MSScriptControl.ScriptControl Public Function os_getcwd() sc.Language = "python" sc.ExecuteStatement ("import os") os_getcwd = sc.Eval("os.getcwd()") End Function With this you can set your Excel formula to =os_getcwd() For me it returns "C:\Documents and Settings\Administrator\My Documents", which I needed to know at the time so I didn't have to screw around with the ever annoying pythonpath. You can put the first two lines of the function in the Workbook_Open hook, but I don't know where that is. I hope to use more Python in Excel soon. Hmm, actually, I suppose you can put those first two lines of the function after the Global declaration as well. I know just about zero VBScript and didn't get a chance to do anything else beyond proof of concept yet. I figured I would write something dynamic which allowed more transparent access to Python, maybe allowing formula like =py("os.getcwd()"), etc.

There are two applications that require using the NumPy array type in C extension modules: Access to numerical libraries: Extension modules can be used to make numerical libraries written in C (or languages linkable to C, such as Fortran) accessible to Python programs. The NumPy array type has the advantage of using the same data layout as arrays in C and Fortran. Mixed-language numerical code: In most numerical applications, only a small part of the total code is CPU time intensive. Only this part should thus be written in C, the rest can be written in Python. NumPy arrays are important for the interface between these two parts, because they provide equally simple access to their contents from Python and from C. This document is a tutorial for using NumPy arrays in C extensions.

 > According to the fact that all Thread run on the same CPU (if i didn't  > understand wrong), i'm asking if python will suffer from the future  > multicore CPU. Will not python use only one core, then a half or a  > quarter of CPU ? It could be a serious problem for the future of python... I agree that it could potentially be a serious hindrance for cpython if "multiple core" CPUs become commonplace. This is in contrast to jython and ironpython, both of which support multiple-cpu parallelism. Although I completely accept the usual arguments offered in defense of the GIL, i.e. that it isn't a problem in the great majority of use cases, I think that position will become more difficult to defend as desktop CPUs sprout more and more execution pipelines. I think that this also fits in with AM Kuchling's recent musing/thesis/prediction that the existing cpython VM may no longer be in use in 5 years, and that it may be superceded by python "interpreters" running on top of other VMs, namely the JVM, the CLR, Smalltalk VM, Parrot, etc, etc, etc. http://www.amk.ca/diary/archives/cat_python.html#003382 I too agree with Andrew's basic position: the Python language needs a period of library consolidation. There is so much duplication of functionality out there, with the situation only getting worse as people re-invent the wheel yet again using newer features such generators, gen-exps and decorators.