Group items tagged

Pypar does not require the Python interpreter to be modified or recompiled: Parallel python programs use the standard Python and need merely import the pypar module. This means for example that you can upgrade Python independently of your parallel codes.

Pypar is an efficient but easy-to-use module that allows programs/scripts written in the Python programming language to run in parallel on multiple processors and communicate using message passing. Pypar provides bindings to an important subset of the message passing interface standard MPI. Other Python MPI bindings available from other developers include: PyMPI, Scientific Python and pythonMPI.

PyACTS- http://www.pyacts.org -- PyACTS is a collection of Python Modules that are very useful to Parallel Computing in a High Performance Computing environment. This packages incorporates several modules like PyBLACS (allows communication data for Linear Algebra), PyPBLAS (distributed Matrix Operations) and PyScaLAPACK (solve linear systems and get the eigenvalue problems). These libraries are part of PyACTS project that provide interfaces to the ACTS Collection. Also is provided a parrallel interpreter for using this package that implements message-based parallel programming using MPI.

 > According to the fact that all Thread run on the same CPU (if i didn't  > understand wrong), i'm asking if python will suffer from the future  > multicore CPU. Will not python use only one core, then a half or a  > quarter of CPU ? It could be a serious problem for the future of python... I agree that it could potentially be a serious hindrance for cpython if "multiple core" CPUs become commonplace. This is in contrast to jython and ironpython, both of which support multiple-cpu parallelism. Although I completely accept the usual arguments offered in defense of the GIL, i.e. that it isn't a problem in the great majority of use cases, I think that position will become more difficult to defend as desktop CPUs sprout more and more execution pipelines. I think that this also fits in with AM Kuchling's recent musing/thesis/prediction that the existing cpython VM may no longer be in use in 5 years, and that it may be superceded by python "interpreters" running on top of other VMs, namely the JVM, the CLR, Smalltalk VM, Parrot, etc, etc, etc. http://www.amk.ca/diary/archives/cat_python.html#003382 I too agree with Andrew's basic position: the Python language needs a period of library consolidation. There is so much duplication of functionality out there, with the situation only getting worse as people re-invent the wheel yet again using newer features such generators, gen-exps and decorators.

Linda is an widely studied distributed computing environment, centered around the notion of a tuple space. A tuple space is a bag (also called a multi-set) of tuples. A tuple is an ordered, typed chunk of data. Tuple spaces exist independently of processes in the system, and the data placed into a tuple space also exist independently. See "Generative communication in Linda" (1985) and "Multiple tuple spaces in Linda" both by David Gelernter for more information on Linda.

[IPython-user] ipython1 and farm tasking Brian Granger ellisonbg.net@gmail.... Wed Feb 27 16:29:03 CST 2008 Previous message: [IPython-user] ipython1 and farm tasking Next message: [IPython-user] yet another leopard/readline question Messages sorted by: [ date ] [ thread ] [ subject ] [ author ] Alex, First, I would suggest updating your ipython1 install from our svn repository. We are about to push out a major new version and the documentation is _much_ better. Also, there are many new features that will hopefully help you. Here is a simple example (using the latest svn of ipython1): In [1]: from ipython1.kernel import client In [2]: mec = client.MultiEngineClient(('127.0.0.1',10105)) In [3]: tc = client.TaskClient(('127.0.0.1',10113)) In [4]: def fold_package(x): ...: return 2.0*x ...: In [5]: mec.push_function(dict(fold_package=fold_package)) Out[5]: [None, None, None, None] In [6]: tasks = [client.Task("y=fold_package(x)",push={'x':x},pull=('y',)) for x in range(128)] In [7]: task_ids = [tc.run(t) for t in tasks] In [8]: tc.barrier(task_ids) In [9]: task_results = [tc.get_task_result(tid) for tid in task_ids] In [10]: results = [tr.ns.y for tr in task_results] In [11]: print results [0.0, 2.0, 4.0, 6.0, 8.0, 10.0, 12.0, 14.0, 16.0, 18.0, 20.0, 22.0, 24.0, 26.0, 28.0, 30.0, 32.0, 34.0, 36.0, 38.0, 40.0, 42.0, 44.0, 46.0, 48.0, 50.0, 52.0, 54.0, 56.0, 58.0, 60.0, 62.0, 64.0, 66.0, 68.0, 70.0, 72.0, 74.0, 76.0, 78.0, 80.0, 82.0, 84.0, 86.0, 88.0, 90.0, 92.0, 94.0, 96.0, 98.0, 100.0, 102.0, 104.0, 106.0, 108.0, 110.0, 112.0, 114.0, 116.0, 118.0, 120.0, 122.0, 124.0, 126.0, 128.0, 130.0, 132.0, 134.0, 136.0, 138.0, 140.0, 142.0, 144.0, 146.0, 148.0, 150.0, 152.0, 154.0, 156.0, 158.0, 160.0, 162.0, 164.0, 166.0, 168.0, 170.0, 172.0, 174.0, 176.0, 178.0, 180.0, 182.0, 184.0, 186.0, 188.0, 190.0, 192.0, 194.0, 196.0, 198.0, 200.0, 202.0, 204.0, 206.0, 208.0, 210.0, 212.0, 214.0, 216.0, 218.0, 220.0, 222.0, 224.0, 226.0, 228.0, 230.0, 232.0, 234.0, 236.0, 238.0, 240.0, 242.0, 244.0, 246.0, 248.0, 250.0, 252.0, 254.0] Or if you don't need load balancing: # This sends the fold_package function for you! results = mec.map(fold_package, range(128)) Let us know if you run into other problems. Cheers, Brian

Note that it's more rebust to run methods on the public IPython api.I.e. do ip = ipshell.api and then ip.magic('px import os')You can explore the api interactively by playing with _ip object.

Osh (Object SHell) is a tool that integrates the processing of structured data, database access, and remote access to a cluster of nodes. These capabilities are made available through a command-line interface (CLI) and a Python application programming interface (API). Osh processes streams of Python objects using simple commands. Complex data processing is achieved by command sequences in which the output from one command is passed to the input of the next. This is similar to composing Unix commands using pipes. However, Unix commands pass strings from one command to the next, and the commands (grep, awk, sed, etc.) are heavily string-oriented. Osh commands send primitive Python types such as strings and numbers; composite types such as tuples, lists and maps; objects representing files, dates and times; or even user-defined objects.