Group items tagged

The first textbook of its kind, Programming Massively Parallel Processors: A Hands-on Approach launches today, authored by Dr. David B. Kirk, NVIDIA Fellow and former chief scientist, and Dr. Wen-mei Hwu, who serves at the University of Illinois at Urbana-Champaign as Chair of Electrical and Computer Engineering in the Coordinated Science Laboratory, co-director of the Universal Parallel Computing Research Center and principal investigator of the CUDA Center of Excellence. The textbook, which is 256 pages, is the first aimed at teaching advanced students and professionals the basic concepts of parallel programming and GPU architectures. Published by Morgan Kaufmann, it explores various techniques for constructing parallel programs and reviews numerous case studies. With conventional CPU-based computing no longer scaling in performance and the world’s computational challenges increasing in complexity, the need for massively parallel processing has never been greater. GPUs have hundreds of cores capable of delivering transformative performance increases across a wide range of computational challenges. The rise of these multi-core architectures has raised the need to teach advanced programmers a new and essential skill: how to program massively parallel processors.

In case you haven’t realized it, the new trend in computer chip technology is multi-core. This is where most of the speed improvements moving forward will come from on our computers. To take full advantage of this however it is necessary to design your applications using Parallel Programming practices, also known as "parallelism". In today’s episode, we will meet with Stephen Toub, who will share with us some of the overarching concepts associated with parallelism, and some of the ways we are trying to empower developers to develop applications to take advantage of it.

In this session, Don McCrady discusses how C++ programmers can fully utilize multicore in their applications using the Concurrency Runtime (ConcRT), the Parallel Pattern Library (PPL), and the Asynchronous Agents Library that ship with Visual Studio 2010.

Ralph Johnson presents a pattern language that he and his colleagues are working on in an attempt to solve the hard issues of parallel programming through a set of design patterns: Structural Patterns, Computational Patterns, Parallel Algorithm Strategy Patterns, Implementation Strategy Patterns, and Concurrent Execution Patterns.

National Instruments has released a vision module for the PXI platform that provides a high-performance parallel processing architecture for hardware-defined timing, control and image pre-processing. The NI 1483 Camera Link adapter module, in combination with an NI FlexRIO field-programmable gate array (FPGA) board, offers a solution for embedding vision and control algorithms directly on FPGAs which are used to process and analyse an image in real time with little to no CPU intervention. The FPGAs can be used to perform operations by pixel, line and region of interest. They can implement many image processing algorithms that are inherently parallel, including fast Fourier transforms (FFTs), thresholding and filtering.

This 75-second video provides a quick introduction to the Concurrency Visualizer, a new profiling tool available in Visual Studio 2010. The Concurrency Visualizer enables you to look under the hood of your parallel applications and quickly discover performance bottlenecks. If you’re hungry to learn more, visit http://blogs.msdn.com/visualizeparallel.

Xilinx Inc. today introduced the architecture for a new Extensible Processing Platform they claim will deliver unrivaled levels of system performance, flexibility and integration to developers of a wide variety of embedded systems. The ARM Cortex-A9 MPCore processor-based platform enables system architects and embedded software developers to apply a combination of serial and parallel processing to address the challenges they face in designing today's embedded systems, which must meet ever-growing demands to perform highly complex functions. The Xilinx Extensible Processing Platform offers embedded systems designers a processor-centric design and development approach for achieving the compute and processing horsepower required to drive tasks involving high-speed access to real-time inputs, high-performance processing and complex digital signal processing - or any combination thereof - needed to meet their application-specific requirements, including lower cost and power.

Erlang allows for massively scalable concurrency, often with millions of lightweight, thread-like components known as actors. Unfortunately, using Erlang requires rewriting all of your legacy code into a rather esoteric language. But there are other options, such as the little known CCR platform that was developed by .NET's robotics department. Actor based languages such as Erlang are able to achieve high degrees of parallelism by using the Actor model. Under this model the fundamental unit of concurrency is not a thread or fiber, but rather something much smaller. Known as a "process" in Erlang, each unit of concurrency has a base overhead of about 1200 bytes on a 32-bit system. By comparison, a thread on the Windows operating system defaults to 1 MB just for the stack, additional space is also needed for bookkeeping and thread local storage. Because they are so lightweight, an application can spawn literally millions of processes simultaneously.

The MATLAB team at MathWorks tested performance scaling of the backslash ("\") matrix division operator to solve for x in the equation A*x = b. In their testing, matrix A occupies far more memory (290 GB) than is available in a single high-end desktop machine—typically a quad core processor with 4-8 GB of RAM, supplying approximately 20 Gigaflops. Therefore, they spread the calculation across machines. In order to solve linear systems of equations they need to be able to access all of the elements of the array even when the array is spread across multiple machines. This problem requires significant amounts of network communication, memory access, and CPU power. They scaled up to a cluster in EC2, giving them the ability to work with larger arrays and to perform calculations at up to 1.3 Teraflops, a 60X improvement. They were able to do this without making any changes to the application code. Here's a graph showing the near-linear scalability of an EC2 cluster across a range of matrix sizes with corresponding increases in cluster size for MATLAB's parallel backslash operator:

When a data (typically a variable) can be accessed by several threads, you must synchronize the access to the data to ensure visibility and correctness.

As many long time readers know, I am an enthusiast of the F# programming language.  I make no apologies for the fact that, if you are developing software on the .NET platform, F# is one of the better choices you can make for numerous reasons.  It is one of the reasons I proudly contributed as a co-author to the book, Professional F# 2.0, which is being published by Wrox in October. Some of the oft cited benefits of F# are that, to distill them quickly, it is good at doing intensely mathematical operations, it is built for parallelism, and it is good at helping define domain specific languages.  Those benefits are so often cited by speakers on the F# speaker circuit that they pretty much seem cliche to me at this point (note, yours truly is proud to call himself a member of said circuit, and often gives this talk!)  As great as these features are, there are a couple features, that in my more mundane F# experiences, seem to stand out as things that “save my ass”, for lack of a better phrase, more often than not.

DMA is used in almost every complex system or subsystems, but it's observed that teams either build the DMA controller from scratch for each project for specific application or take the existing DMAC available from elsewhere. This article discusses the architecture of DMAC that can be used with any kind of Bus, configuration (parallel, serial transfers), can be connected to any kind of ports, most importantly any kind of software assumptions can be implemented in the DMAC very easily.

While the new multicore system on chip (SoC) signal-processing architecture announced by Texas Instruments this week at Mobile World Congress hits all the right notes with respect to what's needed in next-generation basestation designs, it rings a bit hollow given how sketchy the architectural details remain when contrasted with more 'real' announcements from the likes of Freescale. For sure, the requirements of next-generation basestations will push all architectures to their limits and beyond. Balancing lower power and lower cost with increasingly parallel, math-intensive processing to meet multiuser demands for high-data-rate data in 3GPP Long Term Evolution (LTE) Release 8 all-IP networks is not going to be easy, especially with the introduction of MIMO, beam forming, OFDMA and many other enhancements engineered to maximize spectral efficiency.

Researchers at North Carolina State University have developed a new approach to software development that will allow common computer programs to run up to 20 percent faster and possibly incorporate new security measures. The researchers have found a way to run different parts of some hard-to-parallelize programs — such as word processors and web browsers — at the same time, which makes the programs operate more efficiently.

Today sees the launch of Visual Studio 2010, at five launch events around the world, as announced by Bob Muglia, Jason Zander and S. Somasegar, and presented live today in Las Vegas.   Visual Studio 2010 includes the official version 2.0 of the F# language. As is our custom on the F# team, we also release a matching MSI and ZIP of F# 2.0 (for use with Visual Studio 2008 and as a standalone compiler on a range of platforms)   Today represents the culmination of 7 years of work on the language at Microsoft Research, and, more recently, the Microsoft Developer Division. I am immensely proud of what we’ve achieved. F# brings a productive functional and object-oriented programming language to .NET, extending the platform to new audiences in technical, algorithmic, data-rich, parallel and explorative domains, and its inclusion in Visual Studio 2010 represents a huge milestone for the language.   To help understand what we’re doing with F#, I’ve listed some of the common questions people have about the language below.  We thank everyone who has been involved in the production of F#, especially the many users who have given us feedback on the language!