examples

Here you will find example notebooks and scripts for working with IPython Parallel.

Tutorials

• Creating Clusters from Python
  • Cluster as a context manager
  • Launcher classes
  • Connecting to existing clusters
• Data Publication API
  • Setup
  • Simple publication
  • Simulation loop
• Parallel Magics
  • Single-engine views
  • Parallel Exceptions
  • Remote Cell Magics
  • Local Execution
• Visualizing AsyncResults
• Interactive widgets in IPython Parallel
• Broadcast View
• Tuning broadcast views
  • Coalescing replies
• Broadcasting arrays withmemmap
• Broadcasting arrays with MPI

Examples

• Monitoring an MPI Simulation -1
  • MPI simulation example
  • IPython tools to interactively monitor and plot the MPI results
  • Making a simulation object that can be monitored interactively
• Monitoring an MPI Simulation -2
  • Load IPython support for working with MPI tasks
  • MPI simulation example
  • IPython tools to interactively monitor and plot the MPI results
  • Interactive monitoring in the client of the published data
• Parallel decorator and map
• Using MPI with IPythonParallel
• Monte Carlo Options
  • Problem setup
  • Monte-Carlo option pricing function
  • Parallel computation across strike prices and volatilities
  • Process and visualize results
• Random Matrices
  • Wigner’s nearest neighbor eigenvalue distribution
  • Serial calculation of nearest neighbor eigenvalue distribution
  • Parallel calculation of nearest neighbor eigenvalue distribution

Integrating IPython Parallel with other tools

There are lots of cool tools for working with asynchronous and parallel execution. IPython Parallel aims to be fairly compatible with these, both by implementing explicit support via methods such as Client.become_dask, and by using standards such as the concurrent.futures.Future API.

• Using Futures with IPython Parallel
  • Executors
  • Comparing Executors
• IPython Parallel as a joblib backend
• Working with IPython and dask.distributed
  • Debugging distributed with IPython
• Using dill to serialize anything
  • Using dill in IPython Parallel

Non-notebook examples

This directory also contains some examples that are scripts instead of notebooks.

import glob
import os

from IPython.display import FileLink, FileLinks, display

FileLinks(".", included_suffixes=[".py"], recursive=False)

./
  dependencies.py
  task_profiler.py
  customresults.py
  itermapresult.py
  throughput.py
  fetchparse.py
  nwmerge.py
  iopubwatcher.py
  dagdeps.py
  phistogram.py
  task_mod.py

More substantial examples can be found in subdirectories:

for path in os.listdir("."):
    if os.path.isdir(path) and not path.startswith("."):
        display(FileLinks(path, included_suffixes=['.py', '.ipynb']))

interengine/
  communicator.py
  bintree.py
  bintree_script.py
  interengine.py

broadcast/
  Broadcast view.ipynb
  memmap Broadcast.ipynb
  MPI Broadcast.ipynb

daVinci Word Count/
  wordfreq.py
  pwordfreq.py

pi/
  parallelpi.py
  pidigits.py

rmt/
  rmtkernel.py
  rmt.ipynb

wave2D/
  communicator.py
  wavesolver.py
  parallelwave.py
  RectPartitioner.py
  parallelwave-mpi.py

previous

Launchers

next

Cluster API