OpenMPI 0.1.1

- Main Page
- Related Pages
- Modules
- Data Structures
- Files

- File List
- Globals

Data Structures | Macros | Typedefs | Functions | Variables
op.h File Reference

Public interface for the MPI_Op handle. More...

`#include "ompi_config.h"`
`#include <stdio.h>`
`#include "mpi.h"`
`#include "opal/class/opal_object.h"`
`#include "ompi/datatype/ompi_datatype.h"`
`#include "ompi/mpi/f77/fint_2_int.h"`
`#include "ompi/mca/op/op.h"`

Go to the source code of this file.

## Data Structures
struct | ompi_op_t
| Back-end type of MPI_Op. More...
struct | ompi_predefined_op_t

## Macros
#define | OMPI_OP_FLAGS_INTRINSIC 0x0001
| Set if the MPI_Op is a built-in operation.
#define | OMPI_OP_FLAGS_FORTRAN_FUNC 0x0002
| Set if the callback function is in Fortran.
#define | OMPI_OP_FLAGS_CXX_FUNC 0x0004
| Set if the callback function is in C++.
#define | OMPI_OP_FLAGS_ASSOC 0x0008
| Set if the callback function is associative (MAX and SUM will both have ASSOC set – in fact, it will only not be set if we implement some extensions to MPI, because MPI says that all MPI_Op's should be associative, so this flag is really here for future expansion)
#define | OMPI_OP_FLAGS_FLOAT_ASSOC 0x0010
| Set if the callback function is associative for floating point operands (e.g., MPI_SUM will have ASSOC set, but will not have FLOAT_ASSOC set)
#define | OMPI_OP_FLAGS_COMMUTE 0x0020
| Set if the callback function is communative.
#define | PREDEFINED_OP_PAD (sizeof(void*) * 256)
| Padded struct to maintain back compatibiltiy. More...

## Typedefs
typedef void( | ompi_op_fortran_handler_fn_t )(void *, void *, MPI_Fint *, MPI_Fint *)
| Typedef for fortran user-defined MPI_Ops.
typedef void( | ompi_op_cxx_handler_fn_t )(void *, void *, int *, struct ompi_datatype_t **, MPI_User_function *op)
| Typedef for C++ op functions intercept (used for user-defined MPI::Ops). More...
typedef struct ompi_op_t | ompi_op_t
| Convenience typedef.
typedef struct ompi_predefined_op_t | ompi_predefined_op_t

## Functions
BEGIN_C_DECLS typedef void() | ompi_op_c_handler_fn_t (void *, void *, int *, struct ompi_datatype_t **)
| Typedef for C op functions for user-defined MPI_Ops. More...
OMPI_DECLSPEC | OBJ_CLASS_DECLARATION (ompi_op_t)
int | ompi_op_init (void)
| Initialize the op interface. More...
int | ompi_op_finalize (void)
| Finalize the op interface. More...
ompi_op_t * | ompi_op_create_user (bool commute, ompi_op_fortran_handler_fn_t func)
| Create a ompi_op_t with a user-defined callback (vs. More...
OMPI_DECLSPEC void | ompi_op_set_cxx_callback (ompi_op_t *op, MPI_User_function *fn)
| Mark an MPI_Op as holding a C++ callback function, and cache that function in the MPI_Op. More...
static bool | ompi_op_is_intrinsic (ompi_op_t *op)
| Check to see if an op is intrinsic. More...
static bool | ompi_op_is_commute (ompi_op_t *op)
| Check to see if an op is communative or not. More...
static bool | ompi_op_is_float_assoc (ompi_op_t *op)
| Check to see if an op is floating point associative or not. More...
static bool | ompi_op_is_valid (ompi_op_t *op, ompi_datatype_t *ddt, char **msg, const char *func)
| Check to see if an op is valid on a given datatype. More...
static void | ompi_op_reduce (ompi_op_t *op, void *source, void *target, int count, ompi_datatype_t *dtype)
| Perform a reduction operation. More...
static void | ompi_3buff_op_reduce (ompi_op_t *op, void *source1, void *source2, void *target, int count, ompi_datatype_t *dtype)
| Perform a reduction operation. More...

## Variables
OMPI_DECLSPEC int | ompi_op_ddt_map [OMPI_DATATYPE_MAX_PREDEFINED]
| Array to map ddt->id values to the corresponding position in the op function array. More...
OMPI_DECLSPEC ompi_predefined_op_t | ompi_mpi_op_null
| Global variable for MPI_OP_NULL.
OMPI_DECLSPEC ompi_predefined_op_t | ompi_mpi_op_max
| Global variable for MPI_MAX.
OMPI_DECLSPEC ompi_predefined_op_t | ompi_mpi_op_min
| Global variable for MPI_MIN.
OMPI_DECLSPEC ompi_predefined_op_t | ompi_mpi_op_sum
| Global variable for MPI_SUM.
OMPI_DECLSPEC ompi_predefined_op_t | ompi_mpi_op_prod
| Global variable for MPI_PROD.
OMPI_DECLSPEC ompi_predefined_op_t | ompi_mpi_op_land
| Global variable for MPI_LAND.
OMPI_DECLSPEC ompi_predefined_op_t | ompi_mpi_op_band
| Global variable for MPI_BAND.
OMPI_DECLSPEC ompi_predefined_op_t | ompi_mpi_op_lor
| Global variable for MPI_LOR.
OMPI_DECLSPEC ompi_predefined_op_t | ompi_mpi_op_bor
| Global variable for MPI_BOR.
OMPI_DECLSPEC ompi_predefined_op_t | ompi_mpi_op_lxor
| Global variable for MPI_LXOR.
OMPI_DECLSPEC ompi_predefined_op_t | ompi_mpi_op_bxor
| Global variable for MPI_BXOR.
OMPI_DECLSPEC ompi_predefined_op_t | ompi_mpi_op_maxloc
| Global variable for MPI_MAXLOC.
OMPI_DECLSPEC ompi_predefined_op_t | ompi_mpi_op_minloc
| Global variable for MPI_MINLOC.
OMPI_DECLSPEC ompi_predefined_op_t | ompi_mpi_op_replace
| Global variable for MPI_REPLACE.
struct opal_pointer_array_t * | ompi_op_f_to_c_table
| Table for Fortran <-> C op handle conversion.

## Detailed Description

Public interface for the MPI_Op handle.

## Macro Definition Documentation

#define PREDEFINED_OP_PAD (sizeof(void*) * 256)

Padded struct to maintain back compatibiltiy.

See ompi/communicator/communicator.h comments with struct ompi_communicator_t for full explanation why we chose the following padding construct for predefines.

## Typedef Documentation

typedef void( ompi_op_cxx_handler_fn_t)(void *, void *, int *, struct ompi_datatype_t **, MPI_User_function *op)

Typedef for C++ op functions intercept (used for user-defined MPI::Ops).

See the lengthy explanation for why this is different than the C intercept in ompi/mpi/cxx/intercepts.cc in the ompi_mpi_cxx_op_intercept() function.

## Function Documentation

static void ompi_3buff_op_reduce | ( | ompi_op_t * | op,
|  | void * | source1,
|  | void * | source2,
|  | void * | target,
|  | int | count,
|  | ompi_datatype_t * | dtype
| ) |  |
| inline static

Perform a reduction operation.

Parameters

  op | The operation (IN)
  source | Source1 (input) buffer (IN)
  source | Source2 (input) buffer (IN)
  target | Target (output) buffer (IN/OUT)
  count | Number of elements (IN)
  dtype | MPI datatype (IN)

Returns
  void As with MPI user-defined reduction functions, there is no return code from this function.

Perform a reduction operation with count elements of type dtype in the buffers source and target. The target buffer obtains the result (i.e., the original values in the target buffer are reduced with the values in the source buffer and the result is stored in the target buffer).

This function will only be invoked on intrinsic MPI_Ops.

Otherwise, this function is the same as ompi_op_reduce.

References ompi_datatype_t::id, ompi_op_t::o_3buff_intrinsic, and ompi_op_ddt_map.

BEGIN_C_DECLS typedef void() ompi_op_c_handler_fn_t | ( | void * | ,
|  | void * | ,
|  | int * | ,
|  | struct ompi_datatype_t ** |
| ) |  |

Typedef for C op functions for user-defined MPI_Ops.

We don't use MPI_User_function because this would create a confusing dependency loop between this file and mpi.h. So this is repeated code, but it's better this way (and this typedef will never change, so there's not much of a maintenance worry).

ompi_op_t* ompi_op_create_user | ( | bool | commute,
|  | ompi_op_fortran_handler_fn_t | func
| ) |  |

Create a ompi_op_t with a user-defined callback (vs.

creating an intrinsic ompi_op_t).

Parameters

  commute | Boolean indicating whether the operation is communative or not
  func | Function pointer of the error handler

Returns
  op Pointer to the ompi_op_t that will be created and returned

This function is called as the back-end of all the MPI_OP_CREATE function. It creates a new ompi_op_t object, initializes it to the correct object type, and sets the callback function on it.

The type of the function pointer is (arbitrarily) the fortran function handler type. Since this function has to accept 2 different function pointer types (lest we have 2 different functions to create errhandlers), the fortran one was picked arbitrarily. Note that (void*) is not sufficient because at least theoretically, a sizeof(void*) may not necessarily be the same as sizeof(void(*)).

NOTE: It always sets the "fortran" flag to false. The Fortran wrapper for MPI_OP_CREATE is expected to reset this flag to true manually.

References ompi_op_t::fort_fn, ompi_op_t::o_f_to_c_index, ompi_op_t::o_flags, ompi_op_t::o_func, OBJ_RELEASE, OMPI_OP_FLAGS_ASSOC, and OMPI_OP_FLAGS_COMMUTE.

int ompi_op_finalize | ( | void |  | ) |

Finalize the op interface.

Returns
  OMPI_SUCCESS Always

Invokes from ompi_mpi_finalize(); tears down the op interface, and destroys the F2C translation table.

References OBJ_DESTRUCT, and OBJ_RELEASE.

Referenced by ompi_mpi_finalize().

int ompi_op_init | ( | void |  | ) |

Initialize the op interface.

Returns
  OMPI_SUCCESS Upon success
  OMPI_ERROR Otherwise

Invoked from ompi_mpi_init(); sets up the op interface, creates the predefined MPI operations, and creates the corresopnding F2C translation table.

References OMPI_OP_BASE_FORTRAN_BAND, OMPI_OP_BASE_FORTRAN_BOR, OMPI_OP_BASE_FORTRAN_BXOR, OMPI_OP_BASE_FORTRAN_LAND, OMPI_OP_BASE_FORTRAN_LOR, OMPI_OP_BASE_FORTRAN_LXOR, OMPI_OP_BASE_FORTRAN_MAX, OMPI_OP_BASE_FORTRAN_MAXLOC, OMPI_OP_BASE_FORTRAN_MIN, OMPI_OP_BASE_FORTRAN_MINLOC, OMPI_OP_BASE_FORTRAN_NULL, OMPI_OP_BASE_FORTRAN_PROD, OMPI_OP_BASE_FORTRAN_REPLACE, OMPI_OP_BASE_FORTRAN_SUM, OMPI_OP_BASE_TYPE_2DOUBLE_PRECISION, OMPI_OP_BASE_TYPE_2INT, OMPI_OP_BASE_TYPE_2INTEGER, OMPI_OP_BASE_TYPE_2REAL, OMPI_OP_BASE_TYPE_BOOL, OMPI_OP_BASE_TYPE_COMPLEX16, OMPI_OP_BASE_TYPE_COMPLEX32, OMPI_OP_BASE_TYPE_COMPLEX8, OMPI_OP_BASE_TYPE_DOUBLE, OMPI_OP_BASE_TYPE_DOUBLE_INT, OMPI_OP_BASE_TYPE_DOUBLE_PRECISION, OMPI_OP_BASE_TYPE_FLOAT, OMPI_OP_BASE_TYPE_FLOAT_INT, OMPI_OP_BASE_TYPE_INT16_T, OMPI_OP_BASE_TYPE_INT32_T, OMPI_OP_BASE_TYPE_INT64_T, OMPI_OP_BASE_TYPE_INT8_T, OMPI_OP_BASE_TYPE_INTEGER, OMPI_OP_BASE_TYPE_LOGICAL, OMPI_OP_BASE_TYPE_LONG_DOUBLE, OMPI_OP_BASE_TYPE_LONG_DOUBLE_INT, OMPI_OP_BASE_TYPE_LONG_INT, OMPI_OP_BASE_TYPE_REAL, OMPI_OP_BASE_TYPE_SHORT_INT, OMPI_OP_BASE_TYPE_UINT16_T, OMPI_OP_BASE_TYPE_UINT32_T, OMPI_OP_BASE_TYPE_UINT64_T, OMPI_OP_BASE_TYPE_UINT8_T, OMPI_OP_BASE_TYPE_WCHAR, and ompi_op_ddt_map.

Referenced by ompi_mpi_init().

static bool ompi_op_is_commute | ( | ompi_op_t * | op | ) |
| inline static

Check to see if an op is communative or not.

Parameters

  op | The op to check

Returns
  true If the op is communative
  false If the op is not communative

Self-explanitory. This is needed in a few top-level MPI functions; this function is provided to hide the internal structure field names.

References ompi_op_t::o_flags, and OMPI_OP_FLAGS_COMMUTE.

static bool ompi_op_is_float_assoc | ( | ompi_op_t * | op | ) |
| inline static

Check to see if an op is floating point associative or not.

Parameters

  op | The op to check

Returns
  true If the op is floating point associative
  false If the op is not floating point associative

Self-explanitory. This is needed in a few top-level MPI functions; this function is provided to hide the internal structure field names.

References ompi_op_t::o_flags, and OMPI_OP_FLAGS_FLOAT_ASSOC.

Referenced by mca_coll_sm_reduce_intra().

static bool ompi_op_is_intrinsic | ( | ompi_op_t * | op | ) |
| inline static

Check to see if an op is intrinsic.

Parameters

  op | The op to check

Returns
  true If the op is intrinsic
  false If the op is not intrinsic

Self-explanitory. This is needed in a few top-level MPI functions; this function is provided to hide the internal structure field names.

References ompi_op_t::o_flags, and OMPI_OP_FLAGS_INTRINSIC.

Referenced by mca_coll_sm_reduce_intra(), and ompi_op_is_valid().

static bool ompi_op_is_valid | ( | ompi_op_t * | op,
|  | ompi_datatype_t * | ddt,
|  | char ** | msg,
|  | const char * | func
| ) |  |
| inline static

Check to see if an op is valid on a given datatype.

Parameters

  op | The op to check
  ddt | The datatype to check

Returns
  true If the op is valid on that datatype
  false If the op is not valid on that datatype

Self-explanitory. This is needed in a few top-level MPI functions; this function is provided to hide the internal structure field names.

References ompi_datatype_t::id, ompi_op_t::intrinsic, ompi_datatype_t::name, ompi_op_t::o_func, ompi_op_t::o_name, ompi_op_ddt_map, and ompi_op_is_intrinsic().

static void ompi_op_reduce | ( | ompi_op_t * | op,
|  | void * | source,
|  | void * | target,
|  | int | count,
|  | ompi_datatype_t * | dtype
| ) |  |
| inline static

Perform a reduction operation.

Parameters

  op | The operation (IN)
  source | Source (input) buffer (IN)
  target | Target (output) buffer (IN/OUT)
  count | Number of elements (IN)
  dtype | MPI datatype (IN)

Returns
  void As with MPI user-defined reduction functions, there is no return code from this function.

Perform a reduction operation with count elements of type dtype in the buffers source and target. The target buffer obtains the result (i.e., the original values in the target buffer are reduced with the values in the source buffer and the result is stored in the target buffer).

This function figures out which reduction operation function to invoke and whether to invoke it with C- or Fortran-style invocation methods. If the op is intrinsic and has the operation defined for dtype, the appropriate back-end function will be invoked. Otherwise, the op is assumed to be a user op and the first function pointer in the op array will be used.

NOTE: This function assumes that a correct combination will be given to it; it makes no provision for errors (in the name of optimization). If you give it an intrinsic op with a datatype that is not defined to have that operation, it is likely to seg fault.

References ompi_op_t::c_fn, ompi_op_t::cxx_data, ompi_datatype_t::d_f_to_c_index, ompi_op_t::fort_fn, ompi_datatype_t::id, ompi_op_t::intrinsic, ompi_op_t::o_flags, ompi_op_t::o_func, ompi_op_ddt_map, OMPI_OP_FLAGS_CXX_FUNC, OMPI_OP_FLAGS_FORTRAN_FUNC, and OMPI_OP_FLAGS_INTRINSIC.

Referenced by ompi_osc_base_process_op().

OMPI_DECLSPEC void ompi_op_set_cxx_callback | ( | ompi_op_t * | op,
|  | MPI_User_function * | fn
| ) |  |

Mark an MPI_Op as holding a C++ callback function, and cache that function in the MPI_Op.

See a lenghty comment in ompi/mpi/cxx/op.c::ompi_mpi_cxx_op_intercept() for a full expalantion.

References ompi_op_t::cxx_data, ompi_op_t::fort_fn, ompi_op_t::o_flags, ompi_op_t::o_func, and OMPI_OP_FLAGS_CXX_FUNC.

## Variable Documentation

OMPI_DECLSPEC int ompi_op_ddt_map[OMPI_DATATYPE_MAX_PREDEFINED]

Array to map ddt->id values to the corresponding position in the op function array.

NOTE: It is possible to have an implementation without this map. There are basically 3 choices for implementing "how to find the right position in the op array based on the datatype":

1. Use the exact same ordering as ddt->id in the op map. This is nice in that it's always a direct lookup via one memory de-reference. But it makes a sparse op array, and it's at least somewhat wasteful. It also chains the ddt and op implementations together. If the ddt ever changes its ordering, op is screwed. It seemed safer from a maintenance point of view not to do it that way.
2. Re-arrange the ddt ID values so that all the reducable types are at the beginning. This means that we can have a dense array here in op, but then we have the same problem as number one – and so this didn't seem like a good idea from a maintenance point of view.
3. Create a mapping between the ddt->id values and the position in the op array. This allows a nice dense op array, and if we make the map based on symbolic values, then if ddt ever changes its ordering, it won't matter to op. This seemed like the safest thing to do from a maintenance perspective, and since it only costs one extra lookup, and that lookup is way cheaper than the function call to invoke the reduction operation, it seemed like the best idea.

Referenced by ompi_3buff_op_reduce(), ompi_op_init(), ompi_op_is_valid(), and ompi_op_reduce().