OpenMPI  0.1.1
op.h File Reference

Public interface for the MPI_Op handle. More...

#include "ompi_config.h"
#include <stdio.h>
#include "mpi.h"
#include "opal/class/opal_object.h"
#include "ompi/datatype/ompi_datatype.h"
#include "ompi/mpi/f77/fint_2_int.h"
#include "ompi/mca/op/op.h"

Go to the source code of this file.

Data Structures

struct  ompi_op_t
 Back-end type of MPI_Op. More...
 
struct  ompi_predefined_op_t
 

Macros

#define OMPI_OP_FLAGS_INTRINSIC   0x0001
 Set if the MPI_Op is a built-in operation.
 
#define OMPI_OP_FLAGS_FORTRAN_FUNC   0x0002
 Set if the callback function is in Fortran.
 
#define OMPI_OP_FLAGS_CXX_FUNC   0x0004
 Set if the callback function is in C++.
 
#define OMPI_OP_FLAGS_ASSOC   0x0008
 Set if the callback function is associative (MAX and SUM will both have ASSOC set – in fact, it will only not be set if we implement some extensions to MPI, because MPI says that all MPI_Op's should be associative, so this flag is really here for future expansion)
 
#define OMPI_OP_FLAGS_FLOAT_ASSOC   0x0010
 Set if the callback function is associative for floating point operands (e.g., MPI_SUM will have ASSOC set, but will not have FLOAT_ASSOC set)
 
#define OMPI_OP_FLAGS_COMMUTE   0x0020
 Set if the callback function is communative.
 
#define PREDEFINED_OP_PAD   (sizeof(void*) * 256)
 Padded struct to maintain back compatibiltiy. More...
 

Typedefs

typedef void( ompi_op_fortran_handler_fn_t )(void *, void *, MPI_Fint *, MPI_Fint *)
 Typedef for fortran user-defined MPI_Ops.
 
typedef void( ompi_op_cxx_handler_fn_t )(void *, void *, int *, struct ompi_datatype_t **, MPI_User_function *op)
 Typedef for C++ op functions intercept (used for user-defined MPI::Ops). More...
 
typedef struct ompi_op_t ompi_op_t
 Convenience typedef.
 
typedef struct ompi_predefined_op_t ompi_predefined_op_t
 

Functions

BEGIN_C_DECLS typedef void() ompi_op_c_handler_fn_t (void *, void *, int *, struct ompi_datatype_t **)
 Typedef for C op functions for user-defined MPI_Ops. More...
 
OMPI_DECLSPEC OBJ_CLASS_DECLARATION (ompi_op_t)
 
int ompi_op_init (void)
 Initialize the op interface. More...
 
int ompi_op_finalize (void)
 Finalize the op interface. More...
 
ompi_op_tompi_op_create_user (bool commute, ompi_op_fortran_handler_fn_t func)
 Create a ompi_op_t with a user-defined callback (vs. More...
 
OMPI_DECLSPEC void ompi_op_set_cxx_callback (ompi_op_t *op, MPI_User_function *fn)
 Mark an MPI_Op as holding a C++ callback function, and cache that function in the MPI_Op. More...
 
static bool ompi_op_is_intrinsic (ompi_op_t *op)
 Check to see if an op is intrinsic. More...
 
static bool ompi_op_is_commute (ompi_op_t *op)
 Check to see if an op is communative or not. More...
 
static bool ompi_op_is_float_assoc (ompi_op_t *op)
 Check to see if an op is floating point associative or not. More...
 
static bool ompi_op_is_valid (ompi_op_t *op, ompi_datatype_t *ddt, char **msg, const char *func)
 Check to see if an op is valid on a given datatype. More...
 
static void ompi_op_reduce (ompi_op_t *op, void *source, void *target, int count, ompi_datatype_t *dtype)
 Perform a reduction operation. More...
 
static void ompi_3buff_op_reduce (ompi_op_t *op, void *source1, void *source2, void *target, int count, ompi_datatype_t *dtype)
 Perform a reduction operation. More...
 

Variables

OMPI_DECLSPEC int ompi_op_ddt_map [OMPI_DATATYPE_MAX_PREDEFINED]
 Array to map ddt->id values to the corresponding position in the op function array. More...
 
OMPI_DECLSPEC ompi_predefined_op_t ompi_mpi_op_null
 Global variable for MPI_OP_NULL.
 
OMPI_DECLSPEC ompi_predefined_op_t ompi_mpi_op_max
 Global variable for MPI_MAX.
 
OMPI_DECLSPEC ompi_predefined_op_t ompi_mpi_op_min
 Global variable for MPI_MIN.
 
OMPI_DECLSPEC ompi_predefined_op_t ompi_mpi_op_sum
 Global variable for MPI_SUM.
 
OMPI_DECLSPEC ompi_predefined_op_t ompi_mpi_op_prod
 Global variable for MPI_PROD.
 
OMPI_DECLSPEC ompi_predefined_op_t ompi_mpi_op_land
 Global variable for MPI_LAND.
 
OMPI_DECLSPEC ompi_predefined_op_t ompi_mpi_op_band
 Global variable for MPI_BAND.
 
OMPI_DECLSPEC ompi_predefined_op_t ompi_mpi_op_lor
 Global variable for MPI_LOR.
 
OMPI_DECLSPEC ompi_predefined_op_t ompi_mpi_op_bor
 Global variable for MPI_BOR.
 
OMPI_DECLSPEC ompi_predefined_op_t ompi_mpi_op_lxor
 Global variable for MPI_LXOR.
 
OMPI_DECLSPEC ompi_predefined_op_t ompi_mpi_op_bxor
 Global variable for MPI_BXOR.
 
OMPI_DECLSPEC ompi_predefined_op_t ompi_mpi_op_maxloc
 Global variable for MPI_MAXLOC.
 
OMPI_DECLSPEC ompi_predefined_op_t ompi_mpi_op_minloc
 Global variable for MPI_MINLOC.
 
OMPI_DECLSPEC ompi_predefined_op_t ompi_mpi_op_replace
 Global variable for MPI_REPLACE.
 
struct opal_pointer_array_tompi_op_f_to_c_table
 Table for Fortran <-> C op handle conversion.
 

Detailed Description

Public interface for the MPI_Op handle.

Macro Definition Documentation

#define PREDEFINED_OP_PAD   (sizeof(void*) * 256)

Padded struct to maintain back compatibiltiy.

See ompi/communicator/communicator.h comments with struct ompi_communicator_t for full explanation why we chose the following padding construct for predefines.

Typedef Documentation

typedef void( ompi_op_cxx_handler_fn_t)(void *, void *, int *, struct ompi_datatype_t **, MPI_User_function *op)

Typedef for C++ op functions intercept (used for user-defined MPI::Ops).

See the lengthy explanation for why this is different than the C intercept in ompi/mpi/cxx/intercepts.cc in the ompi_mpi_cxx_op_intercept() function.

Function Documentation

static void ompi_3buff_op_reduce ( ompi_op_t op,
void *  source1,
void *  source2,
void *  target,
int  count,
ompi_datatype_t dtype 
)
inlinestatic

Perform a reduction operation.

Parameters
opThe operation (IN)
sourceSource1 (input) buffer (IN)
sourceSource2 (input) buffer (IN)
targetTarget (output) buffer (IN/OUT)
countNumber of elements (IN)
dtypeMPI datatype (IN)
Returns
void As with MPI user-defined reduction functions, there is no return code from this function.

Perform a reduction operation with count elements of type dtype in the buffers source and target. The target buffer obtains the result (i.e., the original values in the target buffer are reduced with the values in the source buffer and the result is stored in the target buffer).

This function will only be invoked on intrinsic MPI_Ops.

Otherwise, this function is the same as ompi_op_reduce.

References ompi_datatype_t::id, ompi_op_t::o_3buff_intrinsic, and ompi_op_ddt_map.

BEGIN_C_DECLS typedef void() ompi_op_c_handler_fn_t ( void *  ,
void *  ,
int *  ,
struct ompi_datatype_t **   
)

Typedef for C op functions for user-defined MPI_Ops.

We don't use MPI_User_function because this would create a confusing dependency loop between this file and mpi.h. So this is repeated code, but it's better this way (and this typedef will never change, so there's not much of a maintenance worry).

ompi_op_t* ompi_op_create_user ( bool  commute,
ompi_op_fortran_handler_fn_t  func 
)

Create a ompi_op_t with a user-defined callback (vs.

creating an intrinsic ompi_op_t).

Parameters
commuteBoolean indicating whether the operation is communative or not
funcFunction pointer of the error handler
Returns
op Pointer to the ompi_op_t that will be created and returned

This function is called as the back-end of all the MPI_OP_CREATE function. It creates a new ompi_op_t object, initializes it to the correct object type, and sets the callback function on it.

The type of the function pointer is (arbitrarily) the fortran function handler type. Since this function has to accept 2 different function pointer types (lest we have 2 different functions to create errhandlers), the fortran one was picked arbitrarily. Note that (void*) is not sufficient because at least theoretically, a sizeof(void*) may not necessarily be the same as sizeof(void(*)).

NOTE: It always sets the "fortran" flag to false. The Fortran wrapper for MPI_OP_CREATE is expected to reset this flag to true manually.

References ompi_op_t::fort_fn, ompi_op_t::o_f_to_c_index, ompi_op_t::o_flags, ompi_op_t::o_func, OBJ_RELEASE, OMPI_OP_FLAGS_ASSOC, and OMPI_OP_FLAGS_COMMUTE.

int ompi_op_finalize ( void  )

Finalize the op interface.

Returns
OMPI_SUCCESS Always

Invokes from ompi_mpi_finalize(); tears down the op interface, and destroys the F2C translation table.

References OBJ_DESTRUCT, and OBJ_RELEASE.

Referenced by ompi_mpi_finalize().

int ompi_op_init ( void  )
static bool ompi_op_is_commute ( ompi_op_t op)
inlinestatic

Check to see if an op is communative or not.

Parameters
opThe op to check
Returns
true If the op is communative
false If the op is not communative

Self-explanitory. This is needed in a few top-level MPI functions; this function is provided to hide the internal structure field names.

References ompi_op_t::o_flags, and OMPI_OP_FLAGS_COMMUTE.

static bool ompi_op_is_float_assoc ( ompi_op_t op)
inlinestatic

Check to see if an op is floating point associative or not.

Parameters
opThe op to check
Returns
true If the op is floating point associative
false If the op is not floating point associative

Self-explanitory. This is needed in a few top-level MPI functions; this function is provided to hide the internal structure field names.

References ompi_op_t::o_flags, and OMPI_OP_FLAGS_FLOAT_ASSOC.

Referenced by mca_coll_sm_reduce_intra().

static bool ompi_op_is_intrinsic ( ompi_op_t op)
inlinestatic

Check to see if an op is intrinsic.

Parameters
opThe op to check
Returns
true If the op is intrinsic
false If the op is not intrinsic

Self-explanitory. This is needed in a few top-level MPI functions; this function is provided to hide the internal structure field names.

References ompi_op_t::o_flags, and OMPI_OP_FLAGS_INTRINSIC.

Referenced by mca_coll_sm_reduce_intra(), and ompi_op_is_valid().

static bool ompi_op_is_valid ( ompi_op_t op,
ompi_datatype_t ddt,
char **  msg,
const char *  func 
)
inlinestatic

Check to see if an op is valid on a given datatype.

Parameters
opThe op to check
ddtThe datatype to check
Returns
true If the op is valid on that datatype
false If the op is not valid on that datatype

Self-explanitory. This is needed in a few top-level MPI functions; this function is provided to hide the internal structure field names.

References ompi_datatype_t::id, ompi_op_t::intrinsic, ompi_datatype_t::name, ompi_op_t::o_func, ompi_op_t::o_name, ompi_op_ddt_map, and ompi_op_is_intrinsic().

static void ompi_op_reduce ( ompi_op_t op,
void *  source,
void *  target,
int  count,
ompi_datatype_t dtype 
)
inlinestatic

Perform a reduction operation.

Parameters
opThe operation (IN)
sourceSource (input) buffer (IN)
targetTarget (output) buffer (IN/OUT)
countNumber of elements (IN)
dtypeMPI datatype (IN)
Returns
void As with MPI user-defined reduction functions, there is no return code from this function.

Perform a reduction operation with count elements of type dtype in the buffers source and target. The target buffer obtains the result (i.e., the original values in the target buffer are reduced with the values in the source buffer and the result is stored in the target buffer).

This function figures out which reduction operation function to invoke and whether to invoke it with C- or Fortran-style invocation methods. If the op is intrinsic and has the operation defined for dtype, the appropriate back-end function will be invoked. Otherwise, the op is assumed to be a user op and the first function pointer in the op array will be used.

NOTE: This function assumes that a correct combination will be given to it; it makes no provision for errors (in the name of optimization). If you give it an intrinsic op with a datatype that is not defined to have that operation, it is likely to seg fault.

References ompi_op_t::c_fn, ompi_op_t::cxx_data, ompi_datatype_t::d_f_to_c_index, ompi_op_t::fort_fn, ompi_datatype_t::id, ompi_op_t::intrinsic, ompi_op_t::o_flags, ompi_op_t::o_func, ompi_op_ddt_map, OMPI_OP_FLAGS_CXX_FUNC, OMPI_OP_FLAGS_FORTRAN_FUNC, and OMPI_OP_FLAGS_INTRINSIC.

Referenced by ompi_osc_base_process_op().

OMPI_DECLSPEC void ompi_op_set_cxx_callback ( ompi_op_t op,
MPI_User_function *  fn 
)

Mark an MPI_Op as holding a C++ callback function, and cache that function in the MPI_Op.

See a lenghty comment in ompi/mpi/cxx/op.c::ompi_mpi_cxx_op_intercept() for a full expalantion.

References ompi_op_t::cxx_data, ompi_op_t::fort_fn, ompi_op_t::o_flags, ompi_op_t::o_func, and OMPI_OP_FLAGS_CXX_FUNC.

Variable Documentation

OMPI_DECLSPEC int ompi_op_ddt_map[OMPI_DATATYPE_MAX_PREDEFINED]

Array to map ddt->id values to the corresponding position in the op function array.

NOTE: It is possible to have an implementation without this map. There are basically 3 choices for implementing "how to find the right position in the op array based on the datatype":

  1. Use the exact same ordering as ddt->id in the op map. This is nice in that it's always a direct lookup via one memory de-reference. But it makes a sparse op array, and it's at least somewhat wasteful. It also chains the ddt and op implementations together. If the ddt ever changes its ordering, op is screwed. It seemed safer from a maintenance point of view not to do it that way.
  2. Re-arrange the ddt ID values so that all the reducable types are at the beginning. This means that we can have a dense array here in op, but then we have the same problem as number one – and so this didn't seem like a good idea from a maintenance point of view.
  3. Create a mapping between the ddt->id values and the position in the op array. This allows a nice dense op array, and if we make the map based on symbolic values, then if ddt ever changes its ordering, it won't matter to op. This seemed like the safest thing to do from a maintenance perspective, and since it only costs one extra lookup, and that lookup is way cheaper than the function call to invoke the reduction operation, it seemed like the best idea.

Referenced by ompi_3buff_op_reduce(), ompi_op_init(), ompi_op_is_valid(), and ompi_op_reduce().