OpenMPI  0.1.1
dss.h File Reference

Data packing subsystem. More...

#include "opal_config.h"
#include "opal/dss/dss_types.h"

Go to the source code of this file.

Data Structures

struct  opal_dss_t
 Base structure for the DSS. More...
 

Typedefs

typedef int(* opal_dss_pack_fn_t )(opal_buffer_t *buffer, const void *src, int32_t num_values, opal_data_type_t type)
 Top-level itnerface function to pack one or more values into a buffer. More...
 
typedef int(* opal_dss_unpack_fn_t )(opal_buffer_t *buffer, void *dest, int32_t *max_num_values, opal_data_type_t type)
 Unpack values from a buffer. More...
 
typedef int(* opal_dss_peek_next_item_fn_t )(opal_buffer_t *buffer, opal_data_type_t *type, int32_t *number)
 Get the type and number of values of the next item in the buffer. More...
 
typedef int(* opal_dss_unload_fn_t )(opal_buffer_t *buffer, void **payload, int32_t *size)
 Unload the data payload from a buffer. More...
 
typedef int(* opal_dss_load_fn_t )(opal_buffer_t *buffer, void *payload, int32_t size)
 Load a data payload into a buffer. More...
 
typedef int(* opal_dss_copy_payload_fn_t )(opal_buffer_t *dest, opal_buffer_t *src)
 Copy a payload from one buffer to another This function will append a copy of the payload in one buffer into another buffer. More...
 
typedef int(* opal_dss_copy_fn_t )(void **dest, void *src, opal_data_type_t type)
 Copy a data value from one location to another. More...
 
typedef int(* opal_dss_compare_fn_t )(const void *value1, const void *value2, opal_data_type_t type)
 Compare two data values. More...
 
typedef int(* opal_dss_size_fn_t )(size_t *size, void *src, opal_data_type_t type)
 Compute size of data value. More...
 
typedef int(* opal_dss_print_fn_t )(char **output, char *prefix, void *src, opal_data_type_t type)
 Print a data value. More...
 
typedef int(* opal_dss_dump_fn_t )(int output_stream, void *src, opal_data_type_t type)
 Print a data value to an output stream for debugging purposes. More...
 
typedef int(* opal_dss_set_fn_t )(opal_dss_value_t *value, void *new_value, opal_data_type_t type)
 Set a data value. More...
 
typedef int(* opal_dss_get_fn_t )(void **data, opal_dss_value_t *value, opal_data_type_t type)
 Get a data value. More...
 
typedef void(* opal_dss_release_fn_t )(opal_dss_value_t *value)
 Release the storage used by a data value. More...
 
typedef int(* opal_dss_register_fn_t )(opal_dss_pack_fn_t pack_fn, opal_dss_unpack_fn_t unpack_fn, opal_dss_copy_fn_t copy_fn, opal_dss_compare_fn_t compare_fn, opal_dss_size_fn_t size_fn, opal_dss_print_fn_t print_fn, opal_dss_release_fn_t release_fn, bool structured, const char *name, opal_data_type_t *type)
 Register a set of data handling functions. More...
 
typedef char *(* opal_dss_lookup_data_type_fn_t )(opal_data_type_t type)
 
typedef void(* opal_dss_dump_data_types_fn_t )(int output)
 
typedef struct opal_dss_t opal_dss_t
 

Functions

OPAL_DECLSPEC int opal_dss_open (void)
 DSS initialization function. More...
 
OPAL_DECLSPEC int opal_dss_close (void)
 DSS finalize function.
 

Variables

BEGIN_C_DECLS typedef int(* opal_dss_set_buffer_type_fn_t )(opal_buffer_t *buffer, opal_dss_buffer_type_t type)
 Set the buffer type. More...
 
OPAL_DECLSPEC opal_dss_t opal_dss
 

Detailed Description

Data packing subsystem.

Typedef Documentation

typedef int(* opal_dss_compare_fn_t)(const void *value1, const void *value2, opal_data_type_t type)

Compare two data values.

Since registered data types can be complex structures, the system needs some way to know how to compare two data values (e.g., when trying to order them in some fashion). This function, which can call other compare functions to build up complex data types, defines the method for comparing two values of the specified data type.

Return values
-1Indicates first value is greater than second value
0Indicates two values are equal
+1Indicates second value is greater than first value
typedef int(* opal_dss_copy_fn_t)(void **dest, void *src, opal_data_type_t type)

Copy a data value from one location to another.

Since registered data types can be complex structures, the system needs some way to know how to copy the data from one location to another (e.g., for storage in the registry). This function, which can call other copy functions to build up complex data types, defines the method for making a copy of the specified data type.

Parameters
**destThe address of a pointer into which the address of the resulting data is to be stored.
*srcA pointer to the memory location from which the data is to be copied.
typeThe type of the data to be copied - must be one of the DSS defined data types.
Return values
OPAL_SUCCESSThe value was successfully copied.
OPAL_ERROR(s)An appropriate error code.
typedef int(* opal_dss_copy_payload_fn_t)(opal_buffer_t *dest, opal_buffer_t *src)

Copy a payload from one buffer to another This function will append a copy of the payload in one buffer into another buffer.

If the destination buffer is NOT empty, then the type of the two buffers MUST match or else an error will be returned. If the destination buffer IS empty, then its type will be set to that of the source buffer. NOTE: This is NOT a destructive procedure - the source buffer's payload will remain intact, as will any pre-existing payload in the destination's buffer.

typedef int(* opal_dss_dump_fn_t)(int output_stream, void *src, opal_data_type_t type)

Print a data value to an output stream for debugging purposes.

Uses the dss.print command to obtain a string version of the data value and prints it to the designated output stream.

Return values
OPAL_SUCCESSThe value was successfully printed.
OPAL_ERROR(s)An appropriate error code.
typedef int(* opal_dss_get_fn_t)(void **data, opal_dss_value_t *value, opal_data_type_t type)

Get a data value.

Since the data values are stored in an opaque manner, the system needs a function by which it can get the data value from within the data_value object. This is the equivalent to a C++ access function.

NOTE: this function does NOT allocate any memory. It simply points the "data" location to that of the value, after ensuring that the value's type matches the specified one. Use "copy" if you want dynamic allocation of memory.

Return values
OPAL_SUCCESSThe value was successfully retrieved
OPAL_ERROR(s)An appropriate error code - usually caused by the specified type not matching the data type within the stored object.
typedef int(* opal_dss_load_fn_t)(opal_buffer_t *buffer, void *payload, int32_t size)

Load a data payload into a buffer.

The load function allows the caller to replace the payload in a buffer with one provided by the caller. If a payload already exists in the buffer, the function will "free" the existing data to release it, and then replace the data payload with the one provided by the caller.

Note
The buffer must be allocated in advance via the OBJ_NEW function call - failing to do so will cause the load function to return an error code.
The caller is responsible for pre-packing the provided payload - the load function cannot convert to network byte order any data contained in the provided payload.
Parameters
bufferA pointer to the buffer into which lthe payload is to be loaded.
payloadA void* pointer to the payload to be loaded into the buffer.
sizeThe size (in bytes) of the provided payload.
Return values
OPAL_SUCCESSThe request was successfully completed
OPAL_ERROR(s)An appropriate error code indicating the problem will be returned. This should be handled appropriately by the caller.
1 opal_buffer_t *buffer;
2 uint8_t bytes;
3 int32_t size;
4 
5 buffer = OBJ_NEW(opal_buffer_t);
6 status_code = opal_dss.load(buffer, (void*)(&bytes), size);
typedef int(* opal_dss_pack_fn_t)(opal_buffer_t *buffer, const void *src, int32_t num_values, opal_data_type_t type)

Top-level itnerface function to pack one or more values into a buffer.

The pack function packs one or more values of a specified type into the specified buffer. The buffer must have already been initialized via an OBJ_NEW or OBJ_CONSTRUCT call - otherwise, the pack_value function will return an error. Providing an unsupported type flag will likewise be reported as an error.

Note that any data to be packed that is not hard type cast (i.e., not type cast to a specific size) may lose precision when unpacked by a non-homogeneous recipient. The DSS will do its best to deal with heterogeneity issues between the packer and unpacker in such cases. Sending a number larger than can be handled by the recipient will return an error code (generated by the DSS upon unpacking) via the RML upon transmission - the DSS cannot detect such errors during packing.

Parameters
*bufferA pointer to the buffer into which the value is to be packed.
<em>srcA void pointer to the data that is to be packed. Note that strings are to be passed as (char **) - i.e., the caller must pass the address of the pointer to the string as the void*. This allows the DSS to use a single interface function, but still allow the caller to pass multiple strings in a single call.
numA size_t value indicating the number of values that are to be packed, beginning at the location pointed to by src. A string value is counted as a single value regardless of length. The values must be contiguous in memory. Arrays of pointers (e.g., string arrays) should be contiguous, although (obviously) the data pointed to need not be contiguous across array entries.
typeThe type of the data to be packed - must be one of the DSS defined data types.
Return values
OPAL_SUCCESSThe data was packed as requested.
OPAL_ERROR(s)An appropriate OPAL error code indicating the problem encountered. This error code should be handled appropriately.
1 opal_buffer_t *buffer;
2 int32_t src;
3 
4 status_code = opal_dss.pack(buffer, &src, 1, OPAL_INT32);
typedef int(* opal_dss_peek_next_item_fn_t)(opal_buffer_t *buffer, opal_data_type_t *type, int32_t *number)

Get the type and number of values of the next item in the buffer.

The peek function looks at the next item in the buffer and returns both its type and the number of values in the item. This is a non-destructive function call that does not disturb the buffer, so it can be called multiple times if desired.

Parameters
bufferA pointer to the buffer in question.
typeA pointer to an opal_data_type_t variable where the type of the next item in the buffer is to be stored. Caller must have memory backing this location.
numberA pointer to a int32_t variable where the number of data values in the next item is to be stored. Caller must have memory backing this location.
Return values
OPAL_SUCCESSRequested info was successfully returned.
OPAL_ERROR(s)An appropriate error code indicating the problem will be returned. This should be handled appropriately by the caller.
typedef int(* opal_dss_print_fn_t)(char **output, char *prefix, void *src, opal_data_type_t type)

Print a data value.

Since registered data types can be complex structures, the system needs some way to know how to print them (i.e., convert them to a string representation).

Return values
OPAL_SUCCESSThe value was successfully printed.
OPAL_ERROR(s)An appropriate error code.
typedef int(* opal_dss_register_fn_t)(opal_dss_pack_fn_t pack_fn, opal_dss_unpack_fn_t unpack_fn, opal_dss_copy_fn_t copy_fn, opal_dss_compare_fn_t compare_fn, opal_dss_size_fn_t size_fn, opal_dss_print_fn_t print_fn, opal_dss_release_fn_t release_fn, bool structured, const char *name, opal_data_type_t *type)

Register a set of data handling functions.

  • This function registers a set of data type functions for a specific type. An integer is returned that should be used a an argument to future invocations of opal_dss.pack(), opal_dss.unpack(), opal_dss.copy(), and opal_dss.compare, which will trigger calls to the appropriate functions. This is most useful when extending the datatypes that the dss can handle; pack and unpack functions can nest calls to opal_dss.pack() / opal_dss.unpack(), so defining small pack/unpack functions can be used recursively to build larger types (e.g., packing/unpacking structs can use calls to opal_dss.pack()/unpack() to serialize / deserialize individual members). This is likewise true for the copy and compare functions.
Parameters
release_fn[IN] Function pointer to the release routine
pack_fn[IN] Function pointer to the pack routine
unpack_fn[IN] Function pointer to the unpack routine
copy_fn[IN] Function pointer to copy routine
compare_fn[IN] Function pointer to compare routine
size_fn[IN] Function pointer to size routine
print_fn[IN] Function pointer to print routine
structured[IN] Boolean indicator as to whether or not the data is structured. A true value indicates that this data type is always passed via reference (i.e., a pointer to the object is passed) as opposed to directly (e.g., the way an int32_t would appear)
name[IN] String name for this pair (mainly for debugging)
type[OUT] Type number for this registration
Returns
OPAL_SUCCESS upon success
typedef void(* opal_dss_release_fn_t)(opal_dss_value_t *value)

Release the storage used by a data value.

Since the data values are stored in an opaque manner, the system needs a function by which it can release the storage associated with a value stored in a data value object.

typedef int(* opal_dss_set_fn_t)(opal_dss_value_t *value, void *new_value, opal_data_type_t type)

Set a data value.

Since the data values are stored in an opaque manner, the system needs a function by which it can set the data value to a specific value. This is the equivalent to a C++ access function.

NOTE: this function does NOT allocate any memory. It only sets the value pointer and type to the specified location and type. Use "copy" if you want dynamic allocation of storage.

Return values
OPAL_SUCCESSThe value was successfully stored
OPAL_ERROR(s)An appropriate error code.
typedef int(* opal_dss_size_fn_t)(size_t *size, void *src, opal_data_type_t type)

Compute size of data value.

Since registered data types can be complex structures, the system needs some way to compute its size. Some of these types, however, involve variable amounts of storage (e.g., a string!). Hence, a pointer to the actual object being "sized" needs to be passed as well.

Parameters
sizeAddress of a size_t value where the size of the data value (in bytes) will be stored - set to zero in event of error.
*srcA pointer to the memory location of the data object. It is okay for this to be NULL - if NULL, the function must return the size of the object itself, not including any data contained in its fields.
typeThe type of the data value - must be one of the DSS defined data types or an error will be returned.
Return values
OPAL_SUCCESSThe value was successfully copied.
OPAL_ERROR(s)An appropriate error code.
typedef int(* opal_dss_unload_fn_t)(opal_buffer_t *buffer, void **payload, int32_t *size)

Unload the data payload from a buffer.

The unload function provides the caller with a pointer to the data payload within the buffer and the size of that payload. This allows the user to directly access the payload - typically used in the RML to unload the payload from the buffer for transmission.

Note
This is a destructive operation. While the payload is undisturbed, the function will clear the buffer's pointers to the payload. Thus, the buffer and the payload are completely separated, leaving the caller free to OBJ_RELEASE the buffer.
Parameters
bufferA pointer to the buffer whose payload is to be unloaded.
payloadThe address to a void* pointer that is to be loaded with the address of the data payload in the buffer.
sizeThe size (in bytes) of the data payload in the buffer.
Return values
OPAL_SUCCESSThe request was succesfully completed.
OPAL_ERROR(s)An appropriate error code indicating the problem will be returned. This should be handled appropriately by the caller.
1 opal_buffer_t *buffer;
2 uint8_t *bytes;
3 int32_t size;
4 
5 status_code = opal_dss.unload(buffer, (void**)(&bytes), &size);
6 OBJ_RELEASE(buffer);
typedef int(* opal_dss_unpack_fn_t)(opal_buffer_t *buffer, void *dest, int32_t *max_num_values, opal_data_type_t type)

Unpack values from a buffer.

The unpack function unpacks the next value (or values) of a specified type from the specified buffer.

The buffer must have already been initialized via an OBJ_NEW or OBJ_CONSTRUCT call (and assumedly filled with some data) - otherwise, the unpack_value function will return an error. Providing an unsupported type flag will likewise be reported as an error, as will specifying a data type that DOES NOT match the type of the next item in the buffer. An attempt to read beyond the end of the stored data held in the buffer will also return an error.

NOTE: it is possible for the buffer to be corrupted and that the DSS will think there is a proper variable type at the beginning of an unpack region - but that the value is bogus (e.g., just a byte field in a string array that so happens to have a value that matches the specified data type flag). Therefore, the data type error check is NOT completely safe. This is true for ALL unpack functions.

Unpacking values is a "destructive" process - i.e., the values are removed from the buffer, thus reducing the buffer size. It is therefore not possible for the caller to re-unpack a value from the same buffer.

Warning: The caller is responsible for providing adequate memory storage for the requested data. The opal_dss_peek() function is provided to assist in meeting this requirement. As noted below, the user must provide a parameter indicating the maximum number of values that can be unpacked into the allocated memory. If more values exist in the buffer than can fit into the memory storage, then the dss will unpack what it can fit into that location and return an error code indicating that the buffer was only partially unpacked.

Note that any data that was not hard type cast (i.e., not type cast to a specific size) when packed may lose precision when unpacked by a non-homogeneous recipient. The DSS will do its best to deal with heterogeneity issues between the packer and unpacker in such cases. Sending a number larger than can be handled by the recipient will return an error code (generated by the DSS upon unpacking) via the RML upon transmission - the DSS cannot detect such errors during packing.

Parameters
*bufferA pointer to the buffer from which the value will be extracted.
<em>destA void pointer to the memory location into which the data is to be stored. Note that these values will be stored contiguously in memory. For strings, this pointer must be to (char **) to provide a means of supporting multiple string operations. The DSS unpack function will allocate memory for each string in the array - the caller must only provide adequate memory for the array of pointers.
*numA pointer to a int32_t value indicating the maximum number of values that are to be unpacked, beginning at the location pointed to by src. This is provided to help protect the caller from memory overrun. Note that a string value is counted as a single value regardless of length.
Note
The unpack function will return the actual number of values unpacked in this location.
Parameters
typeThe type of the data to be unpacked - must be one of the DSS defined data types.
Return values
*max_num_valuesThe number of values actually unpacked. In most cases, this should match the maximum number provided in the parameters - but in no case will it exceed the value of this parameter. Note that if you unpack fewer values than are actually available, the buffer will be in an unpackable state - the dss will return an error code to warn of this condition.
OPAL_SUCCESSThe next item in the buffer was successfully unpacked.
OPAL_ERROR(s)The unpack function returns an error code under one of several conditions: (a) the number of values in the item exceeds the max num provided by the caller; (b) the type of the next item in the buffer does not match the type specified by the caller; or (c) the unpack failed due to either an error in the buffer or an attempt to read past the end of the buffer.
1 opal_buffer_t *buffer;
2 int32_t dest;
3 char **string_array;
4 int32_t num_values;
5 
6 num_values = 1;
7 status_code = opal_dss.unpack(buffer, (void*)&dest, &num_values, OPAL_INT32);
8 
9 num_values = 5;
10 string_array = malloc(num_values*sizeof(char *));
11 status_code = opal_dss.unpack(buffer, (void*)(string_array), &num_values, OPAL_STRING);

Function Documentation

OPAL_DECLSPEC int opal_dss_open ( void  )

DSS initialization function.

In dynamic libraries, declared objects and functions don't get loaded until called. We need to ensure that the opal_dss function structure gets loaded, so we provide an "open" call that is executed as part of the program startup.

set the default buffer type. If we are in debug mode, then we default to fully described buffers. Otherwise, we default to non-described for brevity and performance

References mca_base_param_lookup_int(), mca_base_param_register_int(), OBJ_CONSTRUCT, OPAL_BOOL, OPAL_BYTE, OPAL_BYTE_OBJECT, OPAL_DATA_TYPE, OPAL_DATA_VALUE, opal_dss_initialized, OPAL_INT, OPAL_INT16, OPAL_INT32, OPAL_INT64, OPAL_INT8, OPAL_NODE_STAT, OPAL_NULL, OPAL_PID, opal_pointer_array_init(), OPAL_PSTAT, OPAL_SIZE, OPAL_STRING, OPAL_UINT, OPAL_UINT16, OPAL_UINT32, OPAL_UINT64, and OPAL_UINT8.

Referenced by opal_init_util().

Variable Documentation

BEGIN_C_DECLS typedef int(* opal_dss_set_buffer_type_fn_t)(opal_buffer_t *buffer, opal_dss_buffer_type_t type)

Set the buffer type.

The pack/unpack functions can work with two types of buffer - fully described (i.e., every data object is preceeded by an identifier as to the type of the object) and non-described (i.e., no type identifier included). This function allows the caller to set the buffer type to the specified type - the function first checks to ensure the buffer is empty as the type cannot be changed once data has already entered the buffer.

Parameters
*bufferA pointer to the buffer
typeThe new buffer type
Return values
OPAL_SUCCESSOperation successfully executed
OPAL_ERROR_VALUEAn appropriate error code