Hints

If ROMIO doesn't understand a hint, or if the value is invalid, the hint will be ignored. The values of hints being used by ROMIO for a file can be obtained at any time via MPI_File_get_info.

The following hints control the data sieving optimization and are applicable to all file system types:

ind_rd_buffer_size - Controls the size (in bytes) of the intermediate buffer used by ROMIO when performing data sieving during read operations. Default is 4194304 (4 Mbytes).
ind_wr_buffer_size - Controls the size (in bytes) of the intermediate buffer used by ROMIO when performing data sieving during write operations. Default is 524288 (512 Kbytes).
romio_ds_read - Determines when ROMIO will choose to perform data sieving. Valid values are enable, disable, or automatic. Default value is automatic. In automatic mode ROMIO may choose to enable or disable data sieving based on heuristics.
romio_ds_write - Same as above, only for writes.

The following hints control the two-phase (collective buffering) optimization and are applicable to all file system types:

cb_buffer_size - Controls the size (in bytes) of the intermediate buffer used in two-phase collective I/O. If the amount of data that an aggregator will transfer is larger than this value, then multiple operations are used. The default is 4194304 (4 Mbytes).
cb_nodes - Controls the maximum number of aggregators to be used. By default this is set to the number of unique hosts in the communicator used when opening the file.
romio_cb_read - Controls when collective buffering is applied to collective read operations. Valid values are enable, disable, and automatic. Default is automatic. When enabled, all collective reads will use collective buffering. When disabled, all collective reads will be serviced with individual operations by each process. When set to automatic, ROMIO will use heuristics to determine when to enable the optimization.
romio_cb_write - Controls when collective buffering is applied to collective write operations. Valid values are enable, disable, and automatic. Default is automatic. See the description of romio_cb_read for an explanation of the values.
romio_no_indep_rw - This hint controls when ``deferred open'' is used. When set to true, ROMIO will make an effort to avoid performing any file operation on non-aggregator nodes. The application is expected to use only collective operations. This is discussed in further detail below.
cb_config_list - Provides explicit control over aggregators. This is discussed in further detail below.

For some systems configurations, more control is needed to specify which hardware resources (processors or nodes in an SMP) are preferred for collective I/O, either for performance reasons or because only certain resources have access to storage. The additional MPI_Info key name cb_config_list specifies a comma-separated list of strings, each string specifying a particular node and an optional limit on the number of processes to be used for collective buffering on this node.

This refers to the same processes that cb_nodes refers to, but specifies the available nodes more precisely.

The format of the value of cb_config_list is given by the following BNF:

cb_config_list => hostspec [ ',' cb_config_list ]
hostspec => hostname [ ':' maxprocesses ]
hostname => <alphanumeric string>
         |  '*' 
maxprocesses => <digits>
         |  '*'

The value hostname identifies a processor. This name must match the name returned by MPI_Get_processor_name ²for the specified hardware. The value * as a hostname matches all processors. The value of maxprocesses may be any nonnegative integer (zero is allowed).

The value maxprocesses specifies the maximum number of processes that may be used for collective buffering on the specified host. If no value is specified, the value one is assumed. If * is specified for the number of processes, then all MPI processes with this same hostname will be used..

Leftmost components of the info value take precedence.

Note: Matching of processor names to cb_config_list entries is performed with string matching functions and is independent of the listing of machines that the user provides to mpirun/mpiexec. In other words, listing the same machine multiple times in the list of hosts to run on will not cause a *:1 to assign the same host four aggregators, because the matching code will see that the processor name is the same for all four and will assign exactly one aggregator to the processor.

The value of this info key must be the same for all processes (i.e., the call is collective and each process must receive the same hint value for these collective buffering hints). Further, in the ROMIO implementation the hint is only recognized at MPI_File_open time.

The set of hints used with a file is available through the routine MPI_File_get_info, as documented in the MPI standard. As an additional feature in the ROMIO implementation, wildcards will be expanded to indicate the precise configuration used with the file, with the hostnames in the rank order used for the collective buffering algorithm (this is not implemented at this time).

Here are some examples of how this hint might be used:

*:1 One process per hostname (i.e., one process per node)
box12:30,*:0 Thirty processes on one machine, namely box12, and none anywhere else.
n01,n11,n21,n31,n41 One process on each of these specific nodes only.

When the values specified by cb_config_list conflict with other hints (e.g., the number of collective buffering nodes specified by cb_nodes), the implementation is encouraged to take the minimum of the two values. In other words, if cb_config_list specifies ten processors on which I/O should be performed, but cb_nodes specifies a smaller number, then an implementation is encouraged to use only cb_nodes total aggregators. If cb_config_list specifies fewer processes than cb_nodes, no more than the number in cb_config_list should be used.

The implementation is also encouraged to assign processes in the order that they are listed in cb_config_list.

The following hint controls the deferred open feature of romio and are also applicable to all file system types:

romio_no_indep_rw - If the application plans on performing only collecitve operations and this hint is set to ``true'', then ROMIO can have just the aggregators open a file. The cb_config_list and cb_nodes hints can be given to further control which nodes are aggregators.

For PVFS2:

striping_factor - Controls the number of I/O devices to stripe across. The default is file system dependent, but for PVFS it is -1, indicating that the file should be striped across all I/O devices.
striping_unit - Controls the striping unit (in bytes). For PVFS the default will be the PVFS file system default strip size.
start_iodevice - Determines what I/O device data will first be written to. This is a number in the range of 0 ... striping_factor - 1.

Subsections

Rob Latham 2016-08-01