This section describes two new distribution formats. The syntax is extended as follows:

H810 extended-dist-formatisBLOCK[ (int-expr) ]

orCYCLIC[ (int-expr) ]

orGEN_BLOCK(int-array)

orINDIRECT(int-array)

or*

- An
*int-array*appearing in a*extended-dist-format*of a`DISTRIBUTE`directive or`REDISTRIBUTE`directive must be an integer array of rank 1. - An
*int-array*appearing in a*extended-dist-format*of a`DISTRIBUTE`directive must be a*restricted-expr*. - The size of any
*int-array*appearing with a`GEN_BLOCK`distribution must be equal to the extent of the corresponding dimension of the target processor arrangement. - The size of any
*int-array*appearing with an`INDIRECT`distribution must be equal to the extent of the corresponding dimension of the*distributee*to which the distribution is to be applied.

The ``generalized'' block distribution, `GEN_BLOCK`, allows
contiguous segments of an array, of possibly unequal sizes, to be
mapped onto processors. The sizes of the segments are specified by
values of a user-defined integer mapping array, one value per target
processor of the mapping. That is, the *ith* element of the
mapping array specifies the size of the block to be stored on the
*ith* processor of the target processor arrangement. Thus, the
values of the mapping arrays are restricted to be non-negative numbers
and their sum must be greater than or equal to the extent of the
corresponding dimension the array being distributed.

The mapping array has to be a restricted expression when used in the
`DISTRIBUTE` directive, but can be an array variable in a
`REDISTRIBUTE` directive. In the latter case, changing the
value of the map array after the directive has been executed will not
change the mapping of the distributed array.

Let *l* and *u* be the lower and upper bounds of the
dimension of the *distributee*, *MAP* be the mapping
array and let *BS(i):BE(i)* be the resultant elements mapped to
the *ith* processor in the corresponding dimension of the
target processor arrangements. Then,

BS(1) | = | l, |

BS(i) | = | min(BS(i) + MAP(i) - 1, u), |

BS(i) | = | BE(i - 1) + 1. |

Example:

PARAMETER (S = /2,25,20,0,8,65/) !HPF$ PROCESSORS P(6) REAL A(100), B(200), new(6) !HPF$ DISTRIBUTE A( GEN_BLOCK( S) ) ONTO P !HPF$ DYNAMIC B ... new = ... !HPF$ REDISTRIBUTE ( B( GEN_BLOCK(new) )

Given the above specification, array elements A(1:2) are mapped on
P(1), A(3:27) are mapped on P(2), A(28:47) are mapped on P(3), no
elements are mapped on P(4), A(48:55) are mapped on P(5), and
A(56:100) are mapped on P(6). The array *B* is distributed
based on the array *new* whose values are computed at runtime.

*Advice to implementors.*Accessing elements of an array distributed using the generalized block distribution may require accessing the values of the mapping array at runtime. However, since the size of such an array is equal to that of the processor arrangement, it can in most cases be replicated over all processors.For dynamic arrays, an independent copy of the mapping array will have to be maintained internally so that a change in the values of the mapping array does not affect the access of the distributed array. (

*End of advice to implementors.*)

There are many scientific applications in which the structure of the
underlying domain is such that it does not map directly onto Fortran
data structures. For example, in many CFD applications an unstructured
mesh (consisting of triangles in 2D or tetrahedra in 3D) is used to
represent the underlying domain. The nodes of such a mesh are
generally represented by a one-dimensional array while another is used
to represent their interconnections. Mapping such arrays using the
structured distribution mechanisms, `BLOCK` and
`CYCLIC`, results in mappings in which unrelated elements are
mapped onto the same processor. This in turn leads to massive amounts
of unnecessary communication. What is required is a mechanism to map
a related set of arbitrary array elements onto the same processor.
The `INDIRECT` distribution provides such a mechanism.

The `INDIRECT` distribution allows a many-to-one mapping of
elements of a dimension of a data array to a dimension of the target
processor arrangement. An integer array is used to specify the target
processor of each individual element of the array dimension being
distributed. That is, the *ith* element of the mapping array
provides the processor number onto which the *ith* array
element is to be mapped. Since the mapping array maps array elements
onto processor elements, the extent of the mapping array must match
the extent of the dimension of the array it is distributing. Also, the
values of the mapping array must lie between the lower and upper bound
of the target dimension of the processor arrangement.

The mapping array has to be a restricted expression when used in the
`DISTRIBUTE` directive, but can be an array variable in a
`REDISTRIBUTE` directive. In the latter case, changing the
value of the mapping array after the directive has been executed will
not change the mapping of the distributed array.

Example:

Here, the array!HPF$ PROCESSORS P(4) REAL A(100), B(50) INTEGER map1(100), map2(50) PARAMETER (map1 = /1,3,4,3,3,2,1,4, ..../) !HPF$ DYNAMIC B !HPF$ DISTRIBUTE A( INDIRECT(map1) ) ONTO P !HPF$ DISTRIBUTE map2(BLOCK) ONTO P map2 = ... !HPF$ DISTRIBUTE B( INDIRECT(map2) ) ONTO P ....

A(1)is mapped ontoP(1)

A(2)is mapped ontoP(3)

A(3)is mapped ontoP(4)

A(4)is mapped ontoP(3)

A(5)is mapped ontoP(3)

A(6)is mapped ontoP(2)

A(7)is mapped ontoP(1)

A(5)is mapped ontoP(4)

The array `B` is declared dynamic and is redistributed using the
mapping array `map2`.

*Advice to implementors.*In general, the`INDIRECT`distribution is going to be used in the`REDISTRIBUTE`directive with an array variable as the map array. Also, since the size of the mapping array must be the same as the array being distributed, it will itself be distributed most likely using the`BLOCK`distribution. This raises several issues. To correctly implement this distribution, the runtime system should maintain a (distributed) copy of the mapping array so that if the program modifies the mapping array, the distribution does not change. Using an array variable as a mapping array implies that the location of each element of the array will not be known until runtime. Thus, a communication maybe required to figure out the location of a specific array element. (*End of advice to implementors.*)