Executive Summary
Thirty four people attended the first HPFF94 meeting (see list at end). Activity at the meeting is summarized here very briefly. The full minutes can be consulted for details.Formal voting requirements were discussed. An HPFF MOSAIC home page was proposed (and implemented). See
As a wrap-up, the following responsibilities have been assumed, along with email reflectors for communications. Full committee communications continue on the hpff-core@cs.rice.edu list and general meeting announcements, minutes, documentation releases are posted to hpff@cs.rice.edu.
April 6 Working subgroup sessions were held to discuss requirements and initial review of the requests for document correction/clarification.
April 7 Ken Kennedy formally initiated the second round of High Performance Fortran Forum meetings and led the discussion of initial administrative issues.
There have been two voting mechanisms - straw votes and formal votes. We had continuity in HPFF I meetings and hope that continues. Formal votes are one per organization. Anyone can vote at this meeting. New organizations at the second meeting cannot vote - must have attended the previous meetings. We propose to modify the first/second reading voting procedure for these meetings: some issues will be identified as minor and will be adopted in one meeting. If anyone questions an issue, it will be moved to more formal consideration.The program chair of ICPP has proposed that we coordinate our August meeting since the dates coincide. ICPP has already determined their meeting site and there was some objection to moving the HPFF meeting to that hotel. No overlap was identified in people present at the HPFF meeting who were planning to attend ICPP. The resolution was that if the ICPP organizers wish to propose a joint activity, we will consider it.
An HPFF Mosaic homepage was discussed. Mary was looking for volunteers to complete a language overview, as well as a volunteer for setting up the home page document. For the overview, most of the current document section editors agreed to help. Donna Reese, volunteered to take on the maintenance of the HPFF home page. There was a brief discussion of the possibility of getting the full document on-line. [NB as of the time the minutes are being prepared, the initial home page is available: http://www.erc.msstate.edu/hpff/home.html]
Documents for this round of HPFF meetings were discussed. David Loveman will continue as the main language document editor. Various possibilities for additional documents were discussed, such as a CCI document, implementation experience, limitations (whey we decided not to do something), requirements. The interpretations list might be formalized so it can be on-line. A requirements document would be a form of charge for HPFF II.
Funding for test and validation suites were discussed briefly. Ken contacted NIST about funding, but no program there has this kind of a charter.
Mary outlined the proposal that was submitted to SC94 for a (workshop) roundtable. We propose about 1 hour of report from the HPFF94 meetings - probably in the form of reports from the working group leaders - followed by about 2 hours of implementation and user experience. David Loveman volunteered additional help for the workshop. There will be more discussion of this at a later meeting.
Ken commented on federal research budgeting. Last year HPCC took a beating in the congress. HPCC funding has promoted a lot of good things such as HPFF, purchase of products, and good research. NSF got hit hard for a variety of reasons and steps have been taken. ARPA got hit by misinformation that was passed on to committee staff people (e.g. House Arms Services committee). There may be similar problems again this year. The people who can have the most impact are the HPC companies who should contact their congressmen. Lobbyists are trying to take the HPCC $$$ away.
The mailing reflectors were reviewed. The hpff-comments reflector will be aliased to hpff-interpret. The older working group reflectors will be disabled and replaced by new ones, as the need is identified.
It was verified that we will call this round of HPFF meetings HPFF94, to eliminated the confusion about correspondence between the version of the language and the series of the meetings.
The vendor/implementation survey was updated with the following results:
ANNOUNCED PRODUCTS
ANNOUNCED EFFORTS
INTERESTED
This concluded the initial round of administrative discussion.
Three document handouts for the initial interpret list were considered: the 3/28 list circulated via email (and handout), an addendum from Hank Sips and an second addendum from David Loveman. A numbering correction to the index for items #3.10, 3.11, 3.12 was noted.
In the CCI subgroup, recommendations for the following minor corrections, were made.
These recommendations for minor corrections were accepted by the committee with a vote of 25-0-2.
Two additional items were recommended as "first reading" issues:
The remaining CCI items were broken into two groups with a recommendation that the full group break into two subgroups later in the meeting to consider it issues.
Meeting break.
Following the break, there were reports from three requirements groups. Each group met on Wednesday afternoon to discuss the process and select someone to lead further work.
Irregular computations requirements
Reported by Chuck Koelbel
Methodology: HPF is widely believed to be inadequate for irregular computations (irregular means complex data structures). We want to base new extensions (if any) on user needs. WeUre heavy on compiler experience, light on applications people. So our general approach will be to survey applications we do have, survey established implementations and abstract feasible language and / or library features.
Target Applications: The important areas currently identified are - rapid n-body solvers e.g. multipole (octree), multigrid and adaptive grids, unstructured meshes, (irregularly) coupled regular meshes, mixed problems with substantial regular components, and monte carlo. Collecting a suite of example programs would be an important contribution. (These might also be useful for applications people to have models.)
Application Needs: The following were identified - more general data mapping, nested data structures, arrays of arrays, derived types, mapping to processor subsets, several mechanisms (map arrays, irregular blocks, etc.).
Better control of parallelism: In addition to mapping control, the following kinds of operations can contribute to support of irregular computations - nested and loosely coupled data structures, computational partitioning for loop iterations and array operations, schedule of loop iterations such as ON clause or self-scheduling, precomputing communications.
Other Issues
Action identified: Chuck will circulate a draft outline.
In general discussion following the report, Ken asked what expect to produce after these meetings. Chuck suggested each subgroup produce a report documenting what the issues are and some high-level descriptions of what possible approaches are - e.g. survey of literature, and maybe some recommendations.
Important to focus on requirements of the users and not the requirements for solutions. It was noted that HPF includes F77 so in some sense, it already solves the problem, but also commented that we want to advances the state of the art. Existing program kernels can be quite useful, but there is also a danger in concentrating on existing programs instead of maybe new approaches.
Ken asked about any consensus on some simple things. Some of the suggestions were: the proposal by Chapman at the January meeting that mapping directives by allowed inside of derived data types; processor subsets; reduced restrictions on what is allowed in FORALL.
Ken proposed that the irregular group prepare a formal proposal for minor adjustments to HPF that would enable some solutions. We would like to have such a proposal at the next meeting for straw votes. Terry asked if vendors are adding things? Do they see some easy things? Levesque volunteers to tell about the APR extensions that they are considering.
Guy Steele described the extensions for 64 bit integers as the most difficult thing they experienced at Thinking Machines. Fortran provides most needs, depending on the file location. If one Fortran program writes and another program reads, there may be a mismatch between the number of processors, and the number of disks. Then there are some implications requiring some unscrambling but Fortran I/O spells this out. Part of the reason this works is that the disks are not node attached, they are all are equally distant from the processing nodes.
At this point, the report was interrupted with some informal discussion that went approximately as outlined here. Rob Schreiber asked if implementations can just add keywords. Alok: - adding file layout directives doesn't harm portability because they can be ignored. Guy: Thinking Machines has another set of primitives that are accessed from message passing where all processors are dealing with the same file, but in an asynchronous fashion - also routines to control what happens when all processors are trying to write. These are low level things not visible to Fortran. Jerry Wagener: you need to write things out to be read back in other programs / different configurations. Is there a way to aid the system getting this right? Should we have something like a specification of the number of streams on the reads / writes? Alok: you should always maintain a specific global view of the data - and map in various applications.
Returning to the Parallel I/O report:
Some additional issues are: Is the language sufficient or some additions needed? Can a runtime library do the job? What is the model for parallel I/O? Are there special issues for I/O over high-performance networks?
Bob Knighten noted that users aren't interested in parallel I/O, they want high performance. Mary raised the question of the need for standardization of buffered I/O for overlap of computations with I/O. Bob: Posix has standards for this. Ken: it might be useful to adopt these as part of HPF.
Ken asked if there are simple changes that should be considered such as 64 bit integers, posix I/O, or an out-of-core directive. He invited a simple proposals.
Terry Pratt: overlap of I/O and computations may be a key issue. Guy: we are not necessarily promoting 64 integers. Chuck: F90 already has a KIND mechanism.
lunch break
Tasking group requirements
Reported by Joel Williamson
Support for multiple tasks requires creation, invocation, terminations, coordinations, communication, resource management and scoping. The question of determinism is a major concern. There is no consensus on mechanism. Sample implementations are - Fortran M, SVM, X3H5, Fx Fortran, U. Vienna extensions. There is no unanimous opinion that tasking is needed.
A paper that has been written by Zima, Chapman, Mehrotra , and Rosendale describes something that looks pretty natural.
There is possible overlap with irregular computation. Example applications are needed. A report is expected from a workshop on tasking that was held recently in Pasadena. There should be an ftp address with pointers to electronic copies of various projects/implementations that have proposed language extensions.
The X3H5 documents says that a user must be able to run codes on a single process. Maybe there should be specification of minimum number of tasks and resource? [Some discussion about the distinction between single process, single processor, and serial code.]
Chuck: what about non-determinism? Joel: if there are any place in the language where it can happen, users will fall into the trap. Mary: would like to see a focus on real application requirements. Robert Babb: Is a simple bridge between different tasks on different subsets of processors sufficient? Ian Foster: a virtual file mechanism helps this.
An email list will be established. Questions and suggestions are welcome.
Mary asked about nested WHERE - which subgroup does this belong to? Guy: the current semantics say that the ELSEWHERE can depend on WHERE so it is not really for parallelism, but that could change. This question is delegated to the irregular data requirements group. [Note X3J3 is also addressing this issue.]
This ended the requirements reports.
Ken conducted a straw poll about whether we should consider changing the HPF version. Straw Poll: 5-10-14.
CCI group 1 report - presented by Rob Schreiber for Rick Shapiro. This group worked on issues related to mapping, sequential, and inherit/processor arrangements. The subgroup recommendations were presented.
#D7 - does m have to be positive in block and cyclic directives? - Yes, The document should be modified to clarify.
#4.2 questioner made a mistake
#2.6 add the example to show
#2.7
REAL A(20);
!HPF$ Distribute A(Block(30))
Is this legal? Yes - add an example to the document.
#3.9 Part 1 What about overload of names such as REAL X(10), template x, processors x? This is illegal. There should be just one namespace. The document should be fixed to reflect this.
#3.9 part 2 - Subobjects of derived types are not mappable in HPF. The top-level of any type is mappable. A proposal is needed to clarify and address the mapping of components. We should allow one level of mapping, but not two levels.
David asked about the existing document phrase "non-terminal symbols that are not defined". This needs further action.
#3.13
ALIGN WITH B::A
REAL ALLOCATABLE
DIMENSION (:) :: A,B
#4.1 - Illegal in HPF I. (The example shows a two dimensional array mapped to a single dimensional processors arrangement.)
#3.7 - This item is illustrated with the example
subroutine murky(thick,dense)
!HPF$ distribute(block) :: dense
align with *dense :: thick
subroutine clear (lucid, bright)
!HPF$ align with *lucid :: bright
#3.5/3.6
real a(100,100)
call sub(a)
subroutine sub(b)
real b(100,100)
!HPF$ sequence b
call sub1(b)
end
subroutine sub1(c)
real c(10000)
!HPF$ sequence c
end
#3.10 - 3.12
INHERIT A
DISTRIBUTE A *(BLOCK) ONTO *P is in the subset
DISTRIBUTE * ONTO * is not
Vote to adopt the minor items above where resolution is clear: 27 - 0 - 0.
CCI group 2 report - presented by Mary Zosel. This group worked on the extrinsic/external interface, pointer, inquiries, and pure/forall issues. The subgroup recommendations were presented.
#2.8 GLOBAL_ALIGN/HPF_ALIGN "Furtney test". The answer is 2 in both cases. Both ask about the alignment of an array to its associated template, which is the same template in both cases. The two "quibbles" are dismissed with the comments that (1) is F90 conforming and (2) ok to call HPF_LIBRARY in an HPF_LOCAL.
These items were approved by a vote of 26-0-2.
The following items were either not done or action was postponed as noted.
This completed the reports from the CCI working groups. The next issue for discussion was the charter item about encouraging robust implementations, with discussion of implementation help, tests, etc.
Ken (speak-up or sing) Kennedy asked if we should have an "Advice to implementors document" Ira Baxter: ADA has such a document. There was more on this topic following a discussion of organizing test codes.
Applications subgroup (proposed) - reported by Joel Saltz
Joel reported on an effort being formed to gather a set of small practical kernels and mini-apps of the order 100-30000 lines of code in F77, F90. These would have several data sets, both small and realistic. There would optionally be HPF versions and parallel message passing versions, and possibly some versions from other languages such as Vienna Fortran. Two Gigabytes of a storage repository will be allocated for this purpose. Several codes have already been identified, along with a volunteer to take responsibility for the code:
In the next phase, more codes will be collected:
The purpose of these codes will be to test the limits of HPF and give direction for requirements.
For HPF validation suites, Ken repeated a statement made in January, that funding needs to be identified, and a commercial company given the assignment to produce a validation. This isn't volunteer labor or a student project.
A tutorial was proposed for how to do HPF for irregular problems and where it breaks down.
So far the collecting mini-apps is addressing HPF related to irregular data - but also need general HPF versions.
Chuck volunteered to do another regular code. Alok volunteered an unnamed code and Jerry has a finite difference acoustic model.
Jerry described the CSEP (Computer Science Education Project) which is working on an online text book. He suggested that maybe some of those authors would produce versions. Jerry will provide a mosaic address for more information.
John Levesque also has a suite of benchmarks - that are available over ftp
Joel: sponsorship of codes is important. We hope that over time, that the application will be rewritten in different versions so people can understand the trade-offs between different ways of writing the codes. Chuck: We also need descriptions of what the codes are doing.
An email list hpff-bench will be created for discussion of gathering codes.
Morning break.
Rob Schreiber has volunteered to be the collector of any information related to implementation experience to form a new document. The format and contents of such a document has yet to be determined.
Ken asked if vendors would like to give any implementations experience reports.
John Levesque (APR): We are ignoring restrictions in some case- e.g. on distributing equivalenced variables, or on entries to the routines. We allow for passing work arrays down a call chain and distributing subsections.
David Loveman (Digital): Implementation is very hard. We are targeting an ambitions compiler - full F90 and most of HPF (all except dynamic mappings and few other things). We also have a lot of Digital extensions. We are concentrating on doing well on loops and not paying a lot of attention to COMMON/EQUIVALENCE. We find that the F90 array syntax is just syntactic sugar for FORALL. If you do FORALL right then the array syntax falls out.
Andy Meltzer (CRI) presented a slide about feathers in CRAFT that users have commented most about. They are removing the power of two restriction. Almost everything feature is useful, but most important is for things to run fast. The users prefer high performance over new features. There have been lots of requests for eccentric distributions, even things that will never be available in LPF {actually can't do them in LPF - because LPF doesn't want users!!} [Readers new to HPFF minutes should look to the end of the report and note that LPF is Andy's private project - in no way reflecting on CRI products.]
1. Send any changes to the implementation survey to Ken or Mary.
2. Donna Reese will be responsible for the HPFF Mosaic homepage. Anyone with related research projects is invited to send her http pointers - also any commercial products can be sited from the implementation survey page. Send pointers. The document editors should send text for the language overview.
3. David Loveman will continue as document editor and the current section editors will be responsible for their parts. Drafts will be made available to the committee on FTP in a new directory. (We don't want to confuse people looking for the HPF document with draft versions.) Change bars will be investigated for the macros.
4. For continued CCI work, Rick Shapiro and Mary Zosel will work on issues identified. Each issue should be resolved, document changes verified, and a CCI report should be generated. It is suggested that a formal format be established with the form
5. For Requirements, the subgroup chairs are Chuck Koelbel (irregular data), Alok Choudhary (parallel I/O), and Joel WIlliamson (tasking). The charter for these groups is to collect application requirements, look for simple fixes to propose to the committee, and to create a requirements report. Chuck will assemble this report.
New reflectors will be established: hpff-irreg, hpff-io, hpff-task - all at cs.rice.edu.
6. For the "Encourage Implementations" work, Rob Schreiber will act as a group leader. A goal is an implementor advice document. In addition Joel Saltz will lead a group to collect mini-apps. The reflector addresses for these two grotwo groups are hpff-impl and hpff-bench, both at cs.rice.edu.
The following people attended the April HPFF meeting.
Robert Babb University of Denver babb@cs.du.edu Scott Baden UCSD baden@cs.ucsd.edu Ira Baxter Schlumberger Austin Rsch baxter@austin.sar.slb.com Alok Choudhary Syracuse U. choudhar@cat.syr.edu Jim Cowie Cooperating Systems cowie-james@cs.yale.edu Ian Foster Argonne Lab. foster@mcs.anl.gov Don Heller Ames Laboratory heller@cs.rice.edu Andrew Ingalls ACSET (Belgium) ingalls@acset.be Satoshi Itoh Hitachi CRL sitoh@crl.hitachi.co.jp Andrew Johnson OSF Research Institute andyj@osf.org Ken Kennedy Rice U./CRPC ken@rice.edu Bob Knighten Intel SSD knighten@ssd.intel.com Chuck Koelbel Rice U. chk@cs.rice.edu John Levesque APR levesque@apri.com David Loveman Digital lovemann@hpcgrp.enet.dec.com Larry Meadows PGI lfm@pgroup.com Andy Meltzer Cray Research meltzer@cray.com Nicole Nemer-Preece U. Missouri-Rolla nnemer@cs.umr.edu Bruce Olsen Hewlett Packard bruce@apollo.hp.com Terry Pratt CESDIS/NASA Goddard pratt@ceddis1.gsfc.nasa.gov J. Ramanujam Lousiana State U. jxn@max.ee.lsu.edu Donna Reese Mississippi State U. dreese@erc.msstate.edu Shoichi Sakon NEC sakon@lang1.bs1.fc.nec.co.jp Joel Saltz U. of Maryland saltz@cs.umd.edu Rob Schreiber RIACS schreiber@riacs.edu Richard Shapiro Thinking Machines shapiro@thinkcom Guy Steele Thinking Machines gls@think.com Jon Steidel Cray Research jls@cray.com Jeff Vanderlip Pacific-Sierra Research jeff@psrv.com Jerry Wagener Amoco jwagener@amoco.com Joel Williamson Convex Computer joelw@convex.com Hans Zima U. of Vienna zima@par.univie.ac.at Henry Zongaro IBM Canada zongaro@vnet.ibm.com Mary Zosel LLNL zosel@llnl.gov