CVS log for pvfs2/src/io/flow/flow.h
	Help

Request diff between arbitrary revisions / Display revisions graphically

Merged in changes from Orange-Branch. There were a number of bugs fixed there since this branch was created.

merging Orange Branch changes in

initial merge with Orange-Branch. much will be broken

merge of B2O-Blue-Sync,HEAD,and latest of Orange.

commiting changes from merge of Orange and Blue into test branch.

merge flow deadlock bugfix from trunk to 2-8 branch

bug fix for flow protocol deadlock reported by Bart Taylor; make sure that
we don't deadlock on the completion mutex if a flow cancels immediately and
triggers a callback from within the cancel() function itself.

merged in changes from summer at LANL

*** empty log message ***

merging hints/events code into trunk.

Reverse merged and ported to HEAD.

hints and events.

Get rid of gen_mutex_build.

added hint to flow layer

Merge HEAD into Walt's branch.
Rework new state machines to the new cleanups introduced by Walt.

Reverse merges from HEAD..

reverse merge from trunk.  working for now.

[phil]: flow-proto-tuning: This patch adds "FlowBufferSizeBytes" and "FlowBuffersPerFlow" options to the configuration file format.  They allow you to specify the buffer size that the default flow protocol will use as well as the maximum number of buffers to use per flow.  Note that if you change either of these parameters, then you need to remount any active clients so that they pick up the configuration change before performing any I/O.

[phil]: max-aio: This patch adds "TroveMaxConcurrentIO" to the configuration file format.  It allows you to specify the maximum number of I/O operations that trove will allow to proceed concurrently (currently 16).  Note from the previous email regarding AIO that depending on your access pattern, AIO may queue all of your operations anyway regardless of this setting.  It probably doesn't have much effect unless you are accessing more than one file at a time, or if you are using an alternative to the stock AIO implementation.

merging small I/O changes to trunk.  The majority of this commit includes the
new small I/O state machines on client and server, and the modifications to
the client IO state machine.  It also includes mods to the zero-fill code
to perform zero-fill of holes only if they exist instead of zero-ing the entire
memory region initially.  Also, some debug stuff has been added to the
state machine processing code to print enter/exit of state machines if sm
debug is enabled.  The genconfig now includes ioports/metaports for running
multiple servers on the same host for testing.

fixed a bug in flow caused by outsize being updated correctly in trove now.  Also, s/transfered/transferred/.

Applying Avery's memleak patch for flow descriptors in sys-io.sm.  Modified slightly to fix invalid free for flow descriptors.  Also made internal function in client core static to remove warnings.

updates to my event changes to bring them inline with trunk

Updates for standardized spelling in pint-request.h and new distribution
function signatures.

- added configuration option TroveSyncMeta (can be 'yes' or 'no')
- added configuration option TroveSyncData (can be 'yes' or 'no')
- replaced TroveSyncMode configuration option with separate
  TroveSyncMeta and TroveSyncData options.  the defaults are yes to
  both.  this is a change because previously data was NOT being sync'd
  after reads/writes.  for now, only writes are sync'd if
  TroveSyncData is 'yes'.
- updated pvfs2-genconfig to reflect config option changes
- added top-level and flowproto-multiqueue understanding of the
  data sync mode option via setinfo
- added hook to server to pass data sync mode option to the flow
  implementation via setinfo on a per fs/coll_id basis
- added fp_multiqueue tracking of this to be able to pass the
  appropriate data sync flag to trove on a per fs basis
- misc cleanups

- applied Nathan's expandtab patch (expandtab-2.patch.gz)

  first referenced at:
  http://www.beowulf-underground.org/pipermail/pvfs2-developers/2004-July/000745.html

implemented modification of timeout system for flows: rather than use an
absolute timeout for completion of flows, instead check every "timeout_sec"
seconds to see if progress has been made on it.  Will only cancel if
timeout_sec seconds passes with no data being transfered.  Needs testing
before being turned on

Remove "CACHE_ENDPOINT". The buffer cache still uses TROVE_ENDPOINT.

Add cache endpoint (CACHE_ENDPOINT) to support the bmi-cache flow protocol.

- make flow_mutex a pointer to play nicely with gen_mutex_destroy

exposed a lock used in the flow protocol at a slightly higher level, need it
for some future changes

got rid of some old cruft in flow header

Move from bmi_addr_t to PVFS_BMI_addr_t for consistency.

minor update, changed name of PINT_flow_unpost() -> PINT_flow_cancel(),
still just a stub function

big structural change/simplification of the flow code.  job api remains
unchanged.  notification from flow level back to job level is now done
through callbacks rather than by testing.  works for both threaded and
non threaded code path.

removed some unused flow states

changed some types over to named enumerations

removed some unused and unimplemented flow functions

removed some old #if 0'd code

got rid of FLOWPROTO_ANY and replaced with FLOWPROTO_DEFAULT, no longer
bother searching flow protocols based on endpoint type

Put remaining code in place to handle memory datatypes, not ready yet
(pending a fix to some corner case in req processing).  Everything should
still work as it was prior to this commit.  When we do enable memory
datatypes, I'll need to go back and uncomment some code and adjust all
PVFS_sys_{read/write} calls to pass in the right datatype (they are just
taking NULL for now).  PVFS_sys_{read/write} API is in its final form
(I think...) for now, at least.  For now this code is dependent upon
the fact that the file datatype isn't implicitly tiled yet, need to get
memory datatypes going before relaxing that constraint...

changed reference to datatypes called "io_req..." to "file_req...", to make
the distinction clearer between file datatypes and memory datatypes once the
latter is added

modified flow descriptor so that file data structure is no longer
a pointer; avoids an extra malloc step in some cases

minor cleanup- got rid of an unused "flags" field, will add it back
later if needed

Aieeee....

New datatype request processing interface clumsily merged into tree.  For the
most part, things will work as before, but still need to:
- fix bug in precalculation when datatype offset is used
- fix data corruption when running multiple servers
- update flowproto-dump-offsets and enable again
- take out old #ifdef0'd code in bmi trove flowproto
- clean up flowproto's use of new interface

added file datatype offset to flow and flow descriptor; not being used yet

got rid of the PVFS_coll_id type- it was a duplicate of PVFS_fs_id.

got rid of the PVFS_bitfield type, and replaced with something more
generic everywhere it was used

renamed enum flowproto_type to enum PVFS_flowproto_type and moved to
pvfs2-types.h

added the ability to force selection of a specific flow protocol;
untested

implemented contexts

Added contexts to the flow API.  Not doing anything with it yet.

reformatted all of the flow code

finished pushing max_idle_time semantics into flow interface

big changes to various I/O API's- there are no longer any "wait"
functions in the BMI, flow, or request scheduling APIs.  In the
request scheduler case, they have simply been discarded.  In BMI
and flows, the testXXX() functions have been augmented with a
timeout argument so that they may perform the role of the
former waitXXX() functions.

integrated the flow subsystem into the tree; still need to get matching
test programs in