CVS log for pvfs2/src/io/job/job-time-mgr.c
	Help

Request diff between arbitrary revisions / Display revisions graphically

merge with Orange-Branch

Committed Windows client code to Orange-Branch.

Porting flow etc.

Reverse merged and ported to HEAD.

Update migration branch to current CVS version

reverse merge of HEAD to WALT3 branch.

backmerging of HEAD to branch...

Merge HEAD into Walt's branch.
Rework new state machines to the new cleanups introduced by Walt.

add function names to error messages to help track things down

Reverse merges from HEAD..

reverse merge from trunk.  working for now.

[phil]: alternative lio_listio implementation that spawns a thread and does pwrite/pread.

[phil]: bmi-socket-close: This fixes a bug in the new BMI_set_info(... BMI_TCP_CLOSE_SOCKET ...) mechanism, which is used to reconnect the socket to the initial configuration server if new socket buffer sizes are specified in the config file.  I didn't follow the code path find the exact problem, but at a high level it wasn't being thorough enough in cleaning out the old socket.  This showed up when using epoll and specifying socket buffer sizes in the server configuration- in this case the client will often fail to mount with a cryptic "not a directory" error and leave some epoll() errors in the pvfs2-client.log file.  I think a stale (or possibly reused) file descriptor was being left in the epoll fd set.  At any rate, the fix is to use a different set of functions for tearing down the entire address etc. so that it is reconnected on the next BMI addr lookup.  This path is already used by the server to discard old BMI addresses after critical errors on addresses that cannot be reconnected.  It is triggered from bmi.c without entering the bmi_tcp module, so this patch also adds a check to make sure we don't bother for non-tcp methods.

[phil]: bmi-test-overflow: One of the bmi bandwidth test programs was using types that might overflow if testing large enough transfers.  The fix is to convert to doubles and drop in several type casts to be cautious when performing the computation that was causing trouble.

[phil]: cancel-bugs: The biggest fix here is a change to the job timer code.  It was performing some pointer operations in the wrong order, which could lead to job timers failing to trigger in some cases. This would prevent some operations from ever timing out.  A secondary fix is a minor cleanup in BMI to catch potential race conditions in cancellation where a lock wasn't being held while checking to see if the target operation is complete.

[phil]: flow-post-error: This patch adds checks in the client side I/O state machine to test for failure at post time for flow operations.  This type of error is uncommon unless the flow parameters are faulty, but it should have checked anyway to be safe.

[phil]: dfile-config-check: This is a safety test.  The problem here is that there was no bounds checking for the DefaultNumDFiles option in the config file.  This made it possible to select -1 (which in PVFS1 meant "use the default number").  In PVFS2 this number gets passed verbatim to the client and would cause malloc failures and various other odd results when used. The patch just checks at parse time to make sure the value isn't negative.

[phil]: bmi-flow-logging: This patch just adds a little bit more gossip logging output to BMI and the flow protocol.  In particular, it adds tag information to messages about BMI operations, and adds pointer information to the flow protocol to help match posts and completions of flows in log files.

[phil]: server-freopen: This patch just removes duplicate calls to fdreopen(stdin/stdout/stderr) that were being performed on server startup when the server is run in the background.  The first set should be sufficient.

revert style changes back to previous versions.

run maint/pvfs2indent-80col.sh on all .c files to correct style :)

[pcarns]: improve flow debugging: more messages if flow cancled, log how many
flow and dbpf operations in progress, make it easier to match up the different
flows.

Attached patches implements trusted ports and networks similar to
what PVFS1 implemented.
It adds a new <Security> </Security> tag to the fs.conf config file
and parse utilities accordingly. We can edit these later if there
is a strong need.

Within these tags, you have to specify TrustedPorts <Range of ports> and
TrustedNetwork tcp://<network> tcp://<mask>.
I dont know how to support this cleanly on  non tcp/ip bmi
architectures, so currently this only enables it for the bmi_tcp module
alone. (i guess we could do it for bmi_ib module as well but not done
until someone says that is the right thing to do...)

Also adds support for a --trusted argument to the pvfs2-genconfig script
that will prompt for the trusted port ranges and the network address/masks and write it out
to the fs.conf and server.conf file.

Configure now supports a --enable-trusted to enable all these features.
(NOTE that the fs.conf tags (or lack of one) still allow you to not
select these features even if configure was asked to compile with this
option).

this patch also cleans up all the warnings that arises with gcc
(3.4) and --enable-strict build mode.

updates to my event changes to bring them inline with trunk

show true function name

- added fsync upcall operation on kernel's file fsync calls
- modified wait loop in the bufmap code to avoid potentially infinite
  hangs
- modified some kernel error messages
- added a file_fsync test program that just writes a file and calls fsync
- misc tab to space conversions

phil spotted a case w/o a 'break'.

- applied Nathan's expandtab patch (expandtab-2.patch.gz)

  first referenced at:
  http://www.beowulf-underground.org/pipermail/pvfs2-developers/2004-July/000745.html

removed superfluous debug message (could get triggered on a long running
flow before it has met the criteria for being cancelled)

added new debugging mask ("cancel" string, GOSSIP_CANCEL_DEBUG value) that
shows a little bit more descriptive information when timeouts occur

change job_time_mgr_rem() function to return a void

- add some error handling; null out ptrs that are no longer in use
  (fixes several crash bugs found in the job expiration case); add
  some sanity checks, etc
- make server continue processing, rather than abort on
  BMI_testcontext errors (these are recoverable)

- generalized some remove path error handling (using a macro)
- set msgarray to null after freeing in the remove path to avoid a
  possible incorrect free
- added some assertions
- fixed an error return code path (from an assertion)
- added a vfs test program that does a lot of I/O on a single file

tested and enabled server side timeouts for flow operations.  For now the
server defaults to this behavior: if a flow goes more than 30 seconds
without moving any data at all, it will be cancelled.  Still no timeouts on
the client side yet; coming soon...

implemented modification of timeout system for flows: rather than use an
absolute timeout for completion of flows, instead check every "timeout_sec"
seconds to see if progress has been made on it.  Will only cancel if
timeout_sec seconds passes with no data being transfered.  Needs testing
before being turned on

bug fix; job timeouts on server side seem to work now

tie the time management functions into the bmi job calls, still not doing
anything to trigger the actual timeout check, though

thread safety for job timeout mgr

first cut at a little api for managing timeout values for posted jobs, needs
thread safety and testing before it can be hooked to anything