CVS log for pvfs2/src/io/job/thread-mgr.c
	Help

Request diff between arbitrary revisions / Display revisions graphically

merge with Orange-Branch

merge with Orange-Branch

move gen_mutex_unlock() calls outside of __PVFS2_JOB_THREADED__ #ifdef so the client code correctly releases locks

merge with Orange-Branch

Added bmi_thread_running_mutex to protect bmi_thread_running modifications between
threads.

merged in changes from summer at LANL

merging hints/events code into trunk.

Reverse merged and ported to HEAD.

change which thread calls end events for trove.

reverse merge of latest changes from HEAD to he branch.

hints and events.

merging job/dev bug fix from trunk

merging job/dev bug fix from trunk

Fixed a bug in handling unexpected device jobs.  With a big enough load of
concurrent operations (particularly with threaded pvfs2-client) could cause
client-core assertion, and would likely happen repeatedly when client-core
restarted.

Merge HEAD changes to TAS-branch.

walt3 reverse merge from head includes merges of pw's sm changes (no state declarations), cleanup of state machine code, and other general merging/fixes.

Update migration branch to current CVS version

Sync hint-branch to current CVS version

merge 2.6 branch changes to head

Synchronization with HEAD

fixes for no __PVFS2_JOB_THREADED__

Upgrade to current CVS version

merge jazz fix from trunk

fix for breakage on jazz.  running variable needs to be set _before_ thread gets started.  Thanks RobL!

merge latest changes in 2.6 to head.

fix bug in build of client-core-threaded.  Trove doesn't exist on the client so we shouldn't try to link to it.

reverse merge of HEAD to WALT3 branch.

backmerging of HEAD to branch...

avoid starvation of a thread trying to BMI_cancel by adding another global variable

Merge HEAD into Walt's branch.
Rework new state machines to the new cleanups introduced by Walt.

Reverse merges from HEAD..

Backmerge with HEAD

optimize away a couple bmi_mutex lock operations

revert style changes back to previous versions.

run maint/pvfs2indent-80col.sh on all .c files to correct style :)

Reverse merge from trunk to lockd-branch...

reverse merge of branch

reverse merge of trunk to branch

solve unused ret a different way, consolidate unlock exit points too

remove pointless initializations that cause warnings when #defines make the variable otherwise unused

fixes to small io stuff to get test-zero-fill working again.  Also committing updates from HEAD to make things easier for merge.

committing lld/llu fixes to 1.3.1 branch

moved the llu and lld macros to an internal header so that we don't have to include pvfs2-config.h in the installed pvfs2-types.h header.

* malloc.h is obsolete in freebsd, check for it
* No error.h in freebsd
* linux allows Lu,Lx, and Ld to be interchangeable with llu,llx, and
lld.  Other platforms (solaris, bsd) don't.  Replaced all the L*
with ll*.
* bsd doesn't have mntent.h.  use fstab.h and our own funcs.
* bsd doesn't know about ssize_t.  Use size_t instead.
* bsd doesn't have asm errnos.  Check for each of those
errno macros and if not defined define to -1
* check for db3/db.h db4/db.h header if db path is specified using
--with-db
* darwin has aio support builtin, doesn't need -lrt
* remove kernel stuff in pvfs2-types.h that sets WORDSIZE.  gcc
defines __LP64__ on archs with 64bit longs so we use that instead.
* darwin's open doesn't accept O_LARGEFILE.  off_t is an int64 by
default.  define O_LARGEFILE if it hasn't been.
* need to run ranlib on darwin after libpvfs2.a gets installed to regenerate
table of contents

[pcarns]: add a protocol version to the pvfs2 device communication.  ensures
clients and the kernel module come from the same source tree, but also has the
pleasing side effect of 8-byte aligning access to the header of the request.

don't include pthread.h extraneously.  don't compile pthread versions if
NULL_LCOKING selected

- continue processing trove and bmi jobs in the thread-mgr after a
  null callback has been detected

this fixes (or at least temporarily relieves) the bug reported at:
http://www.beowulf-underground.org/pipermail/pvfs2-developers/2004-December/001038.html

- applied Nathan's expandtab patch (expandtab-2.patch.gz)

  first referenced at:
  http://www.beowulf-underground.org/pipermail/pvfs2-developers/2004-July/000745.html

- add some error handling; null out ptrs that are no longer in use
  (fixes several crash bugs found in the job expiration case); add
  some sanity checks, etc
- make server continue processing, rather than abort on
  BMI_testcontext errors (these are recoverable)

- merging in the pvfs2-nm-nb-branch with the main tree
  see ChangeLog for details, or browse the cvs history of the branch
  for full details

- handle graceful thread termination (if job threaded) or error
  returns in the thread-mgr code
- on shutdown, the thread-mgr callbacks can apparently be null, so handle that
  gracefully as well
- fix kernel bugs that didn't release the bufmap indices in use on some
  error paths (eventually caused all of them to be permanently used)
- move header include poll.h into pvfs2-kernel.h with the rest

- check for poll errors on the device file
- terminate the device testing thread on poll error (as the device can no
  longer be read properly), or return an error in non-threaded mode
  [ this is only really useful for graceful shutdown of the threaded
    client library ]

NOTE THAT THIS IS A BRANCH COMMIT (tagged as nm-nb-branch).  Feel free
to ignore it completey as it's a snapshot of a work in progress and it
will crash your computer and reformat your hard drive.

This is almost the initial draft of the pvfs2-client-core that
operates in a non-blocking manner.  While it runs, there are still
issues that need to be resolved (that are keeping it out of the main
CVS trunk).  Many other changes were made along the way, so it's more
than just that.

- added compile time option for disabling thread-safety in the client
  library (enabled by default; --disable-thread-safety to disable)
- improved configure summary information emitted at configure time
- added missing non-blocking sysint declarations to sysint header
- re-wrote pvfs2-client-core to use sysint non-blocking operations
  where possible
- made sysint test and testsome() calls more useable from a user point
  of view
- merged dev unexp polling/handling with system interface
- added PINT_sys_dev_unexp call that allows posting unexpected device
  messages so that they can be returned from the sysint testsome
  method in addition to completed sysint operations
- many memory leaks fixed -- many more to go (started adding macros
  for freeing the server response objects)
- added a id_gen_fast_unregister macro that is a no op, to make the
  api more consistent with the id_gen_safe_* calls
- server-config-mgr: report mutex still in use if it is, but also make
  sure not to unlock an already unlocked mutex (valgrind complains)
- many assertions added
- many formatting changes
- many ptr assignments to NULL after freeing in the job interface;
  done while tracking down a problem
- modified the pint-dev device interface to make sure it can handle
  the pvfs2 device in a non-blocking manner
- pint-dev was using buffers larger than it needed across the device;
  fixed them to be the right size
- added a method to free the memory region mapped into the kernel
  through the device
- freed that mapped memory region on pvfs2-client-core shutdown
  (valgrind complained)
- modified device driver to work properly in non-blocking mode from
  userspace by implementing the character device poll method
- modified pint-dev test function to properly handle non-blocking
  responses from the device driver
- modified PINT_flow_reset to not allocate a new mutex unless the old
  one was destroyed
- modified all job uses of the id-generator to use the safe, rather
  than fast, methods (useful for several reasons including safe
  cancellation)
- modified cancellation methods to be able to handle ops that have
  already completed
- modified the job_dev_unexp method to have (and honor) the no
  immediate completion flag if passed (used in the pvfs2-client-core)
- modified the job completion callbacks to make sure to NOT add a
  completed job desc to the completion queue if it's already been
  added (by checking a flag, not scanning).  this is a safety and
  should only be used when a non-thread-safe client library is being
  used in a thread safe env, but we should handle it gracefully anyway
- replaced all kernel allocations of ops through the slab allocator
  handled op_cache to be replaced by a wrapper method (op_alloc(), as
  suggested by Murali) -- the other cache allocations will probably be
  replaced later.  this allows the removal of the extern op_cache
  declaration
- added macros for freeing some of the most heavily used server
  response messages (readdir, lookup, getattr) -- the others are
  coming later
- freed dirents coming out of sysint response object in the
  pvfs2-client

reverted earlier timeout tuning change; see previous pvfs2-{cvs/internal}
email for explanation

auto-tune test_timeout to poll frantically when something appears to be going on, else back off to default timeout

- take 2: deadlock removal, now without forced deadlock guarantees

- deadlock fixes on shutdown (always drop the interface mutex before joining)

- work around a deadlock scenario on exit

combo PVFS_id_gen_t -> PVFS_BMI_addr_t, formatting, PVFS error code patch.
Ugly.  Sorry if my formatting pisses someone off; at least I'm not using >
80 columns any more :).

fixed logic bug in thread mgr cancel path, added some lazy #if 0'd print
statements for the moment

loosened up timing in thread mgr cancellation, be more aggressive about
finding pending operations, other minor tweaks

thread-mgr cancel hook for trove operations

where to begin?  ok, we need to proxy bmi cancellation requests (and
eventually trove requests) through the thread-mgr, in case the thread-mgr
has already completed whatever it is we want to cancel but we haven't found
out yet.  thus the addition of thread_mgr_bmi_cancel().  This function is
complicated by races with the thread that is actually trying to complete
stuff, but we can't solve it with simple locks because NPTL will punish us
for holding locks for the duration of a testcontext call... so we use an odd
looking condition variable technique to protect the critical regions

- during dev_stop, drop the dev_mutex, as it's possible that the
  thread can't be joined if we hold this lock (this fixes a hang on
  server shutdown via signal)

shuffled support for device handling thread over into thread mgr; cleans up
core job code and makes sure that thread handling is consistent at job level

started using pthread_join to collect threads when they exit, helps
avoid race conditions.  need to do the same for device thread once it
is integrated into thread mgr...

cleaned up some superfluous locking in the thread manager

reworked the way that the threaded vs. nonthreaded compilation paths are
handled at the job level.  simplifies things a bit and reuses more code.
also paves the way for similar changes at the flow level.

minor bug fix to make thread shutdown safer

started adding trove support to external thread manager for job and flow
components; untested

updated bmi/trove flowprotocol to use external thread manager when
job threading is enabled- all BMI_testcontext() calls now happen
from the same thread in order to cut down on contention

integrated support for the external thread manager into the job interface,
just for BMI operations.  this is the first step towards allowing one
thread to drive BMI for both the job and flow interface.  No performance
benefit yet, but verifies that this stuff works.  Still need to do flow half.

a very rough first cut at a seperate thread management system for BMI that
runs callbacks on completion (so one thread can handle BMI work for
both job and flow subsystems)