CVS log for pvfs2/src/server/small-io.sm
	Help

Request diff between arbitrary revisions / Display revisions graphically

Merged robust security code into Orange-Branch.

merge with Orange-Branch

changing llu handle references to strings

added IO byte counters for small_io
added IO byte counters for ONLY small_io and ONLY flows
minor format edit
debug fixes to usrint code

Modifications to perf counter system
Minor formatting and documentation edits

merging Orange Branch changes in

initial merge with Orange-Branch. much will be broken

Merged failover with tree code.
Modified Files:
  Tag: Orange-Branch
 	prepare src/apps/kernel/linux/pvfs2-client-core.c
 	src/client/sysint/client-state-machine.c
 	src/client/sysint/client-state-machine.h
 	src/client/sysint/finalize.c
	src/client/sysint/sys-getattr.sm
 	src/client/sysint/sys-io.sm
	src/client/sysint/sys-small-io.sm
 	src/common/gossip/gossip.h
	src/common/misc/msgpairarray.sm
 	src/common/misc/pint-event.c
	src/common/misc/pint-event.h
 	src/io/bmi/bmi.c
	src/io/dev/pint-dev.c
	src/io/job/job.c
 	src/kernel/linux-2.6/devpvfs2-req.c
 	src/kernel/linux-2.6/file.c
	src/kernel/linux-2.6/pvfs2-mod.c
 	src/proto/PINT-le-bytefield.c
	src/proto/pvfs2-req-proto.h
 	src/server/create-immutable-copies.sm
	src/server/get-attr.sm
 	src/server/small-io.sm
	src/server/tree-communicate.sm
 	src/server/request-scheduler/request-scheduler.c
 	src/server/request-scheduler/request-scheduler.h

Completed coding and debugging of small-io failover.

merged in changes from summer at LANL

Merged Phil's tcp-peer-comm-branch that allows simultaneous server to server communications.
Added basic allreduce module:
  - implemented a recursive distance doubling algorithm.
  - works only when the # of servers are power of 2.
  - when activated, servers simply exchange their id to each other.

TODO
  - handle arbitrary sizes of buffers of various data types.

Added the server to server communication module within pipeline.sm
to pull the small portion of data when the data is not aligned on strip boundaries.

Changed the default file stripe size to 256KB (originally 64KB)
so that pipeline unit is equal to it.

merging hints/events code into trunk.

reverse merge of direct IO changes to he-branch.

merge of direct IO code to PVFS HEAD.

Reverse merged and ported to HEAD.

changes to trove layer to add direct I/O funcionality (O_DIRECT).

state machine permission checking code cleanup

Updated security checking in state machines.  Added flags where needed.

modified permission checking to use callbacks instead of flags. added custom
checks to the io and small-io state machines.

hints and events.

syncing small-file-branch back up with trunk at small-file-branch-point2 tag
(reverse merge)

Big pile-o' changes to make Pete go mad.

The main changes are the implementation of the stuffed create path (now just the create state machine and operation), and the beginnings of the "unstuff" operation.

This also includes the request scheduler and state machine cleanups (required for using prelude with batch_remove) and whatever else I may have happened to change along the way.

This code is able to create files, but writing to them is almost guaranteed to break.

readonly symbolic

Change access_type functions from 0/1 to use the enum
PINT_server_req_access_type.  There may have been some backward
conditionals in here.  This at least makes them all visible,
and fixes a problem with ReadOnly exports where only reads
(but not writes) were forbidden.

Sam's prelude and scheduler updates

The normal path for a server request operation is to jump to the prelude state machine, which tries schedule the request based on the handle+fsid.  This meant that the request scheduler code had a switch/case returning the handle+fsid for each request we defined.  The function was also overloaded with a readonly parameter (whether the request modified any objects on the server), and a scheduling policy parameter (as the return code), which specified whether the request scheduling could be bypassed for this request.

I moved all that to a per-request parameter structure, that gets specified within the server state machine for the request.  Now there is one array of server requests (moved to pvfs2-server-req.c) which the request scheduler and prelude state machine can poke at to get parameters (defined in the state machine for that request), and based decisions on.

The patch includes some other minor changes:

* A fix to the BMI addr ref count increment/decrement.  Previously, the addr ref count was being incremented/decremented in the prelude/final-response state machines.  That requires that those state machines get invoked, which I don't want to have to depend on.  I've moved the INC call to the unexpected_sm, and the DEC call to where the state machine gets finished in the server.  That way we don't have to worry about whether we're using prelude/final-response.

* Cleanup of the request scheduler.  The way we were changing the mode from normal to admin was sort of embedded into the request scheduling code, so we had to hack the mode change from prelude_sm instead of in setparam.  Also, the request scheduler had to be aware of the setparam fields in the request.  I've moved the mode change code to separate request scheduler and job calls.  I think something similar could be done for the timer code (it doesn't really belong in the request scheduler), but I'll leave that for another patch.

* Changed the way we compile the server binary in Makefile.in to allow for multiple object files.

minor change allows state actions to not be declared at the top of *.sm files.

Merge HEAD changes to TAS-branch.

merge of the WALT3 branch to HEAD.  This patch changes the way state actions
are represented as C structures (what statecomp generates).  It also changes
the main state action parameter from a s_op on the server or an sm_p on the client
to a unified smcb pointer (state machine control block) for all state actions (both client and server).  Finally, initial support for concurrent state machines has
been added to allow state actions to be invoked concurrently .. a first step for
server-to-server.

walt3 reverse merge from head includes merges of pw's sm changes (no state declarations), cleanup of state machine code, and other general merging/fixes.

update to current CVS version because statecomp modifications are neat for my work, thanks!

Remove forward declarations of states.

hint is given down to Trove

Merge HEAD into Walt's branch.
Rework new state machines to the new cleanups introduced by Walt.

Modification for migration added, some functions still need to be adapted after moving to a new branch...

Added request statistics to scheduler, reference from datafiles to
parent metafile (needed for migration) and auto migration tool(unfinished).

Reverse merge changes from HEAD to posix-extensions-branch.

minor update

Simple pint/ls/cp/diff test works

Everything compiles.   Crap!  Now I have to see if it runs!!!

Julian's changes to coalescing and dbpf storage options:

- Changes the coalesce code, now we decide the mode (coalesce or not) during
the coalesce function call and not during the enque process.
- Add the coalesce high / low watermark to the dbpf_collection structure, so
colls can have different values.
- Put per default the TROVE_SYNC flag into trove from different statemachines
where we used the macro before. The macro is still there and should be
removed if we agree to this behavior.
- Removes the coalesce config option
- Moves SMALL_IO_MAX_REGIONS to internal and renames it to IO_MAX_REGIONS (we
had it at least 3 times defined somewhere).

correcting compile errors

updated to support SM control block interface

Reverse merge from trunk to lockd-branch...

reverse merge of branch

reverse merge of trunk to branch

remove unused state declaration

small io fix.  Thanks Avery!

added versioning test.  There's some nasty hacks with set/getenv to be able to test it properly without modifying sysint signatures.  Those will have to be removed at some point.

*** empty log message ***

merge of phil's fixes for small-io and job error codes to HEAD.

reverse merge from trunk.  see if sio tests pass now as well.

if aio threaded isn't supported (or the configure check for libc version fails for some other reason), the non-threaded aio impl doesn't set the result size in the case of reads.  So if the result size doesn't equal the expected bytes found from process_request, we just set the size to the expected bytes for now.

fixed error handling in job api and fixed file descriptor leaks in dbpf open cache.  Both fixes are from Phil.  Thanks Phil!

merging changes from HEAD to dbpf branch

fix for reading a zero byte file.  Also adding fixes from http://www.beowulf-underground.org/pipermail/pvfs2-developers/2006-January001741.html and
http://www.beowulf-underground.org/pipermail/pvfs2-developers/2005-December/001737.html

remove unused variable

woops

This fixes a bug where invalid sizes were being written on servers that were past the first strip.  It was created from the previous fixes to send the total_bytes to the server for the small_io operation (which are in still needed for writes).  The PINT_REQUEST_STATE_SET_FINAL macro actually needs the entire size of the request, not just the bytes sent to this server.

fix for the read request size being sent in small_io operation.  The displayed error this fixes is:  pint_request.c line 89: PINT_process_request: NULL segmax or bytemax!

fixes to small io request to calculate the size of the actual buffer being sent instead of the size of the memory request.

here we go.  avoid a double free.

fix another memory leak.  should be the last in the copy case

last of the 'pvfs2-cp with small blocksize' memleaks

merging small I/O changes to trunk.  The majority of this commit includes the
new small I/O state machines on client and server, and the modifications to
the client IO state machine.  It also includes mods to the zero-fill code
to perform zero-fill of holes only if they exist instead of zero-ing the entire
memory region initially.  Also, some debug stuff has been added to the
state machine processing code to print enter/exit of state machines if sm
debug is enabled.  The genconfig now includes ioports/metaports for running
multiple servers on the same host for testing.

zero-fill appears to work now (finally).

added access check of keyval to improve getattr performance.  updates to small-io to not depend on bstream size for writes.  cleanups to client io analyze results code.

fixes to small io stuff to get test-zero-fill working again.  Also committing updates from HEAD to make things easier for merge.

small I/O changes in sio branch for now

file small-io.sm was initially added on branch slang-sio-branch.