CVS log for pvfs2/src/server/io.sm
	Help

Request diff between arbitrary revisions / Display revisions graphically

Merged robust security code into Orange-Branch.

changing llu handle references to strings

changes to PVFS_handle_X

local layout bug fix during hostname comparison.

added some printf to test bmi/trove timing.

initial merge with Orange-Branch. much will be broken

revert cu-security-branch to before the attempted merge with Orange-Branch

added bmi and trove timing in I/O pipeline.

fixed a bug in case of simple actsto operations in pipeline.sm.
(it was returning local sum instead of global sum)

Fixed a bug in io.sm.

minor cleaning up.

added some timing measurement calls for bmi and trove.

Added reduction kernels in CUDA (from NWU) into source tree.
Revised test programs.

Added "grep" operator into pipeline.sm.
Updated the grep test program in the test/active_storage directory.

Revised simple.c (in test/active-storage directory) to make it parallel.

minor change in pipeline.sm

Merge of Orange-Branch, Orange-FailOver-Becky, and version 1.164 of sys-io.sm (do not unstuff on a read).
This version of Orange-FailOver-Becky supports failover for large files only.  The next iteration will
support small-io failover.

merged in changes from latest Orange-Branch

merged in changes from summer at LANL

Improved stripe alignment algorithm, especially for the 1st and last stripes.
Increased the BMI buffer size: the default was 16MB

Implemented kmeans algorithm based on the state machine, kmeans.sm.
There are two supporting state machines for this: allreduce.sm and bcast.sm.
The simple1_unaligned.c file is updated for testing kmeans.
Sample input file (edge17695.bin) for testing kmeans algorithm is added.

Removed debugging statements.

The buffer_size value in pipeline.sm is now from the fs_config,
and the default size is 1MB.
Added a skeleton for the kmeans operation, which is based on the nested state machine.

Split the previous allreduce.sm into send and recv nested state machines using pjmp.
Changed pipeline.sm to setup allreduce.sm in either SEND or RECV type.

Updated the buffer copy method from the nested state machine to parent.

Used PINT_sm_frame(smcb->parent_smcb, 0) to refer parent smcbs in pipeline.sm.
Removed possible double free() in io.sm.

update printf format for 64 bit values

Added more gossip messages.

fixed invalid free() in io.sm.

Revised the allreduce.sm and other related codes accordingly.

Added the server to server communication module within pipeline.sm
to pull the small portion of data when the data is not aligned on strip boundaries.

Changed the default file stripe size to 256KB (originally 64KB)
so that pipeline unit is equal to it.

Setting up Orange Branch

Applied Phil's patch that fixes size mismatches between js_p->actual_size and *out_size_p in the job_trove_bstream_{read,write}_list call, and revised pipeline.sm accordingly.

Merged read.sm and write.sm into a single file, pipeline.sm.

This file has been renamed to pipeline.h.

Augmented PINT_segpool_take_segments() to handle cases where
there are more segments to read but result.bytes is returned to zero
from the PINT_process_request call.
Added some debugging messages.

Cleaned the data structures related to flow_d.

Initial version of parallel read/write sms is implemented.

modified segpool_init() to set proper file_request_state
and some bugs in the io.sm file.
now it is working without errors.

*** empty log message ***

merging hints/events code into trunk.

reverse merge of direct IO changes to he-branch.

merge of direct IO code to PVFS HEAD.

Reverse merged and ported to HEAD.

changes to trove layer to add direct I/O funcionality (O_DIRECT).

more work on state machine permission checks

modified permission checking to use callbacks instead of flags. added custom
checks to the io and small-io state machines.

hints and events.

syncing small-file-branch back up with trunk at small-file-branch-point2 tag
(reverse merge)

Big pile-o' changes to make Pete go mad.

The main changes are the implementation of the stuffed create path (now just the create state machine and operation), and the beginnings of the "unstuff" operation.

This also includes the request scheduler and state machine cleanups (required for using prelude with batch_remove) and whatever else I may have happened to change along the way.

This code is able to create files, but writing to them is almost guaranteed to break.

readonly symbolic

Change access_type functions from 0/1 to use the enum
PINT_server_req_access_type.  There may have been some backward
conditionals in here.  This at least makes them all visible,
and fixes a problem with ReadOnly exports where only reads
(but not writes) were forbidden.

Sam's prelude and scheduler updates

thought I removed this already...

The normal path for a server request operation is to jump to the prelude state machine, which tries schedule the request based on the handle+fsid.  This meant that the request scheduler code had a switch/case returning the handle+fsid for each request we defined.  The function was also overloaded with a readonly parameter (whether the request modified any objects on the server), and a scheduling policy parameter (as the return code), which specified whether the request scheduling could be bypassed for this request.

I moved all that to a per-request parameter structure, that gets specified within the server state machine for the request.  Now there is one array of server requests (moved to pvfs2-server-req.c) which the request scheduler and prelude state machine can poke at to get parameters (defined in the state machine for that request), and based decisions on.

The patch includes some other minor changes:

* A fix to the BMI addr ref count increment/decrement.  Previously, the addr ref count was being incremented/decremented in the prelude/final-response state machines.  That requires that those state machines get invoked, which I don't want to have to depend on.  I've moved the INC call to the unexpected_sm, and the DEC call to where the state machine gets finished in the server.  That way we don't have to worry about whether we're using prelude/final-response.

* Cleanup of the request scheduler.  The way we were changing the mode from normal to admin was sort of embedded into the request scheduling code, so we had to hack the mode change from prelude_sm instead of in setparam.  Also, the request scheduler had to be aware of the setparam fields in the request.  I've moved the mode change code to separate request scheduler and job calls.  I think something similar could be done for the timer code (it doesn't really belong in the request scheduler), but I'll leave that for another patch.

* Changed the way we compile the server binary in Makefile.in to allow for multiple object files.

minor change allows state actions to not be declared at the top of *.sm files.

Merge HEAD changes to TAS-branch.

merge of the WALT3 branch to HEAD.  This patch changes the way state actions
are represented as C structures (what statecomp generates).  It also changes
the main state action parameter from a s_op on the server or an sm_p on the client
to a unified smcb pointer (state machine control block) for all state actions (both client and server).  Finally, initial support for concurrent state machines has
been added to allow state actions to be invoked concurrently .. a first step for
server-to-server.

walt3 reverse merge from head includes merges of pw's sm changes (no state declarations), cleanup of state machine code, and other general merging/fixes.

Cleanup of performance counter code...

move request scheduler load start/stop directly around state machines

Created a better interface for load values in the performance counter.

update to current CVS version because statecomp modifications are neat for my work, thanks!

Remove forward declarations of states.

Update migration branch to current CVS version

reverse merge of HEAD to WALT3 branch.

add hints to job layer

backmerging of HEAD to branch...

do not try to send a completion ack if the flow completed in error

Merge HEAD into Walt's branch.
Rework new state machines to the new cleanups introduced by Walt.

move frequently used extern declaration into a header file

Modification for migration added, some functions still need to be adapted after moving to a new branch...

PVFS2-hint support + minor changes (verbose request scheduler)

Added request statistics to scheduler, reference from datafiles to
parent metafile (needed for migration) and auto migration tool(unfinished).

Reverse merges from HEAD..

Add pvfs2 client core / kernel modul support for hints.

reverse merge from trunk.  working for now.

[phil]: flow-proto-tuning: This patch adds "FlowBufferSizeBytes" and "FlowBuffersPerFlow" options to the configuration file format.  They allow you to specify the buffer size that the default flow protocol will use as well as the maximum number of buffers to use per flow.  Note that if you change either of these parameters, then you need to remount any active clients so that they pick up the configuration change before performing any I/O.

[phil]: max-aio: This patch adds "TroveMaxConcurrentIO" to the configuration file format.  It allows you to specify the maximum number of I/O operations that trove will allow to proceed concurrently (currently 16).  Note from the previous email regarding AIO that depending on your access pattern, AIO may queue all of your operations anyway regardless of this setting.  It probably doesn't have much effect unless you are accessing more than one file at a time, or if you are using an alternative to the stock AIO implementation.

Simple pint/ls/cp/diff test works

Everything compiles.   Crap!  Now I have to see if it runs!!!

correcting compile errors

updated to support SM control block interface

Reverse merge from trunk to lockd-branch...

reverse merge of branch

merge sync coalescing branch.

fix deadlock bugs in coalescing code.  add proper state machine debugging.

merge from trunk

[pcarns]:  removed unecessary check in io_release() function and made sure that
job_req_sched_release() reports an error if the job descriptor cannot be found.

merging small I/O changes to trunk.  The majority of this commit includes the
new small I/O state machines on client and server, and the modifications to
the client IO state machine.  It also includes mods to the zero-fill code
to perform zero-fill of holes only if they exist instead of zero-ing the entire
memory region initially.  Also, some debug stuff has been added to the
state machine processing code to print enter/exit of state machines if sm
debug is enabled.  The genconfig now includes ioports/metaports for running
multiple servers on the same host for testing.

fixed a bug in flow caused by outsize being updated correctly in trove now.  Also, s/transfered/transferred/.

fixes to small io stuff to get test-zero-fill working again.  Also committing updates from HEAD to make things easier for merge.

committing lld/llu fixes to 1.3.1 branch

moved the llu and lld macros to an internal header so that we don't have to include pvfs2-config.h in the installed pvfs2-types.h header.

* malloc.h is obsolete in freebsd, check for it
* No error.h in freebsd
* linux allows Lu,Lx, and Ld to be interchangeable with llu,llx, and
lld.  Other platforms (solaris, bsd) don't.  Replaced all the L*
with ll*.
* bsd doesn't have mntent.h.  use fstab.h and our own funcs.
* bsd doesn't know about ssize_t.  Use size_t instead.
* bsd doesn't have asm errnos.  Check for each of those
errno macros and if not defined define to -1
* check for db3/db.h db4/db.h header if db path is specified using
--with-db
* darwin has aio support builtin, doesn't need -lrt
* remove kernel stuff in pvfs2-types.h that sets WORDSIZE.  gcc
defines __LP64__ on archs with 64bit longs so we use that instead.
* darwin's open doesn't accept O_LARGEFILE.  off_t is an int64 by
default.  define O_LARGEFILE if it hasn't been.
* need to run ranlib on darwin after libpvfs2.a gets installed to regenerate
table of contents

fixes after branch point shift

updates to my event changes to bring them inline with trunk

merge changes from trunk since branch point into branch.  Effectively moves the branchpoint to the latest trunk.

[pcarns]:
- adds a new prefix ("A") for access style logging
- logs user, group, host for each operation if enabled
- triggered by a macro in the server (PINT_ACCESS_DEBUG) that checks gossip
  level before doing any work
- basic format looks like this (H is handle, S is s_op structure, crdirent is
  operation type, Success is response status sent):

[A 08/02 15:14] pcarns.users@localhost H=1048576 S=0x80f7d20: crdirent:
finish (Success)

- turning on "access" logging gives messages like the above on completion of
  operations, plus messages for some operations that give key information (like
  the path name for lookups, the object type for getattr, etc.)
- turning on "access_detail" logging adds additional messages that print when
  the request is received and when it starts processing

[pcarns]: do a couple things here:
- implements the retry and timeout configuration file options
( http://www.beowulf-underground.org/pipermail/pvfs2-developers/2005-June/001287.html )

- It turns out that it wasn't necessary for clients to choose a value if
  multiple file systems disagree on timeouts.  Instead they just store
  independent settings for each FS.

- This patch adds --client-job-timeout and --server-job-timeout options to
  genconfig (each toggle both flow and bmi timeouts)

- A few operations were multiplying the default client timeout by 5 on the
  premise that those particular operations may take a while.  I removed those
  special cases and instead increased the default client timeout values for
  everything.  My reasoning is that if any one operation can take a long time,
  then by extension any other operation could take an equally long time because
  of queueing behind the slow ones.

* added event logging for states
* moved state debug code to state-machine so all state start/end
points get debug messages automatically
* added textlog stuff.  This allows us to write the default event
queue out to a text file that can be converted to slog2 later.

doxygen-related checkins.

- gossip/debugging cleanups

- fixed trove error codes (yet another pass; this time things look
  sane)
- give trove flush and resize operations the ability to create the
  bstream they're working with if it doesn't exist instead of
  returning an error (easily triggerred on multiple servers if a small
  amount of data is written, followed by a sync -- bstreams that
  weren't yet created fail, causing the call to fail as a whole);
  pointed out by RobL
- fixed bug in (threaded) trove that did not properly reinitialize
  condition variables after the first finalize
- improvements to the storage space and collection removal code
  (rmspace)
- extensive trove cleanups and cruft removal
- check for error codes when doing server side getattr and reading
  metafile information; stop processing if an error is reached.
  pointed out by RobL
- invalidate attr cache on write_at calls (never used)
- changed pvfs2-showcoll coll_id prints from hex to decimal

- make sure job_req_sched_release makes sure that the called supplied
  id looked up is valid (non-null)
- modify I/O state machine to not release an id that wasn't posted
- the above two changes fix a pvfs2-server crash bug on I/O where it
  was possible to try to release a req_sched job that was never posted
  (e.g. i/o during admin mode change)

- more cleanups

- applied Nathan's expandtab patch (expandtab-2.patch.gz)

  first referenced at:
  http://www.beowulf-underground.org/pipermail/pvfs2-developers/2004-July/000745.html

Updates for distribution renamings.

server now reference counts bmi addresses

tested and enabled server side timeouts for flow operations.  For now the
server defaults to this behavior: if a flow goes more than 30 seconds
without moving any data at all, it will be cancelled.  Still no timeouts on
the client side yet; coming soon...

- set a server side timeout on how long it takes to transmit responses; for
  now it is #defined to 30 seconds, should really be a config file option
- still no timeout on flows, need something more clever there

add timeout arguments to bmi and flow related job calls, not used yet

break up enormous print statement so it fits into gossip buffers :)

added some extra gossip debugging messages to help verify if the parameters
to some I/O operations are correct

- compile fix

- server and client side io sm cleanups
- give io it's own debugmask
- improved error handling on the client side

updated to sm syntax: state machines now allow a special "terminate"
transition to indicate the final state in a state machine.  updated
all state machines accordingly.  statecomp will generate a warning if a
statemachine does not contain at least one "terminate" or "return"
transition.  other related cleanups and sanity checks.

remove some more now unnecessary attribute retrievals

remove a now unnecessary disk access in the server-side get-attr path

- metadata storage overhaul (remove the METADATA_KEY from Trove_Common_Keys,
meaning that metadata is now properly stored in the dspace generic attributes
rather than as a separate key/val pair)
- clean up the 'find' test program to handle more than a fixed number of
  dirent entries
- misc other cleanups
- ChangeLog update

Changed iod_num and iod_count to server_nr and server_ct.

whoops, reverting some accidental commits

fixed name, added some bmi repost logic I forgot about

set response type correctly for io write

added an extra argument to all of the job API post fuctions: status_user_tag.
... similar to user_ptr semantically (set by caller at post time, returned
by API at test time), except that it is an integer type and is returned
through the status struct.  Not used yet, but intended to help in client
side state machines that have multiple transitions to the same state and
need a way to differentiate between them.

memory leak fix- write operations send two acks, and I was forgetting
to release the first one before sending the second one...

Put remaining code in place to handle memory datatypes, not ready yet
(pending a fix to some corner case in req processing).  Everything should
still work as it was prior to this commit.  When we do enable memory
datatypes, I'll need to go back and uncomment some code and adjust all
PVFS_sys_{read/write} calls to pass in the right datatype (they are just
taking NULL for now).  PVFS_sys_{read/write} API is in its final form
(I think...) for now, at least.  For now this code is dependent upon
the fact that the file datatype isn't implicitly tiled yet, need to get
memory datatypes going before relaxing that constraint...

changed reference to datatypes called "io_req..." to "file_req...", to make
the distinction clearer between file datatypes and memory datatypes once the
latter is added

modified flow descriptor so that file data structure is no longer
a pointer; avoids an extra malloc step in some cases

minor cleanup- got rid of an unused "flags" field, will add it back
later if needed

renamed PINT_server_job_context to server_job_context (only used in server).  Moved extern declaration into pvfs2-server.h,
and then removed all the externs from all the state machines.

General cleanup and reorganization in src/server.  Got rid of server-state-machine.c, merging the right stuff into pvfs2-server.c.

I've started using server_ for the prefix for functions used only in the server; seems like a good way to differentiate.

I'm still not happy with the error handling in the server; we need to do something about that.

header shuffling- you probably have to do a distclean to build after
updating

unified debugging messages for state transitions under a single macro that
only takes a single string (function name) as an argument and formats
the rest on its own.  Updated all state machines (except for flush) to use
it.

Aieeee....

New datatype request processing interface clumsily merged into tree.  For the
most part, things will work as before, but still need to:
- fix bug in precalculation when datatype offset is used
- fix data corruption when running multiple servers
- update flowproto-dump-offsets and enable again
- take out old #ifdef0'd code in bmi trove flowproto
- clean up flowproto's use of new interface

added error transition out of prelude states for each machine; this should
allow us to properly handle scheduler errors and eventually permission
checking failures

trying to rename some of the state machine handling functions to be more
consistent and reflect what they actually do

Created a common function that can be used to end execution of a state
machine, and started using it in all existing machines.  In particular, this
cleans up any resources that were allocated before the machine started
executing, so that the state machines don't have to worry about resources
that they didn't allocate...

random cleanup; made response structure a static part of server op structure,
rather than dynamically allocating it for each operation

Updated all existing server state machines to use new "prelude" nested
machine.  In theory, when we add permission checking we can do it by only
modifying that one nested machine.  Note, however, that most state machines
don't have an error transition from the initial state yet, so there is no way
to detect scheduling errors or permission failures until we clean that up...

the nomenclature in these machines was driving me bananas;
changed name of job_status_t* variables from "ret" to "js_p", changed
integer variables with names like "job_post_ret" to "ret"

even though phill pointed out how to do this in .vimrc, add the modelines

Added mode lines to emacs directives so emacs knows what to do with these files.

Cleaned up naming of state machines in server so we don't have the "clashing with syscall names" problem any more.

Woohoo look at me!  state machine headers are now obtuse.  and in src/common/misc.  and theoretically reusable between client and server.
New server-state-machine.c has server-specific functions for state machine; we'll need some of those for the client, but different implementations.

More random cleanup.  Still working on making the state machine code movable.

now using pvfs error codes; cleaned up includes all around; still trying to make it possible to move the state machine code in a reasonable way.

modifications to the encoding interface; took encoding type out of argument
and put into structs that represent encoding/decoding state where possible,
other minor cleanups

went ahead and pushed through the addition of a file datatype offset at
sysint, server, and request protocol level.  This stuff needed to happen
regardless of how we actually implement the feature at the lowest level.

Untested!

Piles of changes:
- removed rsize field from request protocol (useless in memory representation,
  duplicate of BMI reported size at wire level, used inconsistently
  anyway)
- fixed some contig encoder bugs (failed to add 4 byte trailer to some
  encodings when it wasn't convenient)
- fixed some unrelated bugs in the job handling path of client side I/O
  operations
- added some memory leaks in contig encoder, will fix later...

got rid of some extra typedefs, cleaned up some naming conventions

turned the "bmi_flag_t buffer_flag" variable into a named enumeration
"enum bmi_buffer_type buffer_type"

put in hooks to manually specify which flowprotocol to use, tested for
a few trial cases

moved the io state machine specific part of the server_op structure into
a server_io_op struct within the union

Added contexts to the job API, and updated all job calls to reflect it.
Contexts aren't actually implemented yet.

renamed statemachine files