CVS log for pvfs2/src/server/module.mk.in
	Help

Request diff between arbitrary revisions / Display revisions graphically

Merged robust security code into Orange-Branch.

initial working versionof usrint code
auto configures usrint on an ucache off (not working yet)
new acl repimplemented - old one has #def
some minor format edits

added new uid management app under src/apps/admin/pvfs2-get-uid
also included are a server and client side sm and the uid mgmt implementation files

remove extents, handle ranges, precreate pools and various other things that aren't needed in their current forms with UUIDs

distributed directory structure for root and lost+found directory. In pvfs2-server.c, it will check if the root directory is set. Need 10~20 seconds for creation when the servers are launched at first time.

Merged in changes from Orange-Branch. There were a number of bugs fixed there since this branch was created.

merge with Orange-Branch

Fixes to lookup.

Corrected memory leaks associated with using the PJMP state machine command.
Files affected:
	src/common/misc/state-machine-fns.c
	src/server/create-immutable-copies.sm
	src/server/get-attr.sm
	src/server/list-attr.sm
	src/server/module.mk.in
	src/server/precreate-pool-refiller.sm
	src/server/pvfs2-server.c
	src/server/pvfs2-server.h
	src/server/set-eattr.sm
	src/server/tree-communicate.sm

*** empty log message ***

merging Orange Branch changes in

initial merge with Orange-Branch. much will be broken

removed anything and everything related to certificates

revert cu-security-branch to before the attempted merge with Orange-Branch

merge of B2O-Blue-Sync,HEAD,and latest of Orange.

commiting changes from merge of Orange and Blue into test branch.

Added reduction kernels in CUDA (from NWU) into source tree.
Revised test programs.

Merge branches

merge from HEAD: http://www.pvfs.org/fisheye/changelog/PVFS/?cs=MAIN:pcarns:20090721184004

Merged NWU's CUDA code.
Modified configure script and Makefile to enable CUDA compilation.

merged in changes from latest Orange-Branch

merged in changes from summer at LANL

initial get-path state machine and associated changes

Initial commit of parallel fsck code

patch from Bart Taylor; moves permission checking code back to common
directory in order to add safety checks for truncate operation permission

Initial import of branch supporting keyval attribute/value lookup

Implemented kmeans algorithm based on the state machine, kmeans.sm.
There are two supporting state machines for this: allreduce.sm and bcast.sm.
The simple1_unaligned.c file is updated for testing kmeans.
Sample input file (edge17695.bin) for testing kmeans algorithm is added.

The buffer_size value in pipeline.sm is now from the fs_config,
and the default size is 1MB.
Added a skeleton for the kmeans operation, which is based on the nested state machine.

Merged Phil's tcp-peer-comm-branch that allows simultaneous server to server communications.
Added basic allreduce module:
  - implemented a recursive distance doubling algorithm.
  - works only when the # of servers are power of 2.
  - when activated, servers simply exchange their id to each other.

TODO
  - handle arbitrary sizes of buffers of various data types.

*** empty log message ***

Setting up Orange Branch

Merged read.sm and write.sm into a single file, pipeline.sm.

*** empty log message ***

new server state machine to handle getcred requests

merging hints/events code into trunk.

tree communication implementation

reverse merge from HEAD.

reverse merge of changes to HEAD since branch.  Includes small file changes.

Merging small files branch to head.  Includes server side precreation of
data files and file stuffing.

db-replication state machine which processes the incomming replication message

db-rep-send state machine added

Reverse merged and ported to HEAD.

got missed in merge of makefile changes.

hints and events.

commit of unstuff code that got missed before.

set up for create-file mod - stubbed out stuff, etc.
a few small bug fixes to statecomp and statemachine stuff

Big pile-o' changes to make Pete go mad.

The main changes are the implementation of the stuffed create path (now just the create state machine and operation), and the beginnings of the "unstuff" operation.

This also includes the request scheduler and state machine cleanups (required for using prelude with batch_remove) and whatever else I may have happened to change along the way.

This code is able to create files, but writing to them is almost guaranteed to break.

Sam's prelude and scheduler updates

The normal path for a server request operation is to jump to the prelude state machine, which tries schedule the request based on the handle+fsid.  This meant that the request scheduler code had a switch/case returning the handle+fsid for each request we defined.  The function was also overloaded with a readonly parameter (whether the request modified any objects on the server), and a scheduling policy parameter (as the return code), which specified whether the request scheduling could be bypassed for this request.

I moved all that to a per-request parameter structure, that gets specified within the server state machine for the request.  Now there is one array of server requests (moved to pvfs2-server-req.c) which the request scheduler and prelude state machine can poke at to get parameters (defined in the state machine for that request), and based decisions on.

The patch includes some other minor changes:

* A fix to the BMI addr ref count increment/decrement.  Previously, the addr ref count was being incremented/decremented in the prelude/final-response state machines.  That requires that those state machines get invoked, which I don't want to have to depend on.  I've moved the INC call to the unexpected_sm, and the DEC call to where the state machine gets finished in the server.  That way we don't have to worry about whether we're using prelude/final-response.

* Cleanup of the request scheduler.  The way we were changing the mode from normal to admin was sort of embedded into the request scheduling code, so we had to hack the mode change from prelude_sm instead of in setparam.  Also, the request scheduler had to be aware of the setparam fields in the request.  I've moved the mode change code to separate request scheduler and job calls.  I think something similar could be done for the timer code (it doesn't really belong in the request scheduler), but I'll leave that for another patch.

* Changed the way we compile the server binary in Makefile.in to allow for multiple object files.

stubbed in state machine to handle refilling precreate pools

inode stuffing operation.  also added a keyval_remove_list.

stubbed in new request type: batch_create.  The plan is for this to react
like create, except that it can create many handles of the same type (on
one server) in one protocol exchange

Hide PVFS2_SEGV_BACKTRACE option down in server code where it applies.
Fix little gossip module.mk.in thinko.

Move some server-only functions out of src/common/misc into src/server.
These functions happen not to compile on clients without getpwuid support.

Merge HEAD changes to TAS-branch.

add code to handle endianness differences between server and client for viewdist and other get-eattr ops.

merge of the WALT3 branch to HEAD.  This patch changes the way state actions
are represented as C structures (what statecomp generates).  It also changes
the main state action parameter from a s_op on the server or an sm_p on the client
to a unified smcb pointer (state machine control block) for all state actions (both client and server).  Finally, initial support for concurrent state machines has
been added to allow state actions to be invoked concurrently .. a first step for
server-to-server.

commit of changes to acl and extended attr checking code.  Hopefully this is will be easier to extend in the future.

Update migration branch to current CVS version

ooops, forgot the new file and to update the makefile.  This completes
the last commit.

Added in help for lock.sm.

backmerging of HEAD to branch...

Merge HEAD into Walt's branch.
Rework new state machines to the new cleanups introduced by Walt.

Merge posix-extensions-branch to HEAD
This branch implemented patches to the 2.6.16 kernel for the proposed POSIX I/O extensions and those patches are under
the patches subdirectory.
It also implements the PVFS2 specific hooks for these system calls.

Tools that may be of immediate use to the pvfs2 general audience is the pvfs2-lsplus utility in src/apps/admin
that should be noticeably faster than the pvfs2-ls utility if there are a lot of objects in a single directory.
Other features are left out by configure and are not even built if the kernels do not support those features
and callbacks.

Modification for migration added, some functions still need to be adapted after moving to a new branch...

Added request statistics to scheduler, reference from datafiles to
parent metafile (needed for migration) and auto migration tool(unfinished).

Added tool which prints file mapping to server aliases. Added
experimental stubs for migration tools. Added kernel tgid to request ID.
Bugfix of PINT_cached_config_get_one_server_str.

First cut of the sys_readdirplus implementation where clients can issue a single
system interface call to get the directory entries as well as the attribute information
of the directory entries.
What this means is that we do not need to call a sys_getattr of the individual directory entries
anymore.
(The kernel version is still untested and won't work until the VFS patch is in place! :))

On the server, there is now a new request/state machine called listattr(poor name, i know!)
which essentially does a vectored getattr for a list of handles as opposed to a getattr
which only does for 1. Perhaps, we need to consolidate them into 1 at a later point of time after
others have given this code a look-see!
On the client, there is a new system interface, and a corresponding statemachine implementation
called readdirplus which does the following algorithm
a) Step 1 - Do a regular readdir
b) Step 2 - Collate all meta handles (obtained from the readdir) and send out 1 listattr
to every server that owns atleast 1 handle from the set.
c) Step 3 - Handles that refer to metafiles (determined by the attributes of the response)
need an additional step to retrieve their file sizes. So we collate all data file handles
of all metafiles, and send out 1 listattr to every server that owns atleast 1 handle from
the set.
d) Step 4 - Copy object attributes obtained for all objects into the sys response structure and
also compute file sizes of regular files since we know the individual dfile handles' file sizes
from Step 3.

Clearly, this is a huge win in terms of performance of listing directories which have large number of directory
entries.
Next step is to hammer out the VFS patches and coreutils patch for ls to get the same benefit.
Also added a new utility called pvfs2-lsplus in src/apps/admin that uses this API.

commit of versioning stuff to branch

merging small I/O changes to trunk.  The majority of this commit includes the
new small I/O state machines on client and server, and the modifications to
the client IO state machine.  It also includes mods to the zero-fill code
to perform zero-fill of holes only if they exist instead of zero-ing the entire
memory region initially.  Also, some debug stuff has been added to the
state machine processing code to print enter/exit of state machines if sm
debug is enabled.  The genconfig now includes ioports/metaports for running
multiple servers on the same host for testing.

small I/O changes in sio branch for now

This patch adds a new system interface (listeattr) to list the extended attributes
of a file.
Also adds support in the VFS kernel module to support the listxattr
system call and a bunch of test programs to debug these.

Updated the FAQ to reflect usage of xattrs through VFS and acls through
VFS.

updates to my event changes to bring them inline with trunk

merge changes from trunk since branch point into branch.  Effectively moves the branchpoint to the latest trunk.

added extended attribute support

cleanup for extended attribute stuff.  removed non-list server ops
in favor of list type and renamed list type.  cleaned up a few items in
request debug programs.

dded delete extended attribute feature

updates for adding extended attributes

More work to make building of just server possible.  There is a hack here
in that we still build server/request-scheduler stuff, which needs to be
moved out of src/server.

- added a client/server operation and mgmt method
  (PVFS_mgmt_get_dirdata_handle/PVFS_imgmt_get_dirdata_handle) for
  retrieving the dirdata handle of a specified 'parent' object (if
  any)
- fixed a crash bug (race) in trove that could return an object on
  queue addition that has already been serviced (since we've signaled
  the trove worker thread before returning the id in the first place);
  should not be a problem if we used safe_register for the ids, but
  otherwise not worth it

- added a client/server operation for removing a directory entry under
  a specified parent object without prejudice
- modified pvfs2-remove-object to have working options to use that
  functionality

- added credential check to PINT_init_sysint_credentials to make sure
  they're valid before trying to copy (and returning -PVFS_EINVAL if
  they're not)
- added low level server/client mgmt operation to remove a pvfs2
  object
  without prejudice
- added src/apps/admin/pvfs2-remove-object tool to remove a pvfs2
  object using the new mgmt operation  (dirent removal is not yet
  supported, but currently arbitrary handles can be removed properly)
  [ this is a low level tool only useful if you know exactly what
  you're doing ]
- modify pvfs2-fs-dump to not crash if it finds a handle it expected
  to get information on isn't available

slight reorganization of how .c files generated from .sm files are tracked
so that the client and server rules are more consistent

- added a chdirent server operation useful during renames of target files
  that already exist.
- still working on cleaning up rename, as it does not properly remove some
  data that is left to be removed

add a mechanism for a server to send a generic response back to the client
when it receives a request with a bad version number or encoding type, so
that the user gets a clear error message at the application level too

add a server state machine (driven by a request scheduler timer) to check
for jobs that have timed out once per second; still not doing anything
because timeouts are set to inf. for now

- Build system:
    - list explicit include directories
    - add cflags provided by module makefile stubs
    - install kernel module if configured
    - simplify dependency generator script, add .po

fixed long standing makefile bug; pvfs2-server.c was not generating
dependency file, which means that it would not be rebuilt if header
files were modified

added stubs for server mgmt operation to retrieve event log

stubbed in server operation for iterate handles

started implementing server request that will retrieve performance
statistics using a mgmt operation

shuffled some files over to src/common/misc, added some calls to perf
counters in flow protocol, need to add locking next

some very early stubs of some ideas for implementing performance counters,
incomplete and untested

added a state machine that runs continuously in the back ground to
manage performance statistics (doesn't actually do anything yet), along
the way added support for starting state machines that are not
triggered by protocol requests

. landing a new and improved client-side truncate.
. completing the landing of a fully-functioning server-side truncate.

whoops- yeah, I forgot to check this in on friday...

don't clear dist info on i/o write; remove broken statfs stubs (Phil: did you
forget to add server/statfs.??)

added statfs request type, and a trivial stub for server state machine

noop support on the server side

server objects now get put into a library to link into server binary

General cleanup and reorganization in src/server.  Got rid of server-state-machine.c, merging the right stuff into pvfs2-server.c.

I've started using server_ for the prefix for functions used only in the server; seems like a good way to differentiate.

I'm still not happy with the error handling in the server; we need to do something about that.

stubs for setparam management operation on server

initial support for "flush"

this is some dirty debugging code that isn't used anywhere except in server test
programs and encoder test programs; I don't know why it was in the src/server
directory, or why it was being added into libpvfs2.  I stuck it all in one
header file to avoid having to write build rules to catch it, and moved it
out of the src subtree

added stubs for another nested state machine

added stubs for a nested state machine to use at the beginning of most
server operations; posts to request scheduler and checks permissions

Moved rule for building .c from .sm into root Makefile.

Woohoo look at me!  state machine headers are now obtuse.  and in src/common/misc.  and theoretically reusable between client and server.
New server-state-machine.c has server-specific functions for state machine; we'll need some of those for the client, but different implementations.

Modified statecomp.c so that it takes an optional output filename.  This helps put the .c files in the right place for the out of tree build.  Modified module.mk.in so that it uses this capability.

Moved the server-config files from the server directory to src/common/misc
in preparation for sharing the configuration file parsing code between the
server and the client.

Added a small amount of code toward getting client side parsing of the
configuration file on sysint initialization.

modified to accomidate new .sm format/extension for state machines, also
uses new arguments to statecomp program

more dead code removal.

Added rmdirent statemachine.

Removed a debugging code fragment from remove

Added comments to statemachine table and added rmdirent

changed module.mk.in to reflect the new addition

Used the correct free for memory leaks.

added the beginnings of a state machine to handle IO operations.  It is
based on Dale's example skeleton- I just wanted to stick it in place to
have something to play with.

Many many changes... extern works, all statemachines updated.  Getconfig works with root handle
server_config does what it is supposed to along with multiple filesystems

Tons more

Comments!!!

Verified that get-attr/setattr/get-config/crdirent/readdir work...
Still working on readdir

shuffled some variables around in the makefiles to make rules a
little cleaner

added in test programs for server

added in core server code and rules to build it