CVS log for pvfs2/include/pvfs2-types.h
	Help

Request diff between arbitrary revisions / Display revisions graphically

Merged robust security code into Orange-Branch.

initial working versionof usrint code
auto configures usrint on an ucache off (not working yet)
new acl repimplemented - old one has #def
some minor format edits

added ERANGE error code

added PVFS_PATH_MAX

merge with Orange-Branch

increase maximum number of eattrs that can be requested, fix bug in nkey check found by hubcap

remove extents, handle ranges, precreate pools and various other things that aren't needed in their current forms with UUIDs

updates to user interface code:
all posix wrappers implemented
socket wrappers added
sendfile implemented
sys-lookup modified to return a path on failure
other files are minor formatting changes

Committed Windows client code to Orange-Branch.

*** empty log message ***

checkpoint on the uuid road

move PVFS_ds_position to different types when appropriate, pass a flag around instead of using special canary values in the handle. Things will be broken for a while.

checkpoint, transition to uuids

changes to PVFS_handle_X

change to #defined calls to uuids

migrate from uint64_t type assignments to uuids

128 bit UUID handles

merge with Orange-Branch

fix to segfault in list-eattr if client provides too large of buffer, found by Mike Marshall. Also includes other changes to enforce attribute name, value and list lengths for eattr operations on the server side and list-eattr on the client side.

merge with Orange-Branch

Namespace collision of ENCODING_DEFAULT with an in-tree kernel module in newer (circa 2.6.33) kernels, just a #define rename.

merge with Orange-Branch

Porting client

Porting flow etc.

Created #define for ALL_FS_META_HINT_FLAGS which include immutable,
append, and noatime flags at this time.

local round-robin layouts for Hadoop.

Porting BMI sockets

Merged in changes from Orange-Branch. There were a number of bugs fixed there since this branch was created.

Finished gen_locks; working on common files

Updated Windows bmi-wintcp, gen-locks

merge with Orange-Branch

Next round of distributed directory changes.

incorporate dist-dir-struct to the source tree. It's compilable but not runnable. still have a lot to change and add. Commit as a touch base.

*** empty log message ***

per-dataspace handle type precreate pools implementation

added distributed directory struct and util functions, includes a md5 implementation. First try, more to change.

worked around security vulnerability in pvfs2-gencred app

merging Orange Branch changes in

encoding fixes

initial merge with Orange-Branch. much will be broken

lots of little fixes that have been in the works for a while

moved security types into main pvfs2-types.h header

revert cu-security-branch to before the attempted merge with Orange-Branch

merge of B2O-Blue-Sync,HEAD,and latest of Orange.

commiting changes from merge of Orange and Blue into test branch.

bug and memory fixes to getvalue operation

checkpoint

Merge branches

checkpoint

merge from HEAD: http://www.pvfs.org/fisheye/changelog/PVFS/?cs=MAIN:pcarns:20090727185444

merge from HEAD: http://www.pvfs.org/fisheye/changelog/PVFS/?cs=MAIN:pcarns:20090727184547

Merge of Orange-Branch, Orange-FailOver-Becky, and version 1.164 of sys-io.sm (do not unstuff on a read).
This version of Orange-FailOver-Becky supports failover for large files only.  The next iteration will
support small-io failover.

merged in changes from latest Orange-Branch

*** empty log message ***

merged in changes from summer at LANL

add flag for not generating paths for result sets

Merge range query changes and other fixes

patches from Dries Kimpe:
- Added check before redefining USE_GNU in pvfs2-server.c
- Made use of endecode_fields_xxx macro's consistent and solved warning
in case they were compiled out.  The macro should be used as a function
call, and thus have a semicolumn at the end.

patch from Dries Kimpe to prevent gcc warning in strict mode for public
pvfs2-types.h header file

Completed most of the getattr changes.  Not all done just yet.

SETEATTR will now update the meta-hint flag if the user is changing the mirroring mode.  If the mirror-mode is
changed to NO_MIRRORING, then the mirror flag is turned off.  Otherwise, the mirror flag is turned on.

Corrected memory leak errors with the seteattr request.

Just making a daily backup.

Merged ORANGE-BECKY-BRANCH - Last of mirror changes.

Testing sticky tag.

Testing sticky tag.

xattr changes

- Change the Trove version to 0.1.4
- Implemented trove_migrate() to update database from 0.1.3 to 0.1.4
- Migration from 0.1.3 to 0.1.4 stats existing bstreams and
  updates the b_size attribute in the database.

change default for layout

merging hints/events code into trunk.

reverse merge of direct IO changes to he-branch.

merge of direct IO code to PVFS HEAD.

reverse merge from HEAD.

reverse merge from HEAD to directio branch

increase xattr value limit from 256 bytes to 8 KB

reverse merge of changes to HEAD since branch.  Includes small file changes.

Merging small files branch to head.  Includes server side precreation of
data files and file stuffing.

Added new blksize field to system attributes returned by PVFS_sys_getattr().
Modified kernel stat function to return this value as the st_blksize field
in the stat() system call.  blksize is distribution specific and normally
indicates the strip size for the file.

Added flag to mgmt iterate handles function to allow it to request reserved
handle values.  Will be used to get a list of known precreated handle values
so that they can be accounted for in fsck and validate tools.  Untested.

adding threaded odirect impl

updated all references in include/, src/client/, src/common/, src/io/,
src/proto/, and src/server/ to use the new PVFS_credential in place of the
old PVFS_credentials. the admin apps in particular need to be updated to use
the new API.

Reverse merged and ported to HEAD.

added a PVFS_ATTR_SYS flag for capabilities

reverse merge of trunk changes since branch.

reverse merge latest trunk changes to small-file-branch and update msgpair
usage where needed

Merge from HEAD.  Fleshed out credential.  Encode/decode added for credential and cleaned up for capability.  Added capability/credential into request/response structures.  Fleshed out comments in security module.

Not sure how useful this is, but added a SETPARAM operation to trigger sync
and drop_caches for benchmarking purposes.  Can be triggered using
pvfs2-drop-caches command line utility.

protocol size limit for layout information; custom encode functions for
layout in order to enforce limit; remember to free decoding memory for
create request fields

reverse merge of latest changes from HEAD to he branch.

whoops, revert last commit for now (on wire server layout list has strings, not
BMI_addr_t types)

account for extra size possibly needed in create request to account for
layout information

Updated branch with code from HEAD

merge pvfs2-touch layout support from trunk to small-file-branch

added argument to pvfs2-touch to exercise file layout functionality.

example:
pvfs2-touch -l tcp://localhost:3334,tcp://localhost:3335 /mnt/pvfs2/foo8

also added default value for layout algorithm.

hints and events.

merge a bunch of fixes made to HEAD since 2.7.0.

removed temporary setparam option used for testing precreate

Big pile-o' changes to make Pete go mad.

The main changes are the implementation of the stuffed create path (now just the create state machine and operation), and the beginnings of the "unstuff" operation.

This also includes the request scheduler and state machine cleanups (required for using prelude with batch_remove) and whatever else I may have happened to change along the way.

This code is able to create files, but writing to them is almost guaranteed to break.

Sam's prelude and scheduler updates

Use setparam state machine as a temporary test harness for get_handles()
interface to precreate pools.  Added test program to trigger it.  Stub for
get_handles() function.

fix definition of PVFS_OP_NULL

Preliminary cut at servers asking each other for handles.  Request is
exchanged, but doesn't do anything yet.  Also some tweaks to how precreate
pool information is assembled at startup.

Start an api for keeping up with the precreate pools.  For now all it does
is find the pools on startup or create them if they don't exist.  All
of the API functions are static in pvfs2-server.c for the moment.  Running
with the assumption we will track allocated handles as keyvals.

fix for PVFS_ERROR_TO_ERRNO returning negative values.

fixup offsetof stuff.

Compile workaround on Cray.

add SETFLAG GETFLAG ioctls for immutable/append flags to work with chattr/lsattr.

A couple more error values not on some random machine.  Also avoid the error in case statements when you find more than one of these unhandled things.

new layout parameter for PVFS_[i]sys_create

Merge HEAD changes to TAS-branch.

walt3 reverse merge from head includes merges of pw's sm changes (no state declarations), cleanup of state machine code, and other general merging/fixes.

Added necessary types for handling lock operations.

merge recent fixes from HEAD

[phil]: This patch corrects a variety of error code problems.

See: http://www.beowulf-underground.org/pipermail/pvfs2-developers/2007-March/003253.html

Changed PVFS_ds_position from int32_t to uint64_t, allowing us to remove DB_RECNUM flag from our databases.

Also changed the dspace comparison function to read the dspace db front to back instead of back to front.

Also changed the dspace_iterate_handles code to use DB_MULTIPLE to improve
server startup times.

Created a better interface for load values in the performance counter.

Added real load statistics for bmi, flow and trove operations.

Update migration branch to current CVS version

Synchronization with HEAD

revert EWARNING

add pvfs2-validate to head.

Upgrade to current CVS version

reverse merge of HEAD to WALT3 branch.

fix comment for EACCES

Added enum PVFS_lock_type.

- Support for immutable files
- Cleanups in the pvfs2-bufmap interfaces (replace int with size_t)
- Utility for displaying extended attributes and setting them
- Support for disabling atime updates at the object level, file system level (mount time)
  and directories.

backmerging of HEAD to branch...

Modified parent handle reference for datafiles behavior.
Also allow client to get parent handle in getattr sm.

fixes from 1.5 branch that never made it to trunk.

Merge HEAD into Walt's branch.
Rework new state machines to the new cleanups introduced by Walt.

Merge posix-extensions-branch to HEAD
This branch implemented patches to the 2.6.16 kernel for the proposed POSIX I/O extensions and those patches are under
the patches subdirectory.
It also implements the PVFS2 specific hooks for these system calls.

Tools that may be of immediate use to the pvfs2 general audience is the pvfs2-lsplus utility in src/apps/admin
that should be noticeably faster than the pvfs2-ls utility if there are a lot of objects in a single directory.
Other features are left out by configure and are not even built if the kernels do not support those features
and callbacks.

Merge from HEAD to posix-extensions-branch...
This should facilitate merging the posix branch to HEAD...

Implement root squash, read only and all squash export options. All of these are server-side
settings and set in the fs.conf files.. Perhaps the docs need to be pushed out to the pvfs.org website
after this check in?
Wild cards and mask-based addresses are allowed as part of the fs.conf export settings..

Also added a mount time option to honor setuid bit in files, mount -o suid...

Removed the -c option from pvfs2-ping and made the config file checks default.

PVFS2-hint support + minor changes (verbose request scheduler)

Reverse merges from HEAD..

Backmerge with HEAD

Added PVFS-hint, changed tests for pwrite/odirect/transactions/aio dbpf
 support to allow users to explicitly enable or disable the functionality.
In the aio implementation is a bug since somebody else modified it.

reverse merge from trunk.  working for now.

Whew.. Final bits of the acl puzzle (hopefully!).
Clearly, ACLs require support at the server. (what was I thinking earlier? :))
Patch adds a PINT_check_acls() function that does that. the pvfs2_acl_entry structure is moved to a common
header file shared between kernel module and server.
When PINT_check_mode() throws up its hand in despair, ACL checks are done. NOTE: ACLs dont impose a performance
overhead when permission checks go through properly. When they fail, we post a job_trove_keyval_read()
and when that completes, we check if ACLs allow the operation to proceed.
Added a lot more debug statements.
It is unfortunately not enough to change only prelude sm, but lookup sm also had to be modified
to check for acls. sigh..

Hopefully, this does not break anything else. Simple preliminary tests dont seem to indicate any borkennes.
All but 2 of the LTP acl tests seem to be working now I think...

Reverse merge changes from HEAD to posix-extensions-branch.

Patch fixes the atime, mtime and ctime problems that people have
reported with PVFS2.
Patch changes the notion of who sets these times. Earlier behavior was the
clients providing the times to the server. New behavior is to have the
server set these times except in  the implementation of utimes() system
call whereby clients explicitly set the time (atime and mtime).
NOTE: We dont allow the clients to dictate the ctime setting at all since
that is consistent with all file systems.
So what we have is:
read, readdir -> atime update required
write, truncate -> mtime update required
ctime is updated on every setattr implicitly on the server.

Since there is no need for parent_*time fields in the server request for chdirent,
crdirent and rmdirent, the next step is to remove those fields, fix the endecode
macros and bump up the major version of the protocol.

Patch also implements these times lazily by flushing them only at the time
of a close rather than after every read/write which could be pretty
expensive.

merge of murali's changes from trunk.

[pcarns]: Patch to allow modifying distribution, distribution parameters and number of
data files to use within a given directory. Such setting will only impact creation of future files
within the same directory. It does not change existing files' parameters. All such hints
are inherited by newly created directories as well..
[pcarns]: Allow trove to continue to read a list of xattrs/keyvals
without stopping if some subset of keys dont exist in the keyval db. Modifies the keyval_read_list() function
to return an array of completely independant error codes for each key that is read.
[pcarns]: adjust encode_string() macro to make it safe to call on NULL strings
[pcarns]: bug in the getattr state machine that would cause it to read dirent_count most of the time
regardless of whether the attribute mask requested it or not.
[slang]: fix a couple of bugs in the get-attr server-side state machine whereby the dirent_count was being read
using the dirent handle instead of the dirdata handle. Fixed a return value from dbpf_keyval_get_handle_info_op_svc
so that get-attr would not needlessly hang..
[vilayann]: fixes for the viewdist utility yet again, since the keys for the xattrs have changed...

reverse merge of recent bug fixes from trunk

commit of keyval handle info changes.  This allows info to be stored on a per handle basis in a 'null key' inside the database.  Right now we just use it to store the count of directory entries, so that we don't have to do a read/modify/write in crdirent and rmdirent.

Also includes changes to the keyval strings to use #define values instead of the actual strings.  This just makes that code lest prone to bugs in the future.

Reverse merge from trunk to lockd-branch...

reverse merge of branch

reverse merge of trunk to branch

apply const to non-modifiable strings

Incorporated Pete's suggestions to make the code less obfuscated by removing straightforward macros
(enc_int32_t, dec_int32_t, etc) and replacing encoding and decoding macros as C functions.
Also added a new module variable and the associated proc entry file (/proc/sys/pvfs2/timing) to control
printing of how long it took to perform certain pvfs2 functions (such as getattr, lookup etc)

CVS/time stamp brain damage. :)

ROMIO defines SIZEOF_VOIDP (well, actually autoconf defined it), so take
measures to avoid future symbol conflicts.

backout offsetof

fix 64bit warnings

added PVFS2_ prefix to ALIGN_VAR

fixes for 32/64 bit issues

commit of murali's 64bit fixes.

onekeyval merge

commit includes:

* a new keyval-pcache component in dbpf that keeps track of positions returned from readdir.  The keys of the cache are handle+position and the values are the key names in the dbpf-keyval DB.  This cache requires some changes to be made to tcache (which its based on), such as entries that don't have a timeout.  If the cache lookup fails for some reason (server failover, etc.), the position is the index of the entry in the directory, so the fallback is to step through the entire list of entries up to that point.

* Murali's patch to the kernel module to fix readdir retries.  Previously the module was ignoring the token (position) returned from previous readdir calls and waiting for count to be 0 (ignored TOKEN_END).  This forced 1 more readdir operation (2x for directories with <32 entries).  Now we keep track of the token returned and pass that in to the next call.

* Changed the keyval code to not use the open-cache.  Now we just initialize the DB once and keep a reference to it.  This also needs to be done for the dspace code, and the open-cache should probably be cleaned up as well.

* fixes to iterate bugs

[pcarns]: most file systems have a maximum name and path of 255 chars.  May as
well make that the maximum for PVFS2 as well.  Has the added benefit of making
one of the LTP name-length tests pass.

merging changes from HEAD to dbpf branch

bad #include of pvfs2-config.h.  cruft from sio merge

merging small I/O changes to trunk.  The majority of this commit includes the
new small I/O state machines on client and server, and the modifications to
the client IO state machine.  It also includes mods to the zero-fill code
to perform zero-fill of holes only if they exist instead of zero-ing the entire
memory region initially.  Also, some debug stuff has been added to the
state machine processing code to print enter/exit of state machines if sm
debug is enabled.  The genconfig now includes ioports/metaports for running
multiple servers on the same host for testing.

don't include pvfs2-config.h

fixes to small io stuff to get test-zero-fill working again.  Also committing updates from HEAD to make things easier for merge.

committing lld/llu fixes to 1.3.1 branch

moved the llu and lld macros to an internal header so that we don't have to include pvfs2-config.h in the installed pvfs2-types.h header.

make changes to 1.3.1 as well

fix LP64 issue with Lu/Ld/Lx that I borked with the darwin patch.

add back in removed #ifdef __KERNEL__ includes check.  Nice catch Robl!

* malloc.h is obsolete in freebsd, check for it
* No error.h in freebsd
* linux allows Lu,Lx, and Ld to be interchangeable with llu,llx, and
lld.  Other platforms (solaris, bsd) don't.  Replaced all the L*
with ll*.
* bsd doesn't have mntent.h.  use fstab.h and our own funcs.
* bsd doesn't know about ssize_t.  Use size_t instead.
* bsd doesn't have asm errnos.  Check for each of those
errno macros and if not defined define to -1
* check for db3/db.h db4/db.h header if db path is specified using
--with-db
* darwin has aio support builtin, doesn't need -lrt
* remove kernel stuff in pvfs2-types.h that sets WORDSIZE.  gcc
defines __LP64__ on archs with 64bit longs so we use that instead.
* darwin's open doesn't accept O_LARGEFILE.  off_t is an int64 by
default.  define O_LARGEFILE if it hasn't been.
* need to run ranlib on darwin after libpvfs2.a gets installed to regenerate
table of contents

dunno how LE_BFIELD got set to 1...

small I/O changes in sio branch for now

[pcarns]: add protocol versioning to PVFS2.  Major version changes are
incompatible up or down.  New clients can't talk to old servers, but new
servers can talk to old clients.

[pcarns]: correct the error code to string mapings

updates to my event changes to bring them inline with trunk

merged acache changes to trunk.

merge changes from trunk since branch point into branch.  Effectively moves the branchpoint to the latest trunk.

commit of phil's set-sync client util.  Also fix a minor build bug in the eattr test

removed refs to flowproto-bmi-trove stuff

Murali's patch to add VFS support for extended attributes.

intermediate commit of acache changes to branch.

added extended attribute support

fixed error reporting for gethostbyname in BMI-TCP
also fixed simple omissions and errors in related code

[pcarns]: "This adds a new directory attribute field that indicates how many
entries (of any type) are in a given directory.  The server already had the
ability to gather this information, but it just wasn't being reported.   A
client can read this with getattr(), but it can't be modified directly."

[pcarns]: This patch adds setgid support to pvfs2 (server side, kernel module,
and admin utitilies), including the semantics of propigating the setgid bit
to new subdirectories or files.  It also makes sure that setattr returns
an error if the user requests permission bits that are not supported
(just the setuid and sticky bits after this patch).

It turns out that there was already a getattr (or maybe a lookup?) in place
that we can get the information from [avoiding an additional getattr call].

updates for adding extended attributes

introduce scanf string define to avoid 64-bit type warnings

*** empty log message ***

* added event logging for states
* moved state debug code to state-machine so all state start/end
points get debug messages automatically
* added textlog stuff.  This allows us to write the default event
queue out to a text file that can be converted to slog2 later.

significant protocol changes to solve 64-bit alignedness issues

export superblock magic number for romio autodetection

doxygen updates.

- added a setattr debugging mask and changed most detailed setattr
  debugging to use it
- added a mkdir debugging mask and changed most detailed mkdir
  debugging to use it
- added some inlined methods in PVFS_util (as they need to be used on
  both the server and the client) for getting the current time in
  PVFS_time format, encoding a PVFS_time as a version (finer grained
  than a 'normal' PVFS_time since we can use the high 32 bits), and
  decoding the version as a PVFS_time
- added a compatibility hack that _should_ allow no noticeable
  breakage on existing storage space, but will eventually migrate to
  the slightly new storage format over time
- modified client side sys-mkdir, sys-create, and sys-symlink to
  encode the mtime as a version when passing it to the server (so it's
  transparent from the server perspective)
- modified server side get-attr to decode the version read from disk
  back into an mtime (so it's transparent from the client perspective)
- modified mkspace method to properly version newly created root and
  lost+found directories
- modified the server side mkdir operation to return -PVFS_EINVAL if
  the object attr type is not a directory object
- misc debugging changes and cleanups

- error code type cleanup that may fix Nathan's 64bit kernel oops (probably
  a bad cast); but may cause additional compilation warnings

- typo fix

- make sure that our error routines strip any known class error bits
  instead of deciding that they're unrecognized if set
- improved trove error handling
- improved pvfs2-server error handling
- fixed crash bug in our config parser that tried to strdup strings
  that might be NULL
- pvfs2-types header file simplification and cleanups

make statfs return load averages from sysinfo()

- changed PVFS_time from int64_t to uint64_t
- removed pvfs2-fsck compile warning
- fixed bug in sys-remove that erroneously tried to back out of the
  remove even on error codes that are acceptable to return.  now those
  codes are returned.  an example is trying to remove a directory that
  isn't empty and getting the wrong error code all together (i.e. NOT
  -PVFS_ENOTEMPTY) since the crdirent will fail and pass back another
  error code on failure
- simplify atime/mtime/ctime updating and the sys_attr to inode
  conversion in the kernel module

- applied Nathan's expandtab patch (expandtab-2.patch.gz)

  first referenced at:
  http://www.beowulf-underground.org/pipermail/pvfs2-developers/2004-July/000745.html

- add a PVFS_EACCES error code that maps to EACCES
- move PVFS_EDETAIL to be a non-errno pvfs2 error code
- re-add the lookup on create error -PVFS_EEXIST, as if enough clients
  try the create, it can still happen (if lookup succeeds, pass
  success to kernel, otherwise -PVFS_EACCES)
- update file_open test program with largefile support and some more tests

- if an invalid or unsupported encoding type is specified in the pvfs2tab,
  cycle through all encodings (until one completes or all are exhausted),
  but dump an error msg pointing out the problem

- change newly created file or directories to be reasonable defaults
  (slightly modified patch submitted by Nathan)
- add functionality to pvfs2-cp that preserves permissions across copies
  (handles unix->unix, unix->pvfs2, pvfs2->pvfs2, pvfs2->unix)

- merging in the pvfs2-nm-nb-branch with the main tree
  see ChangeLog for details, or browse the cvs history of the branch
  for full details

- implement PVFS_strerror_r; an equivalent to strerror_r (the
  thread-safe strerror call) that handles PVFS_error codes [ Brad,
  take note! :-P ]
- move PVFS_util_min to pvfs2-util.h, as it's a useful macro
- removed some debugging noise from pvfs2-client-core
- to avoid long stalls on cancelled I/O operations for now, use a
  specially added BMI_set_info flag to drop the connection on bmi
  cancellations (thanks, Phil!) -- this makes the client more
  responsive after cancellation since the server reqsched would
  normally need to timeout the I/O job it was working on before
  allowing any other jobs through.  the old client was killed, so it
  was similar to dropping the connection like we're doing now until we
  can implement a specific I/O cancellation operation to be sent to
  the servers explicitly
- make sure all internal I/O jobs are accounted for on cancellation
  within the sysint test()/testsome() methods to avoid allowing those
  jobs to pop out.  this is problematic for the caller since they
  should be hidden by the interface.  now we only complete a cancelled
  I/O call when ALL cancelled jobs are accounted for
- fix the final sys-io error code, as it's already resolved in
  io_analyze_results

- prefer spelling of cancelled with two L's to one (they're both
  correct)
- add support in the pvfs2-client for ignoring an upcall retry for an
  op already in progress (as opposed to servicing it multiple times)
- add a method for cancelling operations in progress at the
  pvfs2-client-core level
- don't write the downcall of the completed operation if it matches a
  previously cancelled I/O request (kernel will ignore it anyway, but
  it's faster to not)
- implement the logic in PINT_client_io_cancel to cancel flows and
  write ack recvs still in progress
- clean up client-state-machine.h to make it more readable
- re-factor sys-io.sm to read more cleanly and allow easier access to
  the internal operations and phases in progress required for
  cancellation
- split up particularly large sys-io states into separate inlined
  functions
- add the ability of sys-io.sm to handle run-time cancellation while
  still transitioning properly
- add kernel method that allows the cancellation upcall/downcall cycle
  to progress, ignoring the fact that a signal is pending (since the
  only time a cancellation is issued is when there's a signal pending)
- add kernel macro to service the cancellation operation
- fix kernel oops on setattr (signal) interruption by returning a
  valid error code in this case
- misc cleanups

- added a license notice for the GPL'd kernel module code in COPYING
  and README files
- added Murali to the CREDITS file
- added a configure check for the kernel 2.4.x minor version to know
  if we need to define a few functions or not
- added a new pvfs2-error-code that can be used as an exit code for
  the pvfs2-client-core to indicate to the pvfs2-client that it should
  continue respawning it, but to sleep for a bit first.   [2.4.x in
  particular waits a while to release the device file after the
  process has died, though the same is possible under 2.6.x i guess]
- updated the test mount script with 2.4.x support if running 2.4.x
- added some debugging to client-side create state machine
- murali: fixed a logic bug in mmap-ra-cache
- murali: added matching unlocks to quickhash code functions
- removed MOD_{INC|DEC}_USE_COUNT calls, as at least under my 2.4.26
  testing, they are not necessary
- removed several lock_kernel calls in the 2.4.x code paths that
  caused the kernel to hang in a few places (still investigating)
- murali: fixed the in kernel mount option string parsing
- some misc cleanups
- verified that symlinks now work (at least in basic testing)

[ NOTE: this mostly works under 2.4.26 in my testing, however there
  are some known problems remaining ]

Adjusted parameters to statfs_list call as an example of new "details" interface use.

convert bmi and flow cancel cases to use new non-errno error code,
PVFS_ECANCEL, instead of the confusing PVFS_EINTR.  Will result in error
messages like this:
PVFS_sys_lookup: Operation canceled (possibly due to timeout)

Added PVFS_error_details functions and definitions for use in partial success
handling cases.  Not called anywhere yet.

fix the error class macro so it also strips out error bit and non errno bit

- add ability to add new pvfs2 specific error code and strings that do not
  map to system errno values

- first draft cut of re-working the system interface to be able to work nicely
  with multiple configurations from different servers.  the only restriction
  is that none of the configurations can contain information about file systems
  that have the same fs_id.

this fixes 3 separate things reported by robl and phil.  this changelog lists
phil's description and a description of the fix:

---
1) This one isn't too bad, but if I run this:
      ./pvfs2-client -p pvfs2-client-core
   instead of this:
      ./pvfs2-client -p ./pvfs2-client-core
  ... the output looks the same either way (ie, it appears successful).
In the former case, both the client and client-core actually exited,
because i specified the path to the core incorrectly.  It would be nice
if it printed something in this case to avoid confusion.
---

Ok, done...this is tricky because if the client-core is in the same dir as
the client program, a 'stat' will pass, but the execvp may fail if that dir
isn't in the system PATH.  Now, we print an error if the path is not absolute
and the client core exits.

---
2) I don't know if this happens everywhere or not, but I am running on a
box here where bad things happen if I get the hostname wrong on the
mount command, for example doing something like this:
        mount -t pvfs2 tcp://a43:0/pvfs2-fs /mnt/pvfs2
(note that the port is wrong).  Sorry I didn't catch this when you asked
me to try this stuff out the other day :(  At any rate, the mount command
segfaults, and I get this in dmesg:

Attempting PVFS2 Mount via host tcp://a43:0/pvfs2-fs
Got an unknown pvfs2 error code: -1073741967
pvfs2_fs_mount: got return value of -1073741967
Unable to handle kernel paging request at virtual address 733d4853
 printing eip:
 c016b062
 *pde = 00000000
---

This is due to an error code translation problem.  Previously, the kernel
code didn't translate all pvfs_errno codes to actual errno codes, so I've
re-worked the error code translation functions so that they can be shared
by the kernel and the user space code.

---
3) This one is kinda nasty, and I'm not sure what's going wrong yet.
The scenario is that I have successfully mounted, but then on the next
operation (a getattr), the communication fails causing the msgpair.sm to
call exit(1).  The pvfs2-client and client-core then exit.  I start them
back up, and nothing ever works again.  I get "Input/output error" any
time I try to access pvfs2 from then on, and I get "can't write
superblock" when I try to unmount.  I have to reboot the box to get back
to sanity.
---

This turned out to be a beast of a problem now that we have dynamic mounts on
the client side.  The issue is that on cancelled i/o, or on pvfs2-client-core
restart, there's *no* way the kernel can associate upcalls with the client-core
anymore because the dynamic mount information is lost forever as far as the
client-core is concerned.

The solution I chose for this is to have the pvfs2-client-core issue an ioctl
that causes the kernel to do mount upcalls for every pvfs2 file system it knows
about to avoid having to have the user manually issue another mount command
(or something goofy like that).  Other solutions that I considered and didn't
like are 1) having the pvfs2-client-core store mount info in a file somewhere
and have it remount those on startup, or 2) having the pvfs2-client and
pvfs2-client-core have some ipc going on so that the pvfs2-client can store
the mount info and somehow give it back to the pvfs2-client-core on restart.
both of those are shoddy at best, and do not seem robust at all (i.e.
stale file info will error out on stale remount attempts, ipc requires that
the pvfs2-client never dies along with the pvfs2-client-core, etc).

With the current solution, we have all the information that the kernel knows
about (and so it's more reliable than a file or something), and it's the most
natural way to hand the information up from the kernel (since we have to do
this via upcalls anyway).  Thus, the kernel code now keeps a list of pvfs2
superblocks and we store the info given to the sb at mount time so that it
can dynamically be remounted at any time.  The pvfs2-client-core now issues
an ioctl (in a thread so that it can still service the request) on startup
to 'remount' any file systems the kernel knows about.  while I'm calling this
a remount, it's basically just a mechanism for the kernel to tell the client
core enough information ot rebuild the dynamic mount tables in the system
interface.

because of this, the pvfs2-client-core program *requires* pthreads.  This is
completely independent of whether or not the PVFS2 sysint needs pthreads
at all.

rename PVFS_pinode_reference -> PVFS_object_ref in system interface, a few
related variable name updates

remove spurious yet harmless semicolon

added a named reserved value for fs ids (collection ids), so that we can
cleanly test for uninitialized fs id values

combo PVFS_id_gen_t -> PVFS_BMI_addr_t, formatting, PVFS error code patch.
Ugly.  Sorry if my formatting pisses someone off; at least I'm not using >
80 columns any more :).

really truly once and for all remove the (unneeded) reference to pvfs2-config.h

Added PVFS_EPARTIAL (for now anyway; would like a little better name!).
Added error_array to PVFS_mgmt_statfs_list(); dropped in NULL placeholder in
all calls to it.
Added allocation of error array in karma/comm.c to hold particular errors.
Note: none of this is actually *used* yet.

spelling

Spelling fixes, misc. stuff.

- keep the statfs object the same, even if sysinfo call isn't available

- add configure test for sys/sysinfo.h (should be on all linux machines)
- add memory and uptime information into the (now ill-named) statfs object
- update pvfs2-statfs to display this newly added information

- redefine WORDSIZE definition so as not to need pvfs2-config.h
- remove installation of pvfs2-config.h
- remove configure checks for sizeof(long) [ no longer needed ]
- version bump to pvfs2-0.0.8pre2, because pre1 is old and boring now

- a complete re-write of the client side lookup operation that can properly
  handle '.' and '..' segments, as well as recursive relative/absolute
  symlinks across multiple metadata servers
- addition of a lookup debugging keyword, specific to the lookup operation
- minor changes to test programs to aid development
- added two shell scripts in the test/client/sysint directory to give a
  casual look at some trickier cases that lookup can handle (and to help
  make sure future versions can still handle)

- configure test for size of long
- assign __WORDSIZE in kernel space based on it

- cruft replacement
- allows kernel code to build on alpha

- much improved rename implementation as a state machine (sys-rename.sm)
- better rename error handling
- removed original rename implementation (rename.c)
- fixed a bug in the rename test program
- switched pvfs2-types.h to use LE_BFIELD encoding by default
- made a pvfs2 util call follow links if the link is a directory when resolving
  the parent directory
- updated the ChangeLog

convert everything over to use new flowproto_multiqueue instead of
flowproto_bmi_trove by default

keep kernel land happy

defines and explanation for 64-bit printf friendliness

filled in remaining minimum hooks to be able to test new flow protocol on
some simple cases...

include just stub header from include/ dir

parse wire encoding method from fstab entry

pick up encoding header, add encoding rules for simple types

added hooks to enable and disable various event logging parameters, added
an admin program (pvfs2-set-eventmask) to poke at them.  Still no way to
retrieve values, though.

bump cvs pvfs2 version number;
initial draft of vfs truncate support;
re-enabled intent lookups getattr skips;
temporarily disabled pvfs2-client-core pcache usage

shuffled PVFS_aint in pvfs2-types.h over to job_aint in job.h, hopefully
fixing one of the kernel space type problems

made system interface level attributes expose datafile count (and
added corresponding attribute mask), dfile count may be reported in
getattr, and used at create time to specify number of datafiles to create.
re-enabled command line option to set number of datafiles in pvfs2-import.

iterater special value cleanups: renamed PVFS2_READDIR_START ->
PVFS_READDIR_START, defined PVFS_ITERATOR_START/END at top level types file,
redifined TROVE equivalents in terms of them, used INT32_MAX rather than
INT_MAX to reflect range of type used for position values

do not autogenerate pvfs2-types.h

some configure/type updates:
- new PVFS type: PVFS_aint, which is an integer just large enough
  to also be used to hold a pointer
- configure test to detect typedef to use for above
- move pvfs2-types.h to pvfs2-types.h.in so that autconf can generate it
- added removal of config.cache to make distclean target

added a new variant of PVFS_perror(), called PVFS_perror_gossip(), that
puts the output into gossip

started work on functionality to verify root handle from ping program,
not done yet

modification to error codes:
- added an extra high bit to mask onto all PVFS error codes to give us
  a way to differentiate them from errno values
- added an "IS_PVFS_ERROR" macro that can be used to determine if
  a given error code is a valid PVFS code or not

put in client side and application hooks to verify that the fsid is valid
on all servers, need to figure out why errors aren't propigating
back.  Fixed a couple of silly cut and paste errors

added pvfs2-set-debugmask for toggling debugging level of servers at runtime,
doesn't work yet.  Filled in more hooks to get setparam_all state machines
to run.

Removed all references to PVFS_SYS_IO_{READ,WRITE}, instead using PVFS_IO_xxx versions exclusively.

put a few pieces in place for mgmt_setparam operation, hopefully located
strategically so as not to conflict on merge with the flush...

defined PVFS_HANDLE_NULL and PVFS_OP_NULL, TROVE_HANDLE_NULL is now defined
in terms of the former

more progress in the prelude state machine; it retrieves common attributes
now when possible; not doing anything with them yet.  Also removed
PVFS_ERROR_FS error mask.

Massive cruft removal.  PVFS_credentials.perms no longer exists.  Some
kernel breakage to be fixed asap.

attribute mask cleanups:
- added a new sysint accessible mask (SYS_TYPE) for object type
- added a new convenience mask (SYS_ALL_SETABLE) to specify all masks
  that are legal to set using setattr(), create(), and mkdir()
- made sysint functions stricter about checking masks that get passed in
- updated many sysint callers to correctly set masks, and also to set time
  fields on the way in
- I don't think the time semantics are right yet, but at least we are
  a little more consistent from the client side now

PVFS_sys_getattr() and PVFS_sys_setattr() now operate on system interface
attribute structures; other minor cleanups and additional sanity checking.
Still need to update create and mkdir in the same fashion

prepping for kernel build stuff

more #define / limits cleanup; biggest change is the addition of
PVFS_SEGMENT_MAX to limit the size of each path segment

Renamed id_gen_t to PVFS_id_gen_t and moved it definition into pvfs2-types.h.
Also took the #include for id-generator.h out of header files and into .c
files where possible.

got rid of 3 linux specific errno codes from our errno mapping

changed #include <stdint.h> to #include <inttypes.h> (slightly more portable)

added code stubs for pvfs2-kernel device interface

very minor change :)  pvfs2-types.h cleanup is done, I think.  Still need to
fix some other headers.

changed PVFS_extent (struct extent) to PVFS_handle_extent and moved definition
into pvfs2-storage.h

minor cleanup of dirent definition (I think just comment changes)

cleaned up PVFS_pinode_reference definition

added a new attribute mask to cover the objtype field, and used it
in the appropriate places.  also took out extra checks in
phelper_fill_attr().  I _think_ things are all working on the
sysint side again...

big nasty commit: converted all of the attribute masks to new
style, as found in pvfs2-types.h.  This required changing a
lot of code, most significantly in the pinode cache / pinode
helper area.  Also got rid of a pinode helper function that was
rendered unecessary after the attribute modifications.

System interface tests appear to behave just like before...

Things still aren't right, though!  But I think it will work at
least as well as before the commit, and is slightly more consistent.
Things that are still off:
- the mask should be stored inside the attributes structure,
  rather than kept in a seperate variable everywhere
- server currently ignores attribute masks entirely and does its
  own thing
- a lot of the client code, in particular getattr.c and the pinode
  helper/cache, make big assumptions about how attributes and
  masks can be handled.  In a lot of places they are just copied
  around as atomic units regardless of mask, for example.

I had to stop changing stuff somewhere...

pinode_reference -> PVFS_pinode_reference

turned the PVFS_ds_type into a named enumeration and moved its
definition into pvfs2-types.h

got rid of the U_READ,U_WRITE, etc. family of #defines and replaced with
PVFS_U_READ,PVFS_U_WRITE, etc.

removed the MAX_STRING_SIZE #define, and turned it into a
PVFS_MAX_CONFIG_SIZE #define inside of sysint-utils.c for now; this will
be cleaned up later when we define the protocol limits a little better

got rid of the PVFS_token type and replaced with PVFS_ds_position where
appropriate

got rid of PVFS_type and replaced with PVFS_ds_type where appropriate

got rid of PVFS_count32 type; replaced with int32_t or uint32_t or int
where appropriate

got rid of the PVFS_count64 type

got rid of PVFS_flag type and bmi_flag_t.  need to come back later and
convert the buffer_flag variables in BMI to named enumerations; they are
ints for now

got rid of the PVFS_bitfield type, and replaced with something more
generic everywhere it was used

moved ds_flags from storage to types header

moved ds_position from pvfs2-storage.h to pvfs2-types.h

renamed enum flowproto_type to enum PVFS_flowproto_type and moved to
pvfs2-types.h

changed PVFS_time typedef so that it is based on int64_t rather than
time_t (getting rid of dependance on system specific type).

reorganized file a little, cleaned up some comments

got rid of PVFS_string type and replaced with char* everywhere

got rid of some easily identified unused types and #defines

step one of header cleanup: ran pvfs2-mangle-in-place-80col.sh on these
files to get the code format closer to standard

define a 'struct extent' for pvfs and trove

modified existing BMI API functions to support the concept of
"contexts" and added hooks to create and destroy them.  This is just a
placeholder, contexts aren't actually implemented yet.

add PVFS_perror() function; started using it in pvfs2-import.c in
place of the static pvfs_perror function I was using earlier

filled in a little bit more of import utility (argument parsing and
error handling); added an extern in pvfs2-types.h to get to the errno
mapping table

Defined error codes and classes, built table for mapping from PVFS errors into UNIX errno values.

A handle for the filename would be nice huh????

DW

formatting blues.

Working on setattr, checking in changes to get fs ops to work (for now; will maybe remove later).

Working on dspace setattr.  Nothing useful yet.

Imported sources

Initial revision