CVS log for pvfs2/include/pvfs2-util.h
	Help

Request diff between arbitrary revisions / Display revisions graphically

Merged robust security code into Orange-Branch.

added PVFS_PATH_MAX

added support for group ids in credential generation

added utility function to refresh credentials for long operations

moved security types into main pvfs2-types.h header

brought create credential utility functions up to date

removed unused fsid and serial fields from credential struct

merged in changes from summer at LANL

Initial import of branch supporting keyval attribute/value lookup

updated all references in include/, src/client/, src/common/, src/io/,
src/proto/, and src/server/ to use the new PVFS_credential in place of the
old PVFS_credentials. the admin apps in particular need to be updated to use
the new API.

Reverse merged and ported to HEAD.

created function to create a fake credential then updated every admin app
that uses PVFS_sys_getattr

[on behalf of Phil]:

pvfs2-check-server-gen-mntent.patch
-----------------------------------
This patch does two things.  First, it adds a utilty function called PVFS_util_gen_mntent() that can be used to create a mntent structure in memory without parsing a tabfile.  Previously there wasn't any clean way to construct this on the fly.  Secondly, it adds a utility (using the gen_mntent() function) called pvfs2-check-server.  pvfs2-check-server can be used to probe a specific pvfs2 server to see if it is running and responding to protocol requests.  Since it does not rely on a tab file, it can talk directly to one specific server without having to retrieve configuration information from the primary server listed in the tab file.  This utility is useful for watchdog purposes when you want to find out if one specific server is working properly.

Update migration branch to current CVS version

reverse merge of HEAD to WALT3 branch.

backmerging of HEAD to branch...

Merge HEAD into Walt's branch.
Rework new state machines to the new cleanups introduced by Walt.

apply const to all credentials structs passed in via client interface for correctness and optimization

Merge posix-extensions-branch to HEAD
This branch implemented patches to the 2.6.16 kernel for the proposed POSIX I/O extensions and those patches are under
the patches subdirectory.
It also implements the PVFS2 specific hooks for these system calls.

Tools that may be of immediate use to the pvfs2 general audience is the pvfs2-lsplus utility in src/apps/admin
that should be noticeably faster than the pvfs2-ls utility if there are a lot of objects in a single directory.
Other features are left out by configure and are not even built if the kernels do not support those features
and callbacks.

Override Murali's changes with my own that just copies the function.  Let the edit wars commence!

Changed PVFS2_translate -> PVFS_translate
Added a separate shared file for kernel and user space..

Merge from HEAD to posix-extensions-branch...
This should facilitate merging the posix branch to HEAD...

Implement root squash, read only and all squash export options. All of these are server-side
settings and set in the fs.conf files.. Perhaps the docs need to be pushed out to the pvfs.org website
after this check in?
Wild cards and mask-based addresses are allowed as part of the fs.conf export settings..

Also added a mount time option to honor setuid bit in files, mount -o suid...

Removed the -c option from pvfs2-ping and made the config file checks default.

Added crypto digests utility functions and configure time checks for openssl libs and includes
so that pvfs2-ping utility can now optionally check whether fs.conf files are the same on
all servers..
i.e. pvfs2-ping -m /pvfs -c will now print whether fs.conf integrity checks fail or pass.
Some of the infrastructure used here will be useful for doing SIGHUP handling at the server
so that fs.conf or server.conf files may be refetched etc..

Reverse merges from HEAD..

Backmerge with HEAD

move declaration to be with its peers

reverse merge from trunk.  working for now.

Reverse merge changes from HEAD to posix-extensions-branch.

- Misc. cleanups (replaced a big macro with an inline function)
- Removed parent_*time from the *dirent request protocol since time stamps are now
kept at the server and do not need to be sent over the wire (This is the reason why
major version of the wire protocol has been bumped)
- Adjust macros for the *dirent protocol accordingly.
- Add a few more debugging statements to kmod

Reverse merge from trunk to lockd-branch...

reverse merge of branch

reverse merge of trunk to branch

Consolidates I/O paths through VFS for read, write and readv, writev.
Move duplicated get_object_type function to pvfs2-util.h
Changed the type of a bmi local variable from int to bmi_size_t.
Since new dspace layout makes use of comparision function, update the db-dspace "hack utility" as well

cannot inline code in a different C file, even by declaring it so in the include file

fixes to small io stuff to get test-zero-fill working again.  Also committing updates from HEAD to make things easier for merge.

[pcarns]: somewhere along the line permission translation droped setgid bits.
add them back.

replaced zero-fill code in client io state machine with one that works.  We:

* zero the buffer according to the memory request
* do flow
* calculate return size:
 - check that the ub of the file request is within file data (not a hole or past eof) -> return_size = sizeof(filereq)
 - if that fails, search through retreived datafiles for offset > ub_file_req -> return_size = sizeof(filereq)
 - if that fails, get all the datafiles and search through them for offset > ub_file_req -> return_size = sizeof(filereq)
 - if that fails, calculate correct return size from file req and eof
  * while(next_contig_segment(file_req).ub < eof)
      return_size += sizeof(current_contig_segment(file_req))
    if(current_contig_segment(file_req).lb < eof)
      return_size += (eof - current_contig_segment(file_req).lb)

---

* Added a test for holes and appropriate zero-fill behavior
* Also fixed error reporting for ping to display individual server errors.
* Fixed a bug in PINT_dist_copy

updates to my event changes to bring them inline with trunk

merged acache changes to trunk.

merge changes from trunk since branch point into branch.  Effectively moves the branchpoint to the latest trunk.

more acache updates.

intermediate commit of acache changes to branch.

[pcarns]: This patch adds setgid support to pvfs2 (server side, kernel module,
and admin utitilies), including the semantics of propigating the setgid bit
to new subdirectories or files.  It also makes sure that setattr returns
an error if the user requests permission bits that are not supported
(just the setuid and sticky bits after this patch).

It turns out that there was already a getattr (or maybe a lookup?) in place
that we can get the information from [avoiding an additional getattr call].

Changed PVFS_util_get_mntent to PVFS_util_get_mntent_copy so that mount
entry can be safely accessed while other filesystems are being actively mounted
and unmounted.

Added num_dfiles as a tabfile option.  Added PVFS_util_get_mntent interface
to pvfs2-util.h, a top level include.

- added a setattr debugging mask and changed most detailed setattr
  debugging to use it
- added a mkdir debugging mask and changed most detailed mkdir
  debugging to use it
- added some inlined methods in PVFS_util (as they need to be used on
  both the server and the client) for getting the current time in
  PVFS_time format, encoding a PVFS_time as a version (finer grained
  than a 'normal' PVFS_time since we can use the high 32 bits), and
  decoding the version as a PVFS_time
- added a compatibility hack that _should_ allow no noticeable
  breakage on existing storage space, but will eventually migrate to
  the slightly new storage format over time
- modified client side sys-mkdir, sys-create, and sys-symlink to
  encode the mtime as a version when passing it to the server (so it's
  transparent from the server perspective)
- modified server side get-attr to decode the version read from disk
  back into an mtime (so it's transparent from the client perspective)
- modified mkspace method to properly version newly created root and
  lost+found directories
- modified the server side mkdir operation to return -PVFS_EINVAL if
  the object attr type is not a directory object
- misc debugging changes and cleanups

- moved static copy_mntent method to PVFS_util_copy_mntent method
- added error handling to PVFS_util_copy_mntent
- renamed PVFS_sys_free_mntent to PVFS_util_free_mntent
- fixed successful return value of PVFS_util_copy_sys_attr
- fixed initialize-dyn test program that broken due to mntent changes
- misc cleanups

- added a PVFS_util method to get the current system and return it as
  a valid PVFS_time type
- added arguments to the crdirent/chdirent/rmdirent operations that
  now can take optional parent atime/mtime/ctime fields
- added support on server *dirent operations that update the parent
  time attributes (if specified) on successful completion of the
  *dirent operation; null/zero time fields are not updated
- modified sys-create/sys-mkdir/sys-symlink/sys-rename/sys-remove to
  work with the new operations properly; in general,
  create/remove/rename operations updates the mtime and ctime of the
  parent directory, which is now properly supported
- moved some common server side attribute handling code into a macro
  to make it re-useable across several server state machines
- updated en{de}coding of modified operations
- misc cleanups

- applied Nathan's expandtab patch (expandtab-2.patch.gz)

  first referenced at:
  http://www.beowulf-underground.org/pipermail/pvfs2-developers/2004-July/000745.html

- change newly created file or directories to be reasonable defaults
  (slightly modified patch submitted by Nathan)
- add functionality to pvfs2-cp that preserves permissions across copies
  (handles unix->unix, unix->pvfs2, pvfs2->pvfs2, pvfs2->unix)

- merging in the pvfs2-nm-nb-branch with the main tree
  see ChangeLog for details, or browse the cvs history of the branch
  for full details

- implement PVFS_strerror_r; an equivalent to strerror_r (the
  thread-safe strerror call) that handles PVFS_error codes [ Brad,
  take note! :-P ]
- move PVFS_util_min to pvfs2-util.h, as it's a useful macro
- removed some debugging noise from pvfs2-client-core
- to avoid long stalls on cancelled I/O operations for now, use a
  specially added BMI_set_info flag to drop the connection on bmi
  cancellations (thanks, Phil!) -- this makes the client more
  responsive after cancellation since the server reqsched would
  normally need to timeout the I/O job it was working on before
  allowing any other jobs through.  the old client was killed, so it
  was similar to dropping the connection like we're doing now until we
  can implement a specific I/O cancellation operation to be sent to
  the servers explicitly
- make sure all internal I/O jobs are accounted for on cancellation
  within the sysint test()/testsome() methods to avoid allowing those
  jobs to pop out.  this is problematic for the caller since they
  should be hidden by the interface.  now we only complete a cancelled
  I/O call when ALL cancelled jobs are accounted for
- fix the final sys-io error code, as it's already resolved in
  io_analyze_results

- update branch with changes from the main tree

Remove split_pathname function at Neill's request.

- make unnecessarily exported pvfs2_util string methods internal
  PINT_* methods along with the others

Adding function to split absolute paths into a directory portion and a
filename portion.  Altered PVFS_util_remove_base_dir to return -ENOTDIR instead
of -1 when given a non absolute path.

- improve human readable size reporting
- allow human readable size reporting to use si units
- fix job-dev test program due to job dev api change
- fix memory leak of unexpected sm pointers
- add --si option to pvfs2-ls (similar to ls's --si option)
- add -H option to pvfs2-statfs (similar to du's -H option)
- merge with cvs head

- for non-blocking the passed in sys_attr structures need to be copied
  to avoid stack corruption/bunk values (similar to credential problem;
  there are probably more instances just like this, and they'll be fixed
  as they come)

- another pass over the non-blocking: cleanups, replacing macros with
  functions, making sure credentials live through the sm lifetime,
  removing gotos/confusing logic in i{sys|mgmt} calls

- implement non-blocking operations for all sysint and mgmt operations
- refactor blocking operations in terms of non-blocking operations
- change all sysint credential arguments to credential references
- memory leaks, bug fixes, cleanups

NOTE: this is a non-blocking first draft, updates are sure to come

added missing include needed for PVFS2_NAME_MAX if pvfs2-types.h doesn't
happen to be included by someone else

- properly remove mount information on umount by adding a umount operation

- continuation of dynamic addition/removal of mount information
- this should work, but the resolve hasn't explicitly been tested yet

- add a first draft cut of the ability to dynamically add and remove
  mount entries to the system
- add a simple test program that removes and adds some mount entries -- only
  useful if you have a carefully crafted setup (as I do)
- this *might* work, but needs more testing and work

- consolidate the credential generation in the admin tools
- added a generic/default credential generation function

shuffle tab file structure over to utils header now that system interface
isn't bound to it

added new utility function (PVFS_util_get_default_fsid()) that can be used
to retrieve the default fs identifier out of the set of file systems
that are currently initialized; useful for test programs, or for admin tools
that want a default fs to monitor if not specified by the user.  currently
testing in "client" test program

added a new utility function that will serve as a wrapper for the standard
initialization steps used by most of our tools, tentatively named
PVFS_util_init_defaults(), testing in pvfs2-import

added a new utility function for converting local paths into pvfs relative
path and fs_id, testing in pvfs2-import

modified PVFS_util_parse_pvfstab to return a const pointer to a tab
structure managed within the library; user no longer responsible for
allocating or freeing any of it

renamed pvfs_mntlist -> PVFS_util_tab, also renamed some of its fields.
This is going to look a little weird until some other updates catch up...

added a new argument to parse_pvfstab() to force the use of a particular
tabfile rather than the usual system choices, can be left to NULL for
default behavior

. break out the "make_human_readable" function into pvfs2-util
. make pvfs2-ls and pvfs2-satatfs use the new utility

WARNING: pvfs2tab format has changed- I'll post to list about it in a minute

PVFS_util_parse_pvfstab() cleanup:
- renamed some mnt struct fields
- rewrote parsing, using getmntent() and friends
- removed filename argument to function
- added prioritized searching for entries in PVFSTAB_FILE, /etc/fstab,
  /etc/pvfs2tab, ./pvfs2tab
- changed format of pvfs2tab to be a little cleaner and to fit in nicer
  with system fstab
- corrected the way that tab file options are represented

make symlink attrs cacheable; some pcache doc changes

first cut of getting the pcache integrated properly with sys-getattr.
this stuff still needs a lot of work, though it should work as is.

code format mangling

PINT_remove_base_dir() -> PVFS_util_remove_base_dir()
PINT_remove_dir_prefix() -> PVFS_util_remove_dir_prefix()

updated PVFS_util_lookup_parent() to return an error code, and to take
credentials as an argument

Roooaaarrr!  Header files broken!  Phil smash!
started off innocently as a bid to move lookup_parent_handle() to
PVFS_util_lookup_parent(), but ended up changing all kinds of stuff
in a chain reaction.  Also discarded helper.[ch].
- we need to make PVFS_util_lookup_parent() return an error code one
  of these days...

"new and improved formula pvfs, now with 70% less extern declarations"
renamed parse_pvfstab() to PVFS_util_parse_pvfstab(), and moved from
src/client/unix-io to src/common/misc, took pint-sysint.h out of exposed
header set, stopped compiling the last lingering objects from
src/client/unix-io, a few other little cleanups