CVS log for pvfs2/src/io/trove/trove-internal.h
	Help

Request diff between arbitrary revisions / Display revisions graphically

uuid converstation, PVFS_x_position changes, server/client side first pass done

checkpoint on the uuid road

move PVFS_ds_position to different types when appropriate, pass a flag around instead of using special canary values in the handle. Things will be broken for a while.

Merged in changes from Orange-Branch. There were a number of bugs fixed there since this branch was created.

merge with Orange-Branch

initial transactional bdb work

Bump Trove DBPF version for new precreate-handle storage format. Include migration to 0.1.5. Add job_trove_fs_del_eattr() function and associated lower level calls

initial merge with Orange-Branch. much will be broken

bug and memory fixes to getvalue operation

checkpoint

Merge with work from this summer.  Metadata and data storage spaces can now be located on different disks.  Fixed a small bug in string encoding.  Updated test code to reflect new Trove dual-storage space API.  Genconfig updated as well as affected admin apps. In the correct branch this time.

Undo accidental commit to MAIN.  Reverted each file to pre-commit version.

Merge with work from this summer.  Metadata and data storage spaces can now be located on different disks.  Fixed a small bug in string encoding.  Updated test code to reflect new Trove dual-storage space API.  Genconfig updated as well as affected admin apps.

merged in changes from summer at LANL

get-path implemenation work

initial get-path state machine and associated changes

Merge range query changes and other fixes

Initial import of branch supporting keyval attribute/value lookup

- Change the Trove version to 0.1.4
- Implemented trove_migrate() to update database from 0.1.3 to 0.1.4
- Migration from 0.1.3 to 0.1.4 stats existing bstreams and
  updates the b_size attribute in the database.

Properly shutdown trove collections that have different methods than the global method.  For example, filesystems specified with directio need to call the directio cleanup to shutdown the threads involved.

merging hints/events code into trunk.

reverse merge of direct IO changes to he-branch.

merge of direct IO code to PVFS HEAD.

reverse merge from HEAD.

reverse merge of changes to HEAD since branch.  Includes small file changes.

Merging small files branch to head.  Includes server side precreation of
data files and file stuffing.

fixes for cancel.

dspace, keyval, coll_attr dbs are now created within the environment.
A whole lot of other things try to make replication work

db-replication state machine which processes the incomming replication message

Reverse merged and ported to HEAD.

dspace_remove_list operation for trove; untested

hints and events.

a completely untested dspace_create_list() operation to allow multiple
handles to be created with one trove call

inode stuffing operation.  also added a keyval_remove_list.

Update migration branch to current CVS version

Upgrade to current CVS version

reverse merge of HEAD to WALT3 branch.

commit of trove method capabilities.  We should now be able to support multiple trove methods.  The TroveMethod option to the config file can set the method to something other than the default (dbpf).

hint is given down to Trove

backmerging of HEAD to branch...

Merge HEAD into Walt's branch.
Rework new state machines to the new cleanups introduced by Walt.

Merge posix-extensions-branch to HEAD
This branch implemented patches to the 2.6.16 kernel for the proposed POSIX I/O extensions and those patches are under
the patches subdirectory.
It also implements the PVFS2 specific hooks for these system calls.

Tools that may be of immediate use to the pvfs2 general audience is the pvfs2-lsplus utility in src/apps/admin
that should be noticeably faster than the pvfs2-ls utility if there are a lot of objects in a single directory.
Other features are left out by configure and are not even built if the kernels do not support those features
and callbacks.

Reverse merge changes from HEAD to posix-extensions-branch.

merge of murali's changes from trunk.

[pcarns]: Patch to allow modifying distribution, distribution parameters and number of
data files to use within a given directory. Such setting will only impact creation of future files
within the same directory. It does not change existing files' parameters. All such hints
are inherited by newly created directories as well..
[pcarns]: Allow trove to continue to read a list of xattrs/keyvals
without stopping if some subset of keys dont exist in the keyval db. Modifies the keyval_read_list() function
to return an array of completely independant error codes for each key that is read.
[pcarns]: adjust encode_string() macro to make it safe to call on NULL strings
[pcarns]: bug in the getattr state machine that would cause it to read dirent_count most of the time
regardless of whether the attribute mask requested it or not.
[slang]: fix a couple of bugs in the get-attr server-side state machine whereby the dirent_count was being read
using the dirent handle instead of the dirdata handle. Fixed a return value from dbpf_keyval_get_handle_info_op_svc
so that get-attr would not needlessly hang..
[vilayann]: fixes for the viewdist utility yet again, since the keys for the xattrs have changed...

reverse merge of recent bug fixes from trunk

commit of keyval handle info changes.  This allows info to be stored on a per handle basis in a 'null key' inside the database.  Right now we just use it to store the count of directory entries, so that we don't have to do a read/modify/write in crdirent and rmdirent.

Also includes changes to the keyval strings to use #define values instead of the actual strings.  This just makes that code lest prone to bugs in the future.

First cut of the sys_readdirplus implementation where clients can issue a single
system interface call to get the directory entries as well as the attribute information
of the directory entries.
What this means is that we do not need to call a sys_getattr of the individual directory entries
anymore.
(The kernel version is still untested and won't work until the VFS patch is in place! :))

On the server, there is now a new request/state machine called listattr(poor name, i know!)
which essentially does a vectored getattr for a list of handles as opposed to a getattr
which only does for 1. Perhaps, we need to consolidate them into 1 at a later point of time after
others have given this code a look-see!
On the client, there is a new system interface, and a corresponding statemachine implementation
called readdirplus which does the following algorithm
a) Step 1 - Do a regular readdir
b) Step 2 - Collate all meta handles (obtained from the readdir) and send out 1 listattr
to every server that owns atleast 1 handle from the set.
c) Step 3 - Handles that refer to metafiles (determined by the attributes of the response)
need an additional step to retrieve their file sizes. So we collate all data file handles
of all metafiles, and send out 1 listattr to every server that owns atleast 1 handle from
the set.
d) Step 4 - Copy object attributes obtained for all objects into the sys response structure and
also compute file sizes of regular files since we know the individual dfile handles' file sizes
from Step 3.

Clearly, this is a huge win in terms of performance of listing directories which have large number of directory
entries.
Next step is to hammer out the VFS patches and coreutils patch for ls to get the same benefit.
Also added a new utility called pvfs2-lsplus in src/apps/admin that uses this API.

commit of versioning stuff to branch

- applied Nathan's expandtab patch (expandtab-2.patch.gz)

  first referenced at:
  http://www.beowulf-underground.org/pipermail/pvfs2-developers/2004-July/000745.html

- first cut at supporting threaded-aio trove cancellation
- we now better handle errors by reporting them if they occur in any of
  the requests issued
- added a simple test-trove-cancel program as a sanity check
- misc cleanups

added a new PVFS_statfs structure (incomplete- just a few fields to
start with), implemented trove_collection_getinfo() to retrieve this
information from file system, hooked into server statfs implementation.
completely untested, and no client side support yet

trying to properly hook in testcontext now

added a coll_id arg to trove_[open|close]_context calls.  threaded trove.
things may be badly broken now as it's on by default.  even if things appear
to work, don't be fooled.  much work is left to be done.

Added dbpf thread stubs and a trove_testcontext method that should eventually
replace trove_testsome to better fit with the job interface.

Add storage specific open/close_context methods.  Move flow initialization
in pvfs2-server to be after trove initialization since a trove context is
needed.

small bugfix (related to "kill" error), some arguments were reversed

Added stubs for using contexts in trove.  None of this is working right now,
but the code *should* still work as before.  The idea is to get contexts into
the trove interface to play better with the job interface like the existing
bmi and flow interfaces.  This is the first step toward that end.

Added function for mapping errno values to trove errors.

initial work for flush support in trove

Made necessary changes for sending handle ranges over to the server.  This
required changes to the trove_dspace_create call, so a lot of stuff had to
change to make this compile.  mkdir is broken and may be for a while.

Server receives handle ranges now for creates, but doesn't do anything with
them yet.  That's to come.

Removed bitmask arguments along the trove_dspace_create call path; needs
testing; I'll be looking into this.

get function declarations in sync: collection_{get,set}info doesn't take a handle

vim formatting line fixes, using pvfs2fixbadformat.pl.

Implemented and tested (for simple cases only) collection iterate.  Had to change TROVE_ITERATE_START and TROVE_ITERATE_END values.

Added flags parameters to some of the dspace calls; updated all other code too.
Added coverage target in makefile; not working yet.

Monster checkin:
- removed the trove "fs" files, which are no longer in use.
- added dbpf-collection.c, which keeps up with multiple collections in use
  at one time.
  - hooks in place in dbpf implementation to theoretically handle multiple
    collections; untested (works for one collection).
- added trove-create-stress.c, a stress test for object creation.
- added TROVE_SYNC flag as an option, which is handled in most calls.
- found and removed a number of problems in error handling in dbpf calls.
- created pvfs2codecheck to identify issues in the code to be cleaned up.
- eliminated a number of memory leaks in trove dbpf.
- moved flags into top of the op structure in trove.
- implemented (but did not test) dspace_verify().

Added vim (and emacs) formatting option comments where missing in trove subdir.

Added in all my changes that weren't checked into the old CVS tree.  This code doesn't work yet.

Imported sources

Initial revision