CVS log for pvfs2/src/io/description/pint-request.c
	Help

Request diff between arbitrary revisions / Display revisions graphically

merge with Orange-Branch

Committed Windows client code to Orange-Branch.

Porting job files

initial merge with Orange-Branch. much will be broken

revert cu-security-branch to before the attempted merge with Orange-Branch

merged in changes from latest Orange-Branch

Setting up Orange Branch

This file has been renamed to pipeline.h.

initial support for solaris.  doesn't include /dev/epoll support.  See trac ticket #97.

*** empty log message ***

Reverse merged and ported to HEAD.

Merge HEAD changes to TAS-branch.

Allocate request state and reqstack structures in one go to save some mallocing.  Also export multi-request-state allocators to further reduce mallocs in the many-server IO case.

revert style changes back to previous versions.

run maint/pvfs2indent-80col.sh on all .c files to correct style :)

merge from trunk

fix the overflow issue with adding offsets and storing the result in an int

[pcarns]: when dealing with file sizes larger than 32 bits,
PINT_Process_request would return a negative number (> 32 bits, overflowed).
change PINT_process_requst to return 0 or -PVFS_error, and return the file size through results->bytes (which all callers use already anyway).

merging small I/O changes to trunk.  The majority of this commit includes the
new small I/O state machines on client and server, and the modifications to
the client IO state machine.  It also includes mods to the zero-fill code
to perform zero-fill of holes only if they exist instead of zero-ing the entire
memory region initially.  Also, some debug stuff has been added to the
state machine processing code to print enter/exit of state machines if sm
debug is enabled.  The genconfig now includes ioports/metaports for running
multiple servers on the same host for testing.

adding --ioports,--metaports options to genconfig that allows port ranges to be specified for same host/multiple server testing.  Fix to pint-request for non-contig zero-fill.  Automate zero-fill test.

zero-fill appears to work now (finally).

fixes to small io stuff to get test-zero-fill working again.  Also committing updates from HEAD to make things easier for merge.

committing lld/llu fixes to 1.3.1 branch

moved the llu and lld macros to an internal header so that we don't have to include pvfs2-config.h in the installed pvfs2-types.h header.

* malloc.h is obsolete in freebsd, check for it
* No error.h in freebsd
* linux allows Lu,Lx, and Ld to be interchangeable with llu,llx, and
lld.  Other platforms (solaris, bsd) don't.  Replaced all the L*
with ll*.
* bsd doesn't have mntent.h.  use fstab.h and our own funcs.
* bsd doesn't know about ssize_t.  Use size_t instead.
* bsd doesn't have asm errnos.  Check for each of those
errno macros and if not defined define to -1
* check for db3/db.h db4/db.h header if db path is specified using
--with-db
* darwin has aio support builtin, doesn't need -lrt
* remove kernel stuff in pvfs2-types.h that sets WORDSIZE.  gcc
defines __LP64__ on archs with 64bit longs so we use that instead.
* darwin's open doesn't accept O_LARGEFILE.  off_t is an int64 by
default.  define O_LARGEFILE if it hasn't been.
* need to run ranlib on darwin after libpvfs2.a gets installed to regenerate
table of contents

small I/O changes in sio branch for now

Attached patches implements trusted ports and networks similar to
what PVFS1 implemented.
It adds a new <Security> </Security> tag to the fs.conf config file
and parse utilities accordingly. We can edit these later if there
is a strong need.

Within these tags, you have to specify TrustedPorts <Range of ports> and
TrustedNetwork tcp://<network> tcp://<mask>.
I dont know how to support this cleanly on  non tcp/ip bmi
architectures, so currently this only enables it for the bmi_tcp module
alone. (i guess we could do it for bmi_ib module as well but not done
until someone says that is the right thing to do...)

Also adds support for a --trusted argument to the pvfs2-genconfig script
that will prompt for the trusted port ranges and the network address/masks and write it out
to the fs.conf and server.conf file.

Configure now supports a --enable-trusted to enable all these features.
(NOTE that the fs.conf tags (or lack of one) still allow you to not
select these features even if configure was asked to compile with this
option).

this patch also cleans up all the warnings that arises with gcc
(3.4) and --enable-strict build mode.

replaced zero-fill code in client io state machine with one that works.  We:

* zero the buffer according to the memory request
* do flow
* calculate return size:
 - check that the ub of the file request is within file data (not a hole or past eof) -> return_size = sizeof(filereq)
 - if that fails, search through retreived datafiles for offset > ub_file_req -> return_size = sizeof(filereq)
 - if that fails, get all the datafiles and search through them for offset > ub_file_req -> return_size = sizeof(filereq)
 - if that fails, calculate correct return size from file req and eof
  * while(next_contig_segment(file_req).ub < eof)
      return_size += sizeof(current_contig_segment(file_req))
    if(current_contig_segment(file_req).lb < eof)
      return_size += (eof - current_contig_segment(file_req).lb)

---

* Added a test for holes and appropriate zero-fill behavior
* Also fixed error reporting for ping to display individual server errors.
* Fixed a bug in PINT_dist_copy

updated to handle request type offset correctly

fixed a problem causing memtypes and non-contig requests to fail on the client
added several debug tests to verify correct operation of this

updates to my event changes to bring them inline with trunk

Updating request and distribution code to allow for secondary data servers.

Also converted naming of functions in pint-request.h to the PVFS2 standard.

fixed problems with nexted requests and added support for setting extent

added checks for a few pointers to make sure they are valid

64-bit -1 is different than 32-bit -1; fix ia64 server crash

fixed problem with nested offset calculation

updated PVFS_Request to work correctly with resized requests.  Added
resized constructor, changed logic to use both extent and
num_contig_chunks to determine contiguous types.  Updated tests of
request code.

squeeze multi-line debug messages

consolidate some multi-line debug messages

added code to PINT_Request_do_commit to restore the commit field
to an input request that is already committed, so that committed
requests that are for some reason recommited, don't get uncommited.

Distribution cleanup and refactoring.  All naming is consistent with PVFS2
naming schemes.  Added new methods PINT_dist_methods to enable distribution
to determine the number of datafiles to use.  Removed unused methods (encode
and decode) from distributions (encode_lebf and decode_lebf are the preferred
mechanism).

- rename all gossip masks to GOSSIP_xxx
- some misc cleanups

some 64 bit fixes; maxel has to be big enough to handle > 2G sizes now
that we use tiled requests

Changed iod_num and iod_count to server_nr and server_ct.

add prefix to otherwise confusing debug message

interesting 64-bit related changes

remove remnant of first fix

make messages match function name

fixed bug with mem types - passing out the size of the chunk processed

fixed problem when no mem type given in client mode

working on stupid commit thing

fixed bug with auto-set final size

scattered bug fixes to get rid of warnings (getopt returns an int).  ppc32
platform compiles mostly clean (aside from some parser/scanner.c warnings)
now.  Things seem to work well too, at least locally.

fixed stupi off ty one error

 fixed a bug in client mode that screws up segment combining when it
 happens repeatedly in the same segment.

fixed encode/decode bug - hopfully last commit indicator approach!

reworked commited indicator and tweeked freeing code to get it working

fixed an oops

fixed bug in commit routine

 updated commit code - works better now, but has another recursive
 routine in it - clean that up another day

integrated some request processing updates from walt, mostly to make
memory datatypes work better.  Updated I/O path to expect memory datatypes
and updated all PVFS_sys_read and PVFS_sys_write calls accordingly.
File datatype tiling still isn't quite right, so for now the file datatype
must be large enough to cover whatever it is you are trying to access
in one shot.  Haven't really tested any memory datatypes more
complicated than contiguous with various file datatype offsets- need to
try some fancier stuff...

next version of request processor
removed start_offset and all code referring to it
moved eof_flag from result to request_state struct

- merged in update request processing code from Walt, and did the bare
minimum of PVFS_sys_io and flowproto updates to support it
- more stuff can still be cleaned up, and we need to add hooks to get
to memory datatypes
- re-enabled compilation of request processing test programs

fixed memory problem related to change in interface

Aieeee....

New datatype request processing interface clumsily merged into tree.  For the
most part, things will work as before, but still need to:
- fix bug in precalculation when datatype offset is used
- fix data corruption when running multiple servers
- update flowproto-dump-offsets and enable again
- take out old #ifdef0'd code in bmi trove flowproto
- clean up flowproto's use of new interface

added fix so logicals kip works in client mode too.

hopefully updates reported before are now correct!

added new mode to request processor to skip logical bytes
cleaned up mode code

fixed bug in seek - when seeking to blank area in non-contiguous
request generated bad things - got a negative value for retval.
added a test to fix this.

added a test at the end of the distribute function so that
only if at end of file and NOT extend flag do we return the original
size - signalling completion of the segment

got rid of PVFS_count32 type; replaced with int32_t or uint32_t or int
where appropriate

got rid of PVFS_flag type and bmi_flag_t.  need to come back later and
convert the buffer_flag variables in BMI to named enumerations; they are
ints for now

fixed bug where finished is not signaled when request past EOF has
requested bytes set to the exact number available

fixed bug related to detecting eof in distribute routine

renamed request processing mode from CKSZ2 to CKSIZE_MODIFY_OFFSET

added CKSZ2 mode - does check size without state copy and restore

fixed an error when segs exceed segmax

another bug related to offsets on initial request records

 fixed a problem with not reporting end of request correctly - hope it
 didn't add new problems.

fixed bug when first segment has non-zero offset

fixed a little pointer warning
added logical_file_size method to distributions

working on EOF issues in request processor

fixed some warnings - changed order of args for request commit

brought Walt's io description code over from the old pvfs2 tree