PVFS2 pvfs-2-7-branch: Config File Description

TrustedNetwork bmi-network-address@bmi-network-mask

As an example:

TrustedNetwork tcp://192.168.4.0@24

Alias {alias string} {bmi address}

As an example:

Alias mynode1 tcp://hostname1.clustername1.domainname:12345

This option specifies a range of handle values that can be used for a particular pvfs server in a particular context (meta handles or data handles). The DataHandleRanges and MetaHandleRanges contexts should contain one or more Range options. The format is:

Range {alias} {min value1}-{max value1}[, {min value2}-{max value2},...]

Where {alias} is one of the alias strings already specified in the Aliases context.

{min value} and {max value} are positive integer values that specify the range of possible handles that can be given out for that particular host. {max value} must be less than 18446744073709551615 (UINT64_MAX).

As shown in the specified format, multiple ranges can be specified for the same alias. The format requires that max value of a given range is less than the min value of the next one, i.e. {max value1}<{min value2}

Example of a Range option for data handles:

Range mynode1 2147483651-4294967297

RootHandle {handle value}

Where {handle value} is a positive integer no greater than 18446744073709551615 (UIN64_MAX).

In general its best to let the pvfs-genconfig script specify a RootHandle value for the filesystem.

The ID value can be any positive integer, no greater than 2147483647 (INT32_MAX). It is a required option in the Filesystem context.

The value of the EventLogging option can be a comma separated list of the above values. Individual values can also be negated with a '-'. Examples of possible values are:

EventLogging flow,msgpair,io

EventLogging -storage

EventLogging -flow,-flowproto

A default value is set in the Defaults context which will be be used for all servers. However, the default value can also be overwritten by setting a separate value in the ServerOptions context using the Option tag.

NOTE: This can be overridden in the tag on a per-server basis. Look at the "Option" tag for more details Example:

StorageSpace /tmp/pvfs.storage

The TCPBufferSend and TCPBufferReceive options allows setting the tcp window sizes for the pvfs clients and servers, if using the system wide settings is unacceptable. The values should be large enough to hold the full bandwidth delay product (BDP) of the network. Note that setting these values disables tcp autotuning. See the PSC networking options for details.

Can be set in either Default or ServerOptions contexts.

bmi_tcp bmi_ib bmi_gm

For example:

BMIModules bmi_tcp,bmi_ib

Note that only the bmi modules compiled into pvfs should be specified in this list. The BMIModules option can be specified in either the Defaults or ServerOptions contexts.

flowproto_multiqueue - A flow module that handles all the possible flows, bmi->trove, trove->bmi, mem->bmi, bmi->mem. At present, this is the default and only available flow for production use.

flowproto_bmi_cache - A flow module that enables the use of the NCAC (network-centric adaptive cache) in the pvfs server. Since the NCAC is currently disable and unsupported, this module exists as a proof of concept only.

flowproto_dump_offsets - Used for debugging, this module allows the developer to see what/when flows are being posted, without making any actual BMI or TROVE requests. This should only be used if you know what you're doing.

usec: [%H:%M:%S.%U]

datetime: [%m/%d %H:%M]

thread: [%H:%M:%S.%U (%lu)]

none

The format of the option is one of the above values. For example,

LogStamp datetime

Define options that will influence the way a file-system gets exported to the rest of the world.

ReadOnly tcp://192.168.2.0@24 tcp://10.0.0.* tcp://192.168.* ...

AllSquash tcp://192.168.2.0@24 tcp://10.0.0.* tcp://192.168.* ...

AnonUID 3454 AnonGID 3454

dh - (datafile handles) This will cache the array of datafile handles for each logical file in this filesystem

md - (metafile distribution) This will cache (for each logical file) the file distribution information used to create/manage the datafiles.

de - (directory entries) This will cache the handles of the directory entries in this filesystem

st - (symlink target) This will cache the target path for the symbolic links in this filesystem

The format of this option is a comma-separated list of one or more of the above values. For example:

AttrCacheKeywords dh,md,de,st

Note that this option can be specified in either the Defaults context of the main fs.conf, or in a filesystem specific StorageHints context, but the semantics of TroveMethod in the Defaults context is different from other options. The TroveMethod in the Defaults context only specifies which method is used at server initialization. It does not specify the default TroveMethod for all the filesystems the server supports. To set the TroveMethod for a filesystem, the TroveMethod must be placed in the StorageHints context for that filesystem.

Options specified within the Defaults context are used as default values over all the pvfs2 server specific config files.

This groups options specific to a filesystem and related to the behavior of the storage system. Mostly these options are passed directly to the TROVE storage module which may or may not support them. The DBPF module (currently the only TROVE module available) supports all of them.

Global Context

Options specified within the Security context are used to dictate whether the pvfs2 servers will accept or handle file-system requests. This section is optional and does not need to be specified.

Specifies the beginning of a ExportOptions context. This groups options specific to a filesystem and related to the behavior of how it gets exported to various clients. Most of these options will affect things like what uids get translated to and so on..

This groups the Server specific options.

The ServerOptions context should be defined after the Alias mappings have been defined. The reason is that the ServerOptions context is defined in terms of the aliases defined in that context.

Default options applicable to all servers can be overridden on a per-server basis in the ServerOptions context. To illustrate: Suppose the Option name is X, its default value is Y, and one wishes to override the option for a server to Y'.

.. X Y ..

Server {server alias} .. X Y' ..

The ServerOptions context REQUIRES the Server option specify the server alias, which sets the remaining options specified in the context for that server.

This context groups together the Range options that define valid values for the data handles on a per-host basis for this filesystem.

A DataHandleRanges context is required to be present in a Filesystem context.

This context groups together the Range options that define valid values for meta handles on a per-host basis for this filesystem.

The MetaHandleRanges context is required to be present in a Filesystem context.

Provides a context for defining the filesystem's default distribution to use and the parameters to be set for that distribution.

Valid options within the Distribution context are Name, Param, and Value.

This context is an optional context within the Filesystem context. If not specified, the filesystem defaults to the simple-stripe distribution.

This groups the Alias mapping options.

The Aliases context should be defined before any FileSystem contexts are defined, as options in the FileSystem context usually need to reference the aliases defined in this context.

This groups options specific to a filesystem. A pvfs2 server may manage more than one filesystem, so a config file may have more than one Filesystem context, each defining the parameters of a different Filesystem.