XRootD Configuration Reference
28-April-2020
Release 5.0.0 and above
Andrew Hanushevsky
©2004-2020 by the Board of Trustees of the Leland Stanford, Jr., University
All Rights Reserved
Produced under contract DE-AC02-76-SFO0515 with the Department of Energy
This code is open-sourced under a GNU Lesser General Public license.
For LGPL terms and conditions see http://www.gnu.org/licenses/
1.2 Starting the xrootd Daemon
1.2.1 Multiple Instances and Automatic Fencing
1.2.2 Passing Plug-In Command Line Arguments
1.2.4.1 Environmental Information File
1.2.5 Exported Environment Variables
2 Framework Directives by Category
3 Common Framework Configuration Directives
3.1.1 Administrative Interface
4 Esoteric Framework Configuration Directives
4.2.1 Dual Public/Private Network Guidelines
4.2.2 Resolving Private IP Addresses
5 xrootd Directives by Category
6 Common xrootd Configuration Directives
7 Esoteric xrootd Configuration Directives
7.3.1 Authorizing digFS Access
7.3.2 Optional digFS Directives
8.1.1 Backward Compatibility and Overrides
8.2 Directives to Enhance HTTPS Access
8.2.7 Deprecated HTTPS Directives
This document describes the eXtended Request Daemon (xrd) configuration directives protocols that can be used with xrd: cmsd, HTTP, and XRootd. It also includes the directives for the xrootd daemon that can run xroot and HTTP protocols. The cmsd-specific directives are described in a separate reference manual.
The xrd is a framework that can dynamically support multiple TCP/IP application service layer protocols. It is designed to provide a high performance environment for application services. The xrd is a generalized framework and it makes its primary decision on which protocol to support based on the name given to the executable. Currently, the following executable names are fully supported:
· cmsd daemon for the cms server clustering protocol, and
· xrootd daemon for xroot and other related protocols.
Configuration directives come from a configuration file. Directives are prefixed by the component acronym they apply to, as shown in the following table. This makes creating a single configuration file for all services possible.
Component |
Purpose |
acc |
Access control (i.e., authorization) |
cms |
Cluster Management Services |
frm |
File Residency Manager |
ofs |
Open File System |
oss |
Open Storage System (i.e., file system implementation) |
pfc |
Proxy File Cache |
pss |
Proxy Storage Service |
sec |
Security authentication |
xrd |
Extended Request Daemon |
xrootd |
The xroot protocol implementation. |
http |
The HTTP protocol implementation. |
all |
Applies the directive to all of the above components. |
Records that do not start with a recognized identifier are ignored. This includes blank record and comment lines (i.e., lines starting with a pound sign, #). This guide documents the all, http, xrd, and xrootd configuration directives (i.e., the un-shaded rows). Other directives are documented in supplemental references specific to the component they deal with.
The location of the configuration file is specified on the xrootd command line. Refer to the reference manuals for other components on how they locate their respective configuration files.
Refer to the manual “Configuration File Syntax” on how to specify and use conditional directives and set variables. These features are indispensable for complex configuration files usually encountered in large installations.
The xrd framework relies on the loaded protocol(s) for strong authentication (e.g., Kerberos, GSI, etc.). Therefore, security is a protocol issue. The xroot protocol provide strong authentication should you choose to use it. Refer to each protocol on how to configure strong authentication.
The xrd framework does provide host-based authentication. While this type of authentication can be subverted in a number of ways, it still is a practical mechanism for installations that do not need strong authentication. The allow directive can be used to restrict the range of hosts that can connect to the daemon. This security can be used together with any protocol-provided security.
Because the xrd framework does not intrinsically provide strong authentication; you should not run xrootd as super-user (i.e., Unix root). Any attempt to do so without indicating that you really want to run super-user (see the -R command line option) will cause the program to exit.
Use the following command to start the xrd-based xrootd daemon:
xrootd [ options ] [ path [ path [ . . . ] ] [piargs]
options: [-c cfn] [-l largs]
[-k {num | sz{k|m|g} | sig}] [esoteric]
esoteric: [{-a | -A} apath] [-b] [-d] [-h] [-I {v4 | v6}]
[-L protlib] [-n name] [-[-p {port | any}]
[-P protocol] [-R user] [-s pfn] [-S site]
[{-w | -W} hpath] [-z]
largs: [=]fn | - |
@lib[,bsz=sz][,cse={0|1|2}][,logfn=[=]fn]
sig: fifo|hup|rtmin|rtmin+1|rtmin+2|ttou|winch|xfsz
piargs: -+[tag] [args] [piargs]
Parameters
path An absolute file system path prefix. All requests will be restricted to files with this prefix. You may specify any number of path prefixes. If no path is specified, operations will be restricted to paths starting with /tmp.
Options
-c fn The name of the configuration file. If one is not specified, no configuration file is processed.
-l [=] fn
Specified how messages are to be handled. Options are:
fn Directs messages and any trace output to the indicated file, fn, possibly qualified by the instance name (see the fencing section). If fn is a dash (-), output is sent to standard error; the default.
=fn Same as fn but the fn is not qualified by the instance name, if any. This allows log files to be handled in an arbitrary manual way. For more information see the section on fencing.
@lib Directs messages to a plug-in that is defined in the shared library specified by lib (see the section on log file plug-ins). Additional comma-separated parameters may follow lib, as follows:
bsz=sz Specifies the size of the speed matching buffer. The default is 64K. Messages are placed in the buffer and then forwarded to the plug-in as time permits. A value of 0 disables speed matching and messages are handed off to the plug-in as they occur. See the section on log file plug-ins for more information. A positive value less than 8K is forced to be 8K. The maximum allowed in one megabyte. The sz may be suffixed by k or m to indicate kilobytes or megabyte, respectively.
cse={0|1|2} Specifies how standard error output should be handled:
0 Does not capture standard error output. All such output is sent to the logfn destination, if specified, or is otherwise lost. This is the default.
1 Captures standard error but only forwards it to the logging plug-in if it starts with a standard time stamp. This option may cause an infinite loop. Refer to the logging plug-in section for more information.
2 Captures standard error output and forwards it to the logging plug-in without inspection. Refer to the logging plug-in section for more information.
logfn=[=]fn Specifies that messages are also to be routed to a local log file. The parameter is identical to that described above. To use standard error, specify a dash (-) for fn.
-k num | sz{k|m|g} | sig
Keep no more than num old log files. If sz is specified, the number of log files kept (excluding the current log file) is trimmed to not exceed sz bytes. The sz must be suffixed by k, m, or g to indicate kilobytes, megabyte, or gigbytes, respectively. If a sig value is specified (i.e. hup etc), then an external program is expected to handle log file rotation (e.g. logrotate). Except for fifo, the argument specifies signal that causes the daemon to close and re-open the log file to allow rotation to occur. When fifo is specified, the daemon waits for data to appear on a fifo whose path is identical to the log file path but whose name is prefixed by a dot. Refer to the notes for manual rotation caveats.
Esoteric Options
{-a | -A} apath
Specifies the default administrative path and can be overridden by the adminpath directive in the configuration file. When -A is specified group write access is allowed (see the adminpath directive group option for details).
-b Runs the program in the background. You should also specify -l.
-d Turns on debugging. Warning! This severely impacts performance.
-h Displays help information.
-I {v4 | v6}
Restricts the server’s internet address protocol. When v4 is specified, only hosts with IPV4 addresses can connect or be connected to. When v6 is specified, the default, hosts using IPV6 or IPV4 addresses can connect or be connected to. This option is only useful for systems that have misbehaving IPV6 network stacks. The default is established by the network interface configuration on the machine at the time the program starts.
-L protlib
Specifies the shared library that holds the implementation of the default protocol specified by the –P option.
-n name
Assigns name to the xrootd instance. By default, the xrootd instance is unnamed. See the notes on how to use this option.
-p port
The TCP port, or service name associated with a port, that xrootd is use for new connections. The default is “xrootd” or port 1094, if the TCP service xrootd cannot be found /etc/services.
-p any
Uses any available port.’
-P protocol
The name of the default protocol. Use this option when the name of the executable differs from the name of the default protocol. You may need to specify the –L option as well. See the notes for more information.
-R user
The user name or numeric uid of the user whose effective identity is to be assumed. See the usage notes for more information. The specified user may not have super-user privileges. This option may not be specified unless the program is running as super-user.
-s pfn Specifies the name of the file that is to hold the process id upon start-up.
-S site Specifies a 1- to 15-character site name that is to be included in monitoring records. The name may only contain letters, digits and the symbols “_-:.”; any other characters are converted to a period.
{-w | -W} hpath
Specifies the default home path; i.e. the current working directory during execution. If it is not specified on the command line, it can be specified by the homepath directive in the configuration file. When -W is specified group read access is allowed (see the homepath directive group option for details). The hpath is extended by any specified instance name (i.e. –n option). The path is created should it not exist.
-z provides microsecond resolution for log file message timestamps.
-+ provides a mechanism to pass command line arguments to plug-ins. See the section “Passing Plug-In Command Line Arguments” for more information.
Defaults Arguments
–l - –I v6 –p xrootd –P executable_name /tmp
General Notes
1) For security purposes, only files in /tmp are allowed to be accessed unless you specify otherwise. You may specify other paths either on the command line or using the xrootd export configuration directive.
2) Do not prefix any export path with the oss localroot directive path, if any.
3) If a log file is specified without a signal -k option, the file is closed at midnight, renamed to have a date suffix (i.e., fn.yyyymmdd) and possible sequence number (i.e. fn.yyyymmdd.n), and a new log file is opened.When a signal value is specified, log files are not automatically renamed at midnight. Instead an external program must be used to properly rotate log files. Make sure to choose a signal that is not in use by any plug-in. If unsure, choose one of the obscure signal names and monitor for any odd behavior. Otherwise, use the fifo option. Be aware that on some non-Linux platforms the fifo file descriptor may leak.
4) When fifo is specified the fifo file name must not exists or exist as a fifo file. A simple “echo x >> /path/.lfn” causes the logfile to close and reopen.
5) The sig names, except for fifo, be fully capitalized as well prefixed by “sig” or “SIG” when capitalized.
Notes on Esoteric Options
1) The default port service name, default protocol, and pidfile name is normally determined by the prefix-name of the executable. The prefix-name is defined to be all of the characters in the base filename (i.e., the directory path removed) up to but not including the first dot in the name, if any. If the name starts with a dot, the prefix-name is the complete base filename.
2) The way the prefix-name is derived allows you to maintain several versions of a particular xrd executable (e.g., xrootd and xrootd.debug) without changing the intrinsic way default names (e.g, protocol) are determined.
3) You must use the –P option to set the default protocol name as well as other related naming aspects (e.g., port service name) when the name of the executable does not correspond to the name of the default protocol.
4) The –n option allows you to run multiple instances of the xrootd on the same machine. See the next section on instances and fencing.
5) Use -p any for protocols that manage their own port numbers. This is the case for redirection target xrootd/cmsd combinations. Only the initial point of contact needs a well-known port number. All other connections between clients and servers are routed using whatever port numbers are currently in effect. This allows you to keep a simple configuration file for servers and to run more than one server on the same machine without worrying about conflicting port numbers.
6) The –b option forces the program into the background. If –l is not specified; all output messages are discarded.
7) When –b is specified, the program fails if it cannot write the pid file.
8) The -R option allows the program to run under the super user’s account. This is allowed because the effective user is set to specified user and the effective group to user’s primary group. Thus, the program is not effectively running as super-user. However, the real and saved user ids may still be “root”, depending on how the program was started.
9) The -R option provides a minimal increase in security since it is possible for a loaded protocol to switch back to super user mode. You should not use the -R option unless absolutely necessary.
10) The -b, -p, and -s command line options are meant to be used by start-up scripts (e.g. init.d).
11) Warning: Command line options, except for –a and -s, over-ride corresponding configuration file directives. For -s the pid file is written to the desired location in addition to the location specified by the pidpath directive.
Example
xrootd –c /opt/xrootd/xrootd.cf
You can run multiple instances of xrootd on the same physical machine. This is useful when you want to overlay more than one cluster on top of a file system (e.g. production and test). In order to prevent instances from interfering with each other, you must provide each xrootd that is running on the same hardware a unique instance name. One of the daemons need not have an instance name as it assumes the name “anon” (i.e. anonymous).
Once an instance name is assigned to a daemon using the –n option, the system automatically fences in the daemon so that it does not interfere with any other xrootd processes running with it. Automatic fencing consists of threse actions:
· The instance name is suffixed to the adminpath to create a unique location for temporary server files. For instance, if –n is not specified, xrootd creates /tmp/.xrootd/admin as the path for the administrative interface. If “-n test” is specified, xrootd creates /tmp/test/.xrootd/admin instead. Even the path specified with the adminpath configuration directive is modified.
· The instance name is used to create a new directory in the current working directory. The current working directory is changed to this newly created path. So, if “/home/xrootd” is the current working directory and “-n test” is specified; the current working directory becomes “/home/xrootd/test”. This allows core files to be segregated by instance name.
· The instance name is automatically inserted into the log file path specified via the –l command line directive to create a unique location for server log files files. For instance, if “–l /var/adm/xrootd/xrdlog” is specified along with “-n test”, xrootd modifies the –l argument to be /var/adm/xrootd/test/xrdlog.
Automatic fencing of log files may, for some installations, run counter to the way log files are commonly handled. You can disable fencing of log files by prefix the log file path by an equals sign. However, you are then responsible to make sure that each instance uses a different log file path or name.
Much of this functionality has been subsumed into containerized frameworks. However, the functionality described here is still supported in the cases where containerization does not provide the desired network performance and running host-based applications becomes an imperative.
You can pass command line arguments to various plug-ins that support the feature using the -+ option. The option must be specified after all XRootD options and parameters as plug-in arguments are stripped from the command line. For example,
xrootd [ options ] [ path [ path [ . . . ] ] -+mypi arg1 arg2 -+urpi arg3
places a pointer to the vector containing “argv[0] arg1 arg2 0” into the internal environment passed to the plug-in using the variable “mypi.argv**” and the count in “mypi.argc” (i.e. 3). Similarly, “argv[0] arg3 0” are pointed to by “urpi.argv**” with the count in “urpi.argc” (i.e. 2). If no tag is specified, the leading prefix is missing (i.e. the variables are “.argv**” and “.argc”). Note that argv[0] is the actual value of argv[0] (i.e. the executable name) passed to XRootD.
It is up to the plug-in to extract the appropriate arguments using a documented tag. Tag values that start with “xrd” should not be used as these are reserved for plug-ins normally distributed with the XRootD package.
XRootD allows you to specify a plug-in to handle messages that would otherwise be sent to a regular file or standard error. You do this using the ‘@’ qualifier with the –l option. Logging messages is a critical function in the server and any delay will severely impact server performance. The default logging path is very efficient and any plug-in placed in the path should be just as efficient. To help, a speed matching buffer is used to minimize plug-in vagaries. However, if you choose to not use a speed matching buffer (i.e. a bsz of zero for synchronous operation) then the plug-in becomes the choke point in server performance.
You may also choose to capture standard error output using the cse parameter. However, this option will result in an infinite loop if your logging plug-in writes to standard error for any reason. This may be mitigated by specifying cse=1 which only sends standard error output to the plug-in if it starts with a timestamp of the form “yymmdd hh:mm:ss”. All debugging output starts with such a timestamp.
The
details on how you write a log file plug-in is detailed in the XrdSysLogPI.hh
header file. It is important to realize that if you use the XrdSysLogger
object to route a message from your plug-in, an infinite loop will result.
Additionally, one log file plug-in is used to all XrdSysLogger instances.
The following directories and files are created by xrootd:
Default File |
Changed by |
Contents |
<stderr> |
-l and -n command line options |
Informational and error messages |
/tmp//[name/].xrootd/ |
-n command line option and the adminpath directive |
Directory for various server-related files. |
<cwd>//[name/]core[.pid] |
-n command line options |
Core file |
/tmp/[name/]exec.pid |
pidpath and –n option |
Holds the process id |
/tmp/exec.name.env |
adminpath and –n option |
Holds environmental information (see next section). |
The daemon writes environmental information in the directory specified by adminpath directive, or its default. This information can be used to automatically collect all relevant information about a daemon to facilitate automatic problem resolution.
The environmental file is named “exec.name.env” where exec is the executable’s name and name is the instance name (i.e. -n option) and anon if no instance name was specified. The format of the information is shown below. When parsing this information, you should not depend on the order shown below.
pid=pid&host=host&inst=inst &ver=ver&home=hpath &cfgfn=cfgfn&cwd=cwd&logfn=logfn
Parameters
cfgfn The configuration file used.
cwd The current working directory.
host The host name.
hpath The current working directory.
inst The instance name.
logfn The log file being used.
pid The process id.
ver The version string
These are the minimal elements. Additional elements may be added by specific protocols.
The following table shows the environment variable exported by xrootd. These may be used by external programs and plug-ins, as needed. They should never be modified.
XRD Variable |
Contents |
XRDADMINPATH |
Is the directory for administrative files (i.e. all.adminpath) |
XRDCONFIGFN |
The effective administrative path used for server management files. |
XRDDEBUG |
Set to one when the –d command line option is specified. |
XRDHOST |
The current host’s DNS name. |
XRDINSTANCE |
Is the string of the form “execname instance@hostname”. Where execname is the executable’s name, instance is the name specified via –n or anon if no instance name was specified, and hostname is the current host’s DNS name. |
XRDLOGDIR |
Is the directory where log files are written. |
XRDNAME |
The name specified via –n or anon if no instance name was specified. |
XRDPROG |
The executable’s name. |
XRDSITE |
The site name specified either via the –s command line option or the all.sitename directive. |
If the standard cms client plug-in is being used, the following additional environment variables are exported.
CMS Variable |
Contents |
XRDCMSMAN |
The space separated list of managers for this host each in the form of “host:port”. |
XRDCMSCLUSTERID |
The globally unique cluster identification for this host. |
If the standard ofs plug-in is being used, the following additional environment variables are exported.
OFS Variable |
Contents |
XRDROLE |
The effective value specified on the all.role directive. |
XRDTPC |
Is set when TPC (Third Party Copy) has been configured and represents the version number of the protocol. If the first character is a plus sign, authentication is required. |
If the standard oss plug-in is being used, the following additional environment variables are exported.
OSS Variable |
Contents |
XRDN2NLIB |
The path and name of the name-to plug-in, if specified via the oss.namelib directive. |
XRDRMTROOT |
The local root path specified by the oss.remoteroot directive. |
XRDLCLROOT |
The local root path specified by the oss.localroot directive. |
XRDOSSQUOTAFILE |
The name and location of the file handling disk space quotas. |
XRDOSSUSAGEFILE |
The name and location of the file handling disk space usage. |
If the standard xrootd protocol plug-in is being used, the following additional environment variables are exported.
XROOTD Variable |
Contents |
XRDOFSLIB |
The path and name of the OFS plug-in specified by the xrootd.fslib directive. |
XRDMONRDR |
If monitoring is enabled, how often server identification records are sent. |
This section provides a guide to xrd directives by category that is helpful when you need to address a specific requirement.
xrd.trace Specify which framework activities are to be traced.
xrd.report Specify which execution summary statistics are to be gathered and where they are to be sent.
all.sitename Specify the name of the site to be used for monitoring purposes.
xrd.tcpmonlib Specify the plug-in to be used to collect and report specialized TCP connection statistics.
xrd.network Specify network parameters such as DNS usage, interfaces, keep-alive characteristics, and routing.
all.adminpath Specifies the location of runtime files used for administrative purposes.
xrd.homepath Specifies he location of the working directory during execution.
all.pidpath Specifies where the file containing the server’s process id should be created.
xrd.port Specifies the default port number for incoming requests.
xrd.protocol Load additional protocols such as HTTP.
xrd.allow Restricts hosts that can connect to the server.
xrd.tls Specify the location of the server’s host certificate.
xrd.tlsca Specify the location of CA certificates and CRLs.
xrd.tlsciphers Specify allowable TLS ciphers.
xrd.buffers Limit the amount of memory used for data buffers.
xrd.sched Specify execution parameters such as core file creation, default stack size, and threading.
xrd.timeout Specify various connection handling timeout parameters
all.adminpath path [ group ]
Function
Specify the location of protocol-specific files for administrative purposes.
Parameters
path The absolute path to a directory that is to hold protocol-specific files. The path should be no longer than 76-characters. See the notes for details.
group
Allows read/write group access to any named sockets created in the path. By default, only the owner can use such sockets.
Default (see warning in the notes)
/tmp
Notes
1) The adminpath directive allows you to specify the location of the local TCP sockets used for the command-line administrative functions along with special directories that are needed to handle protocol specific features.
2) Use the -a or -A command line option to set the defaults for the adminpath directive.
3) If -n is specified on the command line, a subdirectory corresponding to the instance name (i.e., -n argument) is created in path, if one does not exist. This becomes the new path. This allows fencing multiple daemons running on the same node.
4) Warning: if idle /tmp directories and socket files are automatically deleted by the system, you should specify a path other than /tmp. Be aware if neither –a command line option nor the adminpath directive is specified, the default becomes /tmp. External removal of files stored in the adminpath may lead to server failure.
5) When specifying an adminpath; be cognizant that the program may be containerized and the path may not be suitable for that environment. Confer with your system administrators on what the suitable path should be if the program will run in a container.
6) Local TCP socket names are limited to 108 characters. Up to 32 characters are needed to define actual socket files; leaving 76 characters that may be specified for the path.
7) The following steps are taken when creating a Unix named socket in path:
· The subdirectory .xrd is created in path and
o the cmsd daemon creates subdirectory .olb while
o the xrootd daemon creates subdirectory .xrootd in path.
· Mode bits for these directories are set to 0700 (rwx for owner). If the directories already exist, the mode settings are reset to correspond the to the adminpath directive.
· If group was specified, the mode setting is extended to 0770 (rwx for owner and group).
· If the directories already exist, the mode settings are reset to correspond the to the adminpath directive specification.
· Server, manager, and supervisor cmsd’s create stream sockets named “olbd.admin”, “olbd.nimda”, and “olbd.super” in the .olb subdirectory, respectively. These sockets are used for administrative communication.
· Server and manager cmsd’s respectively create datagram socket named “olbd.notes” and “olbd.seton” in the .olb subdirectory. These sockets are used for external notifications.
· The xrootd daemon creates a Unix named socket with the name “admin” in the .xrootd subdirectory.
8) The adminpath value is passed to all protocols so that they can create their respective administrative files in path. However, for the cmsd only, you may specify an exception by using the cms.adminpath directive to create a completely different path for its files.
9) Refer to the section “Administrative Interface” for details on how to use the Unix named socket created by various protocols and daemons.
1)
Example
all.adminpath /var/adm/xrd group
The adminpath directive is used to construct the path where local TCP sockets, called named sockets, are created. These sockets are used to communicate requests and receive responses via the administrative interface. Unix domain sockets function identically as INET domain sockets. The only differences are in how the socket is created and the domain in which it operates. Care should also be given as to who creates the socket. Following the next steps will allow the successful use of the administrative interface.
The administrative protocol used for the socket interface is defined elsewhere.
xrd.allow { host | netgroup } name
Function
Restrict the hosts that can connect to xrootd.
Parameters
host name
The DNS host name allowed to connect to xrootd. Substitute for name a host name or IP address. A host name may contain a single asterisk anywhere in the name. This lets you allow a range of hosts should the names follow a regular pattern. . IP addresses may be specified in IPV4 format (i.e. “a.b.c.d”) or in IPV6 format (i.e. “[x:x:x:x:x:x]”).
netgroup name
The NIS netgroup allowed to connect to xrootd. Substitute for name a valid NIS netgroup. Only hosts that are members of the specified netgroup are allowed to connect to xrootd.
Defaults
None. If allow is not specified, any host is allowed to connect.
Notes
1) You may specify any number of hosts and netgroups. Any host matching a specified name or is a member of a specified netgroup is allowed to connect to xrootd.
2) Warning! Using hostname based security relies on the security of the DNS server and the inability of other hosts spoofing and successfully using the “allowed” IP addresses. The two security assumptions have severe limitations.
Example
xrd.allow host objyana*.slac.stanford.edu
xrd.homepath path [ group ]
Function
Specify the location of the current working directory.
Parameters
path The absolute path to a directory that is to be used as the working directory.
group
Allows read group access to path. B
Default
Whatever is the directory on start-up.
Notes
1) Use the -w or -W command line option to set the home path. When the home path is set on the command line, it cannot be overridden with the homepath directive.
2) If -n is specified on the command line, a subdirectory corresponding to the instance name (i.e., -n argument) is created in path, if one does not exist. This becomes the new path. This allows fencing multiple daemons running on the same node.
3) When specifying a homepath; be cognizant that the program may be containerized and the path may not be suitable for that environment. Confer with your system administrators on what the suitable path should be if the program will run in a container.
Example
all.homepath /var/run/xrd group
xrd.tls cpath [ kpath ] [ options ]
options: [[no]detail] [hsto to{h|m|s]
Function
Configure transport layer security (TLS).
Parameters
cpath Specifies the absolute path to the x509 certificate file to use for TLS. The certificate must be in PEM format. The file may only be written by the owner of the file.
kpath Specifies the absolute path to the certificate’s x509 private key file to use for TLS. The key must be in PEM format. The file may only be read and written by the owner of the file. If kpath is not specified then the certificate file must contain the key.
[no]detail
When detail is specified, detailed TLS trace back messages are printed along with explanatory messages. The nodetail option suppresses the TLS trace back messages. The default is nodetail. See the notes why this is so.
to Specifies the maximum amount of time a TLS handshake is allowed to take before the connection is closed. The to value may be suffixed by h for hours, m for minutes, or s for seconds, respectively; otherwise, the to value defaults to seconds. There is no default time limit for the TLS handshake.
Defaults
TLS is not configured and cannot be used. See individual options for the defaults should you configure TLS.
Notes
1) Normally, a host certificate should be used because the client can use it to validate that it connected to the intended host.
2) If you specify the tls directive then you must specify the tlsca directive as well.
3) Most TLS trace back messages do not provide any more information than the companion explanatory messages and, as such, is only useful for debugging purposes. This is why nodetail is the default. However, if you enable TLS tracing using the xrd.trace directive, detail is enabled regardless of what is specified.
Example
xrd.tls /etc/security/xrootd/hostcert.pem
xrd.tlsca noverify | {certdir | certfile} path [options]
options: [crlcheck {all | external | last}]
[log {failure | off}] [[no]proxies]
[refresh rint[h|m|s]] [verdepth vdn]
Function
Configure client certificate verification for transport layer security (TLS).
Parameters
noverify
Disables client certificate verification. All subsequent parameters, if any, are ignored.
certdir path
Specifies the absolute path of the directory containing trusted Certificate Authority certificates that can be used to verify client certificates. Each file in the directory may only contain a single certificate in PEM format. Naming conventions are those required by the version of OpenSSL being used. The directory may only be written to by the owner of the directory.
certfile path
Specifies the absolute path to the file containing one or more trusted Certificate Authority certificates that can be used to verify client certificates. The certificates in the specified file are used first before an attempt is made to find an appropriate certificate in certdir, if specified. The file must be in PEM format. The file may only be written to by the owner of the file.
crlcheck
Specifies the certificate revocation list (crl) is to be handled. Choose one of the following:
all - apply crl checking to the complete certificate chain.
external - crl application is handled by an external plug-in (the default).
last - apply crl checking only to the last certificate in the chain.
log Specifies logging requirements. Logging messages are written to the log file. Choose one of the following:
failure - log failed verifications (the default).
off - verification failures are not to be logged.
vdn Specifies the verification depth. Should the client present a certificate chain, up to the last vdn certificates are verified. Specify a value between 1 and 256, inclusive. The default is 9.
rint Specifies the refresh interval. Suffix the value with h for hours, m for minutes, or s for seconds (the default). The default is 8h (eight hours).
Defaults
xrd.tlsca crlcheck external log failure proxies refresh 8h verdepth 9
Notes
1) You may specify both a directory and a file. The certificates in the certfile will be searched before any certificates in certdir.
2) If all the loaded protocols use strong authentication (e.g. Kerberos, GSI, or SSS) client certificate verification is not necessary as the client will be verified using a strong authentication mechanism and the client’s certificate will only be used to establish a TLS connection.
3) Certain protocols require certificate verification (e.g. HTTPS). If you enable one of these protocols you should enable verification overall to avoid specifying protocol specific directives that duplicate ones that could be specified using the tlsca directive.
4) In the absence of strong authentication, you should always verify client certificates. Generally, you should always verify client certificates. This is why the directive requires that you make an explicit choice.
5) All of the certificates in the directory, as well as the file, must in a format that is recognized by the version of OpenSSL being used.
6) If you use a certdir be aware that OpenSSL requires that the c_rehash utility be run after the certificates in the directory are updated. This introduces a race condition between refreshes and updates to the directory and may produce verification failures should a refresh occur while the directory is being updated. You can avoid this problem by making sure that changes to the directory are visibly done in an atomic fashion.
Example
xrd.tlsca certfile /etc/security/xrootd/cacerts.pem
xrd.buffers memsz[k | m | g] [rint[m | s | h]]
Function
Limits the amount of memory to be used to data buffers.
Parameters
memsz
The maximum number of bytes to be used for data buffers. The memsz can be suffixed by k, m, or g to indicate kilo-, mega-, or giga-bytes; respectively. The default is to use up to 12.5% (one-eight) of the configured memory of the machine.
rint The interval between buffer pool readjustments. Specify a number, optionally suffixed by m for minutes, s for seconds (the default), or h for hours. The default is every 20 minutes.
Defaults
xrd.buffers memsz 20m
Notes
1) The allotted memory for buffers is independent of any other memory allotment to the daemon.
2) Data buffers in the pool are periodically readjusted to reflect the actual working needs of the daemon. The rint interval controls how frequently this adjustment occurs. The default value is usually the best value.
Example
xrd.buffers 512M
xrd.network [buffsz blen[k | m | g]] [cache sec]
[[no]dnr] [[no]dyndns]
[kaparms idle[,itvl[,cnt]] [[no]keepalive]
[routes {split|common|local} [use if1[,if2]]]
[[no]rpipa] [tls]
Function
Specify network parameters.
Parameters
buffsz blen
The buffer size to be set for each connected socket. The blen can be suffixed by k, m, or g to indicate kilo-, mega-, or giga-bytes; respectively. The default is determined by the operating system.
cache sec
The maximum number of seconds that an address to hostname translation can be locally cached for future use. See the notes on its interaction with dyndns. The sec can be suffixed by s, m, h or d to indicate seconds, minutes, or hours, or days; respectively. The default is 3 hours.
dnr Uses Domain Name Resolution to convert IP addresses to host name for connecting clients. Host names are displayed in various messages.
nodnr Avoids using Domain Name Resolution to convert IP addresses to host name for connecting clients. Client IP addresses are displayed in various messages.
dyndns
The network uses a Dynamic DNS for name resolution. See the section on Dynamic DNS for information on when you should specify this option. . See the notes on its interaction with cache.
nodyndns
The network uses a standard DNS whose name entries are stable. This is the default.
kaparms idle[,itvl[,cnt]
Specifies TCP keepalive parameters. The kaparms option is only effective for Linux. Up to three parameters may be specified. Omitted parameters, as well as parameter values of zero, use the system default. The parameters are:
idle The time the connection needs to remain idle before TCP starts sending keepalive probes. The idle value may be optionally suffixed by m for minutes, s for seconds (the default), or h for hours.
itvl The time between individual keepalive probes. The itvl value may be optionally suffixed by m for minutes, s for seconds (the default), or h for hours.
cnt The maximum number of keepalive probes TCP should send before dropping the connection.
keepalive
Uses the operating system’s keep-alive mechanism to determine whether or not a client is still connected to the daemon. This is the default. See the usage notes for other ways of simulating keepalive.
nokeepalive
Does not use the operating system’s keep-alive mechanism to determine whether or not a client is still connected to the daemon.
routes
Specifies that a dual network exists and how public and private addresses are routed within a site. Select one of the below optionally followed by the use keyword and up to two interface names (i.e. if1 and optionally if2 immediately preceded by a comma; use ifconfig to display interfaces and their names):
split Two separate networks exist. Clients connecting with a private address can only be redirected to a server’s private address. Clients connecting with a public address can only be redirected to a server’s public address. If clients can use both types of addresses, all servers must be dual homed with a public and private address. Typically, you must specify the interfaces you are using.
common Two common networks exist. Clients connecting with a private address are preferentially redirected to a server’s private address but may be redirected to a server’s public address if need be. However, clients connecting with a public address can only be redirected to a server’s public address. All servers must have at least a public address. If the machine is dual-homed you must specify the interfaces you are using.
local Two cross-routable networks exist. Clients connecting with a private address are preferentially redirected to a server’s private address but may be redirected to a server’s public address if need be. Clients connecting with a public address are preferentially redirected to a server’s public address but may be redirected to a server’s private address if need be. This is the default mode of operation in order to be compatible with previous releases. If the machine is dual-homed it is advisable to specify the interfaces you are using for predictable access.
Warning, such a network configuration is not suitable for external access and a proxy server must be used for out of domain clients.
rpipa
Tries to resolve private IP addresses to host names. See the section on resolving private IP address for more information.
norpipa
Does not resolve private IP addresses to host names. This is the default.
tls Indicates that the specifications apply to the Transport Layer Security port.
Defaults
xrd.network cache 3h dnr norpipa
Notes
1) For systems that support TCP buffer auto-tuning as a manual option, specify a buffsz blen of 0 to turn on auto-tuning.
2) Setting the buffer size to a large value may cause the operating system’s default value to be used. You should determine the maximum valid value for your system before specifying values greater that 64k.
3) Normally, the best performance is obtained by using TCP buffer auto-tuning.
4) Even if you specific nodnr, domain Resolution may be forced on if you specify an allow directive using a host name or host name fragment or authorize file access via host names, netgroups, or domain names. Resolution is still optimized by caching the results for future use.
5) The daemon’s internal timeout mechanism can be used to discover unconnected clients instead of TCP keepalive and may be more responsive. See the timeout directive. You should avoid using both mechanisms at the same time.
6) For aggressive keepalive processing you can use “kaparms 300,10,6”.
7) By default, public-private networking support is disabled. You must specify the routes option to enable it. If you do specify for one server you must specify for all nodes in your cluster (i.e. servers and redirectors). Failure to do so may result in unreachable nodes..
8) If the supplied information is inconsistent with the server network settings, warning messages are printed and the public-private network support may be turned off for that server. You should verify that your specification is consistent with the server’s networking configuration.
9) Interface names are arbitrary but usually are of the form enx, ethx, etc. One interface must be assigned a public address and the other the private address. This mechanism works best when all of the servers in a cluster use the same interface naming conventions; though the address assignments may differ. If this is not the case, you will need to use the if-else-fi configuration syntax to special case particular nodes.
10) Dual public/private network are fully supported in Linux, MacOS, and in Solaris 11.
11) IP address caching is generally incompatible with a dynamic DNS. When dyndns is specified but no cache option has been specified, the cache timeout is set to zero. This effectively turns off IP address caching.
Example
xrd.network nokeepalive nodnr
Before you configure a dual public/private network, determine if such a network is actually necessary. Administering dual networks is difficult and problem resolution rather onerous. Typical reasons and alternatives to running a dual network are:
· Conserving IPV4 addressees: consider using IPV6, which does not suffer from this problem.
· Performance: if private addresses are routed along with public addresses over the same networking hardware then the switch becomes the bottleneck and performance improvements are moot.
· Security: if you desire to restrict certain kinds of access you may be able to achieve the same result using router and switch settings (e.g. file walls, routing restrictions, etc) thus avoiding a dual network.
If you still wish to run a dual network, you should use the following guidelines to avoid access surprises and mysterious performance issues.
· Never register private addresses in a publicly accessible DNS server. This exposes your private network configuration and is considered bad practice. Private addresses should only be registered in a private DNS or zone registered to prevent external leakage.
· Never register a server’s private and public address under a single host name unless both addresses have equal connectivity. While this practice works for simple applications, complex applications like XRootD and Proof attempt to use all addresses assigned to a particular host name. If connectivity is unequal, performance issues and connection failures are likely to occur.
· Never cluster servers with only a public address with servers that only have a private address if their name spaces overlap. While this may work for certain network routing topologies, it invariably introduces inconsistencies.
· Carefully review the routes option on the xrd.network directive to make sure your network routing topology and interfaces are correctly stated.
By default, private IP addresses are not resolved to host names. This is commonly accepted practice to avoid DNS timeouts as private addresses are usually not registered in DNS. However, some installations may opt to register such addresses in either a private DNS or in a zoned public DNS. This actually may be necessary in cloud deployments where nodes within the cloud receive private addresses and a NAT box is installed to provide outside access. In such cases, here is usually a 1-to1 mapping between a public address (e.g. IPv6) to a corresponding private address (e.g. IPv4). The public address is registered in a public DNS while the private address is registered in a private DNS within the cloud. Both addresses are registered with the same host name. However, to avoid leaking private addresses you must specify the xrd.network rpipa option so that only host names are returned and not actual addresses. This allows clients on the public network to accesses nodes in the private network using the NAT box. Nodes in the private network also receive host names but since these names are registered within the private network, proper address resolution occurs.
The dyndns option specifies that a Dynamic DNS is being used for the network. This is typically the case for containerized cloud deployments (private or public) managed by an orchestration scheme (e.g. Kubernetes). In such schemes, as a container starts with an arbitrary local IP address, its host name and address are entered into a local DNS. When the container stops, the entry is removed. Even on a container restart, the entry may be removed between the time the container stops and the time it restarts with a new IP address. This becomes a problem for servers expecting to contact specific services when those services have not yet been started or are being restarted. In the XRootD framework, cluster management services that supervisor and data servers rely on expect those services to be permanently registered in DNS; albeit with arbitrary non-permanent IP addresses.
The dyndns option notifies all nodes in an XRootD cluster using the network that DNS entries may come and go and resolution of IP addresses needs to be done at the time of contact not during initialization. Failure to specify the dyndns option when a Dynamic DNS is actually being used inevitably leads to random failures.
Of course, you may be able to avoid using a Dynamic DNS if you opt for host networking, at least for management services. However, many times that is not an option and when allowed severely limits orchestration choices.
all.pidpath path
Function
Specify the location where the process id file (i.e. pid file) is to be written.
Parameters
path The path to be used to create the file where the daemon’s process id is stored.
Defaults
The process id file is written into /tmp.
Notes
1) If -n is specified on the command line, a subdirectory corresponding to the instance name (i.e., -n argument) is created in path, if one does not exist. This becomes the new path. This allows fencing multiple daemons running on the same node.
2) The -s command line option may be used to specify the path and filename to be used for the pid file.
3) The name of the pid file corresponds to the executable name suffixed by “.pid”.
Example
all.pidpath /var/run/scalla
xrd.port [tls] {pnum | any} [ if conds ]
Function
Designate the port number to use for incoming requests.
Parameters
tls Sets the preferred Transport Layer Security port number. Otherwise, the default port number is set. See the usage notes on how the default is determined.
pnum The TCP port number or the TCP service name associated with a port in /etc/services file that the daemon should use for incoming requests. See the usage notes on how the default is determined.
any Specifies that any available TCP port number may be used use for incoming requests. See the usage notes on how the default is determined.
conds The conditions that must exist for this directive to apply. Refer to the description of the if directive on how to specify conds.
Defaults
See the usage notes.
Notes
1) The default port number is determined using the following rules:
· The protocol specified port when the protocol is loaded.
· The port specified on the protocol directive entry.
· The port specified on the command line using –p.
· Any available port number if -p any was specified on the command line.
· The port associated with the service name that corresponds to the name of the program used to start the daemon (e.g., xrootd).
· The port value of 1094.
2) The tls port number need only be specified if you wish to have a port that can only communicate using TLS at the outset. A client may discover this port using the kXR_query request with the tlsport as an argument. If not specified, no TLS-only port is created.
3) Not all protocols support a TLS-only port. For instance, the xroot and xroots protocols negotiate TLS with the client and must first communicate without using TLS. So, it cannot be used on a TLS-only port.
4) When running clustered systems, you can keep a single configuration file that is applicable to all types of servers, as follows:
· always specify “xrd.port any” in the configuration file, and
· use the if modifier to identify the top-most servers (i.e., the initial point of contact also known as redirectors) and assign them fixed port numbers immediately following the “xrd.port any” directive.
4) Using the steps outlined above, the initial point of contact will have a well-known port number. While all other servers will choose random port numbers, the ports are communicated to the cluster manager which then automatically manages the port numbers while redirecting clients.
Example
xrd.port xrdnew
xrd.protocol [tls] name[:port] { lib | * } [parms]
Function
Configure a protocol that xrd is to use for incoming requests.
Parameters
tls Indicates that any connection to the port must communicate using TLS at the outset.
name The name of the protocol you wish to configure.
port The port number the protocol is to use for incoming requests. Specify a number, the name of a TCP service, or the word any.
lib The path to the shared library that contains the code the implements the protocol. If lib is an asterisk, the protocol has been statically linked with the daemon and should not be dynamically loaded.
parms Parameters to be passed to the protocol at load time.
.
Defaults
Not applicable.
Notes
1) The daemon expects that only one protocol is statically linked with the daemon’s executable. The name of this protocol must correspond to the name of the program implementing the daemon (e.g., xrootd). You may configure this protocol using the protocol directive.
2) Additional protocols directives naming a non-default protocol include additional protocols. The daemon attempts to match each such protocol with the incoming connection in the order that the protocols are specified. The default protocol is always tried first, if applicable.
3) Additional protocols are dynamically loaded from the indicated lib.
4) If a port is not specified, then the standard port selection rules apply. Refer to the description of the port directive for port selection details.
5) A port of any assigns an arbitrary port number to the protocol.
6) Only those protocols bound to a specific port are matched against incoming connections on that port.
7) Load-time parameters are specific to each protocol. Refer to the protocol requirements for details.
8) The xrootd protocol does not need any load-time parameters.
9) Up to eight different protocols may be specified.
Example
xrd.protocol xrootd *
xrd.protocol xproofd:1093 /opt/xproofd/lib/libXProof.so
xrd.report dest1[,dest2] [every rsec] [-]option
option: all | buff | info | link | poll | process |
prot[ocols] | sched | sgen | sync | syncwp
[[-]option]
Function
Specify execution tracing options.
Parameters
dest1 is a host:port or a UDP named local socket where reports are to be sent. Reports are always sent as a single UDP message.
dest2 is a secondary destination and must differ from dest1. The same report is delivered to dest2 and dest1.
rsec determines how often reports are sent. Specify a number, optionally suffixed by m for minutes, s for seconds (the default), or h for hours. The default is every 10 minutes.
option Specifies the reporting level. One or more options may be specified. The specifications are cumulative and processed left to right. Each option may be optionally prefixed by a minus sign to turn off the setting. The following options produce reports on:
all selects all possible reports
buff I/O buffer activity
link connection and socket I/O activity
poll socket activity other than I/O
process process resources
protocols protocol specific information
sched scheduling and thread activity
sgen statistics generation
sync synchronizes data for completeness (see the notes)
syncwp synchronizes data only when practically possible
Defaults
Reporting is disabled.
Notes
1) Report messages are encoded in XML format. Refer to the xrootd protocol specification on the actual format and embedded information; as described under the response format for the kXR_QStats option of the kXR_query request.
2) By default, statistical values are obtained without data access synchronization. This may occasionally produce incomplete or inaccurate values. However, because information is collected asynchronously this has little impact on the server.
3) If absolute accuracy is required, you should specify the sync option. Be aware that reporting may require a significant amount of elapsed time while the server synchronizes its activities in order to produce accurate and consistent data.
4) If absolute accuracy is desired but not required, you should specify the syncwp option. The server synchronizes its activities only when possible. If the server is too active, an asynchronous report is done. The sgen report segment provides information on whether the report was synchronous or not and how much time it took to generate.
5) For scalability reasons, you should feed all UDP messages to one or more collectors whose sole function is to multiplex the UDP message streams into a single buffered serial stream. Generally, attempting to do more in a UDP message receiver substantially increases the chance for lost UDP messages.
6) An UDP message multiplexor and XML parser, mpxstats, is available as part of the reference XRootD distribution. Refer to the “Monitoring” reference for details.
Example
xrd.report myhost:1234 every 15m all -poll
xrd.sched parms
parms: [avlt avlt] [core {asis | max | off}]
[idle idle] [maxt maxt] [mint mint] [stksz size]
Function
Specify when threads are created, how many can be created, and when they should be destroyed.
Parameters
avlt avlt
The number of threads that must always be available to service a request. These threads are never bound to any connection. Excess threads above this quantity will be allowed to bind with a socket until either the socket becomes idle or the number of available threads falls under avlt. The default is one half of the mint value.
core {assis | max | off}
Sets the limit for core file production. Choices are
assis - leave the current setting alone.
max - allow core files up the hard limit maximum (the default).
off - turn off core file production.
idle idle
The interval between checks for underused threads. Underused threads in excess of the mint value are terminated. Specify a number, optionally suffixed by m for minutes, s for seconds (the default), or h for hours. The default is every 13 minutes (i.e., 13m). Specifying a value of zero prevents threads from being terminated even if they are idle.
maxt maxt
The maximum number of threads that may be created to service requests. The number of threads will dynamically vary between mint and maxt.
mint mint
The minimum number of threads that must exist to handle requests. Once this number has been created, it is never reduced.
stksz size
The default thread stack size. Specify the number of bytes, optionally suffixed by k for kilobytes or m for megabytes. The default is controlled by the target operating system.
Defaults
xrd.sched mint 8 maxt 2048 avlt 512 idle 780
Notes
1) The mint-number of threads are eventually created.
2) The stack size is controlled by the target operating system used to create xrootd. For instance, in Solaris when sizeof(long) is 4 (indicating a 32-bit architecture), a 1M stack size is used. When sizeof(long) is 8 (indicating a 64-bit architecture or an LP64 data model) a 2M stack size is used. In Linux, the stack grows, as needed, to a maximum of 2M.
3) You should periodically review whether or not you have sufficient number of threads. The daemon prints a warning message the first time the maxt value is reached and more threads are needed.
4) If the daemon indicates that the thread limit was reached but less than maxt threads were created; then the target operating system maximum has been reached. This now becomes the new maxt value.
5) Warning, you should not change the thread parameters unless there is an overpowering reason to do so. The system is optimized for having many thread readily available. Constraining the number of threads may yield random failures that are hard to explain.
Example
xrd.sched mint 10 maxt 100 avlt 20
all.sitename sname
Function
Specify the site name to be included in monitoring records.
Parameters
sname
The 1- to 63-character name of the site. The name may include letters, digits, and the symbols “_-:.”. Other symbols are converted to a period. If the name is longer than 63 characters it is truncated to 63 characters.
Defaults
None.
Notes
1) The –S command line option overrides the sitename directive.
2) The first sitename directive takes precedence over any subsequent sitename directives.
Example
all.sitename slac
xrd.tcpmonlib [++] path [parms]
Function
Specify the location of the TCP connection statistics monitoring plug-in.
Parameters
++ The specified plug-in should stack on top of the existing plug-in or default. A stacked plug-in cannot be overridden by a subsequent directive.
path The absolute path to the shared library that contains an implementation of the TCP connection statistics monitor.
parms Optional parameters to be passed to the plug-in object creation function.
Defaults
None.
Notes
1) The TCP connection monitor plug-in interface is defined in the XrdTcpMonPin.hh include file. Refer to this file on how to create a custom monitoring plug-in.
2) You must specify the tcpmon option in the xrootd.monitor directive in order to enable the TCP connection monitoring.
3) The plug-in is called just before the TCP socket to a client is closed.
Example
ofs.authlib /opt/xrootd/lib/libAuth.so
xrd.timeout parms
parms: [hail hlto[h | m | s]] [idle idto[h | m | s]]
[kill klto[h | m | s]] [read rdto[h | m | s]]
Function
Specify timeout parameters.
Parameters
hail hlto
The maximum number of seconds to wait for data to arrive after a connection is accepted. Specify a number optionally suffixed by h for hours, m for minutes, or s for seconds, the default.
idle idto
The number of seconds a connection may remain idle before it is closed. Specify a number optionally suffixed by h for hours, m for minutes, or s for seconds, the default. A value of 0 disables idle timeout processing.
kill klto
The number of seconds to wait for an “end session” request to complete. Specify a number optionally suffixed by h for hours, m for minutes, or s for seconds, the default.
read rdto
The number of seconds a read may wait for data before it is either terminated or rescheduled. Specify a number optionally suffixed by h for hours, m for minutes, or s for seconds, the default.
Defaults
xrd.timeout hail 30 idle 0 kill 3 read 5
Notes
1) The idle timeout prevents accumulation of dead connections which may happen when a client host machine crashes.
2) Currently, idle timeouts are disabled. You may enable them by specifying an idto value greater than zero.
3) Forced closure of connections is safe if the protocol supports dynamic reconnection, as the xroot protocol does.
4) The read timeout forces a link to be closed should the initial protocol identification data not arrive within the timeout interval. After which, connections that do not send all of their data in the indicated period are simply rescheduled to the background.
Example
xrd.timeout idle 120m read 10
xrd.tlsciphers ciphers
Function
Specify the allowed ciphers for transport layer security (TLS).
Parameters
ciphers
A list of colon separated ciphers that are allowed to be used.
Defaults
For OpenSSL versions greater than 1.0.2 the ciphers recommended by mozilla.org version 5.4 guidelines are used. These are strict ciphers. For older OpenSSL versions, generic ciphers are used for compatibility reasons.
Notes
1) The tls.ciphers directive is provided in cases where a default cipher has been shown to be insecure and should be removed. In this case, you need to specify all of the ciphers less the one you wish to eliminate.
Example
xrd.tlsciphers
ALL:!LOW:!EXP:!MD5:!MD2
xrd.trace [-]option
option: {all | conn | debug | mem | net | none | off |
poll | protocol | sched | tls | tlsctx | tlsio |
tlssok} [[-]option]
Function
Specify execution tracing options.
Parameters
option Specifies the tracing level. One or more options may be specified. The specifications are cumulative and processed left to right. Each option may be optionally prefixed by a minus sign to turn off the setting. Valid options are:
all selects all possible trace levels
conn traces connection activity
debug traces internal activities for debugging purposes
mem traces memory management functions
net traces network management functions
none traces nothing
off a synonym for NONE
poll traces I/O interrupt polling activities
protocol traces protocol activity (see the notes)
sched traces scheduling functions
tls a synonym for the combination tlsctx and tlssok
tlxctx traces TLS context activities
tlsio traces TLS I/O activities
tlssok traces TLS socket activities
Defaults
Tracing is disabled.
Notes
1) All tracing is forcibly enabled when the daemon is invoked with the –d option.
2) All previous trace settings are discarded when none or off is encountered.
3) The protocol trace option is passed along to the all loaded protocols that may or may not respect the option or may have their own options.
Example
xrd.trace all -debug
all.export Specify the file system paths that may be accessed.
xrootd.fslib Specify the file system plug-in to be used for data access.
xrootd.redirect Specify client redirection by type of request, access path, and possible errors during access.
xrootd.chksum Enable file checksum calculation.
xrootd.diglib Enable interactive remote debugging.
xrootd.trace Specify execution tracing options.
xrootd.monitor Specify which statistics are to be collected and where they are to be sent.
xrootd.prep Specify how prepare requests tracking should be handled.
xrootd.seclib Specify the location of the security interface layer.
xrootd.log Specify which events are to be logged.
xrootd.tls Specify TLS requirements by request category.
xrootd.async Specify asynchronous data processing features and limits.
xrootd.fsoverload Specify how file system overloads are to be handled.
xrootd.tlsreuse Specify TLS session cache characteristics.
all.export {path | *[?]} [[no]lock] [oss_options]
Function
Specify a valid path prefix for file requests.
Parameters
path An absolute path prefix for valid file requests. Only files starting with this prefix are allowed in requests.
* Allow arbitrary object identifiers (i.e. names that do not start with a slash). The names are not inspected in any way and passed as is to the file system plug-in.
*? Allow arbitrary object identifiers (i.e. names that do not start with a slash). Inspected the names for CGI information and, if present, separate it from the object identifier (i.e. characters before the question mark) before passing the object name and CGI information to the file system plug-in.
lock Uses standard xroot protection against multiple writers. This is the default.
nolock
Does not protect against multiple writers.
oss_options
Optional oss options that affect how the path is processed by the storage system and cluster service. Refer to the “Open File System & Open Storage System Configuration Reference” and the “Clustering Configuration Reference”.
Defaults
xrootd.export /tmp lock
Notes
1) For security purposes, only files in /tmp are allowed to be accessed unless you specify otherwise. You may specify valid paths either on the command line or using the export configuration directive.
2) Do not prefix path with the oss localroot directive path, if any.
3) By default, a file may be opened by a single writer with no readers or multiple readers without any writer. If an external locking mechanism is used or no locking mechanism is needed; specify the nolock option to disable the default.
4) The [no]lock option must appear before any oss options.
5) The underlying file system plug-in as well as the storage system plug-in must support object identifiers in order to use the * or *? export. The default file system plug-in will pass the object identifier to the storage system plug-in for the common set of file operations. However, the default storage system plug-in will not load if object identifiers are being exported.
6) Object identifiers exportation is meant to support object store plug-ins such as the Ceph block storage plug-in.
Example
xrootd.export /store
xrootd.seclib {default | path}
Function
Specify the location of the security interface layer.
Parameters
default
Uses the default security plug-in the security interface.
path The absolute path to the shared library that contains an implementation of the Security (sec) interface that xrootd is to use for strong authentication(e.g., Kerberos, GSI, etc).
Defaults
Strong authentication is disabled unless seclib is specified.
Notes
1) The sec interface allows you to provide an arbitrary authentication implementation (e.g., Kerberos, GSI, etc).
2) A sec implementation requires that compatible interface libraries be used on the server and client sides of the connection.
3) Refer to XrdSecEntity.hh and XrdSecInterface.hh for guideline on how to write a sec interface.
4) It is up to the sfs implementation to use authentication information to restrict access to files.
5) The provided ofs implementation can use authentication information for access control purposes.
6) The default sfs implementation does not provide any access control.
Example
xrootd.seclib /opt/xrootd/lib/libosec.so
xrootd.async parms
parms: [force] [limit clim] [maxsegs smax]
[maxstalls mstall] [maxtot slim]
[minsz reqsz[k | m | g]] [minsfsz sfsz[k | m | g]]
[off] [nosf] [segsize segsz[k | m | g]] [syncw]
Function
Specify how asynchronous I/O is to be handled.
Parameters
force Uses asynchronous I/O for all requests, even if the client did not ask for asynchronous handling.
limit clim
The maximum allowed number of outstanding asynchronous requests per client connection. Any additional requests past clim are synchronously handled. The default is eight (8).
maxsegs smax
The maximum number of simultaneous asynchronous operations that may any one request may have in progress. The default is eight (8).
maxstalls mstall
The maximum number of times a client may fail to deliver data at a sufficient rate to keep up with asynchronous I/O needs before future requests from the client are synchronously handled. Asynchronous handling is tried again after mstall number of synchronously handled requests. The default is eight (8).
maxtot slim
The maximum number of simultaneous asynchronous operations the server may have in progress. The default is 4,096.
minsz reqsz
The minimum number of bytes that must be read or written in a single client request for that request to be asynchronously handled. I/O requests smaller than reqsz are always synchronously handled. The reqsz can be suffixed by k, m, or g to indicate kilo-, mega-, or giga-bytes; respectively. The default is one half of the initial segsz.
minsfsz sfsz
The minimum number of bytes that must be read in a single client request for that request to be handled using sendfile(). I/O requests smaller than sfsz are always handled in the standard way. The sfsz can be suffixed by k, m, or g to indicate kilo-, mega-, or giga-bytes; respectively. The default 8192 for Linux, 1 otherwise.
off Disables asynchronous I/O for all requests.
nosf Disables using sendfile(), where available, for all read requests.
segsize segsz
The ideal asynchronous I/O segment size. The server attempts to quantize asynchronous I/O request into segsz pices. The segsz can be suffixed by k, m, or g to indicate kilo-, mega-, or giga-bytes; respectively. The default is 64k. See the usage notes on how the system dynamically adjusts this value.
syncw
Uses synchronous I/O for all fsync requests. Otherwise, asynchronous I/O is used for fsync requests if the client requested asynchronous I/O or if the force has been specified.
Defaults
xrootd.async limit 8 maxsegs 8 maxstalls 5 maxtot 4096 segsz 64k
Notes
1) Asynchronous requests allow the client to start a number of read operations at one time and wait for the request to complete in optimal order. When properly employed, asynchronous requests may substantially improve overall client processing speed.
2) Asynchronous processing represents a substantial resource commitment on part of the daemon. Each operation requires the dispatching of a separate thread. Rampant asynchronous processing may exhaust resource limits. Use the maxpl and maxps values to limit the amount of asynchronous processing.
3) Asynchronous processing is effective when the disk transfer rate approaches the network transfer rate. Thus, asynchronous processing is enabled only when a sufficiently large amount of data is requested by the client at one time. Use the minsz parameter to control the point where asynchronous operation is effective.
4) The segsz parameter specifies the ideal i/o size for asynchronous operations in order to maintain continuous disk/network transfer overlap. Requests are broken into segsz units. If the requested size is smaller than segsz, the server will use half the segsz. If the number of units is more than twice maxsegs, the server will use twice the segsz.
5) Conversion of asynchronous requests to synchronous requests is transparent to the client.
6) The sendfile() interface allows data to be transferred directly from the kernel’s file system memory cache to a client. Generally, this significantly reduced system overhead. The sendfile() function is available in Solaris 5.8 and above, as well as Linux 2.6 and above.
7) Use the nosf option in cases where you suspect that the sendfile() interface is causing data transfer problems.
Example
xrootd.async minsz 1M
xrootd.chksum [chkcgi] [max num] digest [path [args]]
digest: algorithim [digest]
Function
Specify how file check sums are computed.
Parameters
chkcgi
Always checks the cgi information, if any, for the cks.type element that can be used to select a checksum algorithm. The cgi information is not checked if only one digest is specified for backward compatibility. It is always checked if more than one digest is specified, making chkcgi unnecessary. See the usage notes for more information.
num Maximum number of checksum calculations that may run at the same time. Specifying 0 prevents real-time check summing. See the notes for more information.
algorithm
The name of the checksum digest (e.g., md5) used for the check summing. Specify one or more supported digests, each separated by a space. The first algorithm becomes the default algorithm. See the usage notes on how multiple digests are supported,
path The absolute path of the program that computes the check sum. If path is not specified, checksums are internally performed.
args Initial arguments to be passed to the program identified by path, if any.
Defaults
The default max is 4; otherwise. If path is not specified, checksums are internally performed. File check summing is not supported unless the directive is specified.
Notes
1) When a client issues an xrootd query checksum request, the following steps are performed:
a. A checksum digest is selected as follows:
i. If a single algorithm is specified and chkcgi was not specified, the digest in the configuration file is used.
ii. If a single algorithm is specified and chkgi is specified, a cgi scan is made for cks.type and, if specified, its argument must match the single algorithm in the configuration file or an error results. In any case, the digest in the configuration file is used.
iii. If more than one algorithm has been specified, a cgi scan is made for cks.type and, if specified, its argument must match one of the specified algorithms in the configuration file or an error results. If there is a match, the cks.type argument is used as the desired digest. If cks.type is not found, then the first algorithm specified in the configuration file is used.
b. A check is made that the client has lookup privileges for the file and that a valid checksum has been recorded for the file. If both are true, that checksum is sent back to the client. If the client lacks lookup privileges, an access error is sent back to the client.
c. Since a checksum needs to be computed the max value applies. If it is zero, the client is told that the checksum is not available.
d. If the checksum is natively supported and no program path has been specified, a new checksum is locally computed and recorded for future queries. Otherwise, the program named in path is executed to compute a new checksum and it is not recorded for future queries. Of course, the program may record the checksum in some way for future queries.
e. Either the previously recorded checksum or the computed checksum is provided to the client.
2) Native checksums are adler32, crc32, and md5.
3) Use the ofs.ckslib directive to add new digests or improve the performance of the native digests.
4) Since computation of multiple checksums is CPU and memory intensive choose the max with circumspection. You can control memory usage via the ofs.cksrdsz directive.
5) The ofs directives are documented on the OFS/OSS reference manual.
6) When the program identified by path is invoked, it is passed the path to the file that is to be processed. The actual argument list varies depending on whether or not a single algorithm has been specified, as follows:
a. When a single algorithm has been specified, the program is passed the file path as the last argument and is the only argument if no args have been specified.
b. When multiple algorithms have been specified, the program is passed the file path as the last argument and the checksum algorithm name as the second to the last argument. If no args have been specified, these are the only two arguments that are passed.
7) The program must output on standard out a single checksum value; normally ending with a new-line (‘\n’) character and terminate with a status code of zero. If the program terminates with a non-zero status code or returns no output, the client’s request fails.
8) Upon success, the returned checksum value is passed back to the client, prefixed by the digest token, digest.
9) Warning: If an external checksum program is specified (i.e. path is specified), then neither the oss.localroot nor oss.namelib directives are applied to the logical file name before passing the file name to the specified program that computes the checksum. Hence, the program is responsible for converting a logical file name to a physical file name.
10) When checksums are natively computed (i.e., path is not specified), then the oss.localroot and oss.namelib directives are applied to the logical file name. The checksum is computed against the resulting physical file name.
11) The chkcgi option is provided for backward compatibility. In previous releases only one algorithm could be specified and cgi information was immaterial. This processing mode remains the default when only a single algorithm is specified. You may wish to verify that a client as not requesting an unsupported digest in the case where some servers support multiple checksums and others do not.
12) The administrator’s interface allows you to list and cancel checksum jobs. This applies to external as well as internal computation of the checksum.
13) When max is zero, checksums on demand are prohibited. This requires that checksums to be pre-computed. This can be done using the frm_admin chksum command. See the File Residency Manager reference.
Example
xrootd.chksum max 2 md5
xrootd.diglib * authpath
Function
Enable remote debugging via the digFS read-only file system.
Parameters
* Loads the built-in version of digFS; the only the version currently supported.
authpath
The path to the authorization file that describes who is allowed to access digFS and what kind of information they may view.
Defaults
Bu default, digFS is disabled.
Notes
4) The digFS provides a virtual read-only file system view of key information about xrootd and cmsd that is valuable to remotely debug system problems.
5) Since digFS exposes system information an authorization file describing access permissions is required. See the next section.
6) When diglib is specified, the /=/ directory is automatically exported and available to authorized users. You must not list /=/ in the all.export list.
7) The /=/ path always refers to local storage regardless of server role and is never subject to redirection.
8) Only close, dirlist, locate, open, read, and stat requests can be vectored to digFS. Other requests referring to /=/ are disallowed.
9) The digFS accepts configuration directives starting with dig. Refer to subsequent sections for a description of these directives.
Example
xrootd.diglib * /etc/xrootd/digauth.cf
The file describing digFS access permissions is composed of newline delimited records. Each record describes a single entity that is authorized to access certain information. The format of each record is
info allow aprot ident
info: all | [-]conf | [-]core | [-]logs | [-]proc | [info]
aprot: gsi | host | krb5 | pwd | sss | unix
ident: g=group | h=host | n=name | o=org | r=role | [ident]
Parameters
info Authorizes the entity described in the record to access certain information. Use the word all to allow access to all information. If you specify all, you can remove specific information by specifying subsequent information keywords prefixed by a minus sign. Alternatively, list the info keywords to enable access to the associated information described below.
Keyword |
Information |
Keyword |
Information |
conf |
configuration file |
logs |
log files |
core |
core files |
proc |
process information (Linux only) |
aprot Specifies the authentication protocol that must be used in order to use digFS. Only one protocol per entity description may be specified. Since the information provided by digFS is sensitive in nature you should use the strongest authentication protocol consistent with site policies. The following table lists the default protocols[1] from strongest to weakest. Additionally, the rightmost column lists the ident tags that can be successfully specified relative to that protocol since not every protocol identifies clients in the same way. See the XRootD security reference for more detailed information.
Protocol |
Descritption |
Meaningful ident Codes |
krb5 |
Kerberos Version 5 |
h n |
gsi |
Grid Security Infrastructure (i.e. x.509) |
g h n o r |
sss |
Simple Shared Secret |
g h n |
pwd |
Password |
g h n o r |
host |
DNS resolved hostname |
h |
unix |
NFS V2-Style authentication |
g h n |
ident Authorizes the aprot authenticated entity possessing the specified identity values access to info digFS information. Identity values are specified as key value pairs. The entity must match all the specified pairs in order to be granted access. An imbedded space in a value must be designated as a \s (i.e. two character sequence). Specifying an inappropriate key relative to an authentication protocol prohibits access. The following table describes the possible key value pairs.
Key |
Value |
g |
Group name |
h |
Fully qualified hostname |
n |
Authentication-specific protocol client identity string (see notes) |
o |
Organization name |
r |
Role name |
Notes
1) Valid entries in the authfile are used and syntactically incorrect entries are discarded. At least one valid entry must exist for digFS to be enabled.
2) If the modification time of the authfile changes outside of a 5 second window it is reprocessed. This allows you to modify the authfile on a running system. However, you must atomically update the file as follows:
a. Create a copy of the file.
b. Modify the copy as needed.
c. Rename the copy to be the original name (i.e. use mv).
Distributing a modified copy of the file to other hosts should also use rename to install the new authfile.
3) Each authentication protocol has a specific way of identifying a client. For instance, x.509 (i.e. gsi) uses distinguished name (i.e. dn). Depending on the security configuration the protocol-specific name may be mapped to a Unix name. If so, you must use the mapped name not the original name.
4) If an entity is associated with more than one group name then the specified group (i.e. g=) must match one of the associated group names.
5) Starting in version 4.2 you are able to enable digFS but prevent its use by simply not creating an authorization file or commenting out all authorization entries. This allows you to enable its use in real-time without restarting XRootD by either creating an authorization file or adding authorization lines to an existing file.
6) Prior to 4.2 you must have a valid authorization file with at least one authorization entry. However, that entry may be unsatisfiable. This also allows you to selective enable or disable digFS without an XRootD restart.
Example
all –core allow krb5 h=test.org n=xtestor
conf logs allow gsi g=atlas n=theuser
The digFS accepts the following directives in the configuration file.
dig.addconf path [ fname ]
Function
Add a configuration file reference to the digFS namespace.
Parameters
path The absolute path to a regular file that is to be added to “/=/conf/etc”. The name of the file will be the same as the last component of path unless fname is specified.
fname The name that is to appear in “/=/conf/etc” but refers to path.
Defaults
None.
Notes
1) The path is only added if it is readable by the xrootd server.
2) This directive allows you to make other server related configuration files available via digFS.
Example
dig.addconf /etc/sysconfig/xrootd
dig.log parm [ parm ]
parm: deny | grant | none
Function
Control the level of logging.
Parameters
parm The level of logging; specify one or more of:
deny - log file access denials
grant - log file access approvals
none - turn off logging
Defaults
dig.log deny grant
Notes
1) To enable logging of denials and approvals you must specify both deny and grant parameters.
Example
dig.log deny
The digFS file system can be accessed using standard file system applications. All information is rooted in the /=/ directory and follows a standard layout. The following table describes directory tree.
Directory |
Subdirectory |
Contents |
/=/conf |
|
Configuration files cmsd.cf and xrootd.cf |
|
/etc |
Other site selected configuration files. |
/=/core |
|
Core files |
|
/cmsd |
Directory holding cmsd core files. |
|
/xrootd |
Directory holding xrootd core files. |
/=/logs |
|
Log files |
|
/cmsd |
Directory holding cmsd log files. |
|
/xrootd |
Directory holding xrootd log files. |
/=/proc |
|
/proc files (Linux only) |
|
/cmsd |
Directory holding the cmsd proc files. |
|
/xrootd |
Directory holding the xrootd proc files. |
If the cmsd and the xrootd share the same configuration file the /=/conf/cmsd.cf and /=/conf/xrootd.cf will be identical. If they share the core file directory or the log file directory; the same files may appear in the cmsd and xrootd subdirectories. As other components are added to digFS, additional executable names may appear in each root subdirectory.
To find out your access rights, simply list the entries in the /=/ directory. Only the subdirectories for which you are authorized are displayed.
xrootd.fslib [++] [throttle | path]
Deprecated (see notes):
xrootd.fslib [throttle | path2] {default | path1}
Function
Specify the location of the file system interface layer.
Parameters
++ The specified plug-in should stack on top of the existing plug-in or default. Once specified, it cannot be replaced by a subsequent directive.
throttle
Loads libXrdThrottle.so to wrap the subsequent library specification.
path The path to the shared library that contains the file system plug-in.
path2 The path to the shared library that is to be used as the wrapper for the subsequent library specification.
default
Loads a built-in version of the file system implementation.
path1 The path to the shared library that contains an implementation of the Open File System (ofs) interface that xrootd is to use for file system specific operations (e.g., open, close, read, write, rename, etc).
Defaults
xrootd.fslib default
Notes
1) When you only specify the shared library filename, the library is located using the standard platform-dependent loader rules (e.g. well know places followed by the LD_LIBRARY_PATH envar setting).
2) The sfs interface allows you to provide an arbitrary file system implementation. It is documented in XrdSfsInterface.hh. Refer to this include file for differences between version 1 and 2 instantiation.
3) The deprecated version of his directive should be avoided The general version is a superset of the deprecated version. In the common case
ofs.fslib throttle default
can be simply replaced by
ofs.fslib ++ throttle
Should you mix both styles, the deprecated version is handled before the general version of the directive which may be confusing.
Example
xrootd.fslib /opt/xrootd/lib/libofs.so
xrootd.fsoverload [[no]bypass] [redirect target]
[stall sec]
target: host:port[%host:port]
host: dnsname | [ipv6addr] | ipv4addr
Function
Specify how to handle file system overload.
Parameters
[no]bypass
Specifies whether or not clients should be redirected to the client-specified forwarding destination when the file system indicates it is overloaded. This option is only meaningful for servers configured as forwarding proxies and is ignored if that is not the case. It is only applicable when the client specifies a forwarding destination.
redirect target
The name or address of the host and port number where clients are to be redirected when the file system indicates that it is overloaded. The target consists of one or two host:port specifications with the second separated by a percent sign (%). When a second host:port is specified, then clients connecting using a private IP address are redirected to the second host:port while clients connecting with a public IP address are redirected to the first host:port. If only one host:port is specified, all clients are redirected to that host.
stall sec
Specifies how long the client should be stalled when a redirect target is not available and the file system indicates that it is overloaded. After the stall, clients will re-issue the request. A value of zero passes back an overload error to the client when the file system indicates an overload.
Defaults
xrootd.fsoverload nobypass stall 33
Notes
1) The fsoverload directive is most effective for disk caching proxy servers. Refer to the “Proxy Storage Services Configuration Reference” for additional information on how to effectively use this directive.
2) Currently, the fsoverload directive only applies to file open requests. All other requests encountering a file system overload event fail with an overload error.
3) The bypass directive only applies to release 4.0 or higher clients. Older clients that specify a forwarding path are subject to the stall option should a file system overload event occur.
Example
xrootd.fsoverload bypass redirect foo.proxy.edu:1094
xrootd.log [-]levent [ [-]levent ] [• • •]
levent: all | disc | login
Function
Specify event logging options.
Parameters
levent Specifies the events to be logged level. One ore more events may be specified. The specifications are cumulative and processed left to right. Each event may be optionally prefixed by a minus sign to turn off the setting. Valid events are:
all logs all possible events, the default
disc disconnect events
login login events
Defaults
xrootd.log all
Notes
1) Events messages are routed to the xrootd log file.
Example
xrootd.log all -login
xrootd.monitor [ options ] dest [ dest ]
options: [all] [auth] [flush [io] intvl[m|s|h]]
[fstat intvl[m|s|h] [lfn] [ops] [ssq] [xfr cnt]]
[ident sec] [mbuff size[k] [rbuff size[k]]
[rnums cnt] [window intvl[m|s|h]]
dest: dest events host:port
events: [ccm] [files] [fstat] [io] [info] [pfc] [redir]
[tcpmon] [user]
Function
Enable I/O monitoring.
Parameters
all Automatically enables monitoring for all connections. If all is not specified, monitoring is only enabled upon client request.
auth includes authentication information along with user information, when user is specified and authentication has been configured.
flush [io] intvl
The maximum time event data may be internally buffered before it is sent to the monitoring destination. Specify a number optionally suffixed by h for hours, m for minutes, or s for seconds, the default. When io is specified, io event data is also subject to flushing. Otherwise, only non-io events are flushed. The default only applies to non-io events and is 10 minutes.
fstat intvl [lfn] [ops] [ssq] [xfr cnt]
Enables file activity monitoring using a special “f” stream. The intvl is the maximum time event data may be internally buffered before it is sent to the monitoring destination. Specify a number optionally suffixed by h for hours, m for minutes, or s for seconds. A value of zero disables the “f” stream. The intvl is also used as the basis for xfr event data. By default, only file open and close events are inserted into the stream. Additional information may be requested as follows:
lfn includes the user’s dictionary identifier along with the logical file name being opened in the open event record.
ops includes detailed operation count information along with minimum and maximum values in the close event record.
ssq includes the sum of squares count for read and write sizes in the close event record. Specifying ssq automatically includes ops. This option will impact server performance. See the notes for more information.
xfr cnt inserts the number of bytes read and written from each open file every intvl*cnt elapsed time. The cnt must be 1 or more.
ident sec
The number of seconds between each server identity transmissions (i.e., the ‘=’ map record). Specify a number optionally suffixed by h for hours, m for minutes, or s for seconds, the default. A value of zero transmits the identity only once at start-up time. The default is 1 hour (i.e. 3600 seconds).
mbuff size
The size of the monitoring datagram for file and I/O events. Specify no less than 1024 and no more than 64k as the maximum message size. The size can be suffixed by k to indicate kilo-bytes. The default size is 16k.
rbuff size
The size of the monitoring datagram for redirection events. Specify no less than 2048 and no more than 64k as the maximum message size. The size can be suffixed by k to indicate kilo-bytes. The default size is 32k.
rnums cnt
The number of redirection monitoring streams to start. Specify no less than 1 and no more than 8. The default size is 3.
window intvl
The monitoring window size. Data collected within the window is not differentiated by time. Thus, the window represents the undifferentiated sampling interval. Specify a number optionally suffixed by h for hours, m for minutes, or s for seconds, the default. The default is 60 seconds.
dest host:port
The name of the host where monitoring messages should be sent. The receiving port number must be specified after the colon. All monitoring messages are sent as datagrams (i.e., UDP protocol). The dest parameter must be specified as the last parameter. Up two destinations are allowed. By default, only file event information is sent. The actual events may be specified after the dest keyword. These events are:
ccm cache context management information.
files file-related request monitoring (i.e., open and close requests).
fstat the specified “f” stream information (i.e., open and close requests).
io I/O request monitoring (read and write requests plus files).
iov same as io above but also include details on readv vector elements.
info client specified monitoring data submitted using xrootd protocol.
pfc proxy file cache information (i.e. proxy disk caching).
redir redirection events.
tcpmon TCP connection statistics at time of close.
user client login and disconnect events.
Defaults
While flush 10m ident 1h mbuff 8k rbuff 32k rnums 3 window 60 is in effect; monitoring is not enabled.
Notes
1) Use the monitor directive to enable statistical gathering of file event and I/O requests.
2) The fstat ssq option impacts performance since floating point operations must be carried out for each read and write request in order to accurately compute the sum of squares. The counts can be used to compute the standard deviation for read and write sizes. Do not specify this option unless there is a clear need for such information.
3) The fstat ssq counts are available on platforms that use IEEE 754 floating point format. The fstat ssq option is ignored on non-conforming platforms.
4) The io option reports individual seeks for each read and write request. While this data may be used to determine access patterns or used in I/O trace simulation studies, it reduces server performance by about 7% and generates a large amount of monitoring data. Normally, fstat provides sufficient information about client I/O efficiency at a much lower cost.
5) The flush parameter does not apply to monitor streams that include io event data unless io is specified. By default, monitor streams that include io event data are flushed only when the internal monitor buffer becomes full or when the user owning the stream being monitored disconnects.
6) You may specify two monitoring destinations. This allows you to isolate data high volume streams (i.e., io monitoring) and provide real-time display for low-volume streams (i.e., info, files, fstat, and user).
7) The all option forces monitor data to be collected for all connections. If all is not specified, each client must enable monitoring manually using the xrootd set request code (see the xrootd protocol specification). This allows selective monitoring and gives each client the opportunity to tag io monitor data with the relevant application name.
8) The iov option inserts a read entry for every element in a readv vector. This may explode the amount of monitoring information that is generated. By default, when only io is specified, a summary readv entry is placed in the monitoring stream.
9) Clients cannot enable monitoring that has not been enabled by the monitor directive.
10) Specifying a small datagram buffer size (e.g. less than 8k) increases the number of datagrams that need to be sent and, consequently, adds to server overhead. Large datagram buffer sizes reduce the number of datagrams as well as server overhead but increase memory utilization as each connection allocates a buffer.
11) Approximately 61 requests can fit into a 1K mbuff.
12) On average, 64 to 128 redirection events can fit into a 32K rbuff.
13) Increasing the number of redirection monitoring streams (rnums) reduces the bottlenecks in the monitoring path.
14) Specifying a small window increases the timing accuracy of any individual request entry at the expense of additional datagrams and significantly increased server overhead. Conversely, large window sizes reduce timing accuracy but also reduce server overhead.
15) The tcpmon monitoring information relies on a monitoring plug-in specified by the xrd.tcpmonlib directive. If not specified, the tcpmon option has no effect.
16) Refer to the “XRootD Monitoring” reference for a detailed explanation on the datagram format used by the monitoring subsystem. It is especially important to understand the different between files, fstat, and io monitoring as the information overlaps and there is rarely a need to specify all three.
Example
xrootd.monitor all fstat 5m dest fstat datacoll:5050
xrootd.prep parms
parms: [ keep ksec ] [ scrub time ] [ logdir ldir ]
Function
Specify how prepare request tracking is done.
Parameters
keep ksec
The time that prepare request tracking record are to be held. The time may be suffixed by s (the default), m , or h to indicate seconds, minutes, and hours, respectively. The default is 24 hours.
scrub time
The time between scrubs of the tracking log directory. The time may be suffixed by s (the default), m , or h to indicate seconds, minutes, and hours, respectively. The default is 1 hour.
logdir ldir
The absolute path of the directory that is to hold the preparation tracking records. A directory must be specified, otherwise preparation request tracking is disabled.
Defaults
None. Preparation request tracking is normally disabled. When a logdir directory is specified, the keep and scrub defaults of 24H and 1H apply, respectively.
Notes
1) This directive allows server to track prepare requests. When request tracking is enabled, each prepare is logged in the logdir directory. It then becomes possible to list the requests and cancel them, if need be.
2) Since there can be more than one redirecting xrootd server, prepare requests may be scattered across several servers. It is the client’s responsibility to collect information from each server in order to create a composite preparation request history.
3) Each server uniquely names the files in the logdir directory. When multiple xrootd redirecting servers exist, it is possible to collect full preparation history from any server, if the logdir directory is located in a shared file system (e.g., NFS).
4) When running multiple xrootd servers on the same machine, the instance name (-n command line option) is used to differentiate logdir directories among all instances by appending the instance name to the path.
Example
xrootd.prep keep 12H logdir /nfs/xrootd/preplog
xrootd.redirect target {[-]foper | [?] path [path […]]}
foper: {all | chmod | chksum | dirlist | locate | mkdir
| mv | prepare | prepstage | rm | rmdir | stat
| trunc} [[-]foper]
target: host:port[%host:port]
host: dnsname | [ipv6addr] | ipv4addr
Function
Specify request forwarding.
Parameters
target The name or address of the host and port number where clients are to be redirected based on the subsequent parameters. The target consists of one or two host:port specifications with the second separated by a percent sign (%). When a second host:port is specified, then clients connecting using a private IP address are redirected to the second host:port while clients connecting with a public IP address are redirected to the first host:port. If only one host:port is specified, all clients are redirected to that host.
foper Specifies which metadata operations are to be immediately redirected. One or more operations may be specified. The specifications are cumulative and processed left to right. Each operation may be optionally prefixed by a minus sign to turn off the setting. Valid operations are:
all redirect all possible operations
chmod redirect change mode requests
chksum redirect checksum requests
dirlist redirect directory content listing requests
locate redirect path location requests
mkdir redirect create directory requests
mv redirect rename requests
prepare redirect prepare requests that do not need file staging
prepstage redirect prepare requests that may need file staging
rm redirect file removal requests
rmdir redirect directory removal requests
stat redirect file attribute requests
trunc redirect file truncate requests using a path
path Specifies that when a file open request occurs on the specified path prefix, the client should be redirected to the specified host and port. One or more paths may be specified. However, no more than four different host-port combinations may be specified.
? path Specifies that any client operation on the specified path prefix that ends with a “not found” error (i.e., EONOENT) and has not been specifically covered by another redirect directive, the client should be redirected to the specified host and port. All subsequently specified paths, if any, on the line fall under the “not found” provision.
Defaults
xrootd.redirect -all
Notes
1) Request redirection is typically applicable to the cluster manager. Refer to the role directive in the “Clustering Configuration Reference” for additional information, especially on inter-related directives.
2) Normally, meta-data requests are performed on the local host. However, certain clustered environments may be controlled by a central manager that records the exact state of every file. In such environments, the central manager may perform meta-data requests. When the redirect directive is not specified, the client is directed to perform the operation on a single host, normally the one that has the file. When the request is redirected, the target host is responsible for performing the operation.
3) The redirect path prefixes are always matched from most- to least-specific prefix (i.e., longest to shortest).
Example
xrootd.redirect all -prepare
xrootd.tls [capable] req
req: [-]all | [-]data | [-]login | none | off |
[-]session | [-]tpc | req
Function
Specify transport layer security (TLS) requirement.
Parameters
capable
applies requirements only to TLS capable clients. The default is to apply TLS requirements to all clients irrespective of their ability to use TLS.
req specifies the general requests for which TLS must be used. Clients who attempt these requests without using a TLS connection are rejected with a “TLS Required” error message. Valid req are:
all requires TLS for all requests
data requires that all file data or metadata be transmitted using TLS
login requires TLS for login requests and all subsequent requests
none turns off all requirements (off is a synonym)
session requires TLS for all requests after the login and authentication
tpc requires TLS for third party copy requests
Defaults
xrd.tls none
Notes
1) The capable parameter is meant to provide a migration path for pre-Release 5 clients (i.e. those that do not support TLS). Requiring TLS for all clients essentially disallows older clients from using XRootD.
2) The tls directive is cumulative and allows different settings for TLS-capable clients and those that do not support TLS. See the examples on how to accomplish this.
3) Specifying a minus sign in front of any indicated requirement removes that requirement from the current set of requirements.
4) The session requirement is a subset of the login requirement. If both are specified, login prevails.
5) The data requirement applies to the session connection as well as any additional connections bound to the session. If data is specified without specifying login or session, session is automatically added to the requirements.
6) At least one requirement must be specified, even if it’s none or off.
7) When none or off is encountered, all applicable requirements are discarded.
8) The requirements only apply to the XRootD protocol. They do not apply to any other protocols running in parallel with XRootD.
9) The tls directive fails if TLS has not been configured using the xrd.tls directive.
Example
xrootd.tls tpc
The above requires that any third party copy request use a TLS connection. This also prohibits older client (i.e. those incapable of TLS) from requesting a third party copy. Adding the following directive
xrootd.tls capable session
also requires that TLS capable clients must use a TLS connection after the login and authentication phases. Note that while session also covers third party copy requests, the preceding directive requires that all TPC requests use TLS. Hence, non-capable clients can do anything they are allowed to do but cannot request a third-party copy.
xrootd.tlsreuse off | on [flush ft[h|m|s]]
Function
Specify transport layer security (TLS) session reuse characteristics.
Parameters
off disables the TLS session cache and may substantially increase latency for reconnecting clients.
on enables the TLS session cache. This is the default.
flush ft
flushes the cache every ft interval. Suffix ft with h for hours, m for minutes, or s for seconds; the default. If ft is less than 60 seconds it is set to 60. If ft is greater than 32,767 seconds, it is set to 32,767. The default flush is not time based. Instead, the cache is flushed every 255 TLS connections.
Defaults
xrd.tlsreuse on
Notes
1) The xroots protocol can successfully use the TLS session cache because it does not use TLS peer certificates.
2) Flushing the cache every 255 connection is not ideal. You should flush the cache at regular intervals that are short enough to not substantially increase memory usage. The time interval is largely determined by the highest number of connections expected in any time window. Typically, 1,000 connections is a good start. Set the window to cover that number of connections at its peak.
3) Once you specify a flush there is no way to revert to connection count flushing.
xrootd.trace [-]option [ [-]option ] [• • •]
option: all | debug | emsg | fs | login | mem | off |
stall | redirect | request | response
Function
Specify execution tracing options.
Parameters
option The tracing level. One or more options may be specified. The specifications are cumulative and processed left to right. Each option, other than off, may be optionally prefixed by a minus sign to turn off the setting. Valid options are:
all selects all possible trace levels
debug traces internal activities for debugging purposes
emsg traces errors sent back to the client
fs traces file system requests
login traces login and authentication steps
mem traces memory management functions
off traces nothing
stall traces client deferrals due to resource limitations
redirect traces client redirections to other servers
request traces client request information
response traces request response information
Defaults
xrootd.trace off
Notes
1) Warning: enabling tracing may reduce server performance by 50%!
2) All tracing is enabled when the daemon is invoked with the –d option.
3) All previous trace settings are discarded when off is encountered.
Example
xrootd.trace all -debug
XRootD supports HTTP access via a protocol plug-in. The HTTP protocol can run alongside of standard XRootD protocol without any interference; providing an additional access mode. To Enable HTTP access, add the following configuration parameter to the configuration file.
if exec xrootd
xrd.protocol http[:port] path/libXrdHttp.so [cfgfile]
fi
Parameters
port The port number http is to use for incoming requests. Specify a number, the name of a TCP service, or the word any. If you do not specify port number, port 1094 is used. This is also the default port for XRootD protocol.
path The path to the shared library libXrdHttp.so that contains the code the implements the HTTP protocol.
cfgfile is the path to an external configuration file specific to HTTP. If not specified, the configuration file at daemon start-up is used.
Defaults
Not applicable.
Notes
1) You should surround the xrd.protocol directive with the shown if-fi if you are using a common configuration file for xrootd and cmsd daemons. Failure to do so will prevent the cmsd from starting.
2) All HTTP requests are bound by any “xrd.” and “xrootd.” Directives that exist in the start-up configuration file. Hence, HTTP access cannot exceed any restriction imposed by those directives.
3) Monitoring of HTTP requests is handled as if they are XRootD requests. Consequently, the monitoring stream includes all HTTP requests as well. However, monitoring information is tagged with the fact that the information was generated by the HTTP protocol.
4) While the default port number is the same as for XRootD; the framework directs each request to appropriate protocol and these requests are never intermixed. To avoid any confusion you may wish to use a more standard HTTP port number such as 8080. This is especially true if connection are made using HTTPS.
5) Additional configuration file directives specific to HTTP are always prefixed with “http.” and the following sections describe these directives.
6) HTTP support also includes WebDav support enabling a wider range of access abilities beyond simple get and post capabilities.
7) When HTTP is used in a clustered XRootD deployment, all servers in that deployment must have HTTP enabled. Failure to do so typically results in access failure when a client is redirected to a serer that holds the desired file but for which HTTP was not enabled.
8) Not all HTTP clients support all HTTP features. While the plug-in does not violate the HTTP or WebDav standards, it does implement a wide range of allowable features (e.g. redirection on POST requests) that may not be supported by the HTTP client being used.
When a server is configured to
use HTTPS, each server processes the client’s credentials from the
connection. This allows the client to be authenticated and makes authorization
possible. On the other hand, HTTPS is very resource demanding because it
encrypts and decrypts all of the TCP traffic. Additionally, the process
of establishing an encrypted connection requires several network inter-changes that
increase connection latency which can be substantial on a wide area network.
While HTTP is much faster it is impossible to authenticate the client using HTTP. However, the HTTP plug-in allows a client to initially connect with HTTPS, extract the authentication information, encode that information in a low-overhead encrypted security token and redirect the connection to use HTTP. This is known as HTTPS to HTTP conversion and is much less resource intensive. When a server is configured to do HTTPS to HTTP conversion, it always expects a security token to be present when a client connects via HTTP. If the token is missing or cannot be decrypted the connection is rejected. This mechanism provides a relatively secure authentication but at the expense of privacy as no traffic is encrypted past the authentication stage.
HTTPS to HTTP conversion is especially attractive in clustered environments where a client typically makes contact with a particular node (i.e. redirector) that then redirects the client to a particular server that holds the requested file. Using HTTPS everywhere incurs identical overhead at each contact point. This can be eliminated by using HTTPS at the initial contact point and converting HTTPS to HTTP for subsequent connections. This incurs the overhead just once.
When enabling HTTPS you should consider the following points:
When using HTTP in an XRootD cluster, additional considerations apply on how the cluster redirector interacts with data servers in that cluster. The following table provides reasonable possibilities, depending on the degree of security that is desired.
Redirector Accepts |
Server Accepts |
Configuration |
Remarks |
HTTP |
HTTP |
The is the default |
No security. |
HTTPS |
HTTP with security token |
Specify xrd.tls, xrd.tlsca and http.secretkey directives. |
Central authentication, fast unencrypted data access but higher CPU load in the redirector |
HTTP |
HTTPS |
Specify xrd.tls and xrd.tlsca directives only in data servers. Specify http.desthttps in redirectors. |
Fast redirection, distributed authentication, slow encrypted data access |
HTTP |
HTTPS with self redirection using HTTP with security token |
Specify xrd.tls, xrd.tlsca, http.selfhttps2http and http.secretkey directives only in data servers. Specify http.desthttps in redirectors. |
Fast redirection, distributed authentication, fast unencrypted data access |
HTTPS |
HTTPS |
Specify xrd.tls and xrdtlsca directives in servers and redirectors. |
Fully authenticated but authentication occurs twice, slow encrypted data access, resource consumption can be high |
Exceptions are handled by the http.httpsmode directive. See this directive on how to control backward compatibility warning messages.
By default, HTTPS access is not enabled. You must specify certain critical xrd information in order for HTTPS to be enabled (i.e. xrd.tls and xrd.tlsca).
The xrd framework provides TLS services to all protocols and HTTPS relies on these services. The xrd.tls and xrd.tlsca directives should be used to configure TLS for HTTPS. Previous releases relied on http specific directives to do this (i.e. http.cadir, http.cafile, http.cert and http.key). These are still accepted and still allow HTTPS to be configured. However, when specified a warning message is issued to remind you to use the xrd framework directives instead. It is best to configure TLS for all protocols using a common set of directives to avoid inconsistency. Therefore, the http-specific TLS directives have been deprecated.
However, there are other directives that are specifically oriented to improving the handling of HTTPS. These are described in the following sections.
http.desthttps {no | yes}
Function
Specify whether or not HTTPS is to be used for redirections.
Parameters
no A redirector will always redirect a client using http. The word false and the number 0 are synonyms.
yes A redirector will always redirect a client using https. The word true and the number 1 are synonyms.
Defaults
http.desthttps no
Notes
1) While this directive applies normally applies to redirectors it is used by any node, redirector or data server that redirects a client.
Example
http.desthttps yes
http.gridmap path
Function
Specify the file containing the "grid map file" that the server must use.
Parameters
path The path to the file.
Defaults
None.
Notes
1) This file is loaded at startup and used to translate the requestor’s x509 DN into a short user name for internal authorization usage.
2) This directive requires that HTTPS be enabled with certificate verification.
Example
http.gridmap /etc/grid-security/mapfile
http.httpsmode {auto | disable | manual}
Function
Specify how to handle enabling HTTPS protocol.
Parameters
auto automatically enables the use of HTTPS if the underlying xrd framework was configured to allow TLS (i.e. the xrd.tls and xrd.tlsca directives). This is the default.
disable
disables the use of HTTPS regardless of any other directives.
manual
enables the use of HTTPS but requires that the appropriate http directives are used to specify required HTTPS parameters (i.e. http.cadir, http.cafile, http.cert and http.key).
Defaults
http.httpsmode auto
Notes
1) When auto is in effect, you may still override xrd directives that are used for HTTPS. However, each override produces a warning message. To suppress the warnings specify manual and provide all the required http directives to configure HTTPS.
Example
http.httpsmode manual
http.secretkey {path | token}
Function
Specify the key to be used to encrypt and decrypt redirection tokens.
Parameters
path The absolute path to the file containing a random string of alpha-numeric characters and symbols that are to be used to encrypt and decrypt redirection tokens.
token A random string of alpha-numeric characters and symbols that are to be used to encrypt and decrypt redirection tokens. The token may not start with a slash.
Defaults
None.
Notes
1) The same key must be used by all nodes within a cluster.
2) Specifying a secret key automatically enables HTTPS to HTTP conversion.
Example
http.secretkey 1agq45jkl3400vfghtqzz963
http.selfhttps2http {no | yes}
Function
Specify whether or not a server may redirect an HTTPS connection to itself using HTTP plus a security token.
Parameters
no A server should continue to use HTTPS for all communications. The word false and the number 0 are synonyms.
yes A server should convert an HTTPS session to an HTTP session by redirecting the client to itself using HTTP plus a security token. The word true and the number 1 are synonyms. You must also specify the secretkey directive.
Defaults
http.selfhttps2http no
Notes
1) This option is meant to control the level of data privacy that is desired. Normally, HTTPS connections are converted to HTTP connections after authentication information is extracted from the HTTP stream. This greatly reduces overhead as no data past the authentication stage has to be encrypted.
2) When an HTTPS connection is converted to an HTTP connection, the redirection includes a security token encrypted with the key specified by with the secretkey directive. The HTTP connection is only accepted if the token can be decrypted using the same key.
Example
http.selfhttps2http yes
http.secxtractor path [parms]
Function
Specify the location of the specialized authentication information extractor plug-in.
Parameters
path The path to the shared library containing the plug-in.
parms The parameters to the shared library containing the plug-in.
Defaults
None.
Notes
1) A Security eXtractor plug-in is a component that can be loaded at initialization time in order to provide specialized processing to the certificate passed by the client. Normally, all authentication information comes from the standard part of the client certificate and any extensions are ignored. A Security eXtractor can be used to extract other information from the certificate or any of its extensions. This information is then passed along and may be used for other authorization functions within XRootD.
2) The typical case for which a security extractor library is needed is to extract the extended VO information from a Grid client’s certificate.
3) A general purpose VO Security eXtractor plug-in is available with the gsi package that can be used with HTTPS. See the explanation of the VOMS plug-in in the security reference.
4) The secxtractor plug-in requires that HTTPS be enabled with certificate verification (i.e. xrd.tlsca or http.cadir or http.cafile be specified).
Example
http.secxtractor /usr/lib64/libXrdSecgsiVOMS.so
http.cadir path
Function
Specify the directory containing the CA certificates (see the cafile directive as an alternative).
Parameters
path The path to the directory.
Defaults
None.
Notes
1) Replace this directive with xrd.tlsca to configure all protocols to use the same certificate directory.
2) All of the certificates in the directory must be in a format that is recognized by the version of OpenSSL is being used.
3) If the certificate information is contained in a single file, you should either use the xrd.tlsca certfile (preferable) or the http.cafile directive.
Example
http.cadir /etc/grid-security/certificates
http.cafile path
Function
Specify the file containing the CA certificates.
Parameters
path The path to the file.
Defaults
None.
Notes
1) Replace this directive with xrd.tlsca certfile to configure all protocols to use the same certificate file.
2) All of the certificates in the file must in a format that is recognized by the version of OpenSSL is being used.
3) If certificates are contained in multiple files you should use the xrd.tlsca certdir (preferable) or the http.cadir directive.
Example
http.cafile /etc/myCA.pem
http.cert path
Function
Specify the file containing the x.509 certificate that the server must use.
Parameters
path The path to the file.
Defaults
None.
Notes
1) Replace this directive with xrd.tls to configure all protocols to use the same certificate.
2) The certificate must be in PEM format.
3) See the related http.key directive to specify the location of the private key.
Example
http.cert /etc/grid-security/hostcert.pem
http.cipherfilter ciphers
Function
Specify the allowed ciphers for transport layer security (TLS).
Parameters
ciphers
A list of colon separated ciphers that are allowed to be used.
Defaults
For OpenSSL versions greater than 1.0.2 the ciphers recommended by mozilla.org version 5.4 guidelines are used. These are strict ciphers. For older OpenSSL versions, generic ciphers are used for compatibility reasons.
Notes
1) Replace this directive with xrd.tlscipher to configure all protocols to use the same ciphers.
2) The http.cipherfilter directive is provided in cases where a default cipher has been shown to be insecure and should be removed. In this case, you need to specify all of the ciphers less the one you wish to eliminate.
Example
http.cipherfilter
ALL:!LOW:!EXP:!MD5:!MD2
http.key path
Function
Specify the file containing the x.509 private key that the server must use.
Parameters
path The path to the file.
Defaults
None.
Notes
1) Replace this directive with xrd.tls to configure all protocols to use the same certificate key.
2) The key must be in PEM format.
3) See the related http.cert directive to specify the location of the server’s certificate.
4) Specifying a key without specifying a certificate location is inconsistent and causes the key specification to be ignored along with a warning message.
Example
http.key /etc/grid-security/hostkey.pem
http.embeddedstatic {no | yes}
Function
Specify where CSS template and logo is to come from for formatted listings.
Parameters
no The CSS template and logo information must be over-ridden by another file containing such information. The word false and the number 0 are synonyms.
yes An internal memory-based CSS template and logo should be used. The word true and the number 1 are synonyms.
Defaults
http.embeddedstatic yes
Notes
1) Using the default memory-based CSS template and logo provide much better performance and makes the setup much simpler.
2) If you need to use a custom style sheet, significant performance gains can be achieved by preloading the style sheet file using the staticpreload directive.
Example
http.embeddedstatic yes
http.exthandler name path [token]
Function
Specify the location of the TCP connection statistics monitoring plug-in.
Parameters
name a 1- to-16 character unique name identifying the handler.
path The absolute path to the shared library that contains an implementation of the handler.
token An optional parameter to be passed to the plug-in object creation function. Typically, this is the name of a configuration file.
Defaults
None.
Notes
1) Each exthandler is invoked for every HTTP request to allow special handling for certain requests.
2) No more than 4 handlers may be loaded.
Example
http.exthandler mhandler17 /opt/http/lib/libExtHndlr.so
http.header2cgi hdrkey cgikey
Function
Specify which headers are to be promoted to cgi information and appended to the incoming url.
Parameters
hdrkey the header key that is to be promoted to cgi information.
cgirkey the cgi key name that the promoted header should have.
Defaults
None.
Notes
1) Normally, header information is internally processed and not made available to other plug-ins. The header2cgi directive allows you to pass on header information to external plug-ins via the incoming url by promoting the header payload to a cgi element.
2) Assuming xyzzy is the payload of header with a key of auth, the example shown below would promote the header by appending “authz=xyzzy” to the incoming url as cgi information before it is passed to other system components. This essentially makes the header visible outside of the http plug-in.
3) The header2cgi directive is meant to be used for non-http plug-ins that wish to consider specific information sent via the http protocol.
Example
http.header2cgi auth authz
http.listingdeny {no | yes}
Function
Specify whether or not directory listings are allowed.
Parameters
no Directory listings are not allowed. The word false and the number 0 are synonyms.
yes Directory listings are allowed. The word true and the number 1 are synonyms.
Defaults
http.listingdeny no
Notes
1) In a clustered environment a listing of a directory via HTTP only lists the directory of some arbitrary server in the cluster. Since files are scattered across all of the servers in the cluster; this likely produces an incomplete listing. You may wish to deny directory listings to avoid confusion.
2) Alternatively, you can redirect directory listings to a special node that can produce a composite listing using all nodes in the cluster via the listingredir directive.
3) Note that the XRootD based xrdfs command automatically produces a composite listing.
Example
http.listingdeny yes
http.listingredir desturl
Function
Specify the node to which to redirect clients requesting a directory listing.
Parameters
desturl The redirection URL to use when a directory listing is requested.
Defaults
None.
Notes
None.
Example
http.listingredir http://hostwhichprovideslistings:80/
http.staticpreload url path
Function
Preload a static resource file into memory.
Parameters
url The URL naming a static resource (e.g. style sheet or icon).
path The path to the local file containing the resource.
Defaults
None.
Notes
1) Static resources named in a URL must start with “/static” in order to be recognized.
2) The contents of the static resource contained in path may not exceed 64K. If it does, it is truncated to 64K.
3) Typical static resources are URL resources ending with “.css” and “.ico”.
4) This directive is ineffective if the staticredir directive is specified.
Example
http.staticpreload http://static/mycss.css /etc/mycss
http.staticredir newurl
Function
Preload a static resource file into memory.
Parameters
newurl The URL the client is to be redirected to when requesting a non-local or unsupported static resource.
Defaults
None.
Notes
1) The staticredir directive is only effective when a) embeddedstatic processing is disabled, or b) the resource is neither a content style sheet nor an icon.
Example
http.staticredir http://althost/
http.trace [-]option [ [-]option ] [• • •]
option: all | debug | none | off | request | response
Function
Specify execution tracing options.
Parameters
option The tracing level. One ore more options may be specified. The specifications are cumulative and processed left to right. Each option may be optionally prefixed by a minus sign to turn off the setting. Valid options are:
all selects all possible trace levels
debug traces internal activities for debugging purposes
none traces nothing
off a synonym for none
request traces client request information
response traces request response information
Defaults
Tracing is disabled.
Notes
1) All tracing is enabled when the daemon is invoked with the –d option.
2) All previous trace settings are discarded when none or off is encountered.
3) Tracing seriously degrades server performance. Use this directive only for debugging purposes.
Example
http.trace all -debug
14 March 2005
· Remove documentation on local redirection mode.
· Remove documentation of –s command line option.
· Add ‘-t’ option to the StartXRD documentation.
· Significantly change the port directive, adding “port any” and “if”.
· Discuss using “port any” mode.
26 April 2005
· Further clarified the xrootd monitor flush parameter.
1 June 2005
· Added description of conditional directives (if-fi).
· Added description of the –n command line option.
· Fully explain which run-time files are created.
· Deprecate –r, –t, and –y command line options.
· Deprecate the XRDMODE variable and remove the description of the XRDTYPE variable in the StartXRD.cf script.
· Remove extraneous options from the StartXRD script.
1 Aug 2005
· Document administrative interface portal socket.
· Add file size to open monitor record.
16 Aug 2005
· Add authentication mapping (a-record) to monitoring data.
6 Jan 2006
· Document the -b and -R command line options.
· Document how to independently bind different port numbers to available protocols.
25 Jan 2006
· Add max option to chksum directive.
22 March 2006
· Add exec condition to if/else/fi.
28 February 2007
· Cleaned up documentation relative to role directive and all prefix modifier.
· Documented the xrootd.redirect directive.
· Removed the xrd.connections directive.
· Placed most xrd directives in esoteric status.
28 March 2007
· Move conditional directives to a separate manual.
· Indicate the adminpath now is configured via the all prefix.
· Documented the xrd wan network and protocol directive option.
· Indicate that the xrootd export directive is configured via the all prefix and accepts oss options.
01 October 2007
· Document the locate option of the redirect directive.
01 January 2008
· Remove references to olbd.
01 February 2008
· General clean-up.
11 April 2008
· Document staging (‘s’) monitor record.
29 May 2008
· Document the xrootd async nosf option.
21 July 2008
· Document the xrd network [no]dnr option.
· Document the xrd async minsfsz option.
6 March 2009
· Document the xrootd monitor stage option.
22 June 2009
· Document the xrd.report directive.
7 July 2009
· Document the mpxstats command for monitoring.
· Document the summary variables.
·
17 March 2010
· Document the timeout hail and kill options.
· Document the pid file creation and the pidpath directive.
8 March 2011
· Document the –s command line option.
· Minor editorial changes.
24 May 2011
· Document the auth option in the xrootd.monitor directive.
31 May 2011
· Change the xrootd.chksum directive to support native checksums. Additional wording added explaining native checksums.
29 June 2011
· Document the rbuff and redir options on the xrootd.monitor directive to support redirection monitoring.
27 September 2011
· Document the io flush option on the xrootd.monitor directive.
-------------- Release 3.1.0
10 October 2011
· Document the iov, migr, and purge options on the xrootd.monitor directive.
2 November 2011
· Update documentation on the xrootd.redirect directive. It now accepts additional file operations (chksum and trunc), open targets (previously undocumented feature), and ENOENT targets.
3 December 2011
· Remove the migr, purge and stage options from the xrootd.monitor directive. These have been moved to the frm.all.monitor directive.
· Document the new ident option on the xrootd.monitor directive.
12 December 2011
· Document the rnums option for the xrootd.monitor directive.
-------------- Release 3.2.0
-------------- Release 3.2.1
-------------- Release 3.2.2
-------------- Release 3.2.3
-------------- Release 3.2.4
21 September 2012
· Document the fstat option for the xrootd.monitor directive.
· Remove the rootd configuration section.
-------------- Release 3.2.5
22 October 2012
· Document the –S command line option and the all.sitename directive for specifying a monitoring site name.
-------------- Release 3.2.6
-------------- Release 3.2.7
15 December 2012
· Change the fstat sdv option to fstat ssq in the xrootd.monitor directive.
-------------- Release 3.3.0
-------------- Release 3.3.1
-------------- Release 3.3.2
-------------- Release 3.3.3
-------------- Release 3.3.4
-------------- Release 3.3.5
-------------- Release 3.3.6
11 February 2013
· Enhance the fslib directive to allow one to easily wrap one library with another.
23 February 2013 (IPV6 Introduction)
· Document the –I command line option.
· Document the cache option in the xrd.network directive.
12 August 2013
· Document the extended –k, –l and –z command line options.
· Document exported environment variables.
· Document the environment information file contents.
· General clean-up and better explanations.
2 December 2013
· Document the xrootd.diglib directive.
8 January 2014
· Document the routes option on the xrd.network directive.
· Document enhanced xrootd.redirect directive that can distinguish between public and private IP addresses.
18 February 2014
· Restrict the routes option on the xrd.network directive to prohibit auto-discovery of interface addresses as this may lead to choosing the wrong addresses.
27 March 2014
· Document how to enable HTTP and HTTPS protocols.
· Redesign the routes option on the xrd.network directive to cover the most common case.
6 August 2014
· Document the core option in the xrd.sched directive.
· Document how to export object identifier names via the all.export directive.
8 September 2014
· Document that TCP keepalive is now the default setting.
· Add a nokeepalive option and a kaparms option to the xrd.network directive.
· Indicate the use option in xrd.network accepts one or two interface names.
· Minor corrections to HTTP section.
· Document the http.mapfile, http.staticredir, http.staticpreload, and the http.trace directives.
27 September 2014
· Document multiple checksum support via the xrootd.chksum directive.
· Document the default option on the xrootd.seclib directive.
· Document the –L command line option.
26 November 2014
· Document the version option in the xrootd.fslib directive.
10 February 2015
· Document how to pass command line arguments to plug-ins.
· Document how to enable digFS but prevent its use until needed.
25 November 2015
· Explain the side-effects of the –s command line option on the placement of the environmental file.
15 April 2016
· Document log file plug-ins.
20 June 2016
· Document the cse parameter for logging plug-ins.
· Document the xrd.network [no]rpipa option.
10 March 2017
· Correct http.mapfile directive (it’s really gridmap).
20 May 2017
· Document the xrootd.fsoverload directive.
27 October 2017
· Document the http.header2cgi directive.
19 December 2018
· Document the xrd.tls and the xrootd.tls directives.
31 May 2019
· Document the xrd.tlsca directive.
21 June 2019
· Document the dyndns option in the xrd.network directive.
· Remove all references to the wan option. It is no longer supported.
· Documented the xrd tls network, port, and protocol directive option.
· Correct spelling of xrootd.async segsize option (was segsz).
18 October 2019
· Document the preferred version of the xrootd.fslib directive.
21 December 2019
· Document the interaction between the dyndns and cache option of the xrd.network directive.
31 March 2020
· Document the xrd.tlsciphers directive.
11 April 2020
· General cleanup with better descriptions.
· Document the ccm, pfc, and tcpmon options of the xrootd.monitor directive.
· Document the xrd.tcpmonlib directive.
· Document the http.cipherfilter and http.exthandler directives.
14 April 2020
· Document the -a and -A command line options.
· Document the xrd.homepath and xrd.tcpmonlib directives.
24 April 2020
· Document the xrd.trace directive’s tls, tlsctx, tlsio, and tlssok options.
· Document the xrd.tlsca refresh option.
· Document the detail and cache options of the xrd.tls directive.
· Document the http.httpsmode directive.
28 April 2020
· Remove the xrd.tls directive’s cache option.
· Document the xrd.tlsca directive’s crlcheck and [no]proxies options.
· Document changes in the http.httpsmode directive where enable has changed to manual.
· Re-factor the http protocol section and describe the deprecated directives.
· Document how the TLS session cache has, by default, turned off.
· Document the xrootd.tlsreuse directive.
5 May 2020
· Add directives by category for xrd and xrootd section.
· Cleanup HTTP section.
[1] Authentication is plug-in based and any implemented authentication protocol may be specified as aprot.