vastnfs-ctl

This helper script is installed with the package, and can assist in various situations.

The usage of this script is optional in most cases.

Status

Observe the status of the loaded NFS services and kernel modules

vastnfs-ctl status

Example output:

version: v4.0-pre4-11-ge520dd8eb9cb
kernel modules: sunrpc rpcrdma compat_nfs_ssc lockd nfs_acl auth_rpcgss nfsd rpcsec_gss_krb5 nfs nfsv3 nfsv4
services: rpcbind.socket rpcbind
rpc_pipefs: /var/lib/nfs/rpc_pipefs

Reload

The following command will try to reload the NFS stack. It will first try to unmount all NFS file systems, and then reload all the necessary kernel modules.

vastnfs-ctl reload

Example output:

vastnfs-ctl: stopping service rpcbind.socket
vastnfs-ctl: umounting fs /var/lib/nfs/rpc_pipefs
vastnfs-ctl: unloading kmod nfsv4
vastnfs-ctl: unloading kmod nfsv3
vastnfs-ctl: unloading kmod nfs
vastnfs-ctl: unloading kmod rpcsec_gss_krb5
vastnfs-ctl: unloading kmod nfsd
vastnfs-ctl: unloading kmod auth_rpcgss
vastnfs-ctl: unloading kmod nfs_acl
vastnfs-ctl: unloading kmod lockd
vastnfs-ctl: unloading kmod compat_nfs_ssc
vastnfs-ctl: unloading kmod rpcrdma
vastnfs-ctl: unloading kmod sunrpc
vastnfs-ctl: loading kmod sunrpc
vastnfs-ctl: loading kmod rpcsec_gss_krb5
vastnfs-ctl: loading kmod rpcrdma
vastnfs-ctl: loading kmod nfsv4
vastnfs-ctl: loading kmod nfsv3
vastnfs-ctl: loading kmod nfsd
vastnfs-ctl: loading kmod nfs_acl
vastnfs-ctl: loading kmod nfs
vastnfs-ctl: loading kmod lockd
vastnfs-ctl: loading kmod compat_nfs_ssc
vastnfs-ctl: loading kmod auth_rpcgss
vastnfs-ctl: mounting fs /var/lib/nfs/rpc_pipefs
vastnfs-ctl: starting service rpcbind.socket

Tracing

The vastnfs-ctl utility comes with a helper command for tracing the NFS stack. Please see a short tutorial on using vastnfs-ctl trace.

Show NFS Clients

Use vastnfs-ctl nfs-clients path to show the status of NFS client used by specific path. Use vastnfs-ctl nfs-clients to show status of all active NFS clients.

Example output:

NFS client 3000
	state: 0
	hostname: server.nfs-testing.com
	rpc_clnt: 3

hostname: The used NFS server rpc_clnt: The RPC client's ID used by the NFS client

Show RPC Clients Used by the NFS Protocol

Use vastnfs-ctl rpc-clients <path> to display the status of RPC clients for a specific path. Use vastnfs-ctl rpc-clients to display the status of all active RPC clients.

Note: This command shows RPC clients used by the NFS protocol only. Clients used by other protocols aren't included.

Example output:

rpc_client: 3 switch-2002
	sunrpc-id:	3
	RPC iostats version: 1.1  p/v: 100003/3 (nfs)
	xprt:	tcp 1012 1 2 0 0 6474 6473 0 77192 3579 128 26235 66718
		192.168.23.11, state: CONNECTED BOUND
		remote_port_idx: 0
		role: REG
		pci device:
		hardware gpus:
	remoteports_offset: 0 0
	spread reads: 0 writes: 0
	mdconnect: 0
	per-op statistics
	        NULL: 1 1 0 44 24 0 0 0 0
	     GETATTR: 0 0 0 0 0 0 0 0 0
	     SETATTR: 0 0 0 0 0 0 0 0 0
	      LOOKUP: 0 0 0 0 0 0 0 0 0
	      ACCESS: 0 0 0 0 0 0 0 0 0
	    READLINK: 0 0 0 0 0 0 0 0 0
	        READ: 0 0 0 0 0 0 0 0 0
	       WRITE: 0 0 0 0 0 0 0 0 0
	      CREATE: 0 0 0 0 0 0 0 0 0
	       MKDIR: 0 0 0 0 0 0 0 0 0
	     SYMLINK: 0 0 0 0 0 0 0 0 0
	       MKNOD: 0 0 0 0 0 0 0 0 0
	      REMOVE: 0 0 0 0 0 0 0 0 0
	       RMDIR: 0 0 0 0 0 0 0 0 0
	      RENAME: 0 0 0 0 0 0 0 0 0
	        LINK: 0 0 0 0 0 0 0 0 0
	     READDIR: 0 0 0 0 0 0 0 0 0
	 READDIRPLUS: 0 0 0 0 0 0 0 0 0
	      FSSTAT: 0 0 0 0 0 0 0 0 0
	      FSINFO: 0 0 0 0 0 0 0 0 0
	    PATHCONF: 0 0 0 0 0 0 0 0 0
	      COMMIT: 0 0 0 0 0 0 0 0 0

rpc_client: The unique ID of the RPC client and the associated transport switch. xprt: The brief status of transport/transports.

Show RPC Transport Switches

Use vastnfs-ctl rpc-switches <path> to show the status of RPC transport switches for a specific path. Use vastnfs-ctl rpc-switches to show the status of all active RPC transport switches.

Example output:

RPC switch-2002
        num_xprts=1
        num_active=1
        num_unique_destaddr=1
        queue_len=0
        xprt-2-tcp: CONNECTED   BOUND

queue_len: The number of currently allocated RPC tasks for this transport switch. xprt-2-tcp: The transport (or list of transports) in use, along with their current status (e.g., CONNECTED, BOUND).

Show RPC Transports

Use vastnfs-ctl rpc-transports <path> to show RPC transports associated with a specific path. Use vastnfs-ctl rpc-transports to show all active RPC transports used by the NFS protocol.

This command reads sysfs /sys/kernel/sunrpc/xprt-switches/*/xprt*/xprt_info and related fields for transports belonging to VastNFS mounts.

Example output:

xprt-2-tcp  CONNECTED   BOUND
        dstaddr=192.168.23.11
        binding_q_len=0
        sending_q_len=0
        pending_q_len=0
        backlog_q_len=0
        tasks_queuelen=0
        remote_port_idx=0
        reconnection_attempts=0
        transport=tcp

The first line shows the transport name (xprt-2-tcp) and current state flags (CONNECTED, BOUND, etc.). Queue depth lines come from sysfs xprt_info:

FieldMeaning
binding_q_lenTasks waiting for bind/rpcbind
sending_q_lenTasks waiting to send
pending_q_lenTasks in flight (awaiting reply)
backlog_q_lenTasks waiting for a free request slot
tasks_queuelenTotal tasks associated with this transport

When a transport is disconnected, xprt_info may report <closed> and queue fields are not shown.

For continuous monitoring, the Prometheus exporter exposes these queue depths as queue length gauges with xprt_id, netid, and addr labels. Switch-level queue_len from rpc-switches is not exported by the exporter.

Show RPC Transport Statistics

Use vastnfs-ctl rpc-transports-stats <path> to show per-operation RPC statistics for transports associated with a specific mount path. Use vastnfs-ctl rpc-transports-stats to show statistics for all active RPC transports used by VastNFS mounts.

This command reads the per-op stats section from debugfs /sys/kernel/debug/sunrpc/rpc_xprt/*/extra for transports belonging to VastNFS mounts. Rows where all counters are zero are omitted.

Example output:

xprt-2-tcp  addr=192.168.23.11:2049  CONNECTED BOUND
per-op stats
  op                     ops  trans   t/o         sent         recv    q_ms  rtt_ms  exe_ms   errs
  NULL                     1      1     0           44           24       0       0       0      0
  GETATTR                 12     12     0         1234         5678       1       2       1      0
  READ                   100    100     0      1048576      2097152      10      50      40      0

The header line shows the transport name, remote addr, and current state flags. The table columns map directly to the Prometheus exporter RPC counters. The q_ms column is cumulative queue time (milliseconds), not an instantaneous queue depth — see queue length gauges for live depths from rpc-transports.

For continuous monitoring, use the Prometheus exporter. It exposes these per-op counters with xprt_id, netid, addr, and op labels, plus per-transport queue depths and state flags from the same sysfs and debugfs sources as rpc-transports and rpc-transports-stats.

Show Cached NFS Inode State

Use vastnfs-ctl dump-meta-data <path> to show the inode state using the file's absolute path. Use vastnfs-ctl dump-meta-data :inode@mount_point to show the inode state by inode number and mount point. Use vastnfs-ctl dump-meta-data :inode@fsid to show the inode state by inode number and fsid, where fsid is the value from the third column of /proc/self/mountinfo.

Example output:

junk.fsxlog fh 0x010001010100000066f43ebcefa3000000000000
idx 0 mode 0100644 owner 0:0 flags 0x82:0xc nlink 1
size 285 state 0x100 version 1879990373217750775 count 0
atime 1750877260079976788ns mtime 1750877474684478199ns ctime 1750877474684478199ns
nfsi flags 0x4 cache validity 0x0 gencount 4942822 attrtimeo 12000
timestamp 4297125548 unlock_mtime 0ns jiffies 217558
access_cache 1 entryes, cached 0 pages flags 0x0

First line displays the file name and its NFS file handle.


Document generated on 2026.09.14 19:54 for v4.5.9, branch 4.5