PNFS information on cmspnfs1.fnal.gov

Page generated: Sat Nov 21 03:08:20 CST 2009



TOP sources of PNFS traffic in last 60 seconds:
0% 	 lhc04n02 11
0% 	 cmssrv33 180
1% 	 cmssrv45-phedex-ddt 2460
1% 	 cmswn 2659
7% 	 cmsstor 10898
88% 	 localhost-PnfsManager 128697
144905 PNFS tcpdump responses or 2415/sec

TOP worker node source of PNFS traffic in last 60 seconds:
0% 	 cmswn1393 20
0% 	 cmswn1405 20
0% 	 cmswn958 20
0% 	 cmswn993 20
0% 	 cmswn1747 23

TOP storage node source of PNFS traffic in last 60 seconds:
0% 	 cmsstor173 890
0% 	 cmsstor245 1179
0% 	 cmsstor96 1239
1% 	 cmsstor51 2444
1% 	 cmsstor78 2473

---------------------------------------------------------------------------------------------------------------------------

PNFS statistics during last 60 seconds:
0% 	 null 0
95% 	 getattr 129967
0% 	 setattr 103
0% 	 root 0
1% 	 lookup 2081
0% 	 readlink 0
2% 	 read 3320
0% 	 wrcache 0
0% 	 write 103
0% 	 create 107
0% 	 remove 2
0% 	 rename 0
0% 	 link 0
0% 	 symlink 0
0% 	 mkdir 0
0% 	 rmdir 0
0% 	 readdir 0
0% 	 fsstat 156
135839 PNFS operations or 2263/sec

---------------------------------------------------------------------------------------------------------------------------

UPTIME:
 03:09:23 up 50 days, 12:37,  1 user,  load average: 0.35, 0.29, 0.59

---------------------------------------------------------------------------------------------------------------------------

VMSTAT 60 2:
procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 2  0      0 3524668 177228 27696380    0    0   144   218    1    0  4  9 87  1  0
 0  0      0 3521792 177260 27695764    0    0    13   954 2948 25489  1  3 95  0  0

---------------------------------------------------------------------------------------------------------------------------

atop -c 60 2:

ATOP - cmspnfs1           2009/11/21  03:09:20               60 seconds elapsed
PRC | sys  16.91s | user   8.05s | #proc    315 | #zombie    2 | #exit    878 |
CPU | sys     23% | user     14% | irq       9% | idle    752% | wait      1% |
CPL | avg1   0.35 | avg5    0.29 | avg15   0.59 | csw  1803661 | intr  203082 |
MEM | tot   31.4G | free    3.4G | cache  26.4G | buff  173.1M | slab  713.8M |
SWP | tot   64.0G | free   64.0G |              | vmcom   3.0G | vmlim  64.6G |
DSK |         sdb | busy      1% | read      85 | write   3501 | avio    0 ms |
NET | transport   | tcpi    8170 | tcpo    8546 | udpi  275161 | udpo  275224 |
NET | network     | ipi   290962 | ipo   283771 | ipfrw      0 | deliv 288830 |
NET | lo     ---- | pcki  261658 | pcko  261658 | si 5333 Kbps | so 5333 Kbps |

  PID  CPU COMMAND-LINE                                                  1/1   
29338  10% /usr/java/jdk1.6.0_16/bin/java -server -Xmx2048m -XX:MaxDirectMemory
 4535   5% /usr/sbin/tcpdump -i lo port 2049 and udp
 7982   5% ./pnfsd
 7980   4% ./pnfsd
 7981   4% ./pnfsd
 7979   4% ./pnfsd
 7902   4% ./dbserver cms11
 7866   1% ./dbserver admin
 7900   1% postgres: enstore cms11 [local] BIND                   
 4532   1% /usr/sbin/tcpdump -i eth0 port 2049 and udp
 8316   1% kipmi0

---------------------------------------------------------------------------------------------------------------------------

ipcs

------ Shared Memory Segments --------
key        shmid      owner      perms      bytes      nattch     status      
0x0052e2c1 0          enstore   600        2247483392 66                      
0x00001122 32769      root      600        1480       26                      
0x00000000 65538      root      700        12288      2                       
0x00000000 98307      root      700        12288      22                      
0x00000000 131076     root      700        12288      22                      
0x00000000 163845     root      700        12288      22                      
0x00000000 196614     root      700        12288      22                      
0x7a01809b 229383     zabbix    666        1099804    6                       

------ Semaphore Arrays --------
key        semid      owner      perms      nsems     
0x0052e2c1 3538944    enstore   600        17        
0x0052e2c2 3571713    enstore   600        17        
0x0052e2c3 3604482    enstore   600        17        
0x0052e2c4 3637251    enstore   600        17        
0x0052e2c5 3670020    enstore   600        17        
0x0052e2c6 3702789    enstore   600        17        
0x0052e2c7 3735558    enstore   600        17        
0x002fa327 3768327    root      666        2         
0x00000000 3801096    root      600        1         
0x00000000 3833865    root      700        2         
0x00000000 3866634    root      700        2         
0x00000000 3899403    root      700        2         
0x00000000 3932172    root      700        2         
0x00000000 3964941    root      700        2         
0x00000000 3997710    root      700        2         
0x00000000 4030479    root      700        2         
0x00000000 4063248    root      700        2         
0x00000000 4096017    root      700        2         
0x00000000 4128786    root      700        2         
0x00000000 4161555    root      700        2         
0x00000000 4194324    root      700        2         
0x00000000 4227093    root      700        2         
0x00000000 4259862    root      700        2         
0x00000000 4292631    root      700        2         
0x00000000 4325400    root      700        2         
0x00000000 4358169    root      700        2         
0x00000000 4390938    root      700        2         
0x00000000 4423707    root      700        2         
0x00000000 4456476    root      700        2         
0x00000000 4489245    root      700        2         
0x00000000 4522014    root      700        1         
0x00000000 4554783    root      700        1         
0x00000000 4587552    root      700        1         
0x00000000 4620321    root      700        1         
0x00000000 4653090    root      700        1         
0x00000000 5013539    root      600        1         
0x00000000 5046308    root      600        1         
0x00000000 5079077    root      600        1         
0x00000000 5111846    root      600        1         
0x00000000 5144615    root      600        1         
0x00000000 5177384    root      600        1         
0x00000000 5210153    root      600        1         
0x00000000 5242922    root      600        1         
0x7a01809b 5275691    zabbix    666        1         

------ Message Queues --------
key        msqid      owner      perms      used-bytes   messages    


---------------------------------------------------------------------------------------------------------------------------

time df /pnfs/fs:
Filesystem           1K-blocks      Used Available Use% Mounted on
localhost:/fs           400000     80000    284000  22% /pnfs/fs

real	0m0.001s
user	0m0.000s
sys	0m0.004s

---------------------------------------------------------------------------------------------------------------------------

PNFSMANAGER INFO:
In 63 sec 1352 transactions  = 21.460317 trans/sec


    dCache Admin (VII) (user=enstore)


[cmssrv32.fnal.gov] (local) enstore > cd PnfsManager
[cmssrv32.fnal.gov] (PnfsManager) enstore > info
$Revision: 11501 $
NameSpace Provider: 
$Revision: 11330 $
 Server         : fnal.gov(cmspnfs1.fnal.gov)
 RealMountPoint : 000000000000000000001040 /pnfs/fs
       VirtualMountId : 000000000000000000001040
      VirtualPnfsPath : /root/fs
     VirtualLocalPath : /pnfs/fs
    VirtualGlobalPath : /pnfs/fnal.gov

CacheLocation Provider: 
$Id: SQLNameSpaceProvider.java,v 1.19 2007-08-22 12:24:38 tigran Exp $
StorageInfo Provider: 
$Revision: 11330 $
 Server         : fnal.gov(cmspnfs1.fnal.gov)
 RealMountPoint : 000000000000000000001040 /pnfs/fs
       VirtualMountId : 000000000000000000001040
      VirtualPnfsPath : /root/fs
     VirtualLocalPath : /pnfs/fs
    VirtualGlobalPath : /pnfs/fnal.gov

Store Default AL and RP: false

Threads (4) Queue
    [0] 0
    [1] 0
    [2] 0
    [3] 0

Thread groups (1)
    [0] 0

Cache Location Queues
    [0] 0
    [1] 1
    [2] 0
    [3] 0

Statistics:
  addCacheLocation                   4936613       553         0
  clearCacheLocation                 4554516         0         0
  getCacheLocations                 20070904         0     29293
  createDirectory                       4236        62         0
  createEntry                        2520784       822         0
  deleteEntry                          81453      2569         0
  getStorageInfo                    24099597   6352740   1295369
  setStorageInfo                           0         0         0
  setLength                          2518328      6056         0
  getCacheStatistics                       0         0         0
  updateCacheStatistics                    0         0         0
  rename                                2309         0         0
  setLevelData                             0         0         0
  fileFlushed                         636077       114         0
  mapPath2Id                          428030         3         0
  mapId2Path                               0         0         0
  getMetadataInfo                    4192361    540373     35336
  setMetadataInfo                          0         0         0
[cmssrv32.fnal.gov] (PnfsManager) enstore > ..
[cmssrv32.fnal.gov] (local) enstore > 
[cmssrv32.fnal.gov] (local) enstore > logoff
dmg.util.CommandExitException: (0) Done
[cmssrv32.fnal.gov] (local) enstore > 



Legal Notices