Jex’s Note

XFS - Folder Quota

介紹

xfs 是一種 file system 的 format, 像 ext4 一樣, 不過它特別的地方在於它可以方便的對目錄做 quota 限制

Ubuntu 安裝 :

sudo apt-get install xfsprogs xfsdump

Set quota :

Make the filesystem and mount it:

mount /dev/sda5 /mnt/xfs -o pquota

Create a project named “project1”, which is the “/mnt/xfs/test1” tree:

mkdir test1     #on /mnt/xfs
echo "11:/mnt/xfs/test1" >> /etc/projects
echo "project1:11" >> /etc/projid
xfs_quota -x -c "project -s project1" /dev/sda5

Set the tree quota to 2 MB:

xfs_quota -x -c "limit -p bsoft=1m bhard=2m project1" /dev/sda5

或直接指定project id

xfs_quota -x -c "limit -p bsoft=1m bhard=2m 1" /dev/sda5

也可以使用projects id指定 ex : “limit -p bsoft=1m bhard=2m 11”

That’s it.. Now let’s make some tests:

dd if=/dev/zero of=/mnt/xfs/test1/aaa count=10 bs=1024k

And there’s also a nice report! (looks nicer with a fixed-width console font)

xfs_quota -x -c "report /dev/sda5"

將一個quota綁定多個資料夾:

在/etc/projects設定:

7:/jex_ugc/jexstorage/default/jex_lin
7:/jex_ugc/jexstorage/default/hun

xfs report 中 id=7 的 used欄位會將jex_lin及hun資料夾使用空間加總,超過quota限制的話就會將兩個資料夾鎖住

Set Unlimited Quota

sudo xfs_quota -x -c "limit -p bsoft=0m bhard=0m project1" /dev/mapper/jex--vg-jex--lv

Remove quota :

執行清除project quota指令

$ sudo xfs_quota -xc "project -C 1" /dev/mapper/jex--vg-jex--lv
Clearing project 1 (path /jex_ugc/jexstorage/test)...
Processed 1 (/etc/projects and cmdline) paths for project 1 with recursion depth infinite (-1).

刪除資料夾與 /etc/projects 及 /etc/projid 的對應資料

  • 如果要將/etc/projects其中一個project id的資料夾做更改一定要先執行clear quota再改,不然如果舊的資料夾還存在的話會變成限制成兩個資料夾
  • 要先執行clear quota再刪除資料夾否則會發生找不到資料夾的錯誤

      jex@jex:/jex_ugc/jexstorage$ sudo xfs_quota -xc "project -C 1" /dev/mapper/jex--vg-jex--lv
      xfs_quota: cannot find mount point for path `/jex_ugc/jexstorage/test': No such file or directory
      Processed 0 (/etc/projects and cmdline) paths for project 1 with recursion depth infinite (-1).
    

其他功能:

To suspend (i.e. freeze) an XFS file system, use:

xfs_freeze -f /mount/point

To unfreeze an XFS file system, use:

xfs_freeze -u /mount/point

command :

  • -c cmd xfs_quota commands may be run interactively (the default) or as arguments on the command line. Multiple -c arguments may be given. The commands are run in the sequence given, then the program exits.
  • -p prog Set the program name for prompts and some error messages, the default value is xfs_quota.
  • -x Enable expert mode. All of the administrative commands (see the ADMINISTRATOR COMMANDS section below) which allow modifi - cations to the quota system are available only in expert mode.
  • -d project Project names or numeric identifiers may be specified with this option, which restricts the output of the individual xfs_quota commands to the set of projects specified. Multiple
  • -d arguments may be given.

Command details :

xfs_quota> jex@jex:/jex_ugc/jexstorage/test$ sudo xfs_quota -x
xfs_quota> ?
df [-bir] [-hn] [-f file] -- show free and used counts for blocks and inodes
disable [-gpu] [-v] -- disable quota enforcement
dump [-gpu] [-f file] -- dump quota information for backup utilities
enable [-gpu] [-v] -- enable quota enforcement
help [command] -- help for one or all commands
limit [-gpu] bsoft|bhard|isoft|ihard|rtbsoft|rtbhard=N -d|id|name -- modify quota limits
off [-gpu] [-v] -- permanently switch quota off for a path
path [N] -- set current path, or show the list of paths
print -- list known mount points and projects
project [-c|-s|-C|-d <depth>|-p <path>] project ... -- check, setup or clear project quota trees
quit -- exit the program
quot [-bir] [-gpu] [-acv] [-f file] -- summarize filesystem ownership
quota [-bir] [-gpu] [-hnNv] [-f file] [id|name]... -- show usage and limits
remove [-gpu] [-v] -- remove quota extents from a filesystem
report [-bir] [-gpu] [-ahnt] [-f file] -- report filesystem quota information
restore [-gpu] [-f file] -- restore quota limits from a backup file
state [-gpu] [-a] [-v] [-f file] -- get overall quota state information
timer [-bir] [-gpu] value -d|id|name -- get/set quota enforcement timeouts
warn [-bir] [-gpu] value -d|id|name -- get/set enforcement warning counter

Use 'help commandname' for extended help.

測試紀錄

測試刪除資料夾再新增回來同樣的名稱,quota限制是否還在?

將test資料夾刪除,這時看report,會有之前設定的記錄(只要projects及projid沒刪除,都會顯示project{id}的狀態)

Project ID       Used       Soft       Hard    Warn/Grace
---------- --------------------------------------------------
project1            0       1024       2048     00 [--------]

新增test資料夾

複製10m的檔案,不受quota 2m的影響

jex@jex:/jex_ugc/jexstorage$ dd if=/dev/zero of=/jex_ugc/jexstorage/test/aaa count=10 bs=1024k
10+0 records in
10+0 records out
10485760 bytes (10 MB) copied, 0.00780022 s, 1.3 GB/s

report顯示的Used也不正常 (其實是10m但report那還是顯示0)

jex@jex:/jex_ugc/jexstorage$ du
10240   ./test
10240   .
jex@jex:/jex_ugc/jexstorage$ sudo xfs_quota -xc "report /dev/mapper/jex--vg-jex--lv"
Project quota on /jex_ugc/jexstorage (/dev/mapper/jex--vg-jex--lv)
                               Blocks
Project ID       Used       Soft       Hard    Warn/Grace
---------- --------------------------------------------------
project1            0       1024       2048     00 [--------]

解決方法: 重新將test與project{id}綁定,再觀察report結果,Used那正常顯示

jex@jex:/jex_ugc/jexstorage$ sudo xfs_quota -xc "project -s project1" /dev/mapper/jex--vg-jex--lv
Setting up project project1 (path /jex_ugc/jexstorage/test)...
Processed 1 (/etc/projects and cmdline) paths for project project1 with recursion depth infinite (-1).
jex@jex:/jex_ugc/jexstorage$ sudo xfs_quota -xc "report /dev/mapper/jex--vg-jex--lv"
Project quota on /jex_ugc/jexstorage (/dev/mapper/jex--vg-jex--lv)
                               Blocks
Project ID       Used       Soft       Hard    Warn/Grace
---------- --------------------------------------------------
project1        10240       1024       2048     00 [--none--]

再copy 10m進去,quota就把它擋掉了,得到預期結果

jex@jex:/jex_ugc/jexstorage$ dd if=/dev/zero of=/jex_ugc/jexstorage/test/bbb count=10 bs=1024k
dd: opening /jex_ugc/jexstorage/test/bbb: Disk quota exceeded

就一個已存在的project id指向到一個新建立的資料夾(abc),會發生什麼情形?

2:/jex_ugc/jexstorage/default/test

改成:

2:/jex_ugc/jexstorage/default/abc
  • 綁定後再設quota, 在report顯示出來的used其實是舊的資料夾加上新的資料夾的使用容量,如果超過使用容量就會變成兩個都鎖

解決辦法: XFS在將路徑與project id做綁定時,極大的可能是以inode做綁定,所以只要有綁定都會記錄在xfs資料庫中,所以即使將/etc/projects刪除,但只要曾經綁過就會被紀錄下來,而將quota限制在同一個project id曾綁定過所有資料夾使用的總合

  • 最好的做法是在將/etc/projects變更前先將舊資料夾做Clear Quota,讓舊資料夾跟project id沒有關係後,再變更/etc/projects及將新資料夾與project id綁定
  • 或者是直接刪除該舊資料夾讓xfs找不到它的inode,但可能潛在如果後來生成的資料夾被指派到一樣的indoe,造成xfs也對他做quota 限制

在一個有設定quota的資料夾下將檔案搬移出的問題

假設資料夾 a 有設定quota,project id 為 8 (以下簡稱 p8),裡面有一個檔案test.txt 它的 inode 為 1130

[情況1]

將 a 的 test.txt 檔案搬移到 未設定 quota 的資料夾 b,會產生問題是 p8 仍然還是指向 inode 1130,如果將 a 的 quota 給註銷掉換成資料夾 c,則 p8 的新主人雖然是資料夾 c ,但查看 report 會發現 used 那邊會出錯,因為 inode 1130 這個檔案的使用量會被算到資料夾 c (實際上檔案是在 b 那邊) ,解決方法只要把 inode 1130 的檔案刪除

[情況2]

將 a 的 test.txt 檔案搬移到 已設定 quota 的資料夾 b,就不會發生 p8 指向 inode 錯亂的問題了,因為搬過去時 inode也變重新指派

用 inode 反查

sudo xfs_db -xr -c 'inode 16777350' -c p /dev/sda5

http://xfs.org/docs/xfsdocs-xml-dev/XFS_User_Guide/tmp/en-US/html/ch08s16.html

備註 :

  • 現代的文件系統如JFS和XFS能夠動態地增加inode,因此不會用盡inode。(From: inode wiki)
  • 動態變更上限值 : 直接執行步驟3重新指定上限值即可
  • 步驟2 “11:/mnt/xfs/test1”中的編號11只可以使用數字,並且 不能超過65535 ,超始值 : 1
  • 當資料夾rename時,/etc/projects更改路徑,原有的quota限制也都會對應到
  • About soft and hard : When a user exceeds the soft limit, the timer is enabled. Any time the quota drops below the soft limits, the timer is disabled. If the timer pops, the particular limit that has been exceeded is treated as if the hard limit has been reached, and no more resources are allocated to the user. The only way to reset this condition, short of turning off limit enforcement or increasing the limit, is to reduce usage below quota. Only the superuser (i.e. a sufficiently capable process) can set the time limits and this is done on a per filesystem basis.
  • 中文檔名:

    1. Max. filename length : 255bytes
    2. 85個中文 => OK!
    3. 80個中文+15英文 => OK!
    4. 80個中文+15數字 => OK!
    5. 80個中文+16個英文或數字 => error!
  • 符號:

    1. 允許符號 : ~!@#$%^&*()_+ \‘ ”
    2. 除了 / 不允許其他都OK

其他限制方式:

User Limits

limits xfs_quota -x -c 'limit bsoft=1000m bhard=1200m username'
limits xfs_quota -x -c 'limit isoft=500 ihard=700 username'

Group Limits.

xfs_quota -x -c 'limit -g bsoft=1000m bhard=1200m groupname'
Project Limits

參考:

http://xfs.org/docs/xfsdocs-xml-dev/XFS_User_Guide/tmp/en-US/html/index.html http://www.held.org.il/blog/2008/07/per-directory-quota-not-a-dream/ http://dell9.ma.utexas.edu/cgi-bin/man-cgi?xfs_quota+8 http://docs.fedoraproject.org/en-US/Fedora/14/html/Storage_Administration_Guide/xfsfreeze.html

Comments