BLOG UPDATE 08-NOV-2011 : After posting and handling comments I realize the title is drawing readers’ attention to the wrong thing. I am aware that ls(1) and du(1) can and should disagree about file size and space utilization because they are two different things (e.g., sparse files and non-file data associated with the file). The point of this blog was just to show that within microseconds (or less) ls(1) reported my file was zero bytes and du(1) reported it occupied nearly 300GB space in the file system. That is the “oddity”–or is it? The original post follows:
I love XFS.
I’ve been doing some, shall we say, rather unseemly sorts of things to one of my XFS file systems. Things are generally holding up, however, a little something made me crabby today so I think another installment in my Little Things Doth Crabby Make series is in order.
I don’t think I even need to explain why the following is something which hath crabby made:
# uname -r 2.6.18-194.26.1.el5 # df -h . Filesystem Size Used Avail Use% Mounted on /dev/sdb2 2.0T 1.7T 236G 88% /data1 # ls -l foo -rw-r--r-- 1 root root 0 Nov 7 10:53 foo # du -sh foo 287G foo # rm -f foo $ ls -l foo ls: foo: No such file or directory # df -h . Filesystem Size Used Avail Use% Mounted on /dev/sdb2 2.0T 1.7T 236G 88% /data1
I’ll blog more on what’s happening here as soon as I can. In the meantime, I’m crabby. Not really, that’s just the theme of this blog series.
Kevin Closson's Blog: Platforms, Databases and Storage 
We like XFS, too. I politely asked the s/a about something similar. To reuse your commentary, for the purpose of paraphrasing the response I got, the s/a said he was going to do some unseemly things to me if I didn’t find something better to do. That doth crabby me make.
Hi Paul,
That’s funny! Hmmm…I noticed your email when I approved your comment. I wonder if your s/a has been in your IT shop long enough to remember my on-site (at your data center) days back in the very early 1990s when you folks were one of the very first production sites to go live with Oracle Parallel Server on Unix (Sequent)? Ah, memories.
That aside, may I ask what sort of thing you were doing when you discovered a discrepancy between du and ls opinions of a file’s size? I will blog what got me to this point soon enough, but curious about your situation too.
If ls and du report the same how am I ‘sposed ta know someone still has a deleted file open?
They make powder for that crabby feeling Kevin.
I look forward to your blog series!
Might be a classic case that the foo file was unlinked from the file system put some process still holds fd to this file – so you see no df difference before and after the rm command? But the fact that it showed 0 file size – might simply indicate that the inode is corrupted. Did you try to run “xfs_bmap -l foo”? Did you try to copy this file to see if the copy is also of 0 size but du would still show 287G ?
Hi Piavlo,
As my friend Mark pointed out that would explain why df reported the same but the blog post title is ls versus du 🙂
Mark,
Would that be gunpowder? 🙂
You *do* know, don’t you, that having a file descriptor doesn’t keep the file in the namespace. That aside, pray-tell how you suppose “someone” with an active file descriptor is going to make it so that immediate (scripted), back-to-back ls(1) and du(1) commands are going to differ in their reckoning of how large the file is? Ls(1) tells me zero bytes, du(1) says 287G.
As for the open-but-deleted scenario:
# ls -l foo
-rw-r–r– 1 root root 0 Nov 7 13:58 foo
# ls -l foo
-rw-r–r– 1 root root 0 Nov 7 13:58 foo
#
# rm foo
# ls -l foo
ls: foo: No such file or directory
# pwd
/tmp
# ls -l /proc/22515/fd
total 0
lrwx—— 1 root root 64 Nov 7 13:57 0 -> /dev/pts/14
lrwx—— 1 root root 64 Nov 7 13:57 1 -> /dev/pts/14
lrwx—— 1 root root 64 Nov 7 13:57 2 -> /dev/pts/14
lrwx—— 1 root root 64 Nov 7 13:57 3 -> /tmp/foo (deleted)
#
# echo $$
22442
ls reports the filesize, du reports the blocks allocated to the inode. They’re not supposed to always agree. With the ability to speculatively allocate, there could easily be a reasonable delay or a bug with du.
jgarry,
So why then is it that ls thinks its zero bytes and du sees hundreds of gigabytes of allocated space? Not sparse but allocated space? Your explanation works for huge ls/small-du but not the other way around.
Also… I’m not sure I understand your point about ls and du semantic differences. On linux they both simply stat() the file.
Yes, I have no idea what I am talking about, this is entirely speculative. See http://permalink.gmane.org/gmane.comp.file-systems.xfs.general/36258
So what I am trying to say is, ls is correct, and du is not misleading because xfs has not deallocated the space yet, blocks are still pending to the inode. They potentially only get dropped when the last reference to the inode is dropped. Yes, I know I am going way out on a limb speculating from speculative preallocation to post-deallocation and assuming they both don’t simply stat() the file. But that’s ok because I have no idea what I am talking about. That way I don’t have to be crabby.
Joel wrote: “and assuming they both don’t simply stat() the file.”
…Joel,
You are the one saying you don’t know what you are talking about. Not me.
I specifically pointed out that both ls and du do, indeed, stat the file.
I also pointed out that I was doing “unseemly sorts of things” to my file system. I’ll explain soon.
Come to think of it, I’m probably throwing readers for a loop with the title of the post. Of course ls can report much larger numbers than du in sparse cases but since du proved it isn’t sparse that point is moot.