Apple - Confusion on Metadata | Named Forks | Extended Attributes | Resource Fork -- HFS+

Confusion arises from the fact that the relationship between these concepts is complex and has changed over time. In current systems the difference between a named fork and an extended attribute is largely academic.

For an extended attribute, the actual data is stored in the attribute data record.

For a fork, what's stored is the list of disk allocation blocks that contain the data. A resource fork is still a fork.

Basic file system metadata is stored in dedicated elements of the file system record itself, independent of the attributes and named fork references.

I am not an expert on this, but I have read a bit to try to understand what is going on.

I would say "Confusion arises from the fact that:

• the relationship between these concepts is complex and
• has changed over time and
• Apple have implemented both program level APIs and tools like ls or cp is such a way as to hide many of the differences between the concepts."

AIUI, the HFS+ Catalog file contains Catalog File Records (among other things). The Catalog File Record contains the normal sort of information about the file such as creation date, access date etc. The Catalog File Record also contains two structures that give information about the location and size of the data fork and of the resource fork.

AIUI, HFS+ also has (copied from Wikipedia HFS+) an "Attributes File [which] is a new B-tree in HFS Plus that does not have a corresponding structure in HFS. The Attributes File can store three different types of 4 KB records: Inline Data Attribute records, Fork Data Attribute records and Extension Attribute records. Inline Data Attribute records store small attributes that can fit within the record itself. Fork Data Attribute records contain references to a maximum of eight extents that can hold larger attributes. Extension Attributes are used to extend a Fork Data Attribute record when its eight extent records are already used."

AIUI, the data stored in (or referenced from) the Attributes File (whether inline, Fork Data or Extension Attributes) are known as Extended Attributes.

Those are the data structures, so how are they used?

AIUI, early versions of the operating system (possibly versions before 10.4 Tiger, which John Siracusa seems to indicate had some major changes in this area), pointed to the data fork and the resource fork from the Catalog file.

AIUI, once we get to 10.4 Tiger, the Attributes File becomes widely used to store all sorts of data.

It is possible (but I don't know) that in 10.4 and after, any Resource Forks are pointed to from the Attributes File. I.e. in answer to your first question, I would say named forks are Extended Attributes, unless they are the Resource Fork, and the Resource Fork is referenced from the Catalog file.

The problem in knowing how things are implemented is that in order to preserve backward compatibility, and probably particularly to support access to file systems written by one version of Mac OS from another version, different things and mixtures of things need to be supported transparently.

We can't tell from the normal Terminal command line tools where data is actually being held.

So, access to rsrc might suggest that the Resource Fork in the Catalog file is being accessed.

$ ls -l Icon^M/rsrc
-rwxr-xr-x  1 root  admin  486 23 Jul  2004 Icon?/rsrc

However, we know that although the syntax looks like a file below the Icon^M directory is being accessed, this is not really the case, because

$ ls -lR Icon^M
-rwxr-xr-x@ 1 root  admin  0 23 Jul  2004 Icon?

so Apple have implemented a special case for Resource Forks.

If instead we do

$ ls -l@
-rwxr-xr-x@ 1 root  admin   0 23 Jul  2004 Icon?
    com.apple.FinderInfo    32 
    com.apple.ResourceFork  486 

This suggests that we are accessing the Attributes File. But again, the implementation of ls may have a special case for Resource Forks.

John Siracusa points out here that the ACL lists are stored as 'Extended Attributes', but are specially masked out so they are not shown in xattr. So again there is special case processing in the implementation of xattr.

(Note that this special case processing may be in the code of the tool, or in the code of the underlying APIs that the tools access.)

GregW, if you see this, it would be nice to get a more expert opinion as to whether I am along the right lines, or just hopelessly confused.