File system

For library and office filing systems, see Library classification.

Further information: Filing cabinet

A file system (filesystem) is means to organize data expected to be retained after a program terminates by providing procedures to store, retrieve and update data, as well as manage the available space on the device(s) which contain it. A file system organizes data in an efficient manner and is tuned to the specific characteristics of the device. There is usually a tight coupling between the operating system and the file system. Some filesystems provide mechanisms to control access to the data and metadata. Insuring reliability is a major responsibility of a filesystem. Some filesystems provide a means for multiple programs to update data in the same file nearly at the same time.

Without a filesystem programs would not be able to access data by file name or directory and would need to be able to directly access data regions on a storage device.

File systems are used on data storage devices such as magnetic storage disks or optical discs to maintain the physical location of the computer files. They may provide access to data on a file server by acting as clients for a network protocol (e.g., NFS, SMB, or 9P clients), or they may be virtual and exist only as an access method for virtual data (e.g., procfs). This is distinguished from a directory service and registry.

Aspects of file systems

Space Management

Example of slack space, demonstrated with 4,096-byte NTFS clusters: 100,000 files, each 5 bytes per file, equals 500,000 bytes of actual data, but requires 409,600,000 bytes of disk space to store

File systems allocate space in a granular manner, usually multiple physical units on the device. The file system is responsible for organizing files and directories, and keeping track of which areas of the media belong to which file and which are not being used. For example, in Apple DOS of the early 1980s, 256-byte sectors on 140 kilobyte floppy disk used a track/sector map.

This results in unused space when a file is not an exact multiple of the allocation unit, sometimes referred to as slack space. For a 512-byte allocation, the average unused space is 255 bytes. For a 64KB clusters, the average unused space is 32KB. The size of the allocation unit is chosen when the file system is created. Choosing the allocation size based on the average size of the files expected to be in the filesystem can minimize the amount of unusable space. Frequently the default allocation may provide reasonable usage. If it can be anticipated that a file system can will contain mostly small files a small cluster size should be chosen. Choosing an allocation size that is too small results in excessive overhead if the file system will contain mostly very large files.

File system fragmentation occurs when unused space or single files are not contiguous. As a filesystem is used, files are created, modified and deleted. When a file is created the filesystem allocates space for the data. Some filesystems permit or require specifying an initial space allocation and subsequent incremental allocations as the file grows. As files are deleted the space they were allocated eventually is considered available for use by other files. This creates alternating used and unused areas of various sizes. This is free space fragmentation. When a file is created and there is not an area of contiguous space available for its initial allocation the space must be assigned in fragments. When a file is modified such that it becomes larger it may exceed the space initially allocated to it, another allocation must be assigned elsewhere and the file becomes fragmented.

A file systems may not make use of a storage device but can be used to organize and represent access to any data, whether it is stored or dynamically generated (e.g., procfs).

File names

Main article: Filename

A file name (or filename) is used to reference the storage location in the filesystem. Most filesystems have restrictions on the length of the filename. Some filesystems have case insensitive filenames.

Most file system interface utilities place restrictions on the characters permitted in the filename restricting some special characters to provide a syntax to indicate a device, device type, directory prefix or file type. These are typically not file system restrictions and utilities may provide a means to refer to files with embedded special characters such as enclosing the entire filename within quotes ("). Avoiding using special characters makes it easier for users to refer to files.

Some filesystem utilities, editors and compilers treat prefixes and suffixes in a special way. These are usually merely conventions and not implemented within the filesystem.

Directories

Main article: Directory (file systems)

File systems typically have directories (sometimes called folders) which allow the user to group files. This may be implemented by connecting the file name to an index in a table of contents or an inode in a Unix-like file system. Directory structures may be flat (i.e. linear), or allow hierarchies where directories may contain subdirectories. The first file system to support arbitrary hierarchies of directories was the file system in the Multics operating system. The native file systems of Unix-like systems also support arbitrary directory hierarchies, as do, for example, Apple's Hierarchical File System and its successor HFS+ in classic Mac OS (HFS+ is still used in Mac OS X), the FAT file system in MS-DOS 2.0 and later and Microsoft Windows, the NTFS file system in the Windows NT family of operating systems, and the ODS-2 and higher levels of the Files-11 file system in OpenVMS.

Metadata

Other bookkeeping information is typically associated with each file within a file system. The length of the data contained in a file may be stored as the number of blocks allocated for the file or as a byte count. The time that the file was last modified may be stored as the file's timestamp. File systems might store the file creation time, the time it was last accessed, the time the file's meta-data was changed, or the time the file was last backed up. Other information can include the file's device type (e.g., block, character, socket, subdirectory, etc.), its owner user ID and group ID, and its access permission settings (e.g., whether the file is read-only, executable, etc.).

Additional attributes can be associated on file systems, such as NTFS, XFS, ext2/ext3, some versions of UFS, and HFS+, using extended file attributes. Some file systems provide for user defined attributes such as the author of the document, the character encoding of a document or the size of an image.

Some file systems allow for different data collections to be associated with one file name. These separate collections may be referred to as streams or forks. Apple has long used a forked file system on the Macintosh, and Microsoft supports streams in NTFS. Some file systems maintain multiple past revisions of a file under a single file name; the filename by itself retrieves the most recent version, while prior saved version can be accessed using a special naming convention such as "filename;4" or "filename(-4)" to access the version four saves ago.

Utilities

File systems include utilities to initialize, alter parameters of and remove an instance of the filesystem. Some include the ability to extend or truncate the space allocated to the file system.

Directory utilities create, rename and delete directory entries and alter metadata associated with a directory. They may include a means to create additional links to a directory (hard links in Unix), rename parent links (".." in Unix-like OS), and create bidirectional links to files.

File utilities create, list, copy, move and delete files, alter metadata. They may be able to truncate data, truncate or extend space allocation, append to, move, and modify files in-place. Depending on the underlying structure of the filesystem, they may provide a mechanism to prepend to, or truncate from, the beginning of a file, insert entries into the middle of a file or deletion entries from a file.

Also in this category are utilities to free space for deleted files if the filesystem provides an undelete function.

Some filesystems defer reorganization of free space, secure erasing of free space and rebuilding of hierarchical structures. They provide utilities to perform these functions at times of minimal activity. Included in this category is the infamous defragmentation utility.

Some of the most important features of files system utilities involve supervisory activities which may involve bypassing ownership or direct access to the underlying device. These include high performance backup and recovery, data replication and reorganization of various data structures and allocation tables within the filesystem.

Restricting and permitting access

See also: Secure computing

There are several mechanisms used by file systems to control access to data. Usually the intent is to prevent reading or modifying files by a user or group of users. Another reason is to insure data is modified in a controlled way so access may be restricted to a specific to program. Examples include passwords stored in the metadata of the file or elsewhere and file permissions in the form of permission bits, access control lists, or capabilities. The need for filesystem utilities to be able to access the data at the media level to reorganize the structures and provide efficient backup usually means that these are only effective for polite users but are not effective against intruders.

See also password cracking.

Methods for encrypting file data are sometimes included in the filesystem. This is very effective since there is no need for filesystem utilities to know the encryption seed to effectively manage the data. The risks of relying on encryption include the fact that an attacker can copy the data and use brute force to decrypt the data. Losing the seed means losing the data.

See also filesystem-level encryption, Encrypting File System.

Maintaining integrity

One of the filesystems significant responsibilities is to insure that, regardless of the actions by programs accessing the data, the structure remains consistent. This includes actions taken if a program modifying data terminates abnormally or neglects to inform the filesystem that is has completed its activities. This may include updating the metadata, the directory entry and handling any data that was buffered but not yet updated on the physical storage media.

Other failures which the filesystem must deal with include media failures or loss of connection to remote systems.

In the event of an operating system failure or "soft" power failure, special routines in the filesystem must be invoked similar to when an individual program fails.

The filesystem must also be able to correct damaged structures. These may occur as a result of an operating system failure for which the OS was unable to notify the file system, power failure or reset.

The filesystem must also record events to allow analysis of systemic issues as well as problems with specific files or directories.

Using a filesystem

Utilities, language specific run-time libraries and user programs use Filesystem_APIs to make requests of the file system. These include data transfer, positioning, updating metadata, managing directories, managing access specifications and removal.

Multiple Filesystems within a single system

Frequently retail systems are configured with a single filesystem occupying the entire hard disk.

Another approach is to partition the disk so that several filesystems with different attributes can be used. One filesystem, for use as browser cache, might be configured with a small allocation size. This has the additional advantage of keeping the frantic activity of creating and deleting files typical of browser activity in a narrow area of the disk and not interfering with allocations of other files. A similar partition might be created for email. Another partition, and filesystem might be created for the storage of audio or video files with a relatively large allocation. One of the filesystems may normally be set read only and only periodically be set writable.

Multiple filesystems on a single system has the additional benefit that in the event of a corruption of a single partition, the remaining filesystems will frequently be still intact. This includes virus destruction of the system partition or even a system that will not boot. Filesystem utilities which require dedicated access can effectively be completed piece meal, in addition defragmentation may be more effective. Several system maintenance utilities such as virus scans and backups can also be processed in segments. For example it is not necessary to backup the filesystem containing videos along with all the other files if none have been added since the last backup.

Design limitations

All file systems have some functional limit that defines the maximum storable data capacity within that system. These functional limits are a best-guess effort by the designer to determine how large the storage systems will be right now, and how large storage systems are likely to become in the future. Disk storage has continued to increase at near exponential rates (see Moore's law), so after a few years file systems have kept reaching design limitations that require computer users to repeatedly move to a newer system with ever greater capacity.

File system complexity typically varies proportionally with the available storage capacity. The file systems of early 1980s home computers with 50KB to 512KB of storage would not be a reasonable choice for modern storage systems with hundreds of gigabytes of capacity. Likewise, modern file systems would not be a reasonable choice for these early systems, since the complexity of modern file system structures would consume most or all of the very limited capacity of the early storage systems.

Types of file systems

File system types can be classified into disk/tape file systems, network file systems and special purpose file systems.

Disk file systems

A disk file system takes advantages of the ability to randomly address data on a disk storage media in a short amount of time. Additional considerations include the speed of accessing data following that initially requested and the anticipation that the following data may also be requested. This permits multiple users (or processes) access to various data on the disk without regard to the sequential location of the data. Examples include FAT (FAT12, FAT16, FAT32, exFAT), NTFS, HFS and HFS+, HPFS, UFS, ext2, ext3, ext4, btrfs, ISO 9660, Files-11, Veritas File System, VMFS, ZFS, ReiserFS and UDF. Some disk file systems are journaling file systems or versioning file systems.

Optical discs

ISO 9660 and Universal Disk Format (UDF) are two common formats that target Compact Discs, DVDs and Blu-ray discs. Mount Rainier is an extension to UDF supported by Linux 2.6 series and Windows Vista that facilitates rewriting to DVDs.

Flash file systems

Main article: Flash file system

A flash file system considers the special abilities, performance and restrictions of flash memory devices. Frequently a disk file system can use a flash memory device as the underlying storage media but it is much better to use a filesystem specifically designed for a flash device.

Tape file systems

A tape file system is a file system and tape format designed to store files on tape in a self-describing form. Magnetic tapes are sequential storage media with significantly longer random data access times than disks, posing challenges to the creation and efficient management of a general-purpose file system.

In a disk file system there is typically a master file directory, and a map of used and free data regions. Any file additions, changes, or removals require updating the directory and the used/free maps. Random access to data regions is measured in milliseconds so this system works well for disks.

Tape requires linear motion to wind and unwind potentially very long reels of media. This tape motion may take several seconds to several minutes to move the read/write head from one end of the tape to the other.

Consequently, a master file directory and usage map can extremely slow and inefficient with tape. Writing typically involves reading the block usage map to find free blocks for writing, updating the usage map and directory to add the data, and then advancing the tape to write the data in the correct spot. Each additional file write requires updating the map and directory and writing the data, which may take several seconds to occur for each file.

Tape file systems instead typically allow for the file directory to be spread across the tape intermixed with the data, referred to as streaming, so that time-consuming and repeated tape motions are not required to write new data.

However, a side effect of this design is that reading the file directory of a tape usually requires scanning the entire tape to read all the scattered directory entries. Most data archiving software that works with tape storage will store a local copy of the tape catalog on a disk file system, so that adding files to a tape can be done quickly without having to rescan the tape media. The local tape catalog copy is usually discarded if not used for a specified period of time, at which point the tape must be re-scanned if it is to be used in the future.

IBM has developed a file system for tape called the Linear Tape File System. The IBM implementation of this file system has been released as the open-source IBM Linear Tape File System — Single Drive Edition (LTFS—SDE) product. The Linear Tape File System uses a separate partition on the tape to record the index meta-data, thereby avoiding the problems associated with scattering directory entries across the entire tape.

Tape formatting

Writing data to a tape is often a significantly time-consuming process that may take several hours. Similarly, completely erasing or formatting a tape can also take several hours. With many data tape technologies it is not necessary to format the tape before over-writing new data to the tape. This is due to the inherently destructive nature of overwriting data on sequential media.

Because of the time it can take to format a tape, typically tapes are pre-formatted so that the tape user does not need to spend time preparing each new tape for use. All that is usually necessary is to write an identifying media label to the tape before use, and even this can be automatically written by software when a new tape is used for the first time.

Database file systems

Another concept for file management is the idea of a database-based file system. Instead of, or in addition to, hierarchical structured management, files are identified by their characteristics, like type of file, topic, author, or similar rich metadata. [1]

Although there are no "pure" database file systems, there are initiatives to use the concept:

• A lot of Web-CMS use a relational DBMS to store and retrieve files. Examples: XHTML files are stored as XML or text fields, image files are stored as blob fields; SQL SELECT (with optional XPath) statements retrieve the files, and allow the use of a sophisticated logic and more rich information associations than "usual file systems".
• Very large file systems, embodied by applications like Apache Hadoop and Google File System, use some database file system concepts.

Transactional file systems

Some programs need to update multiple files "all at once." For example, a software installation may write program binaries, libraries, and configuration files. If the software installation fails, the program may be unusable. If the installation is upgrading a key system utility, such as the command shell, the entire system may be left in an unusable state.

Transaction processing introduces the isolation guarantee, which states that operations within a transaction are hidden from other threads on the system until the transaction commits, and that interfering operations on the system will be properly serialized with the transaction. Transactions also provide the atomicity guarantee, that operations inside of a transaction are either all committed, or the transaction can be aborted and the system discards all of its partial results. This means that if there is a crash or power failure, after recovery, the stored state will be consistent. Either the software will be completely installed or the failed installation will be completely rolled back, but an unusable partial install will not be left on the system.

Windows, beginning with Vista, added transaction support to NTFS, abbreviated TxF. TxF is the only commercial implementation of a transactional file system, as transactional file systems are difficult to implement correctly in practice. There are a number of research prototypes of transactional file systems for UNIX systems, including the Valor file system, Amino, LFS, and a transactional ext3 file system on the TxOS kernel, as well as transactional file systems targeting embedded systems, such as TFFS.

Ensuring consistency across multiple file system operations is difficult, if not impossible, without file system transactions. File locking can be used as a concurrency control mechanism for individual files, but it typically does not protect the directory structure or file metadata. For instance, file locking cannot prevent TOCTTOU race conditions on symbolic links. File locking also cannot automatically roll back a failed operation, such as a software upgrade; this requires atomicity.

Journaling file systems are one technique used to introduce transaction-level consistency to file system structures. Journal transactions are not exposed to programs as part of the OS API; they are only used internally to ensure consistency at the granularity of a single system call.

Data backup systems typically do not provide support for direct backup of data stored in a transactional manner, which makes recovery of reliable and consistent data sets difficult. Most backup software simply notes what files have changed since a certain time, regardless of the transactional state shared across multiple files in the overall dataset. As a workaround, some database systems simply produce an archived state file containing all data up to that point, and the backup software only backs that up and does not interact directly with the active transactional databases at all. Recovery requires separate recreation of the database from the state file, after the file has been restored by the backup software.

Network file systems

Main article: Distributed file system

A network file system is a file system that acts as a client for a remote file access protocol, providing access to files on a server. Examples of network file systems include clients for the NFS, AFS, SMB protocols, and file-system-like clients for FTP and WebDAV.

Shared disk file systems

Main article: Shared disk file system

A shared disk file system is one in which a number of machines (usually servers) all have access to the same external disk subsystem (usually a SAN). The file system arbitrates access to that subsystem, preventing write collisions. Examples include GFS from Red Hat, GPFS from IBM, and SFS from DataPlow.

Special file systems

A special file system presents non-file elements of an operating system as files so they can be acted on using file system APIs. This is most commonly done in Unix-like operating systems, but devices are given file names in some non-Unix-like operating systems as well.

Device file systems

A device file system represents I/O devices and pseudo-devices as files, called device files. Examples in Unix-like systems include devfs and, in Linux 2.6 systems, udev. In non-Unix-like systems, such as TOPS-10 and other operating systems influenced by it, where the full filename or pathname of a file can include a device prefix, devices other than those containing file systems are referred to by a device prefix specifying the device, without anything following it.

Others

• In the Linux kernel, configfs and sysfs provide files that can be used to query the kernel for information and configure entities in the kernel.
• procfs maps processes and, on Linux, other operating system structures into a filespace.

File systems and operating systems

Most operating systems provide a file system, as a file system is an integral part of any modern operating system. Early microcomputer operating systems' only real task was file management — a fact reflected in their names (see DOS). Some early operating systems had a separate component for handling file systems which was called a disk operating system. On some microcomputers, the disk operating system was loaded separately from the rest of the operating system. On early operating systems, there was usually support for only one, native, unnamed file system; for example, CP/M supports only its own file system, which might be called "CP/M file system" if needed, but which didn't bear any official name at all.

Because of this, there needs to be an interface provided by the operating system software between the user and the file system. This interface can be textual (such as provided by a command line interface, such as the Unix shell, or OpenVMS DCL) or graphical (such as provided by a graphical user interface, such as file browsers). If graphical, the metaphor of the folder, containing documents, other files, and nested folders is often used (see also: directory and folder).

No file system / Audio-cassette storage

In the late 1970's with the development of the microcomputer, professional disk and tape systems were typically far too expensive for hobbyists to afford. For the first inexpensive home computer systems, a very primitive data storage system was devised that instead used common audio cassette tape.

Due to the very limited memory capacity of these computers, the data access was typically extremely simple: tell the computer to prepare to accept data, and press the Play button on the cassette player. Data was stored sequentially in an un-named format, located by fast-forwarding the tape and looking at the tape counter to find the approximate start of the next data region of the tape. The home computer user might have to listen to the digital sounds to find the right spot to begin playing the next data region.

Flat file systems

In a flat file system, there are no subdirectories—everything is stored at the same (root) level on the media, be it a hard disk, floppy disk, etc. While simple, this system rapidly becomes inefficient as the number of files grows, and makes it difficult for users to organize data into related groups.

For the early home computers, not much data could be stored anyway, so the limitations of flat-file storage was not a hindrance to use of the systems. The 8-bit Atari, for example, could only store 96 kilobytes per 5.25" floppy disk, and the Apple II was only capable of 140 kilobytes per floppy disk. In the early to mid 1980's, a large amount of hard disk storage for homes and small businesses was 5 to 10 megabytes. Some early file systems such as the Apple II DOS 3.3 could have disks customized to allow for hundreds of small files in the flat-file catalog listing.

Like many small systems before it, the original Apple Macintosh featured a flat file system, called Macintosh File System. Its version of Mac OS was unusual in that the file management software (Macintosh Finder) created the illusion of a partially hierarchical filing system on top of EMFS. This structure meant that every file on a disk had to have a unique name, even if it appeared to be in a separate folder. MFS was quickly replaced with Hierarchical File System, which supported real directories.

A recent addition to the flat file system family is Amazon's S3, a remote storage service, which is intentionally simplistic to allow users the ability to customize how their data is stored. The only constructs are buckets (imagine a disk drive of unlimited size) and objects (similar, but not identical to the standard concept of a file). Advanced file management is allowed by being able to use nearly any character (including '/') in the object's name, and the ability to select subsets of the bucket's content based on identical prefixes.

Unix-like operating systems

Unix-like operating systems create a virtual file system, which makes all the files on all the devices appear to exist in a single hierarchy. This means,in those systems, there is one root directory, and every file existing on the system is located under it somewhere. Unix-like systems can use a RAM disk or network shared resource as its root directory.

Unix-like systems assign a device name to each device, but this is not how the files on that device are accessed. Instead, to gain access to files on another device, the operating system must first be informed where in the directory tree those files should appear. This process is called mounting a file system. For example, to access the files on a CD-ROM, one must tell the operating system "Take the file system from this CD-ROM and make it appear under such-and-such directory". The directory given to the operating system is called the mount point – it might, for example, be /media. The /media directory exists on many Unix systems (as specified in the Filesystem Hierarchy Standard) and is intended specifically for use as a mount point for removable media such as CDs, DVDs, USB drives or floppy disks. It may be empty, or it may contain subdirectories for mounting individual devices. Generally, only the administrator (i.e. root user) may authorize the mounting of file systems.

Unix-like operating systems often include software and tools that assist in the mounting process and provide it new functionality. Some of these strategies have been coined "auto-mounting" as a reflection of their purpose.

1. In many situations, file systems other than the root need to be available as soon as the operating system has booted. All Unix-like systems therefore provide a facility for mounting file systems at boot time. System administrators define these file systems in the configuration file fstab (vfstab in Solaris), which also indicates options and mount points.
2. In some situations, there is no need to mount certain file systems at boot time, although their use may be desired thereafter. There are some utilities for Unix-like systems that allow the mounting of predefined file systems upon demand.
3. Removable media have become very common with microcomputer platforms. They allow programs and data to be transferred between machines without a physical connection. Common examples include USB flash drives, CD-ROMs, and DVDs. Utilities have therefore been developed to detect the presence and availability of a medium and then mount that medium without any user intervention.
4. Progressive Unix-like systems have also introduced a concept called supermounting; see, for example, the Linux supermount-ng project. For example, a floppy disk that has been supermounted can be physically removed from the system. Under normal circumstances, the disk should have been synchronized and then unmounted before its removal. Provided synchronization has occurred, a different disk can be inserted into the drive. The system automatically notices that the disk has changed and updates the mount point contents to reflect the new medium. Similar functionality is found on Windows machines.
5. An automounter will automatically mount a file system when a reference is made to the directory atop which it should be mounted. This is usually used for file systems on network servers, rather than relying on events such as the insertion of media, as would be appropriate for removable media.

Linux

Linux supports many different file systems, but common choices for the system disk include the ext* family (such as ext2, ext3 and ext4), XFS, JFS, ReiserFS and btrfs.

Solaris

The Sun Microsystems Solaris operating system in earlier releases defaulted to (non-journaled or non-logging) UFS for bootable and supplementary file systems. Solaris defaulted to, supported, and extended UFS.

Support for other file systems and significant enhancements were added over time, including Veritas Software Corp. (Journaling) VxFS, Sun Microsystems (Clustering) QFS, Sun Microsystems (Journaling) UFS, and Sun Microsystems (open source, poolable, 128 bit compressible, and error-correcting) ZFS.

Kernel extensions were added to Solaris to allow for bootable Veritas VxFS operation. Logging or Journaling was added to UFS in Sun's Solaris 7. Releases of Solaris 10, Solaris Express, OpenSolaris, and other open source variants of the Solaris operating system later supported bootable ZFS.

Logical Volume Management allows for spanning a file system across multiple devices for the purpose of adding redundancy, capacity, and/or throughput. Legacy environments in Solaris may use Solaris Volume Manager (formerly known as Solstice DiskSuite.) Multiple operating systems (including Solaris) may use Veritas Volume Manager. Modern Solaris based operating systems eclipse the need for Volume Management through leveraging virtual storage pools in ZFS.

Mac OS X

Mac OS X uses a file system that it inherited from classic Mac OS called HFS Plus, sometimes called Mac OS Extended. HFS Plus is a metadata-rich and case preserving file system. Due to the Unix roots of Mac OS X, Unix permissions were added to HFS Plus. Later versions of HFS Plus added journaling to prevent corruption of the file system structure and introduced a number of optimizations to the allocation algorithms in an attempt to defragment files automatically without requiring an external defragmenter.

Filenames can be up to 255 characters. HFS Plus uses Unicode to store filenames. On Mac OS X, the filetype can come from the type code, stored in file's metadata, or the filename.

HFS Plus has three kinds of links: Unix-style hard links, Unix-style symbolic links and aliases. Aliases are designed to maintain a link to their original file even if they are moved or renamed; they are not interpreted by the file system itself, but by the File Manager code in userland.

Mac OS X also supports the UFS file system, derived from the BSD Unix Fast File System via NeXTSTEP. However, as of Mac OS X 10.5 (Leopard), Mac OS X can no longer be installed on a UFS volume, nor can a pre-Leopard system installed on a UFS volume be upgraded to Leopard.

Newer versions Mac OS X are capable of reading and writing to the legacy FAT file systems(16 & 32). They are capable of reading, but not writing to the NTFS file system. Third party software is still necessary to write to the NTFS file system.

Plan 9

Plan 9 from Bell Labs treats everything as a file, and accessed as a file would be (i.e., no ioctl or mmap) networking, graphics, debugging, authentication, capabilities, encryption, and other services are accessed via I-O operations on file descriptors. The 9P protocol removes the difference between local and remote files

These file systems are organized with the help of private, per-process namespaces, allowing each process to have a different view of the many file systems that provide resources in a distributed system.

The Inferno operating system shares these concepts with Plan 9.

Microsoft Windows

Directory listing in a Windows command shell

Windows makes use of the FAT and NTFS file systems.

Windows uses a drive letter abstraction at the user level to distinguish one disk or partition from another. For example, the path C:\WINDOWS represents a directory WINDOWS on the partition represented by the letter C. The C drive is most commonly used for the primary hard disk partition, on which Windows is usually installed and from which it boots. This "tradition" has become so firmly ingrained that bugs came about in older applications which made assumptions that the drive that the operating system was installed on was C. The use of drive letters, and the tradition of using "C" as the drive letter for the primary hard disk partition, can be traced to MS-DOS, where the letters A and B were reserved for up to two floppy disk drives. This in turn derived from CP/M in the 1970s, and ultimately from IBM's CP/CMS of 1967.

Network drives may also be mapped to drive letters.

FAT

The File Allocation Table (FAT) filing system, supported by all versions of Microsoft Windows, was an evolution of that used in Microsoft's earlier operating system (MS-DOS which in turn was based on 86-DOS). FAT ultimately traces its roots back to the short-lived M-DOS project and Standalone disk BASIC before it. Over the years various features have been added to it, inspired by similar features found on file systems used by operating systems such as Unix.

Older versions of the FAT file system (FAT12 and FAT16) had file name length limits, a limit on the number of entries in the root directory of the file system and had restrictions on the maximum size of FAT-formatted disks or partitions. Specifically, FAT12 and FAT16 had a limit of 8 characters for the file name, and 3 characters for the extension (such as .exe). This is commonly referred to as the 8.3 filename limit. VFAT, which was an extension to FAT12 and FAT16 introduced in Windows NT 3.5 and subsequently included in Windows 95, allowed long file names (LFN).

FAT32 also addressed many of the limits in FAT12 and FAT16, but remains limited compared to NTFS.

exFAT (also known as FAT64) is the newest iteration of FAT, with certain advantages over NTFS with regards to file system overhead. exFAT is only compatible with newer Windows systems, such as Windows 2003, Windows Vista, Windows 2008, Windows 7 and more recently, support has been added for WinXP.

NTFS

NTFS, introduced with the Windows NT operating system, allowed ACL-based permission control. Hard links, multiple file streams, attribute indexing, quota tracking, sparse files, encryption, compression, reparse points (directories working as mount-points for other file systems, symlinks, junctions, remote storage links) are also supported, though not all these features are well-documented.

OpenVMS

Main article: Files-11

MVS [IBM Mainframe]

Main article: MVS § MVS filesystem

Other file systems

• The Prospero File System is a file system based on the Virtual System Model. The system was created by Dr. B. Clifford Neuman of the Information Sciences Institute at the University of Southern California.
• RSRE FLEX file system - written in ALGOL 68
• The file system of the Michigan Terminal System (MTS) is interesting because: (i) it provides "line files" where record lengths and line numbers are associated as metadata with each record in the file, lines can be added, replaced, updated with the same or different length records, and deleted anywhere in the file without the need to read and rewrite the entire file; (ii) using program keys files may be shared or permitted to commands and programs in addition to users and groups; and (iii) there is a comprehensive file locking mechanism that protects both the file's data and its metadata.

Limitations

Converting the type of a file system

File systems cannot in general be converted from one type to another. More precisely, file systems cannot in general be converted in place, meaning that if one has a file system (on a disk, for instance), one cannot restructure that file system with a different file system containing the same data.

This is because the file systems typically rely on certain common critical data structures such as a partition table, a sector map, and/or a master file directory which are always found starting at a specific location in a disk partition, and this location often overlaps between many different file systems.

Technically, there is no absolute requirement that a file system's core functional data structures must always start at relative sector zero inside a partition. Arbitrary data could precede the file system descriptor, and the critical structures could be stored anywhere inside a partition. But most disk partitioning systems are designed to assume that if a partition begins here and ends there, the file system descriptor will always be found in a specific spot at the absolute beginning of the partition data area and nowhere else. It is not possible to state in the partition table that the partition critical structures can be found in some location other than at the absolute begininning of the partition.

Instead, one may replace a file system with a different file system. For example, if one wishes to "convert" a FAT32 partition to an ext2 partition, one may copy the data to a separate filesystem (such as a different partition or a USB hard drive, via rsync to preserve metadata), then erase the FAT32 filesystem (making the partition empty), create a new ext2 file system in its place ("format" the partition), and then copy the data back to the partition, into the new file system.

In limited cases conversion can be done in-place, though replacing the file system is less dangerous, as it involves a backup copy of the data, and is thus recommended. On Windows, FAT and FAT32 file systems can be converted to NTFS via the convert.exe utility, but not the reverse. On Linux, ext2 can be converted to ext3 (and converted back), and ext3 can be converted to ext4 (but not converted back), and both ext3 and ext4 can be converted to btrfs, and converted back until the undo information is deleted. These conversions are possible due to using the same format for the file data itself, and relocating the metadata into empty space, in some cases using sparse file support.

It is in principle possible to convert in-place between file systems with sparse file support, but this is strongly recommended against and there are no mainstream tools to accomplish this. Technically, this is done by creating a sparse file in the existing file system, creating the new file system in it, moving the files from the old file system to the new file system and, once the move is complete, moving the blocks of the new file system in place to their correct positions, over the old file system. This has been implemented in a proof-of-concept utility on Linux in 2001–2002, convertfs, but has not been further developed, and due to the risk of data loss by destroying the file system, this tool is not included in current Linux distributions. Instead, replacing the file system (as described above) is the suggested course.

Too-long file names and file paths

Typically in hierarchical file systems, files are accessed by means of a file path that identifies a branching list of directories containing the file. The file path usually has a length limit imposed by the operating system, and this length limit varies between operating systems. The files themselves may also have a length limit that may be equal to or shorter than the total file path.

Some file systems allow for the creation of extremely long paths, which are longer than what the operating system is able to process. Some may be able, though, to detect when renaming a parent directory will render child directories inaccessible in this manner, and thus will prevent this condition from developing.

See also

• Comparison of file systems
• Directory structure
• Disk sharing
• Distributed file system
• Filename extension
• File manager
• File system fragmentation
• Filesystem API
• Global filesystem
• Physical and logical storage
• List of file systems
• List of Unix programs
• Virtual file system
• Storage efficiency
• List of default file system

References

General references

• Jonathan de Boyne Pollard (1996). "Disc and volume size limits". Frequently Given Answers. Retrieved February 9, 2005.
• IBM. "OS/2 corrective service fix JR09427". Retrieved February 9, 2005.
• "Attribute - $EA_INFORMATION (0xD0)". NTFS Information, Linux-NTFS Project. Retrieved February 9, 2005.
• "Attribute - $EA (0xE0)". NTFS Information, Linux-NTFS Project. Retrieved February 9, 2005.
• "Attribute - $STANDARD_INFORMATION (0x10)". NTFS Information, Linux-NTFS Project. Retrieved February 21, 2005.
• Apple Computer Inc. "Technical Note TN1150: HFS Plus Volume Format". Detailed HFS Plus and HFSX description. Retrieved May 2, 2006.
• File System Forensic Analysis, Brian Carrier, Addison Wesley, 2005.

Further reading

Books

• Carrier, Brian (2005). File System Forensic Analysis. Addison-Wesley. ISBN 0321268172.
• Custer, Helen (1994). Inside the Windows NT File System. Microsoft Press. ISBN 155615660X.
• Giampaolo, Dominic (1999). Practical File System Design with the Be File System (PDF). Morgan Kaufmann Publishers. ISBN 1558604979. Retrieved 2010-01-22.
• McCoy, Kirby (1990). VMS File System Internals. VAX - VMS Series. Digital Press. ISBN 1555580564.
• Mitchell, Stan (1997). Inside the Windows 95 File System. O'Reilly. ISBN 156592200X.
• Nagar, Rajeev (1997). Windows NT File System Internals : A Developer's Guide. O'Reilly. ISBN 9781565922495.
• Pate, Steve D. (2003). UNIX Filesystems: Evolution, Design, and Implementation. Wiley. ISBN 0471164836.
• Rosenblum, Mendel (1994). The Design and Implementation of a Log-Structured File System. The Springer International Series in Engineering and Computer Science. Springer. ISBN 0792395417.
• Russinovich, Mark; Solomon, David A.; Ionescu, Alex (2009). "File Systems". Windows Internals (5th ed.). Microsoft Press. ISBN 0735625301.
• Prabhakaran, Vijayan (2006). IRON File Systems. PhD disseration, University of Wisconsin-Madison.
• Silberschatz, Abraham; Galvin, Peter Baer; Gagne, Greg (2004). "Storage Management". Operating System Concepts (7th ed.). Wiley. ISBN 0471694665.
• Tanenbaum, Andrew S. (2007). Modern operating Systems (3rd ed.). Prentice Hall. ISBN 0136006639.
• Tanenbaum, Andrew S.; Woodhull, Albert S. (2006). Operating Systems: Design and Implementation (3rd ed.). Prentice Hall. ISBN 0131429388.

Online

• Benchmarking Filesystems (outdated) by Justin Piszcz, Linux Gazette 102, May 2004
• Benchmarking Filesystems Part II using kernel 2.6, by Justin Piszcz, Linux Gazette 122, January 2006
• Filesystems (ext3, ReiserFS, XFS, JFS) comparison on Debian Etch 2006
• Interview With the People Behind JFS, ReiserFS & XFS
• Journal File System Performance (outdated): ReiserFS, JFS, and Ext3FS show their merits on a fast RAID appliance
• Journaled Filesystem Benchmarks (outdated): A comparison of ReiserFS, XFS, JFS, ext3 & ext2
• Large List of File System Summaries
• Linux File System Benchmarks v2.6 kernel with a stress on CPU usage
• Linux Filesystem Benchmarks
• Linux large file support (outdated)
• Local Filesystems for Windows
• Overview of some filesystems (outdated)
• Sparse files support (outdated)
• Jeremy Reimer (March 16, 2008). "From BFS to ZFS: past, present, and future of file systems". arstechnica.com. Retrieved 2008-03-18.

External links

• Filesystem Specifications - Links & Whitepapers
• Interesting File System Projects