When you think of a file system, you might imagine something like this, the file browser.

File browsers, or file explorers, let you see what files and directories are available, open, copy, move, delete them, etc.

But this is just a view of the file system. In reality, files do much more under the hood of the operating system.

That’s basically all files are. No naming, no hardware specifics, no file formats.

Remove (abstract away) differences, leaving what’s common.

What are some different data storage technologies?

• Magnetic, optical, flash, RAM

What’s different about them?

What’s common about them?

• They provide a way to read and write binary on a physical medium.

Files abstract away the specifics of using different hardware technologies. As we’ll see, the abstraction can be applied to other hardware, such as network devices, and even used to create non-physical files, such as pipes.

File systems group together data on storage medium providing a large addressable sequence of bytes (usually as blocks).

Abstractions are defined by how they are used, i.e., their interface.

We need to be able to write data, extending the size of files, delete data, and read it, independently of where it may lie on the physical storage medium.

What other common operations can we do on files? Seek? Open/close?

Do other devices besides storage hardware fit this abstraction?

We’ll be using the UNIX-style file abstraction in this class.

• Used in all modern OSes (GNU/Linux, MacOS, Windows)

We can use the hexdump utility to view the raw bytes of a file in hexadecimal form.

#include <stdio.h>

int main() {
  printf("hello, world!\n");
}

00000000  23 69 6e 63 6c 75 64 65  20 3c 73 74 64 69 6f 2e  |#include <stdio.|
00000010  68 3e 0a 0a 69 6e 74 20  6d 61 69 6e 28 29 20 7b  |h>..int main() {|
00000020  0a 20 20 70 72 69 6e 74  66 28 22 68 65 6c 6c 6f  |.  printf("hello|
00000030  2c 20 77 6f 72 6c 64 21  5c 6e 22 29 3b 0a 7d 0a  |, world!\n");.}.|
00000040

The above output is split into four columns. The first column denotes the offset into the file, in hexadecimal. The second and third columns show the contents of the file in hexadecimal representation. The final column shows the ASCII representation of the file’s contents. See man ascii for more details about ASCII encoding.

What does a file abstraction that doesn’t capture names or file formats mean for file extensions?

file is not bullet-proof. Some malware will modify the magic bytes to avoid detection.

• Link for more info
• Email attachment: LOVE-LETTER-FOR-YOU.TXT.vbs
• .vbs hidden, so users clicked on what they thought was a textfile
• .vbs is a script that gets executed

By Mario23, Public Domain, https://commons.wikimedia.html/w/index.php?curid=19189003

The above command was ran in a compromised MS-SQL server to convert a .txt file containing malicious code into a .exe file so that it could be run without setting off file-extension based malware detection.

Anatomy of an Attack: Detecting and Defeating CRASHOVERRIDE

The following link describes how an information stealer called IcedID used polyglot files, which are files that are written in multiple valid programming languages and whose execution depends on the programming running them.

More Than Meets the Eye: Exposing a Polyglot File That Delivers IcedID

The map is not the territory

Take a class in or read about operating systems class to find out how file systems are implemented.

The phonebook analogy: the phone numbers are linked to the house or phone. The name is connected only by a phone directory (contact list).

Link to paper

Renaming and moving are actually the same command in unix: mv for “move”.

• Hard links: ln source destination
• Soft links: ln -s source destination
• Hard links share the same inode as the source (really just a new directory entry)
  • All files are essentially hard links to inodes.
  • Like adding a new door to a house. Same file (house), new door (directory entry)
  • Remove a door (file), the house (inode) remains
• Soft links have a new inode (new file)
  • Like building a tunnel to a house.
  • Remove the house (inode), tunnel (soft link) leads nowhere.
  • Remove the tunnel (soft link), the house (inode) remains.
• Soft links can do a few things that hard links can’t.
  • Soft links can link to directories, whereas hard links can’t because that would make it possible to create an infinite loop.
  • Soft links can also link to files across different volumes; hard links can’t do this because they must link to an inode number within the same volume.

• inode 0 is a null value, and inode 1 is used to track bad blocks on the disk. See https://docs.kernel.org/filesystems/ext4/special_inodes.html fore more information about special inode numbers on the ext4 file system (the default for many Linux machines).

The above command shows the inode numbers of the /, /usr/ and /usr/bin/ directories.

• The d flag to ls means to print information for directories themselves, not their contents.
• The i flag means to print files’ inode numbers.
• The 1 flag means to print each file’s information on a separate line.

• Most commonly known for persistent hardware storage, but a file does not mean data on a disk (you can have that without a file)

• But it is an abstraction for reading and writing sequences of bytes which can apply to any i/o (stdio.h defines the Unix syscall and libc conventions)

• Directories are special files that store mappings from names to other files

• If a directory contains a mapping to another directory file, we have a directory hierarchy

• Referring to files using relative and absolute naming (using the Unix convention)

• Relative paths are relative to the current working directory, which is stored with a running program (process)