Carving and Small Files

Carving is one of the approaches used in data recovery, in which files are extracted according to known patterns, the so-called signatures. For example, JPEG files begin with the byte sequence FF D8 FF. The process of searching for such files works as follows:

1. A sector is read from the storage device.
2. The beginning of the sector is analyzed to search for known file headers.
3. If a header is found, a file is created, and its length is determined in accordance with the algorithms of the data recovery program. In the most general case, the file is extracted either up to the next known header or up to the beginning of the next cluster.

It is believed that the carving method can extract small files with known headers. Here, “small” effectively means “unfragmented,” since it usually turns out that if a file is large and the storage device it is being written to has been used for a long time in everyday work (with files being written and deleted as usual, that is, quite often), then such a file will most likely be fragmented.

Fragmentation is dangerous for carving because this approach allows you to extract only the first fragment: the location of the remaining fragments is controlled by the file system driver and cannot be determined from the coordinates of the first fragment alone.

It was always believed that problems occur only with large files, while small ones are certainly recovered during carving. In fact, it turns out that the problem also exists with very small files — say, up to 1 KB in size. Let us see why.

NTFS File System

To optimize the use of disk space, this file system stores small files directly in the file system metadata — in the MFT record. Such files are called resident and can be up to 900 bytes in size, depending on the length of the name, the presence of a security descriptor, and other attributes.

The MFT record itself starts at sector offset zero, and the contents of the resident file are stored inside the MFT record. Thus, within the carving approach it will not be possible to find a file that is not at the beginning of the sector but at a different offset. Conclusion: if a file is resident on the NTFS file system, it cannot be extracted as part of standard carving.

To test this hypothesis, we conducted an experiment:

1. We formatted a zeroed 16 GB flash card in NTFS.
2. We placed four files on this volume, with sizes of 102 bytes (.gif), 402 bytes (.png), 80 bytes (.cab), and 680 bytes (.jpg). These formats are well recognized within the carving approach.
3. We found the MFT records for these files.
4. We edited the records in a disk editor, zeroing out the first four bytes so that they would not be detected as NTFS metadata records by data recovery programs.
5. We launched recovery on the edited volume in different data recovery programs: ReclaiMe Pro, R-Studio, and UFS Explorer.

As expected, none of the programs found these files, since their contents are located not at the beginning of the sector but inside the MFT record.

So, before editing the MFT records, we recovered all the files. But after editing the records, there are no files. The results in R-Studio and UFS Explorer are similar.

FAT File System

The same files were placed on a FAT-formatted volume. Since the FAT file system allocates a cluster (a group of sectors) for each file regardless of its size, it is expected that all files were recovered by carving (when the metadata for the files in the file system was intentionally edited so that recovery algorithms could not find them), since their contents began at the beginning of the cluster and therefore at the beginning of the sector.

Other data recovery programs produce the same result.

exFAT File System

The situation is similar to FAT, since this file system does not provide for storing the contents of files inside any metadata, which means the contents of all files will start at the beginning of the cluster and can be detected within the carving approach.

ReFS File System

Here, too, all files were recovered in full, although this file system has the ability to store content inside metadata; in our case, however, all files were stored in their own clusters.

BTRFS File System

Here, the carving approach failed to recover the files.

If we look at where the content is located, we see that it is shifted relative to the beginning of the sector. We look at the Trace tab: the disk offset for our file, 38,959,578, is not evenly divisible by 512 (38,959,578 / 512 = 76,092 with a remainder of 474 = 0x1DA), which means that the beginning of the file will be in sector 76,092 at offset 0x1DA.

EXT File System

All files were fully recovered within the carving approach in all data recovery programs that we tested.

XFS and ZFS File Systems

All files were fully recovered by carving. R-Studio does not recover the ZFS file system, but it does find these small files via carving.

Conclusions

If a file system has the ability to store small files inside metadata records, then such files will not be recovered within the carving approach, since their contents do not begin at the start of a sector, which is a necessary condition for such recovery.

The beginnings of such files can still be found by searching the disk, if one uses a search criterion across all offsets within a sector. Note, however, that this is always manual work and a time-consuming search.