So far I haven't been able to figure out fusion drives to an extent that
would allow me to implement writes. I'm tired of it, and I think the
effort is probably pointless because apfs fusion drives aren't all that
popular. I might reconsider if people request the feature, but for now
just add read-only support, which is almost trivial.
Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
A user has encountered a volume with the dataless snapshots flag:
https://github.com/linux-apfs/linux-apfs-rw/issues/86
I'm not really sure what "dataless snapshots" are, and so far I have
failed to trigger them under macos. I can set the
SNAP_META_PENDING_DATALESS flag in a snapshot myself, and macos will
call it "dataless", but their fsck doesn't complain no matter what I do
with the other fields, so I can't tell what is expected.
I can't imagine any risk in mounting such a volume read-only, so allow
that much from now on. Keep writes banned just in case, and avoid
mounting the dataless snapshots themselves, since I have no idea what
that entails.
Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
I've recently noticed a number of new flags and added them to the fsck.
I heven't quite figured out most of them yet, but copy the information
here anyway. Also add checks for the new incompatible features, so that
they don't get accepted by default.
Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
If the latest newfs_apfs creates an image with the default options,
older versions of fsck_apfs complain about an unknown volume superblock
field called "apfs_doc_id_index_flags". Several such new fields appear
to have been added, and after some bit flipping I managed to retrieve
the names for all of them, except for "apfs_doc_id_tree_oid" which is
only a (probably correct) guess.
I have no plans for these fields right now, but document them all in
apfs_raw.h anyway, so that I don't forget.
Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
According to checkpatch.pl, the use of zero-length arrays is now
deprecated in the kernel. Replace them all with C99 flexible array
members.
Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
Some code that wasn't originally written by me has a style that differs
slightly from the rest of the module. Since I've been refactoring
compress.c quite a lot lately, I have made the inconsistencies more
visible, so I think it's a good time to switch everything to the typical
kernel style.
For this purpose I'm running checkpatch.pl for the first time in years,
and it has detected a handful of other issues that I introduced myself.
Fix most of the issues now, but I prefer to leave the zero-length array
stuff for a separate patch.
For the record, at the moment I'm choosing to ignore style inconsistency
in the compression libraries, even in libzbitmap which I wrote myself. I
prefer to keep them as close to upstream as possible.
Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
Some recently defined object types are missing the "APFS_" prefix. I've
fixed this a while back in the fsck, do it in the driver too.
Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
Support for LZBITMAP was previously limited to resource compression,
simply because I had never encountered any attr-compressed files that I
could use for testing. Using the same tool from the previous patch[0]
and one of our iphone vms, I now managed to create one. Add support.
[0]: https://github.com/Siguza/fscmp
Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
Using the official LZFSE library (which I already dumped here in the
previous patch), add support for LZFSE compression. For testing I used
images created with the following tool:
https://github.com/Siguza/fscmp
I've tested both resource and attribute compression. For resource
compression, I also tested a file that had a single uncompressed block.
That block had 0xff in the first byte, but I don't know if that's a
general rule so it's better to just check for the first byte of the
LZFSE magic.
Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
Use Apple's official LZFSE library[0] to add support for LZVN
compression. Their code for LZFSE compression proper is also dumped with
this patch, but it's not actually rigged up for the driver because I
have no image to test it on.
For the record, the build on Linux 5.15 produces the following objtool
warning:
lzfse/lzvn_decode_base.o: warning: objtool: lzvn_decode()+0x1b0: sibling call from callable instruction with modified stack frame
I'm not sure what this means, but there seems to be no sibling call in
that position, just a regular goto:
1b0: e9 00 00 00 00 jmp 1b5 <lzvn_decode+0x1b5>
I'll ignore it for now, but I guess I should report it if it's a false
positive.
[0]: https://github.com/lzfse/lzfse
Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
There is a weird compression mode that is not actually compressed at
all, add support for it. This patch is just a copy of Stan's code for
our internal tool.
Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
Add suport for reads from sealed volumes. The main differences are that
catalog index nodes are bigger, and that file extents are kept in a
separate tree.
Writes could in theory be supported as well, as we do in our internal
tool. It would probably be a bit too hacky though, so I'll wait to see
if anybody asks for it.
Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
Implement an ioctl that creates a snapshot for the current transaction.
A matching userland tool has already been added to apfsprogs.
Note that, at this point, snapshots won't be handled correctly on later
writes at all. Being able to actually create snapshots for testing is
only the first step in that direction.
Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
Recent work on the fsck has involved a lot of changes to the disk layout
header over there. Bring as many of those changes as possible back to
the kernel, where they will soon be needed, at least for implementing
snapshots.
Also clean up the kernel header by getting rid of duplicated definitions
for the superblock magic, and by using UUID_SIZE everywhere instead of
16. It might have been cleaner to do this stuff in separate commits, but
I would probably have forgottent about it.
Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
The BSD flags used by APFS are not an exact match for the inode flags
used in Linux, but they are close enough that I can at least support
three of them. I don't know if this is important, but the HFS+ module
does it, so follow its lead.
Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
I currently get filesystem corruption after running generic/466 from
xfstests. This is a test of writes to huge filesystem offsets, and the
corruption happens because extent lengths wrap around at 2^56 bytes.
According to the documentation, it appears that the intended file size
limit is 2^63, so the proper way to fix this would involve creating a
large number of maximum-length hole extents. I haven't confirmed this
with the official implementation, but either way it seems like a waste
of effort for current disk sizes. Just ban the creation of files that
wouldn't fit in a single extent.
Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
Luflosi has reported some new build failures:
https://github.com/linux-apfs/linux-apfs-rw/issues/12#issuecomment-890317156
One problem is that the UUID_SIZE macro was not defined until 4.13, so
provide a definition of our own for older kernels. Also, while not
strictly necessary, include the uuid header explicitly (right now it
only works because <linux/fs.h> is included in apfs.h before
"apfs_raw.h").
The other problem is that lockdep_assert_held_write() had a different
name before 5.3, and may not even exist in some older kernels. This
macro is just for debugging, so define it as a no-op when not available.
Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
Separate the in-memory volume and container superblocks, and keep all
mounted containers in a linked list. Each container will hold a pointer
to its block device; when a mount is requested, we traverse the list to
check if it's a new container. Keep one vfs superblock for each mounted
volume, but only assign it a fake anonymous bdev; all disk operations
must be forwarded to the container's bdev, with the use of the new
apfs_sb_bread() and apfs_map_bh() functions.
All the mount changes require that we implement our own ->mount()
function, closely based on bdev_mount() and the btrfs equivalent. I
can't claim to be confident that all changes here are correct, much more
testing is needed.
To simplify access to the container, define two new helpers similar to
APFS_SB(): APFS_NXI() to retrieve the container superblock info, and
APFS_SM() to retrieve the space manager. Also move the usual assertion
that an object is part of the current transaction to its own inline
function; this saves me from rewriting all the callers and has the added
benefit of silencing "unused variable" warnings when the module is built
without APFS_DEBUG.
Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
Start a new out-of-tree repository, like linux-apfs-oot but with write
support.
To get the module to build independently, rewrite the Makefile and
add a definition for the APFS_SUPER_MAGIC macro. Since the intention is
to support a range of kernel versions, use preprocessor checks to handle
kernels without statx, without iversion, and without SB_RDONLY.
Provide a README file based on the original documentation, but with
additional build and mount instructions. Add a LICENSE file as well.
Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
Ernesto A. Fernández
Stan Skowronek