Some spaceman fields provide offsets from where to retrieve the actual
values. Those values aren't declared as part of any on-disk structures,
so special care is needed to remember their endianness.
Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
I can't claim to understand why this field exists, but the rules for its
content seem clear, so implement it. The fsck has already been updated
accordingly.
Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
The count of free blocks in the ring buffer of ip bitmaps should stay
constant, typically with a value of 2. My logic here made no sense, so
fix it.
Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
The last patch accidentally used 64 as the size of a u64 type. This was
used for boundary checks, so the module may incorrectly complain about
filesystem corruption. Fix it.
Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
The Apple driver has trouble mounting containers that have been touched
by my module, throwing many error messages such as
spaceman_freed:4137: failed to free internal pool block 10273: 22
Stan figured out that their driver expects cibs, cabs and allocation
bitmaps themselves to all be allocated in a special area called the
internal pool. This makes sense: if these structures are allocated like
any other, you have to take special measures to prevent recursive calls,
such as allocating a new block in the same chunk as its bitmap and cib.
This could become harder as the disk fills and becomes fragmented.
So do as Apple expects and use the internal pool for this stuff. With
this patch my module passes the new checks added to my fsck, but I'm yet
to test if it mounts well in a mac. All problems are not solved, since I
haven't dealt with the ip_bm_free_next field.
Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
I recently allowed versioned space managers to be mounted, and I left a
warning in place to remember the issue. I didn't stop to consider how
noisy this would get; print the warning only once.
Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
My module and userland tools don't use the internal pool for anything,
so when a block is freed it gets put in the main free queue. The problem
is that, if the filesystem image was not created by me, ip blocks may
actually be in use. When those are freed, put them in their proper
queue.
For the record, this is the error message the Apple fsck throws due to
this bug:
** Checking the space manager free queue trees.
error: sfqe_entry : (range 0x1257+0x1, xid 0x3) : Range in free queue tree lies in the internal pool
error: Spaceman free queue tree of type [1] is invalid
Space manager free queue trees are invalid.
Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
The versioned space manager is not documented, and I don't know what
it's about; but it's enabled by default in images created by an iphone 6
vm, so I can't ignore it. I don't see any obvious problems so far if I
just allow such images to mount, so do that and hope for the best. Keep
a warning in place to make sure I don't forget to figure this out in the
future.
Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
Add support for filesystems that do not use hashes in their dentry
records. I had not encountered these before, but they appear to be the
norm for iphone firmware images.
[ernesto: some small tweaks, fixed endianness issue and dentry cache,
added commit message]
Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
Begin work to support file writes. To this end, add functions to write
file extent records and to deal with the new data stream record type.
A whole new b-tree needs to be used as well: the extent reference tree.
Adjust the node and query code to deal with it.
There is too much I don't understand about the vfs, and writes look
tricky, so try to keep things simple for now. Don't allow writes
through mmap(), only create extents with a single block (even if this
defeats the whole purpose of extents), and neglect to handle short
writes correctly. Don't implement any form of truncation yet, and just
let apfs_delete_inode() fail when the inode is not empty.
Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
Make apfs_transaction_commit() sync all buffers as it iterates through
the list, instead of just marking them as dirty and leaving them for
blkdev_issue_flush(). This change makes it easier to handle objects to
which we still hold references, such as the volume. It also makes the
commit more reliable: for reasons I ignore, it prevents file buffers
from being lost when we try to implement file writes.
Signed-off-by: Ernesto A. Fernández <ernesto.mnd.fernandez@gmail.com>
Don't neglect to initialize the i_extent_id field of a new inode's info
structure. This has been irrelevant so far because nobody is writing to
any inodes, but that's about to change.
Also copy this extent id to disk on apfs_update_inode().
Signed-off-by: Ernesto A. Fernández <ernesto.mnd.fernandez@gmail.com>
The subtype for a new node is currently chosen based on the storage
type. This is of course only possible if we have a single subtype for
each storage type, which will no longer be true once we start working
with physical extents.
To prepare for this, start reading the subtype of nodes into memory on
apfs_read_node(), and give that same subtype to child or sibling nodes
on creation.
The subtype is technically an object property, but only nodes seem to
use it, so place it in the node structure. Since this is shared by all
nodes in a tree, it may be a good idea to use in-memory tree structures
for this sort of thing in the future.
Signed-off-by: Ernesto A. Fernández <ernesto.mnd.fernandez@gmail.com>
A comment on the btrfs aops claims that implementing ->bmap() is not
compatible with CoW, because swapfiles use it to bypass the filesystem
on writes. Follow btrfs and drop this operation.
Signed-off-by: Ernesto A. Fernández <ernesto.mnd.fernandez@gmail.com>
Keeping a header file for each source file doesn't seem to be common
among in-tree filesystems, and I gather that it's disliked by the kernel
developers. I tried to make it work for apfs, but it doesn't make much
sense because we often require almost every header.
So, unify most of them into a single apfs.h file. The main exception
is apfs_raw.h: on-disk structures need to remain separate so that they
will never be modified by mistake. Also ignore unicode.h for now, since
it doesn't quite fit in with the others and might go away when we start
using the official unicode implementation.
Signed-off-by: Ernesto A. Fernández <ernesto.mnd.fernandez@gmail.com>
Put all definitions for on-disk data structures in a dedicated header
file, adapted from the userland tools. The intention is to make it
clear that these definitions can never be changed.
Signed-off-by: Ernesto A. Fernández <ernesto.mnd.fernandez@gmail.com>
When apfs_chunk_allocate_block() is called on a chunk that is entirely
free, a blank bitmap must be allocated. This new bitmap shouldn't go
through copy-on-write, but it does; the result is that it gets put in
the free queue immediately, corrupting the container.
No consequences have been noticed so far, because this module can't yet
allocate enough blocks to fill a whole chunk. Fix the issue anyway; the
patch can still be tested if we tweak the code so that it never performs
any allocations in the first chunk.
Fixes: 3bdb0a8032 ("apfs: implement block allocation")
Signed-off-by: Ernesto A. Fernández <ernesto.mnd.fernandez@gmail.com>
In the spirit of the previous patch, use the __set_bit_le() provided by
the kernel to simplify apfs_chunk_mark_used(). There is no endianness
issue this time, but it's still better to use the common code.
Also change the type of the bitmap from u8* to char*, for consistency
with apfs_chunk_find_free().
Signed-off-by: Ernesto A. Fernández <ernesto.mnd.fernandez@gmail.com>
Before working with an on-disk allocation bitmap, apfs_chunk_find_free()
implicitly casts it to a u64 array, ignoring the cpu endianness. Since
the kernel provides find_next_zero_bit_le(), get rid of my buggy code
and use that instead.
To prevent this in the future, don't declare the bitmap as void* in the
caller.
Fixes: 3bdb0a8032 ("apfs: implement block allocation")
Signed-off-by: Ernesto A. Fernández <ernesto.mnd.fernandez@gmail.com>
Creation of cpm blocks is not yet supported, which may cause free queue
insertions to fail. Currently the return code used is ENOSPC, but this
has a special meaning for the spaceman allocation functions: it forces
them to continue the search in the next chunk or cib, with unpredictable
consequences that seem to trigger a few assertions. To avoid this, use
EOPNOTSUPP instead.
Fixes: 06af0392cf ("apfs: implement creation of free queue nodes")
Signed-off-by: Ernesto A. Fernández <ernesto.mnd.fernandez@gmail.com>
Get rid of apfs_timestamp() and apfs_timespec(), and use instead the
common timespec64_to_ns() and ns_to_timespec64().
Signed-off-by: Ernesto A. Fernández <ernesto.mnd.fernandez@gmail.com>
I can see no reason to keep the name of a file in its header comment. I
think I copied this from ext2, but more recent modules don't seem to do
it. Just remove them.
Signed-off-by: Ernesto A. Fernández <ernesto.mnd.fernandez@gmail.com>
By mistake, the macro guard of extents.h was given a name that is prone
to collision. Rename it to specify the module.
This change should probably have happened as part of commit 5f7efae269
("apfs: rename macro for #include guard in object.h"), but I seem to
have missed it back then.
Fixes: 7132a98d7f ("apfs: move extent code to separate files")
Signed-off-by: Ernesto A. Fernández <ernesto.mnd.fernandez@gmail.com>
Renaming an inode creates a new record from scratch, so all existing
extended fields are lost. This hasn't been a problem so far for files
created by the module, because the only two supported xfields are always
rebuilt.
To do things correctly, move the creation of extended fields to the new
apfs_insert_xfield(), which is capable of handling xfield replacement as
well.
Also define apfs_init_xfields() to initialize the blob for an xfield
collection; use both functions to simplify apfs_build_dentry_val() and
apfs_build_inode_val().
Signed-off-by: Ernesto A. Fernández <ernesto.mnd.fernandez@gmail.com>
Define apfs_find_xfield() in the new xfield.c file; use it to replace
the ad hoc code that finds extended fields in inodes and dentries.
Creation of xfields will soon be moved to common code as well.
Signed-off-by: Ernesto A. Fernández <ernesto.mnd.fernandez@gmail.com>
Ernesto A. Fernández
Stan Skowronek
Ernesto A. Fernández