40 Commits
Author SHA1 Message Date
Ernesto A. Fernández 5a2db9bd68 0.3.12
This release fixes a serious data corruption bug for containers bigger
than ~1.32 TiB, which is immediately triggered by a writable mount. This
goes to show why mounts are still read-only by default.

Another much smaller data loss bug for much bigger containers (~7 TiB)
is also addressed.

Support is added for the 6.12 kernel release candidate. A bit early this
time, so let's hope the build doesn't break again for the final release.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
2024-11-01 20:06:39 -03:00
Ernesto A. Fernández 96e6d41bb7 Silence gcc warnings due to switch to min()
Building an old kernel (5.3) with an old version of gcc (4.9.2), I get
some warnings about "comparison of distinct pointer types". I'm pretty
sure this doesn't matter, and I don't even know if the blame lies with
the old compiler or the old kernel, but just silence this by making the
integer literals of the warning unsigned like the other argument.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
2024-10-31 17:01:30 -03:00
Woody Suwalski 6c2ce34a67 Fix whitespace damage in apfs_aops definition
This has been around for a while so I guess checkpatch isn't picking up
on it for some reason.

[eafer: took this from a bigger patch, added commit message]
Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
2024-10-31 16:46:20 -03:00
Ernesto A. Fernández a34220c93d Switch ->write_begin() and ->write_end() to folios
The build is broken for the 6.12 release candidate. The problem is that
->write_begin() and ->write_end() now take folios as parameters instead
of pages. The same is true, of course, of the associated
__block_write_begin() and generic_write_end(). Update the driver.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
2024-10-31 16:02:06 -03:00
Ernesto A. Fernández 49507c65bc Implement ip bitmap rotation correctly
I never fully understood how the ip bitmap list was supposed to work,
but now I'm noticing that the difference between what I do and what the
official driver does are bad enough to quickly corrupt the list when a
container moves back and forth. Of course this is only an issue for
filesystems above 1.32 TiB or so, but it's still pretty serious.

So, I thought about it harder and I'm pretty sure I finally figured it
out: the array entries for each free bitmap are actually kept in a
linked list, that starts in sm_ip_bm_free_head and ends in
sm_ip_bm_free_tail. All entries outside the list (that is, the entries
for the used blocks) are marked with the invalid index 0xFFFF. The tail
itself also gets 0xFFFF even though it's free, because it has no next
entry - so that accounts for the constant off-by-one weirdness in the
number of free blocks.

So, implement all this stuff correctly. Stop assuming that all ip
bitmaps in use are consecutive, and stop rotating them all in lockstep
as soon as the transaction begins. Instead, read all ip bitmaps into
memory during the first transaction and keep them there, only tracking
if they have been dirtied. At commit time, allocate new blocks for the
dirty bitmaps and write them back to disk.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
2024-10-29 21:13:46 -03:00
Ernesto A. Fernández dfdafd2a07 Wrap ephemeral objects around if there is no room
Currently, the driver doesn't allow multiblock ephemeral objects to wrap
around. When an object doesn't fit in the checkpoint data area, we just
leave some empty space and adjust the mappings accordingly. This seemed
like a simple and reasonable solution to me, but the official driver
actually demands that all ephemeral objects are kept consecutive (and in
the same order as their mappings, which we already do).

In practice this means that if you are working with a container with a
multiblock spaceman (say 7T in size), and if the unmount transaction
just so happens to be the one where the spaceman doesn't fit, then you
will be left with a container that is seen as corrupt by the official
driver. To make matters worse, checkpoint integrity is checked on mount,
so macos will actually mount the container no problem, but silently
discarding my final transaction.

This is all, of course, extremely unlikely. Such large containers are
rare, and checkpoint data areas for them are huge so a wrap-around takes
a while to happen. And it needs to happen precisely during the unmount
transaction, and that same transaction has to involve important data the
user doesn't want to lose. All in all it's not a serious issue, but get
it fixed.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
2024-10-22 17:34:51 -03:00
Integral 2920180ed6 Remove unused macro MIN()/MAX() & replace them with min()/max() (#78) 2024-10-04 16:04:05 -03:00
Ernesto A. Fernández 0667dbff17 0.3.11
This release prevents deadlock if a single volume gets mounted and
unmounted at the same time. It also fixes two other (smaller) mount bugs
found while testing this one.

Support is added for the upcoming 6.11 kernel release, with a patch by
Woody Suwalski and a later patch of my own because things broke a second
time.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
2024-09-12 20:20:49 -03:00
Ernesto A. Fernández af2ee526c0 Avoid double parsing of mount options
Originally all mount options were parsed inside apfs_fill_super(),
because I was following the example of other filesystems (such as ext2)
that have no subvolumes or snapshots. When I implemented those features,
I realized that some mount options would be needed early on mount, so
that sget() could use them to decide if the superblock is already
mounted.

My lazy solution at the time was to leave parse_options() as it was, but
also add some redundant code early on mount to read the needed options.
Just looking at the amount of TODOs in those functions makes it clear
that this was always very silly. And now that the locking during mount
has been simplified, this is causing a race, and I want to take that
stuff seriously.

The bug is that while parse_options() rereads the mount options needed
by sget(), there is nothing preventing another concurrent mount from
testing the superblock. Say the first mount is for volume 1: the volume
number gets briefly reset to the default value of zero, and if the other
mount actually wants volume 0, sget() will consider this a match.

Anyway, split parse_options() into two equally clean functions, one for
early mount and another to get called within apfs_fill_super(). I even
considered simply parsing all the options early during mount, but that
would require a few more changes because the nxi is not attached at that
point. Maybe some other time.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
2024-09-11 16:54:47 -03:00
Ernesto A. Fernández 44f4b4f631 Don't redefine MAX() and MIN() for the 6.11 kernel
The build has become broken once again for the 6.11 kernel:

  https://github.com/linux-apfs/linux-apfs-rw/issues/75

The problem is that the second release candidate introduced generic
MAX() and MIN() macros for everyone to use, and that crashes with our
own definitions. So, only define them for older kernels.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
2024-08-30 20:25:44 -03:00
Ernesto A. Fernández b44d2a37bf Don't forget to initialize the omap cache spinlock
The omap cache has been in place for years, and it seems that its
spinlock was never actually initialized before use. I have no idea what
the consequences of this are, but for one it does cause lockdep to turn
itself off after every mount. Fix it.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
2024-08-15 16:01:07 -03:00
Ernesto A. Fernández 4881d7c707 Don't return 0 on mount if sget() fails
For the previous patch I had to carefully test all the possible errors
during mount and unmount. I noticed that injecting an error inside
apfs_set_super() triggers npd. The cause is that the error code returned
by sget() gets swallowed and the kernel is allowed to think that the
mount succeeded.

This is not a new issue, it's been around since apfs_mount() was first
implemented. I never noticed because apfs_set_super() would only fail if
we ran out of anonymous devices. So fix it in a new commit.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
2024-08-15 16:01:07 -03:00
Ernesto A. Fernández 2982c8655f Don't use the mount mutex so much
Repeatedly running generic/604 of xfstests eventually hangs the kernel.
This is a test that forces a mount to race with an unmount. The problem
is a standard deadlock: ->put_super() gets called while the s_umount
lock is held, and then starts a transaction, which takes nxs_mutex;
meanwhile ->mount() first takes nxs_mutex and then calls sget(), which
takes s_umount.

Before this patch, I had never really put much thought as to where
exactly nxs_mutex is needed. It's obviously the lock for the list of
mounted containers, but it's also used as a generic way to avoid
modifications to a container while another of its subvolumes is getting
mounted. It's always been obvious that I'm holding it for too long and
in too many places, so I want to finally put some effort into figuring
this out now, and leave a record here so that I don't forget.

The first obvious point is that the mutex can be dropped before calling
sget(), and retaken later. There should be no problem if another mount
called it at the same time, and it's enough to fix the bug. But putting
some thought here exposes an existing issue with this code: sget() may
actually test some superblock that is already half-unmounted and had its
info freed, so apfs_test_super() is risking npd or use-after-free.

Looking at other filesystems I realize that my mistake was freeing the
info inside ->put_super(), because at that point the super is still
listed in the fs_supers list for apfs. Instead ->kill_sb() should free
the info itself, after generic_shutdown_super() is done (which calls
->put_super()), and after kill_super_notify() has taken care of the
list. Since kill_super_notify() can't be called from inside a driver, a
little hack is needed to call it through kill_anon_super(). The cleanups
for apfs_mount() also need to be reconsidered, since ->put_super() is
only called by deactivate_locked_super() after the root dentry is set,
but the rest of ->kill_sb() runs either way.

Going back to the mutex: even if a race for sget() itself is not an
issue, dropping it there does have other consequences. The reference
count for the container superblock is increased before sget(), and it's
later checked on some initializations to see if this is the first mount.
That won't do anymore, because a second mount could increase the refcnt
before the mutex is retaken, and the first mount would never complete
its work. Use other fields for this check: in apfs_read_main_super() use
the blocksize (which is zero before the first mount sets it); for
apfs_set_nx_flags(), define a new flag to tell if the other flags have
been set.

This already tells us two functions that definitely need to be protected
by the mutex, so that they can complete their initializations before any
other mount comes to check them. The situation is similar for
apfs_first_read_omap(), which goes through the container's volume list
to check for previous reads. After going line by line through the mount
code, I'm fairly confident that the mutex only needs to be held for
apfs_attach_nxi() and these three functions. For unmount it's the same:
apfs_free_main_super() and apfs_put_omap() require the mutex. The second
will now unset the omap pointer for the volume atomically, so that it
can be checked by apfs_first_read_omap(); it is renamed accordingly.

So far this is all about the races between different mounts and umounts;
but we also don't want the mount to read inconsistent data from disk
because some existing mount happened to write to it. Back when the
driver could only mount one volume at a time, apfs_fill_super() didn't
need any locking, so when multiple mounts got implemented I decided to
just ban transactions during mounts entirely, by taking the nxs_mutex.
This double lock looks very awkward and is a factor on the deadlock we
are fixing, so let's clean that up now: use the read semaphore itself to
protect apfs_fill_super(). Of course this means that two mounts for the
same container can now run through this function at the same time, so be
careful with changes to the shared nxi: we've already dealt with this
issue for apfs_set_nx_flags(), but apfs_set_trans_buffer_limit() might
also be unsafe. Just in case, move it to apfs_read_main_super(), it
makes more sense that way anyway.

By the way, apfs_read_main_super() also reads from disk, but it only
does it on the first mount, so there is no risk of racing against a
transaction. There is one function that writes without a transaction,
namely apfs_make_super_copy(), but that function takes the nxs_mutex to
ensure that no mount will begin during the superblock update.

I think that covers everything? I'm really not sure if this writeup was
clear at all, I have a hard time reasoning about locks. I also put a
simpler list of the uses of nxs_mutex in a comment next to its
declaration, which should be more practical as a reference.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
2024-08-15 16:01:06 -03:00
Ernesto A. Fernández 707d91b486 Undo code reordering in previous patch
I just picked up a patch to add support for the 6.11 kernel. I only did
a shallow review, and now I notice that the code was reordered and a
null buffer head is getting dereferenced. Revert that.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
2024-07-30 17:55:14 -03:00
Woody Suwalski 3e5c902a9a Kernel 6.11: use folio for page ops (#74) 2024-07-30 17:23:17 -03:00
Ernesto A. Fernández 695455bfdb 0.3.10
Transactions are now committed regularly, so it should no longer be
strictly necessary to unmount the filesystem cleanly for writes to take
effect. It's still recommended, of course.

The release also fixes two bugs. One was a big performance regression
from 0.3.9, seen when deleting large numbers of files. The other was
that mounts would go read-only if two snapshots were attempted with the
same name.

Support is added for the new 6.10 kernel version, with a patch by Woody
Suwalski.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
2024-07-15 15:13:17 -03:00
Woody Suwalski 071cdacc32 Small tweaks for building on kernel 6.10 2024-05-28 20:46:02 -03:00
Ernesto A. Fernández 57eb739f59 Don't abort a snapshot if the name is in use
Attempting to create two snapshots with the same name will abort a
transaction and leave the filesystem in a read-only state. From the
comment it seems that I was aware of this issue when I first implemented
snapshots, but I guess I didn't like the obvious solution in this patch
at the time, so I decided to leave it for later.

Since taking snapshots is not expected to be a bottleneck, just run the
query for the name once more ahead of time, as soon as the transaction
opens. Now if the name already exists we don't need to abort, because no
changes have taken place.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
2024-05-28 18:54:56 -03:00
Ernesto A. Fernández f088d1a2e3 Set up a task to commit transactions
Currently my driver only commits transactions "synchronously", that is,
immediately following a user action that triggers it. This implies, in
particular, that the filesystem must get unmounted explicitly or else
data will almost certainly get dropped. While technically valid
behaviour, this is very weird and some people have complained about it
in the past.

So, stop ignoring requests for commit via apfs_transaction_commit() when
they are considered premature. Instead schedule a commit task, with a
small delay to avoid constant commits when the driver is still busy.

Note that the previous two patches were intended to help with this one:
the plan was that the commit task wouldn't need to hold a specific
volume superblock, and could instead work with the nx info. In the end
this proved too messy, so I'm just saving the volume superblock that
requested the commit in the transaction struct.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
2024-05-23 18:46:25 -03:00
Ernesto A. Fernández 8bcaaebb89 Move trans_buffers_max from volume to container
The limit on the number of buffer heads in a transaction depends only on
the ram size, so it's the same for all volumes. Move it to the container
superblock info. Frankly this could just be a global variable, set when
the driver is first inserted, but there is no problem with recalculating
it on all mounts.

The point of this change, along with the previous patch, is that soon we
won't require holding some volume to start and commit transactions.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
2024-05-21 18:26:18 -03:00
Ernesto A. Fernández 781e0024d2 Stop attempting to undo aborted transactions
When I first implemented transactions, my intention was to revert the
in-memory filesystem structures to a consistent state whenever we hit
the kind of errors that can be expected during normal operation. So I
always kept some stuff from the previous transaction around to copy it
back if needed. I never tested any of this though, so it most likely
never worked.

Later improvements mean that the kind of silly errors I had in mind
(things like ENOENT) are no longer supposed to happen during
transactions. The one exception here is ENOSPC, but that's only because
I haven't yet bothered to clean up the space reservations. The errors
that do happen are things like ENOMEM, EIO or EFSCORRUPTED, and I think
it's probably ok if the mount doesn't work perfectly after one of those.
Instead we force it into a read-only state, and let the user decide how
to proceed.

So, there is now little point in keeping my original half-baked plans
littering the transaction code. Remove all that stuff.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
2024-05-20 20:06:53 -03:00
Ernesto A. Fernández 5d41bd7e64 Initialize transaction fields on first mount
The container superblock info keeps a list of inodes and buffers that
have been modified in the current transaction. This is a bit of a hack,
but so far I haven't figured out a way to get rid of it. The lists need
to be initialized before they are first used, and this is currently done
by apfs_transaction_start() itself, after checking if this is the first
start since the last commit. This works fine, but it's really weird,
since the list gets regularly reinitialized to "empty" after getting
emptied naturally by a commit. Initialization should really be separate
from all the code that actually works with the transaction.

Define a new apfs_transaction_init() to take care of this, and call it
from apfs_attach_nxi() which already takes care of the initialization of
all other container info fields.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
2024-05-20 15:18:14 -03:00
Ernesto A. Fernández 4e9984c815 Rename the transaction limit constants
TRANSACTION_MAIN_QUEUE_MAX, TRANSACTION_BUFFERS_MAX and
TRANSACTION_STARTS_MAX were originally desfined inside transaction.c, so
I guess I didn't bother to give them the driver prefix. I don't expect
the names to collide with anything, but it's tidier to follow the same
naming conventions everywhere, so run

  sed -i 's/TRANSACTION_/APFS_TRANS_/' *.{c,h}

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
2024-05-16 20:49:12 -03:00
Ernesto A. Fernández f1c9ab6c38 Adjust TRANSACTION_MAIN_QUEUE_MAX
The iozone automatic tests suffered a big performance regression between
0.3.8 and 0.3.9, in my case from around 14m to close to 20m. The problem
goes away if the newly introduced apfs_dstream_delete_front() is tweaked
to delete files in larger chunks. That said, I don't think my choice for
TRANSACTION_MAIN_QUEUE_MAX was ever based on anything, so it's better to
adjust that instead of coming up with some hack for file deletion alone.

After a lot of testing, I think I get the best results if I raise it
from 4096 to 10000. The iozone benchmark is now better than before, at
around 13m for me.

Of course this will mean that transactions commit less often. I should
look into putting that in a workqueue like the file deletion.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
2024-05-16 20:32:39 -03:00
Ernesto A. Fernández d533b6be3a 0.3.9
This release fixes the build for the soon to be released 6.9 kernel.

Support is tentatively added for the new 0x80 incompatible volume
feature, which appears simple enough so far and isn't causing any
problems.

A console warning that showed up during compressed file reads on recent
kernels is now fixed.

Support is improved for tiny filesystems of under 100 MiB, though they
remain awkward to test due to fragmentation issues.

The behaviour of apfs_modified_by is tweaked to better match the
official driver.

Free queues should now be much harder to fill up. The most important
change here is that files are now deleted across multiple transactions.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
2024-05-07 22:13:48 -03:00
Ernesto A. Fernández 02cb8bb8d9 Suuport volumes with the 0x80 incompatible feature
The official newfs_apfs now sets a 0x80 incompatible feature in all
volumes by default. The feature is undocumented as far as I know, but
after some testing it would seem that its effect is to set a flag in
preallocated extents. I can't imagine why such a feature would be
incompatible instead of read-only compatible, so there is a chance that
I'm missing something important, but for now enable support and let's
see if anything breaks.

By the way, no real changes are needed here because we don't preallocate
extents ourselves, and we put existing preallocated extents through the
same CoW as all the others. The only problem remaining is that writes
after the end of a file will not flip the "preallocation" flags that may
be set in previous blocks, but neither the official fsck nor the driver
seem to care about this. I just don't get what this flag is for, which
is a little scary of course.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
2024-05-03 21:25:50 -03:00
Ernesto A. Fernández 6e7371ae02 Sync apfs_raw.h with the fsck again
I've recently noticed a number of new flags and added them to the fsck.
I heven't quite figured out most of them yet, but copy the information
here anyway. Also add checks for the new incompatible features, so that
they don't get accepted by default.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
2024-05-03 20:54:26 -03:00
Ernesto A. Fernández 2a10ca61e6 Update the volume superblock with latest fields
If the latest newfs_apfs creates an image with the default options,
older versions of fsck_apfs complain about an unknown volume superblock
field called "apfs_doc_id_index_flags". Several such new fields appear
to have been added, and after some bit flipping I managed to retrieve
the names for all of them, except for "apfs_doc_id_tree_oid" which is
only a (probably correct) guess.

I have no plans for these fields right now, but document them all in
apfs_raw.h anyway, so that I don't forget.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
2024-04-23 19:51:27 -03:00
Ernesto A. Fernández 48f6df6262 Improve error reporting for orphan cleanups
Orphan cleanup explicitly reports the error code from apfs_iget() when
it gets swallowed, but it's always ENODATA so it looks silly. Leave the
message in place but ignore the error code.

Instead, report all error codes returned by apfs_clean_orphans() before
they get discarded by apfs_orphan_cleanup_work(). That information might
actually be useful at some point.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
2024-04-17 20:40:12 -03:00
Ernesto A. Fernández 33c9a99e96 Always update apfs_modified_by on unmount
The official reference is a little ambiguous about apfs_modified_by. On
its first introduction it says the following:

  If the implementation's information is already the last entry in this
  field, you can update the field as usual (creating a duplicate), or
  leave the field's value unmodified. Both behaviors are permitted.

So far the driver has followed these instructions and left the field
unmodified, but later on the reference describes the apfs_modified_by
timestamp as:

  The time that the program last modified this volume.

This seems to mean that the timestamp must always change, even if the
implementation in use has not. More importantly, on the matter of the
cloneinfo xid, the reference says the following:

  When unmounting a volume, the value of this field is set to the latest
  transaction identifier, the same as the apfs_modified_by field.

So apfs_modified_by must always get updated on unmount, or so it seems.
I looked at older checkpoints for a few official images, and indeed they
always seem to do this on unmount, and never before. Always by shifting.

Stop lawyering about the specification and just copy what the official
driver does.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
2024-04-17 19:09:38 -03:00
Ernesto A. Fernández ca493f9586 Don't give up when a free queue gets full
I keep running into situations where too much fragmentation gets a free
queue full, and the filesystem goes read-only with the "free queue has
too many nodes" message. I've been patching this up by trying to flush
more often, but in the end it's best to force an extra flush whenever we
run out of space.

Rework apfs_free_queue_insert_nocache() so that it checks for space
before actually inserting anything. If we are out of room, flush and
start again. Make this last-resort flush stronger with a new "force"
argument for apfs_flush_free_queue(), otherwise it might still be
insufficient. Also, stop preserving multiple transactions in the queues,
I don't really see the point and it makes all this stuff much worse.

With these changes I'm hoping that free queues will no longer fill up
in practice, but it's still possible in theory, so leave an error
message in place to identify this issue quickly.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
2024-04-12 17:24:04 -03:00
Ernesto A. Fernández 212cc6ef63 Make apfs_query_found_extent() check ids as well
I've been trying to run tests with a small image of around 100 MiB. I
wasn't expecting any problems, but surprisingly a lot of the tests fail.
The difference with larger containers appears to be that my hack to
avoid fragmentation (allocating data and metada in different chunks)
does not work when you only have one chunk. So this bug actually affects
any image, it's just easier to trigger here.

So, as for the bug itself: when working with large dstream xattrs (like
my apfs/002 test does), I regularly hit the "extent is not tail - bug!"
error. The problem is one that I've faced many times before with my
messy query code: queries looking for an extent without knowing it's
exact logical address can't be "exact" so they don't reliably return
ENODATA when they find nothing of value.

To deal with this, the extent code always checks the retrieved record by
calling apfs_query_found_extent(), but that only checks the record type.
In this particular case, the query is finding a proper file extent
record, but it belongs to a different dstream. I originally didn't think
such things could happen, but that has proved wrong more than once. From
now on, make apfs_query_found_extent() check the extent id as well.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
2024-04-11 16:36:13 -03:00
Ernesto A. Fernández fa6d1d3986 Check if fq root is empty before reading first key
After the last patch I'm regularly getting the following error message
in the console:

  APFS (800010g): index out of bounds (0 of 0) (apfs_node_locate_key:598)

What happens here is that the container xid is set to 1 by the mkfs, but
the ip fq is empty so the oldest xid is set to 0. So when the driver
first starts a transaction, apfs_flush_free_queue() will believe that
there is an actual "0" transaction to flush, and end up trying to update
the oldest xid field for the empty queue.

This is all a bit odd but not a real problem, as soon as the fq starts
getting used all the flush checks will work as expected. The annoying
message comes from apfs_free_queue_oldest_xid(), which directly calls
apfs_node_locate_key() for the first key in the root node, without
verifying if such a key even exists. Avoid that.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
2024-04-05 19:09:24 -03:00
Ernesto A. Fernández 076e715042 Preserve only one transaction in the ip free queue
Currently apfs_flush_free_queue() preserves several transactions in both
the ip and main free queue. This is a problem for tiny filesystems (5
MiB, for example) because the ip only has 6 blocks and inevitably uses 2
per transaction, getting full almost immediately. Allow more recent
transactions to get flushed for the ip.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
2024-04-05 16:10:31 -03:00
Ernesto A. Fernández 38f9100760 Get rid of kmap_atomic()
As mentioned in the code, my implementation of apfs_compress_readpage()
was mostly copied from ext4_read_inline_page() because it was simple
enough and I didn't know what I was doing. As it turns out, I picked the
wrong function to copy: ext4_read_inline_page() only reads in-memory
data and doesn't go to sleep. We do read from disk, and we do go to
sleep, so newer kernels flood the console with warnings.

Replace kmap_atomic() with kmap(). The only expected negative effect of
this change would be worse performance on 32-bit archs, but I haven't
even tested those in years. The alternative kmap_local_page() is not
available in old kernels.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
2024-03-27 21:09:50 -03:00
Ernesto A. Fernández cc320ed9ea Provide an explanation for forced read-only mounts
The driver requires setting an explicit "readwrite" mount option for an
actual writable mount to take place, otherwise the mount is silently
switched to read-only. The point is to prevent users that don't
understand the risks from potentially corrupting their data. Hopefully
this would push them to check the readme, but some might not even notice
that we have one (for example, if they got the driver as a distro
package). Try to explain the situation in the console.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
2024-03-26 20:47:48 -03:00
Ernesto A. Fernández f50c8f2084 Open block devices as files with Linux 6.9
The first 6.9 release candidate is out, and bdev_open_by_path() is gone.
We are no longer supposed to work with block device handles, we must use
a normal open file struct instead. Update the code following what was
done to btrfs in upstream commit 9ae061cf2a46 ("btrfs: port device
access to file").

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
2024-03-26 13:49:24 -03:00
Ernesto A. Fernández 21983fb418 Define SLAB_MEM_SPREAD flag for Linux 6.9
The first 6.9 release candidate is out, and the SLAB_MEM_SPREAD flag is
no longer available for kmem_cache_create(). Apparently it has been a
no-op for a while, so the least invasive way to fix the build is to just
define it as zero.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
2024-03-26 13:48:32 -03:00
Ernesto A. Fernández 514a2164d8 Split orphan cleanup among multiple transactions
Gilgamesh reports via email that the following iozone benchmark is
failing:

  ./iozone -a -i 0 -r 1M -n 1g -g 1g -f /mnt/apfs/test

I've tried this myself on a small (3 GiB) filesystem, and it is the
commit after the unlink that fails, complaining about the main free
queue being full:

  APFS (800010g): free queue has too many nodes (348 > 347) (apfs_free_queue_insert_nocache:849)

The problem is that random writes to a big file create a lot of extents,
and when the file is deleted they all get dumped to the main free queue
as separate records. The queue gets full and the commit fails.

Stop forcing file deletion to happen in a single transaction. Instead,
set up a workqueue whenever an orphan inode gets evicted, and leave the
cleanup for another time. Also allow the cleanup to execute only
partially (deleting the first extents like the official driver) and
reschedule itself (again via inode eviction).

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
2024-03-22 21:20:27 -03:00
Ernesto A. Fernández b8fe780c71 Don't leave stale record data on failed queries
When a query fails with ENODATA, the caller may still need to work with
the result (for instance to insert something in that location), but it
should never be confused for an actual valid record. Some changes I'm
working on trigger this problem regularly for apfs_query_found_extent(),
making it check stale records and report success. I need that function
to be reliable, so always clear the key and value lengths from queries
on failure.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
2024-03-18 18:45:37 -03:00
15 changed files with 1555 additions and 578 deletions

No files matched your search

+44 -31
View File
@@ -117,6 +117,11 @@ static inline bool apfs_node_has_fixed_kv_size(struct apfs_node *node)
return (node->flags & APFS_BTNODE_FIXED_KV_SIZE) != 0;
}
struct apfs_ip_bitmap_block_info {
bool dirty; /* Do we need to commit this block to disk? */
void *block; /* In-memory address of the bitmap block */
};
/*
* Space manager data in memory.
*/
@@ -147,12 +152,12 @@ struct apfs_spaceman {
/* Number of ip bitmaps */
u32 sm_ip_bmaps_count;
/* List of ip bitmaps, in order */
struct buffer_head *sm_ip_bmaps[];
struct apfs_ip_bitmap_block_info sm_ip_bmaps[];
};
#define TRANSACTION_MAIN_QUEUE_MAX 4096
#define TRANSACTION_BUFFERS_MAX 65536
#define TRANSACTION_STARTS_MAX 65536
#define APFS_TRANS_MAIN_QUEUE_MAX 10000
#define APFS_TRANS_BUFFERS_MAX 65536
#define APFS_TRANS_STARTS_MAX 65536
/* Possible states for the container transaction structure */
#define APFS_NX_TRANS_FORCE_COMMIT 1 /* Commit guaranteed */
@@ -166,22 +171,15 @@ struct apfs_spaceman {
struct apfs_nx_transaction {
unsigned int t_state;
struct delayed_work t_work; /* Work task for transaction commits */
struct super_block *t_work_sb; /* sb that last queued a commit */
struct list_head t_inodes; /* List of inodes in the transaction */
struct list_head t_buffers; /* List of buffers in the transaction */
size_t t_buffers_count; /* Count of items on the list */
int t_starts_count; /* Count of starts for transaction */
};
/*
* Structure that keeps track of a volume transaction.
*/
struct apfs_vol_transaction {
struct buffer_head *t_old_vsb; /* Volume superblock being replaced */
struct apfs_node t_old_omap_root; /* Omap root node being replaced */
struct apfs_node t_old_cat_root; /* Catalog root node being replaced */
};
/* State bits for buffer heads in a transaction */
#define BH_TRANS BH_PrivateStart /* Attached to a transaction */
#define BH_CSUM (BH_PrivateStart + 1) /* Requires checksum update */
@@ -224,13 +222,20 @@ struct apfs_ephemeral_object_info {
/* Mount option flags for a container */
#define APFS_CHECK_NODES 1
#define APFS_READWRITE 2
/*
* Mount options for a container are decided on its first mount. This flag lets
* future mounts know that has happened already.
*/
#define APFS_FLAGS_SET 4
/*
* Container superblock data in memory
*/
struct apfs_nxsb_info {
struct block_device *nx_bdev; /* Device for the container */
#if LINUX_VERSION_CODE >= KERNEL_VERSION(6, 8, 0)
#if LINUX_VERSION_CODE >= KERNEL_VERSION(6, 9, 0)
struct file *nx_bdev_file;
#elif LINUX_VERSION_CODE >= KERNEL_VERSION(6, 8, 0)
struct bdev_handle *nx_bdev_handle;
#endif
struct apfs_nx_superblock *nx_raw; /* On-disk main sb */
@@ -252,6 +257,7 @@ struct apfs_nxsb_info {
struct apfs_spaceman *nx_spaceman;
struct apfs_nx_transaction nx_transaction;
int nx_trans_buffers_max;
/* For now, a single semaphore for every operation */
struct rw_semaphore nx_big_sem;
@@ -331,10 +337,8 @@ struct apfs_sb_info {
struct apfs_crypto_state_val *s_dflt_pfk; /* default per-file key */
struct apfs_vol_transaction s_transaction;
int s_trans_buffers_max;
struct inode *s_private_dir; /* Inode for the private directory */
struct work_struct s_orphan_cleanup_work;
};
static inline struct apfs_sb_info *APFS_SB(struct super_block *sb)
@@ -626,20 +630,21 @@ static inline u64 apfs_cat_cnid(struct apfs_key_header *key)
}
/* Flags for the query structure */
#define APFS_QUERY_TREE_MASK 00177 /* Which b-tree we query */
#define APFS_QUERY_OMAP 00001 /* This is a b-tree object map query */
#define APFS_QUERY_CAT 00002 /* This is a catalog tree query */
#define APFS_QUERY_FREE_QUEUE 00004 /* This is a free queue query */
#define APFS_QUERY_EXTENTREF 00010 /* This is an extent reference query */
#define APFS_QUERY_FEXT 00020 /* This is a fext tree query */
#define APFS_QUERY_SNAP_META 00040 /* This is a snapshot meta query */
#define APFS_QUERY_OMAP_SNAP 00100 /* This is an omap snapshots query */
#define APFS_QUERY_NEXT 00200 /* Find next of multiple matches */
#define APFS_QUERY_EXACT 00400 /* Search for an exact match */
#define APFS_QUERY_DONE 01000 /* The search at this level is over */
#define APFS_QUERY_ANY_NAME 02000 /* Multiple search for any name */
#define APFS_QUERY_ANY_NUMBER 04000 /* Multiple search for any number */
#define APFS_QUERY_TREE_MASK 000177 /* Which b-tree we query */
#define APFS_QUERY_OMAP 000001 /* This is a b-tree object map query */
#define APFS_QUERY_CAT 000002 /* This is a catalog tree query */
#define APFS_QUERY_FREE_QUEUE 000004 /* This is a free queue query */
#define APFS_QUERY_EXTENTREF 000010 /* This is an extent reference query */
#define APFS_QUERY_FEXT 000020 /* This is a fext tree query */
#define APFS_QUERY_SNAP_META 000040 /* This is a snapshot meta query */
#define APFS_QUERY_OMAP_SNAP 000100 /* This is an omap snapshots query */
#define APFS_QUERY_NEXT 000200 /* Find next of multiple matches */
#define APFS_QUERY_EXACT 000400 /* Search for an exact match */
#define APFS_QUERY_DONE 001000 /* The search at this level is over */
#define APFS_QUERY_ANY_NAME 002000 /* Multiple search for any name */
#define APFS_QUERY_ANY_NUMBER 004000 /* Multiple search for any number */
#define APFS_QUERY_MULTIPLE (APFS_QUERY_ANY_NAME | APFS_QUERY_ANY_NUMBER)
#define APFS_QUERY_PREV 010000 /* Find previous record */
/*
* Structure used to retrieve data from an APFS B-Tree. For now only used
@@ -755,6 +760,8 @@ struct apfs_inode_info {
bool i_has_dstream; /* Is there a dstream record? */
struct apfs_dstream_info i_dstream; /* Dstream data, if any */
bool i_cleaned; /* Orphan data already deleted */
struct inode vfs_inode;
};
@@ -872,6 +879,7 @@ extern int apfs_btree_remove(struct apfs_query *query);
extern void apfs_btree_change_node_count(struct apfs_query *query, int change);
extern int apfs_btree_replace(struct apfs_query *query, void *key, int key_len,
void *val, int val_len);
extern void apfs_query_direct_forward(struct apfs_query *query);
/* compress.c */
extern int apfs_compress_get_size(struct inode *inode, loff_t *size);
@@ -919,6 +927,7 @@ extern int apfs_unlink(struct inode *dir, struct dentry *dentry);
extern int apfs_rmdir(struct inode *dir, struct dentry *dentry);
extern int apfs_delete_orphan_link(struct inode *inode);
extern int APFS_DELETE_ORPHAN_LINK_MAXOPS(void);
extern u64 apfs_any_orphan_ino(struct super_block *sb, u64 *ino_p);
/* extents.c */
extern int apfs_extent_from_query(struct apfs_query *query,
@@ -934,6 +943,7 @@ extern int apfs_get_new_block(struct inode *inode, sector_t iblock,
struct buffer_head *bh_result, int create);
extern int APFS_GET_NEW_BLOCK_MAXOPS(void);
extern int apfs_truncate(struct apfs_dstream_info *dstream, loff_t new_size);
extern int apfs_inode_delete_front(struct inode *inode);
#if LINUX_VERSION_CODE >= KERNEL_VERSION(4, 20, 0)
extern loff_t apfs_remap_file_range(struct file *src_file, loff_t off, struct file *dst_file, loff_t destoff, loff_t len, unsigned int remap_flags);
#else
@@ -951,6 +961,7 @@ extern int apfs_fsync(struct file *file, loff_t start, loff_t end, int datasync)
extern struct inode *apfs_iget(struct super_block *sb, u64 cnid);
extern int apfs_update_inode(struct inode *inode, char *new_name);
extern int APFS_UPDATE_INODE_MAXOPS(void);
extern void apfs_orphan_cleanup_work(struct work_struct *work);
extern void apfs_evict_inode(struct inode *inode);
extern struct inode *apfs_new_inode(struct inode *dir, umode_t mode,
dev_t rdev);
@@ -1060,6 +1071,7 @@ extern int apfs_read_spaceman(struct super_block *sb);
extern int apfs_free_queue_insert_nocache(struct super_block *sb, u64 bno, u64 count);
extern int apfs_free_queue_insert(struct super_block *sb, u64 bno, u64 count);
extern int apfs_spaceman_allocate_block(struct super_block *sb, u64 *bno, bool backwards);
extern int apfs_write_ip_bitmaps(struct super_block *sb);
/* super.c */
extern int apfs_map_volume_super_bno(struct super_block *sb, u64 bno, bool check);
@@ -1071,6 +1083,7 @@ extern int apfs_sync_fs(struct super_block *sb, int wait);
/* transaction.c */
extern int apfs_cpoint_data_free(struct super_block *sb, u64 bno);
extern void apfs_transaction_init(struct apfs_nx_transaction *trans);
extern int apfs_transaction_start(struct super_block *sb, struct apfs_max_ops maxops);
extern int apfs_transaction_commit(struct super_block *sb);
extern void apfs_inode_join_transaction(struct super_block *sb, struct inode *inode);
+48 -12
View File
@@ -268,6 +268,10 @@ struct apfs_phys_ext_val {
#define APFS_FILE_EXTENT_LEN_MASK 0x00ffffffffffffffULL
#define APFS_FILE_EXTENT_FLAG_MASK 0xff00000000000000ULL
#define APFS_FILE_EXTENT_FLAG_SHIFT 56
#define APFS_FILE_EXTENT_CRYPTO_FLAG 0x01 /* Made-up name */
#define APFS_FILE_EXTENT_PREALLOCATED 0x02 /* Made-up name */
#define APFS_VALID_FILE_EXTENT_FLAGS (APFS_FILE_EXTENT_CRYPTO_FLAG \
| APFS_FILE_EXTENT_PREALLOCATED)
/*
* Put a bound on maximum file size so that a growing truncation will always
@@ -1051,13 +1055,19 @@ struct apfs_checkpoint_map_phys {
#define APFS_FS_SPILLEDOVER 0x00000010LL
#define APFS_FS_RUN_SPILLOVER_CLEANER 0x00000020LL
#define APFS_FS_ALWAYS_CHECK_EXTENTREF 0x00000040LL
#define APFS_FS_PREVIOUSLY_SEALED 0x00000080LL /* Made-up name */
#define APFS_FS_PFK 0x00000100LL /* Made-up name */
#define APFS_FS_UNKNOWN_200 0x00000200LL
#define APFS_FS_FLAGS_VALID_MASK (APFS_FS_UNENCRYPTED \
| APFS_FS_EFFACEABLE \
| APFS_FS_RESERVED_4 \
| APFS_FS_ONEKEY \
| APFS_FS_SPILLEDOVER \
| APFS_FS_RUN_SPILLOVER_CLEANER \
| APFS_FS_ALWAYS_CHECK_EXTENTREF)
| APFS_FS_EFFACEABLE \
| APFS_FS_RESERVED_4 \
| APFS_FS_ONEKEY \
| APFS_FS_SPILLEDOVER \
| APFS_FS_RUN_SPILLOVER_CLEANER \
| APFS_FS_ALWAYS_CHECK_EXTENTREF \
| APFS_FS_PREVIOUSLY_SEALED \
| APFS_FS_PFK \
| APFS_FS_UNKNOWN_200)
#define APFS_FS_CRYPTOFLAGS (APFS_FS_UNENCRYPTED \
| APFS_FS_EFFACEABLE \
@@ -1124,7 +1134,9 @@ struct apfs_checkpoint_map_phys {
#define APFS_INCOMPAT_NORMALIZATION_INSENSITIVE 0x00000008LL
#define APFS_INCOMPAT_INCOMPLETE_RESTORE 0x00000010LL
#define APFS_INCOMPAT_SEALED_VOLUME 0x00000020LL
#define APFS_INCOMPAT_RESERVED_40 0x00000040LL
#define APFS_INCOMPAT_PFK 0x00000040LL /* Made-up name */
#define APFS_INCOMPAT_EXTENT_PREALLOC_FLAG 0x00000080LL /* Made-up name */
#define APFS_INCOMPAT_SECONDARY_FSROOT 0x00000100LL /* Made-up name */
#define APFS_SUPPORTED_INCOMPAT_MASK (APFS_INCOMPAT_CASE_INSENSITIVE \
| APFS_INCOMPAT_DATALESS_SNAPS \
@@ -1132,7 +1144,9 @@ struct apfs_checkpoint_map_phys {
| APFS_INCOMPAT_NORMALIZATION_INSENSITIVE \
| APFS_INCOMPAT_INCOMPLETE_RESTORE \
| APFS_INCOMPAT_SEALED_VOLUME \
| APFS_INCOMPAT_RESERVED_40)
| APFS_INCOMPAT_PFK \
| APFS_INCOMPAT_EXTENT_PREALLOC_FLAG \
| APFS_INCOMPAT_SECONDARY_FSROOT)
#define APFS_MODIFIED_NAMELEN 32
@@ -1162,6 +1176,18 @@ struct apfs_modified_by {
#define APFS_CRYPTO_RESERVED_5 5
#define APFS_UNASSIGNED_CRYPTO_ID (~0ULL)
/* Doc id index flags. I'm making up the names for now. */
#define APFS_DOC_ID_HAS_PREV_TREE 0x00000001
#define APFS_DOC_ID_UNKNOWN_02 0x00000002
#define APFS_DOC_ID_UNKNOWN_04 0x00000004
#define APFS_DOC_ID_UNKNOWN_08 0x00000008
#define APFS_DOC_ID_UNKNOWN_10 0x00000010
#define APFS_DOC_ID_VALID_FLAGS (APFS_DOC_ID_HAS_PREV_TREE \
| APFS_DOC_ID_UNKNOWN_02 \
| APFS_DOC_ID_UNKNOWN_04 \
| APFS_DOC_ID_UNKNOWN_08 \
| APFS_DOC_ID_UNKNOWN_10)
/*
* Structure used to store the encryption state
*/
@@ -1241,15 +1267,24 @@ struct apfs_superblock {
__le64 apfs_snap_meta_ext_oid;
char apfs_volume_group_id[UUID_SIZE];
/*3F0*/ char apfs_volume_group_id[UUID_SIZE];
__le64 apfs_integrity_meta_oid;
/*400*/ __le64 apfs_integrity_meta_oid;
__le64 apfs_fext_tree_oid;
__le32 apfs_fext_tree_type;
/*410*/ __le32 apfs_fext_tree_type;
__le32 reserved_type;
__le64 reserved_oid;
/*420*/ __le64 apfs_doc_id_index_xid;
__le32 apfs_doc_id_index_flags;
__le32 apfs_doc_id_tree_type;
/*430*/ __le64 apfs_doc_id_tree_oid; /* Made-up name */
__le64 apfs_prev_doc_id_tree_oid;
__le64 apfs_doc_id_fixup_cursor;
__le64 apfs_sec_root_tree_oid;
/*450*/ __le32 apfs_sec_root_tree_type;
} __packed;
/* Extended attributes constants */
@@ -1268,9 +1303,10 @@ enum {
APFS_XATTR_DATA_EMBEDDED = 0x00000002,
APFS_XATTR_FILE_SYSTEM_OWNED = 0x00000004,
APFS_XATTR_RESERVED_8 = 0x00000008,
APFS_XATTR_UNKNOWN_10 = 0x00000010,
};
#define APFS_XATTR_VALID_FLAGS 0x0000000f
#define APFS_XATTR_VALID_FLAGS 0x0000001f
/*
* Structure of the value of an extended attributes record
+51 -11
View File
@@ -398,8 +398,6 @@ struct apfs_query *apfs_alloc_query(struct apfs_node *node,
/* To be released by free_query. */
query->node = node;
/* Start the search with the last record and go backwards */
query->index = node->records;
if (parent) {
query->key = parent->key;
@@ -408,6 +406,15 @@ struct apfs_query *apfs_alloc_query(struct apfs_node *node,
query->depth = parent->depth + 1;
}
/*
* We start the search with the last record and go backwards, but
* some queries later use the PREV flag later to list them in order.
*/
if (query->flags & APFS_QUERY_PREV)
query->index = -1;
else
query->index = node->records;
return query;
}
@@ -514,7 +521,8 @@ next_node:
* than enough to map every block.
*/
apfs_err(sb, "btree is too high");
return -EFSCORRUPTED;
err = -EFSCORRUPTED;
goto fail;
}
err = apfs_node_query(sb, *query);
@@ -526,12 +534,16 @@ next_node:
err = apfs_query_set_before_first(sb, query);
if (err) {
apfs_err(sb, "failed to set before the first record");
return err;
goto fail;
}
return -ENODATA;
err = -ENODATA;
goto fail;
} else if (err == -EAGAIN) {
if (!(*query)->parent) /* We are at the root of the tree */
return -ENODATA;
if (!(*query)->parent) {
/* We are at the root of the tree */
err = -ENODATA;
goto fail;
}
/* Move back up one level and continue the query */
parent = (*query)->parent;
@@ -540,7 +552,7 @@ next_node:
*query = parent;
goto next_node;
} else if (err) {
return err;
goto fail;
}
if (apfs_node_is_leaf((*query)->node)) /* All done */
return 0;
@@ -549,14 +561,15 @@ next_node:
if (err) {
apfs_alert(sb, "bad index block: 0x%llx",
(*query)->node->object.block_nr);
return err;
goto fail;
}
/* Now go a level deeper and search the child */
node = apfs_read_node(sb, child_id, storage, false /* write */);
if (IS_ERR(node)) {
apfs_err(sb, "failed to read node 0x%llx", child_id);
return PTR_ERR(node);
err = PTR_ERR(node);
goto fail;
}
if (node->object.oid != child_id)
@@ -571,10 +584,16 @@ next_node:
if (!*query) {
apfs_node_free(node);
*query = parent;
return -ENOMEM;
err = -ENOMEM;
goto fail;
}
node = NULL;
goto next_node;
fail:
/* Don't leave stale record info here or some callers will use it */
(*query)->key_len = (*query)->len = 0;
return err;
}
static int __apfs_btree_replace(struct apfs_query *query, void *key, int key_len, void *val, int val_len);
@@ -1132,3 +1151,24 @@ int apfs_btree_replace(struct apfs_query *query, void *key, int key_len, void *v
apfs_btree_change_rec_count(query, 0 /* change */, key_len, val_len);
return 0;
}
/**
* apfs_query_direct_forward - Set a query to start listing records forwards
* @query: a successfully executed query
*
* Multiple queries list records backwards, but queries marked with this
* function after execution will go in the opposite direction.
*/
void apfs_query_direct_forward(struct apfs_query *query)
{
if (query->flags & APFS_QUERY_PREV)
return;
apfs_assert_query_is_valid(query);
ASSERT(apfs_node_is_leaf(query->node));
while (query) {
query->flags |= APFS_QUERY_PREV;
query = query->parent;
}
}
+3 -3
View File
@@ -417,16 +417,16 @@ static int apfs_compress_read_folio(struct file *filp, struct folio *folio)
static int apfs_compress_readpage(struct file *filp, struct page *page)
{
#endif
void *addr = NULL;
char *addr = NULL;
ssize_t ret;
loff_t off;
/* Mostly copied from ext4_read_inline_page() */
off = page->index << PAGE_SHIFT;
addr = kmap_atomic(page);
addr = kmap(page);
ret = apfs_compress_file_read_page(filp, addr, off);
flush_dcache_page(page);
kunmap_atomic(addr);
kunmap(page);
if (ret >= 0) {
zero_user_segment(page, ret, PAGE_SIZE);
SetPageUptodate(page);
+67 -6
View File
@@ -1158,13 +1158,13 @@ fail:
/**
* apfs_orphan_name - Get the name for an orphan inode's invisible link
* @inode: the vfs inode
* @qname: on return, the name assigned to the link
* @ino: the inode number
* @qname: on return, the name assigned to the link
*
* Returns 0 on success; the caller must remember to free @qname->name after
* use. Returns a negative error code in case of failure.
*/
static int apfs_orphan_name(struct inode *inode, struct qstr *qname)
static int apfs_orphan_name(u64 ino, struct qstr *qname)
{
int max_len;
char *name;
@@ -1174,7 +1174,7 @@ static int apfs_orphan_name(struct inode *inode, struct qstr *qname)
name = kmalloc(max_len, GFP_KERNEL);
if (!name)
return -ENOMEM;
qname->len = snprintf(name, max_len, "0x%llx-dead", apfs_ino(inode));
qname->len = snprintf(name, max_len, "0x%llx-dead", ino);
qname->name = name;
return 0;
}
@@ -1193,7 +1193,7 @@ static int apfs_create_orphan_link(struct inode *inode)
struct qstr qname;
int err = 0;
err = apfs_orphan_name(inode, &qname);
err = apfs_orphan_name(apfs_ino(inode), &qname);
if (err)
return err;
err = apfs_create_dentry_rec(inode, &qname, apfs_ino(priv_dir), 0 /* sibling_id */);
@@ -1236,7 +1236,7 @@ int apfs_delete_orphan_link(struct inode *inode)
struct apfs_drec drec;
int err;
err = apfs_orphan_name(inode, &qname);
err = apfs_orphan_name(apfs_ino(inode), &qname);
if (err)
return err;
@@ -1481,3 +1481,64 @@ out_abort:
apfs_transaction_abort(sb);
return err;
}
/**
* apfs_any_orphan_ino - Find the inode number for any orphaned regular file
* @sb: filesytem superblock
* @ino_p: on return, the inode number found
*
* Returns 0 on success, or a negative error code in case of failure, which may
* be -ENODATA if there are no orphan files.
*/
u64 apfs_any_orphan_ino(struct super_block *sb, u64 *ino_p)
{
struct apfs_sb_info *sbi = APFS_SB(sb);
struct apfs_query *query = NULL;
struct apfs_drec drec = {0};
struct qstr qname = {0};
bool hashed = apfs_is_normalization_insensitive(sb);
bool found = false;
int err;
query = apfs_alloc_query(sbi->s_cat_root, NULL /* parent */);
if (!query)
return -ENOMEM;
apfs_init_drec_key(sb, APFS_PRIV_DIR_INO_NUM, NULL /* name */, 0 /* name_len */, &query->key);
query->flags = APFS_QUERY_CAT | APFS_QUERY_MULTIPLE | APFS_QUERY_EXACT;
while (!found) {
err = apfs_btree_query(sb, &query);
if (err) {
if (err == -ENODATA)
goto out;
apfs_err(sb, "drec query failed for private dir");
goto out;
}
err = apfs_drec_from_query(query, &drec, hashed);
if (err) {
apfs_alert(sb, "bad dentry record in private dir");
goto out;
}
/* These files are deleted immediately by ->evict_inode() */
if (drec.type != DT_REG)
continue;
/*
* Confirm that this is an orphan file, because the official
* reference allows other uses for the private directory.
*/
err = apfs_orphan_name(drec.ino, &qname);
if (err)
goto out;
found = strcmp(drec.name, qname.name) == 0;
kfree(qname.name);
qname.name = NULL;
}
*ino_p = drec.ino;
out:
apfs_free_query(query);
query = NULL;
return err;
}
+1 -1
View File
@@ -1,5 +1,5 @@
PACKAGE_NAME="linux-apfs-rw"
PACKAGE_VERSION="0.3.8"
PACKAGE_VERSION="0.3.12"
BUILT_MODULE_NAME[0]="apfs"
DEST_MODULE_LOCATION[0]="/extra"
+128 -6
View File
@@ -8,8 +8,6 @@
#include <linux/blk_types.h>
#include "apfs.h"
#define MAX(X, Y) ((X) <= (Y) ? (Y) : (X))
/**
* apfs_ext_is_hole - Does this extent represent a hole in a sparse file?
* @extent: the extent to check
@@ -387,7 +385,7 @@ static int apfs_shrink_extent_tail(struct apfs_query *query, struct apfs_dstream
}
/**
* apfs_query_found_extent - Is this query pointing to an extent record?
* apfs_query_found_extent - Did this query find an extent with the right id?
* @query: the (successful) query that found the record
*/
static inline bool apfs_query_found_extent(struct apfs_query *query)
@@ -398,7 +396,12 @@ static inline bool apfs_query_found_extent(struct apfs_query *query)
if (query->key_len < sizeof(*hdr))
return false;
hdr = raw + query->key_off;
return apfs_cat_type(hdr) == APFS_TYPE_FILE_EXTENT;
if (apfs_cat_type(hdr) != APFS_TYPE_FILE_EXTENT)
return false;
if (apfs_cat_cnid(hdr) != query->key.id)
return false;
return true;
}
/**
@@ -1625,6 +1628,125 @@ int apfs_truncate(struct apfs_dstream_info *dstream, loff_t new_size)
return apfs_create_hole(dstream, old_blks, new_blks);
}
/**
* apfs_dstream_delete_front - Deletes as many leading extents as possible
* @sb: filesystem superblock
* @ds_id: id for the dstream to delete
*
* Returns 0 on success, or a negative error code in case of failure, which may
* be -ENODATA if there are no more extents, or -EAGAIN if the free queue is
* getting too full.
*/
static int apfs_dstream_delete_front(struct super_block *sb, u64 ds_id)
{
struct apfs_sb_info *sbi = APFS_SB(sb);
struct apfs_spaceman *sm = APFS_SM(sb);
struct apfs_spaceman_phys *sm_raw = sm->sm_raw;
struct apfs_spaceman_free_queue *fq = NULL;
struct apfs_query *query = NULL;
struct apfs_file_extent head;
bool first_match = true;
int ret;
fq = &sm_raw->sm_fq[APFS_SFQ_MAIN];
if (le64_to_cpu(fq->sfq_count) > APFS_TRANS_MAIN_QUEUE_MAX)
return -EAGAIN;
query = apfs_alloc_query(sbi->s_cat_root, NULL /* parent */);
if (!query)
return -ENOMEM;
apfs_init_file_extent_key(ds_id, 0, &query->key);
query->flags = APFS_QUERY_CAT;
next_extent:
ret = apfs_btree_query(sb, &query);
if (ret && ret != -ENODATA) {
apfs_err(sb, "query failed for first extent of id 0x%llx", ds_id);
goto out;
}
apfs_query_direct_forward(query);
if (!apfs_query_found_extent(query)) {
/*
* After the original lookup, the query may not be set to the
* first extent, but instead to the record that comes right
* before.
*/
if (first_match) {
first_match = false;
goto next_extent;
}
ret = -ENODATA;
goto out;
}
first_match = false;
ret = apfs_extent_from_query(query, &head);
if (ret) {
apfs_err(sb, "bad head extent record on dstream 0x%llx", ds_id);
goto out;
}
ret = apfs_btree_remove(query);
if (ret) {
apfs_err(sb, "removal failed for id 0x%llx, addr 0x%llx", ds_id, head.logical_addr);
goto out;
}
/*
* The official fsck doesn't complain about wrong sparse byte counts
* for orphans, so I guess we don't need to update them here
*/
if (!apfs_ext_is_hole(&head)) {
ret = apfs_range_put_reference(sb, head.phys_block_num, head.len);
if (ret) {
apfs_err(sb, "failed to put range 0x%llx-0x%llx", head.phys_block_num, head.len);
goto out;
}
ret = apfs_crypto_adj_refcnt(sb, head.crypto_id, -1);
if (ret) {
apfs_err(sb, "failed to take crypto id 0x%llx", head.crypto_id);
goto out;
}
}
if (le64_to_cpu(fq->sfq_count) <= APFS_TRANS_MAIN_QUEUE_MAX)
goto next_extent;
ret = -EAGAIN;
out:
apfs_free_query(query);
return ret;
}
/**
* apfs_inode_delete_front - Deletes as many leading extents as possible
* @inode: inode to delete
*
* Tries to delete all extents for @inode, in which case it returns 0. If the
* free queue is getting too full, deletes as much as is reasonable and returns
* -EAGAIN. May return other negative error codes as well.
*/
int apfs_inode_delete_front(struct inode *inode)
{
struct super_block *sb = inode->i_sb;
struct apfs_dstream_info *dstream = NULL;
struct apfs_inode_info *ai = APFS_I(inode);
int ret;
if (!ai->i_has_dstream)
return 0;
dstream = &ai->i_dstream;
ret = apfs_flush_extent_cache(dstream);
if (ret) {
apfs_err(sb, "extent cache flush failed for dstream 0x%llx", dstream->ds_id);
return ret;
}
ret = apfs_dstream_delete_front(sb, dstream->ds_id);
if (ret == -ENODATA)
return 0;
return ret;
}
#if LINUX_VERSION_CODE >= KERNEL_VERSION(4, 20, 0)
loff_t apfs_remap_file_range(struct file *src_file, loff_t off, struct file *dst_file, loff_t destoff, loff_t len, unsigned int remap_flags)
#else
@@ -1849,7 +1971,7 @@ restart:
prev_end = prev_ext.bno + prev_ext.blkcount;
if (prev_end < *paddr_end) {
/* The extent to put is part of a snapshot */
tmp.phys_block_num = MAX(prev_end, paddr_min);
tmp.phys_block_num = max(prev_end, paddr_min);
tmp.len = (*paddr_end - tmp.phys_block_num) << sb->s_blocksize_bits;
ret = apfs_create_update_pext(query, &tmp, -1);
*paddr_end = tmp.phys_block_num;
@@ -1989,7 +2111,7 @@ restart:
prev_end = prev_ext.bno + prev_ext.blkcount;
if (prev_end < *paddr_end) {
/* The extent to take is part of a snapshot */
tmp.phys_block_num = MAX(prev_end, paddr_min);
tmp.phys_block_num = max(prev_end, paddr_min);
tmp.len = (*paddr_end - tmp.phys_block_num) << sb->s_blocksize_bits;
ret = apfs_create_update_pext(query, &tmp, +1);
*paddr_end = tmp.phys_block_num;
+236 -32
View File
@@ -565,7 +565,11 @@ int __apfs_write_begin(struct file *file, struct address_space *mapping, loff_t
}
}
#if LINUX_VERSION_CODE >= KERNEL_VERSION(6, 12, 0)
err = __block_write_begin(page_folio(page), pos, len, apfs_get_new_block);
#else
err = __block_write_begin(page, pos, len, apfs_get_new_block);
#endif
if (err) {
apfs_err(sb, "CoW failed in inode 0x%llx", apfs_ino(inode));
goto out_put_page;
@@ -580,7 +584,11 @@ out_put_page:
return err;
}
#if LINUX_VERSION_CODE >= KERNEL_VERSION(5, 19, 0)
#if LINUX_VERSION_CODE >= KERNEL_VERSION(6, 12, 0)
static int apfs_write_begin(struct file *file, struct address_space *mapping,
loff_t pos, unsigned int len,
struct folio **foliop, void **fsdata)
#elif LINUX_VERSION_CODE >= KERNEL_VERSION(5, 19, 0)
static int apfs_write_begin(struct file *file, struct address_space *mapping,
loff_t pos, unsigned int len,
struct page **pagep, void **fsdata)
@@ -592,6 +600,10 @@ static int apfs_write_begin(struct file *file, struct address_space *mapping,
{
struct inode *inode = mapping->host;
struct super_block *sb = inode->i_sb;
#if LINUX_VERSION_CODE >= KERNEL_VERSION(6, 12, 0)
struct page *page = NULL;
struct page **pagep = &page;
#endif
int blkcount = (len + sb->s_blocksize - 1) >> inode->i_blkbits;
struct apfs_max_ops maxops;
int err;
@@ -615,6 +627,9 @@ static int apfs_write_begin(struct file *file, struct address_space *mapping,
err = __apfs_write_begin(file, mapping, pos, len, flags, pagep, fsdata);
if (err)
goto fail;
#if LINUX_VERSION_CODE >= KERNEL_VERSION(6, 12, 0)
*foliop = page_folio(page);
#endif
return 0;
fail:
@@ -628,7 +643,11 @@ int __apfs_write_end(struct file *file, struct address_space *mapping, loff_t po
struct apfs_dstream_info *dstream = &APFS_I(inode)->i_dstream;
int ret, err;
#if LINUX_VERSION_CODE >= KERNEL_VERSION(6, 12, 0)
ret = generic_write_end(file, mapping, pos, len, copied, page_folio(page), fsdata);
#else
ret = generic_write_end(file, mapping, pos, len, copied, page, fsdata);
#endif
dstream->ds_size = i_size_read(inode);
if (ret < len && pos + len > inode->i_size) {
truncate_pagecache(inode, inode->i_size);
@@ -641,13 +660,22 @@ int __apfs_write_end(struct file *file, struct address_space *mapping, loff_t po
return ret;
}
#if LINUX_VERSION_CODE >= KERNEL_VERSION(6, 12, 0)
static int apfs_write_end(struct file *file, struct address_space *mapping,
loff_t pos, unsigned int len, unsigned int copied,
struct folio *folio, void *fsdata)
#else
static int apfs_write_end(struct file *file, struct address_space *mapping,
loff_t pos, unsigned int len, unsigned int copied,
struct page *page, void *fsdata)
#endif
{
struct inode *inode = mapping->host;
struct super_block *sb = inode->i_sb;
struct apfs_nx_transaction *trans = &APFS_NXI(sb)->nx_transaction;
#if LINUX_VERSION_CODE >= KERNEL_VERSION(6, 12, 0)
struct page *page = &folio->page;
#endif
int ret, err;
ret = __apfs_write_end(file, mapping, pos, len, copied, page, fsdata);
@@ -693,9 +721,9 @@ static const struct address_space_operations apfs_aops = {
#endif
#if LINUX_VERSION_CODE >= KERNEL_VERSION(5, 8, 0)
.readahead = apfs_readahead,
.readahead = apfs_readahead,
#else
.readpages = apfs_readpages,
.readpages = apfs_readpages,
#endif
.write_begin = apfs_write_begin,
@@ -886,7 +914,9 @@ static struct apfs_query *apfs_inode_lookup(const struct inode *inode)
if (!ret)
return query;
apfs_err(sb, "query failed for id 0x%llx", apfs_ino(inode));
/* Don't complain if an orphan is already gone */
if (!current_work() || ret != -ENODATA)
apfs_err(sb, "query failed for id 0x%llx", apfs_ino(inode));
apfs_free_query(query);
return ERR_PTR(ret);
}
@@ -1005,8 +1035,10 @@ struct inode *apfs_iget(struct super_block *sb, u64 cnid)
down_read(&nxi->nx_big_sem);
query = apfs_inode_lookup(inode);
if (IS_ERR(query)) {
apfs_err(sb, "lookup failed for ino 0x%llx", cnid);
err = PTR_ERR(query);
/* Don't complain if an orphan is already gone */
if (!current_work() || err != -ENODATA)
apfs_err(sb, "lookup failed for ino 0x%llx", cnid);
goto fail;
}
err = apfs_inode_from_query(query, inode);
@@ -1540,13 +1572,16 @@ int APFS_UPDATE_INODE_MAXOPS(void)
* apfs_delete_inode - Delete an inode record
* @inode: the vfs inode to delete
*
* Returns 0 on success or a negative error code in case of failure.
* Returns 0 on success or a negative error code in case of failure, which may
* be -EAGAIN if the inode was not deleted in full.
*/
static int apfs_delete_inode(struct inode *inode)
{
struct super_block *sb = inode->i_sb;
struct apfs_inode_info *ai = APFS_I(inode);
struct apfs_dstream_info *dstream = NULL;
struct apfs_query *query;
u64 old_dstream_id;
int ret;
ret = apfs_delete_all_xattrs(inode);
@@ -1555,6 +1590,9 @@ static int apfs_delete_inode(struct inode *inode)
return ret;
}
dstream = &ai->i_dstream;
old_dstream_id = dstream->ds_id;
/*
* This is very wasteful since all the new extents and references will
* get deleted right away, but it only affects clones, so I don't see a
@@ -1566,12 +1604,27 @@ static int apfs_delete_inode(struct inode *inode)
return ret;
}
/* TODO: truncate an orphan inode in multiple transactions */
dstream = &APFS_I(inode)->i_dstream;
ret = apfs_truncate(dstream, 0 /* new_size */);
/* TODO: what about partial deletion of xattrs? Is that allowed? */
ret = apfs_inode_delete_front(inode);
if (ret) {
apfs_err(sb, "truncation failed for ino 0x%llx", apfs_ino(inode));
return ret;
/*
* If the inode had too many extents, only the first few get
* deleted and the inode remains in the orphan list for now.
* I don't know why the deletion starts at the front, but it
* seems to be what the official driver does.
*/
if (ret != -EAGAIN) {
apfs_err(sb, "head deletion failed for ino 0x%llx", apfs_ino(inode));
return ret;
}
if (dstream->ds_id != old_dstream_id) {
ret = apfs_update_inode(inode, NULL /* new_name */);
if (ret) {
apfs_err(sb, "dstream id update failed for orphan 0x%llx", apfs_ino(inode));
return ret;
}
}
return -EAGAIN;
}
ret = apfs_put_dstream_rec(dstream);
@@ -1579,6 +1632,8 @@ static int apfs_delete_inode(struct inode *inode)
apfs_err(sb, "failed to put dstream for ino 0x%llx", apfs_ino(inode));
return ret;
}
dstream = NULL;
ai->i_has_dstream = false;
query = apfs_inode_lookup(inode);
if (IS_ERR(query)) {
@@ -1587,42 +1642,191 @@ static int apfs_delete_inode(struct inode *inode)
}
ret = apfs_btree_remove(query);
apfs_free_query(query);
if (ret)
if (ret) {
apfs_err(sb, "removal failed for ino 0x%llx", apfs_ino(inode));
return ret;
}
ai->i_cleaned = true;
return ret;
}
#define APFS_DELETE_INODE_MAXOPS 1
/**
* apfs_clean_single_orphan - Clean the given orphan file
* @inode: inode for the file to clean
*
* Returns 0 on success or a negative error code in case of failure, which may
* be -EAGAIN if the file could not be deleted in full.
*/
static int apfs_clean_single_orphan(struct inode *inode)
{
struct super_block *sb = inode->i_sb;
struct apfs_max_ops maxops = {0}; /* TODO: rethink this stuff */
u64 ino = apfs_ino(inode);
bool eagain = false;
int err;
err = apfs_transaction_start(sb, maxops);
if (err)
return err;
err = apfs_delete_inode(inode);
if (err) {
if (err != -EAGAIN) {
apfs_err(sb, "failed to delete orphan 0x%llx", ino);
goto fail;
}
eagain = true;
} else {
err = apfs_delete_orphan_link(inode);
if (err) {
apfs_err(sb, "failed to unlink orphan 0x%llx", ino);
goto fail;
}
}
err = apfs_transaction_commit(sb);
if (err)
goto fail;
return eagain ? -EAGAIN : 0;
fail:
apfs_transaction_abort(sb);
return err;
}
/**
* apfs_clean_any_orphan - Pick an orphan and delete as much as reasonable
* @sb: filesystem superblock
*
* Returns 0 on success, or a negative error code in case of failure, which may
* be -ENODATA if there are no more orphan files or -EAGAIN if a file could not
* be deleted in full.
*/
static int apfs_clean_any_orphan(struct super_block *sb)
{
struct apfs_nxsb_info *nxi = APFS_NXI(sb);
struct inode *inode = NULL;
int err;
u64 ino;
down_read(&nxi->nx_big_sem);
err = apfs_any_orphan_ino(sb, &ino);
up_read(&nxi->nx_big_sem);
if (err) {
if (err == -ENODATA)
return -ENODATA;
apfs_err(sb, "failed to find orphan inode numbers");
return err;
}
inode = apfs_iget(sb, ino);
if (IS_ERR(inode)) {
err = PTR_ERR(inode);
if (err != -ENODATA) {
apfs_err(sb, "iget failed for orphan 0x%llx", ino);
return err;
}
/*
* This happens rarely for files with no extents, if we hit a
* race with ->evict_inode(). Not a problem: the file is gone.
*/
apfs_notice(sb, "orphan 0x%llx not found", ino);
return 0;
}
if (atomic_read(&inode->i_count) > 1)
goto out;
err = apfs_clean_single_orphan(inode);
if (err && err != -EAGAIN) {
apfs_err(sb, "failed to clean orphan 0x%llx", ino);
goto out;
}
out:
iput(inode);
return err;
}
/**
* apfs_clean_orphans - Delete as many orphan files as is reasonable
* @sb: filesystem superblock
*
* Returns 0 on success or a negative error code in case of failure.
*/
static int apfs_clean_orphans(struct super_block *sb)
{
int ret, i;
for (i = 0; i < 100; ++i) {
ret = apfs_clean_any_orphan(sb);
if (ret) {
if (ret == -ENODATA)
return 0;
if (ret == -EAGAIN)
break;
apfs_err(sb, "failed to delete an orphan file");
return ret;
}
}
/*
* If a file is too big, or if there are too many files, take a break
* and continue later.
*/
if (atomic_read(&sb->s_active) != 0)
schedule_work(&APFS_SB(sb)->s_orphan_cleanup_work);
return 0;
}
void apfs_evict_inode(struct inode *inode)
{
struct super_block *sb = inode->i_sb;
struct apfs_max_ops maxops;
struct apfs_inode_info *ai = APFS_I(inode);
int err;
if (is_bad_inode(inode) || inode->i_nlink)
goto out_clear;
if (is_bad_inode(inode) || inode->i_nlink || ai->i_cleaned)
goto out;
maxops.cat = APFS_DELETE_INODE_MAXOPS + APFS_DELETE_ORPHAN_LINK_MAXOPS();
maxops.blks = 0;
if (!ai->i_has_dstream || ai->i_dstream.ds_size == 0) {
/* For files with no extents, scheduled cleanup wastes time */
err = apfs_clean_single_orphan(inode);
if (err)
apfs_err(sb, "failed to clean orphan 0x%llx (err:%d)", apfs_ino(inode), err);
goto out;
}
if (apfs_transaction_start(sb, maxops))
goto out_report;
if (apfs_delete_inode(inode))
goto out_abort;
if (apfs_delete_orphan_link(inode))
goto out_abort;
if (apfs_transaction_commit(sb))
goto out_abort;
goto out_clear;
out_abort:
apfs_transaction_abort(sb);
out_report:
apfs_err(sb, "failed to delete orphan inode 0x%llx", apfs_ino(inode));
out_clear:
/*
* If the inode still has extents then schedule cleanup for the rest
* of it. Not during unmount though: completing all cleanup could take
* a while so just leave future mounts to handle the orphans.
*/
if (atomic_read(&sb->s_active))
schedule_work(&APFS_SB(sb)->s_orphan_cleanup_work);
out:
truncate_inode_pages_final(&inode->i_data);
clear_inode(inode);
}
void apfs_orphan_cleanup_work(struct work_struct *work)
{
struct super_block *sb = NULL;
struct apfs_sb_info *sbi = NULL;
struct inode *priv = NULL;
int err;
sbi = container_of(work, struct apfs_sb_info, s_orphan_cleanup_work);
priv = sbi->s_private_dir;
sb = priv->i_sb;
if (sb->s_flags & SB_RDONLY) {
apfs_alert(sb, "attempt to flush orphans in read-only mount");
return;
}
err = apfs_clean_orphans(sb);
if (err)
apfs_err(sb, "orphan cleanup failed (err:%d)", err);
}
/**
* apfs_insert_inode_locked - Wrapper for insert_inode_locked4()
* @inode: vfs inode to insert in cache
-2
View File
@@ -12,8 +12,6 @@
#include <linux/string.h>
#include "libzbitmap.h"
#define MIN(x, y) ((x) > (y) ? (y) : (x))
#define ZBM_MAGIC "ZBM\x09"
#define ZBM_MAGIC_SZ 4
+38
View File
@@ -803,6 +803,42 @@ static int apfs_key_from_query(struct apfs_query *query, struct apfs_key *key)
return err;
}
/**
* apfs_node_prev - Find the previous record in the current node
* @sb: filesystem superblock
* @query: query in execution
*
* Returns 0 on success, -EAGAIN if the previous record is in another node,
* -ENODATA if no more records exist, or another negative error code in case
* of failure.
*
* The meaning of "next" and "previous" is reverted here, because regular
* multiple always start with the final record, and then they go backwards.
* TODO: consider renaming this for clarity.
*/
static int apfs_node_prev(struct super_block *sb, struct apfs_query *query)
{
struct apfs_node *node = query->node;
if (query->index + 1 == node->records) {
/* The next record may be in another node */
return -EAGAIN;
}
++query->index;
query->key_len = apfs_node_locate_key(node, query->index, &query->key_off);
if (query->key_len == 0) {
apfs_err(sb, "bad key for index %d", query->index);
return -EFSCORRUPTED;
}
query->len = apfs_node_locate_data(node, query->index, &query->off);
if (query->len == 0) {
apfs_err(sb, "bad value for index %d", query->index);
return -EFSCORRUPTED;
}
return 0;
}
/**
* apfs_node_next - Find the next matching record in the current node
* @sb: filesystem superblock
@@ -892,6 +928,8 @@ int apfs_node_query(struct super_block *sb, struct apfs_query *query)
int cmp;
int err;
if (query->flags & APFS_QUERY_PREV)
return apfs_node_prev(sb, query);
if (query->flags & APFS_QUERY_NEXT)
return apfs_node_next(sb, query);
+72 -3
View File
@@ -149,9 +149,9 @@ static int apfs_create_snap_name_rec(struct apfs_node *snap_root, const char *na
err = apfs_btree_query(sb, &query);
if (err == 0) {
/* TODO: avoid transaction abort here */
apfs_info(sb, "a snapshot with that name already exists");
err = -EEXIST;
/* This should never happen here, the caller has checked */
apfs_alert(sb, "snapshot name record already exists");
err = -EFSCORRUPTED;
goto fail;
}
if (err != -ENODATA) {
@@ -332,6 +332,56 @@ static int apfs_update_omap_snapshots(struct super_block *sb)
return err;
}
/**
* apfs_snapshot_name_check - Check if a snapshot with the given name exists
* @sb: filesystem superblock
* @name: name to check
* @name_len: length of @name
* @eexist: on return, whether the name exists or not
*
* Returns 0 on success, or a negative error code in case of failure.
*/
static int apfs_snapshot_name_check(struct super_block *sb, const char *name, int name_len, bool *eexist)
{
struct apfs_sb_info *sbi = APFS_SB(sb);
struct apfs_superblock *vsb_raw = sbi->s_vsb_raw;
struct apfs_node *snap_root = NULL;
struct apfs_query *query = NULL;
int err;
snap_root = apfs_read_node(sb, le64_to_cpu(vsb_raw->apfs_snap_meta_tree_oid), APFS_OBJ_PHYSICAL, false /* write */);
if (IS_ERR(snap_root)) {
apfs_err(sb, "failed to read snap meta root 0x%llx", le64_to_cpu(vsb_raw->apfs_snap_meta_tree_oid));
return PTR_ERR(snap_root);
}
vsb_raw = NULL;
query = apfs_alloc_query(snap_root, NULL /* parent */);
if (!query) {
err = -ENOMEM;
goto out;
}
apfs_init_snap_name_key(name, &query->key);
query->flags |= APFS_QUERY_SNAP_META | APFS_QUERY_EXACT;
err = apfs_btree_query(sb, &query);
if (err == 0) {
*eexist = true;
} else if (err == -ENODATA) {
*eexist = false;
err = 0;
} else {
apfs_err(sb, "query failed (%s)", name);
}
out:
apfs_free_query(query);
query = NULL;
apfs_node_free(snap_root);
snap_root = NULL;
return err;
}
/**
* apfs_do_ioc_takesnapshot - Actual work for apfs_ioc_take_snapshot()
* @mntpoint: inode of the mount point to snapshot
@@ -348,12 +398,31 @@ static int apfs_do_ioc_take_snapshot(struct inode *mntpoint, const char *name, i
/* TODO: remember to update the maxops in the future */
struct apfs_max_ops maxops = {0};
u64 sblock_oid;
bool eexist = false;
int err;
err = apfs_transaction_start(sb, maxops);
if (err)
return err;
/*
* This check can fail in normal operation, so run it before making any
* changes and exit the transaction cleanly if necessary. The name
* lookup will have to be repeated later, but it's ok because I don't
* expect snapshot creation to be a bottleneck for anyone.
*/
err = apfs_snapshot_name_check(sb, name, name_len, &eexist);
if (err) {
apfs_err(sb, "snapshot name existence check failed");
goto fail;
}
if (eexist) {
err = apfs_transaction_commit(sb);
if (err)
goto fail;
return -EEXIST;
}
/*
* Flush the extent caches to the extenref tree before it gets moved to
* the snapshot. It seems safer in general to avoid big unpredictable
+347 -163
View File
File diff suppressed because it is too large. Load diff
+392 -188
View File
File diff suppressed because it is too large. Load diff
+125 -116
View File
@@ -20,8 +20,12 @@ static int apfs_checkpoint_end(struct super_block *sb)
struct apfs_nxsb_info *nxi = APFS_NXI(sb);
struct apfs_obj_phys *obj = &nxi->nx_raw->nx_o;
struct buffer_head *bh = NULL;
#if LINUX_VERSION_CODE >= KERNEL_VERSION(6, 10, 0)
struct address_space *bdev_map = nxi->nx_bdev->bd_mapping;
#else
struct inode *bdev_inode = nxi->nx_bdev->bd_inode;
struct address_space *bdev_map = bdev_inode->i_mapping;
#endif
int err;
ASSERT(!(sb->s_flags & SB_RDONLY));
@@ -97,13 +101,18 @@ static bool apfs_transaction_has_room(struct super_block *sb, struct apfs_max_op
static int apfs_read_single_ephemeral_object(struct super_block *sb, struct apfs_checkpoint_mapping *map)
{
struct apfs_nxsb_info *nxi = APFS_NXI(sb);
struct apfs_nx_superblock *raw_sb = NULL;
struct apfs_ephemeral_object_info *list = NULL;
struct buffer_head *bh = NULL;
char *object = NULL;
int count;
u32 size;
u64 bno, oid;
int err, i;
u32 data_blks, size;
u64 data_base, bno, oid;
int err, i, data_idx;
raw_sb = nxi->nx_raw;
data_base = le64_to_cpu(raw_sb->nx_xp_data_base);
data_blks = le32_to_cpu(raw_sb->nx_xp_data_blocks);
list = nxi->nx_eph_list;
count = nxi->nx_eph_count;
@@ -131,8 +140,9 @@ static int apfs_read_single_ephemeral_object(struct super_block *sb, struct apfs
if (!object)
return -ENOMEM;
data_idx = bno - data_base;
for (i = 0; i < size >> sb->s_blocksize_bits; ++i) {
bh = apfs_sb_bread(sb, bno + i);
bh = apfs_sb_bread(sb, data_base + data_idx);
if (!bh) {
apfs_err(sb, "failed to read ephemeral block");
err = -EIO;
@@ -141,6 +151,9 @@ static int apfs_read_single_ephemeral_object(struct super_block *sb, struct apfs
memcpy(object + (i << sb->s_blocksize_bits), bh->b_data, sb->s_blocksize);
brelse(bh);
bh = NULL;
/* Somewhat surprisingly, objects can wrap around */
if (++data_idx == data_blks)
data_idx = 0;
}
/*
@@ -262,6 +275,53 @@ static int apfs_read_ephemeral_objects(struct super_block *sb)
return 0;
}
static void apfs_trans_commit_work(struct work_struct *work)
{
struct super_block *sb = NULL;
struct apfs_nxsb_info *nxi = NULL;
struct apfs_nx_transaction *trans = NULL;
int err;
trans = container_of(to_delayed_work(work), struct apfs_nx_transaction, t_work);
nxi = container_of(trans, struct apfs_nxsb_info, nx_transaction);
sb = trans->t_work_sb;
/*
* If sb is set then the transaction already started, there is no need
* for apfs_transaction_start() here. It would be cleaner to call it
* anyway (and check in there if sb is set), but maxops is a problem
* because we don't need any space. I really need to rethink that stuff
* (TODO).
*/
down_write(&nxi->nx_big_sem);
if (!sb || sb->s_flags & SB_RDONLY) {
/* The commit already took place, or there was an abort */
up_write(&nxi->nx_big_sem);
return;
}
trans->t_state |= APFS_NX_TRANS_FORCE_COMMIT;
err = apfs_transaction_commit(sb);
if (err) {
apfs_err(sb, "queued commit failed (err:%d)", err);
apfs_transaction_abort(sb);
}
}
/**
* apfs_transaction_init - Initialize the transaction struct for the container
* @trans: the transaction structure
*/
void apfs_transaction_init(struct apfs_nx_transaction *trans)
{
trans->t_state = 0;
INIT_DELAYED_WORK(&trans->t_work, apfs_trans_commit_work);
INIT_LIST_HEAD(&trans->t_inodes);
INIT_LIST_HEAD(&trans->t_buffers);
trans->t_buffers_count = 0;
trans->t_starts_count = 0;
}
/**
* apfs_transaction_start - Begin a new transaction
* @sb: superblock structure
@@ -272,18 +332,14 @@ static int apfs_read_ephemeral_objects(struct super_block *sb)
*/
int apfs_transaction_start(struct super_block *sb, struct apfs_max_ops maxops)
{
struct apfs_sb_info *sbi = APFS_SB(sb);
struct apfs_nxsb_info *nxi = APFS_NXI(sb);
struct apfs_vol_transaction *vol_trans = &sbi->s_transaction;
struct apfs_nx_transaction *nx_trans = &nxi->nx_transaction;
int err;
down_write(&nxi->nx_big_sem);
mutex_lock(&nxs_mutex); /* Don't mount during a transaction */
if (sb->s_flags & SB_RDONLY) {
/* A previous transaction has failed; this should be rare */
mutex_unlock(&nxs_mutex);
up_write(&nxi->nx_big_sem);
return -EROFS;
}
@@ -295,7 +351,6 @@ int apfs_transaction_start(struct super_block *sb, struct apfs_max_ops maxops)
if (!nxi->nx_eph_list) {
err = apfs_read_ephemeral_objects(sb);
if (err) {
mutex_unlock(&nxs_mutex);
up_write(&nxi->nx_big_sem);
apfs_err(sb, "failed to read the ephemeral objects");
return err;
@@ -306,9 +361,6 @@ int apfs_transaction_start(struct super_block *sb, struct apfs_max_ops maxops)
++nxi->nx_xid;
nxi->nx_raw->nx_next_xid = cpu_to_le64(nxi->nx_xid + 1);
INIT_LIST_HEAD(&nx_trans->t_inodes);
INIT_LIST_HEAD(&nx_trans->t_buffers);
err = apfs_read_spaceman(sb);
if (err) {
apfs_err(sb, "failed to read the spaceman");
@@ -328,16 +380,7 @@ int apfs_transaction_start(struct super_block *sb, struct apfs_max_ops maxops)
return -ENOSPC;
}
if (!vol_trans->t_old_vsb) {
vol_trans->t_old_vsb = sbi->s_vobject.o_bh;
get_bh(vol_trans->t_old_vsb);
/* Backup the old tree roots; the node struct issues make this ugly */
vol_trans->t_old_cat_root = *sbi->s_cat_root;
get_bh(vol_trans->t_old_cat_root.object.o_bh);
vol_trans->t_old_omap_root = *sbi->s_omap->omap_root;
get_bh(vol_trans->t_old_omap_root.object.o_bh);
if (nx_trans->t_starts_count == 0) {
err = apfs_map_volume_super(sb, true /* write */);
if (err) {
apfs_err(sb, "CoW failed for volume super");
@@ -401,19 +444,17 @@ int apfs_transaction_flush_all_inodes(struct super_block *sb)
list_del_init(&ai->i_list);
nx_trans->t_state |= APFS_NX_TRANS_COMMITTING;
mutex_unlock(&nxs_mutex);
up_write(&nxi->nx_big_sem);
/* Unlocked, so it may call ->evict_inode() */
/* Unlocked, so it may call evict() and wait for writeback */
iput(inode);
down_write(&nxi->nx_big_sem);
mutex_lock(&nxs_mutex);
nx_trans->t_state = 0;
/* Transaction aborted by ->evict_inode(), error code is lost */
/* Transaction aborted during writeback, error code is lost */
if (sb->s_flags & SB_RDONLY) {
apfs_err(sb, "abort during inode eviction");
apfs_err(sb, "abort during inode writeback");
return -EROFS;
}
}
@@ -432,18 +473,24 @@ int apfs_transaction_flush_all_inodes(struct super_block *sb)
static int apfs_write_single_ephemeral_object(struct super_block *sb, struct apfs_obj_phys *obj_raw, const struct apfs_checkpoint_mapping *map)
{
struct apfs_nxsb_info *nxi = APFS_NXI(sb);
struct apfs_nx_superblock *raw_sb = NULL;
struct buffer_head *bh = NULL;
u64 bno;
u32 size;
int err, i;
u64 data_base, bno;
u32 data_blks, size;
int err, i, data_idx;
raw_sb = nxi->nx_raw;
data_base = le64_to_cpu(raw_sb->nx_xp_data_base);
data_blks = le32_to_cpu(raw_sb->nx_xp_data_blocks);
bno = le64_to_cpu(map->cpm_paddr);
size = le32_to_cpu(map->cpm_size);
obj_raw->o_xid = cpu_to_le64(nxi->nx_xid);
apfs_multiblock_set_csum((char *)obj_raw, size);
data_idx = bno - data_base;
for (i = 0; i < size >> sb->s_blocksize_bits; ++i) {
bh = apfs_getblk(sb, bno + i);
bh = apfs_getblk(sb, data_base + data_idx);
if (!bh) {
apfs_err(sb, "failed to map ephemeral block");
return -EIO;
@@ -457,6 +504,9 @@ static int apfs_write_single_ephemeral_object(struct super_block *sb, struct apf
memcpy(bh->b_data, (char *)obj_raw + (i << sb->s_blocksize_bits), sb->s_blocksize);
brelse(bh);
bh = NULL;
/* Somewhat surprisingly, objects can wrap around */
if (++data_idx == data_blks)
data_idx = 0;
}
return 0;
}
@@ -532,15 +582,6 @@ static int apfs_write_ephemeral_objects(struct super_block *sb)
eph_info = &nxi->nx_eph_list[i];
data_next = (data_index + data_len) % data_blks;
obj_blkcnt = eph_info->size >> sb->s_blocksize_bits;
if (obj_blkcnt > data_blks - data_next) {
/*
* This multiblock object does not fit in what's left
* of the ring buffer, so move it to the beginning and
* leave some empty blocks.
*/
data_len += data_blks - data_next;
data_next = 0;
}
err = apfs_create_cpoint_map(sb, cpm, eph_info->object, data_base + data_next, eph_info->size);
if (err) {
@@ -596,11 +637,9 @@ static int apfs_transaction_commit_nx(struct super_block *sb)
{
struct apfs_spaceman *sm = APFS_SM(sb);
struct apfs_nxsb_info *nxi = APFS_NXI(sb);
struct apfs_sb_info *sbi;
struct apfs_nx_transaction *nx_trans = &nxi->nx_transaction;
struct apfs_bh_info *bhi, *tmp;
int err = 0;
u32 bmap_idx;
ASSERT(!(sb->s_flags & SB_RDONLY));
@@ -624,6 +663,16 @@ static int apfs_transaction_commit_nx(struct super_block *sb)
}
sm->sm_free_cache_base = sm->sm_free_cache_blkcnt = 0;
}
/*
* Writing the ip bitmaps modifies the spaceman, so it must happen
* before we commit the ephemeral objects. It must also happen after we
* flush the free queue, in case the last freed range was in the ip.
*/
err = apfs_write_ip_bitmaps(sb);
if (err) {
apfs_err(sb, "failed to commit the ip bitmaps");
return err;
}
err = apfs_write_ephemeral_objects(sb);
if (err)
return err;
@@ -644,7 +693,11 @@ static int apfs_transaction_commit_nx(struct super_block *sb)
}
list_for_each_entry_safe(bhi, tmp, &nx_trans->t_buffers, list) {
struct buffer_head *bh = bhi->bh;
#if LINUX_VERSION_CODE >= KERNEL_VERSION(6, 11, 0)
struct folio *folio = NULL;
#else
struct page *page = NULL;
#endif
bool is_metadata;
ASSERT(buffer_trans(bh));
@@ -664,27 +717,44 @@ static int apfs_transaction_commit_nx(struct super_block *sb)
kfree(bhi);
bhi = NULL;
#if LINUX_VERSION_CODE >= KERNEL_VERSION(6, 11, 0)
folio = page_folio(bh->b_page);
folio_get(folio);
#else
page = bh->b_page;
get_page(page);
#endif
is_metadata = buffer_csum(bh);
clear_buffer_csum(bh);
put_bh(bh);
bh = NULL;
#if LINUX_VERSION_CODE >= KERNEL_VERSION(6, 11, 0)
folio_lock(folio);
folio_mkclean(folio);
#else
/* Future writes to mmapped areas should fault for CoW */
lock_page(page);
page_mkclean(page);
#endif
/* XXX: otherwise, the page cache fills up and crashes the machine */
if (!is_metadata) {
#if LINUX_VERSION_CODE >= KERNEL_VERSION(5, 19, 0)
#if LINUX_VERSION_CODE >= KERNEL_VERSION(6, 11, 0)
try_to_free_buffers(folio);
#elif LINUX_VERSION_CODE >= KERNEL_VERSION(5, 19, 0)
try_to_free_buffers(page_folio(page));
#else
try_to_free_buffers(page);
#endif
}
#if LINUX_VERSION_CODE >= KERNEL_VERSION(6, 11, 0)
folio_unlock(folio);
folio_put(folio);
#else
unlock_page(page);
put_page(page);
#endif
}
err = apfs_checkpoint_end(sb);
if (err) {
@@ -692,30 +762,6 @@ static int apfs_transaction_commit_nx(struct super_block *sb)
return err;
}
list_for_each_entry(sbi, &nxi->vol_list, list) {
struct apfs_vol_transaction *vol_trans = &sbi->s_transaction;
if (!vol_trans->t_old_vsb)
continue;
brelse(vol_trans->t_old_vsb);
vol_trans->t_old_vsb = NULL;
/* XXX: forget the buffers for the b-tree roots */
vol_trans->t_old_omap_root.object.data = NULL;
brelse(vol_trans->t_old_omap_root.object.o_bh);
vol_trans->t_old_omap_root.object.o_bh = NULL;
vol_trans->t_old_cat_root.object.data = NULL;
brelse(vol_trans->t_old_cat_root.object.o_bh);
vol_trans->t_old_cat_root.object.o_bh = NULL;
}
for (bmap_idx = 0; bmap_idx < APFS_SM(sb)->sm_ip_bmaps_count; ++bmap_idx) {
brelse(APFS_SM(sb)->sm_ip_bmaps[bmap_idx]);
APFS_SM(sb)->sm_ip_bmaps[bmap_idx] = NULL;
}
APFS_SM(sb)->sm_raw = NULL;
nx_trans->t_starts_count = 0;
nx_trans->t_buffers_count = 0;
return 0;
@@ -736,7 +782,7 @@ static bool apfs_transaction_need_commit(struct super_block *sb)
return false;
}
/* Avoid nested commits on ->evict_inode() */
/* Avoid nested commits on inode writeback */
if (nx_trans->t_state & APFS_NX_TRANS_COMMITTING)
return false;
@@ -749,9 +795,9 @@ static bool apfs_transaction_need_commit(struct super_block *sb)
struct apfs_spaceman_phys *sm_raw = sm->sm_raw;
struct apfs_spaceman_free_queue *fq_ip = &sm_raw->sm_fq[APFS_SFQ_IP];
struct apfs_spaceman_free_queue *fq_main = &sm_raw->sm_fq[APFS_SFQ_MAIN];
int buffers_max = APFS_SB(sb)->s_trans_buffers_max;
int starts_max = TRANSACTION_STARTS_MAX;
int mq_max = TRANSACTION_MAIN_QUEUE_MAX;
int buffers_max = nxi->nx_trans_buffers_max;
int starts_max = APFS_TRANS_STARTS_MAX;
int mq_max = APFS_TRANS_MAIN_QUEUE_MAX;
/*
* Try to avoid committing halfway through a data block write,
@@ -796,17 +842,24 @@ static bool apfs_transaction_need_commit(struct super_block *sb)
int apfs_transaction_commit(struct super_block *sb)
{
struct apfs_nxsb_info *nxi = APFS_NXI(sb);
struct apfs_nx_transaction *trans = NULL;
int err = 0;
trans = &nxi->nx_transaction;
if (apfs_transaction_need_commit(sb)) {
err = apfs_transaction_commit_nx(sb);
if (err) {
apfs_err(sb, "transaction commit failed");
return err;
}
trans->t_work_sb = NULL;
cancel_delayed_work(&trans->t_work);
} else {
trans->t_work_sb = sb;
mod_delayed_work(system_wq, &trans->t_work, msecs_to_jiffies(100));
}
mutex_unlock(&nxs_mutex);
up_write(&nxi->nx_big_sem);
return 0;
}
@@ -889,19 +942,15 @@ static void apfs_force_readonly(struct apfs_nxsb_info *nxi)
*/
void apfs_transaction_abort(struct super_block *sb)
{
struct apfs_sb_info *sbi;
struct apfs_nxsb_info *nxi = APFS_NXI(sb);
struct apfs_nx_transaction *nx_trans = &nxi->nx_transaction;
struct apfs_bh_info *bhi, *tmp;
struct apfs_inode_info *ai, *ai_tmp;
struct apfs_spaceman *sm = NULL;
u32 bmap_idx;
if (sb->s_flags & SB_RDONLY) {
/* Transaction already aborted, do nothing */
ASSERT(list_empty(&nx_trans->t_inodes));
ASSERT(list_empty(&nx_trans->t_buffers));
mutex_unlock(&nxs_mutex);
up_write(&nxi->nx_big_sem);
return;
}
@@ -924,54 +973,14 @@ void apfs_transaction_abort(struct super_block *sb)
kfree(bhi);
}
/*
* TODO: get rid of all this stuff, it makes little sense. Maybe do an
* actual read-only remount?
*/
list_for_each_entry(sbi, &nxi->vol_list, list) {
struct apfs_vol_transaction *vol_trans = &sbi->s_transaction;
if (!vol_trans->t_old_vsb)
continue;
/* Restore volume state for all volumes */
brelse(sbi->s_vobject.o_bh);
sbi->s_vobject.o_bh = vol_trans->t_old_vsb;
sbi->s_vobject.data = sbi->s_vobject.o_bh->b_data;
sbi->s_vobject.block_nr = vol_trans->t_old_vsb->b_blocknr;
sbi->s_vsb_raw = (void *)vol_trans->t_old_vsb->b_data;
vol_trans->t_old_vsb = NULL;
/* XXX: restore the old b-tree root nodes */
brelse(sbi->s_omap->omap_root->object.o_bh);
*(sbi->s_omap->omap_root) = vol_trans->t_old_omap_root;
vol_trans->t_old_omap_root.object.o_bh = NULL;
vol_trans->t_old_omap_root.object.data = NULL;
brelse(sbi->s_cat_root->object.o_bh);
*(sbi->s_cat_root) = vol_trans->t_old_cat_root;
vol_trans->t_old_cat_root.object.o_bh = NULL;
vol_trans->t_old_cat_root.object.data = NULL;
}
sm = APFS_SM(sb);
if (sm) {
for (bmap_idx = 0; bmap_idx < sm->sm_ip_bmaps_count; ++bmap_idx) {
brelse(sm->sm_ip_bmaps[bmap_idx]);
sm->sm_ip_bmaps[bmap_idx] = NULL;
}
APFS_SM(sb)->sm_raw = NULL;
}
/*
* It's not possible to undo in-memory changes from old operations in
* the aborted transaction. To avoid corruption, never write again.
*/
apfs_force_readonly(nxi);
mutex_unlock(&nxs_mutex);
up_write(&nxi->nx_big_sem);
/* ->evict_inode() will just fail if it starts a new transaction */
list_for_each_entry_safe(ai, ai_tmp, &nx_trans->t_inodes, i_list) {
list_del_init(&ai->i_list);
iput(&ai->vfs_inode);
+3 -4
View File
@@ -12,10 +12,9 @@
#include <linux/types.h>
#include <linux/nls.h>
#include <linux/ctype.h>
#include <linux/version.h>
#include "unicode.h"
#define MIN(X, Y) ((X) <= (Y) ? (X) : (Y))
/* The arrays of unicode data are defined at the bottom of the file */
/* TODO: would a single trie with all the data be more efficient? */
static u16 apfs_nfd_trie[];
@@ -234,7 +233,7 @@ static int apfs_get_normalization_length(const char *utf8str, unsigned int total
while (1) {
if (!total_len || !*utf8str)
return norm_len;
utf8len = utf8_to_utf32(utf8str, MIN(total_len, 4), &utf32char);
utf8len = utf8_to_utf32(utf8str, min(total_len, 4U), &utf32char);
if (utf8len < 0) /* Invalid unicode; don't normalize anything */
return 0;
@@ -306,7 +305,7 @@ new_starter:
unicode_t utf32char;
int utf8len, pos;
utf8len = utf8_to_utf32(utf8str, MIN(total_len, 4), &utf32char);
utf8len = utf8_to_utf32(utf8str, min(total_len, 4U), &utf32char);
for (pos = 0;; pos++, str_pos++) {
unicode_t utf32norm;
u8 ccc;