linux/linux-apfs-rw
1
0 Fork 0
mirror of https://github.com/linux-apfs/linux-apfs-rw.git synced 2026-08-18 02:19:59 -07:00
Code Issues Packages Projects Releases Wiki Activity

Compare commits

base: linux/linux-apfs-rw:v0.3.9
linux/linux-apfs-rw:master linux/linux-apfs-rw:development
linux/linux-apfs-rw:v0.3.21 linux/linux-apfs-rw:v0.3.20 linux/linux-apfs-rw:v0.3.19 linux/linux-apfs-rw:v0.3.18 linux/linux-apfs-rw:v0.3.17 linux/linux-apfs-rw:v0.3.16 linux/linux-apfs-rw:v0.3.15 linux/linux-apfs-rw:v0.3.14 linux/linux-apfs-rw:v0.3.13 linux/linux-apfs-rw:v0.3.12 linux/linux-apfs-rw:v0.3.11 linux/linux-apfs-rw:v0.3.10 linux/linux-apfs-rw:v0.3.9 linux/linux-apfs-rw:v0.3.8 linux/linux-apfs-rw:v0.3.7 linux/linux-apfs-rw:v0.3.6 linux/linux-apfs-rw:v0.3.5 linux/linux-apfs-rw:v0.3.4 linux/linux-apfs-rw:v0.3.3 linux/linux-apfs-rw:v0.3.2 linux/linux-apfs-rw:v0.3.1 linux/linux-apfs-rw:v0.3.0
...
compare: linux/linux-apfs-rw:v0.3.15
linux/linux-apfs-rw:development linux/linux-apfs-rw:master
linux/linux-apfs-rw:v0.3.21 linux/linux-apfs-rw:v0.3.20 linux/linux-apfs-rw:v0.3.19 linux/linux-apfs-rw:v0.3.18 linux/linux-apfs-rw:v0.3.17 linux/linux-apfs-rw:v0.3.16 linux/linux-apfs-rw:v0.3.15 linux/linux-apfs-rw:v0.3.14 linux/linux-apfs-rw:v0.3.13 linux/linux-apfs-rw:v0.3.12 linux/linux-apfs-rw:v0.3.11 linux/linux-apfs-rw:v0.3.10 linux/linux-apfs-rw:v0.3.9 linux/linux-apfs-rw:v0.3.8 linux/linux-apfs-rw:v0.3.7 linux/linux-apfs-rw:v0.3.6 linux/linux-apfs-rw:v0.3.5 linux/linux-apfs-rw:v0.3.4 linux/linux-apfs-rw:v0.3.3 linux/linux-apfs-rw:v0.3.2 linux/linux-apfs-rw:v0.3.1 linux/linux-apfs-rw:v0.3.0
82 Commits
v0.3.9 ... v0.3.15
Author SHA1 Message Date
Ernesto A. Fernández de4083a506 0.3.15 
This release adds support for the upcoming 6.16 kernel version.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2025-07-25 19:27:24 -03:00
Ernesto A. Fernández a3c23dffc3 Don't reference page index in 6.16 
The build is broken for the 6.16 release candidate:

  https://github.com/linux-apfs/linux-apfs-rw/issues/99

The problem is that the page struct no longer has a member called
"index". Use folios for this instead.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2025-07-01 20:45:23 -03:00
Ernesto A. Fernández 7d8dc8821e 0.3.14 
This release adds support for the upcoming 6.15 kernel version, and for
RHEL 9.6.

A longstanding objtool warning has also been fixed.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2025-05-23 21:32:10 -03:00
Ernesto A. Fernández 26bf3c5ba7 Fix build if kernel has no __annotate_jump_table 
I recently started to use the __annotate_jump_table macro to silence an
objtool warning, but it seems that this macro was only added for the 5.3
kernel. Just define it to nothing for older releases, the objtool
warnings probably don't matter that much.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2025-05-16 17:40:41 -03:00
Ernesto A. Fernández c87d7aa77f Fix build for RHEL 9.6 
The build has broken again for RHEL:

  https://github.com/linux-apfs/linux-apfs-rw/issues/95

Fix it.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2025-05-14 20:43:28 -03:00
Ernesto A. Fernández 87cc3f6940 Get rid of "pragma once" in lzfse code 
Use regular macro guards instead.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2025-05-09 18:56:43 -03:00
Ernesto A. Fernández 51a5b742ec Fix the returned type of ->mkdir() for 6.15 
The ->mkdir() method now needs to return a dentry struct instead of an
int, but this is meant for clustered filesystems or something so just
return NULL on success, like most filesystems do.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2025-04-21 19:24:48 -03:00
Ernesto A. Fernández e41b8f1f0e Handle page functions newly removed in 6.15 
It seems that wait_for_stable_page(), page_has_buffers() and
grab_cache_page_write_begin() are no longer available, so put
implementations inside the driver.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2025-04-21 18:31:15 -03:00
Ernesto A. Fernández 68ad507875 Get rid of typedef for fse_value_decoder_entry 
I don't intend to remove all typedefs from the lzfse code, but this one
is triggering a false positive for another issue in checkpatch. Get rid
of it.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2025-04-16 17:32:24 -03:00
Ernesto A. Fernández 1039569db9 Correct brace placement in an lzfse for loop 
Checkpatch complains about the brace placement in this for loop. Change
it.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2025-04-16 17:20:17 -03:00
Ernesto A. Fernández 86cee6ecf8 Add SPDX identifiers to lzfse files 
They are all under the 3-clause BSD license, so annotate them as such.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2025-04-15 20:58:12 -03:00
Ernesto A. Fernández 37e9e2a726 Unify lzfse style for sizeof 
Always use sizeof with brackets, like checkpatch prefers.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2025-04-15 20:39:43 -03:00
Ernesto A. Fernández 991eebd92a Don't use __attribute__ in lzfse 
The kernel has macros for these things, so use them instead and silence
the checkpatch warnings.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2025-04-15 20:30:08 -03:00
Ernesto A. Fernández f47daaf15c Don't use "unsigned" alone in lzfse 
Checkpatch prefers "unsigned int" over "unsigned", so fix this where
needed.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2025-04-15 20:02:59 -03:00
Ernesto A. Fernández be38209c0b Don't use unnecessary braces in lzfse 
Another checkpatch warning: remove all braces around blocks that have a
single statement, except for two cases where it makes more sense to
reorganize long comments so that the warnings stop.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2025-04-15 19:20:22 -03:00
Ernesto A. Fernández 84e57c48c4 Add blank lines after declarations in lzfse 
Continue to silence checkpatch warnings with this change.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2025-04-15 19:08:09 -03:00
Ernesto A. Fernández 5aa52b928f Change the comment style of lzfse 
Change the comment style in the lzfse code (by hand) to more or less
match the rest of the driver.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2025-04-11 20:00:54 -03:00
Ernesto A. Fernández c42445a577 Run clang-format on the remaining lzfse code 
Continuing with the lzfse code cleanup: it seems that clang-format can
at least fix the whitespace reliably, so run it and get that out of the
way. But set the column limit to 100 like checkpatch uses, to avoid too
much awkward line wrapping on the parts of the code that go far to the
right.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2025-04-10 19:39:14 -03:00
Ernesto A. Fernández 6384c5dd50 Get rid of the lzfse inline macros 
The kernel provides its own __always_inline macro, so get rid of those
that came with lzfse.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2025-04-09 18:12:44 -03:00
Ernesto A. Fernández acdad48cbc Get rid of LZFSE_API macro 
The LZFSE_API macro doesn't seem to be used anywhere right now, so clean
it up.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2025-04-09 17:34:09 -03:00
Ernesto A. Fernández 92583d1c8f Get rid of C++ stuff in lzfse 
None of this code will ever get called from C++, so clean that up.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2025-04-09 16:52:50 -03:00
Ernesto A. Fernández efc3c7b5ac Get rid of compiler checks in the lzfse code 
I don't think any rare compilers are used to build the kernel, so just
assume gcc/clang everywhere. Also get rid of the dll checks.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2025-04-09 16:25:36 -03:00
Ernesto A. Fernández d3ea85f068 Allow jumptables without preprocessor checks 
Jumptables are used elsewhere in the kernel, so it's safe to assume that
the compiler supports them. Clean up all the dead fallback code for
lzvn decompression.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2025-04-08 20:53:57 -03:00
Ernesto A. Fernández 21f190f58b Annotate jump table in lzvn decompression code 
Since I first copied the lzfse code into the driver, the build has been
throwing the following error:

  lzfse/lzvn_decode_base.o: warning: objtool: lzvn_decode+0x10: sibling call from callable instruction with modified stack frame

I've been ignoring this because objtool warnings are probably not a big
deal for us; I think they are mostly for live patches. I'm also not
familiar with the lzfse code so I assumed this would be hard to fix.
It's not though: just annotate the jump table correctly.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2025-04-08 20:40:29 -03:00
Ernesto A. Fernández f0982e87d4 Clear unused lzfse/lzvn compression code 
I have no real intention of ever supporting compressed writes. Last I
checked, the official implementation simply decompresses files when you
try to write to them, so maybe I will do that here as well at some
point. For now users that need this can work with a copy of the file
(and delete the original).

So, get rid of all the unused lzfse/lzvn compression code. This will
make it easier to fix the code style in the following patches.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2025-04-08 19:39:22 -03:00
Ernesto A. Fernández 61d8e5b7c4 0.3.13 
This release adds support for the upcoming 6.14 upstream kernel, and for
a handful of RHEL 9 kernels that have been broken for a while.

Some improvements have been made to ENOSPC behaviour, but the work here
isn't quite finished yet.

Read-only support is added for fusion drives. Writes will probably never
get implemented, though I might reconsider this if people actually ask
for the feature.

Read-only mounts of volumes with so-called "dataless snapshots" are also
tentatively allowed. I have no idea what this is about but there should
be no risk.

Free space as reported by statfs is now much more accurate. A few other
bugs were also fixed, but nothing big enough to deserve notice.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2025-03-21 22:30:49 -03:00
Ernesto A. Fernández 129ef5a7e8 Go read-only if we can't read ephemeral objects 
Failing to read an ephemeral object does not abort the transaction, it
just returns an error. The problem is that, by this point, nx_eph_list
is already set: the next time a transaction starts the reads will get
skipped, and things will proceed as if there was no error.

I got this series of messages from xfstests (no idea which test, but
clearly one that fakes I/O errors):

  [18920.212351] Buffer I/O error on dev dm-0, logical block 3932144, async page read
  [18920.515341] APFS (dm-0:0): failed to read cpm block (apfs_read_single_cpm_block:231)
  [18920.520378] APFS (dm-0:0): failed to read cpm block 0 (apfs_read_ephemeral_objects:300)
  [18920.524371] APFS (dm-0:0): failed to read the ephemeral objects (apfs_transaction_start:384)
  [18920.529524] APFS (dm-0:0): no mapping for oid 0x400 (apfs_ephemeral_object_lookup:216)
  [18920.532784] APFS (dm-0:0): no spaceman object for oid 0x400 (apfs_read_spaceman:744)
  [18920.536222] APFS (dm-0:0): failed to read the spaceman (apfs_transaction_start:395)
  [18920.538634] APFS (dm-0:0): aborting transaction (apfs_transaction_abort:1001)
  [18920.646469] APFS (dm-0:0): no mapping for oid 0x400 (apfs_ephemeral_object_lookup:216)
  [18920.653922] APFS (dm-0:0): no spaceman object for oid 0x400 (apfs_spaceman_get_free_blkcnt:1468)
  [18920.766489] APFS (dm-0:0): no mapping for oid 0x400 (apfs_ephemeral_object_lookup:216)
  [18920.770446] APFS (dm-0:0): no spaceman object for oid 0x400 (apfs_spaceman_get_free_blkcnt:1468)

No harm was done because the next transaction still got aborted as soon
as we tried to read the spaceman. But, if at some point in the future we
stop fully flushing free queues on each start, we could run into more
subtle bugs involving missing fq nodes.

This all seems unlikely but, to be safe, make the mount read-only when
we fail to get the ephemeral objects. It's really the same situation as
any other abort, so it's not a problem.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2025-03-19 20:45:23 -03:00
Ernesto A. Fernández c012e6cdb9 Commit when the main free queue has too many nodes 
As mentioned in the previous commit message, generic/269 of xfstests
makes the main free queue unbalanced enough that insertions fail.
Clearing the free queue when the transaction starts is not enough to
prevent it, because the whole disaster can happen without a single
commit.

So, start forcing commits when the main queue has too many nodes. I
don't think there is a problem with being too strict here, because a
truly full main queue would be gigantic for most normal filesystems.
Just force a commit when we reach half the limit (rounded up). Of course
make an exception for filesystems with a single main fq node, since
those can't even become unbalanced in the first place and we don't want
to commit nonstop.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2025-03-18 18:38:13 -03:00
Ernesto A. Fernández 3682c034b0 Fully flush free queues when a transaction starts 
Like so many times before, I'm running out of space in the main free
queue. This time the reproducer is generic/269 of xfstests. The queue
isn't really that full, the problem is that it's very unbalanced. We
don't really have any mechanism in place to rebalance btrees, so the
easiest way forward is to just empty the free queue entirely whenever
possible, that is, when a transaction starts.

Of course this means that I no longer hold on to blocks from old
transactions, but I never had any real reason to do that. Also, there's
no longer any point in retrying fq insertions that fail, since it's just
not possible to safely flush any other blocks.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2025-03-18 16:29:06 -03:00
Ernesto A. Fernández 265f356558 Define macros for transaction space requirements 
The previous patch reused the number "20" from
apfs_transaction_has_room() inside apfs_main_free(). The next commits
might need to tune that value, so it's better to just define a shared
macro for both.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2025-03-17 16:36:43 -03:00
Ernesto A. Fernández 434a595e71 Don't keep retrying orphan cleanups on error 
Running generic/269 of xfstests quickly gets orphan cleanups to hit
ENOSPC, which prints an error message. The problem is that this doesn't
happen just once, but over and over again, completely flooding dmesg for
no good reason.

Retrying cleanups after ENOSPC does have some value, but wait until we
actually made some room. For other errors it's just a bad idea, since
something has gone seriously wrong; we'll usually just get EROFS anyway,
but just don't retry at all.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2025-03-14 23:08:24 -03:00
Ernesto A. Fernández bb291218a4 Log an error if we hit ENOSPC during a transaction 
The intention of my ongoing ENOSPC work is that we should always catch
that error during transaction start (so that we don't need to abort),
but so far xfstests is still beating my checks somehow. That's a bug
though, so start logging an error message to make it easier to spot.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2025-03-13 17:38:49 -03:00
Ernesto A. Fernández 3509b5ed35 Report free space correctly on statfs 
The statfs code was one of the first things I implemented (most likely
before I could even mount the filesystem in any meaningful sense), and I
never tested it much. It was bound to have bugs.

Now that we handle ENOSPC properly (or so I hope), I'm starting to try
my luck with the enospc group from xfstests. The first issue I've
noticed is that generic/275 complains that it "could not sufficiently
fill filesystem". The container is actually very much full, but statfs
reports used space incorrectly: it adds up the blocks allocated by each
volume, neglecting all the blocks that don't belong to any of them.

At the time I didn't know anything about the space manager, so I guess
this seemed like a good enough solution. Fix it now.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2025-03-11 21:13:09 -03:00
Ernesto A. Fernández 3174e363ad Simplify ENOSPC checks before transactions 
If a function throws ENOSPC halfway through an operation, the
transaction will get aborted so that the filesystem isn't left in an
inconsistent state. This means that the mount will go read-only, and
that the user will lose any uncommitted changes.

So, it's better if we detect ENOSPC before starting an operation. I
tried this before with commit a52b73ed97 ("Check free space before
starting a transaction"), but the code was messy and it would have
required constant updates, which I neglected of course. The problem was
that I wanted to be as accurate as possible with my calculations of
required space, but I'm not sure there is any point.

Instead, require a large fixed amount of space (512 KiB) for most
operations, and a much smaller but still almost certainly sufficient
amount of space (80 KiB) for deletions. Hopefully this will be enough
for users to maneuver around ENOSPC situations. It's not perfect, but no
CoW filesystem can be truly perfect in this regard.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2025-03-11 16:15:41 -03:00
Ernesto A. Fernández afe9bac8f2 Actually add support for RHEL 9 
The build is still broken for redhat kernels:

  https://github.com/linux-apfs/linux-apfs-rw/issues/89#issuecomment-2691955523

Fix the remaining issues. For the previous patch I was just looking at
the source, but this time I set up a rhel vm and tried a build and a
mount for each of the 9.x kernels, so hopefully it's over. I didn't know
you could get rhel for free as a developer, I guess it makes sense.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2025-03-03 19:43:17 -03:00
Ernesto A. Fernández 412c4c8f9c Add support for RHEL 9 
The build is broken for redhat kernels:

  https://github.com/linux-apfs/linux-apfs-rw/issues/89

This seems to have been the case for more than a year, but nobody had
reported it so far, so I guess the driver isn't getting much use over
there.

We had encountered a similar problem before in commit 1a0b9fb8af ("Fix
build for RHEL 9 kernels"), but back then I thought it was just a
one-off thing. I was wrong: redhat kernels often backport patches that
break internal apis. I don't know how other out-of-tree drivers handle
this, but for now I will have to rely on user reports.

So, go through all RHEL 9 kernels one by one, find out which api changes
were applied to each, and add the corresponding macro version checks to
the driver.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2025-02-28 19:45:33 -03:00
Ernesto A. Fernández 6a24fdcb2e Update readme with fusion drive support 
Fusion drives now have read-only support, so update the readme.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2025-02-14 18:48:47 -03:00
Ernesto A. Fernández 21acd228b4 Check uuid of fusion drive devices 
If the user mixes up the main and tier 2 devices, the mount fails with
confusing errors. We can do better here: the fusion uuid has a bit to
tell which is the main device, so check that early and fail with a good
explanation. While we are at it, also add checks to make sure what we
are mounting is indeed a fusion drive.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2025-02-13 20:34:45 -03:00
Ernesto A. Fernández c39f8a5a8f Log an error message if a mount option is invalid 
While testing the new "tier2" mount option, I realized that the driver
gives no feedback to the user when a mount option is invalid. I have my
doubts about logging an error here because, while the apfs driver is
inserted, the message may show up for unrelated mounts as well. I think
this sort of issues are unavoidable though, and XFS does log a warning
for invalid options, so do the same thing.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2025-02-13 16:46:34 -03:00
Ernesto A. Fernández 9921ef9205 Implement reads for fusion drives 
So far I haven't been able to figure out fusion drives to an extent that
would allow me to implement writes. I'm tired of it, and I think the
effort is probably pointless because apfs fusion drives aren't all that
popular. I might reconsider if people request the feature, but for now
just add read-only support, which is almost trivial.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2025-02-12 18:27:02 -03:00
Ernesto A. Fernández 3604300b2b Allocate block device info independently of nxi 
Make the block device info field of the container superblock into a
pointer. This will make it easier to tell if we are working with a
fusion drive.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2025-02-10 19:57:40 -03:00
Ernesto A. Fernández 98f07c1262 Keep track of device mode for older kernels 
For kernels before 6.5, blkdev_put() needs to be told the mode of the
block device. This information is stored in the vfs superblock, but I
can't access that from apfs_free_main_super() because that function may
be needed early on mount.

My somewhat hacky solution so far has been to guess the mode from the
setting of the APFS_READWRITE flag. This isn't accurate because the mode
is actually determined by SB_RDONLY, but it still seems to work - I
really haven't checked what happens to the mode inside blkdev_put().

For fusion drives, I will soon need to reuse apfs_blkdev_cleanup()
inside apfs_attach_nxi(). In this case the mode is available, so working
with the different definitions of it for each kernel version would make
a huge mess.

Since we now have apfs_blkdev_info for portablity, just keep the mode in
there and let apfs_blkdev_cleanup() retrieve it.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2025-02-10 15:46:19 -03:00
Ernesto A. Fernández 65b2bc9342 Prepare block device code for reuse 
Put all information about the container's block device into a single
struct, and move all related code into wrapper functions that work with
that struct. Hopefully this will make it easier to add a second block
device for the tier 2 of a fusion drive.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2025-02-07 21:37:07 -03:00
Ernesto A. Fernández 55e30b926c Never pick the blockdev directly for buffer heads 
When obtaining a buffer head, always let some wrapper specify the block
device. This is just a cosmetic thing for now, but it will be important
when dealing with fusion drives.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2025-02-07 17:15:47 -03:00
Ernesto A. Fernández e16e7f4f8a Update apfs_dentry_revalidate() arguments for 6.14 
The build is broken for the 6.14 release candidate:

  https://github.com/linux-apfs/linux-apfs-rw/issues/88

The problem is that ->d_revalidate() now takes two more arguments.
Update our implementation accordingly.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2025-02-06 16:09:14 -03:00
Ernesto A. Fernández 7a3457bdd0 Update header comment for apfs_query 
The header comment from apfs_query says that it only gets used for the
catalog and the object map. That claim is extremely out of date, of
course. Get rid of it.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2024-12-04 22:39:44 -03:00
Ernesto A. Fernández 6fcc69c05d Don't refer to record values as "data" 
The paper I followed in the early days of the driver (before the
official reference was published) referred to record values as "data",
so some of my oldest code uses that name as well. This is inconsistent,
and not very clear because the word "data" shows up everywhere. Reword
all instances of this I can identify.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2024-12-04 21:04:23 -03:00
Ernesto A. Fernández 82324fb015 Make record key size checks more strict 
For safety, I think it's always better for a filesystem to reject any
malformed data it can notice. Omap and free queue keys have a fixed
size, so the size check we run to prevent out-of-bounds reads, while
correct, should be made more strict. This is what we already do for most
record types these days, both keys and values.

Take this opportunity to improve the code style here and make it more
consistent.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2024-12-04 18:28:20 -03:00
Ernesto A. Fernández 41f133bff6 Fix header comment for put/take_single_extent() 
There is no "paddr" argument to either apfs_take_single_extent() or
apfs_put_single_extent(). Fix this.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2024-12-04 16:16:53 -03:00
Ernesto A. Fernández 0fa8eaa812 Properly explain EEXIST during snapshot creation 
Provide a proper error message on dmesg when a user attempts to create a
snapshot with a name that's already in use, otherwise "File exists" may
be a bit confusing.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2024-12-03 21:07:20 -03:00
Ernesto A. Fernández f78e6c876b Allow read-only mounts of dataless snapshots 
A user has encountered a volume with the dataless snapshots flag:

  https://github.com/linux-apfs/linux-apfs-rw/issues/86

I'm not really sure what "dataless snapshots" are, and so far I have
failed to trigger them under macos. I can set the
SNAP_META_PENDING_DATALESS flag in a snapshot myself, and macos will
call it "dataless", but their fsck doesn't complain no matter what I do
with the other fields, so I can't tell what is expected.

I can't imagine any risk in mounting such a volume read-only, so allow
that much from now on. Keep writes banned just in case, and avoid
mounting the dataless snapshots themselves, since I have no idea what
that entails.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2024-12-03 20:33:18 -03:00
Ernesto A. Fernández 7f9ae821f2 Improve the superblock id reported in the logs 
While testing eda21f33e9 ("Check the superblock magic before the block
size"), I noticed that early log messages such as those don't include
device information, just the filesystem type. The problem is that the
s_id field of the vfs superblock is only getting set after the call to
apfs_read_main_super(); there is no real reason for this, so invert it.

While paying attention to this field for the first time, I realized that
I'm making a huge mess of it. Most filesystems use a "%pg" format
specifier: this prints the name of the block_device pointer passed as an
argument, so the logs are very clear. Meanwhile I was just printing the
dev_t... and putting a "g" afterwards? I guess I somehow got confused
while trying to copy the others. Do this right.

That said, an apfs superblock is not uniquely identified by its device:
we have volumes and snapshots as well. The comments in the super_block
structure claim that s_id is just an "Informational name", so I guess I
can set it to whatever I find useful. Provide the volume number and
snapshot name (if any) as well.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2024-11-10 11:56:05 -03:00
Ernesto A. Fernández 1643ae2688 Get rid of the rst note from the last patch 
Github doesn't support rst notes, so it just looks silly over there.
Make it into a regular paragraph.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2024-11-08 20:35:46 -03:00
Ernesto A. Fernández 7e6babba01 Mention the distro packages in the readme 
Some users seem unaware of the existence of packages for the driver, and
I guess that's my fault because the (outdated) readme doesn't mention
them at all. Add a small note about it in the beginning of the "build"
section.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2024-11-08 20:19:31 -03:00
Ernesto A. Fernández c1cae76fb3 Rework the second paragraph of the readme 
These days I'm far more careful about testing multiple kernel versions,
so the warning in the second paragraph of the readme ("only a few of
those have actually been tested") is a bit excessive. Simplify that
stuff. Don't bother mentioning the support for 4.9-4.11 kernels, it has
some performance issues and they are old enough that it probably doesn't
matter much.

The rest of the paragraph is about bug reporting. Move this into a
separate section, so that it's easier to find, and to give me room to
explain the use of dmesg.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2024-11-08 20:02:16 -03:00
Ernesto A. Fernández 240702b9f6 Tone down corruption warnings in the readme 
The driver is not that bad these days. Corruption is still a real risk
of course, but when it happens it's a bug related to some untested
corner case (like the recent issue with filesystems above 1.32 TiB), not
expected behaviour. So tone down the warnings, but make it clear that
it's not all that safe yet.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2024-11-08 18:26:05 -03:00
Ernesto A. Fernández d9e50e76cf Allow choice of mountable writes by default 
A distro packager has requested writable mounts by default, and is
willing to take responsibility for any potential harm to their users:

  https://github.com/linux-apfs/linux-apfs-rw/issues/81

Add a build flag to enable this, but leave it undocumented to avoid
confusing anyone.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2024-11-08 17:20:47 -03:00
Ernesto A. Fernández eda21f33e9 Check the superblock magic before the block size 
When an attempt is made to mount something other than an apfs container,
the driver currently throws a confusing "bad blocksize" error to dmesg.
I would expect this to be encountered fairly often, when users try to
mount the disk as a whole instead of the apfs partition.

Start checking the superblock magic before the block size so that we can
give a more helpful error message.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2024-11-07 21:14:43 -03:00
Ernesto A. Fernández 5a2db9bd68 0.3.12 
This release fixes a serious data corruption bug for containers bigger
than ~1.32 TiB, which is immediately triggered by a writable mount. This
goes to show why mounts are still read-only by default.

Another much smaller data loss bug for much bigger containers (~7 TiB)
is also addressed.

Support is added for the 6.12 kernel release candidate. A bit early this
time, so let's hope the build doesn't break again for the final release.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2024-11-01 20:06:39 -03:00
Ernesto A. Fernández 96e6d41bb7 Silence gcc warnings due to switch to min() 
Building an old kernel (5.3) with an old version of gcc (4.9.2), I get
some warnings about "comparison of distinct pointer types". I'm pretty
sure this doesn't matter, and I don't even know if the blame lies with
the old compiler or the old kernel, but just silence this by making the
integer literals of the warning unsigned like the other argument.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2024-10-31 17:01:30 -03:00
Woody Suwalski 6c2ce34a67 Fix whitespace damage in apfs_aops definition 
This has been around for a while so I guess checkpatch isn't picking up
on it for some reason.

[eafer: took this from a bigger patch, added commit message]
Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2024-10-31 16:46:20 -03:00
Ernesto A. Fernández a34220c93d Switch ->write_begin() and ->write_end() to folios 
The build is broken for the 6.12 release candidate. The problem is that
->write_begin() and ->write_end() now take folios as parameters instead
of pages. The same is true, of course, of the associated
__block_write_begin() and generic_write_end(). Update the driver.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2024-10-31 16:02:06 -03:00
Ernesto A. Fernández 49507c65bc Implement ip bitmap rotation correctly 
I never fully understood how the ip bitmap list was supposed to work,
but now I'm noticing that the difference between what I do and what the
official driver does are bad enough to quickly corrupt the list when a
container moves back and forth. Of course this is only an issue for
filesystems above 1.32 TiB or so, but it's still pretty serious.

So, I thought about it harder and I'm pretty sure I finally figured it
out: the array entries for each free bitmap are actually kept in a
linked list, that starts in sm_ip_bm_free_head and ends in
sm_ip_bm_free_tail. All entries outside the list (that is, the entries
for the used blocks) are marked with the invalid index 0xFFFF. The tail
itself also gets 0xFFFF even though it's free, because it has no next
entry - so that accounts for the constant off-by-one weirdness in the
number of free blocks.

So, implement all this stuff correctly. Stop assuming that all ip
bitmaps in use are consecutive, and stop rotating them all in lockstep
as soon as the transaction begins. Instead, read all ip bitmaps into
memory during the first transaction and keep them there, only tracking
if they have been dirtied. At commit time, allocate new blocks for the
dirty bitmaps and write them back to disk.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2024-10-29 21:13:46 -03:00
Ernesto A. Fernández dfdafd2a07 Wrap ephemeral objects around if there is no room 
Currently, the driver doesn't allow multiblock ephemeral objects to wrap
around. When an object doesn't fit in the checkpoint data area, we just
leave some empty space and adjust the mappings accordingly. This seemed
like a simple and reasonable solution to me, but the official driver
actually demands that all ephemeral objects are kept consecutive (and in
the same order as their mappings, which we already do).

In practice this means that if you are working with a container with a
multiblock spaceman (say 7T in size), and if the unmount transaction
just so happens to be the one where the spaceman doesn't fit, then you
will be left with a container that is seen as corrupt by the official
driver. To make matters worse, checkpoint integrity is checked on mount,
so macos will actually mount the container no problem, but silently
discarding my final transaction.

This is all, of course, extremely unlikely. Such large containers are
rare, and checkpoint data areas for them are huge so a wrap-around takes
a while to happen. And it needs to happen precisely during the unmount
transaction, and that same transaction has to involve important data the
user doesn't want to lose. All in all it's not a serious issue, but get
it fixed.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2024-10-22 17:34:51 -03:00
Integral 2920180ed6 Remove unused macro MIN()/MAX() & replace them with min()/max() (#78) 2024-10-04 16:04:05 -03:00
Ernesto A. Fernández 0667dbff17 0.3.11 
This release prevents deadlock if a single volume gets mounted and
unmounted at the same time. It also fixes two other (smaller) mount bugs
found while testing this one.

Support is added for the upcoming 6.11 kernel release, with a patch by
Woody Suwalski and a later patch of my own because things broke a second
time.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2024-09-12 20:20:49 -03:00
Ernesto A. Fernández af2ee526c0 Avoid double parsing of mount options 
Originally all mount options were parsed inside apfs_fill_super(),
because I was following the example of other filesystems (such as ext2)
that have no subvolumes or snapshots. When I implemented those features,
I realized that some mount options would be needed early on mount, so
that sget() could use them to decide if the superblock is already
mounted.

My lazy solution at the time was to leave parse_options() as it was, but
also add some redundant code early on mount to read the needed options.
Just looking at the amount of TODOs in those functions makes it clear
that this was always very silly. And now that the locking during mount
has been simplified, this is causing a race, and I want to take that
stuff seriously.

The bug is that while parse_options() rereads the mount options needed
by sget(), there is nothing preventing another concurrent mount from
testing the superblock. Say the first mount is for volume 1: the volume
number gets briefly reset to the default value of zero, and if the other
mount actually wants volume 0, sget() will consider this a match.

Anyway, split parse_options() into two equally clean functions, one for
early mount and another to get called within apfs_fill_super(). I even
considered simply parsing all the options early during mount, but that
would require a few more changes because the nxi is not attached at that
point. Maybe some other time.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2024-09-11 16:54:47 -03:00
Ernesto A. Fernández 44f4b4f631 Don't redefine MAX() and MIN() for the 6.11 kernel 
The build has become broken once again for the 6.11 kernel:

  https://github.com/linux-apfs/linux-apfs-rw/issues/75

The problem is that the second release candidate introduced generic
MAX() and MIN() macros for everyone to use, and that crashes with our
own definitions. So, only define them for older kernels.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2024-08-30 20:25:44 -03:00
Ernesto A. Fernández b44d2a37bf Don't forget to initialize the omap cache spinlock 
The omap cache has been in place for years, and it seems that its
spinlock was never actually initialized before use. I have no idea what
the consequences of this are, but for one it does cause lockdep to turn
itself off after every mount. Fix it.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2024-08-15 16:01:07 -03:00
Ernesto A. Fernández 4881d7c707 Don't return 0 on mount if sget() fails 
For the previous patch I had to carefully test all the possible errors
during mount and unmount. I noticed that injecting an error inside
apfs_set_super() triggers npd. The cause is that the error code returned
by sget() gets swallowed and the kernel is allowed to think that the
mount succeeded.

This is not a new issue, it's been around since apfs_mount() was first
implemented. I never noticed because apfs_set_super() would only fail if
we ran out of anonymous devices. So fix it in a new commit.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2024-08-15 16:01:07 -03:00
Ernesto A. Fernández 2982c8655f Don't use the mount mutex so much 
Repeatedly running generic/604 of xfstests eventually hangs the kernel.
This is a test that forces a mount to race with an unmount. The problem
is a standard deadlock: ->put_super() gets called while the s_umount
lock is held, and then starts a transaction, which takes nxs_mutex;
meanwhile ->mount() first takes nxs_mutex and then calls sget(), which
takes s_umount.

Before this patch, I had never really put much thought as to where
exactly nxs_mutex is needed. It's obviously the lock for the list of
mounted containers, but it's also used as a generic way to avoid
modifications to a container while another of its subvolumes is getting
mounted. It's always been obvious that I'm holding it for too long and
in too many places, so I want to finally put some effort into figuring
this out now, and leave a record here so that I don't forget.

The first obvious point is that the mutex can be dropped before calling
sget(), and retaken later. There should be no problem if another mount
called it at the same time, and it's enough to fix the bug. But putting
some thought here exposes an existing issue with this code: sget() may
actually test some superblock that is already half-unmounted and had its
info freed, so apfs_test_super() is risking npd or use-after-free.

Looking at other filesystems I realize that my mistake was freeing the
info inside ->put_super(), because at that point the super is still
listed in the fs_supers list for apfs. Instead ->kill_sb() should free
the info itself, after generic_shutdown_super() is done (which calls
->put_super()), and after kill_super_notify() has taken care of the
list. Since kill_super_notify() can't be called from inside a driver, a
little hack is needed to call it through kill_anon_super(). The cleanups
for apfs_mount() also need to be reconsidered, since ->put_super() is
only called by deactivate_locked_super() after the root dentry is set,
but the rest of ->kill_sb() runs either way.

Going back to the mutex: even if a race for sget() itself is not an
issue, dropping it there does have other consequences. The reference
count for the container superblock is increased before sget(), and it's
later checked on some initializations to see if this is the first mount.
That won't do anymore, because a second mount could increase the refcnt
before the mutex is retaken, and the first mount would never complete
its work. Use other fields for this check: in apfs_read_main_super() use
the blocksize (which is zero before the first mount sets it); for
apfs_set_nx_flags(), define a new flag to tell if the other flags have
been set.

This already tells us two functions that definitely need to be protected
by the mutex, so that they can complete their initializations before any
other mount comes to check them. The situation is similar for
apfs_first_read_omap(), which goes through the container's volume list
to check for previous reads. After going line by line through the mount
code, I'm fairly confident that the mutex only needs to be held for
apfs_attach_nxi() and these three functions. For unmount it's the same:
apfs_free_main_super() and apfs_put_omap() require the mutex. The second
will now unset the omap pointer for the volume atomically, so that it
can be checked by apfs_first_read_omap(); it is renamed accordingly.

So far this is all about the races between different mounts and umounts;
but we also don't want the mount to read inconsistent data from disk
because some existing mount happened to write to it. Back when the
driver could only mount one volume at a time, apfs_fill_super() didn't
need any locking, so when multiple mounts got implemented I decided to
just ban transactions during mounts entirely, by taking the nxs_mutex.
This double lock looks very awkward and is a factor on the deadlock we
are fixing, so let's clean that up now: use the read semaphore itself to
protect apfs_fill_super(). Of course this means that two mounts for the
same container can now run through this function at the same time, so be
careful with changes to the shared nxi: we've already dealt with this
issue for apfs_set_nx_flags(), but apfs_set_trans_buffer_limit() might
also be unsafe. Just in case, move it to apfs_read_main_super(), it
makes more sense that way anyway.

By the way, apfs_read_main_super() also reads from disk, but it only
does it on the first mount, so there is no risk of racing against a
transaction. There is one function that writes without a transaction,
namely apfs_make_super_copy(), but that function takes the nxs_mutex to
ensure that no mount will begin during the superblock update.

I think that covers everything? I'm really not sure if this writeup was
clear at all, I have a hard time reasoning about locks. I also put a
simpler list of the uses of nxs_mutex in a comment next to its
declaration, which should be more practical as a reference.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2024-08-15 16:01:06 -03:00
Ernesto A. Fernández 707d91b486 Undo code reordering in previous patch 
I just picked up a patch to add support for the 6.11 kernel. I only did
a shallow review, and now I notice that the code was reordered and a
null buffer head is getting dereferenced. Revert that.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2024-07-30 17:55:14 -03:00
Woody Suwalski 3e5c902a9a Kernel 6.11: use folio for page ops (#74) 2024-07-30 17:23:17 -03:00
Ernesto A. Fernández 695455bfdb 0.3.10 
Transactions are now committed regularly, so it should no longer be
strictly necessary to unmount the filesystem cleanly for writes to take
effect. It's still recommended, of course.

The release also fixes two bugs. One was a big performance regression
from 0.3.9, seen when deleting large numbers of files. The other was
that mounts would go read-only if two snapshots were attempted with the
same name.

Support is added for the new 6.10 kernel version, with a patch by Woody
Suwalski.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2024-07-15 15:13:17 -03:00
Woody Suwalski 071cdacc32 Small tweaks for building on kernel 6.10 2024-05-28 20:46:02 -03:00
Ernesto A. Fernández 57eb739f59 Don't abort a snapshot if the name is in use 
Attempting to create two snapshots with the same name will abort a
transaction and leave the filesystem in a read-only state. From the
comment it seems that I was aware of this issue when I first implemented
snapshots, but I guess I didn't like the obvious solution in this patch
at the time, so I decided to leave it for later.

Since taking snapshots is not expected to be a bottleneck, just run the
query for the name once more ahead of time, as soon as the transaction
opens. Now if the name already exists we don't need to abort, because no
changes have taken place.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2024-05-28 18:54:56 -03:00
Ernesto A. Fernández f088d1a2e3 Set up a task to commit transactions 
Currently my driver only commits transactions "synchronously", that is,
immediately following a user action that triggers it. This implies, in
particular, that the filesystem must get unmounted explicitly or else
data will almost certainly get dropped. While technically valid
behaviour, this is very weird and some people have complained about it
in the past.

So, stop ignoring requests for commit via apfs_transaction_commit() when
they are considered premature. Instead schedule a commit task, with a
small delay to avoid constant commits when the driver is still busy.

Note that the previous two patches were intended to help with this one:
the plan was that the commit task wouldn't need to hold a specific
volume superblock, and could instead work with the nx info. In the end
this proved too messy, so I'm just saving the volume superblock that
requested the commit in the transaction struct.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2024-05-23 18:46:25 -03:00
Ernesto A. Fernández 8bcaaebb89 Move trans_buffers_max from volume to container 
The limit on the number of buffer heads in a transaction depends only on
the ram size, so it's the same for all volumes. Move it to the container
superblock info. Frankly this could just be a global variable, set when
the driver is first inserted, but there is no problem with recalculating
it on all mounts.

The point of this change, along with the previous patch, is that soon we
won't require holding some volume to start and commit transactions.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2024-05-21 18:26:18 -03:00
Ernesto A. Fernández 781e0024d2 Stop attempting to undo aborted transactions 
When I first implemented transactions, my intention was to revert the
in-memory filesystem structures to a consistent state whenever we hit
the kind of errors that can be expected during normal operation. So I
always kept some stuff from the previous transaction around to copy it
back if needed. I never tested any of this though, so it most likely
never worked.

Later improvements mean that the kind of silly errors I had in mind
(things like ENOENT) are no longer supposed to happen during
transactions. The one exception here is ENOSPC, but that's only because
I haven't yet bothered to clean up the space reservations. The errors
that do happen are things like ENOMEM, EIO or EFSCORRUPTED, and I think
it's probably ok if the mount doesn't work perfectly after one of those.
Instead we force it into a read-only state, and let the user decide how
to proceed.

So, there is now little point in keeping my original half-baked plans
littering the transaction code. Remove all that stuff.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2024-05-20 20:06:53 -03:00
Ernesto A. Fernández 5d41bd7e64 Initialize transaction fields on first mount 
The container superblock info keeps a list of inodes and buffers that
have been modified in the current transaction. This is a bit of a hack,
but so far I haven't figured out a way to get rid of it. The lists need
to be initialized before they are first used, and this is currently done
by apfs_transaction_start() itself, after checking if this is the first
start since the last commit. This works fine, but it's really weird,
since the list gets regularly reinitialized to "empty" after getting
emptied naturally by a commit. Initialization should really be separate
from all the code that actually works with the transaction.

Define a new apfs_transaction_init() to take care of this, and call it
from apfs_attach_nxi() which already takes care of the initialization of
all other container info fields.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2024-05-20 15:18:14 -03:00
Ernesto A. Fernández 4e9984c815 Rename the transaction limit constants 
TRANSACTION_MAIN_QUEUE_MAX, TRANSACTION_BUFFERS_MAX and
TRANSACTION_STARTS_MAX were originally desfined inside transaction.c, so
I guess I didn't bother to give them the driver prefix. I don't expect
the names to collide with anything, but it's tidier to follow the same
naming conventions everywhere, so run

  sed -i 's/TRANSACTION_/APFS_TRANS_/' *.{c,h}

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2024-05-16 20:49:12 -03:00
Ernesto A. Fernández f1c9ab6c38 Adjust TRANSACTION_MAIN_QUEUE_MAX 
The iozone automatic tests suffered a big performance regression between
0.3.8 and 0.3.9, in my case from around 14m to close to 20m. The problem
goes away if the newly introduced apfs_dstream_delete_front() is tweaked
to delete files in larger chunks. That said, I don't think my choice for
TRANSACTION_MAIN_QUEUE_MAX was ever based on anything, so it's better to
adjust that instead of coming up with some hack for file deletion alone.

After a lot of testing, I think I get the best results if I raise it
from 4096 to 10000. The iozone benchmark is now better than before, at
around 13m for me.

Of course this will mean that transactions commit less often. I should
look into putting that in a workqueue like the file deletion.

Signed-off-by: Ernesto A. Fernández <ernesto@corellium.com>
 2024-05-16 20:32:39 -03:00
36 changed files with 3758 additions and 5784 deletions
Expand all files Collapse all files

No files matched your search

Makefile
+5
View File
@@ -13,6 +13,11 @@ apfs-y := btree.o compress.o dir.o extents.o file.o inode.o key.o libzbitmap.o \
lzfse/lzvn_decode_base.o message.o namei.o node.o object.o snapshot.o \
spaceman.o super.o symlink.o transaction.o unicode.o xattr.o xfield.o
# If you want mounts to be writable by default, run the build as:
# make CONFIG=-DCONFIG_APFS_RW_ALWAYS
# This is risky and not generally recommended.
ccflags-y += $(CONFIG)
default:
./genver.sh
make -C $(KERNEL_DIR) M=$(PWD)
README.rst
+26 -12
View File
@@ -4,13 +4,8 @@ Apple File System for Linux
The Apple File System (APFS) is the copy-on-write filesystem currently used on
all Apple devices. This module provides a degree of experimental support on
Linux.
It's supposed to work with a range of kernel versions starting at 4.9 or before,
but only a few of those have actually been tested. Also, kernel versions below
4.12 will be slower for some operations. If you run into any problem, please
send a report to <linux-apfs@googlegroups.com> or file a github issue at
https://github.com/eafer/linux-apfs-rw/issues.
Linux. It's intended to work with all kernel versions since 4.12, but testing
is focused on 5.18 and above.
To help test write support, a set of userland tools is also under development.
The git tree can be retrieved from https://github.com/eafer/apfsprogs.git.
@@ -19,15 +14,32 @@ Known limitations
=================
This module is the result of reverse engineering and testing has been limited.
If you make use of the write support, expect data corruption. Please report any
issues that you find, but I can't promise a quick resolution at this stage.
If you make use of the write support, there is a real risk of data corruption,
so mounts are read-only by default. Please report any issues that you find.
Encryption is not yet implemented even in read-only mode, and neither are
fusion drives.
Writes to fusion drives will likely never be supported, though I might
reconsider this if people actually need it. Encryption is not yet implemented
even in read-only mode.
Reporting bugs
==============
If you encounter any problem, the first thing you should do is run (as root)::
dmesg | grep -i apfs
to see all the error messages. If that doesn't help you, please file a github
issue at https://github.com/eafer/linux-apfs-rw/issues. Or send me an email if
you prefer.
Build
=====
Note that at this point the driver has been packaged for several distros, so
it may be more convenient to install it from there instead of building it
yourself. The distro packages are often outdated though, so be sure to check
the version.
In order to build a module out-of-tree, you will first need the Linux kernel
headers. Depending on your distro, you can get them by running (as root)::
@@ -65,12 +77,14 @@ vol=n Volume number to mount. The default is volume 0.
snap=label Volume snapshot to mount (in read-only mode).
tier2=path Path to the tier 2 device. For fusion drives only.
uid=n, gid=n Override on-disk inode ownership data with given uid/gid.
cknodes Verify the checksum on all metadata nodes. Right now this has a
severe performance cost, so it's not recommended.
readwrite Enable the experimental write support. This **will** corrupt your
readwrite Enable the experimental write support. This may corrupt your
container.
============ =================================================================
apfs.h
+178 -59
View File
@@ -11,6 +11,21 @@
#include <linux/list.h>
#include <linux/types.h>
#include <linux/version.h>
/*
* Redhat kernels regularly break the api, so they need their own version
* checks. Make sure they always fail for non-rhel kernels.
*
* By the way, if rhel keeps picking up breaking patches for old majors after
* a new one is out, this check could break down and things could get more
* complicated. I don't think they do that though...
*/
#ifdef RHEL_RELEASE
#define RHEL_VERSION_GE(a, b) (RHEL_RELEASE_CODE >= RHEL_RELEASE_VERSION(a, b))
#else
#define RHEL_VERSION_GE(a, b) 0
#endif
#include "apfs_raw.h"
#define EFSBADCRC EBADMSG /* Bad CRC detected */
@@ -29,10 +44,10 @@ static inline bool sb_rdonly(const struct super_block *sb) { return sb->s_flags
#endif
/* Compatibility wrapper around submit_bh() */
#if LINUX_VERSION_CODE < KERNEL_VERSION(6, 0, 0)
#define apfs_submit_bh(op, op_flags, bh) submit_bh(op, op_flags, bh)
#else
#if LINUX_VERSION_CODE >= KERNEL_VERSION(6, 0, 0) || RHEL_VERSION_GE(9, 2)
#define apfs_submit_bh(op, op_flags, bh) submit_bh(op | op_flags, bh)
#else
#define apfs_submit_bh(op, op_flags, bh) submit_bh(op, op_flags, bh)
#endif
/*
@@ -81,7 +96,7 @@ struct apfs_node {
int key; /* Offset of the key area in the block */
int free; /* Offset of the free area in the block */
int data; /* Offset of the data area in the block */
int val; /* Offset of the value area in the block */
int key_free_list_len; /* Length of the fragmented free key space */
int val_free_list_len; /* Length of the fragmented free value space */
@@ -117,6 +132,11 @@ static inline bool apfs_node_has_fixed_kv_size(struct apfs_node *node)
return (node->flags & APFS_BTNODE_FIXED_KV_SIZE) != 0;
}
struct apfs_ip_bitmap_block_info {
bool dirty; /* Do we need to commit this block to disk? */
void *block; /* In-memory address of the bitmap block */
};
/*
* Space manager data in memory.
*/
@@ -132,6 +152,7 @@ struct apfs_spaceman {
u32 sm_cib_count; /* Number of chunk-info blocks */
u64 sm_free_count; /* Number of free blocks */
u32 sm_addr_offset; /* Offset of cib addresses in @sm_raw */
u64 sm_main_fq_nodes; /* Number of nodes in the main fq */
/*
* A range of freed blocks not yet put in the free queue. Extend this as
@@ -147,12 +168,12 @@ struct apfs_spaceman {
/* Number of ip bitmaps */
u32 sm_ip_bmaps_count;
/* List of ip bitmaps, in order */
struct buffer_head *sm_ip_bmaps[];
struct apfs_ip_bitmap_block_info sm_ip_bmaps[];
};
#define TRANSACTION_MAIN_QUEUE_MAX 4096
#define TRANSACTION_BUFFERS_MAX 65536
#define TRANSACTION_STARTS_MAX 65536
#define APFS_TRANS_MAIN_QUEUE_MAX 10000
#define APFS_TRANS_BUFFERS_MAX 65536
#define APFS_TRANS_STARTS_MAX 65536
/* Possible states for the container transaction structure */
#define APFS_NX_TRANS_FORCE_COMMIT 1 /* Commit guaranteed */
@@ -160,28 +181,35 @@ struct apfs_spaceman {
#define APFS_NX_TRANS_COMMITTING 4 /* Commit ongoing */
#define APFS_NX_TRANS_INCOMPLETE_BLOCK 8 /* A data block is not written in full */
/*
* ENOSPC checks before transactions are more or less coarse depending on this.
*/
enum apfs_trans_kind {
APFS_TRANS_REG, /* Most transactions */
APFS_TRANS_DEL, /* Transactions that usually free space */
APFS_TRANS_SYNC, /* Sync (or unmount) transaction */
};
/* The space requirements for each kind of transaction, in blocks */
#define APFS_REG_ROOM 128
#define APFS_DEL_ROOM 20
#define APFS_SYNC_ROOM 6
/*
* Structure that keeps track of a container transaction.
*/
struct apfs_nx_transaction {
unsigned int t_state;
struct delayed_work t_work; /* Work task for transaction commits */
struct super_block *t_work_sb; /* sb that last queued a commit */
struct list_head t_inodes; /* List of inodes in the transaction */
struct list_head t_buffers; /* List of buffers in the transaction */
size_t t_buffers_count; /* Count of items on the list */
int t_starts_count; /* Count of starts for transaction */
};
/*
* Structure that keeps track of a volume transaction.
*/
struct apfs_vol_transaction {
struct buffer_head *t_old_vsb; /* Volume superblock being replaced */
struct apfs_node t_old_omap_root; /* Omap root node being replaced */
struct apfs_node t_old_cat_root; /* Catalog root node being replaced */
};
/* State bits for buffer heads in a transaction */
#define BH_TRANS BH_PrivateStart /* Attached to a transaction */
#define BH_CSUM (BH_PrivateStart + 1) /* Requires checksum update */
@@ -196,14 +224,6 @@ struct apfs_bh_info {
struct list_head list; /* List of buffers in the transaction */
};
/*
* Used to report how many operations may be needed for a transaction
*/
struct apfs_max_ops {
int cat; /* Maximum catalog records that may need changing */
int blks; /* Maximum extent blocks that may need changing */
};
/*
* List entry for an in-memory ephemeral object
*/
@@ -224,17 +244,44 @@ struct apfs_ephemeral_object_info {
/* Mount option flags for a container */
#define APFS_CHECK_NODES 1
#define APFS_READWRITE 2
/*
* Mount options for a container are decided on its first mount. This flag lets
* future mounts know that has happened already.
*/
#define APFS_FLAGS_SET 4
/*
* Wrapper around block devices for portability.
*/
struct apfs_blkdev_info {
struct block_device *blki_bdev;
#if LINUX_VERSION_CODE >= KERNEL_VERSION(6, 9, 0) || RHEL_VERSION_GE(9, 5)
struct file *blki_bdev_file;
#elif LINUX_VERSION_CODE >= KERNEL_VERSION(6, 8, 0)
struct bdev_handle *blki_bdev_handle;
#endif
#if LINUX_VERSION_CODE < KERNEL_VERSION(6, 5, 0) && !RHEL_VERSION_GE(9, 4)
fmode_t blki_mode;
#endif
/*
* For tier 2, we need to remember the mount path for show_options().
* We don't need this for the main device of course, but better to
* keep things consistent.
*/
char *blki_path;
};
/*
* Container superblock data in memory
*/
struct apfs_nxsb_info {
struct block_device *nx_bdev; /* Device for the container */
#if LINUX_VERSION_CODE >= KERNEL_VERSION(6, 9, 0)
struct file *nx_bdev_file;
#elif LINUX_VERSION_CODE >= KERNEL_VERSION(6, 8, 0)
struct bdev_handle *nx_bdev_handle;
#endif
/* Block device info for the container */
struct apfs_blkdev_info *nx_blkdev_info;
struct apfs_blkdev_info *nx_tier2_info;
u64 nx_tier2_bno; /* Offset for tier 2 block numbers */
struct apfs_nx_superblock *nx_raw; /* On-disk main sb */
u64 nx_bno; /* Current block number for the checkpoint superblock */
u64 nx_xid; /* Latest transaction id */
@@ -254,6 +301,7 @@ struct apfs_nxsb_info {
struct apfs_spaceman *nx_spaceman;
struct apfs_nx_transaction nx_transaction;
int nx_trans_buffers_max;
/* For now, a single semaphore for every operation */
struct rw_semaphore nx_big_sem;
@@ -318,6 +366,8 @@ struct apfs_sb_info {
dev_t s_anon_dev; /* Anonymous device for this volume-snapshot */
char *s_tier2_path; /* Path to the tier 2 device */
char *s_snap_name; /* Label for the mounted snapshot */
u64 s_snap_xid; /* Transaction id for mounted snapshot */
@@ -333,11 +383,9 @@ struct apfs_sb_info {
struct apfs_crypto_state_val *s_dflt_pfk; /* default per-file key */
struct apfs_vol_transaction s_transaction;
int s_trans_buffers_max;
struct inode *s_private_dir; /* Inode for the private directory */
struct work_struct s_orphan_cleanup_work;
atomic_t s_orphan_cleanup_err; /* Error from last orphan cleanup */
};
static inline struct apfs_sb_info *APFS_SB(struct super_block *sb)
@@ -646,8 +694,7 @@ static inline u64 apfs_cat_cnid(struct apfs_key_header *key)
#define APFS_QUERY_PREV 010000 /* Find previous record */
/*
* Structure used to retrieve data from an APFS B-Tree. For now only used
* on the calalog and the object map.
* Structure used to retrieve data from an APFS B-Tree.
*/
struct apfs_query {
struct apfs_node *node; /* Node being searched */
@@ -660,8 +707,8 @@ struct apfs_query {
int index; /* Index of the entry in the node */
int key_off; /* Offset of the key in the node */
int key_len; /* Length of the key */
int off; /* Offset of the data in the node */
int len; /* Length of the data */
int off; /* Offset of the value in the node */
int len; /* Length of the value */
int depth; /* Put a limit on recursion */
};
@@ -898,7 +945,7 @@ extern int apfs_rename(struct inode *old_dir, struct dentry *old_dentry,
unsigned int flags);
extern int apfs_create(struct inode *dir, struct dentry *dentry, umode_t mode,
bool excl);
#elif LINUX_VERSION_CODE < KERNEL_VERSION(6, 3, 0)
#elif LINUX_VERSION_CODE < KERNEL_VERSION(6, 3, 0) && !RHEL_VERSION_GE(9, 6)
extern int apfs_mknod(struct user_namespace *mnt_userns, struct inode *dir,
struct dentry *dentry, umode_t mode, dev_t rdev);
extern int apfs_mkdir(struct user_namespace *mnt_userns, struct inode *dir,
@@ -908,11 +955,20 @@ extern int apfs_rename(struct user_namespace *mnt_userns, struct inode *old_dir,
struct dentry *new_dentry, unsigned int flags);
extern int apfs_create(struct user_namespace *mnt_userns, struct inode *dir,
struct dentry *dentry, umode_t mode, bool excl);
#else
#elif LINUX_VERSION_CODE < KERNEL_VERSION(6, 15, 0)
extern int apfs_mknod(struct mnt_idmap *idmap, struct inode *dir,
struct dentry *dentry, umode_t mode, dev_t rdev);
extern int apfs_mkdir(struct mnt_idmap *idmap, struct inode *dir,
struct dentry *dentry, umode_t mode);
extern int apfs_rename(struct mnt_idmap *idmap, struct inode *old_dir,
struct dentry *old_dentry, struct inode *new_dir,
struct dentry *new_dentry, unsigned int flags);
extern int apfs_create(struct mnt_idmap *idmap, struct inode *dir,
struct dentry *dentry, umode_t mode, bool excl);
#else
extern int apfs_mknod(struct mnt_idmap *idmap, struct inode *dir,
struct dentry *dentry, umode_t mode, dev_t rdev);
extern struct dentry *apfs_mkdir(struct mnt_idmap *idmap, struct inode *dir, struct dentry *dentry, umode_t mode);
extern int apfs_rename(struct mnt_idmap *idmap, struct inode *old_dir,
struct dentry *old_dentry, struct inode *new_dir,
struct dentry *new_dentry, unsigned int flags);
@@ -925,7 +981,6 @@ extern int apfs_link(struct dentry *old_dentry, struct inode *dir,
extern int apfs_unlink(struct inode *dir, struct dentry *dentry);
extern int apfs_rmdir(struct inode *dir, struct dentry *dentry);
extern int apfs_delete_orphan_link(struct inode *inode);
extern int APFS_DELETE_ORPHAN_LINK_MAXOPS(void);
extern u64 apfs_any_orphan_ino(struct super_block *sb, u64 *ino_p);
/* extents.c */
@@ -940,7 +995,6 @@ extern int apfs_flush_extent_cache(struct apfs_dstream_info *dstream);
extern int apfs_dstream_get_new_bno(struct apfs_dstream_info *dstream, u64 dsblock, u64 *bno);
extern int apfs_get_new_block(struct inode *inode, sector_t iblock,
struct buffer_head *bh_result, int create);
extern int APFS_GET_NEW_BLOCK_MAXOPS(void);
extern int apfs_truncate(struct apfs_dstream_info *dstream, loff_t new_size);
extern int apfs_inode_delete_front(struct inode *inode);
#if LINUX_VERSION_CODE >= KERNEL_VERSION(4, 20, 0)
@@ -959,7 +1013,7 @@ extern int apfs_fsync(struct file *file, loff_t start, loff_t end, int datasync)
/* inode.c */
extern struct inode *apfs_iget(struct super_block *sb, u64 cnid);
extern int apfs_update_inode(struct inode *inode, char *new_name);
extern int APFS_UPDATE_INODE_MAXOPS(void);
extern void apfs_schedule_orphan_cleanup(struct super_block *sb);
extern void apfs_orphan_cleanup_work(struct work_struct *work);
extern void apfs_evict_inode(struct inode *inode);
extern struct inode *apfs_new_inode(struct inode *dir, umode_t mode,
@@ -967,14 +1021,13 @@ extern struct inode *apfs_new_inode(struct inode *dir, umode_t mode,
extern int apfs_create_inode_rec(struct super_block *sb, struct inode *inode,
struct dentry *dentry);
extern int apfs_inode_create_exclusive_dstream(struct inode *inode);
extern int APFS_CREATE_INODE_REC_MAXOPS(void);
extern int __apfs_write_begin(struct file *file, struct address_space *mapping, loff_t pos, unsigned int len, unsigned int flags, struct page **pagep, void **fsdata);
extern int __apfs_write_end(struct file *file, struct address_space *mapping, loff_t pos, unsigned int len, unsigned int copied, struct page *page, void *fsdata);
extern int apfs_dstream_adj_refcnt(struct apfs_dstream_info *dstream, u32 delta);
#if LINUX_VERSION_CODE < KERNEL_VERSION(5, 12, 0)
extern int apfs_setattr(struct dentry *dentry, struct iattr *iattr);
#elif LINUX_VERSION_CODE < KERNEL_VERSION(6, 3, 0)
#elif LINUX_VERSION_CODE < KERNEL_VERSION(6, 3, 0) && !RHEL_VERSION_GE(9, 6)
extern int apfs_setattr(struct user_namespace *mnt_userns,
struct dentry *dentry, struct iattr *iattr);
#else
@@ -996,7 +1049,7 @@ extern int apfs_getattr(struct vfsmount *mnt, struct dentry *dentry,
#elif LINUX_VERSION_CODE < KERNEL_VERSION(5, 12, 0)
extern int apfs_getattr(const struct path *path, struct kstat *stat,
u32 request_mask, unsigned int query_flags);
#elif LINUX_VERSION_CODE < KERNEL_VERSION(6, 3, 0)
#elif LINUX_VERSION_CODE < KERNEL_VERSION(6, 3, 0) && !RHEL_VERSION_GE(9, 6)
extern int apfs_getattr(struct user_namespace *mnt_userns,
const struct path *path, struct kstat *stat, u32 request_mask,
unsigned int query_flags);
@@ -1007,9 +1060,8 @@ extern int apfs_getattr(struct mnt_idmap *idmap,
#endif
extern int apfs_crypto_adj_refcnt(struct super_block *sb, u64 crypto_id, int delta);
extern int APFS_CRYPTO_ADJ_REFCNT_MAXOPS(void);
#if LINUX_VERSION_CODE >= KERNEL_VERSION(6, 3, 0)
#if LINUX_VERSION_CODE >= KERNEL_VERSION(6, 3, 0) || RHEL_VERSION_GE(9, 6)
extern int apfs_fileattr_get(struct dentry *dentry, struct fileattr *fa);
extern int apfs_fileattr_set(struct mnt_idmap *idmap, struct dentry *dentry, struct fileattr *fa);
#elif LINUX_VERSION_CODE >= KERNEL_VERSION(5, 13, 0)
@@ -1070,6 +1122,8 @@ extern int apfs_read_spaceman(struct super_block *sb);
extern int apfs_free_queue_insert_nocache(struct super_block *sb, u64 bno, u64 count);
extern int apfs_free_queue_insert(struct super_block *sb, u64 bno, u64 count);
extern int apfs_spaceman_allocate_block(struct super_block *sb, u64 *bno, bool backwards);
extern int apfs_write_ip_bitmaps(struct super_block *sb);
extern int apfs_spaceman_get_free_blkcnt(struct super_block *sb, u64 *blkcnt);
/* super.c */
extern int apfs_map_volume_super_bno(struct super_block *sb, u64 bno, bool check);
@@ -1081,13 +1135,15 @@ extern int apfs_sync_fs(struct super_block *sb, int wait);
/* transaction.c */
extern int apfs_cpoint_data_free(struct super_block *sb, u64 bno);
extern int apfs_transaction_start(struct super_block *sb, struct apfs_max_ops maxops);
extern void apfs_transaction_init(struct apfs_nx_transaction *trans);
extern int apfs_transaction_start(struct super_block *sb, enum apfs_trans_kind kind);
extern int apfs_transaction_commit(struct super_block *sb);
extern void apfs_inode_join_transaction(struct super_block *sb, struct inode *inode);
extern int apfs_transaction_join(struct super_block *sb,
struct buffer_head *bh);
void apfs_transaction_abort(struct super_block *sb);
extern int apfs_transaction_flush_all_inodes(struct super_block *sb);
extern int apfs_read_ephemeral_objects(struct super_block *sb);
/* xattr.c */
extern int ____apfs_xattr_get(struct inode *inode, const char *name, void *buffer,
@@ -1097,7 +1153,6 @@ extern int __apfs_xattr_get(struct inode *inode, const char *name, void *buffer,
extern int apfs_delete_all_xattrs(struct inode *inode);
extern int apfs_xattr_set(struct inode *inode, const char *name, const void *value,
size_t size, int flags);
extern int APFS_XATTR_SET_MAXOPS(void);
extern ssize_t apfs_listxattr(struct dentry *dentry, char *buffer, size_t size);
extern int apfs_xattr_get_compressed_data(struct inode *inode, const char *name, struct apfs_compressed_data *cdata);
extern void apfs_release_compressed_data(struct apfs_compressed_data *cdata);
@@ -1148,6 +1203,10 @@ static inline void apfs_assert_query_is_valid(const struct apfs_query *query)
ASSERT(apfs_node_is_root(apfs_query_root(query)));
}
#if LINUX_VERSION_CODE >= KERNEL_VERSION(6, 15, 0)
#define page_has_buffers(page) folio_buffers(page_folio(page))
#endif
/*
* TODO: the following are modified variants of buffer head functions that will
* work with the shared block device for the container. The correct approach
@@ -1157,8 +1216,29 @@ static inline void apfs_assert_query_is_valid(const struct apfs_query *query)
static inline void
apfs_map_bh(struct buffer_head *bh, struct super_block *sb, sector_t block)
{
struct apfs_nxsb_info *nxi = NULL;
struct apfs_blkdev_info *info = NULL;
nxi = APFS_NXI(sb);
info = nxi->nx_blkdev_info;
/*
* Block numbers above s_tier2_bno are for the tier 2 device of a
* fusion drive. Don't bother returning an error code if a regular
* drive gets corrupted and reports a tier 2 block number: just treat
* it the same as any other out-of-range block.
*/
if (block >= nxi->nx_tier2_bno) {
if (nxi->nx_tier2_info) {
info = nxi->nx_tier2_info;
block -= nxi->nx_tier2_bno;
} else {
apfs_err(sb, "block number in tier 2 range (0x%llx)", (unsigned long long)block);
}
}
set_buffer_mapped(bh);
bh->b_bdev = APFS_NXI(sb)->nx_bdev;
bh->b_bdev = info->blki_bdev;
bh->b_blocknr = block;
bh->b_size = sb->s_blocksize;
}
@@ -1166,20 +1246,59 @@ apfs_map_bh(struct buffer_head *bh, struct super_block *sb, sector_t block)
static inline struct buffer_head *
apfs_sb_bread(struct super_block *sb, sector_t block)
{
return __bread_gfp(APFS_NXI(sb)->nx_bdev, block, sb->s_blocksize, __GFP_MOVABLE);
struct apfs_nxsb_info *nxi = NULL;
struct apfs_blkdev_info *info = NULL;
nxi = APFS_NXI(sb);
info = nxi->nx_blkdev_info;
if (block >= nxi->nx_tier2_bno) {
if (!nxi->nx_tier2_info) {
/* Not really a fusion drive, so it's corrupted */
apfs_err(sb, "block number in tier 2 range (0x%llx)", (unsigned long long)block);
return NULL;
}
info = nxi->nx_tier2_info;
block -= nxi->nx_tier2_bno;
}
return __bread_gfp(info->blki_bdev, block, sb->s_blocksize, __GFP_MOVABLE);
}
/* Like apfs_getblk(), but doesn't mark the buffer uptodate */
static inline struct buffer_head *
__apfs_getblk(struct super_block *sb, sector_t block)
{
struct apfs_nxsb_info *nxi = NULL;
struct apfs_blkdev_info *info = NULL;
nxi = APFS_NXI(sb);
info = nxi->nx_blkdev_info;
if (block >= nxi->nx_tier2_bno) {
if (!nxi->nx_tier2_info) {
/* Not really a fusion drive, so it's corrupted */
apfs_err(sb, "block number in tier 2 range (0x%llx)", (unsigned long long)block);
return NULL;
}
info = nxi->nx_tier2_info;
block -= nxi->nx_tier2_bno;
}
#if LINUX_VERSION_CODE < KERNEL_VERSION(6, 7, 0)
return __getblk_gfp(info->blki_bdev, block, sb->s_blocksize, __GFP_MOVABLE);
#else
return bdev_getblk(info->blki_bdev, block, sb->s_blocksize, __GFP_MOVABLE);
#endif
}
/* Use instead of apfs_sb_bread() for blocks that will just be overwritten */
static inline struct buffer_head *
apfs_getblk(struct super_block *sb, sector_t block)
{
struct buffer_head *bh;
struct buffer_head *bh = NULL;
#if LINUX_VERSION_CODE < KERNEL_VERSION(6, 7, 0)
bh = __getblk_gfp(APFS_NXI(sb)->nx_bdev, block, sb->s_blocksize, __GFP_MOVABLE);
#else
bh = bdev_getblk(APFS_NXI(sb)->nx_bdev, block, sb->s_blocksize, __GFP_MOVABLE);
#endif
bh = __apfs_getblk(sb, block);
if (bh)
set_buffer_uptodate(bh);
return bh;
apfs_raw.h
+8
View File
@@ -1474,6 +1474,8 @@ struct apfs_compress_rsrc_data {
} __packed block[];
} __packed;
#define APFS_FUSION_TIER2_DEVICE_BYTE_ADDR 0x4000000000000000ULL
/*
* TODO: this is a placeholder, create some snapshots with the official
* implementation to find the actual limit.
@@ -1495,6 +1497,12 @@ struct apfs_snap_metadata_val {
u8 name[];
} __packed;
/* Snapshot metadata record flags */
enum {
APFS_SNAP_META_PENDING_DATALESS = 0x00000001,
APFS_SNAP_MERGE_IN_PROGRESS = 0x00000002,
};
/*
* Structure of the value of a snapshot name record
*/
btree.c
+1 -1
View File
@@ -50,7 +50,7 @@ static int apfs_child_from_query(struct apfs_query *query, u64 *child)
index_val = (struct apfs_btn_index_node_val *)(raw + query->off);
*child = le64_to_cpu(index_val->binv_child_oid) + apfs_catalog_base_oid(query);
} else {
if (query->len != 8) { /* The data on a nonleaf node is the child id */
if (query->len != 8) { /* The value on a nonleaf node is the child id */
apfs_err(sb, "bad index value length (%d)", query->len);
return -EFSCORRUPTED;
}
compress.c
+6 -2
View File
@@ -409,7 +409,7 @@ static ssize_t apfs_compress_file_read_page(struct file *filp, char *buf, loff_t
return step;
}
#if LINUX_VERSION_CODE >= KERNEL_VERSION(5, 19, 0)
#if LINUX_VERSION_CODE >= KERNEL_VERSION(5, 19, 0) || RHEL_VERSION_GE(9, 3)
static int apfs_compress_read_folio(struct file *filp, struct folio *folio)
{
struct page *page = &folio->page;
@@ -422,7 +422,11 @@ static int apfs_compress_readpage(struct file *filp, struct page *page)
loff_t off;
/* Mostly copied from ext4_read_inline_page() */
#if LINUX_VERSION_CODE >= KERNEL_VERSION(6, 16, 0)
off = page_folio(page)->index << PAGE_SHIFT;
#else
off = page->index << PAGE_SHIFT;
#endif
addr = kmap(page);
ret = apfs_compress_file_read_page(filp, addr, off);
flush_dcache_page(page);
@@ -438,7 +442,7 @@ static int apfs_compress_readpage(struct file *filp, struct page *page)
}
const struct address_space_operations apfs_compress_aops = {
#if LINUX_VERSION_CODE >= KERNEL_VERSION(5, 19, 0)
#if LINUX_VERSION_CODE >= KERNEL_VERSION(5, 19, 0) || RHEL_VERSION_GE(9, 3)
.read_folio = apfs_compress_read_folio,
#else
.readpage = apfs_compress_readpage,
dir.c
+16 -56
View File
@@ -414,7 +414,6 @@ fail:
apfs_free_query(query);
return ret;
}
#define APFS_CREATE_DENTRY_REC_MAXOPS 1
/**
* apfs_build_sibling_val - Allocate and initialize a sibling link's value
@@ -491,7 +490,6 @@ fail:
apfs_free_query(query);
return ret;
}
#define APFS_CREATE_SIBLING_LINK_REC_MAXOPS 1
/**
* apfs_create_sibling_map_rec - Create a sibling map record for a dentry
@@ -534,7 +532,6 @@ fail:
apfs_free_query(query);
return ret;
}
#define APFS_CREATE_SIBLING_MAP_REC_MAXOPS 1
/**
* apfs_create_sibling_recs - Create sibling link and map records for a dentry
@@ -567,8 +564,6 @@ static int apfs_create_sibling_recs(struct dentry *dentry,
*sibling_id = cnid;
return 0;
}
#define APFS_CREATE_SIBLING_RECS_MAXOPS (APFS_CREATE_SIBLING_LINK_REC_MAXOPS + \
APFS_CREATE_SIBLING_MAP_REC_MAXOPS)
/**
* apfs_create_dentry - Create all records for a new dentry
@@ -613,9 +608,6 @@ static int apfs_create_dentry(struct dentry *dentry, struct inode *inode)
apfs_inode_join_transaction(parent->i_sb, parent);
return 0;
}
#define APFS_CREATE_DENTRY_MAXOPS (APFS_CREATE_SIBLING_RECS_MAXOPS + \
APFS_CREATE_DENTRY_REC_MAXOPS + \
APFS_UPDATE_INODE_MAXOPS())
/**
* apfs_undo_create_dentry - Clean up apfs_create_dentry()
@@ -643,15 +635,9 @@ int apfs_mkany(struct inode *dir, struct dentry *dentry, umode_t mode,
{
struct super_block *sb = dir->i_sb;
struct inode *inode;
struct apfs_max_ops maxops;
int err;
maxops.cat = APFS_CREATE_INODE_REC_MAXOPS() + APFS_CREATE_DENTRY_MAXOPS;
if (symname)
maxops.cat += APFS_XATTR_SET_MAXOPS();
maxops.blks = 0;
err = apfs_transaction_start(sb, maxops);
err = apfs_transaction_start(sb, APFS_TRANS_REG);
if (err)
return err;
@@ -712,7 +698,7 @@ out_abort:
#if LINUX_VERSION_CODE < KERNEL_VERSION(5, 12, 0)
int apfs_mknod(struct inode *dir, struct dentry *dentry, umode_t mode,
dev_t rdev)
#elif LINUX_VERSION_CODE < KERNEL_VERSION(6, 3, 0)
#elif LINUX_VERSION_CODE < KERNEL_VERSION(6, 3, 0) && !RHEL_VERSION_GE(9, 6)
int apfs_mknod(struct user_namespace *mnt_userns, struct inode *dir,
struct dentry *dentry, umode_t mode, dev_t rdev)
#else
@@ -730,7 +716,7 @@ int apfs_mkdir(struct inode *dir, struct dentry *dentry, umode_t mode)
return apfs_mknod(dir, dentry, mode | S_IFDIR, 0 /* rdev */);
}
#elif LINUX_VERSION_CODE < KERNEL_VERSION(6, 3, 0)
#elif LINUX_VERSION_CODE < KERNEL_VERSION(6, 3, 0) && !RHEL_VERSION_GE(9, 6)
int apfs_mkdir(struct user_namespace *mnt_userns, struct inode *dir,
struct dentry *dentry, umode_t mode)
@@ -738,7 +724,7 @@ int apfs_mkdir(struct user_namespace *mnt_userns, struct inode *dir,
return apfs_mknod(mnt_userns, dir, dentry, mode | S_IFDIR, 0 /* rdev */);
}
#else
#elif LINUX_VERSION_CODE < KERNEL_VERSION(6, 15, 0)
int apfs_mkdir(struct mnt_idmap *idmap, struct inode *dir,
struct dentry *dentry, umode_t mode)
@@ -746,6 +732,13 @@ int apfs_mkdir(struct mnt_idmap *idmap, struct inode *dir,
return apfs_mknod(idmap, dir, dentry, mode | S_IFDIR, 0 /* rdev */);
}
#else
struct dentry *apfs_mkdir(struct mnt_idmap *idmap, struct inode *dir, struct dentry *dentry, umode_t mode)
{
return ERR_PTR(apfs_mknod(idmap, dir, dentry, mode | S_IFDIR, 0 /* rdev */));
}
#endif
@@ -757,7 +750,7 @@ int apfs_create(struct inode *dir, struct dentry *dentry, umode_t mode,
return apfs_mknod(dir, dentry, mode, 0 /* rdev */);
}
#elif LINUX_VERSION_CODE < KERNEL_VERSION(6, 3, 0)
#elif LINUX_VERSION_CODE < KERNEL_VERSION(6, 3, 0) && !RHEL_VERSION_GE(9, 6)
int apfs_create(struct user_namespace *mnt_userns, struct inode *dir,
struct dentry *dentry, umode_t mode, bool excl)
@@ -818,8 +811,6 @@ static int apfs_prepare_dentry_for_link(struct dentry *dentry)
return apfs_create_dentry_rec(d_inode(dentry), &dentry->d_name,
apfs_ino(parent), sibling_id);
}
#define APFS_PREPARE_DENTRY_FOR_LINK_MAXOPS (1 + APFS_CREATE_SIBLING_RECS_MAXOPS + \
APFS_CREATE_DENTRY_REC_MAXOPS)
/**
* __apfs_undo_link - Clean up __apfs_link()
@@ -875,22 +866,15 @@ fail:
drop_nlink(inode);
return err;
}
#define __APFS_LINK_MAXOPS (APFS_UPDATE_INODE_MAXOPS() + \
APFS_PREPARE_DENTRY_FOR_LINK_MAXOPS + \
APFS_CREATE_DENTRY_MAXOPS)
int apfs_link(struct dentry *old_dentry, struct inode *dir,
struct dentry *dentry)
{
struct super_block *sb = dir->i_sb;
struct inode *inode = d_inode(old_dentry);
struct apfs_max_ops maxops;
int err;
maxops.cat = __APFS_LINK_MAXOPS;
maxops.blks = 0;
err = apfs_transaction_start(sb, maxops);
err = apfs_transaction_start(sb, APFS_TRANS_REG);
if (err)
return err;
@@ -954,7 +938,6 @@ fail:
apfs_free_query(query);
return ret;
}
#define APFS_DELETE_SIBLING_LINK_REC_MAXOPS 1
/**
* apfs_delete_sibling_map_rec - Delete the sibling map record for a dentry
@@ -995,7 +978,6 @@ fail:
apfs_free_query(query);
return ret;
}
#define APFS_DELETE_SIBLING_MAP_REC_MAXOPS 1
/**
* apfs_delete_dentry - Delete all records for a dentry
@@ -1048,9 +1030,6 @@ static int apfs_delete_dentry(struct dentry *dentry)
apfs_inode_join_transaction(sb, parent);
return err;
}
#define APFS_DELETE_DENTRY_MAXOPS (1 + APFS_DELETE_SIBLING_LINK_REC_MAXOPS + \
APFS_DELETE_SIBLING_MAP_REC_MAXOPS + \
APFS_UPDATE_INODE_MAXOPS())
/**
* apfs_undo_delete_dentry - Clean up apfs_delete_dentry()
@@ -1217,8 +1196,6 @@ fail:
kfree(qname.name);
return err;
}
#define APFS_CREATE_ORPHAN_LINK_MAXOPS (APFS_CREATE_DENTRY_REC_MAXOPS + \
APFS_UPDATE_INODE_MAXOPS())
/**
* apfs_delete_orphan_link - Delete the link for an orphan inode
@@ -1269,10 +1246,6 @@ fail:
kfree(qname.name);
return err;
}
int APFS_DELETE_ORPHAN_LINK_MAXOPS(void)
{
return 1 + APFS_UPDATE_INODE_MAXOPS();
}
/**
* __apfs_undo_unlink - Clean up __apfs_unlink()
@@ -1368,20 +1341,13 @@ fail:
__apfs_undo_unlink(dentry);
return err;
}
#define __APFS_UNLINK_MAXOPS (APFS_DELETE_DENTRY_MAXOPS + \
APFS_CREATE_ORPHAN_LINK_MAXOPS + \
APFS_UPDATE_INODE_MAXOPS())
int apfs_unlink(struct inode *dir, struct dentry *dentry)
{
struct super_block *sb = dir->i_sb;
struct apfs_max_ops maxops;
int err;
maxops.cat = __APFS_UNLINK_MAXOPS;
maxops.blks = 0;
err = apfs_transaction_start(sb, maxops);
err = apfs_transaction_start(sb, APFS_TRANS_DEL);
if (err)
return err;
@@ -1414,7 +1380,7 @@ int apfs_rmdir(struct inode *dir, struct dentry *dentry)
int apfs_rename(struct inode *old_dir, struct dentry *old_dentry,
struct inode *new_dir, struct dentry *new_dentry,
unsigned int flags)
#elif LINUX_VERSION_CODE < KERNEL_VERSION(6, 3, 0)
#elif LINUX_VERSION_CODE < KERNEL_VERSION(6, 3, 0) && !RHEL_VERSION_GE(9, 6)
int apfs_rename(struct user_namespace *mnt_userns, struct inode *old_dir,
struct dentry *old_dentry, struct inode *new_dir,
struct dentry *new_dentry, unsigned int flags)
@@ -1427,7 +1393,6 @@ int apfs_rename(struct mnt_idmap *idmap, struct inode *old_dir,
struct super_block *sb = old_dir->i_sb;
struct inode *old_inode = d_inode(old_dentry);
struct inode *new_inode = d_inode(new_dentry);
struct apfs_max_ops maxops;
int err;
if (new_inode && APFS_I(new_inode)->i_nchildren)
@@ -1436,12 +1401,7 @@ int apfs_rename(struct mnt_idmap *idmap, struct inode *old_dir,
if (flags & ~RENAME_NOREPLACE) /* TODO: support RENAME_EXCHANGE */
return -EINVAL;
maxops.cat = __APFS_UNLINK_MAXOPS + __APFS_LINK_MAXOPS;
if (new_inode)
maxops.cat += __APFS_UNLINK_MAXOPS;
maxops.blks = 0;
err = apfs_transaction_start(sb, maxops);
err = apfs_transaction_start(sb, APFS_TRANS_REG);
if (err)
return err;
dkms.conf
+1 -1
View File
@@ -1,5 +1,5 @@
PACKAGE_NAME="linux-apfs-rw"
PACKAGE_VERSION="0.3.9"
PACKAGE_VERSION="0.3.15"
BUILT_MODULE_NAME[0]="apfs"
DEST_MODULE_LOCATION[0]="/extra"
extents.c
+9 -27
View File
@@ -8,8 +8,6 @@
#include <linux/blk_types.h>
#include "apfs.h"
#define MAX(X, Y) ((X) <= (Y) ? (Y) : (X))
/**
* apfs_ext_is_hole - Does this extent represent a hole in a sparse file?
* @extent: the extent to check
@@ -760,7 +758,6 @@ static int apfs_update_extent(struct apfs_dstream_info *dstream, const struct ap
}
return apfs_update_mid_extent(dstream, extent);
}
#define APFS_UPDATE_EXTENTS_MAXOPS (1 + 2 * APFS_CRYPTO_ADJ_REFCNT_MAXOPS())
static int apfs_extend_phys_extent(struct apfs_query *query, u64 bno, u64 blkcnt, u64 dstream_id)
{
@@ -1138,7 +1135,6 @@ int apfs_flush_extent_cache(struct apfs_dstream_info *dstream)
dstream->ds_ext_dirty = false;
return 0;
}
#define APFS_FLUSH_EXTENT_CACHE APFS_UPDATE_EXTENTS_MAXOPS
/**
* apfs_create_hole - Create and insert a hole extent for the dstream
@@ -1453,10 +1449,6 @@ static int apfs_dstream_get_new_block(struct apfs_dstream_info *dstream, u64 dsb
dstream->ds_ext_dirty = true;
return 0;
}
int APFS_GET_NEW_BLOCK_MAXOPS(void)
{
return APFS_FLUSH_EXTENT_CACHE;
}
/**
* apfs_dstream_get_new_bno - Allocate a new block inside a dstream
@@ -1651,7 +1643,7 @@ static int apfs_dstream_delete_front(struct super_block *sb, u64 ds_id)
int ret;
fq = &sm_raw->sm_fq[APFS_SFQ_MAIN];
if (le64_to_cpu(fq->sfq_count) > TRANSACTION_MAIN_QUEUE_MAX)
if (le64_to_cpu(fq->sfq_count) > APFS_TRANS_MAIN_QUEUE_MAX)
return -EAGAIN;
query = apfs_alloc_query(sbi->s_cat_root, NULL /* parent */);
@@ -1710,7 +1702,7 @@ next_extent:
}
}
if (le64_to_cpu(fq->sfq_count) <= TRANSACTION_MAIN_QUEUE_MAX)
if (le64_to_cpu(fq->sfq_count) <= APFS_TRANS_MAIN_QUEUE_MAX)
goto next_extent;
ret = -EAGAIN;
out:
@@ -1763,8 +1755,6 @@ int apfs_clone_file_range(struct file *src_file, loff_t off, struct file *dst_fi
struct apfs_dstream_info *dst_ds = &dst_ai->i_dstream;
struct super_block *sb = src_inode->i_sb;
struct apfs_sb_info *sbi = APFS_SB(sb);
/* TODO: remember to update the maxops in the future */
struct apfs_max_ops maxops = {0};
const u64 xfield_flags = APFS_INODE_MAINTAIN_DIR_STATS | APFS_INODE_IS_SPARSE | APFS_INODE_HAS_PURGEABLE_FLAGS;
int err;
@@ -1798,7 +1788,7 @@ int apfs_clone_file_range(struct file *src_file, loff_t off, struct file *dst_fi
return -EOPNOTSUPP;
}
err = apfs_transaction_start(sb, maxops);
err = apfs_transaction_start(sb, APFS_TRANS_REG);
if (err)
return err;
apfs_inode_join_transaction(sb, src_inode);
@@ -1911,7 +1901,7 @@ out:
* @paddr_end: first block after the extent to put
* @paddr_min: don't put references before this block
*
* Puts a reference to the physical extent range that ends in paddr. Sets
* Puts a reference to the physical extent range that ends in @paddr_end. Sets
* @paddr_end to the beginning of the extent, so that the caller can continue
* with the previous one. Returns 0 on success, or a negative error code in
* case of failure.
@@ -1973,7 +1963,7 @@ restart:
prev_end = prev_ext.bno + prev_ext.blkcount;
if (prev_end < *paddr_end) {
/* The extent to put is part of a snapshot */
tmp.phys_block_num = MAX(prev_end, paddr_min);
tmp.phys_block_num = max(prev_end, paddr_min);
tmp.len = (*paddr_end - tmp.phys_block_num) << sb->s_blocksize_bits;
ret = apfs_create_update_pext(query, &tmp, -1);
*paddr_end = tmp.phys_block_num;
@@ -2053,7 +2043,7 @@ static int apfs_range_put_reference(struct super_block *sb, u64 paddr, u64 lengt
* @paddr_end: first block after the extent to take
* @paddr_min: don't take references before this block
*
* Takes a reference to the physical extent range that ends in paddr. Sets
* Takes a reference to the physical extent range that ends in @paddr_end. Sets
* @paddr_end to the beginning of the extent, so that the caller can continue
* with the previous one. Returns 0 on success, or a negative error code in
* case of failure.
@@ -2113,7 +2103,7 @@ restart:
prev_end = prev_ext.bno + prev_ext.blkcount;
if (prev_end < *paddr_end) {
/* The extent to take is part of a snapshot */
tmp.phys_block_num = MAX(prev_end, paddr_min);
tmp.phys_block_num = max(prev_end, paddr_min);
tmp.len = (*paddr_end - tmp.phys_block_num) << sb->s_blocksize_bits;
ret = apfs_create_update_pext(query, &tmp, +1);
*paddr_end = tmp.phys_block_num;
@@ -2297,11 +2287,7 @@ int apfs_nonsparse_dstream_read(struct apfs_dstream_info *dstream, void *buf, si
goto out;
}
#if LINUX_VERSION_CODE < KERNEL_VERSION(6, 7, 0)
bhs[idx] = __getblk_gfp(APFS_NXI(sb)->nx_bdev, bno, sb->s_blocksize, __GFP_MOVABLE);
#else
bhs[idx] = bdev_getblk(APFS_NXI(sb)->nx_bdev, bno, sb->s_blocksize, __GFP_MOVABLE);
#endif
bhs[idx] = __apfs_getblk(sb, bno);
if (!bhs[idx]) {
apfs_err(sb, "failed to map block 0x%llx", bno);
ret = -EIO;
@@ -2373,11 +2359,7 @@ void apfs_nonsparse_dstream_preread(struct apfs_dstream_info *dstream)
if (ret || bno == 0)
return;
#if LINUX_VERSION_CODE < KERNEL_VERSION(6, 7, 0)
bh = __getblk_gfp(APFS_NXI(sb)->nx_bdev, bno, sb->s_blocksize, __GFP_MOVABLE);
#else
bh = bdev_getblk(APFS_NXI(sb)->nx_bdev, bno, sb->s_blocksize, __GFP_MOVABLE);
#endif
bh = __apfs_getblk(sb, bno);
if (!bh)
return;
if (!buffer_uptodate(bh)) {
file.c
+14 -10
View File
@@ -12,6 +12,13 @@
typedef int vm_fault_t;
#endif
#if LINUX_VERSION_CODE >= KERNEL_VERSION(6, 15, 0)
static void wait_for_stable_page(struct page *page)
{
return folio_wait_stable(page_folio(page));
}
#endif
#if LINUX_VERSION_CODE < KERNEL_VERSION(4, 11, 0)
static vm_fault_t apfs_page_mkwrite(struct vm_area_struct *vma, struct vm_fault *vmf)
{
@@ -28,8 +35,6 @@ static vm_fault_t apfs_page_mkwrite(struct vm_fault *vmf)
struct super_block *sb = inode->i_sb;
struct buffer_head *bh, *head;
vm_fault_t ret = VM_FAULT_LOCKED;
struct apfs_max_ops maxops;
int blkcount = PAGE_SIZE >> inode->i_blkbits;
unsigned int blocksize, block_start, len;
u64 size;
int err = 0;
@@ -37,12 +42,7 @@ static vm_fault_t apfs_page_mkwrite(struct vm_fault *vmf)
sb_start_pagefault(inode->i_sb);
file_update_time(vma->vm_file);
/* Placeholder values, I need to get back to this in the future */
maxops.cat = APFS_UPDATE_INODE_MAXOPS() +
blkcount * APFS_GET_NEW_BLOCK_MAXOPS();
maxops.blks = blkcount;
err = apfs_transaction_start(sb, maxops);
err = apfs_transaction_start(sb, APFS_TRANS_REG);
if (err)
goto out;
apfs_inode_join_transaction(sb, inode);
@@ -72,7 +72,11 @@ static vm_fault_t apfs_page_mkwrite(struct vm_fault *vmf)
}
size = i_size_read(inode);
#if LINUX_VERSION_CODE >= KERNEL_VERSION(6, 16, 0)
if (page_folio(page)->index == size >> PAGE_SHIFT)
#else
if (page->index == size >> PAGE_SHIFT)
#endif
len = size & ~PAGE_MASK;
else
len = PAGE_SIZE;
@@ -113,7 +117,7 @@ out_abort:
apfs_transaction_abort(sb);
out:
if (err)
#if LINUX_VERSION_CODE < KERNEL_VERSION(6, 6, 0)
#if LINUX_VERSION_CODE < KERNEL_VERSION(6, 6, 0) && !RHEL_VERSION_GE(9, 5)
ret = block_page_mkwrite_return(err);
#else
ret = vmf_fs_error(err);
@@ -132,7 +136,7 @@ int apfs_file_mmap(struct file *file, struct vm_area_struct *vma)
{
struct address_space *mapping = file->f_mapping;
#if LINUX_VERSION_CODE >= KERNEL_VERSION(5, 19, 0)
#if LINUX_VERSION_CODE >= KERNEL_VERSION(5, 19, 0) || RHEL_VERSION_GE(9, 3)
if (!mapping->a_ops->read_folio)
#else
if (!mapping->a_ops->readpage)
inode.c
+112 -103
View File
File diff suppressed because it is too large. Load diff
key.c
+7 -4
View File
@@ -180,7 +180,7 @@ int apfs_read_free_queue_key(void *raw, int size, struct apfs_key *key)
{
struct apfs_spaceman_free_queue_key *raw_key;
if (size < sizeof(struct apfs_spaceman_free_queue_key)) {
if (size != sizeof(*raw_key)) {
apfs_err(NULL, "bad key length (%d)", size);
return -EFSCORRUPTED;
}
@@ -204,14 +204,17 @@ int apfs_read_free_queue_key(void *raw, int size, struct apfs_key *key)
*/
int apfs_read_omap_key(void *raw, int size, struct apfs_key *key)
{
if (size < sizeof(struct apfs_omap_key)) {
struct apfs_omap_key *raw_key;
if (size != sizeof(*raw_key)) {
apfs_err(NULL, "bad key length (%d)", size);
return -EFSCORRUPTED;
}
raw_key = raw;
key->id = le64_to_cpu(((struct apfs_omap_key *)raw)->ok_oid);
key->id = le64_to_cpu(raw_key->ok_oid);
key->type = 0;
key->number = le64_to_cpu(((struct apfs_omap_key *)raw)->ok_xid);
key->number = le64_to_cpu(raw_key->ok_xid);
key->name = NULL;
return 0;
libzbitmap.c
-2
View File
@@ -12,8 +12,6 @@
#include <linux/string.h>
#include "libzbitmap.h"
#define MIN(x, y) ((x) > (y) ? (y) : (x))
#define ZBM_MAGIC "ZBM\x09"
#define ZBM_MAGIC_SZ 4
lzfse/lzfse.h
+25 -94
View File
@@ -1,23 +1,24 @@
/* SPDX-License-Identifier: BSD-3-Clause */
/*
Copyright (c) 2015-2016, Apple Inc. All rights reserved.
Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer
in the documentation and/or other materials provided with the distribution.
3. Neither the name of the copyright holder(s) nor the names of any contributors may be used to endorse or promote products derived
from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
* Copyright (c) 2015-2016, Apple Inc. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
*
* 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
*
* 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer
* in the documentation and/or other materials provided with the distribution.
*
* 3. Neither the name of the copyright holder(s) nor the names of any contributors may be used to endorse or promote products derived
* from this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
* LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
* COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
* (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#ifndef LZFSE_H
#define LZFSE_H
@@ -25,71 +26,6 @@ ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSI
#include <linux/stddef.h>
#include <linux/types.h>
#ifdef __cplusplus
extern "C" {
#endif
#if defined(_MSC_VER) && !defined(__clang__)
# define __attribute__(X)
# pragma warning(disable : 4068)
#endif
#if defined(LZFSE_DLL)
# if defined(_WIN32) || defined(__CYGWIN__)
# if defined(LZFSE_DLL_EXPORTS)
# define LZFSE_API __declspec(dllexport)
# else
# define LZFSE_API __declspec(dllimport)
# endif
# endif
#endif
#if !defined(LZFSE_API)
# if __GNUC__ >= 4
# define LZFSE_API __attribute__((visibility("default")))
# else
# define LZFSE_API
# endif
#endif
/*! @abstract Get the required scratch buffer size to compress using LZFSE. */
size_t lzfse_encode_scratch_size(void);
/*! @abstract Compress a buffer using LZFSE.
*
* @param dst_buffer
* Pointer to the first byte of the destination buffer.
*
* @param dst_size
* Size of the destination buffer in bytes.
*
* @param src_buffer
* Pointer to the first byte of the source buffer.
*
* @param src_size
* Size of the source buffer in bytes.
*
* @param scratch_buffer
* If non-NULL, a pointer to scratch space for the routine to use as workspace;
* the routine may use up to lzfse_encode_scratch_size( ) bytes of workspace
* during its operation, and will not perform any internal allocations. If
* NULL, the routine may allocate its own memory to use during operation via
* a single call to malloc( ), and will release it by calling free( ) prior
* to returning. For most use, passing NULL is perfectly satisfactory, but if
* you require strict control over allocation, you will want to pass an
* explicit scratch buffer.
*
* @return
* The number of bytes written to the destination buffer if the input is
* successfully compressed. If the input cannot be compressed to fit into
* the provided buffer, or an error occurs, zero is returned, and the
* contents of dst_buffer are unspecified. */
size_t lzfse_encode_buffer(uint8_t *__restrict dst_buffer,
size_t dst_size,
const uint8_t *__restrict src_buffer,
size_t src_size,
void *__restrict scratch_buffer);
/*! @abstract Get the required scratch buffer size to decompress using LZFSE. */
size_t lzfse_decode_scratch_size(void);
@@ -122,15 +58,10 @@ size_t lzfse_decode_scratch_size(void);
* successfully decompressed. If there is not enough space in the destination
* buffer to hold the entire expanded output, only the first dst_size bytes
* will be written to the buffer and dst_size is returned. Note that this
* behavior differs from that of lzfse_encode_buffer. */
size_t lzfse_decode_buffer(uint8_t *__restrict dst_buffer,
size_t dst_size,
const uint8_t *__restrict src_buffer,
size_t src_size,
void *__restrict scratch_buffer);
#ifdef __cplusplus
} /* extern "C" */
#endif
* behavior differs from that of lzfse_encode_buffer.
*/
size_t lzfse_decode_buffer(uint8_t *__restrict dst_buffer, size_t dst_size,
const uint8_t *__restrict src_buffer, size_t src_size,
void *__restrict scratch_buffer);
#endif /* LZFSE_H */
lzfse/lzfse_decode.c
+65 -59
View File
@@ -1,74 +1,80 @@
// SPDX-License-Identifier: BSD-3-Clause
/*
Copyright (c) 2015-2016, Apple Inc. All rights reserved.
* Copyright (c) 2015-2016, Apple Inc. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
*
* 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
*
* 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer
* in the documentation and/or other materials provided with the distribution.
*
* 3. Neither the name of the copyright holder(s) nor the names of any contributors may be used to endorse or promote products derived
* from this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
* LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
* COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
* (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer
in the documentation and/or other materials provided with the distribution.
3. Neither the name of the copyright holder(s) nor the names of any contributors may be used to endorse or promote products derived
from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
// LZFSE decode API
/* LZFSE decode API */
#include <linux/slab.h>
#include "lzfse.h"
#include "lzfse_internal.h"
size_t lzfse_decode_scratch_size(void) { return sizeof(lzfse_decoder_state); }
size_t lzfse_decode_scratch_size(void)
{
return sizeof(lzfse_decoder_state);
}
static size_t lzfse_decode_buffer_with_scratch(uint8_t *__restrict dst_buffer,
size_t dst_size, const uint8_t *__restrict src_buffer,
size_t src_size, void *__restrict scratch_buffer) {
int status;
lzfse_decoder_state *s = (lzfse_decoder_state *)scratch_buffer;
memset(s, 0x00, sizeof(*s));
static size_t lzfse_decode_buffer_with_scratch(uint8_t *__restrict dst_buffer, size_t dst_size,
const uint8_t *__restrict src_buffer,
size_t src_size, void *__restrict scratch_buffer)
{
int status;
lzfse_decoder_state *s = (lzfse_decoder_state *)scratch_buffer;
// Initialize state
s->src = src_buffer;
s->src_begin = src_buffer;
s->src_end = s->src + src_size;
s->dst = dst_buffer;
s->dst_begin = dst_buffer;
s->dst_end = dst_buffer + dst_size;
memset(s, 0x00, sizeof(*s));
// Decode
status = lzfse_decode(s);
if (status == LZFSE_STATUS_DST_FULL)
return dst_size;
if (status != LZFSE_STATUS_OK)
return 0; // failed
return (size_t)(s->dst - dst_buffer); // bytes written
/* Initialize state */
s->src = src_buffer;
s->src_begin = src_buffer;
s->src_end = s->src + src_size;
s->dst = dst_buffer;
s->dst_begin = dst_buffer;
s->dst_end = dst_buffer + dst_size;
/* Decode */
status = lzfse_decode(s);
if (status == LZFSE_STATUS_DST_FULL)
return dst_size;
if (status != LZFSE_STATUS_OK)
return 0; /* failed */
return (size_t)(s->dst - dst_buffer); /* bytes written */
}
size_t lzfse_decode_buffer(uint8_t *__restrict dst_buffer, size_t dst_size,
const uint8_t *__restrict src_buffer,
size_t src_size, void *__restrict scratch_buffer) {
int has_malloc = 0;
size_t ret = 0;
const uint8_t *__restrict src_buffer, size_t src_size,
void *__restrict scratch_buffer)
{
int has_malloc = 0;
size_t ret = 0;
// Deal with the possible NULL pointer
if (scratch_buffer == NULL) {
// +1 in case scratch size could be zero
scratch_buffer = kmalloc(lzfse_decode_scratch_size() + 1, GFP_KERNEL);
has_malloc = 1;
}
if (scratch_buffer == NULL)
return 0;
ret = lzfse_decode_buffer_with_scratch(dst_buffer,
dst_size, src_buffer,
src_size, scratch_buffer);
if (has_malloc)
kfree(scratch_buffer);
return ret;
/* Deal with the possible NULL pointer */
if (scratch_buffer == NULL) {
/* +1 in case scratch size could be zero */
scratch_buffer = kmalloc(lzfse_decode_scratch_size() + 1, GFP_KERNEL);
has_malloc = 1;
}
if (scratch_buffer == NULL)
return 0;
ret = lzfse_decode_buffer_with_scratch(dst_buffer, dst_size, src_buffer, src_size,
scratch_buffer);
if (has_malloc)
kfree(scratch_buffer);
return ret;
}
lzfse/lzfse_decode_base.c
+647 -556
View File
File diff suppressed because it is too large. Load diff
lzfse/lzfse_encode.c
-163
View File
@@ -1,163 +0,0 @@
/*
Copyright (c) 2015-2016, Apple Inc. All rights reserved.
Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer
in the documentation and/or other materials provided with the distribution.
3. Neither the name of the copyright holder(s) nor the names of any contributors may be used to endorse or promote products derived
from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
// LZFSE encode API
#include "lzfse.h"
#include "lzfse_internal.h"
size_t lzfse_encode_scratch_size() {
size_t s1 = sizeof(lzfse_encoder_state);
size_t s2 = lzvn_encode_scratch_size();
return (s1 > s2) ? s1 : s2; // max(lzfse,lzvn)
}
size_t lzfse_encode_buffer_with_scratch(uint8_t *__restrict dst_buffer,
size_t dst_size, const uint8_t *__restrict src_buffer,
size_t src_size, void *__restrict scratch_buffer) {
const size_t original_size = src_size;
// If input is really really small, go directly to uncompressed buffer
// (because LZVN will refuse to encode it, and we will report a failure)
if (src_size < LZVN_ENCODE_MIN_SRC_SIZE)
goto try_uncompressed;
// If input is too small, try encoding with LZVN
if (src_size < LZFSE_ENCODE_LZVN_THRESHOLD) {
// need header + end-of-stream marker
size_t extra_size = 4 + sizeof(lzvn_compressed_block_header);
if (dst_size <= extra_size)
goto try_uncompressed; // DST is really too small, give up
size_t sz = lzvn_encode_buffer(
dst_buffer + sizeof(lzvn_compressed_block_header),
dst_size - extra_size, src_buffer, src_size, scratch_buffer);
if (sz == 0 || sz >= src_size)
goto try_uncompressed; // failed, or no compression, fall back to
// uncompressed block
// If we could encode, setup header and end-of-stream marker (we left room
// for them, no need to test)
lzvn_compressed_block_header header;
header.magic = LZFSE_COMPRESSEDLZVN_BLOCK_MAGIC;
header.n_raw_bytes = (uint32_t)src_size;
header.n_payload_bytes = (uint32_t)sz;
memcpy(dst_buffer, &header, sizeof(header));
store4(dst_buffer + sizeof(lzvn_compressed_block_header) + sz,
LZFSE_ENDOFSTREAM_BLOCK_MAGIC);
return sz + extra_size;
}
// Try encoding with LZFSE
{
lzfse_encoder_state *state = scratch_buffer;
memset(state, 0x00, sizeof *state);
if (lzfse_encode_init(state) != LZFSE_STATUS_OK)
goto try_uncompressed;
state->dst = dst_buffer;
state->dst_begin = dst_buffer;
state->dst_end = &dst_buffer[dst_size];
state->src = src_buffer;
state->src_encode_i = 0;
if (src_size >= 0xffffffffU) {
// lzfse only uses 32 bits for offsets internally, so if the input
// buffer is really huge, we need to process it in smaller chunks.
// Note that we switch over to this path for sizes much smaller
// 2GB because it's actually faster to change algorithms well before
// it's necessary for correctness.
// The first chunk, we just process normally.
const lzfse_offset encoder_block_size = 262144;
state->src_end = encoder_block_size;
if (lzfse_encode_base(state) != LZFSE_STATUS_OK)
goto try_uncompressed;
src_size -= encoder_block_size;
while (src_size >= encoder_block_size) {
// All subsequent chunks require a translation to keep the offsets
// from getting too big. Note that we are always going from
// encoder_block_size up to 2*encoder_block_size so that the
// offsets remain positive (as opposed to resetting to zero and
// having negative offsets).
state->src_end = 2 * encoder_block_size;
if (lzfse_encode_base(state) != LZFSE_STATUS_OK)
goto try_uncompressed;
lzfse_encode_translate(state, encoder_block_size);
src_size -= encoder_block_size;
}
// Set the end for the final chunk.
state->src_end = encoder_block_size + (lzfse_offset)src_size;
}
// If the source buffer is small enough to use 32-bit offsets, we simply
// encode the whole thing in a single chunk.
else
state->src_end = (lzfse_offset)src_size;
// This is either the trailing chunk (if the source file is huge), or
// the whole source file.
if (lzfse_encode_base(state) != LZFSE_STATUS_OK)
goto try_uncompressed;
if (lzfse_encode_finish(state) != LZFSE_STATUS_OK)
goto try_uncompressed;
// No error occured, return compressed size.
return state->dst - dst_buffer;
}
try_uncompressed:
// Compression failed for some reason. If we can fit the data into the
// output buffer uncompressed, go ahead and do that instead.
if (original_size + 12 <= dst_size && original_size < INT32_MAX) {
uncompressed_block_header header = {.magic = LZFSE_UNCOMPRESSED_BLOCK_MAGIC,
.n_raw_bytes = (uint32_t)src_size};
uint8_t *dst_end = dst_buffer;
memcpy(dst_end, &header, sizeof header);
dst_end += sizeof header;
memcpy(dst_end, src_buffer, original_size);
dst_end += original_size;
store4(dst_end, LZFSE_ENDOFSTREAM_BLOCK_MAGIC);
dst_end += 4;
return dst_end - dst_buffer;
}
// Otherwise, there's nothing we can do, so return zero.
return 0;
}
size_t lzfse_encode_buffer(uint8_t *__restrict dst_buffer, size_t dst_size,
const uint8_t *__restrict src_buffer,
size_t src_size, void *__restrict scratch_buffer) {
int has_malloc = 0;
size_t ret = 0;
// Deal with the possible NULL pointer
if (scratch_buffer == NULL) {
// +1 in case scratch size could be zero
scratch_buffer = malloc(lzfse_encode_scratch_size() + 1);
has_malloc = 1;
}
if (scratch_buffer == NULL)
return 0;
ret = lzfse_encode_buffer_with_scratch(dst_buffer,
dst_size, src_buffer,
src_size, scratch_buffer);
if (has_malloc)
free(scratch_buffer);
return ret;
}
lzfse/lzfse_encode_base.c
-826
View File
File diff suppressed because it is too large. Load diff
lzfse/lzfse_encode_tables.h
-218
View File
@@ -1,218 +0,0 @@
/*
Copyright (c) 2015-2016, Apple Inc. All rights reserved.
Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer
in the documentation and/or other materials provided with the distribution.
3. Neither the name of the copyright holder(s) nor the names of any contributors may be used to endorse or promote products derived
from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#ifndef LZFSE_ENCODE_TABLES_H
#define LZFSE_ENCODE_TABLES_H
#if defined(_MSC_VER) && !defined(__clang__)
# define inline __inline
#endif
static inline uint8_t l_base_from_value(int32_t value) {
static const uint8_t sym[LZFSE_ENCODE_MAX_L_VALUE + 1] = {
0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 16,
16, 16, 17, 17, 17, 17, 17, 17, 17, 17, 18, 18, 18, 18, 18, 18, 18, 18,
18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18,
18, 18, 18, 18, 18, 18, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19};
return sym[value];
}
static inline uint8_t m_base_from_value(int32_t value) {
static const uint8_t sym[LZFSE_ENCODE_MAX_M_VALUE + 1] = {
0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 16,
16, 16, 16, 16, 16, 16, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17,
17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17,
17, 17, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18,
18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18,
18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18,
18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18,
18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18,
18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18,
18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18,
18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18,
18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18,
18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18,
18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18,
18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18,
18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18,
18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18,
18, 18, 18, 18, 18, 18, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19};
return sym[value];
}
static inline uint8_t d_base_from_value(int32_t value) {
static const uint8_t sym[64 * 4] = {
0, 1, 2, 3, 4, 4, 5, 5, 6, 6, 7, 7, 8, 8, 8, 8, 9, 9,
9, 9, 10, 10, 10, 10, 11, 11, 11, 11, 12, 12, 12, 12, 12, 12, 12, 12,
13, 13, 13, 13, 13, 13, 13, 13, 14, 14, 14, 14, 14, 14, 14, 14, 15, 15,
15, 15, 15, 15, 15, 15, 16, 16, 16, 16, 16, 17, 18, 19, 20, 20, 21, 21,
22, 22, 23, 23, 24, 24, 24, 24, 25, 25, 25, 25, 26, 26, 26, 26, 27, 27,
27, 27, 28, 28, 28, 28, 28, 28, 28, 28, 29, 29, 29, 29, 29, 29, 29, 29,
30, 30, 30, 30, 30, 30, 30, 30, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32,
32, 32, 32, 33, 34, 35, 36, 36, 37, 37, 38, 38, 39, 39, 40, 40, 40, 40,
41, 41, 41, 41, 42, 42, 42, 42, 43, 43, 43, 43, 44, 44, 44, 44, 44, 44,
44, 44, 45, 45, 45, 45, 45, 45, 45, 45, 46, 46, 46, 46, 46, 46, 46, 46,
47, 47, 47, 47, 47, 47, 47, 47, 48, 48, 48, 48, 48, 49, 50, 51, 52, 52,
53, 53, 54, 54, 55, 55, 56, 56, 56, 56, 57, 57, 57, 57, 58, 58, 58, 58,
59, 59, 59, 59, 60, 60, 60, 60, 60, 60, 60, 60, 61, 61, 61, 61, 61, 61,
61, 61, 62, 62, 62, 62, 62, 62, 62, 62, 63, 63, 63, 63, 63, 63, 63, 63,
0, 0, 0, 0};
int index = 0;
int in_range_k;
in_range_k = (value >= 0 && value < 60);
index |= (((value - 0) >> 0) + 0) & -in_range_k;
in_range_k = (value >= 60 && value < 1020);
index |= (((value - 60) >> 4) + 64) & -in_range_k;
in_range_k = (value >= 1020 && value < 16380);
index |= (((value - 1020) >> 8) + 128) & -in_range_k;
in_range_k = (value >= 16380 && value < 262140);
index |= (((value - 16380) >> 12) + 192) & -in_range_k;
return sym[index & 255];
}
#endif // LZFSE_ENCODE_TABLES_H
lzfse/lzfse_fse.c
+112 -203
View File
@@ -1,217 +1,126 @@
// SPDX-License-Identifier: BSD-3-Clause
/*
Copyright (c) 2015-2016, Apple Inc. All rights reserved.
Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer
in the documentation and/or other materials provided with the distribution.
3. Neither the name of the copyright holder(s) nor the names of any contributors may be used to endorse or promote products derived
from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
* Copyright (c) 2015-2016, Apple Inc. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
*
* 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
*
* 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer
* in the documentation and/or other materials provided with the distribution.
*
* 3. Neither the name of the copyright holder(s) nor the names of any contributors may be used to endorse or promote products derived
* from this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
* LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
* COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
* (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#include "lzfse_internal.h"
// Initialize encoder table T[NSYMBOLS].
// NSTATES = sum FREQ[i] is the number of states (a power of 2)
// NSYMBOLS is the number of symbols.
// FREQ[NSYMBOLS] is a normalized histogram of symbol frequencies, with FREQ[i]
// >= 0.
// Some symbols may have a 0 frequency. In that case, they should not be
// present in the data.
void fse_init_encoder_table(int nstates, int nsymbols,
const uint16_t *__restrict freq,
fse_encoder_entry *__restrict t) {
int offset = 0; // current offset
int n_clz = __builtin_clz(nstates);
int i;
for (i = 0; i < nsymbols; i++) {
int f = (int)freq[i];
int k;
if (f == 0)
continue; // skip this symbol, no occurrences
k = __builtin_clz(f) - n_clz; // shift needed to ensure N <= (F<<K) < 2*N
t[i].s0 = (int16_t)((f << k) - nstates);
t[i].k = (int16_t)k;
t[i].delta0 = (int16_t)(offset - f + (nstates >> k));
t[i].delta1 = (int16_t)(offset - f + (nstates >> (k - 1)));
offset += f;
}
/*
* Initialize decoder table T[NSTATES].
* NSTATES = sum FREQ[i] is the number of states (a power of 2)
* NSYMBOLS is the number of symbols.
* FREQ[NSYMBOLS] is a normalized histogram of symbol frequencies, with FREQ[i]
* >= 0.
* Some symbols may have a 0 frequency. In that case, they should not be
* present in the data.
*/
int fse_init_decoder_table(int nstates, int nsymbols, const uint16_t *__restrict freq,
int32_t *__restrict t)
{
int n_clz = __builtin_clz(nstates);
int sum_of_freq = 0;
int i, j0, j;
for (i = 0; i < nsymbols; i++) {
int f = (int)freq[i];
int k;
if (f == 0)
continue; /* skip this symbol, no occurrences */
sum_of_freq += f;
if (sum_of_freq > nstates)
return -1;
k = __builtin_clz(f) - n_clz; /* shift needed to ensure N <= (F<<K) < 2*N */
j0 = ((2 * nstates) >> k) - f;
/* Initialize all states S reached by this symbol: OFFSET <= S < OFFSET + F */
for (j = 0; j < f; j++) {
fse_decoder_entry e;
e.symbol = (uint8_t)i;
if (j < j0) {
e.k = (int8_t)k;
e.delta = (int16_t)(((f + j) << k) - nstates);
} else {
e.k = (int8_t)(k - 1);
e.delta = (int16_t)((j - j0) << (k - 1));
}
memcpy(t, &e, sizeof(e));
t++;
}
}
return 0; /* OK */
}
// Initialize decoder table T[NSTATES].
// NSTATES = sum FREQ[i] is the number of states (a power of 2)
// NSYMBOLS is the number of symbols.
// FREQ[NSYMBOLS] is a normalized histogram of symbol frequencies, with FREQ[i]
// >= 0.
// Some symbols may have a 0 frequency. In that case, they should not be
// present in the data.
int fse_init_decoder_table(int nstates, int nsymbols,
const uint16_t *__restrict freq,
int32_t *__restrict t) {
int n_clz = __builtin_clz(nstates);
int sum_of_freq = 0;
int i, j0, j;
for (i = 0; i < nsymbols; i++) {
int f = (int)freq[i];
int k;
if (f == 0)
continue; // skip this symbol, no occurrences
/*
* Initialize value decoder table T[NSTATES].
* NSTATES = sum FREQ[i] is the number of states (a power of 2)
* NSYMBOLS is the number of symbols.
* FREQ[NSYMBOLS] is a normalized histogram of symbol frequencies, with FREQ[i]
* >= 0.
* SYMBOL_VBITS[NSYMBOLS] and SYMBOLS_VBASE[NSYMBOLS] are the number of value
* bits to read and the base value for each symbol.
* Some symbols may have a 0 frequency. In that case, they should not be
* present in the data.
*/
void fse_init_value_decoder_table(int nstates, int nsymbols, const uint16_t *__restrict freq,
const uint8_t *__restrict symbol_vbits,
const int32_t *__restrict symbol_vbase,
struct fse_value_decoder_entry *__restrict t)
{
int n_clz = __builtin_clz(nstates);
int i;
sum_of_freq += f;
for (i = 0; i < nsymbols; i++) {
struct fse_value_decoder_entry ei = { 0 };
int f = (int)freq[i];
int k, j0, j;
if (sum_of_freq > nstates) {
return -1;
}
if (f == 0)
continue; /* skip this symbol, no occurrences */
k = __builtin_clz(f) - n_clz; // shift needed to ensure N <= (F<<K) < 2*N
j0 = ((2 * nstates) >> k) - f;
k = __builtin_clz(f) - n_clz; /* shift needed to ensure N <= (F<<K) < 2*N */
j0 = ((2 * nstates) >> k) - f;
// Initialize all states S reached by this symbol: OFFSET <= S < OFFSET + F
for (j = 0; j < f; j++) {
fse_decoder_entry e;
ei.value_bits = symbol_vbits[i];
ei.vbase = symbol_vbase[i];
e.symbol = (uint8_t)i;
if (j < j0) {
e.k = (int8_t)k;
e.delta = (int16_t)(((f + j) << k) - nstates);
} else {
e.k = (int8_t)(k - 1);
e.delta = (int16_t)((j - j0) << (k - 1));
}
/* Initialize all states S reached by this symbol: OFFSET <= S < OFFSET + F */
for (j = 0; j < f; j++) {
struct fse_value_decoder_entry e = ei;
memcpy(t, &e, sizeof(e));
t++;
}
}
if (j < j0) {
e.total_bits = (uint8_t)k + e.value_bits;
e.delta = (int16_t)(((f + j) << k) - nstates);
} else {
e.total_bits = (uint8_t)(k - 1) + e.value_bits;
e.delta = (int16_t)((j - j0) << (k - 1));
}
return 0; // OK
}
// Initialize value decoder table T[NSTATES].
// NSTATES = sum FREQ[i] is the number of states (a power of 2)
// NSYMBOLS is the number of symbols.
// FREQ[NSYMBOLS] is a normalized histogram of symbol frequencies, with FREQ[i]
// >= 0.
// SYMBOL_VBITS[NSYMBOLS] and SYMBOLS_VBASE[NSYMBOLS] are the number of value
// bits to read and the base value for each symbol.
// Some symbols may have a 0 frequency. In that case, they should not be
// present in the data.
void fse_init_value_decoder_table(int nstates, int nsymbols,
const uint16_t *__restrict freq,
const uint8_t *__restrict symbol_vbits,
const int32_t *__restrict symbol_vbase,
fse_value_decoder_entry *__restrict t) {
int n_clz = __builtin_clz(nstates);
int i;
for (i = 0; i < nsymbols; i++) {
fse_value_decoder_entry ei = {0};
int f = (int)freq[i];
int k, j0, j;
if (f == 0)
continue; // skip this symbol, no occurrences
k = __builtin_clz(f) - n_clz; // shift needed to ensure N <= (F<<K) < 2*N
j0 = ((2 * nstates) >> k) - f;
ei.value_bits = symbol_vbits[i];
ei.vbase = symbol_vbase[i];
// Initialize all states S reached by this symbol: OFFSET <= S < OFFSET + F
for (j = 0; j < f; j++) {
fse_value_decoder_entry e = ei;
if (j < j0) {
e.total_bits = (uint8_t)k + e.value_bits;
e.delta = (int16_t)(((f + j) << k) - nstates);
} else {
e.total_bits = (uint8_t)(k - 1) + e.value_bits;
e.delta = (int16_t)((j - j0) << (k - 1));
}
memcpy(t, &e, 8);
t++;
}
}
}
// Remove states from symbols until the correct number of states is used.
static void fse_adjust_freqs(uint16_t *freq, int overrun, int nsymbols) {
int shift;
for (shift = 3; overrun != 0; shift--) {
int sym;
for (sym = 0; sym < nsymbols; sym++) {
if (freq[sym] > 1) {
int n = (freq[sym] - 1) >> shift;
if (n > overrun)
n = overrun;
freq[sym] -= n;
overrun -= n;
if (overrun == 0)
break;
}
}
}
}
// Normalize a table T[NSYMBOLS] of occurrences to FREQ[NSYMBOLS].
void fse_normalize_freq(int nstates, int nsymbols, const uint32_t *__restrict t,
uint16_t *__restrict freq) {
uint32_t s_count = 0;
int remaining = nstates; // must be signed; this may become < 0
int max_freq = 0;
int max_freq_sym = 0;
int shift = __builtin_clz(nstates) - 1;
uint32_t highprec_step;
int i;
// Compute the total number of symbol occurrences
for (i = 0; i < nsymbols; i++)
s_count += t[i];
if (s_count == 0)
highprec_step = 0; // no symbols used
else
highprec_step = ((uint32_t)1 << 31) / s_count;
for (i = 0; i < nsymbols; i++) {
// Rescale the occurrence count to get the normalized frequency.
// Round up if the fractional part is >= 0.5; otherwise round down.
// For efficiency, we do this calculation using integer arithmetic.
int f = (((t[i] * highprec_step) >> shift) + 1) >> 1;
// If a symbol was used, it must be given a nonzero normalized frequency.
if (f == 0 && t[i] != 0)
f = 1;
freq[i] = f;
remaining -= f;
// Remember the maximum frequency and which symbol had it.
if (f > max_freq) {
max_freq = f;
max_freq_sym = i;
}
}
// If there remain states to be assigned, then just assign them to the most
// frequent symbol. Alternatively, if we assigned more states than were
// actually available, then either remove states from the most frequent symbol
// (for minor overruns) or use the slower adjustment algorithm (for major
// overruns).
if (-remaining < (max_freq >> 2)) {
freq[max_freq_sym] += remaining;
} else {
fse_adjust_freqs(freq, -remaining, nsymbols);
}
memcpy(t, &e, 8);
t++;
}
}
}
Loaded 20 of 36 files, more files were not shown because too many files have changed in this diff. Show more