Pull USB patches from Greg KH:
"Here's the big USB pull request for 3.15-rc1.
The normal set of patches, lots of controller driver updates, and a
smattering of individual USB driver updates as well.
All have been in linux-next for a while"
* tag 'usb-3.15-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/usb: (249 commits)
xhci: Transition maintainership to Mathias Nyman.
USB: disable reset-resume when USB_QUIRK_RESET is set
USB: unbind all interfaces before rebinding any
usb: phy: Add ulpi IDs for SMSC USB3320 and TI TUSB1210
usb: gadget: tcm_usb_gadget: stop format strings
usb: gadget: f_fs: add missing spinlock and mutex unlock
usb: gadget: composite: switch over to ERR_CAST()
usb: gadget: inode: switch over to memdup_user()
usb: gadget: f_subset: switch over to PTR_RET
usb: gadget: lpc32xx_udc: fix wrong clk_put() sequence
USB: keyspan: remove dead debugging code
USB: serial: add missing newlines to dev_<level> messages.
USB: serial: add missing braces
USB: serial: continue to write on errors
USB: serial: continue to read on errors
USB: serial: make bulk_out_size a lower limit
USB: cypress_m8: fix potential scheduling while atomic
devicetree: bindings: document lsi,zevio-usb
usb: chipidea: add support for USB OTG controller on LSI Zevio SoCs
usb: chipidea: imx: Use dev_name() for ci_hdrc name to distinguish USBs
...
Pull char/misc driver patches from Greg KH:
"Here's the big char/misc driver updates for 3.15-rc1.
Lots of various things here, including the new mcb driver subsystem.
All of these have been in linux-next for a while"
* tag 'char-misc-3.15-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/char-misc: (118 commits)
extcon: Move OF helper function to extcon core and change function name
extcon: of: Remove unnecessary function call by using the name of device_node
extcon: gpio: Use SIMPLE_DEV_PM_OPS macro
extcon: palmas: Use SIMPLE_DEV_PM_OPS macro
mei: don't use deprecated DEFINE_PCI_DEVICE_TABLE macro
mei: amthif: fix checkpatch error
mei: client.h fix checkpatch errors
mei: use cl_dbg where appropriate
mei: fix Unnecessary space after function pointer name
mei: report consistently copy_from/to_user failures
mei: drop pr_fmt macros
mei: make me hw headers private to me hw.
mei: fix memory leak of pending write cb objects
mei: me: do not reset when less than expected data is received
drivers: mcb: Fix build error discovered by 0-day bot
cs5535-mfgpt: Simplify dependencies
spmi: pm: drop bus-level PM suspend/resume routines
spmi: pmic_arb: make selectable on ARCH_QCOM
Drivers: hv: vmbus: Increase the limit on the number of pfns we can handle
pch_phub: Report error writing MAC back to user
...
Felipe writes:
usb: patches for v3.15
another substantial pull request with new features all over
the place.
dwc3 got a bit closer towards hibernation support with after
a few patches re-factoring code to be reused for hibernation.
Also in dwc3 two new workarounds for known silicon bugs have
been implemented, some randconfig build errors have been fixed,
and it was taught about the new generic phy layer.
MUSB on AM335x now supports isochronous transfers thanks to
George Cherian's work.
The atmel_usba driver got two crash fixes: one when no endpoint
was specified in DeviceTree data and another when stopping the UDC
in DEBUG builds.
Function FS got a much needed fix to ffs_epfile_io() which was
copying too much data to userspace in some cases.
The printer gadget got a fix for a possible deadlock and plugged
a memory leak.
Ethernet drivers now use NAPI for RX which gives improved throughput.
Other than that, the usual miscelaneous fixes, cleanups, and
the like.
Signed-of-by: Felipe Balbi <balbi@ti.com>
This reworks the way SuperSpeed descriptors are added and instead of
having a magic after full and high speed descriptors, it reworks the
whole descriptors block to include a flags field which lists which
descriptors are present and makes future extensions possible.
Signed-off-by: Michal Nazarewicz <mina86@mina86.com>
Signed-off-by: Felipe Balbi <balbi@ti.com>
Allow userspace to pass SuperSpeed descriptors and
handle them in the driver accordingly.
This change doesn't modify existing desc_header and thereby
keeps the ABI changes backward compatible i.e. existing
userspace drivers compiled with old header (functionfs.h)
would continue to work with the updated kernel.
Signed-off-by: Manu Gautam <mgautam@codeaurora.org>
Acked-by: Michal Nazarewicz <mina86@mina86.com>
Signed-off-by: Felipe Balbi <balbi@ti.com>
This allows userspace to use bulk-streams, just like in kernel drivers, see
Documentation/usb/bulk-streams.txt for details on the in kernel API. This
is exported pretty much one on one to userspace.
To use streams an app must first make a USBDEVFS_ALLOC_STREAMS ioctl,
on success this will return the number of streams available (which may be
less then requested). If there are n streams the app can then submit
usbdevfs_urb-s with their stream_id member set to 1-n to use a specific
stream. IE if USBDEVFS_ALLOC_STREAMS returns 4 then stream_id 1-4 can be
used.
When the app is done using streams it should call USBDEVFS_FREE_STREAMS
Note applications are advised to use libusb rather then using the
usbdevfs api directly. The latest version of libusb has support for streams.
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
This patch makes it possible to specify a bulk stream id when submitting
an urb using the async usbfs API. It overloads the number_of_packets
usbdevfs_urb field for this. This is not pretty, but given other
constraints it is the best we can do. The reasoning leading to this goes
as follows:
1) We want to support bulk streams in the usbfs API
2) We do not want to extend the usbdevfs_urb struct with a new member, as
that would mean defining new ioctl numbers for all async API ioctls +
adding compat versions for the old ones (times 2 for 32 bit support)
3) 1 + 2 means we need to re-use an existing field
4) number_of_packets is only used for isoc urbs, and streams are bulk only
so it is the best (and only) candidate for re-using
Note that:
1) This patch only uses number_of_packets as stream_id if the app has
actually allocated streams on the ep, so that old apps which may have
garbage in there (as it was unused until now in the bulk case), will not
break
2) This patch does not add support for allocating / freeing bulk-streams, that
is done in a follow up patch
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Add support for Dual/Quad SPI Transfers to the spidev API.
As this uses SPI mode bits that don't fit in a single byte, two new
ioctls (SPI_IOC_RD_MODE32 and SPI_IOC_WR_MODE32) are introduced.
Signed-off-by: Geert Uytterhoeven <geert+renesas@linux-m68k.org>
Signed-off-by: Mark Brown <broonie@linaro.org>
Implement the file copy service for Linux guests on Hyper-V. This permits the
host to copy a file (over VMBUS) into the guest. This facility is part of
"guest integration services" supported on the Windows platform.
Here is a link that provides additional details on this functionality:
http://technet.microsoft.com/en-us/library/dn464282.aspx
In V1 version of the patch I have addressed comments from
Olaf Hering <olaf@aepfle.de> and Dan Carpenter <dan.carpenter@oracle.com>
In V2 version of this patch I did some minor cleanup (making some globals
static). In V4 version of the patch I have addressed all of Olaf's
most recent set of comments/concerns.
In V5 version of the patch I had addressed Greg's most recent comments.
I would like to thank Greg for suggesting that I use misc device; it has
significantly simplified the code.
In V6 version of the patch I have cleaned up error message based on Olaf's
comments. I have also rebased the patch based on the current tip.
In this version of the patch, I have addressed the latest comments from Greg.
Signed-off-by: K. Y. Srinivasan <kys@microsoft.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Pull btrfs fixes from Chris Mason:
"We have a small collection of fixes in my for-linus branch.
The big thing that stands out is a revert of a new ioctl. Users
haven't shipped yet in btrfs-progs, and Dave Sterba found a better way
to export the information"
* 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/mason/linux-btrfs:
Btrfs: use right clone root offset for compressed extents
btrfs: fix null pointer deference at btrfs_sysfs_add_one+0x105
Btrfs: unset DCACHE_DISCONNECTED when mounting default subvol
Btrfs: fix max_inline mount option
Btrfs: fix a lockdep warning when cleaning up aborted transaction
Revert "btrfs: add ioctl to export size of global metadata reservation"
Pull char/misc fixes from Greg KH:
"Here are some small char/misc driver fixes, along with some
documentation updates, for 3.14-rc3. Nothing major, just a number of
fixes for reported issues"
* tag 'char-misc-3.14-rc3' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/char-misc:
Revert "misc: eeprom: sunxi: Add new compatibles"
Revert "ARM: sunxi: dt: Convert to the new SID compatibles"
misc: mic: fix possible signed underflow (undefined behavior) in userspace API
ARM: sunxi: dt: Convert to the new SID compatibles
misc: eeprom: sunxi: Add new compatibles
misc: genwqe: Fix potential memory leak when pinning memory
Documentation:Update Documentation/zh_CN/arm64/memory.txt
Documentation:Update Documentation/zh_CN/arm64/booting.txt
Documentation:Chinese translation of Documentation/arm64/tagged-pointers.txt
raw: set range for MAX_RAW_DEVS
raw: test against runtime value of max_raw_minors
Drivers: hv: vmbus: Don't timeout during the initial connection with host
Drivers: hv: vmbus: Specify the target CPU that should receive notification
VME: Correct read/write alignment algorithm
mei: don't unset read cb ptr on reset
mei: clear write cb from waiting list on reset
This reverts commit 01e219e806.
David Sterba found a different way to provide these features without adding a new
ioctl. We haven't released any progs with this ioctl yet, so I'm taking this out
for now until we finalize things.
Signed-off-by: Chris Mason <clm@fb.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
CC: Jeff Mahoney <jeffm@suse.com>
Pull networking updates from David Miller:
1) Fix flexcan build on big endian, from Arnd Bergmann
2) Correctly attach cpsw to GPIO bitbang MDIO drive, from Stefan Roese
3) udp_add_offload has to use GFP_ATOMIC since it can be invoked from
non-sleepable contexts. From Or Gerlitz
4) vxlan_gro_receive() does not iterate over all possible flows
properly, fix also from Or Gerlitz
5) CAN core doesn't use a proper SKB destructor when it hooks up
sockets to SKBs. Fix from Oliver Hartkopp
6) ip_tunnel_xmit() can use an uninitialized route pointer, fix from
Eric Dumazet
7) Fix address family assignment in IPVS, from Michal Kubecek
8) Fix ath9k build on ARM, from Sujith Manoharan
9) Make sure fail_over_mac only applies for the correct bonding modes,
from Ding Tianhong
10) The udp offload code doesn't use RCU correctly, from Shlomo Pongratz
11) Handle gigabit features properly in generic PHY code, from Florian
Fainelli
12) Don't blindly invoke link operations in
rtnl_link_get_slave_info_data_size, they are optional. Fix from
Fernando Luis Vazquez Cao
13) Add USB IDs for Netgear Aircard 340U, from Bjørn Mork
14) Handle netlink packet padding properly in openvswitch, from Thomas
Graf
15) Fix oops when deleting chains in nf_tables, from Patrick McHardy
16) Fix RX stalls in xen-netback driver, from Zoltan Kiss
17) Fix deadlock in mac80211 stack, from Emmanuel Grumbach
18) inet_nlmsg_size() forgets to consider ifa_cacheinfo, fix from Geert
Uytterhoeven
19) tg3_change_mtu() can deadlock, fix from Nithin Sujir
20) Fix regression in setting SCTP local source addresses on accepted
sockets, caused by some generic ipv6 socket changes. Fix from
Matija Glavinic Pecotic
21) IPPROTO_* must be pure defines, otherwise module aliases don't get
constructed properly. Fix from Jan Moskyto
22) IPV6 netconsole setup doesn't work properly unless an explicit
source address is specified, fix from Sabrina Dubroca
23) Use __GFP_NORETRY for high order skb page allocations in
sock_alloc_send_pskb and skb_page_frag_refill. From Eric Dumazet
24) Fix a regression added in netconsole over bridging, from Cong Wang
25) TCP uses an artificial offset of 1ms for SRTT, but this doesn't jive
well with TCP pacing which needs the SRTT to be accurate. Fix from
Eric Dumazet
26) Several cases of missing header file includes from Rashika Kheria
27) Add ZTE MF667 device ID to qmi_wwan driver, from Raymond Wanyoike
28) TCP Small Queues doesn't handle nonagle properly in some corner
cases, fix from Eric Dumazet
29) Remove extraneous read_unlock in bond_enslave, whoops. From Ding
Tianhong
30) Fix 9p trans_virtio handling of vmalloc buffers, from Richard Yao
* git://git.kernel.org/pub/scm/linux/kernel/git/davem/net: (136 commits)
6lowpan: fix lockdep splats
alx: add missing stats_lock spinlock init
9p/trans_virtio.c: Fix broken zero-copy on vmalloc() buffers
bonding: remove unwanted bond lock for enslave processing
USB2NET : SR9800 : One chip USB2.0 USB2NET SR9800 Device Driver Support
tcp: tsq: fix nonagle handling
bridge: Prevent possible race condition in br_fdb_change_mac_address
bridge: Properly check if local fdb entry can be deleted when deleting vlan
bridge: Properly check if local fdb entry can be deleted in br_fdb_delete_by_port
bridge: Properly check if local fdb entry can be deleted in br_fdb_change_mac_address
bridge: Fix the way to check if a local fdb entry can be deleted
bridge: Change local fdb entries whenever mac address of bridge device changes
bridge: Fix the way to find old local fdb entries in br_fdb_change_mac_address
bridge: Fix the way to insert new local fdb entries in br_fdb_changeaddr
bridge: Fix the way to find old local fdb entries in br_fdb_changeaddr
tcp: correct code comment stating 3 min timeout for FIN_WAIT2, we only do 1 min
net: vxge: Remove unused device pointer
net: qmi_wwan: add ZTE MF667
3c59x: Remove unused pointer in vortex_eisa_cleanup()
net: fix 'ip rule' iif/oif device rename
...
iovcnt is declared as a signed integer in both the userspace API and
as a local variable in mic_virtio.c. The while() loop in mic_virtio.c
iterates until the local variable iovcnt reaches the value 0. If
userspace passes e.g. INT_MIN as iovcnt field, this loop then appears
to depend on an undefined behavior (signed underflow) to complete.
The fix is to use unsigned integers in both the userspace API and
the local variable.
This issue was reported @ https://lkml.org/lkml/2014/1/10/10
Reported-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Reviewed-by: Ashutosh Dixit <ashutosh.dixit@intel.com>
Signed-off-by: Sudeep Dutt <sudeep.dutt@intel.com>
Cc: stable <stable@vger.kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
This patch adds the hyperv.h header to the uapi folder, and adds it to the Kbuild file.
Doing this enables compiling userspace Hyper-V tools using the installed headers.
Version 2: Split UAPI parts into new header, instead of duplicating.
Signed-off-by: Bjarke Istrup Pedersen <gurligebis@gentoo.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Commit cfd280c912 ("net: sync some IP headers with glibc") changed a set of
define's to an enum (with no explanation why) which introduced a bug
in module mip6 where aliases are generated using the IPPROTO_* defines;
mip6 doesn't load if require_module called with the aliases from
xfrm_get_type().
Reverting this change back to define's to fix the aliases.
modinfo mip6 (before this change)
alias: xfrm-type-10-IPPROTO_DSTOPTS
alias: xfrm-type-10-IPPROTO_ROUTING
modinfo mip6 (after this change)
alias: xfrm-type-10-43
alias: xfrm-type-10-60
Signed-off-by: Jan Moskyto Matejka <mq@suse.cz>
Signed-off-by: David S. Miller <davem@davemloft.net>
Pull NVMe driver update from Matthew Wilcox:
"Looks like I missed the merge window ... but these are almost all
bugfixes anyway (the ones that aren't have been baking for months)"
* git://git.infradead.org/users/willy/linux-nvme:
NVMe: Namespace use after free on surprise removal
NVMe: Correct uses of INIT_WORK
NVMe: Include device and queue numbers in interrupt name
NVMe: Add a pci_driver shutdown method
NVMe: Disable admin queue on init failure
NVMe: Dynamically allocate partition numbers
NVMe: Async IO queue deletion
NVMe: Surprise removal handling
NVMe: Abort timed out commands
NVMe: Schedule reset for failed controllers
NVMe: Device resume error handling
NVMe: Cache dev->pci_dev in a local pointer
NVMe: Fix lockdep warnings
NVMe: compat SG_IO ioctl
NVMe: remove deprecated IRQF_DISABLED
NVMe: Avoid shift operation when writing cq head doorbell
Pull media updates from Mauro Carvalho Chehab:
- a new jpeg codec driver for Samsung Exynos (jpeg-hw-exynos4)
- a new dvb frontend for ds2103 chipset (m88ds2103)
- a new sensor driver for Samsung S5K5BAF UXGA (s5k5baf)
- new drivers for R-Car VSP1
- a new radio driver: radio-raremono
- a new tuner driver for ts2022 chipset (m88ts2022)
- the analog part of em28xx is now a separate module that only
load/runs if the device is not a pure digital TV device
- added a staging driver for bcm2048 radio devices
- the omap 2 video driver (omap24xx) was moved to staging. This driver
is for an old hardware and uses a deprecated Kernel internal API. If
nobody cares enough to fix it, it would be removed on a couple Kernel
releases
- the sn9c102 driver was moved to staging. This driver was replaced by
gspca, and disabled on some distros, as almost all devices are known
to work properly with gspca. It should be removed from kernel on a
couple Kernel releases
- lots of driver fixes, improvements and cleanups
* 'v4l_for_linus' of git://git.kernel.org/pub/scm/linux/kernel/git/mchehab/linux-media: (421 commits)
[media] media: v4l2-dev: fix video device index assignment
[media] rc-core: reuse device numbers
[media] em28xx-cards: properly initialize the device bitmap
[media] Staging: media: Fix line length exceeding 80 characters in as102_drv.c
[media] Staging: media: Fix line length exceeding 80 characters in as102_fe.c
[media] Staging: media: Fix quoted string split across line in as102_fe.c
[media] media: st-rc: Add reset support
[media] m2m-deinterlace: fix allocated struct type
[media] radio-usb-si4713: fix sparse non static symbol warnings
[media] em28xx-audio: remove needless check before usb_free_coherent()
[media] au0828: Fix sparse non static symbol warning
Revert "[media] go7007-usb: only use go->dev after allocated"
[media] em28xx-audio: provide an error code when URB submit fails
[media] em28xx: fix check for audio only usb interfaces when changing the usb alternate setting
[media] em28xx: fix usb alternate setting for analog and digital video endpoints > 0
[media] em28xx: make 'em28xx_ctrl_ops' static
em28xx-alsa: Fix error patch for init/fini
[media] em28xx-audio: flush work at .fini
[media] drxk: remove the option to load firmware asynchronously
[media] em28xx: adjust period size at runtime
...
Pull btrfs updates from Chris Mason:
"This is a pretty big pull, and most of these changes have been
floating in btrfs-next for a long time. Filipe's properties work is a
cool building block for inheriting attributes like compression down on
a per inode basis.
Jeff Mahoney kicked in code to export filesystem info into sysfs.
Otherwise, lots of performance improvements, cleanups and bug fixes.
Looks like there are still a few other small pending incrementals, but
I wanted to get the bulk of this in first"
* 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/mason/linux-btrfs: (149 commits)
Btrfs: fix spin_unlock in check_ref_cleanup
Btrfs: setup inode location during btrfs_init_inode_locked
Btrfs: don't use ram_bytes for uncompressed inline items
Btrfs: fix btrfs_search_slot_for_read backwards iteration
Btrfs: do not export ulist functions
Btrfs: rework ulist with list+rb_tree
Btrfs: fix memory leaks on walking backrefs failure
Btrfs: fix send file hole detection leading to data corruption
Btrfs: add a reschedule point in btrfs_find_all_roots()
Btrfs: make send's file extent item search more efficient
Btrfs: fix to catch all errors when resolving indirect ref
Btrfs: fix protection between walking backrefs and root deletion
btrfs: fix warning while merging two adjacent extents
Btrfs: fix infinite path build loops in incremental send
btrfs: undo sysfs when open_ctree() fails
Btrfs: fix snprintf usage by send's gen_unique_name
btrfs: fix defrag 32-bit integer overflow
btrfs: sysfs: list the NO_HOLES feature
btrfs: sysfs: don't show reserved incompat feature
btrfs: call permission checks earlier in ioctls and return EPERM
...
Pull block IO driver changes from Jens Axboe:
- bcache update from Kent Overstreet.
- two bcache fixes from Nicholas Swenson.
- cciss pci init error fix from Andrew.
- underflow fix in the parallel IDE pg_write code from Dan Carpenter.
I'm sure the 1 (or 0) users of that are now happy.
- two PCI related fixes for sx8 from Jingoo Han.
- floppy init fix for first block read from Jiri Kosina.
- pktcdvd error return miss fix from Julia Lawall.
- removal of IRQF_SHARED from the SEGA Dreamcast CD-ROM code from
Michael Opdenacker.
- comment typo fix for the loop driver from Olaf Hering.
- potential oops fix for null_blk from Raghavendra K T.
- two fixes from Sam Bradshaw (Micron) for the mtip32xx driver, fixing
an OOM problem and a problem with handling security locked conditions
* 'for-3.14/drivers' of git://git.kernel.dk/linux-block: (47 commits)
mg_disk: Spelling s/finised/finished/
null_blk: Null pointer deference problem in alloc_page_buffers
mtip32xx: Correctly handle security locked condition
mtip32xx: Make SGL container per-command to eliminate high order dma allocation
drivers/block/loop.c: fix comment typo in loop_config_discard
drivers/block/cciss.c:cciss_init_one(): use proper errnos
drivers/block/paride/pg.c: underflow bug in pg_write()
drivers/block/sx8.c: remove unnecessary pci_set_drvdata()
drivers/block/sx8.c: use module_pci_driver()
floppy: bail out in open() if drive is not responding to block0 read
bcache: Fix auxiliary search trees for key size > cacheline size
bcache: Don't return -EINTR when insert finished
bcache: Improve bucket_prio() calculation
bcache: Add bch_bkey_equal_header()
bcache: update bch_bkey_try_merge
bcache: Move insert_fixup() to btree_keys_ops
bcache: Convert sorting to btree_keys
bcache: Convert debug code to btree_keys
bcache: Convert btree_iter to struct btree_keys
bcache: Refactor bset_tree sysfs stats
...
Pull more x32 uabi type fixes from Peter Anvin:
"Despite the branch name, **most of these changes are to generic
code**. They change types so that they make an increasing amount of
the exported uapi kernel headers usable for libc.
The ARM64 people are also interested in these changes for their ILP32
ABI"
* 'x86-x32-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
uapi: Use __kernel_long_t in struct mq_attr
uapi: Use __kernel_ulong_t in shmid64_ds/shminfo64/shm_info
x86, uapi, x32: Use __kernel_ulong_t in x86 struct semid64_ds
uapi: Use __kernel_ulong_t in struct msqid64_ds
uapi: Use __kernel_long_t in struct msgbuf
uapi, asm-generic: Use __kernel_ulong_t in uapi struct ipc64_perm
uapi: Use __kernel_long_t/__kernel_ulong_t in <linux/resource.h>
uapi: Use __kernel_long_t in struct timex
This change adds infrastructure to allow for generic properties for
inodes. Properties are name/value pairs that can be associated with
inodes for different purposes. They are stored as xattrs with the
prefix "btrfs."
Properties can be inherited - this means when a directory inode has
inheritable properties set, these are added to new inodes created
under that directory. Further, subvolumes can also have properties
associated with them, and they can be inherited from their parent
subvolume. Naturally, directory properties have priority over subvolume
properties (in practice a subvolume property is just a regular
property associated with the root inode, objectid 256, of the
subvolume's fs tree).
This change also adds one specific property implementation, named
"compression", whose values can be "lzo" or "zlib" and it's an
inheritable property.
The corresponding changes to btrfs-progs were also implemented.
A patch with xfstests for this feature will follow once there's
agreement on this change/feature.
Further, the script at the bottom of this commit message was used to
do some benchmarks to measure any performance penalties of this feature.
Basically the tests correspond to:
Test 1 - create a filesystem and mount it with compress-force=lzo,
then sequentially create N files of 64Kb each, measure how long it took
to create the files, unmount the filesystem, mount the filesystem and
perform an 'ls -lha' against the test directory holding the N files, and
report the time the command took.
Test 2 - create a filesystem and don't use any compression option when
mounting it - instead set the compression property of the subvolume's
root to 'lzo'. Then create N files of 64Kb, and report the time it took.
The unmount the filesystem, mount it again and perform an 'ls -lha' like
in the former test. This means every single file ends up with a property
(xattr) associated to it.
Test 3 - same as test 2, but uses 4 properties - 3 are duplicates of the
compression property, have no real effect other than adding more work
when inheriting properties and taking more btree leaf space.
Test 4 - same as test 3 but with 10 properties per file.
Results (in seconds, and averages of 5 runs each), for different N
numbers of files follow.
* Without properties (test 1)
file creation time ls -lha time
10 000 files 3.49 0.76
100 000 files 47.19 8.37
1 000 000 files 518.51 107.06
* With 1 property (compression property set to lzo - test 2)
file creation time ls -lha time
10 000 files 3.63 0.93
100 000 files 48.56 9.74
1 000 000 files 537.72 125.11
* With 4 properties (test 3)
file creation time ls -lha time
10 000 files 3.94 1.20
100 000 files 52.14 11.48
1 000 000 files 572.70 142.13
* With 10 properties (test 4)
file creation time ls -lha time
10 000 files 4.61 1.35
100 000 files 58.86 13.83
1 000 000 files 656.01 177.61
The increased latencies with properties are essencialy because of:
*) When creating an inode, we now synchronously write 1 more item
(an xattr item) for each property inherited from the parent dir
(or subvolume). This could be done in an asynchronous way such
as we do for dir intex items (delayed-inode.c), which could help
reduce the file creation latency;
*) With properties, we now have larger fs trees. For this particular
test each xattr item uses 75 bytes of leaf space in the fs tree.
This could be less by using a new item for xattr items, instead of
the current btrfs_dir_item, since we could cut the 'location' and
'type' fields (saving 18 bytes) and maybe 'transid' too (saving a
total of 26 bytes per xattr item) from the btrfs_dir_item type.
Also tried batching the xattr insertions (ignoring proper hash
collision handling, since it didn't exist) when creating files that
inherit properties from their parent inode/subvolume, but the end
results were (surprisingly) essentially the same.
Test script:
$ cat test.pl
#!/usr/bin/perl -w
use strict;
use Time::HiRes qw(time);
use constant NUM_FILES => 10_000;
use constant FILE_SIZES => (64 * 1024);
use constant DEV => '/dev/sdb4';
use constant MNT_POINT => '/home/fdmanana/btrfs-tests/dev';
use constant TEST_DIR => (MNT_POINT . '/testdir');
system("mkfs.btrfs", "-l", "16384", "-f", DEV) == 0 or die "mkfs.btrfs failed!";
# following line for testing without properties
#system("mount", "-o", "compress-force=lzo", DEV, MNT_POINT) == 0 or die "mount failed!";
# following 2 lines for testing with properties
system("mount", DEV, MNT_POINT) == 0 or die "mount failed!";
system("btrfs", "prop", "set", MNT_POINT, "compression", "lzo") == 0 or die "set prop failed!";
system("mkdir", TEST_DIR) == 0 or die "mkdir failed!";
my ($t1, $t2);
$t1 = time();
for (my $i = 1; $i <= NUM_FILES; $i++) {
my $p = TEST_DIR . '/file_' . $i;
open(my $f, '>', $p) or die "Error opening file!";
$f->autoflush(1);
for (my $j = 0; $j < FILE_SIZES; $j += 4096) {
print $f ('A' x 4096) or die "Error writing to file!";
}
close($f);
}
$t2 = time();
print "Time to create " . NUM_FILES . ": " . ($t2 - $t1) . " seconds.\n";
system("umount", DEV) == 0 or die "umount failed!";
system("mount", DEV, MNT_POINT) == 0 or die "mount failed!";
$t1 = time();
system("bash -c 'ls -lha " . TEST_DIR . " > /dev/null'") == 0 or die "ls failed!";
$t2 = time();
print "Time to ls -lha all files: " . ($t2 - $t1) . " seconds.\n";
system("umount", DEV) == 0 or die "umount failed!";
Signed-off-by: Filipe David Borba Manana <fdmanana@gmail.com>
Signed-off-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: Chris Mason <clm@fb.com>
btrfs filesystem df output will show the size of the metadata space
and how much of it is used, and the user assumes that the difference
is all usable space. Since that's not actually the case due to the
global metadata reservation, we should provide the full picture to the
user.
This patch adds an ioctl that exports the size of the global metadata
reservation so that btrfs filesystem df can report it.
Signed-off-by: Jeff Mahoney <jeffm@suse.com>
Signed-off-by: Josef Bacik <jbacik@fusionio.com>
Signed-off-by: Chris Mason <clm@fb.com>
