commit 7787380336 upstream.
net_dma can cause data to be copied to a stale mapping if a
copy-on-write fault occurs during dma. The application sees missing
data.
The following trace is triggered by modifying the kernel to WARN if it
ever triggers copy-on-write on a page that is undergoing dma:
WARNING: CPU: 24 PID: 2529 at lib/dma-debug.c:485 debug_dma_assert_idle+0xd2/0x120()
ioatdma 0000:00:04.0: DMA-API: cpu touching an active dma mapped page [pfn=0x16bcd9]
Modules linked in: iTCO_wdt iTCO_vendor_support ioatdma lpc_ich pcspkr dca
CPU: 24 PID: 2529 Comm: linbug Tainted: G W 3.13.0-rc1+ #353
00000000000001e5 ffff88016f45f688 ffffffff81751041 ffff88017ab0ef70
ffff88016f45f6d8 ffff88016f45f6c8 ffffffff8104ed9c ffffffff810f3646
ffff8801768f4840 0000000000000282 ffff88016f6cca10 00007fa2bb699349
Call Trace:
[<ffffffff81751041>] dump_stack+0x46/0x58
[<ffffffff8104ed9c>] warn_slowpath_common+0x8c/0xc0
[<ffffffff810f3646>] ? ftrace_pid_func+0x26/0x30
[<ffffffff8104ee86>] warn_slowpath_fmt+0x46/0x50
[<ffffffff8139c062>] debug_dma_assert_idle+0xd2/0x120
[<ffffffff81154a40>] do_wp_page+0xd0/0x790
[<ffffffff811582ac>] handle_mm_fault+0x51c/0xde0
[<ffffffff813830b9>] ? copy_user_enhanced_fast_string+0x9/0x20
[<ffffffff8175fc2c>] __do_page_fault+0x19c/0x530
[<ffffffff8175c196>] ? _raw_spin_lock_bh+0x16/0x40
[<ffffffff810f3539>] ? trace_clock_local+0x9/0x10
[<ffffffff810fa1f4>] ? rb_reserve_next_event+0x64/0x310
[<ffffffffa0014c00>] ? ioat2_dma_prep_memcpy_lock+0x60/0x130 [ioatdma]
[<ffffffff8175ffce>] do_page_fault+0xe/0x10
[<ffffffff8175c862>] page_fault+0x22/0x30
[<ffffffff81643991>] ? __kfree_skb+0x51/0xd0
[<ffffffff813830b9>] ? copy_user_enhanced_fast_string+0x9/0x20
[<ffffffff81388ea2>] ? memcpy_toiovec+0x52/0xa0
[<ffffffff8164770f>] skb_copy_datagram_iovec+0x5f/0x2a0
[<ffffffff8169d0f4>] tcp_rcv_established+0x674/0x7f0
[<ffffffff816a68c5>] tcp_v4_do_rcv+0x2e5/0x4a0
[..]
---[ end trace e30e3b01191b7617 ]---
Mapped at:
[<ffffffff8139c169>] debug_dma_map_page+0xb9/0x160
[<ffffffff8142bf47>] dma_async_memcpy_pg_to_pg+0x127/0x210
[<ffffffff8142cce9>] dma_memcpy_pg_to_iovec+0x119/0x1f0
[<ffffffff81669d3c>] dma_skb_copy_datagram_iovec+0x11c/0x2b0
[<ffffffff8169d1ca>] tcp_rcv_established+0x74a/0x7f0:
...the problem is that the receive path falls back to cpu-copy in
several locations and this trace is just one of the areas. A few
options were considered to fix this:
1/ sync all dma whenever a cpu copy branch is taken
2/ modify the page fault handler to hold off while dma is in-flight
Option 1 adds yet more cpu overhead to an "offload" that struggles to compete
with cpu-copy. Option 2 adds checks for behavior that is already documented as
broken when using get_user_pages(). At a minimum a debug mode is warranted to
catch and flag these violations of the dma-api vs get_user_pages().
Thanks to David for his reproducer.
Cc: Dave Jiang <dave.jiang@intel.com>
Cc: Vinod Koul <vinod.koul@intel.com>
Cc: Alexander Duyck <alexander.h.duyck@intel.com>
Reported-by: David Whipple <whipple@securedatainnovations.ch>
Acked-by: David S. Miller <davem@davemloft.net>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
commit 21e96c7313 upstream.
When performing continuations there are implied sources that need to be
added to the source count. Quoting dma_set_maxpq:
/* dma_maxpq - reduce maxpq in the face of continued operations
* @dma - dma device with PQ capability
* @flags - to check if DMA_PREP_CONTINUE and DMA_PREP_PQ_DISABLE_P are set
*
* When an engine does not support native continuation we need 3 extra
* source slots to reuse P and Q with the following coefficients:
* 1/ {00} * P : remove P from Q', but use it as a source for P'
* 2/ {01} * Q : use Q to continue Q' calculation
* 3/ {00} * Q : subtract Q from P' to cancel (2)
*
* In the case where P is disabled we only need 1 extra source:
* 1/ {01} * Q : use Q to continue Q' calculation
*/
...fix the selection of the 16 source path to take these implied sources
into account.
Note this also kills the BUG_ON(src_cnt < 9) check in
__ioat3_prep_pq16_lock(). Besides not accounting for implied sources
the check is redundant given we already made the path selection.
Cc: Dave Jiang <dave.jiang@intel.com>
Acked-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
commit 5d48b9b5d8 upstream.
The array to lookup the sed pool based on the number of sources
(pq16_idx_to_sedi) is 16 entries and expects a max source index.
However, we pass the total source count which runs off the end of the
array when src_cnt == 16. The minimal fix is to just pass src_cnt-1,
but given we know the source count is > 8 we can just calculate the sed
pool by (src_cnt - 2) >> 3.
Cc: Dave Jiang <dave.jiang@intel.com>
Acked-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
commit fcaaba6c71 upstream.
We need to free the ld_active list head before jumping into the callback
routine. Otherwise the callback could run into issue_pending and change
our ld_active list head we just going to free. This will run the channel
list into an currupted and undefined state.
Signed-off-by: Michael Grzeschik <m.grzeschik@pengutronix.de>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
Cc: Jonghwan Choi <jhbird.choi@samsung.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
commit fc51446021 upstream.
Allocate a descriptor for each period of a cyclic transfer, not just the first.
Also since the callback needs to be called for each finished period make sure to
initialize the callback and callback_param fields of each descriptor in a cyclic
transfer.
Signed-off-by: Lars-Peter Clausen <lars@metafoo.de>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
When user interrupts ongoing transfers the dmatest may end up with console
lockup, oops, or data mismatch. This patch prevents user to abort any ongoing
test.
Documentation is updated accordingly.
Signed-off-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Reported-by: Will Deacon <will.deacon@arm.com>
Tested-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
Pull slave-dma fixes from Vinod Koul:
"We have two patches from Andy & Rafael fixing the Lynxpoint dma"
* 'fixes' of git://git.infradead.org/users/vkoul/slave-dma:
ACPI / LPSS: register clock device for Lynxpoint DMA properly
dma: acpi-dma: parse CSRT to extract additional resources
devm_ioremap_resource does sanity checks on the given resource. No need to
duplicate this in the driver.
Signed-off-by: Wolfram Sang <wsa@the-dreams.de>
Acked-by: Stephen Warren <swarren@nvidia.com>
Pull slave-dmaengine updates from Vinod Koul:
"This time we have dmatest improvements from Andy along with dw_dmac
fixes. He has also done support for acpi for dmanegine.
Also we have bunch of fixes going in DT support for dmanegine for
various folks. Then Haswell and other ioat changes from Dave and
SUDMAC support from Shimoda."
* 'for-linus' of git://git.infradead.org/users/vkoul/slave-dma: (53 commits)
dma: tegra: implement suspend/resume callbacks
dma:of: Use a mutex to protect the of_dma_list
dma: of: Fix of_node reference leak
dmaengine: sirf: move driver init from module_init to subsys_initcall
sudmac: add support for SUDMAC
dma: sh: add Kconfig
at_hdmac: move to generic DMA binding
ioatdma: ioat3_alloc_sed can be static
ioatdma: Adding write back descriptor error status support for ioatdma 3.3
ioatdma: S1200 platforms ioatdma channel 2 and 3 falsely advertise RAID cap
ioatdma: Adding support for 16 src PQ ops and super extended descriptors
ioatdma: Removing hw bug workaround for CB3.x .2 and earlier
dw_dmac: add ACPI support
dmaengine: call acpi_dma_request_slave_channel as well
dma: acpi-dma: introduce ACPI DMA helpers
dma: of: Remove unnecessary list_empty check
DMA: OF: Check properties value before running be32_to_cpup() on it
DMA: of: Constant names
ioatdma: skip silicon bug workaround for pq_align for cb3.3
ioatdma: Removing PQ val disable for cb3.3
...
Pull ARM SoC device tree updates (part 2) from Arnd Bergmann:
"These are mostly new device tree bindings for existing drivers, as
well as changes to the device tree source files to add support for
those devices, and a couple of new boards, most notably Samsung's
Exynos5 based Chromebook.
The changes depend on earlier platform specific updates and touch the
usual platforms: omap, exynos, tegra, mxs, mvebu and davinci."
* tag 'dt-for-linus-2' of git://git.kernel.org/pub/scm/linux/kernel/git/arm/arm-soc: (169 commits)
ARM: exynos: dts: cros5250: add EC device
ARM: dts: Add sbs-battery for exynos5250-snow
ARM: dts: Add i2c-arbitrator bus for exynos5250-snow
ARM: dts: add mshc controller node for Exynos4x12 SoCs
ARM: dts: Add chip-id controller node on Exynos4/5 SoC
ARM: EXYNOS: Create virtual I/O mapping for Chip-ID controller using device tree
ARM: davinci: da850-evm: add SPI flash support
ARM: davinci: da850: override SPI DT node device name
ARM: davinci: da850: add SPI1 DT node
spi/davinci: add DT binding documentation
spi/davinci: no wildcards in DT compatible property
ARM: dts: mvebu: Convert mvebu device tree files to 64 bits
ARM: dts: mvebu: introduce internal-regs node
ARM: dts: mvebu: Convert all the mvebu files to use the range property
ARM: dts: mvebu: move all peripherals inside soc
ARM: dts: mvebu: fix cpus section indentation
ARM: davinci: da850: add EHRPWM & ECAP DT node
ARM/dts: OMAP3: fix pinctrl-single configuration
ARM: dts: Add OMAP3430 SDP NOR flash memory binding
ARM: dts: Add NOR flash bindings for OMAP2420 H4
...
This is support for the ARM Chromebook, originally scheduled
as a "late" pull request. Since it's already late now, we
can combine this into the existing next/dt2 branch.
* late/dt:
ARM: exynos: dts: cros5250: add EC device
ARM: dts: Add sbs-battery for exynos5250-snow
ARM: dts: Add i2c-arbitrator bus for exynos5250-snow
ARM: dts: Add chip-id controller node on Exynos4/5 SoC
ARM: EXYNOS: Create virtual I/O mapping for Chip-ID controller using device tree
Currently the OF DMA code uses a spin lock to protect the of_dma_list from
concurrent access and a per controller reference count to protect the controller
from being freed while a request operation is in progress. If
of_dma_controller_free() is called for a controller who's reference count is not
zero it will return -EBUSY and not remove the controller. This is fine up until
here, but leaves the question what the caller of of_dma_controller_free() is
supposed to do if the controller couldn't be freed. The only viable solution
for the caller is to spin on of_dma_controller_free() until it returns success.
E.g.
do {
ret = of_dma_controller_free(dev->of_node)
} while (ret != -EBUSY);
This is rather ugly and unnecessary and none of the current users of
of_dma_controller_free() check it's return value anyway. Instead protect the
list by a mutex. The mutex will be held as long as a request operation is in
progress. So if of_dma_controller_free() is called while a request operation is
in progress it will be put to sleep and only wake up once the request operation
has finished.
This means that it is no longer possible to register or unregister OF DMA
controllers from a context where it's not possible to sleep. But I doubt that
we'll ever need this.
Also rename of_dma_get_controller back to of_dma_find_controller.
Signed-off-by: Lars-Peter Clausen <lars@metafoo.de>
Acked-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
of_dma_request_slave_channel() currently does not drop the reference to the
dma_spec of_node if no DMA controller matching the of_node could be found. This
patch fixes it by always calling of_node_put().
Signed-off-by: Lars-Peter Clausen <lars@metafoo.de>
Acked-by: Arnd Bergmann <arnd@arndb.de>
Reviewed-by: Jon Hunter <jon-hunter@ti.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
if we initilize dma driver by module_init, there are still many devices
which will be initilized earlier than dma. these devices will fail to
get dma channel.
this moves dmaengine earlier than device_initcall and make dma available
for all devices.
Reported-by: Renwei Wu <Renwei.Wu@csr.com>
Signed-off-by: Barry Song <Baohua.Song@csr.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
This patch adds Kconfig in the drivers/dma/sh. This patch also adds
a new config "SH_DMAE_BASE" and the "config SH_DMAE" depends on it.
Since some drivers (e.g. sh_mmcif.c) depends on shdma-base.c if
CONFIG_DMA_ENGINE=y, the "config SH_DMAE_BASE" is set as "bool".
Signed-off-by: Yoshihiro Shimoda <yoshihiro.shimoda.uh@renesas.com>
Acked-by: Kuninori Morimoto <kuninori.morimoto.gx@renesas.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
Update at_hdmac driver to support generic DMA device tree binding. Devices
can still request channel with dma_request_channel() then it doesn't break
DMA for non DT boards.
Signed-off-by: Ludovic Desroches <ludovic.desroches@atmel.com>
Acked-by: Nicolas Ferre <nicolas.ferre@atmel.com>
Acked-by: Jean-Christophe PLAGNIOL-VILLARD <plagnioj@jcrosoft.com>
Acked-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
Conflicts:
drivers/net/ethernet/emulex/benet/be_main.c
drivers/net/ethernet/intel/igb/igb_main.c
drivers/net/wireless/brcm80211/brcmsmac/mac80211_if.c
include/net/scm.h
net/batman-adv/routing.c
net/ipv4/tcp_input.c
The e{uid,gid} --> {uid,gid} credentials fix conflicted with the
cleanup in net-next to now pass cred structs around.
The be2net driver had a bug fix in 'net' that overlapped with the VLAN
interface changes by Patrick McHardy in net-next.
An IGB conflict existed because in 'net' the build_skb() support was
reverted, and in 'net-next' there was a comment style fix within that
code.
Several batman-adv conflicts were resolved by making sure that all
calls to batadv_is_my_mac() are changed to have a new bat_priv first
argument.
Eric Dumazet's TS ECR fix in TCP in 'net' conflicted with the F-RTO
rewrite in 'net-next', mostly overlapping changes.
Thanks to Stephen Rothwell and Antonio Quartulli for help with several
of these merge resolutions.
Signed-off-by: David S. Miller <davem@davemloft.net>
