The MIPS GIC driver has previously iterated over bits set in a bitmap
representing pending local IRQs by calling find_first_bit, clearing that
bit then calling find_first_bit again until all bits are clear. If
multiple interrupts are pending then this is wasteful, as find_first_bit
will have to loop over the whole bitmap from the start. Use the
for_each_set_bit macro which performs exactly what we need here instead.
It will use find_next_bit and thus only scan over the relevant part of
the bitmap, and it makes the intent of the code clearer.
This makes the same change for local interrupts that commit cae750bae4
("irqchip: mips-gic: Use for_each_set_bit to iterate over IRQs") made
for shared interrupts.
Signed-off-by: Paul Burton <paul.burton@imgtec.com>
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: linux-mips@linux-mips.org
Cc: Jason Cooper <jason@lakedaemon.net>
Link: http://lkml.kernel.org/r/20160913165427.31686-1-paul.burton@imgtec.com
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Since the device hierarchy domain was added by commit c98c1822ee
("irqchip/mips-gic: Add device hierarchy domain"), GIC local interrupts
have been broken.
Users attempting to setup a per-cpu local IRQ, for example the GIC timer
clock events code in drivers/clocksource/mips-gic-timer.c, the
setup_percpu_irq function would refuse with -EINVAL because the GIC
irqchip driver never called irq_set_percpu_devid so the
IRQ_PER_CPU_DEVID flag was never set for the IRQ. This happens because
irq_set_percpu_devid was being called from the gic_irq_domain_map
function which is no longer called.
Doing only that runs into further problems because gic_dev_domain_alloc
set the struct irq_chip for all interrupts, local or shared, to
gic_level_irq_controller despite that only being suitable for shared
interrupts. The typical outcome of this is that gic_level_irq_controller
callback functions are called for local interrupts, and then hwirq
number calculations overflow & the driver ends up attempting to access
some invalid register with an address calculated from an invalid hwirq
number. Best case scenario is that this then leads to a bus error. This
is fixed by abstracting the setup of the hwirq & chip to a new function
gic_setup_dev_chip which is used by both the root GIC IRQ domain & the
device domain.
Finally, decoding local interrupts failed because gic_dev_domain_alloc
only called irq_domain_alloc_irqs_parent for shared interrupts. Local
ones were therefore never associated with hwirqs in the root GIC IRQ
domain and the virq in gic_handle_local_int would always be 0. This is
fixed by calling irq_domain_alloc_irqs_parent unconditionally & having
gic_irq_domain_alloc handle both local & shared interrupts, which is
easy due to the aforementioned abstraction of chip setup into
gic_setup_dev_chip.
This fixes use of the MIPS GIC timer for clock events, which has been
broken since c98c1822ee ("irqchip/mips-gic: Add device hierarchy
domain") but hadn't been noticed due to a silent fallback to the MIPS
coprocessor 0 count/compare clock events device.
Fixes: c98c1822ee ("irqchip/mips-gic: Add device hierarchy domain")
Signed-off-by: Paul Burton <paul.burton@imgtec.com>
Cc: linux-mips@linux-mips.org
Cc: Jason Cooper <jason@lakedaemon.net>
Cc: Qais Yousef <qsyousef@gmail.com>
Cc: stable@vger.kernel.org
Cc: Marc Zyngier <marc.zyngier@arm.com>
Link: http://lkml.kernel.org/r/20160913165335.31389-1-paul.burton@imgtec.com
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
gic_raise_softirq() walks the list of cpus using for_each_cpu(), it calls
gic_compute_target_list() which advances the iterator by the number of
CPUs in the cluster.
If gic_compute_target_list() reaches the last CPU it leaves the iterator
pointing at the last CPU. This means the next time round the for_each_cpu()
loop cpumask_next() will be called with an invalid CPU.
This triggers a warning when built with CONFIG_DEBUG_PER_CPU_MAPS:
[ 3.077738] GICv3: CPU1: found redistributor 1 region 0:0x000000002f120000
[ 3.077943] CPU1: Booted secondary processor [410fd0f0]
[ 3.078542] ------------[ cut here ]------------
[ 3.078746] WARNING: CPU: 1 PID: 0 at ../include/linux/cpumask.h:121 gic_raise_softirq+0x12c/0x170
[ 3.078812] Modules linked in:
[ 3.078869]
[ 3.078930] CPU: 1 PID: 0 Comm: swapper/1 Not tainted 4.8.0-rc5+ #5188
[ 3.078994] Hardware name: Foundation-v8A (DT)
[ 3.079059] task: ffff80087a1a0080 task.stack: ffff80087a19c000
[ 3.079145] PC is at gic_raise_softirq+0x12c/0x170
[ 3.079226] LR is at gic_raise_softirq+0xa4/0x170
[ 3.079296] pc : [<ffff0000083ead24>] lr : [<ffff0000083eac9c>] pstate: 200001c9
[ 3.081139] Call trace:
[ 3.081202] Exception stack(0xffff80087a19fbe0 to 0xffff80087a19fd10)
[ 3.082269] [<ffff0000083ead24>] gic_raise_softirq+0x12c/0x170
[ 3.082354] [<ffff00000808e614>] smp_send_reschedule+0x34/0x40
[ 3.082433] [<ffff0000080e80a0>] resched_curr+0x50/0x88
[ 3.082512] [<ffff0000080e89d0>] check_preempt_curr+0x60/0xd0
[ 3.082593] [<ffff0000080e8a60>] ttwu_do_wakeup+0x20/0xe8
[ 3.082672] [<ffff0000080e8bb8>] ttwu_do_activate+0x90/0xc0
[ 3.082753] [<ffff0000080ea9a4>] try_to_wake_up+0x224/0x370
[ 3.082836] [<ffff0000080eabc8>] default_wake_function+0x10/0x18
[ 3.082920] [<ffff000008103134>] __wake_up_common+0x5c/0xa0
[ 3.083003] [<ffff0000081031f4>] __wake_up_locked+0x14/0x20
[ 3.083086] [<ffff000008103f80>] complete+0x40/0x60
[ 3.083168] [<ffff00000808df7c>] secondary_start_kernel+0x15c/0x1d0
[ 3.083240] [<00000000808911a4>] 0x808911a4
[ 3.113401] Detected PIPT I-cache on CPU2
Avoid updating the iterator if the next call to cpumask_next() would
cause the for_each_cpu() loop to exit.
There is no change to gic_raise_softirq()'s behaviour, (cpumask_next()s
eventual call to _find_next_bit() will return early as start >= nbits),
this patch just silences the warning.
Fixes: 021f653791 ("irqchip: gic-v3: Initial support for GICv3")
Signed-off-by: James Morse <james.morse@arm.com>
Acked-by: Marc Zyngier <marc.zyngier@arm.com>
Cc: linux-arm-kernel@lists.infradead.org
Cc: Jason Cooper <jason@lakedaemon.net>
Link: http://lkml.kernel.org/r/1474306155-3303-1-git-send-email-james.morse@arm.com
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Let ACPI build ITS PCI MSI domain. ACPI code is responsible for retrieving
inner domain token and passing it on to its_pci_msi_init_one generic
init call.
IORT maintains list of registered domain tokens and allows to find
corresponding domain based on MADT ITS subtable ID info.
Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
Acked-by: Marc Zyngier <marc.zyngier@arm.com>
Reviewed-by: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Firmware agnostic code lands in common functions which do necessary
domain initialization based on unique domain handler. DT specific
code goes to DT specific init call.
Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
Acked-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
ITS is prepared for being initialized different than DT,
therefore we can initialize it in ACPI way. We collect register base
address from MADT table and pass mandatory info to firmware-agnostic
ITS init call.
Use here IORT lib to register ITS domain which then can be found and
used on to build another PCI MSI domain in hierarchical stack domain.
NOTE: Waiting for proper ITS and NUMA node relation description in IORT
table, we pass around NUMA_NO_NODE to the its_probe_one init call.
This means that Cavium ThunderX erratum 23144 (pass1.1 only)
is not supported for ACPI boot method yet.
Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
Acked-by: Marc Zyngier <marc.zyngier@arm.com>
Reviewed-by: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
In order to add ACPI support we need to isolate ACPI&DT common code and
move DT logic to corresponding functions. To achieve this we are using
firmware agnostic handle which can be unpacked to either DT or ACPI node.
No functional changes other than a very minor one:
1. Terminate its_init call with -ENODEV for non-DT case which allows
to remove hack from its-gic-v3.c.
2. Fix ITS base register address type (from 'unsigned long' to 'phys_addr_t'),
as a bonus we get nice string formatting.
3. Since there is only one of ITS parent domain convert it to static global
variable and drop the parameter from its_probe_one. Users can refer to it
in more convenient way then.
Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
There is no point to initialize ITS without having msi-controller
property in corresponding DT node. However, its_probe is checking
msi-controller presence at the end, so we can save our time and do that
check prior to its_probe call. Also, for the code clarity purpose,
we put domain initialization to separate function.
Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
Acked-by: Marc Zyngier <marc.zyngier@arm.com>
Reviewed-by: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
We get 1 warning when building kernel with W=1:
drivers/irqchip/irq-gic.c:917:13: warning: no previous prototype for 'gic_init_physaddr' [-Wmissing-prototypes]
In fact, this function is only used in the file in which it is
declared and don't need a declaration, but can be made static.
so this patch marks this function with 'static'.
Signed-off-by: Baoyou Xie <baoyou.xie@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Commit 498b5fdd40 ("PM / clk: Add support for adding a specific clock
from device-tree") add a new helper function for adding a clock from
device-tree to a device. Update the GIC-PM driver to use this new
function to simplify the driver.
Signed-off-by: Jon Hunter <jonathanh@nvidia.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Currently, when running on FVP, CPU 0 boots up with its BPR changed from
the reset value. This renders it impossible to (preemptively) prioritize
interrupts on CPU 0.
This is harmless on normal systems since Linux typically does not
support preemptive interrupts. It does however cause problems in
systems with additional changes (such as patches for NMI simulation).
Many thanks to Andrew Thoelke for suggesting the BPR as having the
potential to harm preemption.
Suggested-by: Andrew Thoelke <andrew.thoelke@arm.com>
Signed-off-by: Daniel Thompson <daniel.thompson@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
The BL switcher code manipulates the logical/physical CPU mapping,
forcing a lock to be taken on the IPI path. With an IPI heavy load,
this single lock becomes contended.
But when CONFIG_BL_SWITCHER is not enabled, there is no reason
to take this lock at all since the CPU mapping is immutable.
This patch allows the lock to be entierely removed when BL_SWITCHER
is not enabled (which is the case in most configurations), leading
to a small improvement of "perf bench sched pipe" (measured on
an 8 core AMD Seattle system):
Before: 101370 ops/sec
After: 103680 ops/sec
Take this opportunity to remove a useless lock being taken when
handling an interrupt on a secondary GIC.
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
The MIPS GIC driver has previously iterated over bits set in a bitmap
representing pending IRQs by calling find_first_bit, clearing that bit
then calling find_first_bit again until all bits are clear. If multiple
interrupts are pending then this is wasteful, as find_first_bit will
have to loop over the whole bitmap from the start. Use the
for_each_set_bit macro which performs exactly what we need here instead.
It will use find_next_bit and thus only scan over the relevant part of
the bitmap, and it makes the intent of the code more clear.
Signed-off-by: Paul Burton <paul.burton@imgtec.com>
Link: https://lkml.kernel.org/r/20160819171119.28121-1-paul.burton@imgtec.com
Signed-off-by: Jason Cooper <jason@lakedaemon.net>
The Marvell Armada 7K/8K integrates a secondary interrupt controller
very originally named "PIC". It is connected to the main GIC via a
PPI. Amongst other things, this PIC is used for the ARM PMU.
This commit adds a simple irqchip driver for this interrupt
controller. Since this interrupt controller is not needed early at boot
time, we make the driver a proper platform driver rather than use the
IRQCHIP_DECLARE() mechanism.
Signed-off-by: Yehuda Yitschak <yehuday@marvell.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Link: https://lkml.kernel.org/r/1470408921-447-3-git-send-email-thomas.petazzoni@free-electrons.com
Signed-off-by: Jason Cooper <jason@lakedaemon.net>
If an IRQ is setup using __setup_irq(), which is used by the
request_irq() family of functions, and we are using an SMP kernel then
the affinity of the IRQ will be set via setup_affinity() immediately
after the IRQ is enabled. This call to gic_set_affinity() will lead to
the interrupt being mapped to a VPE. However there are other ways to use
IRQs which don't cause affinity to be set, for example if it is used to
chain to another IRQ controller with irq_set_chained_handler_and_data().
The irq_set_chained_handler_and_data() code path will enable the IRQ,
but will not trigger a call to gic_set_affinity() and in this case
nothing will map the interrupt to a VPE, meaning that the interrupt is
never received.
Fix this by implementing the activate operation for the GIC device IRQ
domain, using gic_shared_irq_domain_map() to map the interrupt to the
correct pin of cpu 0.
Fixes: c98c1822ee ("irqchip/mips-gic: Add device hierarchy domain")
Signed-off-by: Paul Burton <paul.burton@imgtec.com>
Cc: linux-mips@linux-mips.org
Cc: Jason Cooper <jason@lakedaemon.net>
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: stable@vger.kernel.org
Link: http://lkml.kernel.org/r/20160819170715.27820-2-paul.burton@imgtec.com
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
gic_shared_irq_domain_map() is called from gic_irq_domain_alloc() where
the wrong chip has been set, and is then overwritten. Tidy this up by
setting the correct chip the first time, and setting the
handle_level_irq handler from gic_irq_domain_alloc() too.
gic_shared_irq_domain_map() is also called from gic_irq_domain_map(),
which now calls irq_set_chip_and_handler() to retain its previous
behaviour.
This patch prepares for a follow-on which will call
gic_shared_irq_domain_map() from a callback where the lock on the struct
irq_desc is held, which without this change would cause the call to
irq_set_chip_and_handler() to lead to a deadlock.
Fixes: c98c1822ee ("irqchip/mips-gic: Add device hierarchy domain")
Signed-off-by: Paul Burton <paul.burton@imgtec.com>
Cc: linux-mips@linux-mips.org
Cc: Jason Cooper <jason@lakedaemon.net>
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: stable@vger.kernel.org
Link: http://lkml.kernel.org/r/20160819170715.27820-1-paul.burton@imgtec.com
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
When starting a kexec/kdump kernel, the GIC ITS will already have been
enabled. According to the ARM Generic Interrupt Controller
Architecture Specification (GIC architecture Version 3.0 and version
4.0), writing to GITS_BASER<n> or GITS_CBASER is "UNPREDICTABLE" when
the ITS is enabled. On Cavium Thunder systems, this prevents the ITS
from being initializing in the kexec/kdump kernel, resulting in
failure to register/enable interrupts for all devices.
The fix is to disable the ITS if it is not already in the disabled
state. This allows the ITS to be properly initialized and then
re-enabled in the kexec/kdump kernel.
Acked-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: David Daney <david.daney@cavium.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Thomas Gleixner
Alexandre TORGUE
Paul Burton
James Morse
Boris Brezillon
Tomasz Nowicki
Baoyou Xie
Jon Hunter
Daniel Thompson
Marc Zyngier
Colin Ian King
Jason Cooper
Wei Yongjun
Thomas Petazzoni
David Daney