There were two copy and paste type of errors where I swapped 'complete'
with 'submit' names. Use the correct variables and definitions.
This problem was found by Dmitry Bogdanov and this patch builds on top of
his patch to fix a second instance that was found.
Fixes: 06933066d8 ("scsi: target: Add support for completing commands from backend context")
Signed-off-by: Dmitry Bogdanov <d.bogdanov@yadro.com>
[mnc: Fixed second instance]
Signed-off-by: Mike Christie <michael.christie@oracle.com>
Link: https://patch.msgid.link/20260307220630.131008-1-michael.christie@oracle.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
To complete a command several drivers just drop their reference and add
it to list to be processed by a driver specific thread. So there's no
need to go from backend context to the LIO thread then to the driver's
thread. When avoiding the LIO thread, IOPS can increase from 20-30% for
workloads like:
fio --filename=/dev/sdb --direct=1 --rw=randrw --bs=8K \
--ioengine=libaio --iodepth=128 --numjobs=$jobs
where increasing jobs increases the performance improvement (this is
using NVMe drives with LIO's submit_type=1 to directly submit).
Add the infrastructure so drivers and userspace can control how to
complete a command like is done for the submission path. In this commit
there is no behavior change and we continue to defer to the LIO
workqueue thread. In the subsequent commits we will allow drivers to
report what they support and allow userspace to control the behavior.
Signed-off-by: Mike Christie <michael.christie@oracle.com>
Link: https://patch.msgid.link/20260222232946.7637-2-michael.christie@oracle.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
This was done entirely with mindless brute force, using
git grep -l '\<k[vmz]*alloc_objs*(.*, GFP_KERNEL)' |
xargs sed -i 's/\(alloc_objs*(.*\), GFP_KERNEL)/\1)/'
to convert the new alloc_obj() users that had a simple GFP_KERNEL
argument to just drop that argument.
Note that due to the extreme simplicity of the scripting, any slightly
more complex cases spread over multiple lines would not be triggered:
they definitely exist, but this covers the vast bulk of the cases, and
the resulting diff is also then easier to check automatically.
For the same reason the 'flex' versions will be done as a separate
conversion.
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
This is the result of running the Coccinelle script from
scripts/coccinelle/api/kmalloc_objs.cocci. The script is designed to
avoid scalar types (which need careful case-by-case checking), and
instead replace kmalloc-family calls that allocate struct or union
object instances:
Single allocations: kmalloc(sizeof(TYPE), ...)
are replaced with: kmalloc_obj(TYPE, ...)
Array allocations: kmalloc_array(COUNT, sizeof(TYPE), ...)
are replaced with: kmalloc_objs(TYPE, COUNT, ...)
Flex array allocations: kmalloc(struct_size(PTR, FAM, COUNT), ...)
are replaced with: kmalloc_flex(*PTR, FAM, COUNT, ...)
(where TYPE may also be *VAR)
The resulting allocations no longer return "void *", instead returning
"TYPE *".
Signed-off-by: Kees Cook <kees@kernel.org>
The atomic use in the main I/O path is causing perf issues when using
higher performance backend devices and multiple queues (more than
10 when using vhost-scsi) like with this fio workload:
[global]
bs=4K
iodepth=128
direct=1
ioengine=libaio
group_reporting
time_based
runtime=120
name=standard-iops
rw=randread
numjobs=16
cpus_allowed=0-15
To fix this issue, move the LUN stats to per CPU.
Note: I forgot to include this patch with the delayed/ordered per CPU
tracking and per device/device entry per CPU stats. With this patch you
get the full 33% improvements when using fast backends, multiple queues
and multiple IO submiters.
Signed-off-by: Mike Christie <michael.christie@oracle.com>
Reviewed-by: Dmitry Bogdanov <d.bogdanov@yadro.com>
Link: https://patch.msgid.link/20250917221338.14813-4-michael.christie@oracle.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
The names RESERVE and RELEASE are not only used in <scsi/scsi_proto.h> but
also elsewhere in the kernel:
$ git grep -nHE 'define[[:blank:]]*(RESERVE|RELEASE)[[:blank:]]'
drivers/input/joystick/walkera0701.c:13:#define RESERVE 20000
drivers/s390/char/tape_std.h:56:#define RELEASE 0xD4 /* 3420 NOP, 3480 REJECT */
drivers/s390/char/tape_std.h:58:#define RESERVE 0xF4 /* 3420 NOP, 3480 REJECT */
Additionally, while the names of the symbolic constants RESERVE_10 and
RELEASE_10 include the command length, the command length is not included
in the RESERVE and RELEASE names. Address both issues by renaming the
RESERVE and RELEASE constants into RESERVE_6 and RELEASE_6 respectively.
Reviewed-by: Christoph Hellwig <hch@lst.de>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Bart Van Assche <bvanassche@acm.org>
Link: https://lore.kernel.org/r/20250210205031.2970833-1-bvanassche@acm.org
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Pull SCSI fixes from James Bottomley:
"Fixes all in drivers. The largest is the mpi3mr which corrects a phy
count limit that should only apply to the controller but was being
incorrectly applied to expander phys"
* tag 'scsi-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi:
scsi: target: core: Fix null-ptr-deref in target_alloc_device()
scsi: mpi3mr: Validate SAS port assignments
scsi: ufs: core: Set SDEV_OFFLINE when UFS is shut down
scsi: ufs: core: Requeue aborted request
scsi: ufs: core: Fix the issue of ICU failure
There is a null-ptr-deref issue reported by KASAN:
BUG: KASAN: null-ptr-deref in target_alloc_device+0xbc4/0xbe0 [target_core_mod]
...
kasan_report+0xb9/0xf0
target_alloc_device+0xbc4/0xbe0 [target_core_mod]
core_dev_setup_virtual_lun0+0xef/0x1f0 [target_core_mod]
target_core_init_configfs+0x205/0x420 [target_core_mod]
do_one_initcall+0xdd/0x4e0
...
entry_SYSCALL_64_after_hwframe+0x76/0x7e
In target_alloc_device(), if allocing memory for dev queues fails, then
dev will be freed by dev->transport->free_device(), but dev->transport
is not initialized at that time, which will lead to a null pointer
reference problem.
Fixing this bug by freeing dev with hba->backend->ops->free_device().
Fixes: 1526d9f10c ("scsi: target: Make state_list per CPU")
Signed-off-by: Wang Hai <wanghai38@huawei.com>
Link: https://lore.kernel.org/r/20241011113444.40749-1-wanghai38@huawei.com
Reviewed-by: Mike Christie <michael.christie@oracle.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
asm/unaligned.h is always an include of asm-generic/unaligned.h;
might as well move that thing to linux/unaligned.h and include
that - there's nothing arch-specific in that header.
auto-generated by the following:
for i in `git grep -l -w asm/unaligned.h`; do
sed -i -e "s/asm\/unaligned.h/linux\/unaligned.h/" $i
done
for i in `git grep -l -w asm-generic/unaligned.h`; do
sed -i -e "s/asm-generic\/unaligned.h/linux\/unaligned.h/" $i
done
git mv include/asm-generic/unaligned.h include/linux/unaligned.h
git mv tools/include/asm-generic/unaligned.h tools/include/linux/unaligned.h
sed -i -e "/unaligned.h/d" include/asm-generic/Kbuild
sed -i -e "s/__ASM_GENERIC/__LINUX/" include/linux/unaligned.h tools/include/linux/unaligned.h
An abort that is responded to by iSCSI itself is added to tmr_list but does
not go to target core. A LUN_RESET that goes through tmr_list takes a
refcounter on the abort and waits for completion. However, the abort will
be never complete because it was not started in target core.
Unable to locate ITT: 0x05000000 on CID: 0
Unable to locate RefTaskTag: 0x05000000 on CID: 0.
wait_for_tasks: Stopping tmf LUN_RESET with tag 0x0 ref_task_tag 0x0 i_state 34 t_state ISTATE_PROCESSING refcnt 2 transport_state active,stop,fabric_stop
wait for tasks: tmf LUN_RESET with tag 0x0 ref_task_tag 0x0 i_state 34 t_state ISTATE_PROCESSING refcnt 2 transport_state active,stop,fabric_stop
...
INFO: task kworker/0:2:49 blocked for more than 491 seconds.
task:kworker/0:2 state:D stack: 0 pid: 49 ppid: 2 flags:0x00000800
Workqueue: events target_tmr_work [target_core_mod]
Call Trace:
__switch_to+0x2c4/0x470
_schedule+0x314/0x1730
schedule+0x64/0x130
schedule_timeout+0x168/0x430
wait_for_completion+0x140/0x270
target_put_cmd_and_wait+0x64/0xb0 [target_core_mod]
core_tmr_lun_reset+0x30/0xa0 [target_core_mod]
target_tmr_work+0xc8/0x1b0 [target_core_mod]
process_one_work+0x2d4/0x5d0
worker_thread+0x78/0x6c0
To fix this, only add abort to tmr_list if it will be handled by target
core.
Signed-off-by: Dmitry Bogdanov <d.bogdanov@yadro.com>
Link: https://lore.kernel.org/r/20240111125941.8688-1-d.bogdanov@yadro.com
Reviewed-by: Mike Christie <michael.christie@oracle.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Pull SCSI updates from James Bottomley:
"Updates to the usual drivers (ufs, megaraid_sas, lpfc, target, ibmvfc,
scsi_debug) plus the usual assorted minor fixes and updates.
The major change this time around is a prep patch for rethreading of
the driver reset handler API not to take a scsi_cmd structure which
starts to reduce various drivers' dependence on scsi_cmd in error
handling"
* tag 'scsi-misc' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi: (132 commits)
scsi: ufs: core: Leave space for '\0' in utf8 desc string
scsi: ufs: core: Conversion to bool not necessary
scsi: ufs: core: Fix race between force complete and ISR
scsi: megaraid: Fix up debug message in megaraid_abort_and_reset()
scsi: aic79xx: Fix up NULL command in ahd_done()
scsi: message: fusion: Initialize return value in mptfc_bus_reset()
scsi: mpt3sas: Fix loop logic
scsi: snic: Remove useless code in snic_dr_clean_pending_req()
scsi: core: Add comment to target_destroy in scsi_host_template
scsi: core: Clean up scsi_dev_queue_ready()
scsi: pmcraid: Add missing scsi_device_put() in pmcraid_eh_target_reset_handler()
scsi: target: core: Fix kernel-doc comment
scsi: pmcraid: Fix kernel-doc comment
scsi: core: Handle depopulation and restoration in progress
scsi: ufs: core: Add support for parsing OPP
scsi: ufs: core: Add OPP support for scaling clocks and regulators
scsi: ufs: dt-bindings: common: Add OPP table
scsi: scsi_debug: Add param to control sdev's allow_restart
scsi: scsi_debug: Add debugfs interface to fail target reset
scsi: scsi_debug: Add new error injection type: Reset LUN failed
...
This allows userspace to request the fabric drivers do direct submissions
if they support it. With the new device file, submit_type, users can
write 0 - 2 to control how commands are submitted to the backend:
0 - TARGET_FABRIC_DEFAULT_SUBMIT - LIO will use the fabric's default
submission type. This is the default for compat.
1 - TARGET_DIRECT_SUBMIT - LIO will submit the cmd to the backend from the
calling context if the fabric the cmd was received on supports it,
else it will use the fabric's default type.
2 - TARGET_QUEUE_SUBMIT - LIO will queue the cmd to the LIO submission
workqueue which will pass it to the backend.
When using an NVMe drive and vhost-scsi with direct submission we see
around a 20% improvement in 4K I/Os:
fio jobs 1 2 4 8 10
--------------------------------------------------
defer 94K 190K 394K 770K 890K
direct 128K 252K 488K 950K -
And when using the queueing mode, we now no longer see issues like where
the iSCSI tx thread is blocked in the block layer waiting on a tag so it
can't respond to a nop or perform I/Os for other LUs.
Signed-off-by: Mike Christie <michael.christie@oracle.com>
Link: https://lore.kernel.org/r/20230928020907.5730-6-michael.christie@oracle.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
The following call trace shows a deadlock issue due to recursive locking of
mutex "device_mutex". First lock acquire is in target_for_each_device() and
second in target_free_device().
PID: 148266 TASK: ffff8be21ffb5d00 CPU: 10 COMMAND: "iscsi_ttx"
#0 [ffffa2bfc9ec3b18] __schedule at ffffffffa8060e7f
#1 [ffffa2bfc9ec3ba0] schedule at ffffffffa8061224
#2 [ffffa2bfc9ec3bb8] schedule_preempt_disabled at ffffffffa80615ee
#3 [ffffa2bfc9ec3bc8] __mutex_lock at ffffffffa8062fd7
#4 [ffffa2bfc9ec3c40] __mutex_lock_slowpath at ffffffffa80631d3
#5 [ffffa2bfc9ec3c50] mutex_lock at ffffffffa806320c
#6 [ffffa2bfc9ec3c68] target_free_device at ffffffffc0935998 [target_core_mod]
#7 [ffffa2bfc9ec3c90] target_core_dev_release at ffffffffc092f975 [target_core_mod]
#8 [ffffa2bfc9ec3ca0] config_item_put at ffffffffa79d250f
#9 [ffffa2bfc9ec3cd0] config_item_put at ffffffffa79d2583
#10 [ffffa2bfc9ec3ce0] target_devices_idr_iter at ffffffffc0933f3a [target_core_mod]
#11 [ffffa2bfc9ec3d00] idr_for_each at ffffffffa803f6fc
#12 [ffffa2bfc9ec3d60] target_for_each_device at ffffffffc0935670 [target_core_mod]
#13 [ffffa2bfc9ec3d98] transport_deregister_session at ffffffffc0946408 [target_core_mod]
#14 [ffffa2bfc9ec3dc8] iscsit_close_session at ffffffffc09a44a6 [iscsi_target_mod]
#15 [ffffa2bfc9ec3df0] iscsit_close_connection at ffffffffc09a4a88 [iscsi_target_mod]
#16 [ffffa2bfc9ec3df8] finish_task_switch at ffffffffa76e5d07
#17 [ffffa2bfc9ec3e78] iscsit_take_action_for_connection_exit at ffffffffc0991c23 [iscsi_target_mod]
#18 [ffffa2bfc9ec3ea0] iscsi_target_tx_thread at ffffffffc09a403b [iscsi_target_mod]
#19 [ffffa2bfc9ec3f08] kthread at ffffffffa76d8080
#20 [ffffa2bfc9ec3f50] ret_from_fork at ffffffffa8200364
Fixes: 36d4cb460b ("scsi: target: Avoid that EXTENDED COPY commands trigger lock inversion")
Signed-off-by: Junxiao Bi <junxiao.bi@oracle.com>
Link: https://lore.kernel.org/r/20230918225848.66463-1-junxiao.bi@oracle.com
Reviewed-by: Mike Christie <michael.christie@oracle.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Mike Christie <michael.christie@oracle.com> says:
The following patches apply over Martin's 6.4 branches and Linus's tree.
They fix a couple regressions in iscsit that occur when there are TMRs
executing and a connection is closed. It also includes Dimitry's fixes in
related code paths for cmd cleanup when ERL2 is used and the write pending
hang during conn cleanup.
This version of the patchset brings it back to just regressions and fixes
for bugs we have a lot of users hitting. I'm going to fix isert and get it
hooked into iscsit properly in a second patchset, because this one was
getting so large. I've also moved my cleanup type of patches for a 3rd
patchset.
Link: https://lore.kernel.org/r/20230319015620.96006-1-michael.christie@oracle.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
This fixes a bug where an initiator thinks a LUN_RESET has cleaned up
running commands when it hasn't. The bug was added in commit 51ec502a32
("target: Delete tmr from list before processing").
The problem occurs when:
1. We have N I/O cmds running in the target layer spread over 2 sessions.
2. The initiator sends a LUN_RESET for each session.
3. session1's LUN_RESET loops over all the running commands from both
sessions and moves them to its local drain_task_list.
4. session2's LUN_RESET does not see the LUN_RESET from session1 because
the commit above has it remove itself. session2 also does not see any
commands since the other reset moved them off the state lists.
5. sessions2's LUN_RESET will then complete with a successful response.
6. sessions2's inititor believes the running commands on its session are
now cleaned up due to the successful response and cleans up the running
commands from its side. It then restarts them.
7. The commands do eventually complete on the backend and the target
starts to return aborted task statuses for them. The initiator will
either throw a invalid ITT error or might accidentally lookup a new
task if the ITT has been reallocated already.
Fix the bug by reverting the patch, and serialize the execution of
LUN_RESETs and Preempt and Aborts.
Also prevent us from waiting on LUN_RESETs in core_tmr_drain_tmr_list,
because it turns out the original patch fixed a bug that was not
mentioned. For LUN_RESET1 core_tmr_drain_tmr_list can see a second
LUN_RESET and wait on it. Then the second reset will run
core_tmr_drain_tmr_list and see the first reset and wait on it resulting in
a deadlock.
Fixes: 51ec502a32 ("target: Delete tmr from list before processing")
Signed-off-by: Mike Christie <michael.christie@oracle.com>
Link: https://lore.kernel.org/r/20230319015620.96006-8-michael.christie@oracle.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
