Replace the deprecated[1] strncpy() with strscpy() in
transport_dump_vpd_proto_id(), transport_dump_vpd_assoc(),
transport_dump_vpd_ident_type(), and transport_dump_vpd_ident().
All four functions follow the same pattern: a local
buf[VPD_TMP_BUF_SIZE] (254 bytes) is zeroed with memset(), populated
via sprintf()/snprintf() (always NUL-terminated), then conditionally
copied to p_buf with strncpy(). The p_buf destination is used as a
C string by all callers in target_core_configfs.c: strlen(buf) and
sprintf(page+len, "%s", buf) to build sysfs output.
NUL-padding is not required: callers in target_core_configfs.c
pre-zero p_buf with memset() or initializer before calling these
functions, and consume p_buf only as a NUL-terminated C string via
strlen() and "%s", never exposing trailing bytes.
No behavioral change: the source buf is always NUL-terminated and
shorter than VPD_TMP_BUF_SIZE, so strscpy() produces identical output.
Link: https://github.com/KSPP/linux/issues/90 [1]
Signed-off-by: Kees Cook <kees@kernel.org>
Reviewed-by: Kees Cook <keescook@chromium.org>
Link: https://patch.msgid.link/20260323171311.work.101-kees@kernel.org
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
There were two copy and paste type of errors where I swapped 'complete'
with 'submit' names. Use the correct variables and definitions.
This problem was found by Dmitry Bogdanov and this patch builds on top of
his patch to fix a second instance that was found.
Fixes: 06933066d8 ("scsi: target: Add support for completing commands from backend context")
Signed-off-by: Dmitry Bogdanov <d.bogdanov@yadro.com>
[mnc: Fixed second instance]
Signed-off-by: Mike Christie <michael.christie@oracle.com>
Link: https://patch.msgid.link/20260307220630.131008-1-michael.christie@oracle.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
To complete a command several drivers just drop their reference and add
it to list to be processed by a driver specific thread. So there's no
need to go from backend context to the LIO thread then to the driver's
thread. When avoiding the LIO thread, IOPS can increase from 20-30% for
workloads like:
fio --filename=/dev/sdb --direct=1 --rw=randrw --bs=8K \
--ioengine=libaio --iodepth=128 --numjobs=$jobs
where increasing jobs increases the performance improvement (this is
using NVMe drives with LIO's submit_type=1 to directly submit).
Add the infrastructure so drivers and userspace can control how to
complete a command like is done for the submission path. In this commit
there is no behavior change and we continue to defer to the LIO
workqueue thread. In the subsequent commits we will allow drivers to
report what they support and allow userspace to control the behavior.
Signed-off-by: Mike Christie <michael.christie@oracle.com>
Link: https://patch.msgid.link/20260222232946.7637-2-michael.christie@oracle.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
This was done entirely with mindless brute force, using
git grep -l '\<k[vmz]*alloc_objs*(.*, GFP_KERNEL)' |
xargs sed -i 's/\(alloc_objs*(.*\), GFP_KERNEL)/\1)/'
to convert the new alloc_obj() users that had a simple GFP_KERNEL
argument to just drop that argument.
Note that due to the extreme simplicity of the scripting, any slightly
more complex cases spread over multiple lines would not be triggered:
they definitely exist, but this covers the vast bulk of the cases, and
the resulting diff is also then easier to check automatically.
For the same reason the 'flex' versions will be done as a separate
conversion.
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
This is the result of running the Coccinelle script from
scripts/coccinelle/api/kmalloc_objs.cocci. The script is designed to
avoid scalar types (which need careful case-by-case checking), and
instead replace kmalloc-family calls that allocate struct or union
object instances:
Single allocations: kmalloc(sizeof(TYPE), ...)
are replaced with: kmalloc_obj(TYPE, ...)
Array allocations: kmalloc_array(COUNT, sizeof(TYPE), ...)
are replaced with: kmalloc_objs(TYPE, COUNT, ...)
Flex array allocations: kmalloc(struct_size(PTR, FAM, COUNT), ...)
are replaced with: kmalloc_flex(*PTR, FAM, COUNT, ...)
(where TYPE may also be *VAR)
The resulting allocations no longer return "void *", instead returning
"TYPE *".
Signed-off-by: Kees Cook <kees@kernel.org>
Marco Crivellari <marco.crivellari@suse.com> says:
Hi,
=== Current situation: problems ===
Let's consider a nohz_full system with isolated CPUs: wq_unbound_cpumask is
set to the housekeeping CPUs, for !WQ_UNBOUND the local CPU is selected.
This leads to different scenarios if a work item is scheduled on an
isolated CPU where "delay" value is 0 or greater then 0:
schedule_delayed_work(, 0);
This will be handled by __queue_work() that will queue the work item on the
current local (isolated) CPU, while:
schedule_delayed_work(, 1);
Will move the timer on an housekeeping CPU, and schedule the work there.
Currently if a user enqueue a work item using schedule_delayed_work() the
used wq is "system_wq" (per-cpu wq) while queue_delayed_work() use
WORK_CPU_UNBOUND (used when a cpu is not specified). The same applies to
schedule_work() that is using system_wq and queue_work(), that makes use
again of WORK_CPU_UNBOUND.
This lack of consistency cannot be addressed without refactoring the API.
=== Recent changes to the WQ API ===
The following, address the recent changes in the Workqueue API:
- commit 128ea9f6cc ("workqueue: Add system_percpu_wq and system_dfl_wq")
- commit 930c2ea566 ("workqueue: Add new WQ_PERCPU flag")
The old workqueues will be removed in a future release cycle.
=== Introduced Changes by this series ===
1) [P 1] Replace uses of system_wq and system_unbound_wq
system_unbound_wq is to be used when locality is not required.
Because of that, system_unbound_wq has been replaced with
system_dfl_wq, to make clear it should be used when locality
is not required.
2) [P 2-3-4] WQ_PERCPU added to alloc_workqueue()
This change adds a new WQ_PERCPU flag to explicitly request
alloc_workqueue() to be per-cpu when WQ_UNBOUND has not been specified.
Thanks!
Link: https://patch.msgid.link/20251031095643.74246-1-marco.crivellari@suse.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Currently if a user enqueues a work item using schedule_delayed_work()
the used wq is "system_wq" (per-cpu wq) while queue_delayed_work() use
WORK_CPU_UNBOUND (used when a cpu is not specified). The same applies to
schedule_work() that is using system_wq and queue_work(), that makes use
again of WORK_CPU_UNBOUND. This lack of consistency cannot be addressed
without refactoring the API.
alloc_workqueue() treats all queues as per-CPU by default, while unbound
workqueues must opt-in via WQ_UNBOUND.
This default is suboptimal: most workloads benefit from unbound queues,
allowing the scheduler to place worker threads where they’re needed and
reducing noise when CPUs are isolated.
This continues the effort to refactor workqueue APIs, which began with
the introduction of new workqueues and a new alloc_workqueue flag in:
commit 128ea9f6cc ("workqueue: Add system_percpu_wq and system_dfl_wq")
commit 930c2ea566 ("workqueue: Add new WQ_PERCPU flag")
This change adds a new WQ_PERCPU flag to explicitly request
alloc_workqueue() to be per-cpu when WQ_UNBOUND has not been specified.
With the introduction of the WQ_PERCPU flag (equivalent to !WQ_UNBOUND),
any alloc_workqueue() caller that doesn’t explicitly specify WQ_UNBOUND
must now use WQ_PERCPU.
Once migration is complete, WQ_UNBOUND can be removed and unbound will
become the implicit default.
Suggested-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Marco Crivellari <marco.crivellari@suse.com>
Link: https://patch.msgid.link/20251107154008.304127-1-marco.crivellari@suse.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
The atomic use in the main I/O path is causing perf issues when using
higher performance backend devices and multiple queues (more than
10 when using vhost-scsi) like with this fio workload:
[global]
bs=4K
iodepth=128
direct=1
ioengine=libaio
group_reporting
time_based
runtime=120
name=standard-iops
rw=randread
numjobs=16
cpus_allowed=0-15
To fix this issue, move the LUN stats to per CPU.
Note: I forgot to include this patch with the delayed/ordered per CPU
tracking and per device/device entry per CPU stats. With this patch you
get the full 33% improvements when using fast backends, multiple queues
and multiple IO submiters.
Signed-off-by: Mike Christie <michael.christie@oracle.com>
Reviewed-by: Dmitry Bogdanov <d.bogdanov@yadro.com>
Link: https://patch.msgid.link/20250917221338.14813-4-michael.christie@oracle.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
asm/unaligned.h is always an include of asm-generic/unaligned.h;
might as well move that thing to linux/unaligned.h and include
that - there's nothing arch-specific in that header.
auto-generated by the following:
for i in `git grep -l -w asm/unaligned.h`; do
sed -i -e "s/asm\/unaligned.h/linux\/unaligned.h/" $i
done
for i in `git grep -l -w asm-generic/unaligned.h`; do
sed -i -e "s/asm-generic\/unaligned.h/linux\/unaligned.h/" $i
done
git mv include/asm-generic/unaligned.h include/linux/unaligned.h
git mv tools/include/asm-generic/unaligned.h tools/include/linux/unaligned.h
sed -i -e "/unaligned.h/d" include/asm-generic/Kbuild
sed -i -e "s/__ASM_GENERIC/__LINUX/" include/linux/unaligned.h tools/include/linux/unaligned.h
An abort that is responded to by iSCSI itself is added to tmr_list but does
not go to target core. A LUN_RESET that goes through tmr_list takes a
refcounter on the abort and waits for completion. However, the abort will
be never complete because it was not started in target core.
Unable to locate ITT: 0x05000000 on CID: 0
Unable to locate RefTaskTag: 0x05000000 on CID: 0.
wait_for_tasks: Stopping tmf LUN_RESET with tag 0x0 ref_task_tag 0x0 i_state 34 t_state ISTATE_PROCESSING refcnt 2 transport_state active,stop,fabric_stop
wait for tasks: tmf LUN_RESET with tag 0x0 ref_task_tag 0x0 i_state 34 t_state ISTATE_PROCESSING refcnt 2 transport_state active,stop,fabric_stop
...
INFO: task kworker/0:2:49 blocked for more than 491 seconds.
task:kworker/0:2 state:D stack: 0 pid: 49 ppid: 2 flags:0x00000800
Workqueue: events target_tmr_work [target_core_mod]
Call Trace:
__switch_to+0x2c4/0x470
_schedule+0x314/0x1730
schedule+0x64/0x130
schedule_timeout+0x168/0x430
wait_for_completion+0x140/0x270
target_put_cmd_and_wait+0x64/0xb0 [target_core_mod]
core_tmr_lun_reset+0x30/0xa0 [target_core_mod]
target_tmr_work+0xc8/0x1b0 [target_core_mod]
process_one_work+0x2d4/0x5d0
worker_thread+0x78/0x6c0
To fix this, only add abort to tmr_list if it will be handled by target
core.
Signed-off-by: Dmitry Bogdanov <d.bogdanov@yadro.com>
Link: https://lore.kernel.org/r/20240111125941.8688-1-d.bogdanov@yadro.com
Reviewed-by: Mike Christie <michael.christie@oracle.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Mike Christie <michael.christie@oracle.com> says:
The following patches were made over Linus's tree but apply over
Martin's branches. They allow userspace to configure how fabric
drivers submit cmds to backend drivers.
Right now loop and vhost use a worker thread, and the other drivers
submit from the contexts they receive/process the cmd from. For
multiple LUN cases where the target can queue more cmds than the
backend can handle then deferring to a worker thread is safest because
the backend driver can block when doing things like waiting for a free
request/tag. Deferring also helps when the target has to handle
transport level requests from the recv context.
For cases where the backend devices can queue everything the target
sends, then there is no need to defer to a workqueue and you can see a
perf boost of up to 26% for small IO workloads. For a nvme device and
vhost-scsi I can see with 4K IOs:
fio jobs 1 2 4 8 10
--------------------------------------------------
workqueue
submit 94K 190K 394K 770K 890K
direct
submit 128K 252K 488K 950K -
Link: https://lore.kernel.org/r/1b1f7a5c-0988-45f9-b103-dfed2c0405b1@oracle.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
This allows userspace to request the fabric drivers do direct submissions
if they support it. With the new device file, submit_type, users can
write 0 - 2 to control how commands are submitted to the backend:
0 - TARGET_FABRIC_DEFAULT_SUBMIT - LIO will use the fabric's default
submission type. This is the default for compat.
1 - TARGET_DIRECT_SUBMIT - LIO will submit the cmd to the backend from the
calling context if the fabric the cmd was received on supports it,
else it will use the fabric's default type.
2 - TARGET_QUEUE_SUBMIT - LIO will queue the cmd to the LIO submission
workqueue which will pass it to the backend.
When using an NVMe drive and vhost-scsi with direct submission we see
around a 20% improvement in 4K I/Os:
fio jobs 1 2 4 8 10
--------------------------------------------------
defer 94K 190K 394K 770K 890K
direct 128K 252K 488K 950K -
And when using the queueing mode, we now no longer see issues like where
the iSCSI tx thread is blocked in the block layer waiting on a tag so it
can't respond to a nop or perform I/Os for other LUs.
Signed-off-by: Mike Christie <michael.christie@oracle.com>
Link: https://lore.kernel.org/r/20230928020907.5730-6-michael.christie@oracle.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Move the hack to clear some buffers to transport_handle_cdb_direct() so we
can eventually merge transport_handle_cdb_direct() and target_submit().
This also fixes up the comment so it's clear it was only for udev and
reflects that the referenced function does not exist and we now allow more
than 1 page for control CDBs.
Signed-off-by: Mike Christie <michael.christie@oracle.com>
Link: https://lore.kernel.org/r/20230928020907.5730-4-michael.christie@oracle.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Move core_alua_check_nonop_delay() to transport_handle_cdb_direct() so the
iSCSI target driver doesn't have to call as many core functions
directly. We will eventually merge transport_handle_cdb_direct and
target_submit so iSCSI and the other drivers call a common function.
It will also be helpful as preparation for future changes which allow the
iSCSI target to defer command submission to the LIO submission workqueue,
because we will have a common submission function for that which will be
based on transport_handle_cdb_direct()/target_submit().
Signed-off-by: Mike Christie <michael.christie@oracle.com>
Link: https://lore.kernel.org/r/20230928020907.5730-3-michael.christie@oracle.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
In the error exits in target_setup_session(), if a branch is taken to
free_sess: transport_free_session() may call to target_free_cmd_counter()
and then fall through to call target_free_cmd_counter() a second time.
This can, and does, sometimes cause seg faults since the data field in
cmd_cnt->refcnt has been freed in the first call.
Fix this problem by simply returning after the call to
transport_free_session(). The second call is redundant for those cases.
Fixes: 4edba7e4a8 ("scsi: target: Move cmd counter allocation")
Signed-off-by: Bob Pearson <rpearsonhpe@gmail.com>
Link: https://lore.kernel.org/r/20230613144259.12890-1-rpearsonhpe@gmail.com
Reviewed-by: Mike Christie <michael.christie@oracle.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
iSCSI needs to allocate its cmd counter per connection for MCS support
where we need to stop and wait on commands running on a connection instead
of per session. This moves the cmd counter allocation to
target_setup_session() which is used by drivers that need the stop+wait
behavior per session.
xcopy doesn't need stop+wait at all, so we will be OK moving the cmd
counter allocation outside of transport_init_session().
Signed-off-by: Mike Christie <michael.christie@oracle.com>
Link: https://lore.kernel.org/r/20230319015620.96006-3-michael.christie@oracle.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
iSCSI needs to wait on outstanding commands like how SRP and the FC/FCoE
drivers do. It can't use target_stop_session() because for MCS support we
can't stop the entire session during recovery because if other connections
are OK then we want to be able to continue to execute I/O on them.
Move the per session cmd counters to a new struct so iSCSI can allocate
them per connection. The xcopy code can also just not allocate in the
future since it doesn't need to track commands.
Signed-off-by: Mike Christie <michael.christie@oracle.com>
Link: https://lore.kernel.org/r/20230319015620.96006-2-michael.christie@oracle.com
Reviewed-by: Maurizio Lombardi <mlombard@redhat.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
