Pull SCSI updates from James Bottomley:
"Final round of fixes that came in too late to send in the first
request.
It's nine bug fixes and one version update (because of a bug fix) and
one set of PCI ID additions. There's one bug fix in the core which is
really a one liner (except that an additional sdev pointer was added
for convenience) and the rest are in drivers"
* tag 'scsi-misc' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi:
scsi: target: core: Add TMF to tmr_list handling
scsi: core: Kick the requeue list after inserting when flushing
scsi: fnic: unlock on error path in fnic_queuecommand()
scsi: fcoe: Fix unsigned comparison with zero in store_ctlr_mode()
scsi: mpi3mr: Fix mpi3mr_fw.c kernel-doc warnings
scsi: smartpqi: Bump driver version to 2.1.26-030
scsi: smartpqi: Fix logical volume rescan race condition
scsi: smartpqi: Add new controller PCI IDs
scsi: ufs: qcom: Remove unnecessary goto statement from ufs_qcom_config_esi()
scsi: ufs: core: Remove the ufshcd_hba_exit() call from ufshcd_async_scan()
scsi: ufs: core: Simplify power management during async scan
An abort that is responded to by iSCSI itself is added to tmr_list but does
not go to target core. A LUN_RESET that goes through tmr_list takes a
refcounter on the abort and waits for completion. However, the abort will
be never complete because it was not started in target core.
Unable to locate ITT: 0x05000000 on CID: 0
Unable to locate RefTaskTag: 0x05000000 on CID: 0.
wait_for_tasks: Stopping tmf LUN_RESET with tag 0x0 ref_task_tag 0x0 i_state 34 t_state ISTATE_PROCESSING refcnt 2 transport_state active,stop,fabric_stop
wait for tasks: tmf LUN_RESET with tag 0x0 ref_task_tag 0x0 i_state 34 t_state ISTATE_PROCESSING refcnt 2 transport_state active,stop,fabric_stop
...
INFO: task kworker/0:2:49 blocked for more than 491 seconds.
task:kworker/0:2 state:D stack: 0 pid: 49 ppid: 2 flags:0x00000800
Workqueue: events target_tmr_work [target_core_mod]
Call Trace:
__switch_to+0x2c4/0x470
_schedule+0x314/0x1730
schedule+0x64/0x130
schedule_timeout+0x168/0x430
wait_for_completion+0x140/0x270
target_put_cmd_and_wait+0x64/0xb0 [target_core_mod]
core_tmr_lun_reset+0x30/0xa0 [target_core_mod]
target_tmr_work+0xc8/0x1b0 [target_core_mod]
process_one_work+0x2d4/0x5d0
worker_thread+0x78/0x6c0
To fix this, only add abort to tmr_list if it will be handled by target
core.
Signed-off-by: Dmitry Bogdanov <d.bogdanov@yadro.com>
Link: https://lore.kernel.org/r/20240111125941.8688-1-d.bogdanov@yadro.com
Reviewed-by: Mike Christie <michael.christie@oracle.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Pull SCSI updates from James Bottomley:
"Updates to the usual drivers (ufs, mpi3mr, mpt3sas, lpfc, fnic,
hisi_sas, arcmsr, ) plus the usual assorted minor fixes and updates.
This time around there's only a single line update to the core, so
nothing major and barely anything minor"
* tag 'scsi-misc' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi: (135 commits)
scsi: ufs: core: Simplify ufshcd_auto_hibern8_update()
scsi: ufs: core: Rename ufshcd_auto_hibern8_enable() and make it static
scsi: ufs: qcom: Fix ESI vector mask
scsi: ufs: host: Fix kernel-doc warning
scsi: hisi_sas: Correct the number of global debugfs registers
scsi: hisi_sas: Rollback some operations if FLR failed
scsi: hisi_sas: Check before using pointer variables
scsi: hisi_sas: Replace with standard error code return value
scsi: hisi_sas: Set .phy_attached before notifing phyup event HISI_PHYE_PHY_UP_PM
scsi: ufs: core: Add sysfs node for UFS RTC update
scsi: ufs: core: Add UFS RTC support
scsi: ufs: core: Add ufshcd_is_ufs_dev_busy()
scsi: ufs: qcom: Remove unused definitions
scsi: ufs: qcom: Use ufshcd_rmwl() where applicable
scsi: ufs: qcom: Remove support for host controllers older than v2.0
scsi: ufs: qcom: Simplify ufs_qcom_{assert/deassert}_reset
scsi: ufs: qcom: Initialize cycles_in_1us variable in ufs_qcom_set_core_clk_ctrl()
scsi: ufs: qcom: Sort includes alphabetically
scsi: ufs: qcom: Remove unused ufs_qcom_hosts struct array
scsi: ufs: qcom: Use dev_err_probe() to simplify error handling of devm_gpiod_get_optional()
...
Pull SCSI updates from James Bottomley:
"Updates to the usual drivers (ufs, megaraid_sas, lpfc, target, ibmvfc,
scsi_debug) plus the usual assorted minor fixes and updates.
The major change this time around is a prep patch for rethreading of
the driver reset handler API not to take a scsi_cmd structure which
starts to reduce various drivers' dependence on scsi_cmd in error
handling"
* tag 'scsi-misc' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi: (132 commits)
scsi: ufs: core: Leave space for '\0' in utf8 desc string
scsi: ufs: core: Conversion to bool not necessary
scsi: ufs: core: Fix race between force complete and ISR
scsi: megaraid: Fix up debug message in megaraid_abort_and_reset()
scsi: aic79xx: Fix up NULL command in ahd_done()
scsi: message: fusion: Initialize return value in mptfc_bus_reset()
scsi: mpt3sas: Fix loop logic
scsi: snic: Remove useless code in snic_dr_clean_pending_req()
scsi: core: Add comment to target_destroy in scsi_host_template
scsi: core: Clean up scsi_dev_queue_ready()
scsi: pmcraid: Add missing scsi_device_put() in pmcraid_eh_target_reset_handler()
scsi: target: core: Fix kernel-doc comment
scsi: pmcraid: Fix kernel-doc comment
scsi: core: Handle depopulation and restoration in progress
scsi: ufs: core: Add support for parsing OPP
scsi: ufs: core: Add OPP support for scaling clocks and regulators
scsi: ufs: dt-bindings: common: Add OPP table
scsi: scsi_debug: Add param to control sdev's allow_restart
scsi: scsi_debug: Add debugfs interface to fail target reset
scsi: scsi_debug: Add new error injection type: Reset LUN failed
...
Mike Christie <michael.christie@oracle.com> says:
The following patches were made over Linus's tree but apply over
Martin's branches. They allow userspace to configure how fabric
drivers submit cmds to backend drivers.
Right now loop and vhost use a worker thread, and the other drivers
submit from the contexts they receive/process the cmd from. For
multiple LUN cases where the target can queue more cmds than the
backend can handle then deferring to a worker thread is safest because
the backend driver can block when doing things like waiting for a free
request/tag. Deferring also helps when the target has to handle
transport level requests from the recv context.
For cases where the backend devices can queue everything the target
sends, then there is no need to defer to a workqueue and you can see a
perf boost of up to 26% for small IO workloads. For a nvme device and
vhost-scsi I can see with 4K IOs:
fio jobs 1 2 4 8 10
--------------------------------------------------
workqueue
submit 94K 190K 394K 770K 890K
direct
submit 128K 252K 488K 950K -
Link: https://lore.kernel.org/r/1b1f7a5c-0988-45f9-b103-dfed2c0405b1@oracle.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
This exports the fabric driver's direct submit settings, so users know what
the driver supports. It will be helpful when they are exporting a device
through different targets and one doesn't support direct submission.
The new files allow the fabric to report what submission types they default
to and if they support direct submission:
default_submit_type:
1 - TARGET_DIRECT_SUBMIT - If the user has not requested a specific value
then the fabric requests direct submission.
2 - TARGET_QUEUE_SUBMIT - If the user has not requested a specific value
then the fabric requests queued submission.
Note that these fabric values are based on what the fabric driver currently
defaults to for compat with exiting setups.
direct_submit_supported:
0 - The fabric does not support direct submission.
1 - The fabric supports direct submission.
Signed-off-by: Mike Christie <michael.christie@oracle.com>
Link: https://lore.kernel.org/r/20230928020907.5730-8-michael.christie@oracle.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
This allows userspace to request the fabric drivers do direct submissions
if they support it. With the new device file, submit_type, users can
write 0 - 2 to control how commands are submitted to the backend:
0 - TARGET_FABRIC_DEFAULT_SUBMIT - LIO will use the fabric's default
submission type. This is the default for compat.
1 - TARGET_DIRECT_SUBMIT - LIO will submit the cmd to the backend from the
calling context if the fabric the cmd was received on supports it,
else it will use the fabric's default type.
2 - TARGET_QUEUE_SUBMIT - LIO will queue the cmd to the LIO submission
workqueue which will pass it to the backend.
When using an NVMe drive and vhost-scsi with direct submission we see
around a 20% improvement in 4K I/Os:
fio jobs 1 2 4 8 10
--------------------------------------------------
defer 94K 190K 394K 770K 890K
direct 128K 252K 488K 950K -
And when using the queueing mode, we now no longer see issues like where
the iSCSI tx thread is blocked in the block layer waiting on a tag so it
can't respond to a nop or perform I/Os for other LUs.
Signed-off-by: Mike Christie <michael.christie@oracle.com>
Link: https://lore.kernel.org/r/20230928020907.5730-6-michael.christie@oracle.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Move the hack to clear some buffers to transport_handle_cdb_direct() so we
can eventually merge transport_handle_cdb_direct() and target_submit().
This also fixes up the comment so it's clear it was only for udev and
reflects that the referenced function does not exist and we now allow more
than 1 page for control CDBs.
Signed-off-by: Mike Christie <michael.christie@oracle.com>
Link: https://lore.kernel.org/r/20230928020907.5730-4-michael.christie@oracle.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Move core_alua_check_nonop_delay() to transport_handle_cdb_direct() so the
iSCSI target driver doesn't have to call as many core functions
directly. We will eventually merge transport_handle_cdb_direct and
target_submit so iSCSI and the other drivers call a common function.
It will also be helpful as preparation for future changes which allow the
iSCSI target to defer command submission to the LIO submission workqueue,
because we will have a common submission function for that which will be
based on transport_handle_cdb_direct()/target_submit().
Signed-off-by: Mike Christie <michael.christie@oracle.com>
Link: https://lore.kernel.org/r/20230928020907.5730-3-michael.christie@oracle.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
In some cases, like with multiple LUN targets or where the target has to
respond to transport level requests from the receiving context it can be
better to defer cmd submission to a helper thread. If the backend driver
blocks on something like request/tag allocation it can block the entire
target submission path and other LUs and transport IO on that session.
In other cases like single LUN targets with storage that can support all
the commands that the target can queue, then it's best to submit the cmd
to the backend from the target's cmd receiving context.
Subsequent commits will allow the user to config what they prefer, but
drivers like loop can't directly submit because they can be called from a
context that can't sleep. And, drivers like vhost-scsi can support direct
submission, but need to keep their default behavior of deferring execution
to avoid possible regressions where the backend can block.
Make the drivers tell LIO core if they support direct submissions and their
current default, so we can prevent users from misconfiguring the system and
initialize devices correctly.
Signed-off-by: Mike Christie <michael.christie@oracle.com>
Link: https://lore.kernel.org/r/20230928020907.5730-2-michael.christie@oracle.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
The following call trace shows a deadlock issue due to recursive locking of
mutex "device_mutex". First lock acquire is in target_for_each_device() and
second in target_free_device().
PID: 148266 TASK: ffff8be21ffb5d00 CPU: 10 COMMAND: "iscsi_ttx"
#0 [ffffa2bfc9ec3b18] __schedule at ffffffffa8060e7f
#1 [ffffa2bfc9ec3ba0] schedule at ffffffffa8061224
#2 [ffffa2bfc9ec3bb8] schedule_preempt_disabled at ffffffffa80615ee
#3 [ffffa2bfc9ec3bc8] __mutex_lock at ffffffffa8062fd7
#4 [ffffa2bfc9ec3c40] __mutex_lock_slowpath at ffffffffa80631d3
#5 [ffffa2bfc9ec3c50] mutex_lock at ffffffffa806320c
#6 [ffffa2bfc9ec3c68] target_free_device at ffffffffc0935998 [target_core_mod]
#7 [ffffa2bfc9ec3c90] target_core_dev_release at ffffffffc092f975 [target_core_mod]
#8 [ffffa2bfc9ec3ca0] config_item_put at ffffffffa79d250f
#9 [ffffa2bfc9ec3cd0] config_item_put at ffffffffa79d2583
#10 [ffffa2bfc9ec3ce0] target_devices_idr_iter at ffffffffc0933f3a [target_core_mod]
#11 [ffffa2bfc9ec3d00] idr_for_each at ffffffffa803f6fc
#12 [ffffa2bfc9ec3d60] target_for_each_device at ffffffffc0935670 [target_core_mod]
#13 [ffffa2bfc9ec3d98] transport_deregister_session at ffffffffc0946408 [target_core_mod]
#14 [ffffa2bfc9ec3dc8] iscsit_close_session at ffffffffc09a44a6 [iscsi_target_mod]
#15 [ffffa2bfc9ec3df0] iscsit_close_connection at ffffffffc09a4a88 [iscsi_target_mod]
#16 [ffffa2bfc9ec3df8] finish_task_switch at ffffffffa76e5d07
#17 [ffffa2bfc9ec3e78] iscsit_take_action_for_connection_exit at ffffffffc0991c23 [iscsi_target_mod]
#18 [ffffa2bfc9ec3ea0] iscsi_target_tx_thread at ffffffffc09a403b [iscsi_target_mod]
#19 [ffffa2bfc9ec3f08] kthread at ffffffffa76d8080
#20 [ffffa2bfc9ec3f50] ret_from_fork at ffffffffa8200364
Fixes: 36d4cb460b ("scsi: target: Avoid that EXTENDED COPY commands trigger lock inversion")
Signed-off-by: Junxiao Bi <junxiao.bi@oracle.com>
Link: https://lore.kernel.org/r/20230918225848.66463-1-junxiao.bi@oracle.com
Reviewed-by: Mike Christie <michael.christie@oracle.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
The write back throttling (WBT) code checks if REQ_SYNC | REQ_IDLE is set
to determine if a write is O_DIRECT vs buffered. If the bits are not set
then it assumes it's a buffered write and will throttle LIO if we hit
certain metrics. LIO itself is not using the buffer cache and is doing
direct I/O, so this has us set the direct bits so we are not throttled.
When the initiator application is doing direct I/O this can greatly improve
performance. It depends on the backend device but we have seen where the
WBT code is throttling writes to only 20K IOPs with 4K I/Os when the device
can support 100K+.
Signed-off-by: Mike Christie <michael.christie@oracle.com>
Link: https://lore.kernel.org/r/20230817192902.346791-1-michael.christie@oracle.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
