This patch has the SW FCoE driver and the bnx2fc
driver make use of the new fcoe_sysfs API added
earlier in this patch series.
After this patch a fcoe_ctlr_device is allocated with
private data in this order.
+------------------+ +------------------+
| fcoe_ctlr_device | | fcoe_ctlr_device |
+------------------+ +------------------+
| fcoe_ctlr | | fcoe_ctlr |
+------------------+ +------------------+
| fcoe_interface | | bnx2fc_interface |
+------------------+ +------------------+
libfcoe also takes part in this new model since it
discovers and manages fcoe_fcf instances. The memory
allocation is different for FCFs. I didn't want to
impact libfcoe's fcoe_fcf processing, so this patch
creates fcoe_fcf_device instances for each discovered
fcoe_fcf. The two are paired using a (void * priv)
member of the fcoe_ctlr_device. This allows libfcoe
to continue maintaining its list of fcoe_fcf instances
and simply attaches and detaches them from existing
or new fcoe_fcf_device instances.
Signed-off-by: Robert Love <robert.w.love@intel.com>
Tested-by: Ross Brattain <ross.b.brattain@intel.com>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>
This patch adds a 'fcoe bus' infrastructure to the kernel
that is driven by changes to libfcoe which allow LLDs to
present FIP (FCoE Initialization Protocol) discovered
entities and their attributes to user space via sysfs.
This patch adds the following APIs-
fcoe_ctlr_device_add
fcoe_ctlr_device_delete
fcoe_fcf_device_add
fcoe_fcf_device_delete
They allow the LLD to expose the FCoE ENode Controller
and any discovered FCFs (Fibre Channel Forwarders, e.g.
FCoE switches) to the user. Each of these new devices
has their own bus_type so that they are grouped together
for easy lookup from a user space application. Each
new class has an attribute_group to expose attributes
for any created instances. The attributes are-
fcoe_ctlr_device
* fcf_dev_loss_tmo
* lesb_link_fail
* lesb_vlink_fail
* lesb_miss_fka
* lesb_symb_err
* lesb_err_block
* lesb_fcs_error
fcoe_fcf_device
* fabric_name
* switch_name
* priority
* selected
* fc_map
* vfid
* mac
* fka_peroid
* fabric_state
* dev_loss_tmo
A device loss infrastructre similar to the FC Transport's
is also added by this patch. It is nice to have so that a
link flapping adapter doesn't continually advance the count
used to identify the discovered FCF. FCFs will exist in a
"Disconnected" state until either the timer expires or the
FCF is rediscovered and becomes "Connected."
This patch generates a few checkpatch.pl WARNINGS that
I'm not sure what to do about. They're macros modeled
around the FC Transport attribute building macros, which
have the same 'feature' where the caller can ommit a cast
in the argument list and no cast occurs in the code. I'm
not sure how to keep the code condensed while keeping the
macros. Any advice would be appreciated.
Signed-off-by: Robert Love <robert.w.love@intel.com>
Tested-by: Ross Brattain <ross.b.brattain@intel.com>
Acked-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>
Currently the fcoe_ctlr associated with an interface is allocated
as a member of struct fcoe_interface. This causes problems when
attempting to use the new fcoe_sysfs APIs which allow us to allocate
the fcoe_interface as private data to the fcoe_ctlr_device instance.
The problem is that libfcoe wants to be able use pointer math to find a
fcoe_ctlr's fcoe_ctlr_device as well as finding a fcoe_ctlr_device's
assocated fcoe_ctlr. To do this we need to allocate the
fcoe_ctlr_device, with private data for the LLD. The private data
contains the fcoe_ctlr and its private data is the fcoe_interface.
This patch only allocates the fcoe_interface with the fcoe_ctlr, the
fcoe_ctlr_device will be added in a later patch, which will complete
the below diagram-
+------------------+
| fcoe_ctlr_device |
+------------------+
| fcoe_ctlr |
+------------------+
| fcoe_interface |
+------------------+
This prep work will allow us to go from a fcoe_ctlr_device instance
to its fcoe_ctlr as well as from a fcoe_ctlr to its fcoe_ctlr_device
once the fcoe_sysfs API is in use (later patches in this series).
Signed-off-by: Robert Love <robert.w.love@intel.com>
Tested-by: Ross Brattain <ross.b.brattain@intel.com>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>
FC-BB-6 v1.04 7.9.8.14 N_Port_ID Beacon:
"A N_Port_ID Beacon is multicast and uses the VN_Port MAC address as source
address."
Currently, libfcoe is using ENode MAC, this seems ok and functionality wise
not a problem in my back to back testing setup, however, just fix this to
make libfcoe VN2VN support more spec compliant.
Signed-off-by: Yi Zou <yi.zou@intel.com>
Signed-off-by: Robert Love <robert.w.love@intel.com>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>
The rtnl_mutex was held to protect calls to dev_uc_add
and dev_uc_del. Holding rtnl is not required as those
functions make use of the netif_addr_lock* API to
protect the MAC changing.
This change fixes the following regression by removing
the rtnl usage when fcoe_update_src_mac is called.
https://bugzilla.kernel.org/show_bug.cgi?id=42918
the existing dependency chain (in reverse order) is:
-> #1 (&fip->ctlr_mutex){+.+...}:
[<c1091f70>] lock_acquire+0x80/0x1b0
[<c147655d>] mutex_lock_nested+0x6d/0x340
[<f8970c32>] fcoe_ctlr_link_up+0x22/0x180 [libfcoe]
[<f894620e>] fcoe_create+0x47e/0x6e0 [fcoe]
[<f8973dd3>] fcoe_transport_create+0x143/0x250 [libfcoe]
[<c10527e0>] param_attr_store+0x30/0x60
[<c1052696>] module_attr_store+0x26/0x40
[<c11a201e>] sysfs_write_file+0xae/0x100
[<c11449df>] vfs_write+0x8f/0x160
[<c1144cbd>] sys_write+0x3d/0x70
[<c147a0c4>] syscall_call+0x7/0xb
-> #0 (rtnl_mutex){+.+.+.}:
[<c109164b>] __lock_acquire+0x140b/0x1720
[<c1091f70>] lock_acquire+0x80/0x1b0
[<c147655d>] mutex_lock_nested+0x6d/0x340
[<c13a10c4>] rtnl_lock+0x14/0x20
[<f89445ac>] fcoe_update_src_mac+0x2c/0xb0 [fcoe]
[<f8971712>] fcoe_ctlr_timer_work+0x712/0xb60 [libfcoe]
[<c104fb69>] process_one_work+0x179/0x5d0
[<c10502f1>] worker_thread+0x121/0x2d0
[<c10550ed>] kthread+0x7d/0x90
[<c1481a82>] kernel_thread_helper+0x6/0x10
other info that might help us debug this:
Possible unsafe locking scenario:
CPU0 CPU1
---- ----
lock(&fip->ctlr_mutex);
lock(rtnl_mutex);
lock(&fip->ctlr_mutex);
lock(rtnl_mutex);
*** DEADLOCK ***
Signed-off-by: Robert Love <robert.w.love@intel.com>
Tested-by: Bart Van Assche <bvanassche@acm.org>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>
Remove lport from net device and then do synchronize net device to flush
inflight rx frames for the lport before doing fcoe_percpu_clean.
In case of master lport, remove all rx packet handlers completely and
then only do fcoe_percpu_clean. This required splitting fcoe_interface_cleanup
to do remove part separately and for that added func fcoe_interface_remove
and then call it from fcoe_if_destory before doing fcoe_percpu_clean.
However if fcoe_interface_remove() is already called then
don't call again from fcoe_interface_cleanup() to preserve its
existing flows.
This patch along with Neil's other patch to avoid soft irq context
on ingress will avoid passing up frames on disabled lport as
discussed in this mail thread:-
http://lists.open-fcoe.org/pipermail/devel/2012-February/011947.html
Signed-off-by: Vasu Dev <vasu.dev@intel.com>
Acked-by: Neil Horman <nhorman@tuxdriver.com>
Tested-by: Ross Brattain <ross.b.brattain@intel.com>
Signed-off-by: Robert Love <robert.w.love@intel.com>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>
Pull SCSI updates from James Bottomley:
"This is primarily another round of driver updates (lpfc, bfa, fcoe,
ipr) plus a new ufshcd driver. There shouldn't be anything
controversial in here (The final deletion of scsi proc_ops which
caused some build breakage has been held over until the next merge
window to give us more time to stabilise it).
I'm afraid, with me moving continents at exactly the wrong time,
anything submitted after the merge window opened has been held over to
the next merge window."
* tag 'scsi-misc' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi-misc-2.6: (63 commits)
[SCSI] ipr: Driver version 2.5.3
[SCSI] ipr: Increase alignment boundary of command blocks
[SCSI] ipr: Increase max concurrent oustanding commands
[SCSI] ipr: Remove unnecessary memory barriers
[SCSI] ipr: Remove unnecessary interrupt clearing on new adapters
[SCSI] ipr: Fix target id allocation re-use problem
[SCSI] atp870u, mpt2sas, qla4xxx use pci_dev->revision
[SCSI] fcoe: Drop the rtnl_mutex before calling fcoe_ctlr_link_up
[SCSI] bfa: Update the driver version to 3.0.23.0
[SCSI] bfa: BSG and User interface fixes.
[SCSI] bfa: Fix to avoid vport delete hang on request queue full scenario.
[SCSI] bfa: Move service parameter programming logic into firmware.
[SCSI] bfa: Revised Fabric Assigned Address(FAA) feature implementation.
[SCSI] bfa: Flash controller IOC pll init fixes.
[SCSI] bfa: Serialize the IOC hw semaphore unlock logic.
[SCSI] bfa: Modify ISR to process pending completions
[SCSI] bfa: Add fc host issue lip support
[SCSI] mpt2sas: remove extraneous sas_log_info messages
[SCSI] libfc: fcoe_transport_create fails in single-CPU environment
[SCSI] fcoe: reduce contention for fcoe_rx_list lock [v2]
...
The rtnl_lock is primarily used to serialize networking
driver changes as well as to ensure that a networking driver
is not removed when making changes to it. fcoe also uses
the rtnl_lock to protect the fcoe hostlist.
fcoe_create holds the rtnl_lock over the entirity of the
routine including a the call to fcoe_ctlr_link_up.
This causes the below deadlock because fcoe_ctlr_link_up
acquires the fcoe_ctlr ctlr_mutex and this deadlocks with
a libfcoe thread that acquires the fcoe_ctlr ctlr_mutex and
then the rtnl_lock (to update a MAC address).
This patch drops the rtnl_lock before calling
fcoe_ctlr_link_up and therefore the deadlock is prevented.
https://bugzilla.kernel.org/show_bug.cgi?id=42918
the existing dependency chain (in reverse order) is:
-> #1 (&fip->ctlr_mutex){+.+...}:
[<c1091f70>] lock_acquire+0x80/0x1b0
[<c147655d>] mutex_lock_nested+0x6d/0x340
[<f8970c32>] fcoe_ctlr_link_up+0x22/0x180 [libfcoe]
[<f894620e>] fcoe_create+0x47e/0x6e0 [fcoe]
[<f8973dd3>] fcoe_transport_create+0x143/0x250 [libfcoe]
[<c10527e0>] param_attr_store+0x30/0x60
[<c1052696>] module_attr_store+0x26/0x40
[<c11a201e>] sysfs_write_file+0xae/0x100
[<c11449df>] vfs_write+0x8f/0x160
[<c1144cbd>] sys_write+0x3d/0x70
[<c147a0c4>] syscall_call+0x7/0xb
-> #0 (rtnl_mutex){+.+.+.}:
[<c109164b>] __lock_acquire+0x140b/0x1720
[<c1091f70>] lock_acquire+0x80/0x1b0
[<c147655d>] mutex_lock_nested+0x6d/0x340
[<c13a10c4>] rtnl_lock+0x14/0x20
[<f89445ac>] fcoe_update_src_mac+0x2c/0xb0 [fcoe]
[<f8971712>] fcoe_ctlr_timer_work+0x712/0xb60 [libfcoe]
[<c104fb69>] process_one_work+0x179/0x5d0
[<c10502f1>] worker_thread+0x121/0x2d0
[<c10550ed>] kthread+0x7d/0x90
[<c1481a82>] kernel_thread_helper+0x6/0x10
other info that might help us debug this:
Possible unsafe locking scenario:
CPU0 CPU1
---- ----
lock(&fip->ctlr_mutex);
lock(rtnl_mutex);
lock(&fip->ctlr_mutex);
lock(rtnl_mutex);
*** DEADLOCK ***
Signed-off-by: Robert Love <robert.w.love@intel.com>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>
There is potentially lots of contention for the rx_list_lock. On a cpu that is
receiving lots of fcoe traffic, the softirq context has to add and release the
lock for every frame it receives, as does the receiving per-cpu thread. We can
reduce this contention somewhat by altering the per-cpu threads loop such that
when traffic is detected on the fcoe_rx_list, we splice it to a temporary list.
In this way, we can process multiple skbs while only having to acquire and
release the fcoe_rx_list lock once.
[ Braces around single statement while loop removed by Robert Love
to satisfy checkpath.pl. ]
Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
Acked-by: Vasu Dev <vasu.dev@intel.com>
Signed-off-by: Robert Love <robert.w.love@intel.com>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>
commit e7a51997da ([SCSI] fcoe: flush per-cpu
thread work when destroying interface) added a skb flush to the fcoe_rx_list,
which ensures that we push any pending frames on the list through the per-cpu
receive thread. Because of this, its redundant to lock and scan the list
first, dropping any arriving frames.
Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
Acked-by: Vasu Dev <vasu.dev@intel.com>
Signed-off-by: Robert Love <robert.w.love@intel.com>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>
The fcoe sw recive packet function (fcoe_rcv) only ever executes in softirq
context. Given that, and the fact that no use of the fcoe_rx_list is made in
irq context, its not necessecary to disable bottom halves while actually
receiving the frame. Convert spin_*_bh calls in that function to their
lock-only equivalents
Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
Acked-by: Vasu Dev <vasu.dev@intel.com>
Signed-off-by: Robert Love <robert.w.love@intel.com>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>
Some switch implementations (eg., HP virtual connect FlexFabric) send two MAC
descriptors in FIP FLOGI response, with first MAC descriptor (granted_mac) used
as FPMA, and the second one (fcoe_mac) used as destination address for
sending/receiving FCoE packets. fip_mac continues to be used for FIP traffic.
This patch introduces fcoe_mac in fcoe_fcf structure. For regular switches,
both fcoe_mac and fip_mac will be the same. For the switches that send
additional MAC descriptor, fcoe_mac is updated.
Signed-off-by: Bhanu Prakash Gollapudi <bprakash@broadcom.com>
Signed-off-by: Robert Love <robert.w.love@intel.com>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>
When handling CVL with no Vx port descriptors, lports for NPIV ports are reset
before issuing the ctlr_reset. This causes FDISCs to be issued before
successful FLOGI. Fix it by resetting the controller before resetting the
lports.
Signed-off-by: Bhanu Prakash Gollapudi <bprakash@broadcom.com>
Signed-off-by: Robert Love <robert.w.love@intel.com>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>
commit 859b7b649a introduced the ability to call
fcoe_recv_frame in softirq context. While this is beneficial to performance,
its not safe to do, as it breaks the serialization of access to the lport
structure (i.e. when an fcoe interface is being torn down, theres no way to
serialize the teardown effort with the completion of receieve operations
occuring in softirq context. As a result, lport (and other) data structures can
be read and modified in parallel leading to corruption. Most notable is the
vport list, which is protected by a mutex, that will cause a panic if a softirq
receive while said mutex is locked. Additionaly, the ema_list, discussed here:
http://lists.open-fcoe.org/pipermail/devel/2012-February/011947.html
Can be corrupted if a list traversal occurs in softirq context at the same time
as a list delete in process context. And generally the lport state variables
will not be stable, and may lead to unpredictable results.
The most direct fix is to remove the bits from the above commit that allowed
fcoe_recv_frame to be called in softirq context. We just force all frames to be
handled by the per-cpu rx threads. This will allow the fcoe_if_destroy's use of
fcoe_percpu_clean to function properly, ensuring that no frames are being
received while the lport is being torn down.
Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
Reviewed-by: Vasu Dev <vasu.dev@intel.com>
Signed-off-by: Robert Love <robert.w.love@intel.com>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>
SCSI updates from James Bottomley:
"The update includes the usual assortment of driver updates (lpfc,
qla2xxx, qla4xxx, bfa, bnx2fc, bnx2i, isci, fcoe, hpsa) plus a huge
amount of infrastructure work in the SAS library and transport class
as well as an iSCSI update. There's also a new SCSI based virtio
driver."
* tag 'scsi-misc' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi-misc-2.6: (177 commits)
[SCSI] qla4xxx: Update driver version to 5.02.00-k15
[SCSI] qla4xxx: trivial cleanup
[SCSI] qla4xxx: Fix sparse warning
[SCSI] qla4xxx: Add support for multiple session per host.
[SCSI] qla4xxx: Export CHAP index as sysfs attribute
[SCSI] scsi_transport: Export CHAP index as sysfs attribute
[SCSI] qla4xxx: Add support to display CHAP list and delete CHAP entry
[SCSI] iscsi_transport: Add support to display CHAP list and delete CHAP entry
[SCSI] pm8001: fix endian issue with code optimization.
[SCSI] pm8001: Fix possible racing condition.
[SCSI] pm8001: Fix bogus interrupt state flag issue.
[SCSI] ipr: update PCI ID definitions for new adapters
[SCSI] qla2xxx: handle default case in qla2x00_request_firmware()
[SCSI] isci: improvements in driver unloading routine
[SCSI] isci: improve phy event warnings
[SCSI] isci: debug, provide state-enum-to-string conversions
[SCSI] scsi_transport_sas: 'enable' phys on reset
[SCSI] libsas: don't recover end devices attached to disabled phys
[SCSI] libsas: fixup target_port_protocols for expanders that don't report sata
[SCSI] libsas: set attached device type and target protocols for local phys
...
Pull kmap_atomic cleanup from Cong Wang.
It's been in -next for a long time, and it gets rid of the (no longer
used) second argument to k[un]map_atomic().
Fix up a few trivial conflicts in various drivers, and do an "evil
merge" to catch some new uses that have come in since Cong's tree.
* 'kmap_atomic' of git://github.com/congwang/linux: (59 commits)
feature-removal-schedule.txt: schedule the deprecated form of kmap_atomic() for removal
highmem: kill all __kmap_atomic() [swarren@nvidia.com: highmem: Fix ARM build break due to __kmap_atomic rename]
drbd: remove the second argument of k[un]map_atomic()
zcache: remove the second argument of k[un]map_atomic()
gma500: remove the second argument of k[un]map_atomic()
dm: remove the second argument of k[un]map_atomic()
tomoyo: remove the second argument of k[un]map_atomic()
sunrpc: remove the second argument of k[un]map_atomic()
rds: remove the second argument of k[un]map_atomic()
net: remove the second argument of k[un]map_atomic()
mm: remove the second argument of k[un]map_atomic()
lib: remove the second argument of k[un]map_atomic()
power: remove the second argument of k[un]map_atomic()
kdb: remove the second argument of k[un]map_atomic()
udf: remove the second argument of k[un]map_atomic()
ubifs: remove the second argument of k[un]map_atomic()
squashfs: remove the second argument of k[un]map_atomic()
reiserfs: remove the second argument of k[un]map_atomic()
ocfs2: remove the second argument of k[un]map_atomic()
ntfs: remove the second argument of k[un]map_atomic()
...
Fix a bug when using 'ethtool -K ethx tx off' to turn off tx ip checksum,
FCoE CRC offload should not be impacte. The skb_checksum_help() is needed
only if it's not FCoE traffic for ip checksum, regardless of ethtool toggling
the tx ip checksum on or off. Instead of using CHECKSUM_PARTIAL, we will
use CHECKSUM_UNNECESSARY as a proper indication to avoid sw ip checksum
on FCoE frames.
Ref. to original discussion thread:
http://patchwork.ozlabs.org/patch/146567/
CC: "James E.J. Bottomley" <JBottomley@parallels.com>
CC: Robert Love <robert.w.love@intel.com>
Signed-off-by: Yi Zou <yi.zou@intel.com>
Tested-by: Ross Brattain <ross.b.brattain@intel.com>
Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
The reference counting was necessary on these instances
because it was possible for NPIV ports to be destroyed
after the N_Port. A previous patch ensures that all NPIV
ports are destroyed before the N_Port making the need to
track references on the interface unnecessary.
Signed-off-by: Robert Love <robert.w.love@intel.com>
Tested-by: Ross Brattain <ross.b.brattain@intel.com>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>
Currently all port deletion is routed though the FCoE
workqueue (fcoe_wq). When fc_remove_host is called on
an N_Port (for example, from fcoe_destroy) the vports
are queued into a FC Transport workqueue. fc_remove_host
flushes that queue and each vport is passed to fcoe's
fcoe_vport_destroy, which simply queues the associated
fcoe_ports for later deletion. This queue cannot be
flushed within the N_Ports destroy path because of
circular locking issues. The result is that the NPIV
ports are destroyed after the N_Port, which is reverse
of how they are created.
This quirk causes fcoe to keep references on the
fcoe_interface shared by each of these ports (N_Port
and NPIV). Changing the ordering such that NPIV ports
are destroyed before the N_Port will allow us to remove
reference counting on the fcoe_interface instances.
This patch simply allows fcoe_vport_destory to destroy
NPIV ports without deferring them to a workqueue context.
This ensures that when fc_remove_host is called the
NPIV ports will be destroyed first before the N_Port and
allows reference counting on the fcoe's fcoe_interface
to be remove in a later patch.
Signed-off-by: Robert Love <robert.w.love@intel.com>
Tested-by: Ross Brattain <ross.b.brattain@intel.com>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>
The label implies that it should be called when
there is 'nomod.' I read that to mean that the
module reference 'get' failed. However, it's only
called when the module reference 'get' succeeded.
I think it makes more sense to name the label,
'out_putmod' since it should be called when we
need to 'put' the module reference taken in the
routine before returning.
Signed-off-by: Robert Love <robert.w.love@intel.com>
Tested-by: Ross Brattain <ross.b.brattain@intel.com>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>
This is more of a debug statement. As a KERN_ERR we generate
log entries anytime any netdev goes up or down, so when booting
there are notification log entries for all system interfaces
including 'lo'. This is too much. Let's just log when necessary.
Signed-off-by: Robert Love <robert.w.love@intel.com>
Tested-by: Ross Brattain <ross.b.brattain@intel.com>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>
