fnic doesn't use any of the create/destroy/enable/disable interfaces
either from the (legacy) module paramaters or the (new) fcoe_sysfs
interfaces. When fcoe_sysfs was introduced fnic wasn't changed since
it wasn't using the interfaces. libfcoe incorrectly assumed that that
all of its users were using fcoe_sysfs and when adding and deleting
FCFs would assume the existance of a fcoe_ctlr_device. fnic was not
allocating this structure because it doesn't care about the standard
user interfaces (fnic starts on link only). If/When libfcoe tried to use
the fcoe_ctlr_device's lock for the first time a NULL pointer exception
would be triggered.
Since fnic doesn't care about sysfs or user interfaces, the solution
is to drop libfcoe's assumption that all drivers are using fcoe_sysfs.
This patch accomplishes this by changing some of the structure
relationships.
We need a way to determine when a LLD is using fcoe_sysfs or not and
we can do that by checking for the existance of the fcoe_ctlr_device.
Prior to this patch, it was assumed that the fcoe_ctlr structure was
allocated with the fcoe_ctlr_device and immediately followed it in
memory. To reach the fcoe_ctlr_device we would simply go back in memory
from the fcoe_ctlr to get the fcoe_ctlr_device.
Since fnic doesn't allocate the fcoe_ctlr_device, we cannot keep that
assumption. This patch adds a pointer from the fcoe_ctlr to the
fcoe_ctlr_device. For bnx2fc and fcoe we will continue to allocate the
two structures together, but then we'll set the ctlr->cdev pointer
to point at the fcoe_ctlr_device. fnic will not change and will continue
to allocate the fcoe_ctlr itself, and ctlr->cdev will remain NULL.
When libfcoe adds fcoe_fcf's to the fcoe_ctlr it will check if ctlr->cdev
is set and only if so will it continue to interact with fcoe_sysfs.
Signed-off-by: Robert Love <robert.w.love@intel.com>
Acked-by: Neil Horman <nhorman@tuxdriver.com>
Tested-by: Hiral Patel <hiralpat@cisco.com>
Similarly they can be moved into libfcoe instead of being private to fcoe now.
Also add comments particularly on the term LESB to the corresponding function.
Signed-off-by: Yi Zou <yi.zou@intel.com>
Cc: Bhanu Prakash Gollapudi <bprakash@broadcom.com>
Tested-by: Marcus Dennis <marcusx.e.dennis@intel.com>
Signed-off-by: Robert Love <robert.w.love@intel.com>
Currently, in the default kernel fcoe driver, it is needed to get to the underlying
private per fcoe transport's private structure, e.g., fcoe_interface in
fcoe.ko, and returns the associated netdev. The similar logic exists in other
fcoe drivers, e.g., bnx2fc, so we add a function pointer into the common
fcoe_port struct to allow individual fcoe transport implementaion (fcoe
and bnx2fc) to get the corresponding netdev associated with a give lport.
Then a inline fcoe_get_netdev() is added as part of libfcoe for all underlying
fcoe transport drivers to use regardless of its individual fcoe transport
driver, and also allows move more common code such as fcoe_link_speed_update or
fcoe_ctlr_get_lesb to be in libfcoe, rather than specific to fcoe.
This patch is a prep work that adds aforementioned fucntion pointer, and
followed by the actual code changes to make use of it.
Signed-off-by: Yi Zou <yi.zou@intel.com>
Cc: Bhanu Prakash Gollapudi <bprakash@broadcom.com>
Tested-by: Marcus Dennis <marcusx.e.dennis@intel.com>
Signed-off-by: Robert Love <robert.w.love@intel.com>
This patch does a few things.
1) Makes /sys/bus/fcoe/ctlr_{create,destroy} interfaces.
These interfaces take an <ifname> and will either
create an FCoE Controller or destroy an FCoE
Controller depending on which file is written to.
The new FCoE Controller will start in a DISABLED
state and will not do discovery or login until it
is ENABLED. This pause will allow us to configure
the FCoE Controller before enabling it.
2) Makes the 'mode' attribute of a fcoe_ctlr_device
writale. This allows the user to configure the mode
in which the FCoE Controller will start in when it
is ENABLED.
Possible modes are 'Fabric', or 'VN2VN'.
The default mode for a fcoe_ctlr{,_device} is 'Fabric'.
Drivers must implement the set_fcoe_ctlr_mode routine
to support this feature.
libfcoe offers an exported routine to set a FCoE
Controller's mode. The mode can only be changed
when the FCoE Controller is DISABLED.
This patch also removes the get_fcoe_ctlr_mode pointer
in the fcoe_sysfs function template, the code in
fcoe_ctlr.c to get the mode and the assignment of
the fcoe_sysfs function pointer to the fcoe_ctlr.c
implementation (in fcoe and bnx2fc). fcoe_sysfs can
return that value for the mode without consulting the
LLD.
3) Make a 'enabled' attribute of a fcoe_ctlr_device. On a
read, fcoe_sysfs will return the attribute's value. On
a write, fcoe_sysfs will call the LLD (if there is a
callback) to notifiy that the enalbed state has changed.
This patch maintains the old FCoE control interfaces as
module parameters, but it adds comments pointing out that
the old interfaces are deprecated.
Signed-off-by: Robert Love <robert.w.love@intel.com>
Acked-by: Neil Horman <nhorman@tuxdriver.com>
SCSI errors were generated while writing to LUNs
connected via NPIV ports.
Debugging this it was found that the FCoE packets
transmitted via the NPIV ports were not tagged with
correct user priority as negotiated with peer by DCB
agent. This resulted in FCoE traffic going with priority
zero(0) that did not have priority flow control (PFC)
enabled for it. The initiator after transferring data
to the target never saw any reply indicating the transfer
was complete. This resulted in error recovery (ABTS) and
SCSI command retries by the scsi-mid layer; eventually
resulting in I/O errors.
This patch fixes this issue by keeping the FCoE user
priority information in the fcoe_interface instance
that is common for both the physical port as well as
NPIV ports connected to that physical port; instead
of storing it in fcoe_port structure that has a per
port instance.
Signed-off-by: Neerav Parikh <Neerav.Parikh@intel.com>
Acked-by: Yi Zou <yi.zou@intel.com>
Acked-by: John Fastabend <john.r.fastabend@intel.com>
Tested-by: Marcus Dennis <marcusx.e.dennis@intel.com>
Signed-off-by: Robert Love <robert.w.love@intel.com>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>
This patch has the SW FCoE driver and the bnx2fc
driver make use of the new fcoe_sysfs API added
earlier in this patch series.
After this patch a fcoe_ctlr_device is allocated with
private data in this order.
+------------------+ +------------------+
| fcoe_ctlr_device | | fcoe_ctlr_device |
+------------------+ +------------------+
| fcoe_ctlr | | fcoe_ctlr |
+------------------+ +------------------+
| fcoe_interface | | bnx2fc_interface |
+------------------+ +------------------+
libfcoe also takes part in this new model since it
discovers and manages fcoe_fcf instances. The memory
allocation is different for FCFs. I didn't want to
impact libfcoe's fcoe_fcf processing, so this patch
creates fcoe_fcf_device instances for each discovered
fcoe_fcf. The two are paired using a (void * priv)
member of the fcoe_ctlr_device. This allows libfcoe
to continue maintaining its list of fcoe_fcf instances
and simply attaches and detaches them from existing
or new fcoe_fcf_device instances.
Signed-off-by: Robert Love <robert.w.love@intel.com>
Tested-by: Ross Brattain <ross.b.brattain@intel.com>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>
This patch adds a 'fcoe bus' infrastructure to the kernel
that is driven by changes to libfcoe which allow LLDs to
present FIP (FCoE Initialization Protocol) discovered
entities and their attributes to user space via sysfs.
This patch adds the following APIs-
fcoe_ctlr_device_add
fcoe_ctlr_device_delete
fcoe_fcf_device_add
fcoe_fcf_device_delete
They allow the LLD to expose the FCoE ENode Controller
and any discovered FCFs (Fibre Channel Forwarders, e.g.
FCoE switches) to the user. Each of these new devices
has their own bus_type so that they are grouped together
for easy lookup from a user space application. Each
new class has an attribute_group to expose attributes
for any created instances. The attributes are-
fcoe_ctlr_device
* fcf_dev_loss_tmo
* lesb_link_fail
* lesb_vlink_fail
* lesb_miss_fka
* lesb_symb_err
* lesb_err_block
* lesb_fcs_error
fcoe_fcf_device
* fabric_name
* switch_name
* priority
* selected
* fc_map
* vfid
* mac
* fka_peroid
* fabric_state
* dev_loss_tmo
A device loss infrastructre similar to the FC Transport's
is also added by this patch. It is nice to have so that a
link flapping adapter doesn't continually advance the count
used to identify the discovered FCF. FCFs will exist in a
"Disconnected" state until either the timer expires or the
FCF is rediscovered and becomes "Connected."
This patch generates a few checkpatch.pl WARNINGS that
I'm not sure what to do about. They're macros modeled
around the FC Transport attribute building macros, which
have the same 'feature' where the caller can ommit a cast
in the argument list and no cast occurs in the code. I'm
not sure how to keep the code condensed while keeping the
macros. Any advice would be appreciated.
Signed-off-by: Robert Love <robert.w.love@intel.com>
Tested-by: Ross Brattain <ross.b.brattain@intel.com>
Acked-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>
Currently the fcoe_ctlr associated with an interface is allocated
as a member of struct fcoe_interface. This causes problems when
attempting to use the new fcoe_sysfs APIs which allow us to allocate
the fcoe_interface as private data to the fcoe_ctlr_device instance.
The problem is that libfcoe wants to be able use pointer math to find a
fcoe_ctlr's fcoe_ctlr_device as well as finding a fcoe_ctlr_device's
assocated fcoe_ctlr. To do this we need to allocate the
fcoe_ctlr_device, with private data for the LLD. The private data
contains the fcoe_ctlr and its private data is the fcoe_interface.
This patch only allocates the fcoe_interface with the fcoe_ctlr, the
fcoe_ctlr_device will be added in a later patch, which will complete
the below diagram-
+------------------+
| fcoe_ctlr_device |
+------------------+
| fcoe_ctlr |
+------------------+
| fcoe_interface |
+------------------+
This prep work will allow us to go from a fcoe_ctlr_device instance
to its fcoe_ctlr as well as from a fcoe_ctlr to its fcoe_ctlr_device
once the fcoe_sysfs API is in use (later patches in this series).
Signed-off-by: Robert Love <robert.w.love@intel.com>
Tested-by: Ross Brattain <ross.b.brattain@intel.com>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>
Some switch implementations (eg., HP virtual connect FlexFabric) send two MAC
descriptors in FIP FLOGI response, with first MAC descriptor (granted_mac) used
as FPMA, and the second one (fcoe_mac) used as destination address for
sending/receiving FCoE packets. fip_mac continues to be used for FIP traffic.
This patch introduces fcoe_mac in fcoe_fcf structure. For regular switches,
both fcoe_mac and fip_mac will be the same. For the switches that send
additional MAC descriptor, fcoe_mac is updated.
Signed-off-by: Bhanu Prakash Gollapudi <bprakash@broadcom.com>
Signed-off-by: Robert Love <robert.w.love@intel.com>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>
Use DCB notifiers to set the skb priority to allow packets
to be steered and tagged correctly over DCB enabled drivers
that setup traffic classes.
This allows queue_mapping() routines to be removed in these
drivers that were previously inspecting the ethertype of
every skb to mark FCoE/FIP frames.
Signed-off-by: John Fastabend <john.r.fastabend@intel.com>
Signed-off-by: Robert Love <robert.w.love@intel.com>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>
Except for obtaining the netdev from lport, fcoe_get_lesb is the common code
for the LLDs.
Signed-off-by: Bhanu Prakash Gollapudi <bprakash@broadcom.com>
Acked-by: Yi Zou <yi.zou@intel.com>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>
Export fcoe_get_wwn, fcoe_validate_vport_create and fcoe_wwn_to_str so that all
LLDs can use these common function.
Signed-off-by: Bhanu Prakash Gollapudi <bprakash@broadcom.com>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>
To facilitate LLDDs to reuse the code, skb queue related functions are moved to
libfcoe, so that both fcoe and bnx2fc drivers can use them. The common structures
fcoe_port, fcoe_percpu_s are moved to libfcoe. fcoe_port will now have an
opaque pointer that points to corresponding driver's interface structure.
Also, fcoe_start_io and fcoe_fc_crc are moved to libfcoe.
As part of this change, fixed fcoe_start_io to return ENOMEM if
skb_clone fails.
Signed-off-by: Bhanu Prakash Gollapudi <bprakash@broadcom.com>
Signed-off-by: Robert Love <robert.w.love@intel.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
add the fcoe_transport struct to the common libfcoe.h header so all fcoe
transport provides can use it to attach itself as an fcoe transport. This
is the header part, and the next patch will be the transport code itself.
Signed-off-by: Yi Zou <yi.zou@intel.com>
Signed-off-by: Bhanu Prakash Gollapudi <bprakash@broadcom.com>
Signed-off-by: Robert Love <robert.w.love@intel.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
Switches using multiple-FCFs may reject FLOGI in order to
balance the load between multiple FCFs. Even though the FCF
was available, it may have more load at the point we actually
send the FLOGI.
If the FLOGI fails, select a different FCF
if possible, among those with the same priority. If no other
FCF is available, just deliver the reject to libfc for retry.
Signed-off-by: Joe Eykholt <jeykholt@cisco.com>
Signed-off-by: Robert Love <robert.w.love@intel.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
The FC-BB-6 committee is proposing a new FIP usage model called
VN_port to VN_port mode. It allows VN_ports to discover each other
over a loss-free L2 Ethernet without any FCF or Fibre-channel fabric
services. This is point-to-multipoint. There is also a variant
of this called point-to-point which provides for making sure there
is just one pair of ports operating over the Ethernet fabric.
We add these new states: VNMP_START, _PROBE1, _PROBE2, _CLAIM, and _UP.
These usually go quickly in that sequence. After waiting a random
amount of time up to 100 ms in START, we select a pseudo-random
proposed locally-unique port ID and send out probes in states PROBE1
and PROBE2, 100 ms apart. If no probe responses are heard, we
proceed to CLAIM state 400 ms later and send a claim notification.
We wait another 400 ms to receive claim responses, which give us
a list of the other nodes on the network, including their FC-4
capabilities. After another 400 ms we go to VNMP_UP state and
should start interoperating with any of the nodes for whic we
receivec claim responses. More details are in the spec.j
Add the new mode as FIP_MODE_VN2VN. The driver must specify
explicitly that it wants to operate in this mode. There is
no automatic detection between point-to-multipoint and fabric
mode, and the local port initialization is affected, so it isn't
anticipated that there will ever be any such automatic switchover.
It may eventually be possible to have both fabric and VN2VN
modes on the same L2 network, which may be done by two separate
local VN_ports (lports).
When in VN2VN mode, FIP replaces libfc's fabric-oriented discovery
module with its own simple code that adds remote ports as they
are discovered from incoming claim notifications and responses.
These hooks are placed by fcoe_disc_init().
A linear list of discovered vn_ports is maintained under the
fcoe_ctlr struct. It is expected to be short for now, and
accessed infrequently. It is kept under RCU for lock-ordering
reasons. The lport and/or rport mutexes may be held when we
need to lookup a fcoe_vnport during an ELS send.
Change fcoe_ctlr_encaps() to lookup the destination vn_port in
the list of peers for the destination MAC address of the
FIP-encapsulated frame.
Add a new function fcoe_disc_init() to initialize just the
discovery portion of libfcoe for VN2VN mode.
Signed-off-by: Joe Eykholt <jeykholt@cisco.com>
Signed-off-by: Robert Love <robert.w.love@intel.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
There are three modes that libfcoe currently supports, and a new one
is coming. Change the fcoe_ctlr_init() interface to add the mode
desired. This should not change any functionality.
Signed-off-by: Joe Eykholt <jeykholt@cisco.com>
Signed-off-by: Robert Love <robert.w.love@intel.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
It turns out most of the FIP work is now done from worker threads
or process context now, so there's no need to use a spin lock.
Change to use mutex instead of spin lock and delayed_work instead
of a timer.
This will make it nicer for the VN_port to VN_port feature that
will interact more with the libfc layers requiring that
spinlocks not be held.
Signed-off-by: Joe Eykholt <jeykholt@cisco.com>
Signed-off-by: Robert Love <robert.w.love@intel.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
This is per FC-BB-5 Annex-D recommendation and per that
if address checking fails then drop the frame.
FIP code paths are already doing this so only needed for fcoe
frames.
The src address checking is limited to only fip mode since
this might break non-fip mode used in p2p due to used OUI
based addressing in some p2p code paths, going forward FIP
will be the only mode, therefore limited this to only FIP
mode so that it won't break non-fip p2p mode for now.
-v2
Removes FCOE packet type checking since fcoe_rcv is
registered to receive only FCoE type packets from netdev
and it is already checked by netdev.
Signed-off-by: Vasu Dev <vasu.dev@intel.com>
Signed-off-by: Robert Love <robert.w.love@intel.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
No reason to restrict CDB size to 12 bytes in fcoe, so
increased to 16 so that 16 bytes SCSI CDB doesn't fail.
Uses common define to set max_cmd_len for fcoe and fnic,
fnic is already setting max_cmd_len to 16.
sg_readcap -l fails without this fix.
Signed-off-by: Vasu Dev <vasu.dev@intel.com>
Signed-off-by: Robert Love <robert.w.love@intel.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
The link and last_link fields in the fcoe_ctlr struct are no
longer useful, since they are always set to the same value,
and FIP always calls libfc to pass link information to the lport.
Eliminate those fields and rename link_work to timer_work, since
it no longer has any link change work to do.
Thanks to Brian Uchino for discovering this issue.
Signed-off-by: Joe Eykholt <jeykholt@cisco.com>
Signed-off-by: Robert Love <robert.w.love@intel.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
When the D bit is set if the FKA_ADV_Period of the FIP Discovery
Advertisement, the ENode should not transmit period ENode FIP Keep Alive and
VN_Port FIP Keep Alive (FC-BB-5 Rev2, 7.8.3.13).
Note that fcf->flags is taken directly from the fip_header, I am claiming one
bit for the purpose of the FIP_FKA_Period D bit as FIP_FL_FK_ADV_B, and use
FIP_HEADER_FLAGS as bitmask for bits used in fip_header.
Signed-off-by: Yi Zou <yi.zou@intel.com>
Signed-off-by: Robert Love <robert.w.love@intel.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
