Vlad Buslov
653cd284a8
net: sched: always disable bh when taking tcf_lock
...
Recently, ops->init() and ops->dump() of all actions were modified to
always obtain tcf_lock when accessing private action state. Actions that
don't depend on tcf_lock for synchronization with their data path use
non-bh locking API. However, tcf_lock is also used to protect rate
estimator stats in softirq context by timer callback.
Change ops->init() and ops->dump() of all actions to disable bh when using
tcf_lock to prevent deadlock reported by following lockdep warning:
[ 105.470398] ================================
[ 105.475014] WARNING: inconsistent lock state
[ 105.479628] 4.18.0-rc8+ #664 Not tainted
[ 105.483897] --------------------------------
[ 105.488511] inconsistent {SOFTIRQ-ON-W} -> {IN-SOFTIRQ-W} usage.
[ 105.494871] swapper/16/0 [HC0[0]:SC1[1]:HE1:SE0] takes:
[ 105.500449] 00000000f86c012e (&(&p->tcfa_lock)->rlock){+.?.}, at: est_fetch_counters+0x3c/0xa0
[ 105.509696] {SOFTIRQ-ON-W} state was registered at:
[ 105.514925] _raw_spin_lock+0x2c/0x40
[ 105.519022] tcf_bpf_init+0x579/0x820 [act_bpf]
[ 105.523990] tcf_action_init_1+0x4e4/0x660
[ 105.528518] tcf_action_init+0x1ce/0x2d0
[ 105.532880] tcf_exts_validate+0x1d8/0x200
[ 105.537416] fl_change+0x55a/0x268b [cls_flower]
[ 105.542469] tc_new_tfilter+0x748/0xa20
[ 105.546738] rtnetlink_rcv_msg+0x56a/0x6d0
[ 105.551268] netlink_rcv_skb+0x18d/0x200
[ 105.555628] netlink_unicast+0x2d0/0x370
[ 105.559990] netlink_sendmsg+0x3b9/0x6a0
[ 105.564349] sock_sendmsg+0x6b/0x80
[ 105.568271] ___sys_sendmsg+0x4a1/0x520
[ 105.572547] __sys_sendmsg+0xd7/0x150
[ 105.576655] do_syscall_64+0x72/0x2c0
[ 105.580757] entry_SYSCALL_64_after_hwframe+0x49/0xbe
[ 105.586243] irq event stamp: 489296
[ 105.590084] hardirqs last enabled at (489296): [<ffffffffb507e639>] _raw_spin_unlock_irq+0x29/0x40
[ 105.599765] hardirqs last disabled at (489295): [<ffffffffb507e745>] _raw_spin_lock_irq+0x15/0x50
[ 105.609277] softirqs last enabled at (489292): [<ffffffffb413a6a3>] irq_enter+0x83/0xa0
[ 105.618001] softirqs last disabled at (489293): [<ffffffffb413a800>] irq_exit+0x140/0x190
[ 105.626813]
other info that might help us debug this:
[ 105.633976] Possible unsafe locking scenario:
[ 105.640526] CPU0
[ 105.643325] ----
[ 105.646125] lock(&(&p->tcfa_lock)->rlock);
[ 105.650747] <Interrupt>
[ 105.653717] lock(&(&p->tcfa_lock)->rlock);
[ 105.658514]
*** DEADLOCK ***
[ 105.665349] 1 lock held by swapper/16/0:
[ 105.669629] #0 : 00000000a640ad99 ((&est->timer)){+.-.}, at: call_timer_fn+0x10b/0x550
[ 105.678200]
stack backtrace:
[ 105.683194] CPU: 16 PID: 0 Comm: swapper/16 Not tainted 4.18.0-rc8+ #664
[ 105.690249] Hardware name: Supermicro SYS-2028TP-DECR/X10DRT-P, BIOS 2.0b 03/30/2017
[ 105.698626] Call Trace:
[ 105.701421] <IRQ>
[ 105.703791] dump_stack+0x92/0xeb
[ 105.707461] print_usage_bug+0x336/0x34c
[ 105.711744] mark_lock+0x7c9/0x980
[ 105.715500] ? print_shortest_lock_dependencies+0x2e0/0x2e0
[ 105.721424] ? check_usage_forwards+0x230/0x230
[ 105.726315] __lock_acquire+0x923/0x26f0
[ 105.730597] ? debug_show_all_locks+0x240/0x240
[ 105.735478] ? mark_lock+0x493/0x980
[ 105.739412] ? check_chain_key+0x140/0x1f0
[ 105.743861] ? __lock_acquire+0x836/0x26f0
[ 105.748323] ? lock_acquire+0x12e/0x290
[ 105.752516] lock_acquire+0x12e/0x290
[ 105.756539] ? est_fetch_counters+0x3c/0xa0
[ 105.761084] _raw_spin_lock+0x2c/0x40
[ 105.765099] ? est_fetch_counters+0x3c/0xa0
[ 105.769633] est_fetch_counters+0x3c/0xa0
[ 105.773995] est_timer+0x87/0x390
[ 105.777670] ? est_fetch_counters+0xa0/0xa0
[ 105.782210] ? lock_acquire+0x12e/0x290
[ 105.786410] call_timer_fn+0x161/0x550
[ 105.790512] ? est_fetch_counters+0xa0/0xa0
[ 105.795055] ? del_timer_sync+0xd0/0xd0
[ 105.799249] ? __lock_is_held+0x93/0x110
[ 105.803531] ? mark_held_locks+0x20/0xe0
[ 105.807813] ? _raw_spin_unlock_irq+0x29/0x40
[ 105.812525] ? est_fetch_counters+0xa0/0xa0
[ 105.817069] ? est_fetch_counters+0xa0/0xa0
[ 105.821610] run_timer_softirq+0x3c4/0x9f0
[ 105.826064] ? lock_acquire+0x12e/0x290
[ 105.830257] ? __bpf_trace_timer_class+0x10/0x10
[ 105.835237] ? __lock_is_held+0x25/0x110
[ 105.839517] __do_softirq+0x11d/0x7bf
[ 105.843542] irq_exit+0x140/0x190
[ 105.847208] smp_apic_timer_interrupt+0xac/0x3b0
[ 105.852182] apic_timer_interrupt+0xf/0x20
[ 105.856628] </IRQ>
[ 105.859081] RIP: 0010:cpuidle_enter_state+0xd8/0x4d0
[ 105.864395] Code: 46 ff 48 89 44 24 08 0f 1f 44 00 00 31 ff e8 cf ec 46 ff 80 7c 24 07 00 0f 85 1d 02 00 00 e8 9f 90 4b ff fb 66 0f 1f 44 00 00 <4c> 8b 6c 24 08 4d 29 fd 0f 80 36 03 00 00 4c 89 e8 48 ba cf f7 53
[ 105.884288] RSP: 0018:ffff8803ad94fd20 EFLAGS: 00000246 ORIG_RAX: ffffffffffffff13
[ 105.892494] RAX: 0000000000000000 RBX: ffffe8fb300829c0 RCX: ffffffffb41e19e1
[ 105.899988] RDX: 0000000000000007 RSI: dffffc0000000000 RDI: ffff8803ad9358ac
[ 105.907503] RBP: ffffffffb6636300 R08: 0000000000000004 R09: 0000000000000000
[ 105.914997] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000004
[ 105.922487] R13: ffffffffb6636140 R14: ffffffffb66362d8 R15: 000000188d36091b
[ 105.929988] ? trace_hardirqs_on_caller+0x141/0x2d0
[ 105.935232] do_idle+0x28e/0x320
[ 105.938817] ? arch_cpu_idle_exit+0x40/0x40
[ 105.943361] ? mark_lock+0x8c1/0x980
[ 105.947295] ? _raw_spin_unlock_irqrestore+0x32/0x60
[ 105.952619] cpu_startup_entry+0xc2/0xd0
[ 105.956900] ? cpu_in_idle+0x20/0x20
[ 105.960830] ? _raw_spin_unlock_irqrestore+0x32/0x60
[ 105.966146] ? trace_hardirqs_on_caller+0x141/0x2d0
[ 105.971391] start_secondary+0x2b5/0x360
[ 105.975669] ? set_cpu_sibling_map+0x1330/0x1330
[ 105.980654] secondary_startup_64+0xa5/0xb0
Taking tcf_lock in sample action with bh disabled causes lockdep to issue a
warning regarding possible irq lock inversion dependency between tcf_lock,
and psample_groups_lock that is taken when holding tcf_lock in sample init:
[ 162.108959] Possible interrupt unsafe locking scenario:
[ 162.116386] CPU0 CPU1
[ 162.121277] ---- ----
[ 162.126162] lock(psample_groups_lock);
[ 162.130447] local_irq_disable();
[ 162.136772] lock(&(&p->tcfa_lock)->rlock);
[ 162.143957] lock(psample_groups_lock);
[ 162.150813] <Interrupt>
[ 162.153808] lock(&(&p->tcfa_lock)->rlock);
[ 162.158608]
*** DEADLOCK ***
In order to prevent potential lock inversion dependency between tcf_lock
and psample_groups_lock, extract call to psample_group_get() from tcf_lock
protected section in sample action init function.
Fixes: 4e232818bd764e9a2448729e012609d772849566e8917f4370b6a2b971c02142236b45vladbu@mellanox.com >
Signed-off-by: David S. Miller <davem@davemloft.net >
2018-08-19 10:46:21 -07:00
Vlad Buslov
32039eac4c
net: sched: act_ife: always release ife action on init error
...
Action init API was changed to always take reference to action, even when
overwriting existing action. Substitute conditional action release, which
was executed only if action is newly created, with unconditional release in
tcf_ife_init() error handling code to prevent double free or memory leak in
case of overwrite.
Fixes: 4e8ddd7f17xiyou.wangcong@gmail.com >
Signed-off-by: Vlad Buslov <vladbu@mellanox.com >
Acked-by: Cong Wang <xiyou.wangcong@gmail.com >
Signed-off-by: David S. Miller <davem@davemloft.net >
2018-08-16 12:12:12 -07:00
Hangbin Liu
a51c76b4df
cls_matchall: fix tcf_unbind_filter missing
...
Fix tcf_unbind_filter missing in cls_matchall as this will trigger
WARN_ON() in cbq_destroy_class().
Fixes: fd62d9f5c5shuali@redhat.com >
Signed-off-by: Hangbin Liu <liuhangbin@gmail.com >
Acked-by: Cong Wang <xiyou.wangcong@gmail.com >
Signed-off-by: David S. Miller <davem@davemloft.net >
2018-08-16 12:08:26 -07:00
Hangbin Liu
008369dcc5
net_sched: Fix missing res info when create new tc_index filter
...
Li Shuang reported the following warn:
[ 733.484610] WARNING: CPU: 6 PID: 21123 at net/sched/sch_cbq.c:1418 cbq_destroy_class+0x5d/0x70 [sch_cbq]
[ 733.495190] Modules linked in: sch_cbq cls_tcindex sch_dsmark rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache xt_CHECKSUM iptable_mangle ipt_MASQUERADE iptable_nat l
[ 733.574155] syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm igb ixgbe ahci libahci i2c_algo_bit libata i40e i2c_core dca mdio megaraid_sas dm_mirror dm_region_hash dm_log dm_mod
[ 733.592500] CPU: 6 PID: 21123 Comm: tc Not tainted 4.18.0-rc8.latest+ #131
[ 733.600169] Hardware name: Dell Inc. PowerEdge R730/0WCJNT, BIOS 2.1.5 04/11/2016
[ 733.608518] RIP: 0010:cbq_destroy_class+0x5d/0x70 [sch_cbq]
[ 733.614734] Code: e7 d9 d2 48 8b 7b 48 e8 61 05 da d2 48 8d bb f8 00 00 00 e8 75 ae d5 d2 48 39 eb 74 0a 48 89 df 5b 5d e9 16 6c 94 d2 5b 5d c3 <0f> 0b eb b6 0f 1f 44 00 00 66 2e 0f 1f 84
[ 733.635798] RSP: 0018:ffffbfbb066bb9d8 EFLAGS: 00010202
[ 733.641627] RAX: 0000000000000001 RBX: ffff9cdd17392800 RCX: 000000008010000f
[ 733.649588] RDX: ffff9cdd1df547e0 RSI: ffff9cdd17392800 RDI: ffff9cdd0f84c800
[ 733.657547] RBP: ffff9cdd0f84c800 R08: 0000000000000001 R09: 0000000000000000
[ 733.665508] R10: ffff9cdd0f84d000 R11: 0000000000000001 R12: 0000000000000001
[ 733.673469] R13: 0000000000000000 R14: 0000000000000001 R15: ffff9cdd17392200
[ 733.681430] FS: 00007f911890a740(0000) GS:ffff9cdd1f8c0000(0000) knlGS:0000000000000000
[ 733.690456] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 733.696864] CR2: 0000000000b5544c CR3: 0000000859374002 CR4: 00000000001606e0
[ 733.704826] Call Trace:
[ 733.707554] cbq_destroy+0xa1/0xd0 [sch_cbq]
[ 733.712318] qdisc_destroy+0x62/0x130
[ 733.716401] dsmark_destroy+0x2a/0x70 [sch_dsmark]
[ 733.721745] qdisc_destroy+0x62/0x130
[ 733.725829] qdisc_graft+0x3ba/0x470
[ 733.729817] tc_get_qdisc+0x2a6/0x2c0
[ 733.733901] ? cred_has_capability+0x7d/0x130
[ 733.738761] rtnetlink_rcv_msg+0x263/0x2d0
[ 733.743330] ? rtnl_calcit.isra.30+0x110/0x110
[ 733.748287] netlink_rcv_skb+0x4d/0x130
[ 733.752576] netlink_unicast+0x1a3/0x250
[ 733.756949] netlink_sendmsg+0x2ae/0x3a0
[ 733.761324] sock_sendmsg+0x36/0x40
[ 733.765213] ___sys_sendmsg+0x26f/0x2d0
[ 733.769493] ? handle_pte_fault+0x586/0xdf0
[ 733.774158] ? __handle_mm_fault+0x389/0x500
[ 733.778919] ? __sys_sendmsg+0x5e/0xa0
[ 733.783099] __sys_sendmsg+0x5e/0xa0
[ 733.787087] do_syscall_64+0x5b/0x180
[ 733.791171] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[ 733.796805] RIP: 0033:0x7f9117f23f10
[ 733.800791] Code: c3 48 8b 05 82 6f 2c 00 f7 db 64 89 18 48 83 cb ff eb dd 0f 1f 80 00 00 00 00 83 3d 8d d0 2c 00 00 75 10 b8 2e 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 31 c3 48 83 ec 08 e8
[ 733.821873] RSP: 002b:00007ffe96818398 EFLAGS: 00000246 ORIG_RAX: 000000000000002e
[ 733.830319] RAX: ffffffffffffffda RBX: 000000005b71244c RCX: 00007f9117f23f10
[ 733.838280] RDX: 0000000000000000 RSI: 00007ffe968183e0 RDI: 0000000000000003
[ 733.846241] RBP: 00007ffe968183e0 R08: 000000000000ffff R09: 0000000000000003
[ 733.854202] R10: 00007ffe96817e20 R11: 0000000000000246 R12: 0000000000000000
[ 733.862161] R13: 0000000000662ee0 R14: 0000000000000000 R15: 0000000000000000
[ 733.870121] ---[ end trace 28edd4aad712ddca ]---
This is because we didn't update f->result.res when create new filter. Then in
tcindex_delete() -> tcf_unbind_filter(), we will failed to find out the res
and unbind filter, which will trigger the WARN_ON() in cbq_destroy_class().
Fix it by updating f->result.res when create new filter.
Fixes: 6e0565697ashuali@redhat.com >
Signed-off-by: Hangbin Liu <liuhangbin@gmail.com >
Acked-by: Cong Wang <xiyou.wangcong@gmail.com >
Signed-off-by: David S. Miller <davem@davemloft.net >
2018-08-13 19:37:42 -07:00
Hangbin Liu
2df8bee565
net_sched: fix NULL pointer dereference when delete tcindex filter
...
Li Shuang reported the following crash:
[ 71.267724] BUG: unable to handle kernel NULL pointer dereference at 0000000000000004
[ 71.276456] PGD 800000085d9bd067 P4D 800000085d9bd067 PUD 859a0b067 PMD 0
[ 71.284127] Oops: 0000 [#1 ] SMP PTI
[ 71.288015] CPU: 12 PID: 2386 Comm: tc Not tainted 4.18.0-rc8.latest+ #131
[ 71.295686] Hardware name: Dell Inc. PowerEdge R730/0WCJNT, BIOS 2.1.5 04/11/2016
[ 71.304037] RIP: 0010:tcindex_delete+0x72/0x280 [cls_tcindex]
[ 71.310446] Code: 00 31 f6 48 87 75 20 48 85 f6 74 11 48 8b 47 18 48 8b 40 08 48 8b 40 50 e8 fb a6 f8 fc 48 85 db 0f 84 dc 00 00 00 48 8b 73 18 <8b> 56 04 48 8d 7e 04 85 d2 0f 84 7b 01 00
[ 71.331517] RSP: 0018:ffffb45207b3f898 EFLAGS: 00010282
[ 71.337345] RAX: ffff8ad3d72d6360 RBX: ffff8acc84393680 RCX: 000000000000002e
[ 71.345306] RDX: ffff8ad3d72c8570 RSI: 0000000000000000 RDI: ffff8ad847a45800
[ 71.353277] RBP: ffff8acc84393688 R08: ffff8ad3d72c8400 R09: 0000000000000000
[ 71.361238] R10: ffff8ad3de786e00 R11: 0000000000000000 R12: ffffb45207b3f8c7
[ 71.369199] R13: ffff8ad3d93bd2a0 R14: 000000000000002e R15: ffff8ad3d72c9600
[ 71.377161] FS: 00007f9d3ec3e740(0000) GS:ffff8ad3df980000(0000) knlGS:0000000000000000
[ 71.386188] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 71.392597] CR2: 0000000000000004 CR3: 0000000852f06003 CR4: 00000000001606e0
[ 71.400558] Call Trace:
[ 71.403299] tcindex_destroy_element+0x25/0x40 [cls_tcindex]
[ 71.409611] tcindex_walk+0xbb/0x110 [cls_tcindex]
[ 71.414953] tcindex_destroy+0x44/0x90 [cls_tcindex]
[ 71.420492] ? tcindex_delete+0x280/0x280 [cls_tcindex]
[ 71.426323] tcf_proto_destroy+0x16/0x40
[ 71.430696] tcf_chain_flush+0x51/0x70
[ 71.434876] tcf_block_put_ext.part.30+0x8f/0x1b0
[ 71.440122] tcf_block_put+0x4d/0x70
[ 71.444108] cbq_destroy+0x4d/0xd0 [sch_cbq]
[ 71.448869] qdisc_destroy+0x62/0x130
[ 71.452951] dsmark_destroy+0x2a/0x70 [sch_dsmark]
[ 71.458300] qdisc_destroy+0x62/0x130
[ 71.462373] qdisc_graft+0x3ba/0x470
[ 71.466359] tc_get_qdisc+0x2a6/0x2c0
[ 71.470443] ? cred_has_capability+0x7d/0x130
[ 71.475307] rtnetlink_rcv_msg+0x263/0x2d0
[ 71.479875] ? rtnl_calcit.isra.30+0x110/0x110
[ 71.484832] netlink_rcv_skb+0x4d/0x130
[ 71.489109] netlink_unicast+0x1a3/0x250
[ 71.493482] netlink_sendmsg+0x2ae/0x3a0
[ 71.497859] sock_sendmsg+0x36/0x40
[ 71.501748] ___sys_sendmsg+0x26f/0x2d0
[ 71.506029] ? handle_pte_fault+0x586/0xdf0
[ 71.510694] ? __handle_mm_fault+0x389/0x500
[ 71.515457] ? __sys_sendmsg+0x5e/0xa0
[ 71.519636] __sys_sendmsg+0x5e/0xa0
[ 71.523626] do_syscall_64+0x5b/0x180
[ 71.527711] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[ 71.533345] RIP: 0033:0x7f9d3e257f10
[ 71.537331] Code: c3 48 8b 05 82 6f 2c 00 f7 db 64 89 18 48 83 cb ff eb dd 0f 1f 80 00 00 00 00 83 3d 8d d0 2c 00 00 75 10 b8 2e 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 31 c3 48 83 ec 08 e8
[ 71.558401] RSP: 002b:00007fff6f893398 EFLAGS: 00000246 ORIG_RAX: 000000000000002e
[ 71.566848] RAX: ffffffffffffffda RBX: 000000005b71274d RCX: 00007f9d3e257f10
[ 71.574810] RDX: 0000000000000000 RSI: 00007fff6f8933e0 RDI: 0000000000000003
[ 71.582770] RBP: 00007fff6f8933e0 R08: 000000000000ffff R09: 0000000000000003
[ 71.590729] R10: 00007fff6f892e20 R11: 0000000000000246 R12: 0000000000000000
[ 71.598689] R13: 0000000000662ee0 R14: 0000000000000000 R15: 0000000000000000
[ 71.606651] Modules linked in: sch_cbq cls_tcindex sch_dsmark xt_CHECKSUM iptable_mangle ipt_MASQUERADE iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_coni
[ 71.685425] libahci i2c_algo_bit i2c_core i40e libata dca mdio megaraid_sas dm_mirror dm_region_hash dm_log dm_mod
[ 71.697075] CR2: 0000000000000004
[ 71.700792] ---[ end trace f604eb1acacd978b ]---
Reproducer:
tc qdisc add dev lo handle 1:0 root dsmark indices 64 set_tc_index
tc filter add dev lo parent 1:0 protocol ip prio 1 tcindex mask 0xfc shift 2
tc qdisc add dev lo parent 1:0 handle 2:0 cbq bandwidth 10Mbit cell 8 avpkt 1000 mpu 64
tc class add dev lo parent 2:0 classid 2:1 cbq bandwidth 10Mbit rate 1500Kbit avpkt 1000 prio 1 bounded isolated allot 1514 weight 1 maxburst 10
tc filter add dev lo parent 2:0 protocol ip prio 1 handle 0x2e tcindex classid 2:1 pass_on
tc qdisc add dev lo parent 2:1 pfifo limit 5
tc qdisc del dev lo root
This is because in tcindex_set_parms, when there is no old_r, we set new
exts to cr.exts. And we didn't set it to filter when r == &new_filter_result.
Then in tcindex_delete() -> tcf_exts_get_net(), we will get NULL pointer
dereference as we didn't init exts.
Fix it by moving tcf_exts_change() after "if (old_r && old_r != r)" check.
Then we don't need "cr" as there is no errout after that.
Fixes: bf63ac73b3shuali@redhat.com >
Signed-off-by: Hangbin Liu <liuhangbin@gmail.com >
Acked-by: Cong Wang <xiyou.wangcong@gmail.com >
Signed-off-by: David S. Miller <davem@davemloft.net >
2018-08-13 19:37:42 -07:00
Vlad Buslov
42c625a486
net: sched: act_ife: disable bh when taking ife_mod_lock
...
Lockdep reports deadlock for following locking scenario in ife action:
Task one:
1) Executes ife action update.
2) Takes tcfa_lock.
3) Waits on ife_mod_lock which is already taken by task two.
Task two:
1) Executes any path that obtains ife_mod_lock without disabling bh (any
path that takes ife_mod_lock while holding tcfa_lock has bh disabled) like
loading a meta module, or creating new action.
2) Takes ife_mod_lock.
3) Task is preempted by rate estimator timer.
4) Timer callback waits on tcfa_lock which is taken by task one.
In described case tasks deadlock because they take same two locks in
different order. To prevent potential deadlock reported by lockdep, always
disable bh when obtaining ife_mod_lock.
Lockdep warning:
[ 508.101192] =====================================================
[ 508.107708] WARNING: SOFTIRQ-safe -> SOFTIRQ-unsafe lock order detected
[ 508.114728] 4.18.0-rc8+ #646 Not tainted
[ 508.119050] -----------------------------------------------------
[ 508.125559] tc/5460 [HC0[0]:SC0[2]:HE1:SE0] is trying to acquire:
[ 508.132025] 000000005a938c68 (ife_mod_lock){++++}, at: find_ife_oplist+0x1e/0xc0 [act_ife]
[ 508.140996]
and this task is already holding:
[ 508.147548] 00000000d46f6c56 (&(&p->tcfa_lock)->rlock){+.-.}, at: tcf_ife_init+0x6ae/0xf40 [act_ife]
[ 508.157371] which would create a new lock dependency:
[ 508.162828] (&(&p->tcfa_lock)->rlock){+.-.} -> (ife_mod_lock){++++}
[ 508.169572]
but this new dependency connects a SOFTIRQ-irq-safe lock:
[ 508.178197] (&(&p->tcfa_lock)->rlock){+.-.}
[ 508.178201]
... which became SOFTIRQ-irq-safe at:
[ 508.189771] _raw_spin_lock+0x2c/0x40
[ 508.193906] est_fetch_counters+0x41/0xb0
[ 508.198391] est_timer+0x83/0x3c0
[ 508.202180] call_timer_fn+0x16a/0x5d0
[ 508.206400] run_timer_softirq+0x399/0x920
[ 508.210967] __do_softirq+0x157/0x97d
[ 508.215102] irq_exit+0x152/0x1c0
[ 508.218888] smp_apic_timer_interrupt+0xc0/0x4e0
[ 508.223976] apic_timer_interrupt+0xf/0x20
[ 508.228540] cpuidle_enter_state+0xf8/0x5d0
[ 508.233198] do_idle+0x28a/0x350
[ 508.236881] cpu_startup_entry+0xc7/0xe0
[ 508.241296] start_secondary+0x2e8/0x3f0
[ 508.245678] secondary_startup_64+0xa5/0xb0
[ 508.250347]
to a SOFTIRQ-irq-unsafe lock: (ife_mod_lock){++++}
[ 508.256531]
... which became SOFTIRQ-irq-unsafe at:
[ 508.267279] ...
[ 508.267283] _raw_write_lock+0x2c/0x40
[ 508.273653] register_ife_op+0x118/0x2c0 [act_ife]
[ 508.278926] do_one_initcall+0xf7/0x4d9
[ 508.283214] do_init_module+0x18b/0x44e
[ 508.287521] load_module+0x4167/0x5730
[ 508.291739] __do_sys_finit_module+0x16d/0x1a0
[ 508.296654] do_syscall_64+0x7a/0x3f0
[ 508.300788] entry_SYSCALL_64_after_hwframe+0x49/0xbe
[ 508.306302]
other info that might help us debug this:
[ 508.315286] Possible interrupt unsafe locking scenario:
[ 508.322771] CPU0 CPU1
[ 508.327681] ---- ----
[ 508.332604] lock(ife_mod_lock);
[ 508.336300] local_irq_disable();
[ 508.342608] lock(&(&p->tcfa_lock)->rlock);
[ 508.349793] lock(ife_mod_lock);
[ 508.355990] <Interrupt>
[ 508.358974] lock(&(&p->tcfa_lock)->rlock);
[ 508.363803]
*** DEADLOCK ***
[ 508.370715] 2 locks held by tc/5460:
[ 508.374680] #0 : 00000000e27e4fa4 (rtnl_mutex){+.+.}, at: rtnetlink_rcv_msg+0x583/0x7b0
[ 508.383366] #1 : 00000000d46f6c56 (&(&p->tcfa_lock)->rlock){+.-.}, at: tcf_ife_init+0x6ae/0xf40 [act_ife]
[ 508.393648]
the dependencies between SOFTIRQ-irq-safe lock and the holding lock:
[ 508.403505] -> (&(&p->tcfa_lock)->rlock){+.-.} ops: 1001553 {
[ 508.409646] HARDIRQ-ON-W at:
[ 508.413136] _raw_spin_lock_bh+0x34/0x40
[ 508.419059] gnet_stats_start_copy_compat+0xa2/0x230
[ 508.426021] gnet_stats_start_copy+0x16/0x20
[ 508.432333] tcf_action_copy_stats+0x95/0x1d0
[ 508.438735] tcf_action_dump_1+0xb0/0x4e0
[ 508.444795] tcf_action_dump+0xca/0x200
[ 508.450673] tcf_exts_dump+0xd9/0x320
[ 508.456392] fl_dump+0x1b7/0x4a0 [cls_flower]
[ 508.462798] tcf_fill_node+0x380/0x530
[ 508.468601] tfilter_notify+0xdf/0x1c0
[ 508.474404] tc_new_tfilter+0x84a/0xc90
[ 508.480270] rtnetlink_rcv_msg+0x5bd/0x7b0
[ 508.486419] netlink_rcv_skb+0x184/0x220
[ 508.492394] netlink_unicast+0x31b/0x460
[ 508.507411] netlink_sendmsg+0x3fb/0x840
[ 508.513390] sock_sendmsg+0x7b/0xd0
[ 508.518907] ___sys_sendmsg+0x4c6/0x610
[ 508.524797] __sys_sendmsg+0xd7/0x150
[ 508.530510] do_syscall_64+0x7a/0x3f0
[ 508.536201] entry_SYSCALL_64_after_hwframe+0x49/0xbe
[ 508.543301] IN-SOFTIRQ-W at:
[ 508.546834] _raw_spin_lock+0x2c/0x40
[ 508.552522] est_fetch_counters+0x41/0xb0
[ 508.558571] est_timer+0x83/0x3c0
[ 508.563912] call_timer_fn+0x16a/0x5d0
[ 508.569699] run_timer_softirq+0x399/0x920
[ 508.575840] __do_softirq+0x157/0x97d
[ 508.581538] irq_exit+0x152/0x1c0
[ 508.586882] smp_apic_timer_interrupt+0xc0/0x4e0
[ 508.593533] apic_timer_interrupt+0xf/0x20
[ 508.599686] cpuidle_enter_state+0xf8/0x5d0
[ 508.605895] do_idle+0x28a/0x350
[ 508.611147] cpu_startup_entry+0xc7/0xe0
[ 508.617097] start_secondary+0x2e8/0x3f0
[ 508.623029] secondary_startup_64+0xa5/0xb0
[ 508.629245] INITIAL USE at:
[ 508.632686] _raw_spin_lock_bh+0x34/0x40
[ 508.638557] gnet_stats_start_copy_compat+0xa2/0x230
[ 508.645491] gnet_stats_start_copy+0x16/0x20
[ 508.651719] tcf_action_copy_stats+0x95/0x1d0
[ 508.657992] tcf_action_dump_1+0xb0/0x4e0
[ 508.663937] tcf_action_dump+0xca/0x200
[ 508.669716] tcf_exts_dump+0xd9/0x320
[ 508.675337] fl_dump+0x1b7/0x4a0 [cls_flower]
[ 508.681650] tcf_fill_node+0x380/0x530
[ 508.687366] tfilter_notify+0xdf/0x1c0
[ 508.693031] tc_new_tfilter+0x84a/0xc90
[ 508.698820] rtnetlink_rcv_msg+0x5bd/0x7b0
[ 508.704869] netlink_rcv_skb+0x184/0x220
[ 508.710758] netlink_unicast+0x31b/0x460
[ 508.716627] netlink_sendmsg+0x3fb/0x840
[ 508.722510] sock_sendmsg+0x7b/0xd0
[ 508.727931] ___sys_sendmsg+0x4c6/0x610
[ 508.733729] __sys_sendmsg+0xd7/0x150
[ 508.739346] do_syscall_64 +0x7a/0x3f0
[ 508.744943] entry_SYSCALL_64_after_hwframe+0x49/0xbe
[ 508.751930] }
[ 508.753964] ... key at: [<ffffffff916b3e20>] __key.61145+0x0/0x40
[ 508.760946] ... acquired at:
[ 508.764294] _raw_read_lock+0x2f/0x40
[ 508.768513] find_ife_oplist+0x1e/0xc0 [act_ife]
[ 508.773692] tcf_ife_init+0x82f/0xf40 [act_ife]
[ 508.778785] tcf_action_init_1+0x510/0x750
[ 508.783468] tcf_action_init+0x1e8/0x340
[ 508.787938] tcf_action_add+0xc5/0x240
[ 508.792241] tc_ctl_action+0x203/0x2a0
[ 508.796550] rtnetlink_rcv_msg+0x5bd/0x7b0
[ 508.801200] netlink_rcv_skb+0x184/0x220
[ 508.805674] netlink_unicast+0x31b/0x460
[ 508.810129] netlink_sendmsg+0x3fb/0x840
[ 508.814611] sock_sendmsg+0x7b/0xd0
[ 508.818665] ___sys_sendmsg+0x4c6/0x610
[ 508.823029] __sys_sendmsg+0xd7/0x150
[ 508.827246] do_syscall_64+0x7a/0x3f0
[ 508.831483] entry_SYSCALL_64_after_hwframe+0x49/0xbe
the dependencies between the lock to be acquired
[ 508.838945] and SOFTIRQ-irq-unsafe lock:
[ 508.851177] -> (ife_mod_lock){++++} ops: 95 {
[ 508.855920] HARDIRQ-ON-W at:
[ 508.859478] _raw_write_lock+0x2c/0x40
[ 508.865264] register_ife_op+0x118/0x2c0 [act_ife]
[ 508.872071] do_one_initcall+0xf7/0x4d9
[ 508.877947] do_init_module+0x18b/0x44e
[ 508.883819] load_module+0x4167/0x5730
[ 508.889595] __do_sys_finit_module+0x16d/0x1a0
[ 508.896043] do_syscall_64+0x7a/0x3f0
[ 508.901734] entry_SYSCALL_64_after_hwframe+0x49/0xbe
[ 508.908827] HARDIRQ-ON-R at:
[ 508.912359] _raw_read_lock+0x2f/0x40
[ 508.918043] find_ife_oplist+0x1e/0xc0 [act_ife]
[ 508.924692] tcf_ife_init+0x82f/0xf40 [act_ife]
[ 508.931252] tcf_action_init_1+0x510/0x750
[ 508.937393] tcf_action_init+0x1e8/0x340
[ 508.943366] tcf_action_add+0xc5/0x240
[ 508.949130] tc_ctl_action+0x203/0x2a0
[ 508.954922] rtnetlink_rcv_msg+0x5bd/0x7b0
[ 508.961024] netlink_rcv_skb+0x184/0x220
[ 508.966970] netlink_unicast+0x31b/0x460
[ 508.972915] netlink_sendmsg+0x3fb/0x840
[ 508.978859] sock_sendmsg+0x7b/0xd0
[ 508.984400] ___sys_sendmsg+0x4c6/0x610
[ 508.990264] __sys_sendmsg+0xd7/0x150
[ 508.995952] do_syscall_64+0x7a/0x3f0
[ 509.001643] entry_SYSCALL_64_after_hwframe+0x49/0xbe
[ 509.008722] SOFTIRQ-ON-W at:\
[ 509.012242] _raw_write_lock+0x2c/0x40
[ 509.018013] register_ife_op+0x118/0x2c0 [act_ife]
[ 509.024841] do_one_initcall+0xf7/0x4d9
[ 509.030720] do_init_module+0x18b/0x44e
[ 509.036604] load_module+0x4167/0x5730
[ 509.042397] __do_sys_finit_module+0x16d/0x1a0
[ 509.048865] do_syscall_64+0x7a/0x3f0
[ 509.054551] entry_SYSCALL_64_after_hwframe+0x49/0xbe
[ 509.061636] SOFTIRQ-ON-R at:
[ 509.065145] _raw_read_lock+0x2f/0x40
[ 509.070854] find_ife_oplist+0x1e/0xc0 [act_ife]
[ 509.077515] tcf_ife_init+0x82f/0xf40 [act_ife]
[ 509.084051] tcf_action_init_1+0x510/0x750
[ 509.090172] tcf_action_init+0x1e8/0x340
[ 509.096124] tcf_action_add+0xc5/0x240
[ 509.101891] tc_ctl_action+0x203/0x2a0
[ 509.107671] rtnetlink_rcv_msg+0x5bd/0x7b0
[ 509.113811] netlink_rcv_skb+0x184/0x220
[ 509.119768] netlink_unicast+0x31b/0x460
[ 509.125716] netlink_sendmsg+0x3fb/0x840
[ 509.131668] sock_sendmsg+0x7b/0xd0
[ 509.137167] ___sys_sendmsg+0x4c6/0x610
[ 509.143010] __sys_sendmsg+0xd7/0x150
[ 509.148718] do_syscall_64+0x7a/0x3f0
[ 509.154443] entry_SYSCALL_64_after_hwframe+0x49/0xbe
[ 509.161533] INITIAL USE at:
[ 509.164956] _raw_read_lock+0x2f/0x40
[ 509.170574] find_ife_oplist+0x1e/0xc0 [act_ife]
[ 509.177134] tcf_ife_init+0x82f/0xf40 [act_ife]
[ 509.183619] tcf_action_init_1+0x510/0x750
[ 509.189674] tcf_action_init+0x1e8/0x340
[ 509.195534] tcf_action_add+0xc5/0x240
[ 509.201229] tc_ctl_action+0x203/0x2a0
[ 509.206920] rtnetlink_rcv_msg+0x5bd/0x7b0
[ 509.212936] netlink_rcv_skb+0x184/0x220
[ 509.218818] netlink_unicast+0x31b/0x460
[ 509.224699] netlink_sendmsg+0x3fb/0x840
[ 509.230581] sock_sendmsg+0x7b/0xd0
[ 509.235984] ___sys_sendmsg+0x4c6/0x610
[ 509.241791] __sys_sendmsg+0xd7/0x150
[ 509.247425] do_syscall_64+0x7a/0x3f0
[ 509.253007] entry_SYSCALL_64_after_hwframe+0x49/0xbe
[ 509.259975] }
[ 509.261998] ... key at: [<ffffffffc1554258>] ife_mod_lock+0x18/0xffffffffffff8dc0 [act_ife]
[ 509.271569] ... acquired at:
[ 509.274912] _raw_read_lock+0x2f/0x40
[ 509.279134] find_ife_oplist+0x1e/0xc0 [act_ife]
[ 509.284324] tcf_ife_init+0x82f/0xf40 [act_ife]
[ 509.289425] tcf_action_init_1+0x510/0x750
[ 509.294068] tcf_action_init+0x1e8/0x340
[ 509.298553] tcf_action_add+0xc5/0x240
[ 509.302854] tc_ctl_action+0x203/0x2a0
[ 509.307153] rtnetlink_rcv_msg+0x5bd/0x7b0
[ 509.311805] netlink_rcv_skb+0x184/0x220
[ 509.316282] netlink_unicast+0x31b/0x460
[ 509.320769] netlink_sendmsg+0x3fb/0x840
[ 509.325248] sock_sendmsg+0x7b/0xd0
[ 509.329290] ___sys_sendmsg+0x4c6/0x610
[ 509.333687] __sys_sendmsg+0xd7/0x150
[ 509.337902] do_syscall_64+0x7a/0x3f0
[ 509.342116] entry_SYSCALL_64_after_hwframe+0x49/0xbe
[ 509.349601]
stack backtrace:
[ 509.354663] CPU: 6 PID: 5460 Comm: tc Not tainted 4.18.0-rc8+ #646
[ 509.361216] Hardware name: Supermicro SYS-2028TP-DECR/X10DRT-P, BIOS 2.0b 03/30/2017
Fixes: ef6980b6bevladbu@mellanox.com >
Acked-by: Jamal Hadi Salim <jhs@mojatatu.com >
Signed-off-by: David S. Miller <davem@davemloft.net >
2018-08-13 11:45:06 -07:00
Jamal Hadi Salim
7c5790c4da
net: sched: act_mirred method rename for grep-ability and consistency
...
Signed-off-by: Jamal Hadi Salim <jhs@mojatatu.com >
Signed-off-by: David S. Miller <davem@davemloft.net >
2018-08-13 09:06:17 -07:00
Jamal Hadi Salim
8aa7f22e56
net: sched: act_vlan method rename for grep-ability and consistency
...
Signed-off-by: Jamal Hadi Salim <jhs@mojatatu.com >
Signed-off-by: David S. Miller <davem@davemloft.net >
2018-08-13 09:06:17 -07:00
Jamal Hadi Salim
353d2c253f
net: sched: act_skbmod method rename for grep-ability and consistency
...
Signed-off-by: Jamal Hadi Salim <jhs@mojatatu.com >
Signed-off-by: David S. Miller <davem@davemloft.net >
2018-08-13 09:06:17 -07:00
Jamal Hadi Salim
45da1dac61
net: sched: act_skbedit method rename for grep-ability and consistency
...
Signed-off-by: Jamal Hadi Salim <jhs@mojatatu.com >
Signed-off-by: David S. Miller <davem@davemloft.net >
2018-08-13 09:06:17 -07:00
Jamal Hadi Salim
798de374e5
net: sched: act_simple method rename for grep-ability and consistency
...
Signed-off-by: Jamal Hadi Salim <jhs@mojatatu.com >
Signed-off-by: David S. Miller <davem@davemloft.net >
2018-08-13 09:06:17 -07:00
Jamal Hadi Salim
2ac063474d
net: sched: act_police method rename for grep-ability and consistency
...
Signed-off-by: Jamal Hadi Salim <jhs@mojatatu.com >
Signed-off-by: David S. Miller <davem@davemloft.net >
2018-08-13 09:06:17 -07:00
Jamal Hadi Salim
6a2b401cd1
net: sched: act_pedit method rename for grep-ability and consistency
...
Signed-off-by: Jamal Hadi Salim <jhs@mojatatu.com >
Signed-off-by: David S. Miller <davem@davemloft.net >
2018-08-13 09:06:17 -07:00
Jamal Hadi Salim
0390514fe1
net: sched: act_nat method rename for grep-ability and consistency
...
Signed-off-by: Jamal Hadi Salim <jhs@mojatatu.com >
Signed-off-by: David S. Miller <davem@davemloft.net >
2018-08-13 09:06:16 -07:00
Jamal Hadi Salim
11b9695b3f
net: sched: act_ipt method rename for grep-ability and consistency
...
Signed-off-by: Jamal Hadi Salim <jhs@mojatatu.com >
Signed-off-by: David S. Miller <davem@davemloft.net >
2018-08-13 09:06:16 -07:00
Jamal Hadi Salim
1740005e2a
net: sched: act_gact method rename for grep-ability and consistency
...
Signed-off-by: Jamal Hadi Salim <jhs@mojatatu.com >
Signed-off-by: David S. Miller <davem@davemloft.net >
2018-08-13 09:06:16 -07:00
Jamal Hadi Salim
c831549c3f
net: sched: act_sum method rename for grep-ability and consistency
...
Signed-off-by: Jamal Hadi Salim <jhs@mojatatu.com >
Signed-off-by: David S. Miller <davem@davemloft.net >
2018-08-13 09:06:16 -07:00
Jamal Hadi Salim
2fbec27f81
net: sched: act_bpf method rename for grep-ability and consistency
...
Signed-off-by: Jamal Hadi Salim <jhs@mojatatu.com >
Signed-off-by: David S. Miller <davem@davemloft.net >
2018-08-13 09:06:16 -07:00
Jamal Hadi Salim
962ad1f937
net: sched: act_connmark method rename for grep-ability and consistency
...
Signed-off-by: Jamal Hadi Salim <jhs@mojatatu.com >
Signed-off-by: David S. Miller <davem@davemloft.net >
2018-08-13 09:06:16 -07:00
Vlad Buslov
e329bc4273
net: sched: act_police: remove dependency on rtnl lock
...
Use tcf spinlock to protect police action private data from concurrent
modification during dump. (init already uses tcf spinlock when changing
police action state)
Pass tcf spinlock as estimator lock argument to gen_replace_estimator()
during action init.
Signed-off-by: Vlad Buslov <vladbu@mellanox.com >
Signed-off-by: David S. Miller <davem@davemloft.net >
2018-08-11 12:37:10 -07:00
Vlad Buslov
4e232818bd
net: sched: act_mirred: remove dependency on rtnl lock
...
Re-introduce mirred list spinlock, that was removed some time ago, in order
to protect it from concurrent modifications, instead of relying on rtnl
lock.
Use tcf spinlock to protect mirred action private data from concurrent
modification in init and dump. Rearrange access to mirred data in order to
be performed only while holding the lock.
Rearrange net dev access to always hold reference while working with it,
instead of relying on rntl lock.
Signed-off-by: Vlad Buslov <vladbu@mellanox.com >
Signed-off-by: David S. Miller <davem@davemloft.net >
2018-08-11 12:37:10 -07:00
Vlad Buslov
84a75b329b
net: sched: extend action ops with put_dev callback
...
As a preparation for removing dependency on rtnl lock from rules update
path, all users of shared objects must take reference while working with
them.
Extend action ops with put_dev() API to be used on net device returned by
get_dev().
Modify mirred action (only action that implements get_dev callback):
- Take reference to net device in get_dev.
- Implement put_dev API that releases reference to net device.
Signed-off-by: Vlad Buslov <vladbu@mellanox.com >
Signed-off-by: David S. Miller <davem@davemloft.net >
2018-08-11 12:37:10 -07:00
Vlad Buslov
764e9a2448
net: sched: act_vlan: remove dependency on rtnl lock
...
Use tcf spinlock to protect vlan action private data from concurrent
modification during dump and init. Use rcu swap operation to reassign
params pointer under protection of tcf lock. (old params value is not used
by init, so there is no need of standalone rcu dereference step)
Remove rtnl assertion that is no longer necessary.
Signed-off-by: Vlad Buslov <vladbu@mellanox.com >
Signed-off-by: David S. Miller <davem@davemloft.net >
2018-08-11 12:37:10 -07:00
Vlad Buslov
729e012609
net: sched: act_tunnel_key: remove dependency on rtnl lock
...
Use tcf lock to protect tunnel key action struct private data from
concurrent modification in init and dump. Use rcu swap operation to
reassign params pointer under protection of tcf lock. (old params value is
not used by init, so there is no need of standalone rcu dereference step)
Remove rtnl lock assertion that is no longer required.
Signed-off-by: Vlad Buslov <vladbu@mellanox.com >
Signed-off-by: David S. Miller <davem@davemloft.net >
2018-08-11 12:37:10 -07:00
Vlad Buslov
c8814552fe
net: sched: act_skbmod: remove dependency on rtnl lock
...
Move read of skbmod_p rcu pointer to be protected by tcf spinlock. Use tcf
spinlock to protect private skbmod data from concurrent modification during
dump.
Signed-off-by: Vlad Buslov <vladbu@mellanox.com >
Signed-off-by: David S. Miller <davem@davemloft.net >
2018-08-11 12:37:09 -07:00
