TCP LAST ACK incorrectly treated as invalid

Discussion:

vDev

2014-10-21 01:58:10 UTC

It looks like TCP conntrack is incorrectly treating "LAST ACK" (ACK
from peer for last FIN) as the sequence number comparison does not
take into account that the last ACK sequence number will be one
greater than the previous sequence number.

Here's the code that seems to be the cause of it in tcp_in_window()
[nf_conntrack_proto_tcp.c]:

pr_debug("tcp_in_window: I=%i II=%i III=%i IV=%i\n",
before(seq, sender->td_maxend + 1),
after(end, sender->td_end - receiver->td_maxwin - 1),
before(sack, receiver->td_end + 1),
after(sack, receiver->td_end - MAXACKWINDOW(sender) - 1));

if (before(seq, sender->td_maxend + 1) &&
after(end, sender->td_end - receiver->td_maxwin - 1) &&
before(sack, receiver->td_end + 1) &&
after(sack, receiver->td_end - MAXACKWINDOW(sender) - 1)) {

III in pr_debug will be 0 as "before(sack, receiver->td_end + 1)" will
evaluate to false as sack will be equal to (receiver->td_end + 1) for
the last ACK. The sequence number will be one greater in the last ACK.

Any feedback will be helpful. Thanks for looking into this.
--
To unsubscribe from this list: send the line "unsubscribe netfilter" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Jozsef Kadlecsik

2014-10-21 06:57:42 UTC

Permalink

Post by vDev
It looks like TCP conntrack is incorrectly treating "LAST ACK" (ACK
from peer for last FIN) as the sequence number comparison does not
take into account that the last ACK sequence number will be one
greater than the previous sequence number.
Here's the code that seems to be the cause of it in tcp_in_window()
pr_debug("tcp_in_window: I=%i II=%i III=%i IV=%i\n",
before(seq, sender->td_maxend + 1),
after(end, sender->td_end - receiver->td_maxwin - 1),
before(sack, receiver->td_end + 1),
after(sack, receiver->td_end - MAXACKWINDOW(sender) - 1));
if (before(seq, sender->td_maxend + 1) &&
after(end, sender->td_end - receiver->td_maxwin - 1) &&
before(sack, receiver->td_end + 1) &&
after(sack, receiver->td_end - MAXACKWINDOW(sender) - 1)) {
III in pr_debug will be 0 as "before(sack, receiver->td_end + 1)" will
evaluate to false as sack will be equal to (receiver->td_end + 1) for
the last ACK. The sequence number will be one greater in the last ACK.
Any feedback will be helpful. Thanks for looking into this.

Maybe you are not taking into account that the last ACK acks FIN+ACK and
FIN is counted in the sequence numbers and it's stored in td_end. Please
see segment_seq_plus_len() in nf_conntrack_proto_tcp.c.

Best regards,
Jozsef
-
E-mail : ***@blackhole.kfki.hu, ***@wigner.mta.hu
PGP key : http://www.kfki.hu/~kadlec/pgp_public_key.txt
Address : Wigner Research Centre for Physics, Hungarian Academy of Sciences
H-1525 Budapest 114, POB. 49, Hungary
--
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

vDev

2014-10-21 15:44:57 UTC

Permalink

Thanks, Jozsef. I see sequence number compensation in
segment_seq_plus_len() but does that not cover the ACK sequence
numbers. The variables (sack and receiver->td_end) are off by 1. It
seems like receiver->td_end must be incremented by 1 for last ACK.
Here's a trace:

[ 775.980000] seq=726609914 ack=94044192+(0) sack=94044192+(0)
win=14600 end=726609914
[ 775.980000] tcp_in_window: sender end=726609914 maxend=726611414
maxwin=14600 scale=0 receiver end=94044191 maxend=94058791 maxwin=1500
scale=0
[ 775.980000] tcp_in_window: I=1 II=1 III=0 IV=1
[ 775.980000] tcp_in_window: res=0 sender end=726609914
maxend=726611414 maxwin=14600 receiver end=94044191 maxend=94058791
maxwin=1500

As you can see above sack is 94044192 and receiver end is 94044191. As
a result III is 0 and res=0 which causes -NF_ACCEPT to be returned.

On Tue, Oct 21, 2014 at 1:57 AM, Jozsef Kadlecsik

Post by Jozsef Kadlecsik

Maybe you are not taking into account that the last ACK acks FIN+ACK and
FIN is counted in the sequence numbers and it's stored in td_end. Please
see segment_seq_plus_len() in nf_conntrack_proto_tcp.c.
Best regards,
Jozsef
-
PGP key : http://www.kfki.hu/~kadlec/pgp_public_key.txt
Address : Wigner Research Centre for Physics, Hungarian Academy of Sciences
H-1525 Budapest 114, POB. 49, Hungary

--
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Jozsef Kadlecsik

2014-10-21 18:39:56 UTC

Permalink

Post by vDev
Thanks, Jozsef. I see sequence number compensation in
segment_seq_plus_len() but does that not cover the ACK sequence
numbers. The variables (sack and receiver->td_end) are off by 1. It
seems like receiver->td_end must be incremented by 1 for last ACK.
[ 775.980000] seq=726609914 ack=94044192+(0) sack=94044192+(0)
win=14600 end=726609914
[ 775.980000] tcp_in_window: sender end=726609914 maxend=726611414
maxwin=14600 scale=0 receiver end=94044191 maxend=94058791 maxwin=1500
scale=0
[ 775.980000] tcp_in_window: I=1 II=1 III=0 IV=1
[ 775.980000] tcp_in_window: res=0 sender end=726609914
maxend=726611414 maxwin=14600 receiver end=94044191 maxend=94058791
maxwin=1500
As you can see above sack is 94044192 and receiver end is 94044191. As
a result III is 0 and res=0 which causes -NF_ACCEPT to be returned.

It's a single packet, it doesn't tell anything about the TCP stream.
A full TCP stream dump is required from the very first SYN to the very
last packet.

Best regards,
Jozsef
-
E-mail : ***@blackhole.kfki.hu, ***@wigner.mta.hu
PGP key : http://www.kfki.hu/~kadlec/pgp_public_key.txt
Address : Wigner Research Centre for Physics, Hungarian Academy of Sciences
H-1525 Budapest 114, POB. 49, Hungary
--
To unsubscribe from this list: send the line "unsubscribe netfilter" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

vDev

2014-10-21 19:48:46 UTC

Permalink

OK. Here it is. Please look for III=0 and subsequent res=0.

<7>[ 2238.060000] tcp_new: sender end=4004184065 maxend=4004184065
maxwin=1500 scale=0 receiver end=0 maxend=0 maxwin=0 scale=0
<7>[ 2238.060000] tcp_in_window: START
<7>[ 2238.060000] tcp_in_window:
<7>[ 2238.060000] seq=4004184064 ack=0+(0) sack=0+(0) win=1500 end=4004184065
<7>[ 2238.060000] tcp_in_window: sender end=4004184065
maxend=4004184065 maxwin=1500 scale=0 receiver end=0 maxend=0 maxwin=0
scale=0
<7>[ 2238.060000] tcp_in_window:
<7>[ 2238.060000] seq=4004184064 ack=0+(0) sack=0+(0) win=1500 end=4004184065
<7>[ 2238.060000] tcp_in_window: sender end=4004184065
maxend=4004184065 maxwin=1500 scale=0 receiver end=0 maxend=0 maxwin=0
scale=0
<7>[ 2238.060000] tcp_in_window: I=1 II=1 III=1 IV=1
<7>[ 2238.060000] tcp_in_window: res=1 sender end=4004184065
maxend=4004184065 maxwin=1500 receiver end=0 maxend=1500 maxwin=0
<7>[ 2238.060000] tcp_conntracks:
<7>[ 2238.060000] syn=1 ack=0 fin=0 rst=0 old=0 new=1
<7>[ 2238.110000] tcp_in_window: START
<7>[ 2238.110000] tcp_in_window:
<7>[ 2238.110000] seq=201772607 ack=4004184065+(0) sack=4004184065+(0)
win=14600 end=201772608
<7>[ 2238.110000] tcp_in_window: sender end=0 maxend=1500 maxwin=0
scale=0 receiver end=4004184065 maxend=4004184065 maxwin=1500 scale=0
<7>[ 2238.110000] tcp_in_window:
<7>[ 2238.110000] seq=201772607 ack=4004184065+(0) sack=4004184065+(0)
win=14600 end=201772608
<7>[ 2238.110000] tcp_in_window: sender end=201772608 maxend=201772608
maxwin=14600 scale=0 receiver end=4004184065 maxend=4004184065
maxwin=1500 scale=0
<7>[ 2238.110000] tcp_in_window: I=1 II=1 III=1 IV=1
<7>[ 2238.110000] tcp_in_window: res=1 sender end=201772608
maxend=201772608 maxwin=14600 receiver end=4004184065
maxend=4004198665 maxwin=1500
<7>[ 2238.110000] tcp_conntracks:
<7>[ 2238.110000] syn=1 ack=1 fin=0 rst=0 old=1 new=2
<7>[ 2238.120000] tcp_in_window: START
<7>[ 2238.120000] tcp_in_window:
<7>[ 2238.120000] seq=4004184065 ack=201772608+(0) sack=201772608+(0)
win=1500 end=4004184065
<7>[ 2238.120000] tcp_in_window: sender end=4004184065
maxend=4004198665 maxwin=1500 scale=0 receiver end=201772608
maxend=201772608 maxwin=14600 scale=0
<7>[ 2238.120000] tcp_in_window:
<7>[ 2238.120000] seq=4004184065 ack=201772608+(0) sack=201772608+(0)
win=1500 end=4004184065
<7>[ 2238.120000] tcp_in_window: sender end=4004184065
maxend=4004198665 maxwin=1500 scale=0 receiver end=201772608
maxend=201772608 maxwin=14600 scale=0
<7>[ 2238.120000] tcp_in_window: I=1 II=1 III=1 IV=1
<7>[ 2238.120000] tcp_in_window: res=1 sender end=4004184065
maxend=4004198665 maxwin=1500 receiver end=201772608 maxend=201774108
maxwin=14600
<7>[ 2238.120000] tcp_conntracks:
<7>[ 2238.120000] syn=0 ack=1 fin=0 rst=0 old=2 new=3
<7>[ 2238.810000] tcp_in_window: START
<7>[ 2238.810000] tcp_in_window:
<7>[ 2238.810000] seq=4004184065 ack=201772608+(0) sack=201772608+(0)
win=1500 end=4004184095
<7>[ 2238.810000] tcp_in_window: sender end=4004184065
maxend=4004198665 maxwin=1500 scale=0 receiver end=201772608
maxend=201774108 maxwin=14600 scale=0
<7>[ 2238.810000] tcp_in_window:
<7>[ 2238.810000] seq=4004184065 ack=201772608+(0) sack=201772608+(0)
win=1500 end=4004184095
<7>[ 2238.810000] tcp_in_window: sender end=4004184065
maxend=4004198665 maxwin=1500 scale=0 receiver end=201772608
maxend=201774108 maxwin=14600 scale=0
<7>[ 2238.810000] tcp_in_window: I=1 II=1 III=1 IV=1
<7>[ 2238.810000] tcp_in_window: res=1 sender end=4004184095
maxend=4004198665 maxwin=1500 receiver end=201772608 maxend=201774108
maxwin=14600
<7>[ 2238.810000] tcp_conntracks:
<7>[ 2238.810000] syn=0 ack=1 fin=0 rst=0 old=3 new=3
<7>[ 2238.870000] tcp_in_window: START
<7>[ 2238.870000] tcp_in_window:
<7>[ 2238.870000] seq=201772608 ack=4004184095+(0) sack=4004184095+(0)
win=14600 end=201772608
<7>[ 2238.870000] tcp_in_window: sender end=201772608 maxend=201774108
maxwin=14600 scale=0 receiver end=4004184095 maxend=4004198665
maxwin=1500 scale=0
<7>[ 2238.870000] tcp_in_window:
<7>[ 2238.870000] seq=201772608 ack=4004184095+(0) sack=4004184095+(0)
win=14600 end=201772608
<7>[ 2238.870000] tcp_in_window: sender end=201772608 maxend=201774108
maxwin=14600 scale=0 receiver end=4004184095 maxend=4004198665
maxwin=1500 scale=0
<7>[ 2238.870000] tcp_in_window: I=1 II=1 III=1 IV=1
<7>[ 2238.870000] tcp_in_window: res=1 sender end=201772608
maxend=201774108 maxwin=14600 receiver end=4004184095
maxend=4004198695 maxwin=1500
<7>[ 2238.870000] tcp_conntracks:
<7>[ 2238.870000] syn=0 ack=1 fin=0 rst=0 old=3 new=3
<7>[ 2238.870000] tcp_in_window: START
<7>[ 2238.870000] tcp_in_window:
<7>[ 2238.870000] seq=201772608 ack=4004184095+(0) sack=4004184095+(0)
win=14600 end=201772626
<7>[ 2238.870000] tcp_in_window: sender end=201772608 maxend=201774108
maxwin=14600 scale=0 receiver end=4004184095 maxend=4004198695
maxwin=1500 scale=0
<7>[ 2238.870000] tcp_in_window:
<7>[ 2238.870000] seq=201772608 ack=4004184095+(0) sack=4004184095+(0)
win=14600 end=201772626
<7>[ 2238.870000] tcp_in_window: sender end=201772608 maxend=201774108
maxwin=14600 scale=0 receiver end=4004184095 maxend=4004198695
maxwin=1500 scale=0
<7>[ 2238.870000] tcp_in_window: I=1 II=1 III=1 IV=1
<7>[ 2238.870000] tcp_in_window: res=1 sender end=201772626
maxend=201774108 maxwin=14600 receiver end=4004184095
maxend=4004198695 maxwin=1500
<7>[ 2238.870000] tcp_conntracks:
<7>[ 2238.870000] syn=0 ack=1 fin=0 rst=0 old=3 new=3
<7>[ 2238.870000] tcp_conntracks:
<7>[ 2238.870000] syn=0 ack=1 fin=1 rst=0 old=3 new=4
<7>[ 2238.880000] tcp_in_window: START
<7>[ 2238.880000] tcp_in_window:
<7>[ 2238.880000] seq=4004184095 ack=201772626+(0) sack=201772626+(0)
win=1500 end=4004184095
<7>[ 2238.880000] tcp_in_window: sender end=4004184095
maxend=4004198695 maxwin=1500 scale=0 receiver end=201772626
maxend=201774108 maxwin=14600 scale=0
<7>[ 2238.880000] tcp_in_window:
<7>[ 2238.880000] seq=4004184095 ack=201772626+(0) sack=201772626+(0)
win=1500 end=4004184095
<7>[ 2238.880000] tcp_in_window: sender end=4004184095
maxend=4004198695 maxwin=1500 scale=0 receiver end=201772626
maxend=201774108 maxwin=14600 scale=0
<7>[ 2238.880000] tcp_in_window: I=1 II=1 III=1 IV=1
<7>[ 2238.880000] tcp_in_window: res=1 sender end=4004184095
maxend=4004198695 maxwin=1500 receiver end=201772626 maxend=201774126
maxwin=14600
<7>[ 2238.880000] tcp_conntracks:
<7>[ 2238.880000] syn=0 ack=1 fin=0 rst=0 old=4 new=5
<7>[ 2238.880000] tcp_conntracks:
<7>[ 2238.880000] syn=0 ack=1 fin=1 rst=0 old=5 new=6
<7>[ 2238.940000] tcp_in_window: START
<7>[ 2238.940000] tcp_in_window:
<7>[ 2238.940000] seq=201772627 ack=4004184096+(0) sack=4004184096+(0)
win=14600 end=201772627
<7>[ 2238.940000] tcp_in_window: sender end=201772626 maxend=201774126
maxwin=14600 scale=0 receiver end=4004184095 maxend=4004198695
maxwin=1500 scale=0
<7>[ 2238.940000] tcp_in_window:
<7>[ 2238.940000] seq=201772626 ack=4004184096+(0) sack=4004184096+(0)
win=14600 end=201772626
<7>[ 2238.940000] tcp_in_window: sender end=201772626 maxend=201774126
maxwin=14600 scale=0 receiver end=4004184095 maxend=4004198695
maxwin=1500 scale=0
<7>[ 2238.940000] tcp_in_window: I=1 II=1 III=0 IV=1
<7>[ 2238.940000] tcp_in_window: res=0 sender end=201772626
maxend=201774126 maxwin=14600 receiver end=4004184095
maxend=4004198695 maxwin=1500
<7>[ 2238.940000] nf_ct_tcp: invalid new deleting.
<7>[ 2241.930000] tcp_conntracks:
<7>[ 2241.930000] syn=0 ack=1 fin=1 rst=0 old=6 new=6
<7>[ 2244.990000] tcp_conntracks:
<7>[ 2244.990000] syn=0 ack=1 fin=1 rst=0 old=6 new=6
<7>[ 2248.060000] tcp_conntracks:
<7>[ 2248.060000] syn=0 ack=1 fin=1 rst=0 old=6 new=6
<7>[ 2251.120000] tcp_conntracks:
<7>[ 2251.120000] syn=0 ack=1 fin=1 rst=0 old=6 new=6
<7>[ 2254.190000] tcp_conntracks:
<7>[ 2254.190000] syn=0 ack=1 fin=1 rst=0 old=6 new=6
<7>[ 2257.250000] tcp_conntracks:
<7>[ 2257.250000] syn=0 ack=1 fin=1 rst=0 old=6 new=6
<7>[ 2260.320000] tcp_conntracks:
<7>[ 2260.320000] syn=0 ack=1 fin=1 rst=0 old=6 new=6
<7>[ 2263.380000] tcp_conntracks:
<7>[ 2263.380000] syn=0 ack=1 fin=1 rst=0 old=6 new=6
<7>[ 2266.440000] tcp_conntracks:
<7>[ 2266.440000] syn=0 ack=1 fin=1 rst=0 old=6 new=6
<7>[ 2269.510000] tcp_conntracks:
<7>[ 2269.510000] syn=0 ack=1 fin=1 rst=0 old=6 new=6
<7>[ 2272.570000] tcp_conntracks:
<7>[ 2272.570000] syn=0 ack=0 fin=0 rst=1 old=6 new=8

On Tue, Oct 21, 2014 at 1:39 PM, Jozsef Kadlecsik

Post by Jozsef Kadlecsik

It's a single packet, it doesn't tell anything about the TCP stream.
A full TCP stream dump is required from the very first SYN to the very
last packet.
Best regards,
Jozsef
-
PGP key : http://www.kfki.hu/~kadlec/pgp_public_key.txt
Address : Wigner Research Centre for Physics, Hungarian Academy of Sciences
H-1525 Budapest 114, POB. 49, Hungary

--
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Jozsef Kadlecsik

2014-10-21 20:36:53 UTC

Permalink

Post by vDev
OK. Here it is. Please look for III=0 and subsequent res=0.

This is not the dump of the TCP stream. Please send a tcpdump recording!

[...]

Post by vDev
<7>[ 2238.870000] syn=0 ack=1 fin=0 rst=0 old=3 new=3
<7>[ 2238.870000] syn=0 ack=1 fin=1 rst=0 old=3 new=4

It seems there was an ignored packet here. But without a tcpdump, nothing
much can be said about it.

Best regard,
Jozsef
-
E-mail : ***@blackhole.kfki.hu, ***@wigner.mta.hu
PGP key : http://www.kfki.hu/~kadlec/pgp_public_key.txt
Address : Wigner Research Centre for Physics, Hungarian Academy of Sciences
H-1525 Budapest 114, POB. 49, Hungary
--
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Jozsef Kadlecsik

2014-10-21 20:49:50 UTC

Permalink

Post by Jozsef Kadlecsik

Post by vDev
OK. Here it is. Please look for III=0 and subsequent res=0.

This is not the dump of the TCP stream. Please send a tcpdump recording!

And do not send it as an ascii dump but as a pcap file...

Best regards,
Joysef
-
E-mail : ***@blackhole.kfki.hu, ***@wigner.mta.hu
PGP key : http://www.kfki.hu/~kadlec/pgp_public_key.txt
Address : Wigner Research Centre for Physics, Hungarian Academy of Sciences
H-1525 Budapest 114, POB. 49, Hungary
--
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

vDev

2014-10-21 22:14:29 UTC

Permalink

I am unable to post a pcap file at this point but here is an ASCII
output. The last ACK (ID: 6200) was considered invalid due to III
evaluating to 0 and res=0 in the earlier trace.

tcpdump: listening on eth2, link-type EN10MB (Ethernet), capture size
262144 bytes
16:39:34.462344 IP (tos 0x0, ttl 63, id 11, offset 0, flags [none],
proto TCP (6), length 44)
Linux_Router.1026 > REMOTE_SERVER.63001: Flags [S], cksum 0xeaa4
(correct), seq 1278607360, win 1500, options [mss 536], length 0
16:39:34.518272 IP (tos 0x0, ttl 52, id 0, offset 0, flags [DF], proto
TCP (6), length 44)
REMOTE_SERVER.63001 > Linux_Router.1026: Flags [S.], cksum 0xaf17
(correct), seq 913887703, ack 1278607361, win 14600, options [mss
536], length 0
16:39:34.529264 IP (tos 0x0, ttl 63, id 12, offset 0, flags [none],
proto TCP (6), length 40)
Linux_Router.1026 > REMOTE_SERVER.63001: Flags [.], cksum 0xf664
(correct), seq 1, ack 1, win 1500, length 0
16:39:35.199013 IP (tos 0x0, ttl 63, id 13, offset 0, flags [none],
proto TCP (6), length 70)
Linux_Router.1026 > REMOTE_SERVER.63001: Flags [P.], cksum 0x54a7
(correct), seq 1:31, ack 1, win 1500, length 30
16:39:35.255619 IP (tos 0x0, ttl 52, id 6197, offset 0, flags [DF],
proto TCP (6), length 40)
REMOTE_SERVER.63001 > Linux_Router.1026: Flags [.], cksum 0xc31a
(correct), seq 1, ack 31, win 14600, length 0
16:39:35.256164 IP (tos 0x0, ttl 52, id 6198, offset 0, flags [DF],
proto TCP (6), length 58)
REMOTE_SERVER.63001 > Linux_Router.1026: Flags [P.], cksum 0xc6ca
(correct), seq 1:19, ack 31, win 14600, length 18
16:39:35.256521 IP (tos 0x0, ttl 52, id 6199, offset 0, flags [DF],
proto TCP (6), length 40)
REMOTE_SERVER.63001 > Linux_Router.1026: Flags [F.], cksum 0xc307
(correct), seq 19, ack 31, win 14600, length 0
16:39:35.266911 IP (tos 0x0, ttl 63, id 14, offset 0, flags [none],
proto TCP (6), length 40)
Linux_Router.1026 > REMOTE_SERVER.63001: Flags [.], cksum 0xf634
(correct), seq 31, ack 19, win 1500, length 0
16:39:35.273232 IP (tos 0x0, ttl 63, id 15, offset 0, flags [none],
proto TCP (6), length 40)
Linux_Router.1026 > REMOTE_SERVER.63001: Flags [F.], cksum 0xf632
(correct), seq 31, ack 20, win 1500, length 0
16:39:35.328857 IP (tos 0x0, ttl 52, id 6200, offset 0, flags [DF],
proto TCP (6), length 40)
REMOTE_SERVER.63001 > Linux_Router.1026: Flags [.], cksum 0xc306
(correct), seq 20, ack 32, win 14600, length 0
16:39:35.329297 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto
TCP (6), length 40)
Linux_Router.1026 > REMOTE_SERVER.63001: Flags [R], cksum 0x047f
(correct), seq 1278607392, win 0, length 0
16:39:38.293532 IP (tos 0x0, ttl 63, id 16, offset 0, flags [none],
proto TCP (6), length 40)
Linux_Router.1026 > REMOTE_SERVER.63001: Flags [F.], cksum 0xf632
(correct), seq 31, ack 20, win 1500, length 0
16:39:41.333370 IP (tos 0x0, ttl 63, id 17, offset 0, flags [none],
proto TCP (6), length 40)
Linux_Router.1026 > REMOTE_SERVER.63001: Flags [F.], cksum 0xf632
(correct), seq 31, ack 20, win 1500, length 0
16:39:44.373300 IP (tos 0x0, ttl 63, id 18, offset 0, flags [none],
proto TCP (6), length 40)
Linux_Router.1026 > REMOTE_SERVER.63001: Flags [F.], cksum 0xf632
(correct), seq 31, ack 20, win 1500, length 0
16:39:47.413089 IP (tos 0x0, ttl 63, id 19, offset 0, flags [none],
proto TCP (6), length 40)
Linux_Router.1026 > REMOTE_SERVER.63001: Flags [F.], cksum 0xf632
(correct), seq 31, ack 20, win 1500, length 0
16:39:50.452954 IP (tos 0x0, ttl 63, id 20, offset 0, flags [none],
proto TCP (6), length 40)
Linux_Router.1026 > REMOTE_SERVER.63001: Flags [F.], cksum 0xf632
(correct), seq 31, ack 20, win 1500, length 0
16:39:53.492753 IP (tos 0x0, ttl 63, id 21, offset 0, flags [none],
proto TCP (6), length 40)
Linux_Router.1026 > REMOTE_SERVER.63001: Flags [F.], cksum 0xf632
(correct), seq 31, ack 20, win 1500, length 0
16:39:56.532626 IP (tos 0x0, ttl 63, id 22, offset 0, flags [none],
proto TCP (6), length 40)
Linux_Router.1026 > REMOTE_SERVER.63001: Flags [F.], cksum 0xf632
(correct), seq 31, ack 20, win 1500, length 0
16:39:59.572464 IP (tos 0x0, ttl 63, id 23, offset 0, flags [none],
proto TCP (6), length 40)
Linux_Router.1026 > REMOTE_SERVER.63001: Flags [F.], cksum 0xf632
(correct), seq 31, ack 20, win 1500, length 0
16:40:02.612307 IP (tos 0x0, ttl 63, id 24, offset 0, flags [none],
proto TCP (6), length 40)
Linux_Router.1026 > REMOTE_SERVER.63001: Flags [F.], cksum 0xf632
(correct), seq 31, ack 20, win 1500, length 0
16:40:05.652150 IP (tos 0x0, ttl 63, id 25, offset 0, flags [none],
proto TCP (6), length 40)
Linux_Router.1026 > REMOTE_SERVER.63001: Flags [F.], cksum 0xf632
(correct), seq 31, ack 20, win 1500, length 0
16:40:08.691960 IP (tos 0x0, ttl 63, id 26, offset 0, flags [none],
proto TCP (6), length 40)
Linux_Router.1026 > REMOTE_SERVER.63001: Flags [R], cksum 0xcb77
(correct), seq 1278621991, win 0, length 0

On Tue, Oct 21, 2014 at 3:49 PM, Jozsef Kadlecsik

Post by Jozsef Kadlecsik

Post by vDev
OK. Here it is. Please look for III=0 and subsequent res=0.

This is not the dump of the TCP stream. Please send a tcpdump recording!

And do not send it as an ascii dump but as a pcap file...
Best regards,
Joysef
-
PGP key : http://www.kfki.hu/~kadlec/pgp_public_key.txt
Address : Wigner Research Centre for Physics, Hungarian Academy of Sciences
H-1525 Budapest 114, POB. 49, Hungary

--
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Jozsef Kadlecsik

2014-10-22 19:37:53 UTC

Permalink

Post by vDev
I am unable to post a pcap file at this point but here is an ASCII
output. The last ACK (ID: 6200) was considered invalid due to III
evaluating to 0 and res=0 in the earlier trace.
Linux_Router.1026 > REMOTE_SERVER.63001: Flags [F.], cksum 0xf632
(correct), seq 31, ack 20, win 1500, length 0
16:39:35.328857 IP (tos 0x0, ttl 52, id 6200, offset 0, flags [DF],
proto TCP (6), length 40)

However, this is still not enough. The debug lines corresponding to the
captured packets are required too. Moreover, previously there were packets
skipping the window checking in the debug log, without enough debug data

Post by vDev
<7>[ 2238.870000] syn=0 ack=1 fin=0 rst=0 old=3 new=3
<7>[ 2238.870000] syn=0 ack=1 fin=1 rst=0 old=3 new=4

So please apply this small patch, which adds the missing debug info:

diff --git a/net/netfilter/nf_conntrack_proto_tcp.c b/net/netfilter/nf_conntrack_proto_tcp.c
index 44d1ea3..655170f 100644
--- a/net/netfilter/nf_conntrack_proto_tcp.c
+++ b/net/netfilter/nf_conntrack_proto_tcp.c
@@ -830,12 +830,16 @@ static int tcp_packet(struct nf_conn *ct,
th = skb_header_pointer(skb, dataoff, sizeof(_tcph), &_tcph);
BUG_ON(th == NULL);

+ pr_debug("tcp_packet: ");
spin_lock_bh(&ct->lock);
old_state = ct->proto.tcp.state;
dir = CTINFO2DIR(ctinfo);
index = get_conntrack_index(th);
new_state = tcp_conntracks[dir][index][old_state];
tuple = &ct->tuplehash[dir].tuple;
+ pr_debug("dir=%u, seq=%u ack=%u win=%u end=%u\n",
+ dir, ntohl(th->seq), ntohl(th->ack_seq), ntohs(th->window),
+ segment_seq_plus_len(ntohl(th->seq), skb->len, dataoff, th));

switch (new_state) {
case TCP_CONNTRACK_SYN_SENT:

and send me the kernel debug messages and the tcpdump recording of the
packet flow. If you must send the tcpdump in ascii, then print absolute
and not relative TCP sequence numbers.

At the moment the only thing I see about the packet flow is that the first
reply packet answering the first FIN does not ack the FIN flag itself.
However the TCP conntrack state table advances the state to CLOSE_WAIT,
instead of keeping the FIN_WAIT state. But that's still not a problem
in handling the next packets.

Best regards,
Jozsef
-
E-mail : ***@blackhole.kfki.hu, ***@wigner.mta.hu
PGP key : http://www.kfki.hu/~kadlec/pgp_public_key.txt
Address : Wigner Research Centre for Physics, Hungarian Academy of Sciences
H-1525 Budapest 114, POB. 49, Hungary
--
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

vDev

2014-10-22 21:42:09 UTC

Permalink

Thanks, Jozsef. Attached is the new packet capture and trace with patch applied.

On Wed, Oct 22, 2014 at 2:37 PM, Jozsef Kadlecsik

Post by Jozsef Kadlecsik

Post by vDev
<7>[ 2238.870000] syn=0 ack=1 fin=0 rst=0 old=3 new=3
<7>[ 2238.870000] syn=0 ack=1 fin=1 rst=0 old=3 new=4

diff --git a/net/netfilter/nf_conntrack_proto_tcp.c b/net/netfilter/nf_conntrack_proto_tcp.c
index 44d1ea3..655170f 100644
--- a/net/netfilter/nf_conntrack_proto_tcp.c
+++ b/net/netfilter/nf_conntrack_proto_tcp.c
@@ -830,12 +830,16 @@ static int tcp_packet(struct nf_conn *ct,
th = skb_header_pointer(skb, dataoff, sizeof(_tcph), &_tcph);
BUG_ON(th == NULL);
+ pr_debug("tcp_packet: ");
spin_lock_bh(&ct->lock);
old_state = ct->proto.tcp.state;
dir = CTINFO2DIR(ctinfo);
index = get_conntrack_index(th);
new_state = tcp_conntracks[dir][index][old_state];
tuple = &ct->tuplehash[dir].tuple;
+ pr_debug("dir=%u, seq=%u ack=%u win=%u end=%u\n",
+ dir, ntohl(th->seq), ntohl(th->ack_seq), ntohs(th->window),
+ segment_seq_plus_len(ntohl(th->seq), skb->len, dataoff, th));
switch (new_state) {
and send me the kernel debug messages and the tcpdump recording of the
packet flow. If you must send the tcpdump in ascii, then print absolute
and not relative TCP sequence numbers.
At the moment the only thing I see about the packet flow is that the first
reply packet answering the first FIN does not ack the FIN flag itself.
However the TCP conntrack state table advances the state to CLOSE_WAIT,
instead of keeping the FIN_WAIT state. But that's still not a problem
in handling the next packets.
Best regards,
Jozsef
-
PGP key : http://www.kfki.hu/~kadlec/pgp_public_key.txt
Address : Wigner Research Centre for Physics, Hungarian Academy of Sciences
H-1525 Budapest 114, POB. 49, Hungary

vDev

2014-10-22 21:50:38 UTC

Permalink

Post by vDev
Linux_Router.1026 > REMOTE_SERVER.63001: Flags [F.], cksum 0xf632
(correct), seq 31, ack 20, win 1500, length 0
16:39:35.328857 IP (tos 0x0, ttl 52, id 6200, offset 0, flags [DF],
proto TCP (6), length 40)

You looked at the previous packet. The correct packet is:

16:39:35.328857 IP (tos 0x0, ttl 52, id 6200, offset 0, flags [DF],
proto TCP (6), length 40)
REMOTE_SERVER.63001 > Linux_Router.1026: Flags [.], cksum 0xc306
(correct), seq 20, ack 32, win 14600, length 0

Post by vDev
Thanks, Jozsef. Attached is the new packet capture and trace with patch applied.
On Wed, Oct 22, 2014 at 2:37 PM, Jozsef Kadlecsik

Post by Jozsef Kadlecsik

Post by vDev
<7>[ 2238.870000] syn=0 ack=1 fin=0 rst=0 old=3 new=3
<7>[ 2238.870000] syn=0 ack=1 fin=1 rst=0 old=3 new=4

--
To unsubscribe from this list: send the line "unsubscribe netfilter" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html