1 |
Hi! |
2 |
Коллеги, кто может раскрыть секрет: что, хотя бы в теории, может убивать |
3 |
пакеты _до_ iptables??? |
4 |
|
5 |
Симптомы: |
6 |
|
7 |
Есть раутер локальной сети, стоящий между юзерами и DMZ. |
8 |
eth1 смотрит в DMZ, на eth0 взведена куча vlan'он, каждый из которых |
9 |
смотрит |
10 |
в нужный кусочек пользовательской сети. На раутере накручены |
11 |
iptables+ipset. |
12 |
Шейпер и ebtables не используются |
13 |
|
14 |
В первом приближении (и с ОЧЕНЬ хорошей точностью) все работает. Но!!! |
15 |
Совершенно случайно обнаружил проблему, возникновение которой объяснить |
16 |
не могу уже неделю :( |
17 |
|
18 |
В DMZ есть машина 172.16.0.11 (этот момент не принципиален, т.к. пробовал с |
19 |
разных машин и даже из разных IP-сетей, как из DMZ, так и |
20 |
пользовательских). С |
21 |
этой машины пускается пинг на 2 адреса в одной из пользовательских сетей. |
22 |
Для определенности, пусть это будут 172.17.18.19 и 172.17.18.40. |
23 |
|
24 |
Получаем такой результат: |
25 |
=============================================================== |
26 |
sabitov@fig ~ $ ping 172.17.18.19 |
27 |
PING 172.17.18.19 (172.17.18.19) 56(84) bytes of data. |
28 |
^C |
29 |
--- 172.17.18.19 ping statistics --- |
30 |
6 packets transmitted, 0 received, 100% packet loss, time 5002ms |
31 |
|
32 |
sabitov@fig ~ $ ping 172.17.18.40 |
33 |
PING 172.17.18.40 (172.17.18.40) 56(84) bytes of data. |
34 |
64 bytes from 172.17.18.40: icmp_seq=1 ttl=63 time=5.59 ms |
35 |
64 bytes from 172.17.18.40: icmp_seq=2 ttl=63 time=4.90 ms |
36 |
^C |
37 |
--- 172.17.18.40 ping statistics --- |
38 |
2 packets transmitted, 2 received, 0% packet loss, time 1001ms |
39 |
rtt min/avg/max/mdev = 4.900/5.247/5.594/0.347 ms |
40 |
=============================================================== |
41 |
|
42 |
Перед пинганьем на раутере запускаем: |
43 |
tcpdump -n -i eth0.0018 net 172.17.18.0/24 and host 172.16.0.11 |
44 |
tcpdump -n -i eth1 net 172.17.18.0/24 and host 172.16.0.11 |
45 |
|
46 |
На eth0.0018 имеем: |
47 |
=============================================================== |
48 |
12:38:00.277614 IP 172.16.0.11 > 172.17.18.19: ICMP echo request, id |
49 |
26647, seq 1, length 64 |
50 |
12:38:00.283405 IP 172.17.18.19 > 172.16.0.11: ICMP echo reply, id |
51 |
26647, seq 1, length 64 |
52 |
12:38:01.280200 IP 172.16.0.11 > 172.17.18.19: ICMP echo request, id |
53 |
26647, seq 2, length 64 |
54 |
12:38:01.285974 IP 172.17.18.19 > 172.16.0.11: ICMP echo reply, id |
55 |
26647, seq 2, length 64 |
56 |
12:38:02.280152 IP 172.16.0.11 > 172.17.18.19: ICMP echo request, id |
57 |
26647, seq 3, length 64 |
58 |
12:38:02.285944 IP 172.17.18.19 > 172.16.0.11: ICMP echo reply, id |
59 |
26647, seq 3, length 64 |
60 |
12:38:03.280164 IP 172.16.0.11 > 172.17.18.19: ICMP echo request, id |
61 |
26647, seq 4, length 64 |
62 |
12:38:03.285961 IP 172.17.18.19 > 172.16.0.11: ICMP echo reply, id |
63 |
26647, seq 4, length 64 |
64 |
12:38:04.280186 IP 172.16.0.11 > 172.17.18.19: ICMP echo request, id |
65 |
26647, seq 5, length 64 |
66 |
12:38:04.286543 IP 172.17.18.19 > 172.16.0.11: ICMP echo reply, id |
67 |
26647, seq 5, length 64 |
68 |
12:38:05.280201 IP 172.16.0.11 > 172.17.18.19: ICMP echo request, id |
69 |
26647, seq 6, length 64 |
70 |
12:38:05.286016 IP 172.17.18.19 > 172.16.0.11: ICMP echo reply, id |
71 |
26647, seq 6, length 64 |
72 |
12:38:13.170502 IP 172.16.0.11 > 172.17.18.40: ICMP echo request, id |
73 |
26648, seq 1, length 64 |
74 |
12:38:13.175813 IP 172.17.18.40 > 172.16.0.11: ICMP echo reply, id |
75 |
26648, seq 1, length 64 |
76 |
12:38:14.172288 IP 172.16.0.11 > 172.17.18.40: ICMP echo request, id |
77 |
26648, seq 2, length 64 |
78 |
12:38:14.176868 IP 172.17.18.40 > 172.16.0.11: ICMP echo reply, id |
79 |
26648, seq 2, length 64 |
80 |
=============================================================== |
81 |
|
82 |
На eth1: |
83 |
=============================================================== |
84 |
12:38:00.277575 IP 172.16.0.11 > 172.17.18.19: ICMP echo request, id |
85 |
26647, seq 1, length 64 |
86 |
12:38:01.280166 IP 172.16.0.11 > 172.17.18.19: ICMP echo request, id |
87 |
26647, seq 2, length 64 |
88 |
12:38:02.280124 IP 172.16.0.11 > 172.17.18.19: ICMP echo request, id |
89 |
26647, seq 3, length 64 |
90 |
12:38:03.280132 IP 172.16.0.11 > 172.17.18.19: ICMP echo request, id |
91 |
26647, seq 4, length 64 |
92 |
12:38:04.280154 IP 172.16.0.11 > 172.17.18.19: ICMP echo request, id |
93 |
26647, seq 5, length 64 |
94 |
12:38:05.280164 IP 172.16.0.11 > 172.17.18.19: ICMP echo request, id |
95 |
26647, seq 6, length 64 |
96 |
12:38:13.170466 IP 172.16.0.11 > 172.17.18.40: ICMP echo request, id |
97 |
26648, seq 1, length 64 |
98 |
12:38:13.175902 IP 172.17.18.40 > 172.16.0.11: ICMP echo reply, id |
99 |
26648, seq 1, length 64 |
100 |
12:38:14.172256 IP 172.16.0.11 > 172.17.18.40: ICMP echo request, id |
101 |
26648, seq 2, length 64 |
102 |
12:38:14.176978 IP 172.17.18.40 > 172.16.0.11: ICMP echo reply, id |
103 |
26648, seq 2, length 64 |
104 |
=============================================================== |
105 |
|
106 |
Видно, что с .40 все хорошо, а ответы от .19 не проходят через раутер. |
107 |
Возникло подозрение, |
108 |
что где-то в iptables я это блокирую, как результат напихал кучу правил |
109 |
для логирования |
110 |
прохождения пакета по цепочкам. |
111 |
|
112 |
Это реальный кусок без купюр: |
113 |
=============================================================== |
114 |
iptables -t raw -A PREROUTING -s 172.17.18.0/24 -j LOG --log-prefix |
115 |
"172.17.18.0 (raw): " |
116 |
iptables -t mangle -A PREROUTING -s 172.17.18.0/24 -j LOG --log-prefix |
117 |
"172.17.18.0 (mangle): " |
118 |
#маркируем прямые обращения к squid-у для блокировки в цепочке INPUT |
119 |
iptables -t mangle -A PREROUTING -p TCP -s 172.16.0.0/15 --dport 3128 -j |
120 |
MARK --set-mark 3128 |
121 |
|
122 |
iptables -t nat -A PREROUTING -s 172.17.18.0/24 -j LOG --log-prefix |
123 |
"172.17.18.0 (nat): " |
124 |
iptables -t nat -A PREROUTING -p TCP \ |
125 |
-s 172.16.0.0/15 \ |
126 |
-m set ! --match-set "ip4_direct_http_clients" src \ |
127 |
-m set ! --match-set "ip4_all_vlans" dst \ |
128 |
-m set --match-set "ip4_proxying_port_list" dst \ |
129 |
-j REDIRECT --to-ports 3128 |
130 |
|
131 |
|
132 |
iptables -A FORWARD -s 172.17.18.0/24 -j LOG --log-prefix "172.17.18.0 |
133 |
(1): " |
134 |
iptables -A FORWARD -p ALL -m set --match-set "ip4_black_list" src |
135 |
-j DROP |
136 |
iptables -A FORWARD -s 172.17.18.0/24 -j LOG --log-prefix "172.17.18.0 |
137 |
(2): " |
138 |
|
139 |
iptables -A FORWARD -p ALL ! -s 172.17.32.0/24 -m set --match-set |
140 |
"ip4_all_vlans" src -m set ! --match-set "mac" src,src -m limit |
141 |
--limit 3/minute --limit-burst 3 -j LOG --log-prefix "Wrong MAC-IP |
142 |
pair (FORWARD): " |
143 |
iptables -A FORWARD -p ALL ! -s 172.17.32.0/24 -m set --match-set |
144 |
"ip4_all_vlans" src -m set ! --match-set "mac" src,src -j DROP |
145 |
|
146 |
iptables -A FORWARD -s 172.17.18.0/24 -j LOG --log-prefix "172.17.18.0 |
147 |
(3): " |
148 |
|
149 |
iptables -A FORWARD -p ALL -m set --match-set "ip4_white_list_2_inet" |
150 |
src -j ACCEPT |
151 |
iptables -A FORWARD -p ALL -m set --match-set "ip4_white_list_2_inet" |
152 |
dst -j ACCEPT |
153 |
|
154 |
iptables -A FORWARD -p ALL -m addrtype --dst-type |
155 |
BROADCAST -j DROP |
156 |
|
157 |
iptables -A FORWARD -s 172.17.18.0/24 -j LOG --log-prefix "172.17.18.0 |
158 |
(4): " |
159 |
|
160 |
iptables -A FORWARD -p ALL -m conntrack --ctstate |
161 |
ESTABLISHED,RELATED -j ACCEPT |
162 |
|
163 |
iptables -A FORWARD -s 172.17.18.0/24 -j LOG --log-prefix "172.17.18.0 |
164 |
(5): " |
165 |
|
166 |
iptables -A FORWARD -p TCP -j forward_tcp_packets |
167 |
iptables -A FORWARD -p UDP -j forward_udp_packets |
168 |
iptables -A FORWARD -p ICMP -j forward_icmp_packets |
169 |
|
170 |
iptables -A FORWARD -s 172.17.18.0/24 -j LOG --log-prefix "172.17.18.0 |
171 |
(6): " |
172 |
|
173 |
iptables -A forward_icmp_packets -s 172.17.18.0/24 -j LOG --log-prefix |
174 |
"172.17.18.0: " |
175 |
iptables -A forward_icmp_packets -p ICMP -j ACCEPT |
176 |
=============================================================== |
177 |
|
178 |
Если верить диаграмме http://inai.de/images/nf-packet-flow.png , то у |
179 |
меня должно в первую |
180 |
очередь отработать логирование входящего пакета в цепочке PREROUTING |
181 |
таблицы raw. |
182 |
Дык вот, для ответов от .40 это так и есть, а для .19 -- тишина! |
183 |
=============================================================== |
184 |
router ~ # grep 172.17.18. /var/log/messages | grep 172.16.0.11 |
185 |
Jun 4 12:38:13 router kernel: 172.17.18.0 (raw): IN=eth0.0018 OUT= |
186 |
MAC=00:13:21:ae:e7:4b:00:1d:b3:91:29:a0:08:00 SRC=172.17.18.40 |
187 |
DST=172.16.0.11 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=100 PROTO=ICMP |
188 |
TYPE=0 CODE=0 ID=26648 SEQ=1 |
189 |
Jun 4 12:38:13 router kernel: 172.17.18.0 (raw): IN=eth0.0018 OUT= |
190 |
MAC=00:13:21:ae:e7:4b:00:1d:b3:91:29:a0:08:00 SRC=172.17.18.40 |
191 |
DST=172.16.0.11 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=100 PROTO=ICMP |
192 |
TYPE=0 CODE=0 ID=26648 SEQ=1 |
193 |
Jun 4 12:38:13 router kernel: 172.17.18.0 (mangle): IN=eth0.0018 OUT= |
194 |
MAC=00:13:21:ae:e7:4b:00:1d:b3:91:29:a0:08:00 SRC=172.17.18.40 |
195 |
DST=172.16.0.11 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=100 PROTO=ICMP |
196 |
TYPE=0 CODE=0 ID=26648 SEQ=1 |
197 |
Jun 4 12:38:13 router kernel: 172.17.18.0 (1): IN=eth0.0018 OUT=eth1 |
198 |
MAC=00:13:21:ae:e7:4b:00:1d:b3:91:29:a0:08:00 SRC=172.17.18.40 |
199 |
DST=172.16.0.11 LEN=84 TOS=0x00 PREC=0x00 TTL=63 ID=100 PROTO=ICMP |
200 |
TYPE=0 CODE=0 ID=26648 SEQ=1 |
201 |
Jun 4 12:38:13 router kernel: 172.17.18.0 (2): IN=eth0.0018 OUT=eth1 |
202 |
MAC=00:13:21:ae:e7:4b:00:1d:b3:91:29:a0:08:00 SRC=172.17.18.40 |
203 |
DST=172.16.0.11 LEN=84 TOS=0x00 PREC=0x00 TTL=63 ID=100 PROTO=ICMP |
204 |
TYPE=0 CODE=0 ID=26648 SEQ=1 |
205 |
Jun 4 12:38:13 router kernel: 172.17.18.0 (3): IN=eth0.0018 OUT=eth1 |
206 |
MAC=00:13:21:ae:e7:4b:00:1d:b3:91:29:a0:08:00 SRC=172.17.18.40 |
207 |
DST=172.16.0.11 LEN=84 TOS=0x00 PREC=0x00 TTL=63 ID=100 PROTO=ICMP |
208 |
TYPE=0 CODE=0 ID=26648 SEQ=1 |
209 |
Jun 4 12:38:13 router kernel: 172.17.18.0 (4): IN=eth0.0018 OUT=eth1 |
210 |
MAC=00:13:21:ae:e7:4b:00:1d:b3:91:29:a0:08:00 SRC=172.17.18.40 |
211 |
DST=172.16.0.11 LEN=84 TOS=0x00 PREC=0x00 TTL=63 ID=100 PROTO=ICMP |
212 |
TYPE=0 CODE=0 ID=26648 SEQ=1 |
213 |
Jun 4 12:38:14 router kernel: 172.17.18.0 (raw): IN=eth0.0018 OUT= |
214 |
MAC=00:13:21:ae:e7:4b:00:1d:b3:91:29:a0:08:00 SRC=172.17.18.40 |
215 |
DST=172.16.0.11 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=101 PROTO=ICMP |
216 |
TYPE=0 CODE=0 ID=26648 SEQ=2 |
217 |
Jun 4 12:38:14 router kernel: 172.17.18.0 (raw): IN=eth0.0018 OUT= |
218 |
MAC=00:13:21:ae:e7:4b:00:1d:b3:91:29:a0:08:00 SRC=172.17.18.40 |
219 |
DST=172.16.0.11 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=101 PROTO=ICMP |
220 |
TYPE=0 CODE=0 ID=26648 SEQ=2 |
221 |
Jun 4 12:38:14 router kernel: 172.17.18.0 (mangle): IN=eth0.0018 OUT= |
222 |
MAC=00:13:21:ae:e7:4b:00:1d:b3:91:29:a0:08:00 SRC=172.17.18.40 |
223 |
DST=172.16.0.11 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=101 PROTO=ICMP |
224 |
TYPE=0 CODE=0 ID=26648 SEQ=2 |
225 |
Jun 4 12:38:14 router kernel: 172.17.18.0 (1): IN=eth0.0018 OUT=eth1 |
226 |
MAC=00:13:21:ae:e7:4b:00:1d:b3:91:29:a0:08:00 SRC=172.17.18.40 |
227 |
DST=172.16.0.11 LEN=84 TOS=0x00 PREC=0x00 TTL=63 ID=101 PROTO=ICMP |
228 |
TYPE=0 CODE=0 ID=26648 SEQ=2 |
229 |
Jun 4 12:38:14 router kernel: 172.17.18.0 (2): IN=eth0.0018 OUT=eth1 |
230 |
MAC=00:13:21:ae:e7:4b:00:1d:b3:91:29:a0:08:00 SRC=172.17.18.40 |
231 |
DST=172.16.0.11 LEN=84 TOS=0x00 PREC=0x00 TTL=63 ID=101 PROTO=ICMP |
232 |
TYPE=0 CODE=0 ID=26648 SEQ=2 |
233 |
Jun 4 12:38:14 router kernel: 172.17.18.0 (3): IN=eth0.0018 OUT=eth1 |
234 |
MAC=00:13:21:ae:e7:4b:00:1d:b3:91:29:a0:08:00 SRC=172.17.18.40 |
235 |
DST=172.16.0.11 LEN=84 TOS=0x00 PREC=0x00 TTL=63 ID=101 PROTO=ICMP |
236 |
TYPE=0 CODE=0 ID=26648 SEQ=2 |
237 |
Jun 4 12:38:14 router kernel: 172.17.18.0 (4): IN=eth0.0018 OUT=eth1 |
238 |
MAC=00:13:21:ae:e7:4b:00:1d:b3:91:29:a0:08:00 SRC=172.17.18.40 |
239 |
DST=172.16.0.11 LEN=84 TOS=0x00 PREC=0x00 TTL=63 ID=101 PROTO=ICMP |
240 |
TYPE=0 CODE=0 ID=26648 SEQ=2 |
241 |
=============================================================== |
242 |
|
243 |
Эта проблема касается не только пингов, а трафика "вообще". Например, |
244 |
через раз срабатывает обход устройств по SNMP. |