mirror of
https://github.com/DanielLavrushin/b4.git
synced 2026-10-19 15:33:18 +00:00
feat(tun): enhance local-only traffic handling and first-N packet cap… (#387)
Some checks are pending
Deploy documentation / build-and-deploy (push) Waiting to run
Some checks are pending
Deploy documentation / build-and-deploy (push) Waiting to run
This commit is contained in:
parent
c2f8760122
commit
886adfa267
21 changed files with 787 additions and 37 deletions
|
|
@ -7,6 +7,8 @@
|
|||
- FIXED: **The settings page asked for a restart of b4 after a change to the SOCKS5 proxy, to the web interface's username, password or language, or to the MCP settings, none of which needs one** - it counted every change on the Core tab as needing a restart, and the MCP settings of the Integrations tab as Core ones.
|
||||
- FIXED: **In TUN mode, with packet duplication on a set that covers many addresses, such as a GeoIP category or an ASN, the web interface came up a minute or more after start, with a routing set the bypass then stopped for about as long again, and stopping b4 in that time timed out and left its capture rules behind** - b4 gave each duplication address a firewall rule of its own and added them one command at a time, ahead of the rules that capture the first packets of every connection, and on iptables-legacy every command rewrites the whole table, so the capture chain took longer to build with every address, at start and again right after it, once the routing sets were in place.
|
||||
- FIXED: **One malformed address, such as `10.0.0.0/99` or a name, in a set with packet duplication kept the NFQUEUE packet engine from starting** - b4 handed the set's addresses to ipset or nft unchecked, and one entry the tool rejects fails the whole load.
|
||||
- FIXED: **In TUN mode on a kernel without the iptables raw table or CT target, such as Synology DSM, connections the device forwarded for others, like Docker containers on a bridge network or LAN clients, failed past the local network while the device's own connections worked** - without b4's NOTRACK rule, the copies of their packets that b4 sent on went through connection tracking a second time, and the kernel's NAT gave them a new source port, so the replies came back to a port nothing listened on.
|
||||
- FIXED: **In TUN mode on a kernel without xt_connbytes, a connection that matched a set got the set's DPI bypass on every packet instead of only its first ones, which slowed traffic through it many times over and wrote a log line per packet** - b4 left it to the kernel's connbytes match to hand over only the first packets of a connection, and without that match the TUN engine received and processed whole connections.
|
||||
|
||||
## [1.83.1] - 2026-09-27
|
||||
|
||||
|
|
|
|||
|
|
@ -7,6 +7,8 @@
|
|||
- ИСПРАВЛЕНО: **Страница настроек требовала перезапустить b4 после изменения SOCKS5-прокси, имени пользователя, пароля или языка веб-интерфейса или настроек MCP, хотя ни одному из них перезапуск не нужен** - она считала требующим перезапуска любое изменение на вкладке «Основные», а настройки MCP с вкладки «Интеграции» относила к «Основным».
|
||||
- ИСПРАВЛЕНО: **В режиме TUN при включённом дублировании пакетов у сета, который охватывает много адресов, например категорию GeoIP или ASN, веб-интерфейс появлялся через минуту и больше после запуска, при наличии сета маршрутизации обход затем останавливался ещё примерно на столько же, а остановка b4 в это время завершалась по тайм-ауту и оставляла правила перехвата** - b4 давал каждому адресу дублирования отдельное правило файрвола и добавлял их по одной команде, раньше правил, перехватывающих первые пакеты каждого соединения, а на iptables-legacy каждая команда переписывает всю таблицу, поэтому цепочка перехвата строилась тем дольше, чем больше адресов, при запуске и ещё раз сразу после него, когда появлялись сеты маршрутизации.
|
||||
- ИСПРАВЛЕНО: **Один неверный адрес, например `10.0.0.0/99` или имя, в сете с дублированием пакетов не давал запуститься движку пакетов NFQUEUE** - b4 передавал адреса сета в ipset или nft без проверки, а одна запись, которую инструмент отвергает, срывает загрузку целиком.
|
||||
- ИСПРАВЛЕНО: **В режиме TUN на ядре без таблицы raw или цели CT в iptables, например в Synology DSM, соединения, которые устройство пересылало для других, например для Docker-контейнеров в сети bridge или клиентов LAN, не работали дальше локальной сети, хотя собственные соединения устройства работали** - без правила NOTRACK копии их пакетов, которые b4 отправлял дальше, второй раз проходили через отслеживание соединений, и NAT ядра давал им новый порт источника, поэтому ответы приходили на порт, который никто не слушал.
|
||||
- ИСПРАВЛЕНО: **В режиме TUN на ядре без xt_connbytes соединение, совпавшее с сетом, получало обход DPI этого сета на каждом пакете, а не только на первых, из-за чего трафик через него шёл во много раз медленнее, а в лог попадала строка на каждый пакет** - b4 полагался на правило connbytes ядра, которое передаёт ему только первые пакеты соединения, а без него TUN-движок получал и обрабатывал соединения целиком.
|
||||
|
||||
## [1.83.1] - 2026-09-27
|
||||
|
||||
|
|
|
|||
|
|
@ -219,5 +219,5 @@ Every decision b4 makes about a query is recorded on the [Traffic](./connections
|
|||
| `dns-heal+ipv6-stripped` | Both of the above happened to the same answer: unreachable addresses were replaced and the IPv6 addresses were removed |
|
||||
|
||||
:::note
|
||||
In TUN mode, requests to port 53 are captured, but replies are only seen when b4 carries the whole default route. Redirects and pins work either way, since b4 produces those answers itself.
|
||||
In TUN mode, requests to port 53 are captured, but replies are only seen when b4 carries the whole default route. Redirects and pins work either way, since b4 produces those answers itself. When b4 cannot install its NOTRACK rule, TUN captures only the device's own traffic, so the DNS of other hosts reaches b4 only when a resolver on this device handles it, see [Packet engine](settings/core.md#packet-engine).
|
||||
:::
|
||||
|
|
|
|||
|
|
@ -69,6 +69,10 @@ Turning IPv6 support on or off changes which rules exist in the firewall for eve
|
|||
|
||||
Selecting TUN adds the **TUN settings** group: uplink interface, uplink gateway, TUN address and TUN device name.
|
||||
|
||||
:::info TUN without the raw table or the CT target
|
||||
The TUN engine sends each packet it reads on through a raw socket. For a packet the device forwards for another host, such as a LAN client or a Docker container on a bridge network, that copy has to skip connection tracking, which b4 arranges with a `NOTRACK` rule in the iptables `raw` table; without it the kernel's NAT gives the copy a new source port and the reply never reaches the client. A kernel without the `raw` table (`iptable_raw`) or without the `CT` target (`xt_CT`), such as the Synology DSM kernel, cannot hold that rule, and b4 treats a failed SNAT into the TUN device the same way. There b4 captures only the connections the device makes itself, those of its SOCKS5 and MTProto proxies included. Forwarded traffic stays out of the TUN device, so it gets no bypass strategies and b4 does not see its DNS queries, while routing and blocking sets still apply to it. The capture ports are TCP and UDP 443, the ports in the sets' port filters and TCP 80 when a set uses **Empty line before HTTP method**; where `xt_connbytes` is missing as well, every packet of those connections goes through the TUN device, and b4 counts them itself so that it processes only the first `tcp_conn_bytes_limit` / `udp_conn_bytes_limit` packets (19 and 8 by default). The log says so at startup, and System Info shows it in the **Captured Traffic** row under Engine. With **Skip IPTables/NFTables setup** on, b4 installs neither rule and keeps capturing forwarded traffic.
|
||||
:::
|
||||
|
||||
#### When the engine does not start
|
||||
|
||||
b4 starts the packet engine before the web interface. When the engine fails, b4 removes the rules it had installed and keeps running without it:
|
||||
|
|
@ -103,7 +107,7 @@ With the NFQUEUE engine, setting this to 0 turns off rule monitoring completely.
|
|||
The **Expose to internet** switches of the web server, the SOCKS5 proxy and the listeners on the Telegram tab add accept rules to the host's own input chains, separately from the rules of the packet engine. The monitor interval also sets how often b4 checks them and puts back the ones a firewall reload removed, whatever the packet engine; at `0` only `SIGUSR1` starts that check. **Skip IPTables/NFTables setup** stops b4 from adding them. See [Access from the internet](./security.md#expose-to-internet).
|
||||
:::
|
||||
|
||||
In TUN mode the Firewall group holds only NAT Masquerade and the monitor interval. At that interval the TUN engine checks its capture chain `B4_TUN` and the jumps into it, and the firewall monitor checks the masquerade, MSS clamp and routing-set rules. Each puts back what the router's own firewall removed, for example when the router restarts its firewall after a port-forwarding change. In this mode the interval is at least 10 seconds, and 0 turns neither check off. `SIGUSR1` starts both checks without waiting for the interval. NAT Masquerade leaves the TUN device out: the TUN engine rewrites the source address of captured packets to the uplink address itself, and keeps that rule ahead of any masquerade rule that names no outgoing interface.
|
||||
In TUN mode the Firewall group holds only NAT Masquerade and the monitor interval. At that interval the TUN engine checks its capture chain `B4_TUN` and the jumps into it, and the firewall monitor checks the masquerade, MSS clamp and routing-set rules. Each puts back what the router's own firewall removed, for example when the router restarts its firewall after a port-forwarding change. In this mode the interval is at least 10 seconds, and 0 turns neither check off. `SIGUSR1` starts both checks without waiting for the interval. NAT Masquerade leaves the TUN device out: the TUN engine rewrites the source address of captured packets to the uplink address itself, and keeps that rule ahead of any masquerade rule that names no outgoing interface. When TUN captures only the device's own traffic (see [TUN without the raw table or the CT target](#packet-engine)), that rule is a NAT exemption, `-o <device> -j ACCEPT`, which leaves the source address as it is.
|
||||
|
||||
:::info Rule restores in System Info
|
||||
When TUN captures by port (the Capture row reads `ports`), System Info compares the number of rules in the capture chain with the number b4 installed. After the first restore it also shows how many times the capture rules were restored and, in either engine mode, how many times the firewall monitor restored its rules, each with the time of the last restore. A count that keeps growing points to another service on the router rewriting the firewall.
|
||||
|
|
|
|||
|
|
@ -221,5 +221,5 @@ Fail-closed вместе с недоступным резолвером озна
|
|||
| `dns-heal+ipv6-stripped` | С одним ответом произошло и то, и другое: недоступные адреса заменены, а IPv6-адреса убраны |
|
||||
|
||||
:::note
|
||||
В режиме TUN запросы на порт 53 перехватываются, а ответы видны только тогда, когда b4 держит весь маршрут по умолчанию. Перенаправления и закрепления работают в любом случае, потому что эти ответы b4 формирует сам.
|
||||
В режиме TUN запросы на порт 53 перехватываются, а ответы видны только тогда, когда b4 держит весь маршрут по умолчанию. Перенаправления и закрепления работают в любом случае, потому что эти ответы b4 формирует сам. Если b4 не может поставить правило NOTRACK, TUN захватывает только трафик самого устройства, поэтому DNS других хостов доходит до b4, только когда его обрабатывает резолвер на этом устройстве, см. [Движок пакетов](settings/core.md#движок-пакетов).
|
||||
:::
|
||||
|
|
|
|||
|
|
@ -71,6 +71,10 @@ title: Основные
|
|||
|
||||
При выборе TUN появляется группа **Настройки TUN**: аплинк-интерфейс, шлюз аплинка, адрес TUN и имя устройства TUN.
|
||||
|
||||
:::info TUN без таблицы raw или цели CT
|
||||
Каждый прочитанный пакет TUN-движок отправляет дальше через raw-сокет. Если устройство пересылает пакет для другого хоста, например клиента LAN или Docker-контейнера в сети bridge, эта копия должна миновать отслеживание соединений, и b4 обеспечивает это правилом `NOTRACK` в таблице `raw` iptables; без него NAT ядра даёт копии новый порт источника, и ответ до клиента не доходит. На ядре без таблицы `raw` (`iptable_raw`) или без цели `CT` (`xt_CT`), например на ядре Synology DSM, такое правило поставить нельзя, и так же b4 поступает, если не удалось поставить SNAT для трафика в TUN-устройство. Тогда b4 захватывает только соединения самого устройства, в том числе соединения его SOCKS5- и MTProto-прокси. Пересылаемый трафик в TUN-устройство не попадает: стратегии обхода к нему не применяются и его DNS-запросы b4 не видит, а сеты маршрутизации и блокировки на него действуют. Порты захвата - это TCP и UDP 443, порты из фильтров портов сетов и TCP 80, если в сете включена **Пустая строка перед методом HTTP**; если нет и `xt_connbytes`, через TUN-устройство проходят все пакеты этих соединений, а обрабатывает b4 только первые `tcp_conn_bytes_limit` / `udp_conn_bytes_limit` (по умолчанию 19 и 8), считая их сам. Об этом сообщает лог при запуске, а в окне «Информация о системе» это видно в строке **Захватываемый трафик** раздела «Движок». Если включено **Пропустить настройку IPTables/NFTables**, b4 не ставит ни одно из этих правил и продолжает захватывать пересылаемый трафик.
|
||||
:::
|
||||
|
||||
#### Если движок не запустился
|
||||
|
||||
b4 запускает движок пакетов раньше веб-интерфейса. Если движок не запустился, b4 удаляет уже установленные правила и продолжает работать без него:
|
||||
|
|
@ -105,7 +109,7 @@ b4 повторяет запуск сам, перезапускаясь на м
|
|||
Переключатели **Доступ из интернета** у веб-сервера, SOCKS5-прокси и слушателей на вкладке Telegram добавляют разрешающие правила в собственные цепочки input хоста, отдельно от правил движка пакетов. Интервал мониторинга задаёт и то, как часто b4 проверяет их и возвращает удалённые перезагрузкой файрвола, при любом движке пакетов; при значении `0` эту проверку запускает только `SIGUSR1`. **Пропустить настройку IPTables/NFTables** не даёт b4 их добавлять. См. [Доступ из интернета](./security.md#expose-to-internet).
|
||||
:::
|
||||
|
||||
В режиме TUN в группе «Фаервол» остаются только NAT Masquerade и интервал мониторинга. С этим интервалом движок TUN проверяет свою цепочку захвата `B4_TUN` и переходы в неё, а монитор фаервола - правила маскарада, MSS clamp и сетов маршрутизации. Каждый возвращает то, что удалил собственный фаервол роутера, например когда роутер перезапускает фаервол после изменения проброса портов. В этом режиме интервал не меньше 10 секунд, и значение 0 ни одну из проверок не отключает. `SIGUSR1` запускает обе проверки, не дожидаясь интервала. NAT Masquerade не трогает устройство TUN: адрес источника у захваченных пакетов движок TUN сам заменяет на адрес внешнего интерфейса и держит это правило выше любого правила маскарада без указанного выходного интерфейса.
|
||||
В режиме TUN в группе «Фаервол» остаются только NAT Masquerade и интервал мониторинга. С этим интервалом движок TUN проверяет свою цепочку захвата `B4_TUN` и переходы в неё, а монитор фаервола - правила маскарада, MSS clamp и сетов маршрутизации. Каждый возвращает то, что удалил собственный фаервол роутера, например когда роутер перезапускает фаервол после изменения проброса портов. В этом режиме интервал не меньше 10 секунд, и значение 0 ни одну из проверок не отключает. `SIGUSR1` запускает обе проверки, не дожидаясь интервала. NAT Masquerade не трогает устройство TUN: адрес источника у захваченных пакетов движок TUN сам заменяет на адрес внешнего интерфейса и держит это правило выше любого правила маскарада без указанного выходного интерфейса. Если TUN захватывает только трафик самого устройства (см. [TUN без таблицы raw или цели CT](#движок-пакетов)), вместо этого правила стоит исключение из NAT, `-o <устройство> -j ACCEPT`, и адрес источника не меняется.
|
||||
|
||||
:::info Восстановления правил в «Информации о системе»
|
||||
Когда TUN захватывает трафик по портам (в строке «Захват» стоит `ports`), «Информация о системе» сравнивает число правил в цепочке захвата с числом установленных b4. После первого восстановления там же появляется, сколько раз восстанавливались правила захвата и, в любом режиме движка, сколько раз монитор фаервола восстанавливал свои правила, со временем последнего восстановления. Растущий счётчик указывает на другую службу на роутере, которая переписывает фаервол.
|
||||
|
|
|
|||
|
|
@ -600,6 +600,7 @@ func collectTUNInfo(cfg *config.Config) *DiagTUN {
|
|||
}
|
||||
t.ResolvedSrc = di.ResolvedSrc
|
||||
t.Capture = di.Capture
|
||||
t.LocalOnly = di.LocalOnly
|
||||
t.ReplyCapture = di.ReplyCapture
|
||||
t.PacketsForwarded = di.PacketsForwarded
|
||||
t.ForwardErrors = di.ForwardErrors
|
||||
|
|
|
|||
|
|
@ -129,6 +129,7 @@ type DiagTUN struct {
|
|||
OutGateway string `json:"out_gateway,omitempty"`
|
||||
ResolvedSrc string `json:"resolved_src,omitempty"`
|
||||
Capture string `json:"capture,omitempty"`
|
||||
LocalOnly bool `json:"local_only,omitempty"`
|
||||
RouteTable int `json:"route_table,omitempty"`
|
||||
ReplyCapture bool `json:"reply_capture"`
|
||||
PacketsForwarded uint64 `json:"packets_forwarded"`
|
||||
|
|
|
|||
|
|
@ -422,6 +422,11 @@ export const SystemInfoDialog = ({ open, onClose }: SystemInfoDialogProps) => {
|
|||
t("settings.SystemInfo.tunCapture"),
|
||||
data.engine.tun.capture,
|
||||
)}
|
||||
{data.engine.tun.local_only &&
|
||||
row(
|
||||
t("settings.SystemInfo.tunScope"),
|
||||
boolChip(false, "", t("settings.SystemInfo.tunScopeLocal")),
|
||||
)}
|
||||
{!!data.engine.tun.route_table &&
|
||||
row(
|
||||
t("settings.SystemInfo.tunRouteTable"),
|
||||
|
|
@ -474,6 +479,7 @@ export const SystemInfoDialog = ({ open, onClose }: SystemInfoDialogProps) => {
|
|||
}),
|
||||
)}
|
||||
{data.engine.tun.capture === "ports" &&
|
||||
!data.engine.tun.local_only &&
|
||||
row(
|
||||
t("settings.SystemInfo.tunSteerConflicts"),
|
||||
data.engine.tun.steer_conflicts?.length
|
||||
|
|
|
|||
|
|
@ -856,6 +856,8 @@
|
|||
"tunGateway": "Egress Gateway",
|
||||
"tunSource": "Source IP",
|
||||
"tunCapture": "Capture",
|
||||
"tunScope": "Captured Traffic",
|
||||
"tunScopeLocal": "This device only",
|
||||
"tunRouteTable": "Route Table",
|
||||
"tunForwarded": "Packets Forwarded",
|
||||
"tunForwardErrors": "Forward Errors",
|
||||
|
|
|
|||
|
|
@ -859,6 +859,8 @@
|
|||
"tunGateway": "Исходящий шлюз",
|
||||
"tunSource": "IP источника",
|
||||
"tunCapture": "Захват",
|
||||
"tunScope": "Захватываемый трафик",
|
||||
"tunScopeLocal": "Только это устройство",
|
||||
"tunRouteTable": "Таблица маршрутов",
|
||||
"tunForwarded": "Переслано пакетов",
|
||||
"tunForwardErrors": "Ошибки пересылки",
|
||||
|
|
|
|||
|
|
@ -87,6 +87,7 @@ interface DiagTUN {
|
|||
out_gateway?: string;
|
||||
resolved_src?: string;
|
||||
capture?: string;
|
||||
local_only?: boolean;
|
||||
route_table?: number;
|
||||
reply_capture: boolean;
|
||||
packets_forwarded: number;
|
||||
|
|
|
|||
|
|
@ -42,10 +42,15 @@ func (r *routeManager) steerMarkStr() string {
|
|||
}
|
||||
|
||||
func (r *routeManager) resolveCaptureMode() string {
|
||||
if iptablesMatchSupported([]string{
|
||||
r.noConnbytes = !iptablesMatchSupported([]string{
|
||||
"-p", "tcp", "-m", "connbytes", "--connbytes-dir", "original",
|
||||
"--connbytes-mode", "packets", "--connbytes", "0:10", "-j", "ACCEPT",
|
||||
}) {
|
||||
})
|
||||
if !r.noConnbytes {
|
||||
return "ports"
|
||||
}
|
||||
if r.localOnly {
|
||||
log.Warnf("TUN: xt_connbytes not available; whole connections on the capture ports go through %s instead of their first N packets (install xtables-addons / linux-modules-extra for first-N capture)", r.tunName)
|
||||
return "ports"
|
||||
}
|
||||
log.Warnf("TUN: xt_connbytes not available; capturing the whole default route instead of first-N packets (install xtables-addons / linux-modules-extra for first-N capture)")
|
||||
|
|
@ -71,16 +76,29 @@ func (r *routeManager) setupPortCapture(srcIP string) error {
|
|||
return err
|
||||
}
|
||||
r.clientBypassOK.Store(true)
|
||||
r.conflicts = r.warnOnSteerConflicts()
|
||||
if !r.localOnly {
|
||||
r.conflicts = r.warnOnSteerConflicts()
|
||||
}
|
||||
r.multiport = iptablesMatchSupported([]string{"-p", "tcp", "-m", "multiport", "--dports", "80,443", "-j", "ACCEPT"})
|
||||
r.ensureCaptureChain()
|
||||
r.rebuildCaptureChain()
|
||||
r.ensureCaptureJumps()
|
||||
log.Infof("TUN: port-capture mode - first %d tcp / %d udp packets on ports %s + DNS routed into %s (steer mark %s, ip rule priority %d, table %d; everything b4 re-injects follows this router's own routing)",
|
||||
r.tcpLimit, r.udpLimit, strings.Join(r.tcpPorts, ","), r.tunName, r.steerMarkStr(), r.capturePrio, r.captureTable)
|
||||
log.Infof("TUN: port-capture mode - %s, routed into %s (steer mark %s, ip rule priority %d, table %d; everything b4 re-injects follows this router's own routing)",
|
||||
r.captureSummary(), r.tunName, r.steerMarkStr(), r.capturePrio, r.captureTable)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (r *routeManager) captureSummary() string {
|
||||
what := fmt.Sprintf("first %d tcp / %d udp packets on tcp ports %s, udp ports %s + DNS", r.tcpLimit, r.udpLimit, strings.Join(r.tcpPorts, ","), strings.Join(r.udpPorts, ","))
|
||||
if r.noConnbytes {
|
||||
what = fmt.Sprintf("whole connections on tcp ports %s, udp ports %s + DNS", strings.Join(r.tcpPorts, ","), strings.Join(r.udpPorts, ","))
|
||||
}
|
||||
if r.localOnly {
|
||||
what += ", this device's own traffic only"
|
||||
}
|
||||
return what
|
||||
}
|
||||
|
||||
func (r *routeManager) setupCaptureTable() error {
|
||||
tableStr := strconv.Itoa(r.captureTable)
|
||||
steer := r.steerMarkStr()
|
||||
|
|
@ -162,6 +180,12 @@ func (r *routeManager) ensureCaptureJumps() int {
|
|||
restored++
|
||||
}
|
||||
|
||||
if r.localOnly {
|
||||
r.removeJump("PREROUTING", "-j", tunCaptureChain)
|
||||
r.removeJump("PREROUTING", "-j", tunGateChain)
|
||||
return restored
|
||||
}
|
||||
|
||||
if r.deviceFilterActive() {
|
||||
if r.ensureGateChain() {
|
||||
restored++
|
||||
|
|
@ -254,6 +278,7 @@ type captureRule struct {
|
|||
spec []string
|
||||
local string
|
||||
soft bool
|
||||
steer bool
|
||||
}
|
||||
|
||||
func (r *routeManager) captureChainRules(excl, local []string) []captureRule {
|
||||
|
|
@ -276,16 +301,23 @@ func (r *routeManager) captureChainRules(excl, local []string) []captureRule {
|
|||
locals = append(locals, captureRule{spec: []string{"-d", cidr, "-j", "RETURN"}, local: cidr})
|
||||
}
|
||||
|
||||
if r.reinjectReachesLocal() {
|
||||
switch {
|
||||
case r.localOnly:
|
||||
if r.outGateway != "" {
|
||||
rules = append(rules, captureRule{spec: []string{"-d", r.outGateway, "-p", "udp", "--dport", "53", "-j", "MARK", "--set-xmark", r.steerMarkStr()}})
|
||||
}
|
||||
rules = append(rules, locals...)
|
||||
rules = append(rules, replies...)
|
||||
case r.reinjectReachesLocal():
|
||||
rules = append(rules, replies...)
|
||||
rules = append(rules, locals...)
|
||||
} else {
|
||||
default:
|
||||
rules = append(rules, locals...)
|
||||
rules = append(rules, replies...)
|
||||
}
|
||||
|
||||
for _, spec := range r.steerSpecs() {
|
||||
rules = append(rules, captureRule{spec: spec})
|
||||
rules = append(rules, captureRule{spec: spec, steer: true})
|
||||
}
|
||||
return rules
|
||||
}
|
||||
|
|
@ -310,16 +342,23 @@ func (r *routeManager) rebuildCaptureChain() {
|
|||
|
||||
applied := make([]string, 0, len(local))
|
||||
installed, missing := 0, 0
|
||||
steerWanted, steerInstalled := 0, 0
|
||||
for _, rule := range r.captureChainRules(excl, local) {
|
||||
if r.stopping() {
|
||||
r.captureInstalled = installed
|
||||
log.Infof("TUN: stopping, so the rebuild of %s ends after %d rule(s)", tunCaptureChain, installed)
|
||||
return
|
||||
}
|
||||
if rule.steer {
|
||||
steerWanted++
|
||||
}
|
||||
_, err := run(append([]string{"iptables", "-t", "mangle", "-A", tunCaptureChain}, rule.spec...)...)
|
||||
switch {
|
||||
case err == nil:
|
||||
installed++
|
||||
if rule.steer {
|
||||
steerInstalled++
|
||||
}
|
||||
if rule.local != "" {
|
||||
applied = append(applied, rule.local)
|
||||
}
|
||||
|
|
@ -335,6 +374,9 @@ func (r *routeManager) rebuildCaptureChain() {
|
|||
if missing == 0 {
|
||||
r.captureRetries = 0
|
||||
}
|
||||
if steerWanted > 0 && steerInstalled == 0 {
|
||||
log.Errorf("TUN: none of the %d rules that steer traffic into %s could be installed (see the warnings above; the MARK target may be missing from this kernel), so b4 processes no traffic", steerWanted, r.tunName)
|
||||
}
|
||||
r.captureInstalled = installed
|
||||
r.captureMissing = missing
|
||||
if len(applied) > 0 {
|
||||
|
|
@ -512,6 +554,9 @@ func (r *routeManager) steerSpecs() [][]string {
|
|||
|
||||
var specs [][]string
|
||||
cb := func(portRange string) []string {
|
||||
if r.noConnbytes {
|
||||
return nil
|
||||
}
|
||||
return []string{"-m", "connbytes", "--connbytes-dir", "original", "--connbytes-mode", "packets", "--connbytes", portRange}
|
||||
}
|
||||
|
||||
|
|
@ -587,7 +632,9 @@ func (r *routeManager) ensurePortCapture() {
|
|||
r.captureDirty, r.gateDirty = false, false
|
||||
present := r.ensureCaptureChain()
|
||||
hooks := r.ensureCaptureJumps()
|
||||
r.refreshSteerConflicts()
|
||||
if !r.localOnly {
|
||||
r.refreshSteerConflicts()
|
||||
}
|
||||
desired := r.desiredCaptureExclusions()
|
||||
localNow, localOK := r.desiredLocalNets()
|
||||
if !localOK {
|
||||
|
|
@ -614,7 +661,7 @@ func (r *routeManager) ensurePortCapture() {
|
|||
log.Infof("TUN: retrying the rebuild of %s, whose flush failed at the previous check", tunCaptureChain)
|
||||
r.rebuildCaptureChain()
|
||||
case dirty:
|
||||
log.Infof("TUN: capture settings changed, rebuilding %s (first %d tcp / %d udp packets on tcp ports %s, udp ports %s)", tunCaptureChain, r.tcpLimit, r.udpLimit, strings.Join(r.tcpPorts, ","), strings.Join(r.udpPorts, ","))
|
||||
log.Infof("TUN: capture settings changed, rebuilding %s (%s)", tunCaptureChain, r.captureSummary())
|
||||
r.rebuildCaptureChain()
|
||||
case !equalStringSet(desired, r.captureExcl):
|
||||
log.Infof("TUN: reconcile refreshing capture exclusions (%d routing set(s))", len(desired))
|
||||
|
|
|
|||
|
|
@ -74,17 +74,31 @@ func clearTunSNAT(device string) bool {
|
|||
}
|
||||
cleared := false
|
||||
for _, line := range strings.Split(out, "\n") {
|
||||
line = strings.TrimSpace(line)
|
||||
if !strings.HasPrefix(line, "-A POSTROUTING") {
|
||||
spec, ok := ownTunNatRule(line, device)
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
if ruleFieldValue(line, "-o") != device || ruleFieldValue(line, "-j") != "SNAT" {
|
||||
continue
|
||||
}
|
||||
spec := strings.Fields(strings.TrimPrefix(line, "-A POSTROUTING"))
|
||||
if _, err := run(append([]string{"iptables", "-t", "nat", "-D", "POSTROUTING"}, spec...)...); err == nil {
|
||||
cleared = true
|
||||
}
|
||||
}
|
||||
return cleared
|
||||
}
|
||||
|
||||
func ownTunNatRule(line, device string) ([]string, bool) {
|
||||
line = strings.TrimSpace(line)
|
||||
if !strings.HasPrefix(line, "-A POSTROUTING ") {
|
||||
return nil, false
|
||||
}
|
||||
spec := strings.Fields(strings.TrimPrefix(line, "-A POSTROUTING"))
|
||||
if len(spec) < 4 || spec[0] != "-o" || spec[1] != device || spec[2] != "-j" {
|
||||
return nil, false
|
||||
}
|
||||
switch {
|
||||
case len(spec) == 4 && spec[3] == "ACCEPT":
|
||||
return spec, true
|
||||
case len(spec) == 6 && spec[3] == "SNAT" && spec[4] == "--to-source":
|
||||
return spec, true
|
||||
}
|
||||
return nil, false
|
||||
}
|
||||
|
|
|
|||
|
|
@ -13,6 +13,7 @@ type DiagInfo struct {
|
|||
OutGateway string
|
||||
ResolvedSrc string
|
||||
Capture string
|
||||
LocalOnly bool
|
||||
RouteTable int
|
||||
Mark uint
|
||||
ReplyCapture bool
|
||||
|
|
@ -44,6 +45,7 @@ func (e *Engine) DiagInfo() DiagInfo {
|
|||
di.OutGateway = r.outGateway
|
||||
di.ResolvedSrc = r.srcIP
|
||||
di.Capture = r.resolvedCapture
|
||||
di.LocalOnly = r.localOnly
|
||||
di.RouteTable = r.activeTable()
|
||||
di.Mark = r.mark
|
||||
di.ReplyCapture = r.replyCapture
|
||||
|
|
@ -51,7 +53,7 @@ func (e *Engine) DiagInfo() DiagInfo {
|
|||
for _, c := range r.conflicts {
|
||||
di.SteerConflicts = append(di.SteerConflicts, c.String())
|
||||
}
|
||||
di.CaptureRulesExpected = r.captureInstalled
|
||||
di.CaptureRulesExpected = r.captureInstalled + r.captureMissing
|
||||
di.CaptureRestores = r.captureRestores
|
||||
di.LastCaptureRestore = r.lastCaptureRestore
|
||||
if di.Capture == "ports" {
|
||||
|
|
|
|||
198
src/tun/firstn.go
Normal file
198
src/tun/firstn.go
Normal file
|
|
@ -0,0 +1,198 @@
|
|||
package tun
|
||||
|
||||
import (
|
||||
"encoding/binary"
|
||||
"net"
|
||||
"sync"
|
||||
"sync/atomic"
|
||||
"time"
|
||||
)
|
||||
|
||||
const (
|
||||
firstNMaxFlows = 1 << 16
|
||||
firstNTCPIdle = 30 * time.Minute
|
||||
firstNUDPIdle = 2 * time.Minute
|
||||
firstNClosedIdle = 10 * time.Second
|
||||
firstNSweepGap = time.Second
|
||||
firstNEvictSample = 8
|
||||
)
|
||||
|
||||
type flowKey struct {
|
||||
proto uint8
|
||||
src, dst [4]byte
|
||||
sport, dport uint16
|
||||
}
|
||||
|
||||
type flowCount struct {
|
||||
packets uint32
|
||||
isn uint32
|
||||
synSeen bool
|
||||
closing bool
|
||||
seen time.Time
|
||||
}
|
||||
|
||||
type firstNParams struct {
|
||||
tcpLimit uint32
|
||||
udpLimit uint32
|
||||
dupHosts map[[4]byte]struct{}
|
||||
dupNets []*net.IPNet
|
||||
}
|
||||
|
||||
type firstNLimiter struct {
|
||||
params atomic.Pointer[firstNParams]
|
||||
passed atomic.Uint64
|
||||
|
||||
mu sync.Mutex
|
||||
flows map[flowKey]*flowCount
|
||||
lastSweep time.Time
|
||||
}
|
||||
|
||||
func newFirstNLimiter(p captureParams) *firstNLimiter {
|
||||
l := &firstNLimiter{flows: make(map[flowKey]*flowCount)}
|
||||
l.setParams(p)
|
||||
return l
|
||||
}
|
||||
|
||||
func (l *firstNLimiter) setParams(p captureParams) {
|
||||
np := &firstNParams{
|
||||
tcpLimit: uint32(max(p.tcpLimit, 1)),
|
||||
udpLimit: uint32(max(p.udpLimit, 1)),
|
||||
dupHosts: make(map[[4]byte]struct{}),
|
||||
}
|
||||
for _, s := range p.dupIPs {
|
||||
if _, ipnet, err := net.ParseCIDR(s); err == nil {
|
||||
if v4 := ipnet.IP.To4(); v4 != nil {
|
||||
if ones, _ := ipnet.Mask.Size(); ones == 32 {
|
||||
np.dupHosts[[4]byte(v4)] = struct{}{}
|
||||
} else {
|
||||
np.dupNets = append(np.dupNets, ipnet)
|
||||
}
|
||||
}
|
||||
continue
|
||||
}
|
||||
if ip := net.ParseIP(s).To4(); ip != nil {
|
||||
np.dupHosts[[4]byte(ip)] = struct{}{}
|
||||
}
|
||||
}
|
||||
l.params.Store(np)
|
||||
}
|
||||
|
||||
func (p *firstNParams) duplicated(dst [4]byte) bool {
|
||||
if _, ok := p.dupHosts[dst]; ok {
|
||||
return true
|
||||
}
|
||||
for _, n := range p.dupNets {
|
||||
if n.Contains(net.IP(dst[:])) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func (l *firstNLimiter) admit(raw []byte, now time.Time) bool {
|
||||
if len(raw) < 20 || raw[0]>>4 != 4 {
|
||||
return true
|
||||
}
|
||||
ihl := int(raw[0]&0x0f) * 4
|
||||
proto := raw[9]
|
||||
if ihl < 20 || (proto != 6 && proto != 17) || len(raw) < ihl+4 {
|
||||
return true
|
||||
}
|
||||
if (uint16(raw[6])<<8|uint16(raw[7]))&0x3fff != 0 {
|
||||
return true
|
||||
}
|
||||
k := flowKey{proto: proto, sport: uint16(raw[ihl])<<8 | uint16(raw[ihl+1]), dport: uint16(raw[ihl+2])<<8 | uint16(raw[ihl+3])}
|
||||
copy(k.src[:], raw[12:16])
|
||||
copy(k.dst[:], raw[16:20])
|
||||
|
||||
p := l.params.Load()
|
||||
limit := p.udpLimit
|
||||
if proto == 17 {
|
||||
if k.sport == 53 || k.dport == 53 {
|
||||
return true
|
||||
}
|
||||
} else {
|
||||
if p.duplicated(k.dst) {
|
||||
return true
|
||||
}
|
||||
limit = p.tcpLimit
|
||||
}
|
||||
|
||||
var syn, fin bool
|
||||
var seq uint32
|
||||
if proto == 6 && len(raw) >= ihl+14 {
|
||||
flags := raw[ihl+13]
|
||||
syn = flags&0x02 != 0 && flags&0x10 == 0
|
||||
fin = flags&0x05 != 0
|
||||
seq = binary.BigEndian.Uint32(raw[ihl+4 : ihl+8])
|
||||
}
|
||||
|
||||
l.mu.Lock()
|
||||
defer l.mu.Unlock()
|
||||
f, ok := l.flows[k]
|
||||
if !ok {
|
||||
if len(l.flows) >= firstNMaxFlows {
|
||||
if now.Sub(l.lastSweep) >= firstNSweepGap {
|
||||
l.sweepLocked(now)
|
||||
}
|
||||
if len(l.flows) >= firstNMaxFlows {
|
||||
l.evictStalestLocked()
|
||||
}
|
||||
}
|
||||
f = &flowCount{}
|
||||
l.flows[k] = f
|
||||
}
|
||||
if syn && (!f.synSeen || f.isn != seq) {
|
||||
*f = flowCount{isn: seq, synSeen: true}
|
||||
}
|
||||
f.packets++
|
||||
f.seen = now
|
||||
if fin {
|
||||
f.closing = true
|
||||
}
|
||||
pass := f.packets <= limit
|
||||
if !pass {
|
||||
l.passed.Add(1)
|
||||
}
|
||||
return pass
|
||||
}
|
||||
|
||||
func (l *firstNLimiter) sweep(now time.Time) {
|
||||
l.mu.Lock()
|
||||
l.sweepLocked(now)
|
||||
l.mu.Unlock()
|
||||
}
|
||||
|
||||
func (l *firstNLimiter) evictStalestLocked() {
|
||||
var victim flowKey
|
||||
var oldest time.Time
|
||||
n := 0
|
||||
for k, f := range l.flows {
|
||||
if n == 0 || f.seen.Before(oldest) {
|
||||
victim, oldest = k, f.seen
|
||||
}
|
||||
n++
|
||||
if n == firstNEvictSample {
|
||||
break
|
||||
}
|
||||
}
|
||||
if n > 0 {
|
||||
delete(l.flows, victim)
|
||||
}
|
||||
}
|
||||
|
||||
func (l *firstNLimiter) sweepLocked(now time.Time) {
|
||||
l.lastSweep = now
|
||||
for k, f := range l.flows {
|
||||
idle := firstNUDPIdle
|
||||
switch {
|
||||
case f.closing:
|
||||
idle = firstNClosedIdle
|
||||
case k.proto == 6:
|
||||
idle = firstNTCPIdle
|
||||
}
|
||||
if now.Sub(f.seen) > idle {
|
||||
delete(l.flows, k)
|
||||
}
|
||||
}
|
||||
}
|
||||
228
src/tun/firstn_test.go
Normal file
228
src/tun/firstn_test.go
Normal file
|
|
@ -0,0 +1,228 @@
|
|||
package tun
|
||||
|
||||
import (
|
||||
"net"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func testPacket(proto uint8, src, dst string, sport, dport uint16, tcpFlags byte) []byte {
|
||||
ihl := 20
|
||||
l4 := 8
|
||||
if proto == 6 {
|
||||
l4 = 20
|
||||
}
|
||||
raw := make([]byte, ihl+l4)
|
||||
raw[0] = 0x45
|
||||
raw[9] = proto
|
||||
copy(raw[12:16], net.ParseIP(src).To4())
|
||||
copy(raw[16:20], net.ParseIP(dst).To4())
|
||||
raw[ihl], raw[ihl+1] = byte(sport>>8), byte(sport)
|
||||
raw[ihl+2], raw[ihl+3] = byte(dport>>8), byte(dport)
|
||||
if proto == 6 {
|
||||
raw[ihl+13] = tcpFlags
|
||||
}
|
||||
return raw
|
||||
}
|
||||
|
||||
const (
|
||||
tcpSYN = 0x02
|
||||
tcpACK = 0x10
|
||||
tcpFIN = 0x01
|
||||
)
|
||||
|
||||
func admitted(l *firstNLimiter, raw []byte, n int, now time.Time) int {
|
||||
got := 0
|
||||
for i := 0; i < n; i++ {
|
||||
if l.admit(raw, now) {
|
||||
got++
|
||||
}
|
||||
}
|
||||
return got
|
||||
}
|
||||
|
||||
func TestFirstNLimiterProcessesOnlyTheFirstPacketsOfAFlow(t *testing.T) {
|
||||
l := newFirstNLimiter(captureParams{tcpLimit: 3, udpLimit: 2})
|
||||
now := time.Now()
|
||||
|
||||
tcp := testPacket(6, "192.168.0.10", "203.0.113.10", 40000, 443, tcpACK)
|
||||
if got := admitted(l, tcp, 10, now); got != 3 {
|
||||
t.Fatalf("tcp: %d of 10 packets processed, want 3", got)
|
||||
}
|
||||
other := testPacket(6, "192.168.0.10", "203.0.113.10", 40001, 443, tcpACK)
|
||||
if !l.admit(other, now) {
|
||||
t.Fatal("a second connection must start its own count")
|
||||
}
|
||||
udp := testPacket(17, "192.168.0.10", "203.0.113.10", 50000, 443, 0)
|
||||
if got := admitted(l, udp, 5, now); got != 2 {
|
||||
t.Fatalf("udp: %d of 5 packets processed, want 2", got)
|
||||
}
|
||||
if got := l.passed.Load(); got != 7+3 {
|
||||
t.Fatalf("passed = %d, want 10", got)
|
||||
}
|
||||
}
|
||||
|
||||
func withSeq(raw []byte, seq uint32) []byte {
|
||||
raw[24], raw[25], raw[26], raw[27] = byte(seq>>24), byte(seq>>16), byte(seq>>8), byte(seq)
|
||||
return raw
|
||||
}
|
||||
|
||||
func TestFirstNLimiterCountsRetransmittedSYNs(t *testing.T) {
|
||||
l := newFirstNLimiter(captureParams{tcpLimit: 2, udpLimit: 2})
|
||||
now := time.Now()
|
||||
syn := withSeq(testPacket(6, "192.168.0.10", "203.0.113.10", 40000, 443, tcpSYN), 1000)
|
||||
if got := admitted(l, syn, 4, now); got != 2 {
|
||||
t.Fatalf("retransmitted SYNs belong to the same connection and count against its limit, got %d of 4 processed", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestFirstNLimiterRestartsTheCountForANewConnectionOnTheSameTuple(t *testing.T) {
|
||||
l := newFirstNLimiter(captureParams{tcpLimit: 2, udpLimit: 2})
|
||||
now := time.Now()
|
||||
data := testPacket(6, "192.168.0.10", "203.0.113.10", 40000, 443, tcpACK)
|
||||
first := withSeq(testPacket(6, "192.168.0.10", "203.0.113.10", 40000, 443, tcpSYN), 1000)
|
||||
second := withSeq(testPacket(6, "192.168.0.10", "203.0.113.10", 40000, 443, tcpSYN), 777777)
|
||||
|
||||
l.admit(first, now)
|
||||
admitted(l, data, 5, now)
|
||||
if !l.admit(second, now) {
|
||||
t.Fatal("a SYN with a new initial sequence number opens a new connection and must be processed")
|
||||
}
|
||||
if !l.admit(data, now) || l.admit(data, now) {
|
||||
t.Fatal("the new connection gets its own first packets")
|
||||
}
|
||||
}
|
||||
|
||||
func TestFirstNLimiterKeepsCountingAfterFINAndExpiresClosedFlowsSoon(t *testing.T) {
|
||||
l := newFirstNLimiter(captureParams{tcpLimit: 2, udpLimit: 2})
|
||||
start := time.Now()
|
||||
data := testPacket(6, "192.168.0.10", "203.0.113.10", 40000, 443, tcpACK)
|
||||
fin := testPacket(6, "192.168.0.10", "203.0.113.10", 40000, 443, tcpFIN|tcpACK)
|
||||
live := testPacket(6, "192.168.0.10", "203.0.113.10", 40001, 443, tcpACK)
|
||||
|
||||
admitted(l, data, 2, start)
|
||||
l.admit(live, start)
|
||||
if l.admit(fin, start) || l.admit(data, start) {
|
||||
t.Fatal("packets after a FIN belong to the same connection and must not restart its count")
|
||||
}
|
||||
l.sweep(start.Add(firstNClosedIdle + time.Second))
|
||||
if len(l.flows) != 1 {
|
||||
t.Fatalf("a closed flow must expire after %s while a live one stays, %d left", firstNClosedIdle, len(l.flows))
|
||||
}
|
||||
}
|
||||
|
||||
func TestFirstNLimiterAlwaysProcessesDNSDuplicatesAndWhatItCannotKey(t *testing.T) {
|
||||
l := newFirstNLimiter(captureParams{tcpLimit: 1, udpLimit: 1, dupIPs: []string{"198.51.100.7", "198.51.100.64/26"}})
|
||||
now := time.Now()
|
||||
cases := map[string][]byte{
|
||||
"dns query": testPacket(17, "192.168.0.10", "192.168.0.1", 50000, 53, 0),
|
||||
"dns answer": testPacket(17, "192.168.0.10", "192.168.0.20", 53, 50000, 0),
|
||||
"duplicated host": testPacket(6, "192.168.0.10", "198.51.100.7", 40000, 443, tcpACK),
|
||||
"duplicated range": testPacket(6, "192.168.0.10", "198.51.100.70", 40000, 443, tcpACK),
|
||||
"icmp": testPacket(1, "192.168.0.10", "203.0.113.10", 0, 0, 0),
|
||||
}
|
||||
for name, raw := range cases {
|
||||
if got := admitted(l, raw, 5, now); got != 5 {
|
||||
t.Errorf("%s: %d of 5 packets processed, want all", name, got)
|
||||
}
|
||||
}
|
||||
|
||||
frag := testPacket(6, "192.168.0.10", "203.0.113.10", 40000, 443, tcpACK)
|
||||
frag[6] = 0x20
|
||||
if got := admitted(l, frag, 3, now); got != 3 {
|
||||
t.Errorf("fragments cannot be keyed and must all be processed, got %d of 3", got)
|
||||
}
|
||||
if !l.admit([]byte{0x60, 0, 0, 0}, now) {
|
||||
t.Error("IPv6 is left to the engine")
|
||||
}
|
||||
|
||||
dupUDP := testPacket(17, "192.168.0.10", "198.51.100.7", 50000, 443, 0)
|
||||
if got := admitted(l, dupUDP, 3, now); got != 1 {
|
||||
t.Errorf("duplication covers tcp only, udp to a duplicated host keeps the udp limit: got %d of 3", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestFirstNLimiterSweepsIdleFlowsAndStaysBounded(t *testing.T) {
|
||||
l := newFirstNLimiter(captureParams{tcpLimit: 1, udpLimit: 1})
|
||||
start := time.Now()
|
||||
l.admit(testPacket(17, "192.168.0.10", "203.0.113.10", 50000, 443, 0), start)
|
||||
l.admit(testPacket(6, "192.168.0.10", "203.0.113.10", 40000, 443, tcpACK), start)
|
||||
|
||||
l.sweep(start.Add(firstNUDPIdle + time.Second))
|
||||
if len(l.flows) != 1 {
|
||||
t.Fatalf("an idle udp flow must go and a tcp flow stay, %d left", len(l.flows))
|
||||
}
|
||||
l.sweep(start.Add(firstNTCPIdle + time.Second))
|
||||
if len(l.flows) != 0 {
|
||||
t.Fatalf("an idle tcp flow must go too, %d left", len(l.flows))
|
||||
}
|
||||
|
||||
for i := 0; i < firstNMaxFlows; i++ {
|
||||
l.flows[flowKey{proto: 6, sport: uint16(i), dport: uint16(i >> 16)}] = &flowCount{packets: 9, seen: start}
|
||||
}
|
||||
for i := 0; i < 50; i++ {
|
||||
fresh := testPacket(6, "192.168.0.10", "203.0.113.10", uint16(41000+i), 443, tcpACK)
|
||||
if got := admitted(l, fresh, 3, start); got != 1 {
|
||||
t.Fatalf("flow %d: with the table full of live flows a new flow must still get a counter, got %d of 3 processed", i, got)
|
||||
}
|
||||
}
|
||||
if len(l.flows) != firstNMaxFlows {
|
||||
t.Fatalf("the table must stay at its cap by evicting, has %d entries", len(l.flows))
|
||||
}
|
||||
}
|
||||
|
||||
func TestFirstNLimiterEvictsTheStalestOfItsSample(t *testing.T) {
|
||||
l := newFirstNLimiter(captureParams{tcpLimit: 1, udpLimit: 1})
|
||||
start := time.Now()
|
||||
stale := flowKey{proto: 6, sport: 1}
|
||||
l.flows[stale] = &flowCount{packets: 1, seen: start.Add(-time.Hour)}
|
||||
for i := 2; i <= firstNEvictSample-1; i++ {
|
||||
l.flows[flowKey{proto: 6, sport: uint16(i)}] = &flowCount{packets: 1, seen: start}
|
||||
}
|
||||
l.evictStalestLocked()
|
||||
if _, ok := l.flows[stale]; ok {
|
||||
t.Fatal("with fewer flows than the sample size the stalest one must be the one evicted")
|
||||
}
|
||||
if len(l.flows) != firstNEvictSample-2 {
|
||||
t.Fatalf("exactly one flow must go, %d left", len(l.flows))
|
||||
}
|
||||
}
|
||||
|
||||
func TestLocalOnlyChainExemptsLocalDNSButKeepsTheGateway(t *testing.T) {
|
||||
r := &routeManager{
|
||||
mark: 0x8000,
|
||||
localOnly: true,
|
||||
outGateway: "192.168.0.1",
|
||||
tcpPorts: []string{"443"},
|
||||
udpPorts: []string{"443"},
|
||||
tcpLimit: 19,
|
||||
udpLimit: 8,
|
||||
noConnbytes: true,
|
||||
}
|
||||
rules := r.captureChainRules(nil, []string{"172.17.0.0/16", "192.168.0.0/24"})
|
||||
|
||||
const steer = " -j MARK --set-xmark 0x40000000/0x40000000"
|
||||
is := func(want string) func(string) bool { return func(s string) bool { return s == want } }
|
||||
gateway := captureRuleIndex(rules, is("-d 192.168.0.1 -p udp --dport 53"+steer))
|
||||
bridge := captureRuleIndex(rules, is("-d 172.17.0.0/16 -j RETURN"))
|
||||
lan := captureRuleIndex(rules, is("-d 192.168.0.0/24 -j RETURN"))
|
||||
dns := captureRuleIndex(rules, is("-p udp --dport 53"+steer))
|
||||
forward := captureRuleIndex(rules, is("-p tcp --dport 443"+steer))
|
||||
|
||||
for name, idx := range map[string]int{"gateway dns": gateway, "bridge": bridge, "lan": lan, "dns": dns, "forward": forward} {
|
||||
if idx < 0 {
|
||||
t.Fatalf("%s rule missing from the chain: %v", name, rules)
|
||||
}
|
||||
}
|
||||
if !(gateway < bridge && gateway < lan && bridge < dns && lan < dns && dns < forward) {
|
||||
t.Fatalf("want gateway dns < local exemptions < dns < forward, got %d %d %d %d %d", gateway, bridge, lan, dns, forward)
|
||||
}
|
||||
for _, rule := range rules {
|
||||
s := strings.Join(rule.spec, " ")
|
||||
port := s == "-p tcp --dport 443"+steer || s == "-p udp --dport 443"+steer
|
||||
if rule.steer != port {
|
||||
t.Fatalf("only the port rules count as steer rules: %q steer=%v", s, rule.steer)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -53,6 +53,9 @@ type routeManager struct {
|
|||
snatAdded bool
|
||||
notrackAdded bool
|
||||
clientNotrackAdded bool
|
||||
localOnly bool
|
||||
natGuardAdded bool
|
||||
noConnbytes bool
|
||||
|
||||
captureTable int
|
||||
tcpPorts []string
|
||||
|
|
@ -156,7 +159,7 @@ func (r *routeManager) installNotrack(markStr, label string) bool {
|
|||
return true
|
||||
}
|
||||
if _, err := run(append([]string{"iptables", "-t", "raw", "-A", "OUTPUT"}, notrack...)...); err != nil {
|
||||
log.Infof("TUN: NOTRACK not installed for mark %s (no raw table here); conntrack sysctls + SNAT cover it, so this is harmless: %v", markStr, err)
|
||||
log.Infof("TUN: NOTRACK not installed for %s (mark %s): the kernel has no raw table or no CT target: %v", label, markStr, err)
|
||||
return false
|
||||
}
|
||||
log.Infof("TUN: NOTRACK installed for %s (mark %s)", label, markStr)
|
||||
|
|
@ -185,6 +188,7 @@ func (r *routeManager) removeSNAT() {
|
|||
|
||||
func (r *routeManager) teardownNAT() {
|
||||
r.removeSNAT()
|
||||
r.removeNatGuard()
|
||||
if r.notrackAdded {
|
||||
r.removeNotrack(reinjectMarkMatch())
|
||||
r.notrackAdded = false
|
||||
|
|
@ -302,9 +306,12 @@ func (r *routeManager) setup() error {
|
|||
if r.skipTables {
|
||||
log.Infof("TUN: --skip-tables set; skipping rp_filter/FORWARD/SNAT/NOTRACK - manage NAT and forwarding yourself (b4 still sets up routing: device, capture, bypass table)")
|
||||
} else {
|
||||
r.loosenRPFilter()
|
||||
r.setupForwarding()
|
||||
r.setupNAT()
|
||||
r.decideScope()
|
||||
if !r.localOnly {
|
||||
r.loosenRPFilter()
|
||||
}
|
||||
}
|
||||
|
||||
r.resolvedCapture = r.resolveCaptureMode()
|
||||
|
|
@ -465,11 +472,14 @@ func (r *routeManager) refreshEgress() bool {
|
|||
}
|
||||
log.Infof("TUN: default route changed; re-pointing egress %s(gw %q) -> %s(gw %q)", r.outIface, r.outGateway, iface, gw)
|
||||
|
||||
if !r.skipTables {
|
||||
if !r.skipTables && !r.localOnly {
|
||||
r.restoreRPFilter()
|
||||
}
|
||||
r.removeSNAT()
|
||||
r.outIface = iface
|
||||
if r.localOnly && gw != r.outGateway {
|
||||
r.captureDirty = true
|
||||
}
|
||||
r.outGateway = gw
|
||||
if src != "" {
|
||||
r.srcIP = src
|
||||
|
|
@ -484,7 +494,7 @@ func (r *routeManager) refreshEgress() bool {
|
|||
r.savedDefault += " src " + src
|
||||
}
|
||||
}
|
||||
if !r.skipTables {
|
||||
if !r.skipTables && !r.localOnly {
|
||||
r.savedRPFilter = ""
|
||||
r.loosenRPFilter()
|
||||
}
|
||||
|
|
@ -558,7 +568,9 @@ func (r *routeManager) reconcile() {
|
|||
}
|
||||
|
||||
func (r *routeManager) ensureNAT() {
|
||||
if r.srcIP != "" {
|
||||
if r.localOnly {
|
||||
r.ensureNatGuard()
|
||||
} else if r.srcIP != "" {
|
||||
snat := []string{"-o", r.tunName, "-j", "SNAT", "--to-source", r.srcIP}
|
||||
if _, err := run(append([]string{"iptables", "-t", "nat", "-C", "POSTROUTING"}, snat...)...); err != nil {
|
||||
if _, err := run(append([]string{"iptables", "-t", "nat", "-I", "POSTROUTING", "1"}, snat...)...); err == nil {
|
||||
|
|
@ -567,30 +579,34 @@ func (r *routeManager) ensureNAT() {
|
|||
}
|
||||
} else {
|
||||
r.snatAdded = true
|
||||
r.keepSNATAhead(snat)
|
||||
r.keepNatRuleAhead(snat, "the SNAT")
|
||||
}
|
||||
}
|
||||
r.ensureNotrack(&r.notrackAdded, reinjectMarkMatch())
|
||||
r.ensureNotrack(&r.clientNotrackAdded, clientMarkMatch())
|
||||
}
|
||||
|
||||
func (r *routeManager) keepSNATAhead(snat []string) {
|
||||
func (r *routeManager) keepNatRuleAhead(spec []string, label string) {
|
||||
own := "-o " + r.tunName + " -j " + ruleFieldValue(strings.Join(spec, " "), "-j")
|
||||
out, err := run("iptables", "-t", "nat", "-S", "POSTROUTING")
|
||||
if err != nil || !snatShadowed(out, r.tunName) {
|
||||
if err != nil || !natRuleShadowed(out, own, r.tunName) {
|
||||
return
|
||||
}
|
||||
if _, err := run(append([]string{"iptables", "-t", "nat", "-D", "POSTROUTING"}, snat...)...); err != nil {
|
||||
if _, err := run(append([]string{"iptables", "-t", "nat", "-D", "POSTROUTING"}, spec...)...); err != nil {
|
||||
return
|
||||
}
|
||||
if _, err := run(append([]string{"iptables", "-t", "nat", "-I", "POSTROUTING", "1"}, snat...)...); err != nil {
|
||||
log.Warnf("TUN: failed to move the SNAT for %s to the top of POSTROUTING: %v", r.tunName, err)
|
||||
if _, err := run(append([]string{"iptables", "-t", "nat", "-I", "POSTROUTING", "1"}, spec...)...); err != nil {
|
||||
log.Warnf("TUN: failed to move %s for %s to the top of POSTROUTING: %v", label, r.tunName, err)
|
||||
return
|
||||
}
|
||||
log.Infof("TUN: moved the SNAT for %s above a masquerade rule that would have rewritten captured packets to the %s address", r.tunName, r.tunName)
|
||||
log.Infof("TUN: moved %s for %s above a masquerade rule that would have rewritten captured packets to the %s address", label, r.tunName, r.tunName)
|
||||
}
|
||||
|
||||
func snatShadowed(dump, tunName string) bool {
|
||||
own := "-o " + tunName + " -j SNAT"
|
||||
return natRuleShadowed(dump, "-o "+tunName+" -j SNAT", tunName)
|
||||
}
|
||||
|
||||
func natRuleShadowed(dump, own, tunName string) bool {
|
||||
for _, line := range strings.Split(dump, "\n") {
|
||||
line = strings.TrimSpace(line)
|
||||
if !strings.HasPrefix(line, "-A POSTROUTING ") {
|
||||
|
|
|
|||
83
src/tun/scope.go
Normal file
83
src/tun/scope.go
Normal file
|
|
@ -0,0 +1,83 @@
|
|||
package tun
|
||||
|
||||
import (
|
||||
"strings"
|
||||
|
||||
"github.com/daniellavrushin/b4/log"
|
||||
)
|
||||
|
||||
func missingForwardRules(notrack, clientNotrack, snat bool, tunName string) []string {
|
||||
var missing []string
|
||||
if !notrack || !clientNotrack {
|
||||
missing = append(missing, "the NOTRACK rule for the packets it sends on (the raw table or the CT target is missing)")
|
||||
}
|
||||
if !snat {
|
||||
missing = append(missing, "the SNAT for "+tunName)
|
||||
}
|
||||
return missing
|
||||
}
|
||||
|
||||
func (r *routeManager) decideScope() {
|
||||
missing := missingForwardRules(r.notrackAdded, r.clientNotrackAdded, r.snatAdded || r.srcIP == "", r.tunName)
|
||||
if len(missing) == 0 {
|
||||
r.deleteNatGuardRules()
|
||||
return
|
||||
}
|
||||
r.localOnly = true
|
||||
hadSNAT := r.snatAdded
|
||||
r.removeSNAT()
|
||||
r.installNatGuard()
|
||||
if hadSNAT && r.natGuardAdded {
|
||||
log.Infof("TUN: replaced the SNAT for %s with a NAT exemption, so captured connections keep their source address", r.tunName)
|
||||
}
|
||||
log.Warnf("TUN: b4 could not install %s, so it captures only connections this device makes itself, its SOCKS5 and MTProto proxies included; traffic this device forwards for other hosts, such as LAN clients or Docker containers on bridge networks, stays out of %s, because without those rules the packets b4 sends on for them lose their NAT mapping and the replies never arrive",
|
||||
strings.Join(missing, " and "), r.tunName)
|
||||
}
|
||||
|
||||
func (r *routeManager) natGuardSpec() []string {
|
||||
return []string{"-o", r.tunName, "-j", "ACCEPT"}
|
||||
}
|
||||
|
||||
func (r *routeManager) installNatGuard() {
|
||||
guard := r.natGuardSpec()
|
||||
if _, err := run(append([]string{"iptables", "-t", "nat", "-C", "POSTROUTING"}, guard...)...); err == nil {
|
||||
r.natGuardAdded = true
|
||||
r.keepNatRuleAhead(guard, "the NAT exemption")
|
||||
return
|
||||
}
|
||||
if _, err := run(append([]string{"iptables", "-t", "nat", "-I", "POSTROUTING", "1"}, guard...)...); err != nil {
|
||||
log.Warnf("TUN: failed to exempt traffic into %s from NAT: %v", r.tunName, err)
|
||||
return
|
||||
}
|
||||
r.natGuardAdded = true
|
||||
}
|
||||
|
||||
func (r *routeManager) ensureNatGuard() {
|
||||
guard := r.natGuardSpec()
|
||||
if _, err := run(append([]string{"iptables", "-t", "nat", "-C", "POSTROUTING"}, guard...)...); err != nil {
|
||||
if _, err := run(append([]string{"iptables", "-t", "nat", "-I", "POSTROUTING", "1"}, guard...)...); err == nil {
|
||||
r.natGuardAdded = true
|
||||
log.Infof("TUN: reconcile restored the NAT exemption for %s", r.tunName)
|
||||
}
|
||||
return
|
||||
}
|
||||
r.natGuardAdded = true
|
||||
r.keepNatRuleAhead(guard, "the NAT exemption")
|
||||
}
|
||||
|
||||
func (r *routeManager) deleteNatGuardRules() {
|
||||
guard := r.natGuardSpec()
|
||||
for {
|
||||
if _, err := run(append([]string{"iptables", "-t", "nat", "-D", "POSTROUTING"}, guard...)...); err != nil {
|
||||
return
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (r *routeManager) removeNatGuard() {
|
||||
if !r.natGuardAdded {
|
||||
return
|
||||
}
|
||||
r.deleteNatGuardRules()
|
||||
r.natGuardAdded = false
|
||||
}
|
||||
117
src/tun/scope_test.go
Normal file
117
src/tun/scope_test.go
Normal file
|
|
@ -0,0 +1,117 @@
|
|||
package tun
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestMissingForwardRules(t *testing.T) {
|
||||
if got := missingForwardRules(true, true, true, "b4tun0"); len(got) != 0 {
|
||||
t.Fatalf("with NOTRACK and SNAT in place forwarded traffic is safe to capture, got %v", got)
|
||||
}
|
||||
cases := []struct {
|
||||
name string
|
||||
notrack, clientNotrack, sn bool
|
||||
want string
|
||||
}{
|
||||
{"no raw table", false, false, true, "NOTRACK"},
|
||||
{"client NOTRACK only", true, false, true, "NOTRACK"},
|
||||
{"no SNAT", true, true, false, "the SNAT for b4tun0"},
|
||||
}
|
||||
for _, c := range cases {
|
||||
got := strings.Join(missingForwardRules(c.notrack, c.clientNotrack, c.sn, "b4tun0"), " and ")
|
||||
if !strings.Contains(got, c.want) {
|
||||
t.Errorf("%s: missing = %q, want it to name %q", c.name, got, c.want)
|
||||
}
|
||||
}
|
||||
if got := missingForwardRules(false, false, false, "b4tun0"); len(got) != 2 {
|
||||
t.Fatalf("both failures must be named once each, got %v", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestOwnTunNatRuleMatchesOnlyExactSpecs(t *testing.T) {
|
||||
cases := []struct {
|
||||
line string
|
||||
own bool
|
||||
}{
|
||||
{"-A POSTROUTING -o b4tun0 -j ACCEPT", true},
|
||||
{"-A POSTROUTING -o b4tun0 -j SNAT --to-source 192.168.0.10", true},
|
||||
{"-A POSTROUTING -s 10.1.0.0/16 -o b4tun0 -j ACCEPT", false},
|
||||
{"-A POSTROUTING ! -o b4tun0 -j ACCEPT", false},
|
||||
{"-A POSTROUTING -s 10.3.0.0/16 ! -o b4tun0 -j SNAT --to-source 10.9.0.2", false},
|
||||
{"-A POSTROUTING -o b4tun0 -j MASQUERADE", false},
|
||||
{"-A POSTROUTING -o b4tun1 -j ACCEPT", false},
|
||||
{"-A POSTROUTING -o b4tun0 -j SNAT --to-source 10.9.0.2 --random", false},
|
||||
{"-P POSTROUTING ACCEPT", false},
|
||||
}
|
||||
for _, c := range cases {
|
||||
if _, got := ownTunNatRule(c.line, "b4tun0"); got != c.own {
|
||||
t.Errorf("ownTunNatRule(%q) = %v, want %v", c.line, got, c.own)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestCaptureSummaryNamesTheScope(t *testing.T) {
|
||||
r := &routeManager{tcpPorts: []string{"443"}, udpPorts: []string{"443"}, tcpLimit: 19, udpLimit: 8}
|
||||
if got := r.captureSummary(); got != "first 19 tcp / 8 udp packets on tcp ports 443, udp ports 443 + DNS" {
|
||||
t.Errorf("full scope summary = %q", got)
|
||||
}
|
||||
r.noConnbytes, r.localOnly = true, true
|
||||
if got := r.captureSummary(); got != "whole connections on tcp ports 443, udp ports 443 + DNS, this device's own traffic only" {
|
||||
t.Errorf("local-only summary without connbytes = %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestNatGuardSpec(t *testing.T) {
|
||||
r := &routeManager{tunName: "b4tun0"}
|
||||
if got := strings.Join(r.natGuardSpec(), " "); got != "-o b4tun0 -j ACCEPT" {
|
||||
t.Fatalf("natGuardSpec = %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestNatGuardShadowedLikeTheSNAT(t *testing.T) {
|
||||
own := "-A POSTROUTING -o b4tun0 -j ACCEPT\n"
|
||||
guard := "-o b4tun0 -j ACCEPT"
|
||||
cases := []struct {
|
||||
name string
|
||||
dump string
|
||||
want bool
|
||||
}{
|
||||
{"guard first", own + "-A POSTROUTING -j MASQUERADE\n", false},
|
||||
{"catch-all masquerade first", "-A POSTROUTING -j MASQUERADE\n" + own, true},
|
||||
{"docker style negated interface first", "-A POSTROUTING -s 172.17.0.0/16 ! -o docker0 -j MASQUERADE\n" + own, true},
|
||||
{"masquerade pinned to the uplink first", "-A POSTROUTING -o eth0 -j MASQUERADE\n" + own, false},
|
||||
{"stale SNAT for the tunnel first", "-A POSTROUTING -o b4tun0 -j SNAT --to-source 192.168.0.10\n" + own, true},
|
||||
}
|
||||
for _, c := range cases {
|
||||
if got := natRuleShadowed(c.dump, guard, "b4tun0"); got != c.want {
|
||||
t.Errorf("%s: natRuleShadowed = %v, want %v", c.name, got, c.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestSteerSpecsWithoutConnbytesTakeWholeConnections(t *testing.T) {
|
||||
for _, multiport := range []bool{true, false} {
|
||||
r := &routeManager{
|
||||
multiport: multiport,
|
||||
noConnbytes: true,
|
||||
tcpPorts: []string{"443", "8443"},
|
||||
udpPorts: []string{"443"},
|
||||
tcpLimit: 19,
|
||||
udpLimit: 8,
|
||||
}
|
||||
specs := r.steerSpecs()
|
||||
if len(specs) == 0 {
|
||||
t.Fatalf("multiport=%v: no steer specs", multiport)
|
||||
}
|
||||
for _, s := range specs {
|
||||
j := strings.Join(s, " ")
|
||||
if strings.Contains(j, "connbytes") {
|
||||
t.Errorf("multiport=%v: a kernel without xt_connbytes cannot load %q", multiport, j)
|
||||
}
|
||||
if !strings.HasSuffix(j, "-j MARK --set-xmark 0x40000000/0x40000000") {
|
||||
t.Errorf("multiport=%v: steer spec does not end in MARK: %q", multiport, j)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -26,6 +26,7 @@ type Engine struct {
|
|||
tunFile *os.File
|
||||
tunName string
|
||||
routes *routeManager
|
||||
limiter *firstNLimiter
|
||||
sender *sock.Sender
|
||||
clientSender *sock.Sender
|
||||
trigger chan struct{}
|
||||
|
|
@ -54,7 +55,11 @@ func (e *Engine) config() *config.Config {
|
|||
|
||||
func (e *Engine) UpdateConfig(cfg *config.Config) {
|
||||
e.cfg.Store(cfg)
|
||||
if r := e.routes; r != nil && r.updateCapture(captureParamsFrom(cfg)) {
|
||||
capture := captureParamsFrom(cfg)
|
||||
if l := e.limiter; l != nil {
|
||||
l.setParams(capture)
|
||||
}
|
||||
if r := e.routes; r != nil && r.updateCapture(capture) {
|
||||
e.triggerReconcile()
|
||||
}
|
||||
}
|
||||
|
|
@ -148,10 +153,15 @@ func (e *Engine) Start() error {
|
|||
log.Infof("TUN: reply-direction RST capture enabled (experimental; RST protection / escalation). Validate on a real device")
|
||||
}
|
||||
|
||||
if !cfg.System.Tables.SkipSetup {
|
||||
if !cfg.System.Tables.SkipSetup && !e.routes.localOnly {
|
||||
e.pool.EnableTUNSourceResolver(e.routes.currentSrcIP())
|
||||
}
|
||||
|
||||
if e.routes.noConnbytes {
|
||||
e.limiter = newFirstNLimiter(capture)
|
||||
log.Infof("TUN: without xt_connbytes b4 counts packets itself and processes the first %d tcp / %d udp packets of each connection, as the kernel match would", routes.tcpLimit, routes.udpLimit)
|
||||
}
|
||||
|
||||
threads := cfg.Queue.Threads
|
||||
if threads < 1 {
|
||||
threads = 1
|
||||
|
|
@ -195,11 +205,14 @@ func (e *Engine) reconcileLoop() {
|
|||
e.routes.reconcile()
|
||||
e.pool.UpdateTUNSourceWAN(e.routes.currentSrcIP())
|
||||
}
|
||||
case <-ticker.C:
|
||||
case now := <-ticker.C:
|
||||
if e.routes != nil {
|
||||
e.routes.reconcile()
|
||||
e.pool.UpdateTUNSourceWAN(e.routes.currentSrcIP())
|
||||
}
|
||||
if e.limiter != nil {
|
||||
e.limiter.sweep(now)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -233,6 +246,10 @@ func (e *Engine) readLoop(workerIdx int) {
|
|||
}
|
||||
|
||||
raw := buf[:n]
|
||||
if l := e.limiter; l != nil && !l.admit(raw, time.Now()) {
|
||||
e.forwardPacket(raw)
|
||||
continue
|
||||
}
|
||||
if worker.ProcessPacket(raw) == engine.VerdictAccept {
|
||||
e.forwardPacket(raw)
|
||||
}
|
||||
|
|
@ -357,5 +374,8 @@ func (e *Engine) Stop() {
|
|||
|
||||
log.Infof("TUN: engine stopped (%d packets forwarded, %d forward errors, %d ipv6 dropped)",
|
||||
atomic.LoadUint64(&e.fwdCount), atomic.LoadUint64(&e.fwdErrCount), atomic.LoadUint64(&e.v6DropCount))
|
||||
if e.limiter != nil {
|
||||
log.Infof("TUN: %d packets past the first ones of their connection were forwarded without processing", e.limiter.passed.Load())
|
||||
}
|
||||
})
|
||||
}
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue