You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
DOCS-2997: Write the guide for preparing an existing bridge
Fill in the guide for nodes whose only uplink also carries the node's own IP
address, where Calico cannot create the bridge itself.
The guide is written around configuring the host to bring the bridge up
correctly at boot. Moving a running IP address onto a bridge is the dangerous
way to reach the same end state, so the page does not teach it.
The page states the three properties Calico checks, and what it deliberately
does not require. It covers why the bridge MAC address must be set explicitly
and must not be the trunk's, the VLAN membership the bridge device needs in its
own right, and two hazards that are not settings: a bridge named with the
prefix Calico treats as its own, and the startup cost of spanning tree.
Verification runs in two rounds, because the failures that matter most only
appear when the first workload attaches.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@@ -10,62 +10,329 @@ L2 bridge networking is a tech preview feature. APIs and behavior may change bef
10
10
11
11
:::
12
12
13
-
Use this guide when the uplink that must carry your VLANs is also the one your node's IP address depends on. $[prodname] cannot take that interface over on its own, so you build the bridge and $[prodname] uses it.
13
+
Use this guide when the interface that must carry your VLANs is also the one your node's
14
+
IP address depends on. $[prodname] cannot take that interface over on its own, so you
15
+
build the bridge and $[prodname] uses it. For why, see
16
+
[Who owns the host network configuration](about-l2-bridge.mdx#who-owns-the-host-network-configuration).
14
17
15
-
The goal is not to move a running address onto a bridge. It is to have the host bring the bridge up correctly at boot, so the node's address arrives in the right place the first time.
18
+
If $[prodname] can have an interface of its own, you do not need this guide. Use
19
+
[Connect workloads to an existing VLAN](connect-vlan.mdx) instead.
20
+
21
+
:::note
22
+
23
+
The goal is not to move a running IP address onto a bridge. It is to have the host bring
24
+
the bridge up correctly at boot, so the node's address arrives on the right device the
25
+
first time. Configure this the way you configure the rest of your host networking, and
26
+
verify it survives a reboot.
27
+
28
+
:::
16
29
17
30
## Before you begin
18
31
19
-
This procedure changes how your nodes get their IP addresses. Read this section before you start.
32
+
This procedure changes how your nodes obtain their IP addresses. A mistake takes the node
33
+
off the network, and you will not be able to fix it over the network.
34
+
35
+
You need:
36
+
37
+
- Root access to the node, and ownership of its network configuration.
38
+
- Out-of-band access to the node, such as a console, or a tested way to roll the
39
+
configuration back.
40
+
- The name of the interface that will become the trunk.
41
+
- The VLAN your host's own traffic uses, or confirmation that it is untagged.
42
+
- The VLAN IDs your workloads will use.
43
+
- The node's IP address, prefix length, and default gateway.
44
+
45
+
Work on one non-critical node first, confirm it, and only then apply the same
46
+
configuration to the rest of the cluster.
20
47
21
48
## What you are building
22
49
23
-
Before and after, so the configuration further down makes sense.
50
+
Today, your node's IP address sits directly on a physical interface, and that interface
51
+
carries the node's traffic.
52
+
53
+
Afterwards, that interface is a port on a bridge and carries no address of its own. The
54
+
bridge carries your workload VLANs, and the node's address sits either on the bridge or
55
+
on a VLAN device layered on top of it. $[prodname] attaches workloads to the same bridge
56
+
as access ports.
24
57
25
58
## What $[prodname] requires
26
59
27
-
Three settings on the bridge. $[prodname] checks all three and will not use a bridge that is missing any of them.
60
+
$[prodname] checks three properties of the bridge and refuses to use it if any are
61
+
missing. It does not set them for you, because your host network configuration owns them
62
+
and would undo the change.
63
+
64
+
| Requirement | Setting | Why |
65
+
| ----------- | ------- | --- |
66
+
| VLAN filtering enabled |`vlan_filtering=1`| Without it the bridge ignores VLAN tags entirely, so segments are not isolated from each other. |
67
+
| An explicitly set MAC address | Any address, set deliberately | Stops the bridge's address changing as workloads come and go. See below. |
68
+
| The 802.1Q tag protocol |`vlan_protocol=802.1Q`| $[prodname] programs 802.1Q VLAN membership and creates 802.1Q sub-devices. On an 802.1ad bridge none of it matches. |
69
+
70
+
:::caution
71
+
72
+
Enable VLAN filtering **before** you put the node's address on the bridge, or as part of
73
+
the same configuration change. Turning it on makes the kernel start enforcing VLAN
74
+
membership immediately, and an address on a bridge with no VLAN membership stops
75
+
receiving and sending traffic the moment filtering comes on.
76
+
77
+
:::
28
78
29
79
### Why the MAC address must be pinned, and must not be the trunk's
30
80
31
-
Two separate failures, both avoided by the same setting.
81
+
A Linux bridge has no MAC address of its own. By default the kernel borrows one from
82
+
whichever attached port has the numerically lowest address, and it re-runs that choice
83
+
every time a port is added or removed.
84
+
85
+
On an L2 bridge the ports are workload interfaces, which appear and disappear constantly.
86
+
Left to itself, the bridge's address changes under your node, and if the node's IP address
87
+
is on that bridge, its traffic stops being delivered.
88
+
89
+
Setting an address explicitly stops the kernel from ever reconsidering it.
90
+
91
+
Do not use the trunk interface's own MAC address for the bridge. When two devices in the
92
+
same path present the same address, some network card drivers reject the second
93
+
programming attempt and the bridge is left non-functional. Use a distinct address.
94
+
95
+
A locally administered address is a good choice, because it cannot collide with a real
96
+
card. Set the second least significant bit of the first byte: `02:00:00:00:00:01` and
97
+
`0a:1b:2c:3d:4e:5f` are both locally administered.
32
98
33
99
### What $[prodname] does not require
34
100
35
-
Just as useful as the required list, and it keeps the required list credible.
101
+
Just as useful as the list above, and it keeps that list credible.
102
+
103
+
-**Enslaving the trunk.** $[prodname] adds the trunk interface to the bridge itself. The
104
+
interface only has to exist and be named in the `Network`.
105
+
-**A particular default PVID.** $[prodname] does not read or change the bridge's default
106
+
PVID.
107
+
-**Removing an existing VLAN device.** If you already have a VLAN sub-device on the
108
+
bridge, $[prodname] reuses it rather than failing.
109
+
-**Bringing the bridge up.** $[prodname] brings up a bridge that is administratively
110
+
down.
111
+
-**Self VLAN memberships.** $[prodname] never adds or removes the bridge device's own
112
+
VLAN memberships, or their PVID and untagged flags. They are yours to manage.
36
113
37
114
### Two hazards that are not settings
38
115
39
-
A naming collision that deletes your bridge, and an option that quietly slows workload startup.
116
+
**Do not name the bridge with a `calb-` prefix.** $[prodname] treats that prefix as
117
+
meaning it owns the bridge, and it deletes bridges it owns when they are no longer needed.
118
+
A bridge of your own named that way can be deleted from under you. Nothing rejects the
119
+
name when you create the `Network`, so this is on you to avoid.
120
+
121
+
**Spanning Tree Protocol is allowed, but it costs workload startup time.** Every new
122
+
access port waits out a forwarding delay before it passes traffic, which delays every
123
+
workload attaching to the bridge. $[prodname] warns once and leaves STP alone on a bridge
124
+
it did not create. Turn it off unless you need it.
40
125
41
126
## Configure the host
42
127
43
-
Write this as persistent configuration. A change that does not survive a reboot is a trap.
128
+
Write this as persistent configuration, using whatever manages networking on your nodes.
129
+
A change made with `ip` commands alone disappears at the next reboot, and the node comes
130
+
back with its address on an interface that is now a bridge port.
131
+
132
+
The steps below describe what the configuration has to achieve. The example that follows
133
+
shows one way to express it.
134
+
135
+
1. Create a bridge, with VLAN filtering enabled, the 802.1Q tag protocol, STP disabled,
136
+
and an explicitly set MAC address that is not the trunk's.
137
+
2. Add the trunk interface to the bridge as a port, and give that port membership of
138
+
every VLAN it must carry, including the VLAN your host traffic uses.
139
+
3. Give the bridge device itself membership of the VLAN your host traffic uses. See
140
+
[Self VLAN membership](#self-vlan-membership).
141
+
4. Remove the IP configuration from the physical interface, so it carries no address.
142
+
5. Put the node's IP address, prefix, and default gateway on the bridge or on a VLAN
143
+
device above it, depending on whether your host traffic is tagged. See
144
+
[Where the host's address goes](#where-the-hosts-address-goes).
145
+
6. Apply the configuration and confirm the node is still reachable.
146
+
7. Reboot the node and confirm it comes back with the same addressing.
44
147
45
148
### Where the host's address goes
46
149
47
-
Two shapes, depending on whether your host's own traffic is tagged.
150
+
Two shapes, depending on your host's own traffic.
151
+
152
+
**Untagged host traffic.** Put the node's address directly on the bridge device. The
153
+
bridge port facing the trunk needs the host's traffic to arrive untagged, which means
154
+
setting a native VLAN on the `Network` later so $[prodname] knows which VLAN untagged
155
+
traffic belongs to.
156
+
157
+
**Host traffic on its own VLAN.** Create a VLAN device on top of the bridge for that VLAN
158
+
— for example, VLAN 100 on `br-l2` gives `br-l2.100` — and put the node's address on the
159
+
VLAN device. The bridge itself carries no address.
160
+
161
+
The distinction matters and is easy to invert. A VLAN device **above** the bridge, holding
162
+
the host's address, is the supported arrangement. A VLAN device **below** the bridge,
163
+
standing in for the trunk, is not supported.
48
164
49
165
### Self VLAN membership
50
166
51
-
The step that is easiest to miss, and the one most likely to take a node off the network.
167
+
For the host to receive traffic on a VLAN through the bridge, the bridge device needs
168
+
membership of that VLAN in its own right. This is separate from the membership you give
169
+
the trunk port, and it is the step most likely to be missed.
170
+
171
+
Without it, the bridge forwards that VLAN's traffic between its ports but never delivers
172
+
any of it to the host, so a node whose address depends on that VLAN goes dark.
173
+
174
+
$[prodname] does not add this membership and does not remove one you have added.
175
+
176
+
Some configuration tools have no field for it. Where that is the case, set it with a hook
177
+
that runs when the bridge comes up:
178
+
179
+
```bash
180
+
bridge vlan add dev br-l2 vid 100 self
181
+
```
182
+
183
+
Confirm it with `bridge vlan show`, where the bridge device appears in the list alongside
184
+
its ports.
52
185
53
186
### An annotated example
54
187
55
-
A complete configuration with every line explained.
188
+
This example uses NetworkManager keyfiles, for a node whose management traffic is on VLAN
189
+
100 and whose workloads will use VLANs 10 and 20. The trunk is `eno1`.
0 commit comments