Skip to content

Add unit tests for conductor SONiC BGP/VLAN/Loopback/VRF helpers#2290

Open
berendt wants to merge 1 commit into
mainfrom
gh2223
Open

Add unit tests for conductor SONiC BGP/VLAN/Loopback/VRF helpers#2290
berendt wants to merge 1 commit into
mainfrom
gh2223

Conversation

@berendt
Copy link
Copy Markdown
Member

@berendt berendt commented May 19, 2026

Closes #2223.

Adds tests/unit/tasks/conductor/sonic/test_config_generator_bgp_vlan_vrf.py covering the BGP, VLAN, Loopback and VRF helpers in osism/tasks/conductor/sonic/config_generator.py.

Coverage

  • _add_bgp_configurationsBGP_NEIGHBOR_AF / BGP_NEIGHBOR for connected interfaces and port channels, plus VLAN-SVI BGP: untagged-member exclusion, direct vs. transfer-role IPv4, switch-to-switch / l2vpn_evpn tag handling, non-default VRF, peer-IP local-addr resolution and peer-IP dedup
  • _get_connected_device_for_interface — delegation
  • _determine_peer_type — AS-mapping vs. computed ASN, missing primary_ip4, and exception handling
  • _add_vlan_configuration — members / VLAN_MEMBER / SVI, unmapped-member warning
  • _add_loopback_configurationLoopback0 BGP_GLOBALS_AF_NETWORK (v4/v6), invalid-IP path
  • _get_vrf_info — both naming conventions, per-interface and top-level error paths
  • _add_vrf_configuration — VNI/table_id/neither, BGP_GLOBALS deep copy, VXLAN sections, interface/port-channel VRF assignment

Notes

  • _add_vlan_configuration does not actually sort members by SONiC name (the issue text says it does); tests assert the real behaviour via an order-independent comparison.
  • Log-only branches are asserted via the shared loguru_logs fixture.

Checks

  • flake8, python-black green locally
  • mypy runs as a separate Zuul job; the file follows the annotation-free SimpleNamespace style of the existing, CI-green SONiC test files
  • pytest left to the python-osism-unit-tests Zuul job

🤖 Generated with Claude Code

Copy link
Copy Markdown

@sourcery-ai sourcery-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 1 issue

Prompt for AI Agents
Please address the comments from this code review:

## Individual Comments

### Comment 1
<location path="tests/unit/tasks/conductor/sonic/test_config_generator_bgp_vlan_vrf.py" line_range="758-772" />
<code_context>
+# ---------------------------------------------------------------------------
+
+
[email protected]
+def vrf_config():
+    return {
+        "VRF": {},
+        "VLAN": {},
+        "VLAN_INTERFACE": {},
+        "BGP_GLOBALS_AF": {},
+        "BGP_GLOBALS_ROUTE_ADVERTISE": {},
+        "BGP_GLOBALS": {},
+        "VXLAN_TUNNEL": {},
+        "VXLAN_EVPN_NVO": {},
+        "VXLAN_TUNNEL_MAP": {},
+        "INTERFACE": {},
+        "PORTCHANNEL_INTERFACE": {},
+    }
+
</code_context>
<issue_to_address>
**suggestion (testing):** Initialize ROUTE_REDISTRIBUTE in vrf_config fixture to make the intent explicit

`TestAddVrfConfiguration.test_vrf_with_vni_full_config` asserts that `"vrfStorage|connected|bgp|ipv4" in vrf_config["ROUTE_REDISTRIBUTE"]`, but the fixture doesn’t define `"ROUTE_REDISTRIBUTE"`. The test currently depends on `_add_vrf_configuration` to create this key. Initializing `"ROUTE_REDISTRIBUTE": {}` in the `vrf_config` fixture would make the test more explicit and its failures less dependent on implementation side effects.

```suggestion
@pytest.fixture
def vrf_config():
    return {
        "VRF": {},
        "VLAN": {},
        "VLAN_INTERFACE": {},
        "BGP_GLOBALS_AF": {},
        "BGP_GLOBALS_ROUTE_ADVERTISE": {},
        "BGP_GLOBALS": {},
        "VXLAN_TUNNEL": {},
        "VXLAN_EVPN_NVO": {},
        "VXLAN_TUNNEL_MAP": {},
        "INTERFACE": {},
        "PORTCHANNEL_INTERFACE": {},
        "ROUTE_REDISTRIBUTE": {},
    }
```
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

@berendt berendt force-pushed the gh2223 branch 2 times, most recently from e4ca3a2 to 8aa1b24 Compare May 19, 2026 14:48
Cover the BGP, VLAN, Loopback and VRF helpers in
osism/tasks/conductor/sonic/config_generator.py:

- _add_bgp_configurations: BGP_NEIGHBOR_AF and BGP_NEIGHBOR for
  connected interfaces and port channels, plus VLAN-SVI BGP
  (untagged-member, transfer-role, switch-to-switch / l2vpn_evpn,
  non-default VRF and dedup branches)
- _get_connected_device_for_interface delegation
- _determine_peer_type AS comparison and error handling
- _add_vlan_configuration members / VLAN_MEMBER / SVI
- _add_loopback_configuration including Loopback0 BGP_GLOBALS_AF_NETWORK
- _get_vrf_info naming conventions and error paths
- _add_vrf_configuration VNI/table_id/VXLAN/interface assignment

Closes #2223

AI-assisted: Claude Code
Signed-off-by: Christian Berendt <[email protected]>
Copy link
Copy Markdown
Contributor

@ideaship ideaship left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review deferred until #2297 is done.

@berendt berendt moved this from In review to In progress in Human Board May 28, 2026
ideaship added a commit that referenced this pull request May 28, 2026
The SONiC config generator owns config_db.json in full: it generates the
file from NetBox data and hardcoded SONiC policy, and operators must not
make manual adjustments to it.  This ownership model was implicit and
undocumented, so each PR touching the generator relitigated which
sections it owns (surfaced in review of #2279 and #2290).

Document the model in the generate_sonic_config() docstring as whole-file
ownership: the generator overwrites every section it manages, and no
sections pass operator edits through unchanged.  Note the two fields the
generator does not itself emit and instead inherits from the device's
image-provided base config (the DEVICE_METADATA localhost device
attributes and the DATABASE schema VERSION); these come from the image,
not from operator hand-edits.

Bring the code into line with the documented model:

  - Rebuild owned tables from scratch instead of overwriting only the
    entries touched in the current run.  generate_sonic_config() deep-
    copies the base config and most helpers assign only the keys they
    currently generate, so an entry removed from NetBox but still present
    in config_db.json (e.g. an old VLAN_MEMBER, BGP_GLOBALS VRF, or
    VXLAN_TUNNEL_MAP) survived regen as stale, active config.  Drop the
    content of every owned table up front (OWNED_TABLE_KEYS: every
    generated table except the inherited DEVICE_METADATA and VERSIONS) so
    only regenerated entries remain.
  - BGP_GLOBALS['default'] was written with key-level updates that merged
    into any pre-existing entry, so operator fields survived regen.
    Replace the entry wholesale when a primary IP is present, like every
    other generated VRF.
  - STATIC_ROUTE merged the management default route into the dict loaded
    from config_db.json, so pre-existing routes (e.g. an operator's
    custom blackhole or VRF route) survived regen.  Clearing the owned
    tables now drops them before the management route is written.

Add tests pinning the model.  Illustrative helper tests
(test_config_generator_ownership.py) show that _add_vrf_configuration and
_add_vlan_configuration replace BGP_GLOBALS[vrf], VLAN entries and
ROUTE_REDISTRIBUTE keys wholesale.  Orchestrator tests
(test_config_generator_orchestrator.py) assert that pre-existing
BGP_GLOBALS['default'] fields, STATIC_ROUTE entries, and stale owned-table
entries are dropped on regen while DEVICE_METADATA and VERSIONS survive.

AI-assisted: Claude Code
Signed-off-by: Roger Luethi <[email protected]>
ideaship added a commit that referenced this pull request May 29, 2026
The SONiC config generator owns config_db.json in full: it generates the
file from NetBox data and hardcoded SONiC policy, and operators must not
make manual adjustments to it.  This ownership model was implicit and
undocumented, so each PR touching the generator relitigated which
sections it owns (surfaced in review of #2279 and #2290).

Document the model in the generate_sonic_config() docstring as whole-file
ownership: the generator overwrites every section it manages, and no
sections pass operator edits through unchanged.  Note the two fields the
generator does not itself emit and instead inherits from the device's
image-provided base config (the DEVICE_METADATA localhost device
attributes and the DATABASE schema VERSION); these come from the image,
not from operator hand-edits.

Bring the code into line with the documented model:

  - Rebuild owned tables from scratch instead of overwriting only the
    entries touched in the current run.  generate_sonic_config() deep-
    copies the base config and most helpers assign only the keys they
    currently generate, so an entry removed from NetBox but still present
    in config_db.json (e.g. an old VLAN_MEMBER, BGP_GLOBALS VRF, or
    VXLAN_TUNNEL_MAP) survived regen as stale, active config.  Drop the
    content of every owned table up front and only regenerated entries
    remain.  OWNED_TABLE_KEYS is composed from two named sub-categories:
    the scaffolded-owned tables (every scaffold key except the inherited
    DEVICE_METADATA and VERSIONS, created up front via setdefault) and
    the on-demand owned tables (ROUTE_REDISTRIBUTE, the SNMP_* tables and
    SYSLOG_SERVER, which helpers assign only when NetBox carries their
    data).
  - BGP_GLOBALS['default'] was written with key-level updates that merged
    into any pre-existing entry, so operator fields survived regen.
    Replace the entry wholesale when a primary IP is present, like every
    other generated VRF.
  - STATIC_ROUTE merged the management default route into the dict loaded
    from config_db.json, so pre-existing routes (e.g. an operator's
    custom blackhole or VRF route) survived regen.  Clearing the owned
    tables now drops them before the management route is written.

Add tests pinning the model.  Illustrative helper tests
(test_config_generator_ownership.py) show that _add_vrf_configuration and
_add_vlan_configuration replace BGP_GLOBALS[vrf], VLAN entries and
ROUTE_REDISTRIBUTE keys wholesale.  Orchestrator tests
(test_config_generator_orchestrator.py) assert that pre-existing
BGP_GLOBALS['default'] fields, STATIC_ROUTE entries, and stale owned-table
entries are dropped on regen while DEVICE_METADATA and VERSIONS survive.

The ownership tests also pin the table-classification taxonomy: that the
scaffold set partitions into scaffolded-owned and inherited tables, that
owned and inherited tables are disjoint, that OWNED_TABLE_KEYS has no
duplicates, and that the scaffolded-owned and on-demand owned sets do not
overlap.  Most of the taxonomy holds by construction (OWNED_TABLE_KEYS is
derived), so these checks exist to fail when the hand-written
INHERITED_TABLE_KEYS or ON_DEMAND_OWNED_TABLE_KEYS literals are edited in
a way that would silently break the model.

AI-assisted: Claude Code
Signed-off-by: Roger Luethi <[email protected]>
ideaship added a commit that referenced this pull request May 29, 2026
The SONiC config generator owns config_db.json in full: it generates the
file from NetBox data and hardcoded SONiC policy, and operators must not
make manual adjustments to it.  This ownership model was implicit and
undocumented, so each PR touching the generator relitigated which
sections it owns (surfaced in review of #2279 and #2290).

Document the model in the generate_sonic_config() docstring as whole-file
ownership: the generator overwrites every section it manages, and no
sections pass operator edits through unchanged.  Note the two fields the
generator does not itself emit and instead inherits from the device's
image-provided base config (the DEVICE_METADATA localhost device
attributes and the DATABASE schema VERSION); these come from the image,
not from operator hand-edits.

Bring the code into line with the documented model:

  - Rebuild owned tables from scratch instead of overwriting only the
    entries touched in the current run.  generate_sonic_config() deep-
    copies the base config and most helpers assign only the keys they
    currently generate, so an entry removed from NetBox but still present
    in config_db.json (e.g. an old VLAN_MEMBER, BGP_GLOBALS VRF, or
    VXLAN_TUNNEL_MAP) survived regen as stale, active config.  Drop the
    content of every owned table up front and only regenerated entries
    remain.  OWNED_TABLE_KEYS is composed from two named sub-categories:
    the scaffolded-owned tables (every scaffold key except the inherited
    DEVICE_METADATA and VERSIONS, created up front via setdefault) and
    the on-demand owned tables (ROUTE_REDISTRIBUTE, the SNMP_* tables and
    SYSLOG_SERVER, which helpers assign only when NetBox carries their
    data).
  - BGP_GLOBALS['default'] was written with key-level updates that merged
    into any pre-existing entry, so operator fields survived regen.
    Replace the entry wholesale when a primary IP is present, like every
    other generated VRF.
  - STATIC_ROUTE merged the management default route into the dict loaded
    from config_db.json, so pre-existing routes (e.g. an operator's
    custom blackhole or VRF route) survived regen.  Clearing the owned
    tables now drops them before the management route is written.

Add tests pinning the model.  Illustrative helper tests
(test_config_generator_ownership.py) show that _add_vrf_configuration and
_add_vlan_configuration replace BGP_GLOBALS[vrf], VLAN entries and
ROUTE_REDISTRIBUTE keys wholesale.  Orchestrator tests
(test_config_generator_orchestrator.py) assert that pre-existing
BGP_GLOBALS['default'] fields, STATIC_ROUTE entries, and stale owned-table
entries are dropped on regen while DEVICE_METADATA and VERSIONS survive.

The ownership tests also pin the table-classification taxonomy: that the
scaffold set partitions into scaffolded-owned and inherited tables, that
owned and inherited tables are disjoint, that OWNED_TABLE_KEYS has no
duplicates, and that the scaffolded-owned and on-demand owned sets do not
overlap.  Most of the taxonomy holds by construction (OWNED_TABLE_KEYS is
derived), so these checks exist to fail when the hand-written
INHERITED_TABLE_KEYS or ON_DEMAND_OWNED_TABLE_KEYS literals are edited in
a way that would silently break the model.

AI-assisted: Claude Code
Signed-off-by: Roger Luethi <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: In progress

Development

Successfully merging this pull request may close these issues.

Unit tests for osism/tasks/conductor/sonic/config_generator.py — BGP/VLAN/Loopback/VRF

3 participants