Skip to content

e2e-tests (3/4): Add gNMI integration tests with envtest#409

Open
nikatza wants to merge 3 commits into
mainfrom
refactor-tests-3
Open

e2e-tests (3/4): Add gNMI integration tests with envtest#409
nikatza wants to merge 3 commits into
mainfrom
refactor-tests-3

Conversation

@nikatza

@nikatza nikatza commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

This PR introduces envtest-based gNMI integration tests for the OpenConfig provider, enabling faster validation of the Kubernetes resources to gNMI sequence without requiring a full cluster deployment.

The test architecture uses a per-process gNMI server design that enables parallel test execution with Ginkgo's --procs flag. Each process creates its own suite-level server in BeforeSuite and shares it across tests running serially within that process.

@nikatza
nikatza force-pushed the refactor-tests-3 branch from 78654a7 to d200e98 Compare June 16, 2026 22:11
@nikatza nikatza changed the title e2e-tests (3/4): Add test resources for NXOS e2e-tests (3/4): Add envtest mode for faster in-process e2e testing Jun 17, 2026
@nikatza
nikatza force-pushed the refactor-tests-2 branch from 82d714b to 829e761 Compare June 17, 2026 23:40
@nikatza
nikatza force-pushed the refactor-tests-3 branch from d200e98 to 823e208 Compare June 17, 2026 23:40
@github-actions github-actions Bot added size/XL and removed size/XXL labels Jun 17, 2026
@nikatza
nikatza force-pushed the refactor-tests-3 branch from 823e208 to f735c68 Compare June 17, 2026 23:45
@nikatza
nikatza force-pushed the refactor-tests-2 branch 2 times, most recently from be16d2f to e20ae93 Compare June 18, 2026 07:59
@nikatza
nikatza force-pushed the refactor-tests-3 branch 2 times, most recently from 51d400f to ace1a56 Compare June 18, 2026 08:15
@nikatza
nikatza force-pushed the refactor-tests-2 branch from 8b82070 to e0403cc Compare June 18, 2026 08:54
@nikatza
nikatza force-pushed the refactor-tests-3 branch from ace1a56 to bce5bb0 Compare June 18, 2026 08:54
@nikatza
nikatza changed the base branch from refactor-tests-2 to refactor-tests-1 July 14, 2026 12:20
@nikatza
nikatza force-pushed the refactor-tests-3 branch from bce5bb0 to 2cf82d6 Compare July 14, 2026 17:16
@github-actions github-actions Bot added size/XXL and removed size/XL labels Jul 14, 2026
@nikatza nikatza changed the title e2e-tests (3/4): Add envtest mode for faster in-process e2e testing e2e-tests (3/4): Add gNMI integration tests with envtest Jul 14, 2026
@nikatza
nikatza force-pushed the refactor-tests-3 branch 2 times, most recently from dd3f643 to 5b100a6 Compare July 15, 2026 08:53
@nikatza
nikatza force-pushed the refactor-tests-1 branch 5 times, most recently from f81d9b2 to 47fda5a Compare July 15, 2026 14:22
Base automatically changed from refactor-tests-1 to main July 15, 2026 15:34
@nikatza
nikatza force-pushed the refactor-tests-3 branch 3 times, most recently from 268ae9b to 625b160 Compare July 16, 2026 13:23
@nikatza
nikatza force-pushed the refactor-tests-3 branch 5 times, most recently from e22db40 to 6a91fa7 Compare July 16, 2026 17:09
@github-actions github-actions Bot added size/XL and removed size/XXL labels Jul 16, 2026
nikatza added 3 commits July 16, 2026 19:13
This commit introduces envtest-based integration tests for gNMI functionality
using the OpenConfig provider. The tests verify the end-to-end workflow from
Kubernetes resources to provider gNMI payloads without requiring a full cluster
deployment.

The test architecture uses a per-process gNMI test server to enable parallel
test execution with Ginkgo's --procs flag. Each Ginkgo process creates its own
suite-level server in BeforeSuite, shares it across all tests within that
process, and cleans it up in AfterSuite. Tests within each process run serially,
ensuring proper isolation through per-test namespaces and state clearing between
tests.

Signed-off-by: Pujol <enric.pujol@sap.com>
…lacks VLAN support)

The OpenConfig provider does not yet support VLAN interface configuration,
causing the interface_routed_vlan test to fail. Skip this test by renaming
the fixture file to .skip extension until VLAN support is implemented.

Signed-off-by: Pujol <enric.pujol@sap.com>
Add a dedicated GitHub Actions workflow for gNMI integration tests using the
OpenConfig provider. The workflow runs in parallel with other test jobs,
executing tests with 4 processes via Ginkgo's --procs flag.

Signed-off-by: Pujol <enric.pujol@sap.com>
@github-actions

Copy link
Copy Markdown

Merging this branch will not change overall coverage

Impacted Packages Coverage Δ 🤖
github.com/ironcore-dev/network-operator/test/gnmi 0.00% (ø)
github.com/ironcore-dev/network-operator/test/gnmi/server 0.00% (ø)

Coverage by file

Changed files (no unit tests)

Changed File Coverage Δ Total Covered Missed 🤖
github.com/ironcore-dev/network-operator/test/gnmi/server/server.go 0.00% (ø) 0 0 0

Please note that the "Total", "Covered", and "Missed" counts above refer to code statements instead of lines of code. The value in brackets refers to the test coverage of that file in the old version of the code.

Changed unit test files

  • github.com/ironcore-dev/network-operator/test/gnmi/gnmi_suite_test.go
  • github.com/ironcore-dev/network-operator/test/gnmi/gnmi_test.go
  • github.com/ironcore-dev/network-operator/test/gnmi/json_test.go

@nikatza
nikatza marked this pull request as ready for review July 16, 2026 18:32
Comment on lines +10 to +12
paths-ignore:
- 'docs/**'
- '**/*.md'

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Proposal for a bit more aggressive filtering. This should only trigger e2e on code or dependency changes.

Suggested change
paths-ignore:
- 'docs/**'
- '**/*.md'
paths:
- '**.go'
- 'hack/**'
- 'config/**'
- 'go.mod'
- 'go.sum'
- 'Makefile'

Comment thread test/gnmi/gnmi_test.go
Comment on lines +455 to +460
if envProvider == "" {
It("requires PROVIDER to be set", func() {
Fail(ProviderEnvVar + " not set. Please set " + ProviderEnvVar + " to one of: " + strings.Join(supportedProviders, ", "))
})
return
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can be rewritten using only gomega matchers:

Suggested change
if envProvider == "" {
It("requires PROVIDER to be set", func() {
Fail(ProviderEnvVar + " not set. Please set " + ProviderEnvVar + " to one of: " + strings.Join(supportedProviders, ", "))
})
return
}
Expect(envProvider).NotTo(BeEmpty(), "PROVIDER environment variable must be set to one of: %s", strings.Join(supportedProviders, ", "))

Same for some other if checks in this file

Comment thread test/gnmi/json_test.go
// For comparison, it unmarshals both into interface{} and uses reflect.DeepEqual
// after sorting any arrays and removing empty arrays/objects to ignore ordering
// and cleanup artifacts.
func CompareJSON(got, want string) error {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you tried the gomega build-in matcher for JSON? https://onsi.github.io/gomega/#matchjsonjson-any

# SPDX-FileCopyrightText: 2026 SAP SE or an SAP affiliate company and IronCore contributors
# SPDX-License-Identifier: Apache-2.0

name: Test gNMI integration

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
name: Test gNMI integration
name: Test gNMI integration

why does this file have a two space indentation by default?

Comment on lines +26 to +27
- name: Install ginkgo
run: go install github.com/onsi/ginkgo/v2/ginkgo@latest

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this install step required if the 'make test-gnmi' already takes care of installing ginkgo if it's not present?


jobs:
test-gnmi:
name: Run gNMI Integration Tests (OpenConfig)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
name: Run gNMI Integration Tests (OpenConfig)
name: gNMI Integration (${{ matrix.provider }})
strategy:
fail-fast: false
matrix:
provider:
- openconfig

Should we already integrate a matrix build, such that it is easier to integrate more providers later?

- name: Install ginkgo
run: go install github.com/onsi/ginkgo/v2/ginkgo@latest
- name: Running gNMI tests
run: make test-gnmi TESTGNMI_PROVIDER=openconfig GINKGO_PROCS=4

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
run: make test-gnmi TESTGNMI_PROVIDER=openconfig GINKGO_PROCS=4
run: make test-gnmi TESTGNMI_PROVIDER=openconfig GINKGO_PROCS=$(nproc)

why not let the platform use all available?

Comment on lines 118 to 120
if s.grpcServer != nil {
s.grpcServer.GracefulStop()
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if s.grpcServer != nil {
s.grpcServer.GracefulStop()
}
if s.grpcServer != nil {
const timeout = 5 * time.Second
t := time.AfterFunc(timeout, func() {
log.Printf("Graceful shutdown timed out, forcing stop")
s.grpcServer.Stop()
})
s.grpcServer.GracefulStop()
t.Stop()
}

What about this? This completely removes the need to create a done chan and spawing a gorountine in a goroutine. So the go func in the NewTestServer can just become:

	go func() {
		<-ctx.Done()
		server.Close()
	}()

Comment thread test/gnmi/gnmi_test.go
Comment on lines +656 to +684
// Delete and wait for resources with finalizers - controller needs Device alive to clean up gNMI state
for _, gvk := range Resources {
list := &unstructured.UnstructuredList{}
list.SetGroupVersionKind(schema.GroupVersionKind{
Group: gvk.Group,
Version: gvk.Version,
Kind: gvk.Kind + "List",
})

if err := c.List(cleanupCtx, list, client.InNamespace(namespace)); err != nil {
if apimeta.IsNoMatchError(err) {
continue
}
Expect(err).NotTo(HaveOccurred())
}

for _, item := range list.Items {
Expect(client.IgnoreNotFound(c.Delete(cleanupCtx, &item))).To(Succeed())
}

if len(list.Items) > 0 {
Eventually(func(g Gomega) {
var remaining unstructured.UnstructuredList
remaining.SetGroupVersionKind(list.GroupVersionKind())
g.Expect(c.List(cleanupCtx, &remaining, client.InNamespace(namespace))).To(Succeed())
g.Expect(remaining.Items).To(BeEmpty())
}).WithContext(cleanupCtx).Should(Succeed())
}
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess a

c.Delete(cleanupCtx, device, client.PropagationPolicy(metav1.DeletePropagationForeground))

should already work for deleting in the right order.

If that is not the case, then we clearly have some internal errors elsewhere.

Comment thread test/gnmi/gnmi_test.go
controllerCancel()
}

if gnmiServerCancel != nil {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is a race, as ggnmiServerCancel()triggers a context cancelation, which means the the goroutine inside ofNewTestServerwill continue to callGracefulStop(), and Close()later also callsGracefulStop(). I believe this is also the reason why sync.Once` was introduced as a quickfix, but I feel like that shows that the implementation is a bit muddy here.

Comment thread test/gnmi/gnmi_test.go
gnmiServer *testserver.Server
)

// Lifecycle — cancel functions called in AfterSuite

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why two separate contexts? Isn't this just one from t.Context()?

}

func (s *State) Set(path *gpb.Path, raw []byte) {
func (s *State) Set(path *gpb.Path, raw []byte) error {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
func (s *State) Set(path *gpb.Path, raw []byte) error {
func (s *State) Set(path *gpb.Path, raw []byte) (err error) {

nit: avoid having multiple var err error later.

Comment on lines +352 to +363
elems := path.GetElem()
if len(elems) == 0 {
s.Buf = buf
return nil
}

for k, v := range elems[len(elems)-1].GetKey() {
buf, err = sjson.SetBytes(buf, sb.String()+"."+k, v)
if err != nil {
return fmt.Errorf("sjson.SetBytes final key %s: %w", k, err)
}
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
elems := path.GetElem()
if len(elems) == 0 {
s.Buf = buf
return nil
}
for k, v := range elems[len(elems)-1].GetKey() {
buf, err = sjson.SetBytes(buf, sb.String()+"."+k, v)
if err != nil {
return fmt.Errorf("sjson.SetBytes final key %s: %w", k, err)
}
}
if elems := path.GetElem(); len(elems) > 0 {
for k, v := range elems[len(elems)-1].GetKey() {
buf, err = sjson.SetBytes(buf, sb.String()+"."+k, v)
if err != nil {
return fmt.Errorf("sjson.SetBytes final key %s: %w", k, err)
}
}
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants