Skip to content

Add Scrutiny Helm chart - #502

Open
votdev wants to merge 1 commit into
harvester:masterfrom
votdev:scrutiny
Open

Add Scrutiny Helm chart#502
votdev wants to merge 1 commit into
harvester:masterfrom
votdev:scrutiny

Conversation

@votdev

@votdev votdev commented Mar 26, 2026

Copy link
Copy Markdown
Member

Problem:

Prometheus node-exporter does not collect SMART values of the disks connected to the cluster nodes.

Solution:

Make use of the scrutiny project which provides a nice small UI to display the SMART values of the disks that are auto-detected on each node in the cluster.

Bildschirmfoto vom 2026-03-26 11-53-28 Bildschirmfoto vom 2026-03-26 10-45-24

Related Issue(s):

n/a

Test plan:

  • Download the Helm chart
curl -sL "https://github.com/votdev/harvester-charts/archive/refs/heads/scrutiny.tar.gz" \
  | tar -xz --strip-components=2 harvester-charts-scrutiny/charts/scrutiny
  • Validate the Helm chart
helm template scrutiny ./scrutiny -n scrutiny
helm lint ./scrutiny -f values.yaml
helm install scrutiny ./scrutiny -n scrutiny --dry-run --debug
  • Install the Helm chart
helm install scrutiny -n harvester-system --dry-run ./scrutiny
helm install scrutiny -n harvester-system ./scrutiny
  • Validate installation (on a 3 node cluster)
    Check the running pods.
  • 3 collector pods
  • 1 Influx DB pod
  • 1 Web pod
kubectl get deployment -n harvester-system
  • Go to Service Discovery > Services in the embedded Rancher. Look out for scrutiny-web. Click on the IP in the Targetcolumn to open the Scrutiny WebUI.

Additional documentation or context

@votdev votdev self-assigned this Mar 26, 2026
Copilot AI review requested due to automatic review settings March 26, 2026 11:12

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new scrutiny Helm chart to deploy the Scrutiny stack (web UI, collector DaemonSet, and InfluxDB) so SMART disk metrics can be collected and viewed in-cluster.

Changes:

  • Introduces a new Helm chart (charts/scrutiny) with default values and chart metadata.
  • Adds Kubernetes manifests for Scrutiny Web + InfluxDB Deployments, Services, PVCs, and a collector DaemonSet.
  • Adds Helm helper templates for naming/namespace overrides.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 13 comments.

Show a summary per file
File Description
charts/scrutiny/Chart.yaml Declares the new Helm chart metadata (name/version/appVersion).
charts/scrutiny/values.yaml Provides default images, persistence sizes, tolerations, and collector scheduling.
charts/scrutiny/templates/_helpers.tpl Adds helper templates for name/fullname/namespace.
charts/scrutiny/templates/deployment-web.yaml Deploys the Scrutiny web UI and config PVC mount.
charts/scrutiny/templates/service-web.yaml Exposes the web UI via a ClusterIP Service.
charts/scrutiny/templates/deployment-influxdb.yaml Deploys InfluxDB with a data PVC mount.
charts/scrutiny/templates/service-influxdb.yaml Exposes InfluxDB internally to the web/collector components.
charts/scrutiny/templates/pvc-influxdb.yaml Creates persistent storage for InfluxDB data.
charts/scrutiny/templates/pvc-config.yaml Creates persistent storage for Scrutiny config.
charts/scrutiny/templates/daemonset-collector.yaml Runs the Scrutiny collector on each node with host access for SMART collection.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread charts/scrutiny/templates/deployment-web.yaml
Comment thread charts/scrutiny/templates/service-influxdb.yaml
Comment thread charts/scrutiny/templates/deployment-web.yaml
Comment thread charts/scrutiny/templates/deployment-web.yaml
Comment thread charts/scrutiny/values.yaml
Comment thread charts/scrutiny/templates/service-web.yaml
Comment thread charts/scrutiny/templates/deployment-web.yaml Outdated
Comment thread charts/scrutiny/templates/deployment-influxdb.yaml
Comment thread charts/scrutiny/templates/deployment-influxdb.yaml
Comment thread charts/scrutiny/values.yaml

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 2 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread charts/scrutiny/templates/pvc-influxdb.yaml
Comment thread charts/scrutiny/templates/pvc-config.yaml

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 8 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread charts/scrutiny/templates/deployment-influxdb.yaml
Comment thread charts/scrutiny/templates/daemonset-collector.yaml
Comment thread charts/scrutiny/templates/pvc-influxdb.yaml
Comment thread charts/scrutiny/Chart.yaml
Comment thread charts/scrutiny/values.yaml
Comment thread charts/scrutiny/templates/_helpers.tpl
Comment thread charts/scrutiny/templates/pvc-config.yaml
Comment thread charts/scrutiny/templates/daemonset-collector.yaml
Signed-off-by: Volker Theile <vtheile@suse.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 3 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +45 to +49
mountPath: /opt/scrutiny/config
volumes:
- name: config
persistentVolumeClaim:
claimName: {{ include "scrutiny.fullname" . }}-config-pvc

Copilot AI Mar 26, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The web Deployment mounts a single ReadWriteOnce PVC (config-pvc) but allows scaling via replicas. Scaling above 1 will leave extra replicas Pending due to the RWO volume, and the default RollingUpdate strategy can also stall rollouts when a surge pod tries to mount the same PVC. Consider constraining this component to 1 replica and/or setting spec.strategy.type: Recreate (or switching to StatefulSet / RWX storage) to make updates reliable.

Copilot uses AI. Check for mistakes.
Comment on lines +36 to +40
mountPath: /var/lib/influxdb2
volumes:
- name: influxdb-data
persistentVolumeClaim:
claimName: {{ include "scrutiny.fullname" . }}-influxdb-pvc

Copilot AI Mar 26, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This Deployment mounts a single ReadWriteOnce PVC for /var/lib/influxdb2 but does not set spec.strategy. With the default RollingUpdate strategy, upgrades can stall because a surge pod may be created before the old pod is terminated, and both cannot mount the same RWO volume. Set spec.strategy.type: Recreate (see charts/harvester-vm-import-controller/templates/deployment.yaml:9-10) or use a StatefulSet to make rollouts reliable.

Copilot uses AI. Check for mistakes.
- effect: NoSchedule
key: node-role.kubernetes.io/master
- effect: NoExecute
operator: Exists

Copilot AI Mar 26, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The default toleration effect: NoExecute with operator: Exists and no tolerationSeconds tolerates all NoExecute taints indefinitely (including node.kubernetes.io/not-ready / unreachable). For the web/influxdb Deployments this can reduce availability by preventing eviction/rescheduling when a node becomes unhealthy. Consider scoping this toleration to the specific taints you need, or adding tolerationSeconds for node-condition taints.

Suggested change
operator: Exists
operator: Exists
tolerationSeconds: 300

Copilot uses AI. Check for mistakes.
@w13915984028

Copy link
Copy Markdown
Member

Thanks for the PR, some open questions for discussion,

(1) How to represent this chart to user.

It could be a subchart of rancher-monitoring https://github.com/rancher/charts/tree/dev-v2.14/charts/rancher-monitoring ; or an addon/experimental-addon on Harvester.

Rancher mirror all/most of the images by third-party like https://github.com/harvester/harvester-installer/blob/c8fd2f4e80cdeef1d79396ef75c6e49deb4d0ba8/scripts/images/rancher-images.txt#L20

(2) License https://github.com/analogj/scrutiny?tab=readme-ov-file#licenses, I am not an expert on this area.

(3) Test plans, coutinuous upgrade plans.

maybe we need a HEP.

@innobead

Copy link
Copy Markdown
Contributor

As discussed, we should consider enhancing our node disk manager to provide enough capacity to monitor disk statistics and export them via our own Prometheus metrics API. When using an upstream chart, consider maintenance, resource consumption, release workload, and feature scope, among other factors. This requires strong justification and analysis to ensure alignment with our project goals. (Otherwise, users can simply install the upstream chart on their own, since there are no integration benefits here.)

Suggest reviewing harvester/harvester#7416 (comment) first for ideas from Longhorn SMART monitoring (a minimal implementation of the required capabilities), including kernel-managed disks, filesystems, or user-space disks managed by SPDK, to help shape our design. Once you review this, you'll have a clearer understanding of the effort involved and whether any open-source projects could help us.

cc @harvester/storage @harvester/pm

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants