feat(health): add health check for flink.apache.org/FlinkSessionJob

Project: argoproj/argo-cd | Language: Go (Lua for health check) | Concepts: testing-strategy, error-handling, observability | Difficulty: junior
Link: https://github.com/argoproj/argo-cd/pull/26626

The problem

Argo CD did not have a built-in health check for the FlinkSessionJob custom resource from flink.apache.org. This meant that users could not see clear health status for these resources in the Argo CD UI, making troubleshooting and monitoring harder.

Why it's non-trivial

A naive solution might just check for the presence of a status field or a single state, but FlinkSessionJob objects can have multiple states (e.g., RUNNING, FINISHED, FAILED, SUSPENDED, RECONCILING, CREATED) and error messages. Properly mapping all these states to meaningful Argo CD health statuses requires careful handling, including fallback logic for missing fields.

The approach

The PR adds a Lua script to define the health check logic, and comprehensive test cases to verify it.

1. Health check logic in Lua:

if obj.status ~= nil and obj.status.jobStatus ~= nil then
  if obj.status.jobStatus.state == "RUNNING" or obj.status.jobStatus.state == "FINISHED" then
    health_status.status = "Healthy"
    health_status.message = obj.status.jobStatus.state
    return health_status
  end

  if obj.status.jobStatus.state == "RECONCILING" or obj.status.jobStatus.state == "CREATED" then
    health_status.status = "Progressing"
    health_status.message = obj.status.jobStatus.state
    return health_status
  end

  if obj.status.jobStatus.state == "SUSPENDED" or obj.status.jobStatus.state == "CANCELED" then
    health_status.status = "Suspended"
    health_status.message = obj.status.jobStatus.state
    return health_status
  end

  if obj.status.jobStatus.state == "FAILED" then
    health_status.status = "Degraded"
    if obj.status.error ~= nil then
      health_status.message = obj.status.error
    else
      health_status.message = "FlinkSessionJob failed"
    end
    return health_status
  end
end

This logic checks the state of the job and maps it to Argo CD's health statuses. It handles all expected states and provides a custom message, using an error message if available for failures.

if obj.status ~= nil and obj.status.error ~= nil then
  health_status.status = "Degraded"
  health_status.message = obj.status.error
  return health_status
end

health_status.status = "Progressing"
health_status.message = "Waiting for FlinkSessionJob to be reconciled"
return health_status

If there is an error but no jobStatus, it still marks as Degraded. If neither status nor error is present, it defaults to Progressing.

2. Test coverage for all cases:

tests:
- healthStatus:
    status: Healthy
    message: RUNNING
  inputPath: testdata/healthy_running.yaml
- healthStatus:
    status: Degraded
    message: "Job submission failed: Could not find a registered TaskManager"
  inputPath: testdata/degraded_failed.yaml
- healthStatus:
    status: Progressing
    message: Waiting for FlinkSessionJob to be reconciled
  inputPath: testdata/progressing_no_status.yaml

Test YAML covers all state and error combinations, ensuring the script works as intended.

From the review thread

No notable review discussion on this PR.

Takeaways

• When adding health checks for custom resources, enumerate all possible status values and map them explicitly to health states to avoid silent failures or misleading UI signals.
• Always provide fallback logic for missing or incomplete status fields to prevent scripts from breaking on unexpected input.
• Use automated tests with real-world scenarios to validate health check logic, ensuring coverage for both typical and edge cases.