SlimFaas Jobs

SlimFaas lets you run one‑off, batch, and scheduled (cron) jobs either on‑demand via simple HTTP calls or automatically on a cron‑like cadence. Each job definition includes guard‑rails: cap concurrency with NumberParallelJob, enforce per‑pod CPU & memory limits, and decide whether the endpoint is Public (external) or Private (in‑cluster). You get a powerful REST API while keeping your cluster safe from resource spikes.

From a trigger to a completed job

HTTP requests and schedules feed the same job execution path. A job configuration supplies the executable/image, dependencies, concurrency and retry limits; it is distinct from each execution created from that configuration.

In the dashboard, Jobs Overview shows configurations and running work. List executions with GET /job/name and create/delete your own schedules in the job exercise. CPU/memory resource limits depend on the orchestrator; native local processes do not enforce container limits.

Recovery after a temporary SlimData interruption

An HTTP 202 Accepted response means the job has been queued; execution still waits for dependencies, an available concurrency slot and the cluster leader. A temporary SlimData interruption can delay dispatch. Once the cluster can make progress again, SlimFaas retries queue operations without requiring a node restart.

Queue reads wait at most five seconds per attempt for the committed Raft log to be applied locally. An expiration is reported as a SlimData unavailability error and retried; if those retries are exhausted, the jobs worker can try again on a later cycle. Internal mutation requests keep their HTTP timeout active until the complete response body arrives, including when the response headers have already arrived. The default internal HTTP timeout is 100 seconds.

If a mutation response is lost or times out, SlimFaas retries the same ordered batch against the current leader. Its producer, generation, sequence and request IDs are preserved so SlimData can return the previously committed result without applying the queue mutation twice. These transport retries are separate from retries of the job program itself. This does not provide an exactly-once guarantee for external side effects performed by a job.

If jobs remain queued after recovery, check dependency readiness, active jobs and the node logs for Raft local log application timed out or Retrying the same ordered SlimData batch. Retained Succeeded and Failed jobs do not occupy concurrency slots.

1. Why Use Jobs?

• Short‑lived or periodic tasks — perform a specialised computation once or on a fixed cadence and then shut down.
• Separate scaling — jobs can have different concurrency settings from standard functions.
• Infrastructure safety — cap concurrent pods with NumberParallelJob and enforce per‑pod CPU/memory limits to protect your cluster.
• Secure yet open API — trigger jobs & schedules through REST, choosing Public visibility for external access or Private for in‑cluster only.
• Dependency handling — automatically wait for required deployments before launching.
• Cost optimisation — jobs and its dependencies run on‑demand and scale to zero when finished.
• Built‑in scheduling — declare cron schedules next to the job definition or create them on the fly with the API.

2. Defining Jobs via Configuration

You configure SlimFaas jobs by providing a JSON configuration. The most common method is to store this JSON in a Kubernetes ConfigMap and pass it to SlimFaas via the SlimFaas__JobsConfiguration environment variable.

Example ConfigMap + StatefulSet

Below is a comprehensive example showing both the job configuration and how it’s referenced in the SlimFaas StatefulSet:

---
apiVersion: v1
kind: Namespace
metadata:
  name: slimfaas-demo
  labels:
    name: slimfaas
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: slimfaas-config
  namespace: slimfaas-demo
data:
  SlimFaas__JobsConfiguration: |
    {
      "Configurations": {
        "fibonacci": {
          "Image": "axaguildev/fibonacci-batch:latest",
          "ImagesWhitelist": [],
          "Resources": {
            "Requests": {
              "cpu": "400m",
              "memory": "400Mi"
            },
            "Limits": {
              "cpu": "400m",
              "memory": "400Mi"
            }
          },
          "DependsOn": ["fibonacci1"],
          "Environments": [],
          "BackoffLimit": 1,
          "Visibility": "Public",
          "NumberParallelJob": 2,
          "TtlSecondsAfterFinished": 36000,
          "RestartPolicy": "Never"
        }
      },
      "Schedules": {
          "fibonacci": [
            {
              "Schedule": "0 0 * * *",  # every day at midnight
              "Args": ["42"]
            },
            {
              "Schedule": "0 0 * * 0",  # every Sunday at midnight
              "Args": ["42"],
              "DependsOn": ["fibonacci2"],
            }
          ]
        }
    }
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: slimfaas
  namespace: slimfaas-demo
spec:
  replicas: 1
  podManagementPolicy: Parallel
  serviceName: slimfaas
  selector:
    matchLabels:
      app: slimfaas
  template:
    metadata:
      labels:
        app: slimfaas
    spec:
      volumes:
        - name: slimfaas-volume
          emptyDir:
            sizeLimit: 10Mi
      serviceAccountName: slimfaas
      containers:
        - name: slimfaas
          image: axaguildev/slimfaas:latest
          livenessProbe:
            httpGet:
              path: /health
              port: 5000
            initialDelaySeconds: 3
            periodSeconds: 10
            timeoutSeconds: 8
            terminationGracePeriodSeconds: 30
          env:
            - name: SlimFaas__JobsConfiguration
              valueFrom:
                configMapKeyRef:
                  name: slimfaas-config
                  key: SlimFaas__JobsConfiguration
          resources:
            limits:
              memory: "76Mi"
              cpu: "400m"
            requests:
              memory: "76Mi"
              cpu: "250m"
          ports:
            - containerPort: 5000
              name: slimfaas
            - containerPort: 3262
              name: slimdata
---
apiVersion: v1
kind: Service
metadata:
  name: slimfaas
  namespace: slimfaas-demo
spec:
  selector:
    app: slimfaas
  ports:
    - name: "http"
      port: 5000
    - name: "slimdata"
      port: 3262

Explanation of Each Configuration Field

In SlimFaas__JobsConfigurationenvironment variable, you define jobs under the "Configurations" key and their schedules under the "Schedules" key. Here’s a breakdown of the configuration structure:

{
  "Configurations": {
    "fibonacci": {
      "Image": "axaguildev/fibonacci-batch:latest",
      "ImagesWhitelist": ["axaguildev/fibonacci-batch:*"],
      "Resources": {
        "Requests": {
          "cpu": "400m",
          "memory": "400Mi"
        },
        "Limits": {
          "cpu": "400m",
          "memory": "400Mi"
        }
      },
      "DependsOn": ["fibonacci1"],
      "Environments": [],
      "BackoffLimit": 1,
      "Visibility": "Public",
      "NumberParallelJob": 2,
      "TtlSecondsAfterFinished": 36000,
      "RestartPolicy": "Never"
    }
  },
  "Schedules": {
      "fibonacci": [
          { "Schedule": "0 0 * * *", "Args": ["43"] },  # every day at midnight
          { "Schedule": "0 0 * * 0", "Args": ["43"] }   # every Sunday at midnight
      ]
  }
}

Configurations

You can define multiple jobs definition under the "Configurations" key. Each job has a unique name (e.g., "fibonacci").

Warning: These definitions are protections about actions you can do via API calls, such as triggering a job or creating a schedule. For example, we cannot ask for more resources than the ones defined in the job configuration. The same for the number of parallel jobs, the image used, etc.

Schedules

You can also define cron schedules for jobs under the "Schedules" key. Each schedule entry contains:

{
  "Schedules": {
    "fibonacci": [
      {
        "Schedule": "0 0 * * *",  # Every day at midnight
        "Args": ["42"]
      },
      {
        "Schedule": "0 0 * * 0",  # Every Sunday at midnight
        "Args": ["42"],
        "DependsOn": ["fibonacci2"]
      }
    ]
  }
}

Default configuration values

If you trigger a job whose name does not exist in SlimFaas__JobsConfiguration, SlimFaas falls back to a built‑in entry called Default. It ships with conservative resource requests/limits (cpu: 100m, memory: 100Mi), no dependencies, and the same visibility/retry defaults. You can customise these defaults by adding or overriding the "Default" key in your configuration map.

2.3 Defining Jobs from Kubernetes CronJob resources

SlimFaas can discover job definitions from Kubernetes CronJob resources when the CronJob carries the annotation:

SlimFaas/Job: "true"

When this annotation is present, SlimFaas's ExtractJobConfigurations scans the CronJob and converts parts of it into a SlimFaas job configuration (SlimfaasJob). Below you'll find the exact demo CronJob (from demo/deployment-cron.yaml) followed by a table showing how fields and annotations are mapped to SlimFaas configuration.

Example CronJob (exact content from demo/deployment-cron.yaml):

apiVersion: batch/v1
kind: CronJob
metadata:
  name: fibonacci5
  namespace: slimfaas-demo
  annotations:
    SlimFaas/Job: "true"
    SlimFaas/DefaultVisibility: "Public"
    SlimFaas/NumberParallelJob: "1"
    SlimFaas/DependsOn: "fibonacci1,fibonacci2"
    SlimFaas/Schedules: '[{"Schedule":"0/2 * * * *","Args":["39"]}]'
spec:
  schedule: "0 0 * * *"
  suspend: true
  jobTemplate:
    spec:
      backoffLimit: 1
      ttlSecondsAfterFinished: 60
      template:
        spec:
          containers:
            - name: fibonacci11
              image: axaguildev/fibonacci-batch:latest
              resources:
                requests:
                  cpu: "400m"
                  memory: "400Mi"
                limits:
                  cpu: "400m"
                  memory: "400Mi"
          restartPolicy: Never

CronJob → SlimFaas Job Configuration Mapping

Note: The CronJob's suspend field must be set to true for SlimFaas to consider it a valid job definition. This ensures SlimFaas manages the scheduling and triggering of the job, preventing Kubernetes from automatically creating Job resources outside of SlimFaas's control.

The native local manifest uses the same annotations under jobs.<name>:

jobs:
  fibonacci5:
    command: ["dotnet", "run", "--project", "FibonacciBatch.csproj", "--"]
    workingDirectory: samples/FibonacciBatch
    annotations:
      SlimFaas/Job: "true"
      SlimFaas/DefaultVisibility: "Public"
      SlimFaas/NumberParallelJob: "1"
      SlimFaas/DependsOn: "fibonacci1,fibonacci2"
      SlimFaas/Schedules: '[{"Schedule":"*/2 * * * *","Args":["39"]}]'

In native local mode, command replaces the container image and the Kubernetes Job-spec settings remain structural YAML fields. The old local metadata fields parallelism, visibility, dependsOn, and schedules are not aliases and are rejected.

2.3.1 Declaring Schedules via SlimFaas/Schedules Annotation

You can embed one or more cron schedules directly on the CronJob using the SlimFaas/Schedules annotation. Its value is a JSON array of schedule entries:

annotations:
  SlimFaas/Job: "true"
  SlimFaas/Schedules: |
    [
      { "Schedule": "*/2 * * * *", "Args": ["39"] },  # every 2 minutes
      { "Schedule": "0 6 * * 1", "Args": ["99"], "DependsOn": ["fibonacci2"] }
    ]

Note: The schedule */2 * * * * means the job will run every 2 minutes, every hour, every day.

Each entry is merged with the CronJob's own configuration, so fields you omit are inherited from the CronJob spec (image, resources, backoffLimit, ttlSecondsAfterFinished, restartPolicy, dependsOn, environments). Fields you provide explicitly override the inherited value.

Tip: Schedules declared via the SlimFaas/Schedules annotation are merged at synchronisation time with any schedules defined in SlimFaas__JobsConfiguration. This lets you keep recurring schedules close to the workload definition (in the CronJob manifest) while still being able to add dynamic schedules at runtime via the /job-schedules API.

3. Invoking and Managing Jobs

SlimFaas exposes HTTP endpoints to trigger, list, and delete jobs.

3.1 Triggering a Job

To trigger a job, make an HTTP request to the job endpoint:

POST http://<slimfaas>/job/<jobName>

• <jobName> corresponds to the name defined in "Configurations" (e.g., "fibonacci" in the example). The route has no arbitrary path suffix. Creation returns 202 with a JSON Id; use that ID when inspecting or deleting an execution.

Example 1 (default parameters):

curl -X POST http://localhost:30021/job/fibonacci \
  -H 'Content-Type: application/json' --data '{"Args":["10"]}'

Example 2 (overrides default configuration):

curl -X POST http://localhost:30021/job/fibonacci \
  -H 'Content-Type: application/json' \
  --data '{"Args":["10"],"DependsOn":["fibonacci2"],"Resources":{"Requests":{"cpu":"200m","memory":"200Mi"},"Limits":{"cpu":"200m","memory":"200Mi"}},"BackoffLimit":1,"TtlSecondsAfterFinished":100,"RestartPolicy":"Never"}'

• SlimFaas merge parameters with the default configured value in ConfigMap
• SlimFaas checks if the job is configured and, if so, spawns a Kubernetes Job resource in slimfaas-demo (or whichever namespace you deployed SlimFaas to).
• If the job is Public, you can call it from anywhere.
• If the job is Private, SlimFaas verifies that the request originates from trusted pods or is internal to the cluster.

3.2 Listing Jobs

To list jobs (queued and already running/finished) for a queue or job family, send a GET request:

GET http://<slimfaas>/job/<queueName>

Example:

curl -X GET http://localhost:30021/job/fibonacci

Example response:

[
  { "Id": "1", "Name": "daisy-slimfaas-job-12772", "Status": "Queued",   "PositionInQueue": 1, "InQueueTimestamp": 1, "StartTimestamp": -1 },
  { "Id": "2", "Name": "daisy-slimfaas-job-12732", "Status": "Running",  "PositionInQueue": -1, "InQueueTimestamp": 1, "StartTimestamp": 1 }
]

3.3 Deleting a Job

To delete a job after it has left the queue, send a DELETE request with the job name and job ID:

DELETE http://<slimfaas>/job/<jobName>/{id}

4. Visibility: Public vs. Private

As with functions, you can mark jobs as Public or Private:

• Public: Accessible from outside the cluster (e.g., direct HTTP calls).
• Private: Accessible only from within the cluster or trusted pods.

Use the Visibility field in the job configuration:

{
  "Visibility": "Public"
}

5 Managing Cron Schedules

SlimFaas ships with a companion API to create, list, and delete cron schedules at runtime. The routes live under /job-schedules. Creation and deletion enforce the job configuration’s visibility. Listing dynamic schedules does not perform that same visibility check; control access at your gateway when needed.

Request body for POST

{
  "Schedule": "0 0 * * *",           // cron expression (required)
  "Args": ["42", "43"],             // optional
  "Image": "axaguildev/fibonacci-batch:1.0.1",  // optional — must be in ImagesWhitelist
  "DependsOn": ["fibonacci2"],
  "Resources": {                       // optional overrides
    "Requests": {"cpu": "200m", "memory": "200Mi"},
    "Limits":   {"cpu": "200m", "memory": "200Mi"}
  },
  "Environments": [],
  "BackoffLimit": 1,
  "TtlSecondsAfterFinished": 100,
  "RestartPolicy": "Never"
}

Only Schedule is mandatory. Any additional keys follow the same override/merge rules as a one-off job trigger. Creation returns 201 {"Id":"..."}; the returned ID is not a numeric position in the list.

Examples

# Set BASE_URL to your demo; native local mode uses port 30020.
export BASE_URL=http://127.0.0.1:30021
SCHEDULE_ID=$(curl -fsS -X POST "$BASE_URL/job-schedules/fibonacci" \
  -H 'Content-Type: application/json' \
  --data '{"Schedule":"0 0 * * *","Args":["10"]}' | jq -r .Id)
curl -fsS "$BASE_URL/job-schedules/fibonacci"
curl -i -X DELETE "$BASE_URL/job-schedules/fibonacci/$SCHEDULE_ID"

Behaviour At the scheduled time, SlimFaas triggers the job exactly as if you had called POST /job/<jobName> with the provided overrides. Dependency checks, visibility rules, and concurrency limits all apply identically for manual and scheduled executions.

5.1 Backup & Restore of Dynamic Schedules

Dynamic schedules (created at runtime via POST /job-schedules) are stored in the internal SlimData Raft database. Deleting Raft storage also deletes these schedules. Export the schedules before a planned storage reset; ordinary follower recovery should preserve storage.

To prevent data loss, SlimFaas can automatically back up all dynamic schedule data to a separate volume and restore it on cold start when the database is empty.

Enabling backup

Set the following environment variables. SlimData__BackupDirectory is required; SlimData__BackupIntervalSeconds is optional.

How it works

1. Periodic backup — Every BackupIntervalSeconds seconds, each node reads all ScheduleJob:* hashsets from its local Raft state and computes a SHA-256 hash. The backup file (schedule-jobs-backup.json) is written to the backup directory only when the hash differs from the previous write, avoiding unnecessary I/O. All nodes perform this backup independently on their own volume.
2. Startup sync — When any node starts and the Raft state already contains schedule data (e.g., it joined an existing cluster), it immediately writes a backup before entering the periodic loop.
3. Restore — On cold start (coldStart=true) with an empty database, the master node checks the backup directory. If a non-empty backup file exists, it restores all schedule entries into the Raft cluster automatically.

Important: The backup volume must be separate from the Raft database volume. This way, when you delete the Raft volumes to fix desynchronisation, the backup volume is preserved and the schedules can be restored on the next cold start.

Kubernetes example

volumeClaimTemplates:
  - metadata:
      name: slimfaas-volume        # Raft database
    spec:
      accessModes: [ ReadWriteOnce ]
      resources:
        requests:
          storage: 2Gi
  - metadata:
      name: slimfaas-backup        # Schedule backup (separate volume)
    spec:
      accessModes: [ ReadWriteOnce ]
      resources:
        requests:
          storage: 1Gi

env:
  - name: SlimData__Directory
    value: "/database"
  - name: SlimData__BackupDirectory
    value: "/backup"
  - name: SlimData__BackupIntervalSeconds
    value: "60"
volumeMounts:
  - name: slimfaas-volume
    mountPath: /database
  - name: slimfaas-backup
    mountPath: /backup

Docker Compose example

services:
  slimfaas:
    environment:
      SlimData__Directory: "/database"
      SlimData__BackupDirectory: "/backup"
      SlimData__BackupIntervalSeconds: "60"
    volumes:
      - slimfaas_data:/database
      - slimfaas_backup:/backup

volumes:
  slimfaas_data:
  slimfaas_backup:

6. Visibility: Public vs. Private

As with functions, you can mark jobs and their schedules as Public or Private.

• Public — job triggers (manual or scheduled) can originate from outside the cluster.
• Private — only trusted pods or in‑cluster callers may trigger or create schedules.

Set the Visibility field inside Configurations.

7. Concurrency and Scaling

• NumberParallelJob limits concurrent executions for each job configuration. Pending and Running executions reserve slots, including jobs waiting for a retry. Succeeded and Failed executions release their slots on the next scheduler refresh, even while their resources are retained. The existing ImagePullBackOff exemption also applies.
• TtlSecondsAfterFinished controls how long finished resources remain available for inspection. A high TTL does not delay queued work or keep function dependencies awake. Finished jobs keep their terminal status in job listings until cleanup; lowering TTL is not required to free concurrency slots.
• In Kubernetes, terminal Complete=True and Failed=True conditions determine completion. Failed pod attempts or partial successes alone do not finish a job that is still executing or retrying.
• BackoffLimit controls total retries.
• SlimFaas manages creation, dependency checks, and honouring cron schedules without exceeding your limits.

To verify slot reuse with a one-hour TTL using native local mode:

dotnet build src/SlimFaas/SlimFaas.csproj
dotnet build samples/FibonacciBatch/FibonacciBatch.csproj
python3 tests/SlimFaas.Tests/Local/test_job_concurrency.py

The smoke test uses temporary state and free loopback ports. It completes four executions at a parallel limit of four, then verifies a fifth succeeds within 60 seconds while the first four remain listed. It covers both retained successes and retained failures, and stops its local supervisor when finished.

8. Summary

• One‑off & batch jobs — POST /job/<jobName>.
• Cron (scheduled) jobs — attach them inside SlimFaas__JobsConfiguration > Schedules or manage at runtime via /job‑schedules endpoints.
• List jobs: GET /job/<queueName>; list schedules: GET /job‑schedules/<jobName>.
• Delete a (running, waiting, finished) job: DELETE /job/<jobName>/{id}; delete a schedule: DELETE /job‑schedules/<jobName>/{id}.
• Visibility, overrides, resource limits, dependency checks, retries, and TTL behave identically for manual and scheduled executions.

Use SlimFaas Jobs & Cron Schedules to handle asynchronous, on‑demand, or recurring workloads with minimal operational overhead. Happy automating! 🚀