Shared Services

If many tests use the same service (like a database), defining it in every test file gets repetitive. Instead, define it once in the configuration file and reference it with ref:. Update the image or config in one place, and every test picks up the change.

Defining shared services

Add services to spark.yaml in your project root:

# spark.yaml
services:
  postgres:
    image: postgres:15
    environment:
      POSTGRES_PASSWORD: secret
    healthcheck:
      test: pg_isready
      retries: 10

  redis:
    image: redis:7-alpine
    healthcheck:
      test: redis-cli ping
      retries: 10

Referencing in tests

Use ref: with the service key from the config. You still need name: to set the hostname in the test's Docker network.

services:
  - ref: postgres
    name: db
  - ref: redis
    name: cache

DRY principle in action. When you upgrade Postgres from 15 to 16, change it once in spark.yaml and every test that references postgres gets the update. No find-and-replace across dozens of files.

Overriding values

You can override any field from the shared definition. Environment variables are merged (your override wins on key conflict), other fields are replaced entirely.

services:
  - ref: postgres
    name: db
    environment:
      POSTGRES_DB: myapp_test     # added on top of POSTGRES_PASSWORD from config

Artifacts are also merged — the shared service's artifacts are collected first, then any artifacts defined on the ref in the test are appended:

# spark.yaml — shared definition always collects pg logs
services:
  postgres:
    image: postgres:15
    healthcheck:
      test: pg_isready
      retries: 10
    artifacts:
      - path: /var/log/postgresql/

# test.spark — adds a dump on top of the template's artifacts
services:
  - ref: postgres
    name: db
    artifacts:
      - path: /tmp/dump.sql

Before and after

Here's what the same test looks like with and without shared services:

# Clean and focused
services:
  - ref: postgres
    name: db
  - ref: redis
    name: cache

# Repeated in every test file
services:
  - name: db
    image: postgres:15
    environment:
      POSTGRES_PASSWORD: secret
    healthcheck:
      test: pg_isready
      retries: 10
  - name: cache
    image: redis:7-alpine
    healthcheck:
      test: redis-cli ping
      retries: 10