Skip to content

Environment variables ​

The API server reads .env at the repository root. The template is .env.example. .env holds credentials, so make it readable only by you with chmod 600 .env, and never paste it into repositories or chats.

sh
cp .env.example .env
chmod 600 .env

Restart the API after changing values. If a value is malformed or a setting is missing, the API prints only the setting's name and does not start.

API server: basics ​

VariableDefaultContent
MMT_DATABASE_URLNone (required)PostgreSQL connection URL
NODE_ENVdevelopmentproduction in production. HTTPS URLs become mandatory and development features are disabled
HOST127.0.0.1Address the API listens on
PORT4182Port the API listens on
MMT_PUBLIC_URLhttp://127.0.0.1:4182The URL users open. The SSO callback is built from it
MMT_WEB_ORIGINhttp://127.0.0.1:5182Origin of the web UI. Also used for links in notifications
MMT_ALLOW_PRIVATE_ORIGINSfalsetrue also allows origins with LAN/VPN private IPs and localhost. For development; keep false on a server
MMT_TRUSTED_PROXY_HOPS0 (1 in compose)Proxies in front of the API that append the client to X-Forwarded-For. Used for login rate limits and the client address in audit events. 2 when another proxy sits in front of compose's web
MMT_ALLOW_SEEDfalseAllows loading sample data. Only with AUTH_MODE=development
MMT_ALLOW_LOCAL_EXECUTORfalseAllows the local executor. Only with AUTH_MODE=development
DEVELOPMENT_ADMIN_EMAILadmin@localhostEmail that becomes a global administrator with the development login

Authentication and SSO ​

See Authentication and local accounts and Authentik (SSO) for details.

VariableDefaultContent
AUTH_MODEhybridlocal, oidc, hybrid, development
AUTH_SESSION_IDLE_SECONDS604800 (7 days)Seconds without activity until a session ends. At least 300
AUTH_SESSION_ABSOLUTE_SECONDS604800 (7 days)Seconds from login until a session ends. At least 3600, and not less than the idle limit
OIDC_ISSUER_URLNoneIssuer of the Authentik application. Required for oidc and hybrid. HTTPS
OIDC_CLIENT_IDNoneProvider client ID. Required for oidc and hybrid
OIDC_CLIENT_SECRETNoneProvider client secret
OIDC_ALLOWED_GROUPSNoneGroups allowed to sign in (comma-separated). Required for oidc and hybrid
OIDC_ROLE_MAPPING_JSONNoneGroup → global role mapping. Example: {"mmt-admins":"admin","mmt-users":"user"}
OIDC_DEFAULT_ROLEuserRole of users in no mapped group
OIDC_SCOPESopenid profile email offline_accessRequested scopes. openid is required. Refresh tokens also require the Provider's offline_access mapping
OIDC_LABELAuthentikLabel of the SSO button on the login screen
OIDC_AUTO_LINK_VERIFIED_EMAILfalseWhether to link to a local account with the same email automatically
OIDC_ALLOW_INSECURE_HTTPfalseAllows a loopback HTTP issuer (for tests). Not allowed in production
MMT_SESSION_ENCRYPTION_KEYNoneKey that encrypts Authentik tokens stored with sessions (base64 of 32 bytes). Required for oidc and hybrid
OIDC_RECHECK_SECONDS60Interval (seconds) for rechecking an SSO session's groups
OIDC_TOKEN_SYNC_MAX_AGE_SECONDS604800 (7 days)Seconds after the last group sync during which SSO users' API tokens work. At least 60
MMT_TOKEN_DEFAULT_LIFETIME_DAYS365Default lifetime in days when token creation omits its expiry. 1 to 3650. expiresAt: null never expires; explicit future timestamps have no lifetime cap.

Generate keys in a terminal. Use different values for MMT_SESSION_ENCRYPTION_KEY and MMT_STORAGE_SECRET_KEY.

sh
openssl rand -base64 32

Artifacts and storage ​

Storage settings are described in Storage.

VariableDefaultContent
ARTIFACT_FILESYSTEM_ROOTvar/artifacts under the API's working directoryLocation of the filesystem storage. A relative path is resolved from the API's working directory, so use an absolute path in production
S3_BUCKETNoneWhen set, the S3 storage from environment variables can be selected
S3_ENDPOINTNone (AWS S3)Endpoint of an S3-compatible storage
S3_REGIONus-east-1Region
S3_PREFIXNone (bucket root)Prefix in the bucket. .env.example uses mado-model-tracking
S3_FORCE_PATH_STYLEfalseUse path-style URLs
S3_ACCESS_KEY_ID, S3_SECRET_ACCESS_KEYNoneS3 credentials. Without them, the AWS SDK's default provider chain is used
MMT_STORAGE_SECRET_KEYNoneKey that encrypts in the database the secrets of S3 backends added on the screen (base64 of 32 bytes)
MMT_HOOK_SECRET_KEYNoneKey that encrypts in the database the secrets of webhook hooks (base64 of 32 bytes). Without it, webhook hooks cannot be created and stored ones cannot be verified
MMT_ARTIFACT_MAX_BYTES214748364800 (200 GiB)Limit per Artifact
MMT_ARTIFACT_DELETE_GRACE_DAYS7Days before a deleted Artifact's content is removed. 0 to 3650
MMT_UPLOAD_REQUEST_TIMEOUT_MS0 (disabled)Deadline for a whole upload request
MMT_UPLOAD_IDLE_TIMEOUT_MS120000Time before a stalled upload is cut
MMT_MLFLOW_MULTIPART_UPLOADStrueAccept multipart uploads from the MLflow SDK
MMT_MLFLOW_MULTIPART_DOWNLOADSfalseDo not change. true breaks downloads of MLflow SDK 3.17 and later
MMT_UPLOAD_FINALIZE_WAIT_MS100000How long completing an MLflow multipart upload waits

Notifications ​

See Notifications and operations alerts.

VariableDefaultContent
MMT_NOTIFICATION_*NoneWebhook URLs and signing keys. Choose any name and register it in the channel
MMT_SMTP_URLNoneSMTP server URL: smtp:// (STARTTLS) or smtps://
MMT_SMTP_FROMNoneSender address. Example: mado ML Tracking <mmt@example.com>
NODE_EXTRA_CA_CERTSNonePath of an internal CA certificate (PEM), used to verify SMTP servers and webhook destinations

Mado integration ​

The Mado integration is built into the API server. There is no separate service and no .env of its own. The only variable it needs in the API server's .env is the key that encrypts Mado access tokens. Mado URLs and access tokens are not environment variables: a global administrator saves them in Integrations in the Project. Namespaces are not configured anywhere; Mado reports them for the access token (Mado integration).

VariableDefaultContent
MMT_PLUGIN_SECRET_KEYNoneKey that encrypts Mado access tokens in the database (base64 of 32 bytes). Not needed if you do not use the Mado integration. Without it, access tokens cannot be saved and the integration cannot be enabled
  • Generate the key with openssl rand -base64 32, and use a value different from the other encryption keys (MMT_SESSION_ENCRYPTION_KEY, MMT_STORAGE_SECRET_KEY, MMT_HOOK_SECRET_KEY).
  • Once access tokens are saved, do not change or remove the key: the saved tokens could no longer be decrypted. If the key is lost, restore it or re-enter every saved access token. Restoring the database needs it too, so keep the key somewhere safe.

Other API server settings ​

VariableDefaultContent
MMT_CHECKPOINT_KEEP_COUNT5Checkpoints listed per Run
MMT_CSV_EXPORT_MAX_ROWS50000Maximum rows of a Run search CSV export
MMT_REPORT_SNAPSHOT_MAX_BYTES52428800Limit (bytes) of fixed data in one revision of a shared report
MMT_GIT_SSH_KEY_PATH, MMT_GIT_KNOWN_HOSTS_PATHNonePrivate key and known_hosts (absolute paths) for loading SSH Git repositories into the editor. Set both

Preview worker ​

Settings of the worker that renders previews of long audio and video (npm run preview-worker -w @mmt/api, or the compose preview service). Database and storage settings use the same values as the API server.

VariableDefaultContent
MMT_PREVIEW_FFMPEG_PATHffmpegPath of ffmpeg
MMT_PREVIEW_FFPROBE_PATHffprobePath of ffprobe
MMT_PREVIEW_POLL_INTERVAL_MS5000Interval for looking for Artifacts to process
MMT_PREVIEW_TOOL_TIMEOUT_MS1800000 (30 minutes)Limit of one ffmpeg or ffprobe run. Up to 40 minutes
MMT_PREVIEW_WORK_DIRThe OS temporary directoryWhere content is placed temporarily. Needs room for MMT_PREVIEW_MAX_SOURCE_BYTES
MMT_PREVIEW_MAX_SOURCE_BYTES10737418240 (10 GiB)Larger Artifacts get no preview (skipped, source_too_large)

Worker ​

Settings of mado-tracking-worker. When installed with install, they are written to ~/.config/mado-tracking-worker/<worker-id>.env.

VariableDefaultContent
MMT_API_URLNone (required)URL of the API server. Must also be reachable from Compute targets
MMT_API_TOKENNone (required)Service Account token
MMT_API_TOKEN_FILENoneFile to read the token from (for containers). MMT_API_TOKEN wins if set
MMT_WORKER_IDNone (required)Worker ID. Keep the same value across restarts
MMT_WORKER_TARGET_IDSNone (all allowed)Compute target IDs to serve (comma-separated). Connection checks for which the worker is chosen in Worker for this connection are picked up regardless
MMT_WORKER_SSH_DIRNoneA directory searched, besides ~/.ssh, for SSH key and known_hosts candidates. Only paths are reported (Report SSH key candidates)
MMT_WORKER_STATE_DIR~/.local/state/mado-tracking-worker/<hash of the ID>Directory for the journal of running Jobs
MMT_WORKER_MAX_OUTPUT_FILES10000Limit of output files collected from one Job. 1 to 1000000
MMT_ALLOW_LOCAL_EXECUTORNonetrue enables the local executor (development; the API must allow it too)

With Docker Compose, MMT_WORKER_TOKEN_FILE, MMT_WORKER_SSH_DIR, MMT_WORKER_UID, MMT_WORKER_GID, and MMT_WORKER_API_URL are also used (Worker). Compose's MMT_WORKER_SSH_DIR is the host directory mounted at the container's ~/.ssh.

Python SDK ​

VariableDefaultContent
MMT_API_URLNoneURL of the API server
MMT_API_TOKENNoneAPI token
MMT_OFFLINE_DIR~/.local/share/mado-tracking/offlineDirectory for offline records

Variables passed to running code ​

The worker passes these variables to a Job's code. The code only reads them; you do not set them yourself.

VariableContent
MMT_API_URL, MMT_API_TOKEN, MLFLOW_TRACKING_TOKENAPI URL and the Job token
MMT_PROJECT_ID, MMT_EXPERIMENT_ID, MMT_RUN_ID, MMT_JOB_ID, MMT_JOB_KINDIDs and kind of the Run being executed
MMT_JOB_CONTEXT_FILEJSON with the Job, Run, parameters, model version, input datasets, and more
MMT_PARAMETERS_FILE, MMT_PARAMETERS_JSONParameters
MMT_MODEL_VERSION_FILE, MMT_MODEL_VERSION_IDThe model version to use
MMT_DATASET_VERSIONS_FILE, MMT_INPUT_DATASET_VERSION_IDS, MMT_INPUT_DATASET_DIRSInput dataset versions and the directories holding their content
MMT_UPSTREAM_RUN_ID, MMT_UPSTREAM_RUN_FILEThe upstream Run (only when there is one)
MMT_RESUME_CHECKPOINT_DIR, MMT_RESUME_STEP, MMT_RESUME_CHECKPOINT_FILEThe checkpoint to resume from (only when resuming)
MMT_OUTPUTS_DIR, MMT_RESULT_FILEThe output directory and the path of result.json