Skip to content

Running Locally

The whole Control Plane stack can be run locally for development, with low latency, fast reloading, and interactive debugging.

Follow Getting Started for initial setup instructions before trying to run the system.

The hippodrome orchestrator is the recommended way to run the control plane locally. It starts all core services with proper configuration and service discovery.

See the Hippodrome documentation for full documentation.

Quick start:

# Start core services
PANTS_CONCURRENT=True pants run //components/hippodrome/hippodrome/cli.py -- start

# Start with e-commerce services
PANTS_CONCURRENT=True pants run //components/hippodrome/hippodrome/cli.py -- start --profile ecom

This starts the dashboard at http://localhost:9000 with all services running and aggregated logs.

Backend (Individual Services)

The Monolith server can be run locally in two different modes: online or offline. Online will connect to GCP services in a development project, whereas offline will mock all external dependencies (except for Firestore which has a local emulator).

# Online
pants run //components/monolith:gcp

# Offline
pants run //components/monolith:local

In general, the offline mode is preferred for local development, and online for integration testing.

Note: Offline mode is dependent on mocks of the third-party services, so you may find that you need to extend the capabilities of those mocks to support new features. The see where the mocks are defined, look in the section of components/monolith/monolith/mock_cloud_injector.py commented # GCP mocks.

Debugging

Debugging processes run with Pants is hard, so for debugging in your IDE, you can run the Monolith server directly with python components/monolith/run_local.py. However this requires some environment setup:

  • Create and activate a virtual environment with Python 3.11+
  • Install the requirements with pip install -r components/monolith/requirements.txt

In VS Code, you can open the Debug view (Cmd + Shift + D) and run Debug Monolith (online) or Debug Monolith (local) (offline). This will start the server with the debugger attached.

Admin Console

The Admin Console consists of two services: the Admin Worker (React frontend on Cloudflare Workers) and the Admin Lambda (FastAPI backend). Both need to be running for the full local experience.

Prerequisites

  1. AWS SSO login — the Admin Lambda needs AWS credentials to access DynamoDB tables: 

    aws sso login --profile <your-profile>

  2. Admin Lambda environment — copy and configure the example env: 

    cp components/admin_lambda/example.env components/admin_lambda/.env.local
    Fill in the required values (table names, environment, etc.) for the environment you want to connect to.

You can pull most values from a deployed lambda's environment: 

aws lambda get-function-configuration \
  --function-name "<env>-AdminLambda" \
  --query "Environment.Variables" \
  --output json --profile <your-profile>

For the Cloudflare API token (needed for Query Configs), fetch it from Secrets Manager: 

aws secretsmanager get-secret-value \
  --secret-id "<CF_API_TOKEN_SECRET_NAME from above>" \
  --query "SecretString" --output text --profile <your-profile>
Set this as CF_API_TOKEN in your .env.local.

  1. Admin Worker environment — copy and configure the dev vars: 
    cp components/admin_worker/.dev.vars.example components/admin_worker/.dev.vars
    The default ADMIN_SERVICE_BASE_URL=http://localhost:8000 points to the local Admin Lambda.

Running

Start both services in separate terminals:

# Terminal 1: Admin Lambda (backend API on port 8000)
pants run //components/admin_lambda:local

# Terminal 2: Admin Worker (frontend on port 5173)
cd components/admin_worker
npm install  # first time only
npm run dev

Open http://localhost:5173 in your browser.

Connecting to a deployed environment instead

If you don't want to run the Admin Lambda locally, you can point the worker at a deployed API Gateway:

  1. Get the API Gateway URL: 

    aws cloudformation describe-stacks \
  --stack-name "<env>-AdminInternalApiStack" \
  --query "Stacks[0].Outputs" \
  --output json \
  | jq -r '.[] | select(.OutputKey | test("ApiUrl"; "i")).OutputValue'
    Replace <env> with staging, dev-main, etc.

  2. Get a Cloudflare Access token — go to the deployed admin console (e.g. https://staging-admin.dev-marqo.org), authenticate, then copy the CF_Authorization cookie from browser dev tools.

  3. Update components/admin_worker/.dev.vars: 

    ADMIN_SERVICE_BASE_URL=<api-gateway-url-from-step-1>
LOCAL_CF_ACCESS_TOKEN=<cf-access-token-from-step-2>

  4. Run npm run dev from components/admin_worker.

Console

To run the Console locally, first install the dependencies:

cd components/console
npm ci

Then start the development server:

npm run dev

Open http://localhost:3000 in your browser to see the Console.