LakeQL - Contributing - Local Development

Requirements #

Install these tools before working on LakeQL:

Node.js v24 or later (see .nvmrc )
pnpm v11+ (declared in packageManager field)
Docker — container runtime for the local Trino cluster
uv — Python package manager (required for minitrino)

Quick Start #

Clone and install

Clone the repository and install dependencies:

1
2
3
4
git clone https://github.com/noxify/lakeql.git
cd lakeql
pnpm install

Start the local Trino cluster

The project uses minitrino to provision a local Trino cluster with Hive, MinIO, LDAP, and OAuth2 modules:

1
2
pnpm mt:start

Seed test data

Generate Parquet files, upload them to MinIO, and create the corresponding Trino schemas and tables:

1
2
pnpm seed --all

Verify the setup

Run a quick query to confirm everything is working:

1
2
pnpm query "SELECT count(*) FROM hive.test.products"

The local environment uses minitrino to provision a Trino cluster with all necessary modules. The configuration lives in .minitrino/lib/ within the project root, so every developer gets the same setup.

Available Commands #

Command	Description
`pnpm mt:start`	Provision and start the cluster (including MinIO proxy)
`pnpm mt:restart`	Restart the cluster
`pnpm mt:stop`	Stop the cluster (keep volumes)
`pnpm mt:clean`	Stop and remove everything

MinIO S3 Proxy #

Minitrino exposes only the MinIO Console (port 9001) by default. The mt:start command also starts a lightweight proxy that makes the S3 API available on localhost:9000 for local tooling.

Seeding Test Data #

The seed command generates Parquet files, uploads them to MinIO, and creates the corresponding Trino schemas and tables. Each run is a full reset. It replaces existing tables and data with freshly generated records.

Command	Description
`pnpm seed --all`	Seed all definitions
`pnpm seed -d <name>`	Seed a specific definition
`pnpm seed -d <name> -d <name>`	Seed multiple definitions
`pnpm seed --all --amount 500`	Custom record count (default: 1000)

Adding a Dataset #

Seed definitions live in tooling/test-data/seed.config.ts. To add a custom dataset, define columns and generate inline, or create a new file under tooling/test-data/src/datasets/:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
import type { ColumnDefinition } from "../seed/config"

export const myColumns: ColumnDefinition[] = [
  { name: "id", type: "BIGINT" },
  { name: "value", type: "VARCHAR" },
]

export async function myGenerate(
  amount: number,
  targetDir: string
): Promise<string> {
  // Generate parquet file, return path
}

Then import and register the dataset in tooling/test-data/seed.config.ts.

Querying Trino #

A lightweight query tool ships with the project for quick verification:

1
2
3
4
5
6
7
8
9
# Table output (default)
pnpm query "SELECT * FROM hive.test.products LIMIT 5"

# JSON output
pnpm query "SELECT * FROM hive.test.orders LIMIT 3" -f json

# CSV output
pnpm query "SELECT count(*) FROM hive.test.products" -f csv

Connection Details #

Service	URL	Credentials
Trino	`http://localhost:8080`	user: `admin`, no password
Trino UI	`https://localhost:8443/ui`	`admin@minitrino.com`
MinIO Console	`http://localhost:9001`	`access-key` / `secret-key`
MinIO S3 API	`http://localhost:9000` (via proxy)	`access-key` / `secret-key`

The Trino UI requires https:// since the setup uses OAuth2. Your browser may show a certificate warning that you need to bypass.

Default Users #

The local setup uses minitrino's default users for LDAP and OAuth:

Troubleshooting #

Seed fails with "bucket does not exist" #

Check that the MinIO proxy is running. It starts with pnpm mt:start, but you can restart it manually:

1
2
pnpm mt:proxy

Seed fails with "401 Unauthorized" #

Trino uses user-only auth (no password) for local development. This works out of the box with minitrino defaults. Confirm the cluster is running with pnpm mt:start.

Query fails with connection error #

Confirm minitrino is running:

1
2
pnpm mt:start