Data Forestry Service · Field Manual No. 3

S3quoia

Massive data. Clear trails.

An S3 query engine — accurate and simple. With local caching, an MCP server, and a 1 GB scan guardrail.

★ View on GitHub $ npm install s3quoia ⧉

§ 01 · Get on the trail

Running in minutes

Point it at your bucket, write SQL with date tokens, get results. No cluster, no config server — just a package.

  1. ①  npm install s3quoia
  2. ②  point at your S3 bucket
  3. ③  query with {yyyy}/{MM}/{dd} tokens
lake.js 
import s3quoia from 's3quoia';

const rows = await s3quoia({
  accessKeyId: process.env.S3_ACCESS_KEY_ID,
  secretAccessKey: process.env.S3_SECRET_ACCESS_KEY,
  defaultEndpoint: process.env.S3_ENDPOINT,
  defaultBucket: 'events',
  bucketsDir: '/tmp/s3-cache',
  from: new Date('2025-01-01').getTime(),
  to: new Date('2025-01-31').getTime(),
  query: `SELECT region, COUNT(*) AS hits
  FROM read_parquet(
    'events/{yyyy}/{MM}/{dd}/*.parquet'
  )
  GROUP BY region`,
  format: 'jsonRecords',
});

§ 02 · Standard Issue Equipment

Time-range queries

Give it a from/to range and S3quoia reads only the Hive partitions inside it — resolving {yyyy}/{MM}/{dd} paths for you. No manual globbing, no over-fetching.

Local disk cache

Downloaded files are cached and reused across queries. Automatic eviction keeps disk usage in check — repeat queries skip S3 entirely.

Built-in MCP server

Connect Claude or any MCP-compatible client to your data lake with a single npx command. No server to provision.

Cross-bucket joins

Reference multiple S3 buckets in a single query using the same clean date-token syntax. No manual URI construction — as long as your buckets share credentials, S3quoia handles the rest.

§ 03 · Why not just DuckDB + S3?

Capability DuckDB + S3 S3quoia
01 Time-partition path pruning manual globbing automatic ✓
02 Local disk cache + eviction ✕ built in ✓
03 MCP server for AI clients ✕ one command ✓
04 Runaway-scan guardrail ✕ 1 GB cap ✓
05 Cross-bucket joins manual URI construction date-token syntax ✓
06 Concurrent duplicate downloads same file, twice lock queue ✓
07 Setup wire it yourself npm install ✓

The Ranger Station

Ask your data lake a question.

S3quoia ships a Model Context Protocol server. Connect Claude Desktop, Claude Code, or any MCP client and query your S3 data in plain language.

$ claude mcp add s3quoia \ -e S3_ENDPOINT=https://s3.amazonaws.com \ -e S3_BUCKET=my-bucket \ -e S3_ACCESS_KEY_ID=key \ -e S3_SECRET_ACCESS_KEY=secret \ -- npx -y s3quoia
  • Claude Code
  • Claude Desktop
  • Any MCP client