Bright Data Local Search

Run powerful web searches locally using Bright Data's SERP API. This skill sets up the unfancy-search pipeline — a local search engine with query expansion, multi-engine SERP retrieval, RRF reranking, deduplication, and domain clustering.

Important: This skill uses the LOCAL version only. Do not use the hosted endpoint.

Prerequisites

• Bright Data account with SERP API access
• Anthropic API key (for query expansion, optional)
• Docker (recommended) or Node.js 18+ with pnpm

Setup

Step 1: Clone and Configure

git clone https://github.com/yaronbeen/unfancy-search.git
cd unfancy-search
cp .env.example .env

Step 2: Set Environment Variables

Edit .env with your credentials:

BRIGHT_DATA_API_TOKEN=your_brightdata_token
BRIGHT_DATA_SERP_ZONE=serp_api1
ANTHROPIC_API_KEY=your_anthropic_key   # Optional: enables AI query expansion

Get your Bright Data token from: https://brightdata.com (SERP API section)

Step 3: Start the Local Server

Docker (recommended):

docker compose up -d
# Server runs at http://localhost:3000

Node.js:

pnpm install
pnpm dev
# Server runs at http://localhost:3000

API Endpoints

All requests go to http://localhost:3000:

Running a Search

Step 1: Submit Search

curl -X POST http://localhost:3000/api/search \
  -H "Content-Type: application/json" \
  -d '{"query": "your search term"}'

The response returns a jobId.

Step 2: Poll for Results

curl http://localhost:3000/api/search-status/{jobId}

Poll every 3 seconds until status is "done".

Search Parameters

Search Modes

• Basic (expand: false): Single query, fastest, no AI cost
• Expanded (expand: true): Claude Haiku generates 3 sub-queries for broader coverage
• Research (research: true): 12 sub-queries for maximum coverage

Usage Examples

Basic Search from an Agent

# Start search
JOB_ID=$(curl -s -X POST http://localhost:3000/api/search \
  -H "Content-Type: application/json" \
  -d '{"query": "best practices for API rate limiting"}' | jq -r '.jobId')

# Poll until done
while true; do
  RESULT=$(curl -s http://localhost:3000/api/search-status/$JOB_ID)
  STATUS=$(echo $RESULT | jq -r '.status')
  if [ "$STATUS" = "done" ]; then
    echo $RESULT | jq '.results'
    break
  fi
  sleep 3
done

Research Mode with Domain Filtering

curl -X POST http://localhost:3000/api/search \
  -H "Content-Type: application/json" \
  -d '{
    "query": "kubernetes scaling strategies",
    "research": true,
    "excludeDomains": ["pinterest.com", "quora.com"]
  }'

Adding Search to an Existing Agent

To give your Claude Code agent search capabilities:

1. Ensure the local server is running (docker compose up -d in the unfancy-search directory)
2. Your agent can use curl or fetch to query http://localhost:3000/api/search
3. Parse the ranked results to ground responses with real web data

Response Format

Results include:

• Ranked URLs with RRF scores
• Domain clustering (grouped by source)
• Cost transparency (per-search expense breakdown)
• Raw and unique result counts
• Search duration

Troubleshooting