Harbor Scale OSS

Harbor Scale OSS is the open-source ingestion and visualization stack behind Harbor Scale. Self-host your own telemetry backend with full control over your data and infrastructure.

Repo Link: https://github.com/harborscale/telemetry-harbor-oss

What's Included

• 🚀 API Ingestion Layer - Go Fiber-based REST API with Redis-backed queue
• ⚡ Background Worker - Efficient data processing and TimescaleDB insertion
• 📊 Grafana Integration - Pre-configured dashboards and datasource
• 🗄️ TimescaleDB - Optimized time-series database for telemetry data
• 🔄 Redis Queue - Reliable message queue management
• 🛠️ SDK Compatible - Works with all official Harbor Scale SDKs

🚀 Quick Start

Get your self-hosted Harbor Scale instance running in minutes:

git clone https://github.com/harborscale/telemetry-harbor-oss.git
cd telemetry-harbor-oss
docker compose up -d

Once started:

• API available at → http://localhost:8000/api/v2
• Grafana available at → http://localhost:3000 (default: admin / StrongAdminPassword!)

📋 System Requirements

• Docker with Docker Compose support
• Minimum 2GB RAM (4GB+ recommended for production)
• 10GB+ disk space (depends on data retention needs)

🔐 Security Configuration

⚠️ Security Notice This repository ships with default credentials for ease of testing. Before using in production, you must change the following in docker-compose.yml:

• POSTGRES_PASSWORD - TimescaleDB password
• REDIS_PASSWORD - Redis authentication password
• API_KEY - API authentication key for data ingestion
• GF_SECURITY_ADMIN_PASSWORD - Grafana admin password

Failure to do so will leave your system vulnerable.

📡 API Ingestion

We support Flexible Ingestion. You can send data in a strict format or a flat, user-friendly format. The system automatically "explodes" flat JSON into individual metrics.

1. Flexible Format (Recommended)

Send multiple metrics in a single flat JSON object.

POST /api/v2/ingest

curl -X POST "http://localhost:8000/api/v2/ingest" \
  -H "X-API-Key: your_api_key_here" \
  -H "Content-Type: application/json" \
  -d '{
    "ship_id": "server-01",
    "cpu_load": 45.2,
    "ram_usage": 1024,
    "temperature": 55.0
  }'

Result: Creates 3 separate metrics.

2. Strict Format (For SDK)

Useful if you need to map a specific metric ID dynamically.

POST /api/v2/ingest

curl -X POST "http://localhost:8000/api/v2/ingest" \
  -H "X-API-Key: your_api_key_here" \
  -H "Content-Type: application/json" \
  -d '{
    "ship_id": "server-01",
    "cargo_id": "cpu_load",
    "value": 45.2
  }'

3. Batch Ingestion

You can send an array containing mixed formats (Flexible and Strict).

POST /api/v2/ingest/batch

curl -X POST "http://localhost:8000/api/v2/ingest/batch" \
  -H "X-API-Key: your_api_key_here" \
  -H "Content-Type: application/json" \
  -d '[
    { "ship_id": "server-A", "cpu": 50, "temp": 60 },
    { "ship_id": "server-B", "cargo_id": "pressure", "value": 1013 }
  ]'

The Things Network (TTN) Webhook

Harbor Scale OSS supports direct ingestion from TTN v3 Webhooks. We automatically extract decoded_payload values, RSSI, SNR, and Frequency.

Configuration in TTN Console:

• Base URL: http://yourdomain.com/api/v2/ingest/ttn
• HTTP Method: POST
• Headers: X-API-Key: your_api_key_here

Note: Ensure your TTN payload formatter returns a flat JSON object with numeric values in decoded_payload.

📊 Visualization with Grafana

Grafana comes pre-configured with:

• Harbor Scale Datasource (TimescaleDB connection)
• Comprehensive Telemetry Dashboard
• Ready-to-use panels for time-series visualization

Log into Grafana at http://localhost:3000 and start exploring your telemetry data immediately.

🗄️ Data Retention

By default, your telemetry data is kept for 365 days. Want a different retention period? Just tweak it in init.sql before starting the stack.

🛠️ SDK Compatibility

Harbor Scale OSS is fully compatible with all official Harbor Scale SDKs:

• Harbor Scale SDKs

Just replace your ingest endpoint with your OSS URL - no code changes needed!

☁️ OSS vs Cloud Comparison

Feature	OSS Self-hosted	Harbor Scale Cloud
Pricing	Free	Free/Paid
Rate Limits	No rate limits (your hardware)	Rate limits based on plan
Data Retention	365 days (configurable)	Varies by plan
Storage	Based On Your Hardware	Unlimited
Grafana Access	Dedicated instance	Shared/dedicated
Infrastructure	Self-managed	Fully managed
Harbor AI	Not included	Available
Backup & Recovery	Not Available	All Plans
High Availability	Not Available	All Plans
Scalability	Manual scaling	Fully managed
Support	Community	Priority Email Support
Updates	Manual	Automatic
Security	Self-managed	Enterprise-grade

When to Choose OSS Self-hosted

✅ Choose OSS if you:

• Need full control over your data and infrastructure
• Want to customize the stack for specific needs
• Have technical expertise to manage infrastructure
• Require on-premises deployment for compliance

✅ Choose Cloud if you:

• Want managed infrastructure and automatic updates
• Need Harbor AI features for data insights
• Require enterprise-grade backup and high availability
• Prefer dedicated support and SLAs
• Want to focus on your application, not infrastructure
• Need guaranteed uptime and scalability

📜 License

This project is licensed under the Apache License 2.0.

• ✅ Free to use (commercial + personal)
• ✅ Free to modify and redistribute
• ⚠️ Must include attribution (keep copyright + NOTICE file)

See LICENSE for details.

If you use this project in your product, please credit Harbor Scale with a link to https://harborscale.com.

🤝 Contributing

We welcome issues, pull requests, and feature suggestions!

• Open a GitHub issue for bugs or feature requests
• Fork the repo and submit PRs for improvements
• Join our community discussions

Repo Link: https://github.com/harborscale/telemetry-harbor-oss