# The IoT World — Platform Architecture Blueprint

> Complete 4-layer technical architecture breakdown of The IoT World enterprise AI platform, covering Ingestion, RAG, Fine-Tuning, and Inference Serving.

## Architecture Layers

### Layer 01: Ingestion Layer
- **Tech Stack**: Apache Kafka, Apache Flink, gRPC, MQTT, AWS Kinesis, Schema Registry
- **Pipeline Topology**:
  - `IoT Devices / Sensors` → `MQTT / gRPC Gateway` → `Kafka (Tenant-Partitioned)` → `Flink Processor` → `Schema Registry + Validator` → `Tenant Router → Downstream`
- **Key Specifications**:
  - Kafka topics partitioned by `tenant_id` for strict multi-tenant isolation.
  - Flink stateful stream processing for deduplication, windowed aggregation, and enrichment.
  - Sub-20ms p95 ingest latency with Snappy/Zstandard compression.
  - 5M+ daily device log capacity with horizontal partition scaling.
  - Schema Registry enforcing Avro/Protobuf contracts per tenant.

### Layer 02: RAG & Document Intelligence
- **Tech Stack**: LangChain, pgvector, ChromaDB, FastAPI, SentenceTransformers, Cross-Encoder Reranker
- **Pipeline Topology**:
  - `Raw Documents / Clinical Records` → `PHI-Aware Chunker` → `Dense Embeddings (SentenceTransformers)` → `pgvector / ChromaDB Index` → `Hybrid Retrieval (BM25 + Dense)` → `LLM Reranker → Response`
- **Key Specifications**:
  - Hybrid dense (pgvector HNSW) + sparse (BM25) retrieval for high clinical accuracy.
  - PHI-aware chunking preventing patient data leakage across tenants.
  - Cross-encoder reranking improving answer relevance by ~30%.
  - Calibration document workflows with structured metadata extraction.
  - <200ms end-to-end retrieval latency at p95.

### Layer 03: LLM Fine-Tuning Pipeline
- **Tech Stack**: PyTorch, HuggingFace PEFT, LoRA, QLoRA, Weights & Biases, MLflow, FSDP
- **Pipeline Topology**:
  - `Domain Dataset (labeled)` → `4-bit Quantization (QLoRA)` → `LoRA Adapter Training` → `FSDP Distributed Training` → `W&B Experiment Tracking` → `Fine-tuned Checkpoint → Registry`
- **Key Specifications**:
  - LoRA rank-16 adapters adding <0.1% parameters vs. full fine-tuning.
  - QLoRA 4-bit NF4 quantization reducing GPU VRAM requirements by 60–70%.
  - FSDP sharding across 8–32 GPUs for scalable distributed training.
  - +15–25% improvement in domain classification accuracy on holdout benchmarks.
  - W&B and MLflow for experiment comparison, metrics, and model lineage tracking.

### Layer 04: Secure Inference Architecture
- **Tech Stack**: vLLM, Ray Serve, Kubernetes, Istio, OPA, HashiCorp Vault, KEDA
- **Pipeline Topology**:
  - `Tenant API Request` → `Istio Gateway + mTLS` → `OPA Policy Enforcement` → `PHI NER Redaction Layer` → `Ray Serve Router → vLLM Engine` → `Namespace-Isolated Response`
- **Key Specifications**:
  - Kubernetes namespace-per-tenant for hard compute and memory isolation.
  - Istio service mesh with mTLS for all inter-service communication.
  - OPA admission controller enforcing tenant authorization and RBAC.
  - vLLM PagedAttention + continuous batching maximizing GPU utilization.
  - KEDA scales GPU worker pods to zero on idle, reducing cloud spend by 25–35%.
  - Vault-managed inference secrets and tenant API keys.

## Machine-Readable Resources
- [Full Documentation](https://www.theiotworld.io/llms-full.txt)
- [LLMs Index](https://www.theiotworld.io/llms.txt)
- [Homepage](https://www.theiotworld.io/)