Chief Technology Officer

Skarnode is building the world's first platform to quantify narrative-driven risk. We transform millions of daily news articles into real-time volatility indexes that help banks, insurers, and corporations anticipate geopolitical, regulatory, and reputational shocks before they hit markets and supply chains.

We are looking for a senior technical leader to design, build, and scale our data ingestion and real-time processing infrastructure. This is a high-impact role at the heart of our mission to become the benchmark for narrative volatility worldwide.

About Skarnode

Skarnode is building the world's first platform to quantify narrative-driven risk. We transform millions of daily news articles into real-time volatility indexes that help banks, insurers, and corporations anticipate geopolitical, regulatory, and reputational shocks before they hit markets and supply chains.

We are looking for a senior technical leader to design, build, and scale our data ingestion and real-time processing infrastructure. This is a high-impact role at the heart of our mission to become the benchmark for narrative volatility worldwide.

Technical Requirements

Event-Driven Architecture

• Expertise in Apache Kafka (or equivalent) and Apache Flink for high-performance streaming and distributed real-time processing.
• Must handle millions of events per day with fault tolerance and horizontal scalability.

Web Scraping & Data Ingestion

• Large-scale distributed scraping using headless browsers (Playwright, Puppeteer).
• Anti-bot strategies including proxy pools, IP rotation, fingerprinting, and quota management.
• High-performance orchestration and parallelization of scraping tasks.

Backend Engineering

• Expertise in Python, Django Rest Framework, Go, Rust, C++.
• Experience designing REST APIs, GraphQL, WebSockets, and microservices at scale.

Frontend

• Proficiency in JavaScript (ES6+), Vue.js, React, or equivalent frameworks.
• Building real-time dashboards and interactive client interfaces.

Databases & Storage

• Strong experience with PostgreSQL, Redis/Valkey, and S3-compatible object storage.
• Managing massive data pipelines and daily snapshots.

Cloud & Infrastructure

• Docker and Kubernetes for deployment and scaling.
• Hybrid on-premise/cloud environments, multi-region architecture, and failover strategies.

MLOps & NLP

• Experience with PyTorch, multilingual embeddings, dynamic clustering, and Vector Similarity Search (VSS).
• Building production ML pipelines with full lifecycle management.

Responsibilities

Real-Time Data Pipeline

• Design and deploy distributed web scraping systems using Playwright and Puppeteer.
• Reverse-engineer complex APIs via HAR file analysis.
• Orchestrate data streams with Kafka + Flink + Redis.
• Implement retry logic, failure handling, and large-scale resilience.

Real-Time Skarnode Metrics

• Implement mathematical and statistical models to calculate volatility indexes in real time.
• Experience in financial mathematics is a strong plus.

Client Delivery Layer

• Build secure REST APIs, GraphQL, and WebSockets for real-time client access.
• Manage enterprise-grade authentication and permissions.

Infrastructure & Reliability

• Oversee CI/CD pipelines, monitoring, and observability tools.
• Build internal dashboards to track pipeline performance and detect anomalies.

Team Leadership

• Recruit and mentor a high-performing team across:
• Backend & frontend engineering
• Data engineering & streaming
• Machine learning & NLP
• Web scraping & parsing
• Define technical standards, enforce code reviews, and set best practices.

Key Objectives

• Build a fully automated, resilient ingestion system capable of monitoring hundreds of sources in parallel.
• Ensure real-time processing scalability using Kafka, Flink, and Redis.
• Guarantee data quality, security, and compliance with GDPR and international standards.
• Develop internal tools for scraping management, proxy monitoring, error logs, and bot detection.
• Deploy advanced ML models to produce Skarnode's volatility indexes in production.

Profile

• Proven expertise in data streaming, advanced web scraping, and real-time processing.
• Deep knowledge of distributed architectures and messaging systems (Kafka, Flink).
• Strong leadership skills and experience building and scaling teams.
• Fluent English required; Chinese or Russian are a major plus.

• Executive

• Full-time

• Information Technology

• Technology, Information and Media