Chief Technology Officer

About Skarnode

Skarnode is building the world's first platform to quantify narrative-driven risk. We transform millions of daily news articles into real-time volatility indexes that help banks, insurers, and corporations anticipate geopolitical, regulatory, and reputational shocks before they hit markets and supply chains.

We are looking for a senior technical leader
to design, build, and scale our data ingestion and real-time processing infrastructure. This is a high-impact role at the heart of our mission to become the benchmark for narrative volatility worldwide.

Technical Requirements

Event-Driven Architecture

• Expertise in
  Apache Kafka
  (or equivalent) and
  Apache Flink
  for high-performance streaming and distributed real-time processing.
• Must handle
  millions of events per day
  with fault tolerance and horizontal scalability.

Web Scraping & Data Ingestion

• Large-scale distributed scraping using
  headless browsers
  (Playwright, Puppeteer).
• Anti-bot strategies including
  proxy pools, IP rotation, fingerprinting, and quota management
  .
• High-performance orchestration and parallelization of scraping tasks.

Backend Engineering

• Expertise in
  Python
  ,
  Django Rest Framework
  ,
  Go
  ,
  Rust
  ,
  C++
  .
• Experience designing
  REST APIs
  ,
  GraphQL
  ,
  WebSockets
  , and microservices at scale.

Frontend

• Proficiency in
  JavaScript (ES6+)
  ,
  
  ,
  React
  , or equivalent frameworks.
• Building
  real-time dashboards
  and interactive client interfaces.

Databases & Storage

• Strong experience with
  PostgreSQL
  ,
  Redis/Valkey
  , and
  S3-compatible object storage
  .
• Managing massive data pipelines and daily snapshots.

Cloud & Infrastructure

• Docker
  and
  Kubernetes
  for deployment and scaling.
• Hybrid on-premise/cloud environments, multi-region architecture, and failover strategies.

MLOps & NLP

• Experience with
  PyTorch
  ,
  multilingual embeddings
  ,
  dynamic clustering
  , and
  Vector Similarity Search (VSS)
  .
• Building production ML pipelines with full lifecycle management.

Responsibilities

Real-Time Data Pipeline

• Design and deploy distributed web scraping systems using Playwright and Puppeteer.
• Reverse-engineer complex APIs via HAR file analysis.
• Orchestrate data streams with
  Kafka + Flink + Redis
  .
• Implement retry logic, failure handling, and large-scale resilience.

Real-Time Skarnode Metrics

• Implement mathematical and statistical models to calculate volatility indexes in real time.
• Experience in financial mathematics is a strong plus.

Client Delivery Layer

• Build secure
  REST APIs
  ,
  GraphQL
  , and
  WebSockets
  for real-time client access.
• Manage enterprise-grade authentication and permissions.

Infrastructure & Reliability

• Oversee
  CI/CD pipelines
  , monitoring, and observability tools.
• Build internal dashboards to track pipeline performance and detect anomalies.

Team Leadership

• Recruit and mentor a high-performing team across:
• Backend & frontend engineering
• Data engineering & streaming
• Machine learning & NLP
• Web scraping & parsing
• Define technical standards, enforce code reviews, and set best practices.

Key Objectives

1. Build a
   fully automated, resilient ingestion system
   capable of monitoring hundreds of sources in parallel.
2. Ensure
   real-time processing scalability
   using Kafka, Flink, and Redis.
3. Guarantee
   data quality, security, and compliance
   with GDPR and international standards.
4. Develop
   internal tools
   for scraping management, proxy monitoring, error logs, and bot detection.
5. Deploy advanced ML models to
   produce Skarnode's volatility indexes
   in production.

Profile

• Proven expertise in
  data streaming, advanced web scraping, and real-time processing
  .
• Deep knowledge of
  distributed architectures and messaging systems
  (Kafka, Flink).
• Strong leadership skills and experience
  building and scaling teams
  .
• Fluent
  English
  required;
  Chinese or Russian
  are a major plus.