Apply on Employer Site

WIS International · 9 hours ago

Software Engineer Streaming

Plano, TX

Full-time

Hybrid

Mid Level

$120K/yr - $150K/yr

WIS International is seeking a Streaming Platform Engineer to design, implement, and operate high-throughput, low-latency event streaming systems. The role involves managing streaming clusters, developing real-time pipelines, and ensuring data integration and optimization across various databases.

Data Collection and LabelingRetail

Responsibilities

Design and manage streaming clusters (Kafka, Pulsar, Event Hubs, Kinesis) across cloud environments

Configure topics, partitions, retention, replication, and geo-redundancy

Implement schema management (Schema Registry, Avro, Protobuf, JSON)

Ensure high availability and disaster recovery

Build fault-tolerant producers/consumers in Java, Scala, Python, or Go

Develop real-time pipelines using Kafka Streams, Pulsar Functions, Flink, Spark Streaming, or Kinesis Analytics

Guarantee reliable delivery and ordered processing

Deploy CDC pipelines (Debezium, Maxwell, MongoDB Change Streams)

Sync data to warehouses (Snowflake, BigQuery, Redshift) and operational DBs (PostgreSQL, Cassandra, DynamoDB)

Design schemas, indexes, and partitioning in RDBMS (AzureSQL, PostgreSQL, MySQL) and NoSQL (Cassandra, DynamoDB, MongoDB) for high-velocity writes

Optimize query performance for event-sourced or streaming-derived data

Manage data consistency between streams and persistent stores (event sourcing, CQRS patterns)

Implement monitoring (Prometheus, Grafana, Datadog)

Set up alerts, schema checks, and dead-letter queues

Automate deployments with Terraform, Helm, Kubernetes Operators

Write integration and resilience tests

Qualification

Apache KafkaApache PulsarGoKafka StreamsAvroDebeziumAzureSQLAWSDockerKubernetesTerraformCI/CDFlinkSpark StreamingJSON SchemaSchema RegistryCassandraDynamoDBMongoDBPostgreSQLMySQL