Apply on Employer Site

Shulman Fleming & Partners · 1 month ago

Data Platform Engineer

New York City, NY, United States

Full-time

Hybrid

Mid, Senior Level

$0/yr - $150K/yr

Shulman Fleming & Partners is seeking a highly skilled Data Platform Engineer to design, build, and operate their enterprise event-streaming platform using Red Hat AMQ Streams (Kafka on OpenShift). The role involves ensuring a reliable and scalable streaming ecosystem while collaborating with application teams to implement event-driven integration patterns.

Staffing & Recruiting

No H1B

Responsibilities

Design, deploy, and operate AMQ Streams (Kafka) clusters on Red Hat OpenShift

Configure and manage Kafka components including brokers, Kraft, MirrorMaker 2

Explore Kafka Connect, and Schema Registry concepts and implementations

Ensure performance, reliability, scalability, and high availability of the Kafka platform

Implement cluster monitoring, logging, and alerting using enterprise observability tools

Manage capacity planning, partition strategies, retention policies, and performance tuning

Define and document standardized event-driven integration patterns, including:

Event sourcing

CQRS

Pub/sub messaging

Change data capture

Stream processing & enrichment

Request-reply over Kafka

Guide application teams on using appropriate patterns that align with enterprise architecture

Establish best practices for schema design, topic governance, data contracts, and message lifecycle management

Implement enterprise-grade security for Kafka, including RBAC, TLS, ACLs, and authentication/authorization integration. (SSO and OAuth)

Maintain governance for topic creation, schema evolution, retention policies, and naming standards

Ensure adherence to compliance, auditing, and data protection requirements (Encryption at Rest and flight)

Provide platform guidance and troubleshooting expertise to development and integration teams

Partner with architects, SREs, and developers to drive adoption of event-driven architecture

Create documentation, runbooks, and internal knowledge-sharing materials

Develop CI/CD pipelines in GitLab, enabling automated builds, infrastructure updates, and configuration promotion across environments

Maintain Infrastructure-as-Code (IaC) repositories and templates for Kafka resources

Qualification

Kafka administrationOpenShiftCI/CD pipelinesPythonDevOps practicesProblem-solvingCommunicationCollaboration

Required

Bachelor's degree in computer science, Engineering, or a related field

Proven experience with Kafka administration and management

Strong knowledge of OpenShift and container orchestration

Proficiency in scripting languages such as Python or Bash

Experience with monitoring and logging tools (e.g., Splunk, Prometheus, Grafana)

Excellent problem-solving skills and attention to detail

Strong communication and collaboration skills

Preferred

Experience with Red Hat OpenShift administration

Knowledge of service mesh patterns (Istio, OpenShift Service Mesh)

Familiarity with stream processing frameworks (Kafka Streams, ksqlDB, Flink)

Experience using observability stacks (Prometheus, Grafana)

Background working in regulated or enterprise-scale environments

Knowledge of DevOps practices and tools (e.g., ArgoCD, Ansible, Terraform)

Knowledge of SRE Monitoring and logging tools (e.g., Splunk, Prometheus, Grafana)

Company

Shulman Fleming & Partners

SF&P is a recruitment firm specializing as a creative search resource for early stage and growing tech companies.

Founded in 2001

NYC, NY, US

11-50 employees

http://www.sfp8.com

Funding

Current Stage

Early Stage

Leadership Team

Norm Shulman

CEO

Company data provided by crunchbase