About the Job

At Xendit, we seek a Senior/Staff Data engineer who will bring technical expertise to help us build a reliable and scalable Data platform to empower internal teams to work independently and unlock full potential of our data. You’ll bring a diverse set of skills across data engineering including data pipeline design at scale (batch/real-time), data infrastructure, data governance, data quality testing, development & deployment best practices, and more. You will be part of the Data engineering core team, working among 8 other talented data engineers. With your relentless drive for successful outcomes, you will work on the most complex projects within the core team as well as contribute to highly impactful cross-team initiatives. You can expect to own a technical roadmap ensuring the tools and services that the team provides are reliable, scalable, easy-to-use, and cost-efficient. You possess a strong commitment to going the extra mile and delivering projects of high quality on time. You enjoy working with others and are able to level up other engineers. Your contributions will help stakeholders to generate value from data and indirectly drive revenues.

Minimum Qualifications

• 5+ years of relevant experience as a data engineer.
• 2+ years of relevant experience as senior/staff/principal data engineer.
• Excellent knowledge of Python, SQL, and Apache Spark.
• Excellent knowledge of at least 1 real-time data processing framework such as Spark Streaming, Flink, or Kafka.
• Demonstrated ability to design and build high-volume batch and streaming pipelines.
• Demonstrated ability to design and build scalable data infrastructure.
• Working experience with designing and implementing data quality checks and alerting systems.
• Working experience in optimizing SQL queries in OLAP cluster (e.g. data partitioning, bucketing, z-ordering, indexing).
• You are coachable. Able to own mistakes, reflect, and take feedback with maturity and a willingness to improve.
• Strong written and verbal communication skills.
• Bachelor's degree in a technical field or equivalent work experience.

Preferred Qualifications

• Working experience in developing a Python library used by internal teams. Including best practices for development and deployment.
• You have a good knowledge of Terraform, CloudFormation, or other IaaC tools.
• You have built data products that have scaled on AWS or another cloud.
• You have experience with modern data tools such as Airflow, dbt, Kafka, Trino, Databricks, Looker, or similar.
• You have experience with different databases or SQL engines and understand their trade-offs (e.g. Trino, Druid, PostgreSQL, MongoDB, etc.).
• You have worked with sensitive data and ensured secure access through a data governance tool.
• You have worked on building a Data platform that enables stakeholders to self-serve.

Responsibilities

• Improve data pipeline logic to ensure reliability, scalability, and cost efficiency (Python, Spark, Airflow).
• Ensure fast and reliable execution of analytical queries by building robust OLAP cluster (Trino, Terraform, Databricks SQL warehouse).
• Design and implement a data governance policy to minimize security risks (Unity Catalog, Apache Ranger).
• Own design and development of Data engineering’s internal library (Python, Spark, dbt).
• Enable internal teams to build real-time applications on top of the Data lakehouse. (Spark streaming, Kafka)
• Automate common data requests and unlock self-service (Retool, Flask).
• Ensure high data quality through automated tests and data contracts (dbt, Great expectations).
• Improve deployment process for various applications (Buddy).
• Collaborate with other data engineers, analysts, and business users to design effective solutions.
• Guide junior engineers and set engineering standards for the team.
• Minimize detection and recovery time from incidents and ensure the team meets important metrics and SLOs.
• Research innovative technologies and integrate them into our data infrastructure.
• Do whatever it takes to make Xendit succeed.