Principal Data Processing Engineer-OSS

{
  "content_hash": "87393b679c31a8c360fe779bbda362fa87b6e9084a3be36a487b36126814df6a",
  "source_hash": "2baaa523da2a9b40c514efa590b0ef8961987ec69ff20feb4857b9940939f30e",
  "last_changed_at": "2026-05-30T06:11:22.133Z",
  "active_status": "active"
}

{
  "language": "en",
  "location": {
    "raw": "Mountain View, California, 94041, United States",
    "city": "Mountain View",
    "region": null,
    "country": "United States",
    "is_remote": true,
    "confidence": 0.95
  },
  "salary_max": null,
  "salary_min": null,
  "inferred_at": "2026-06-06T10:26:05.936Z",
  "launch_scope": {
    "reason": "bamboohr_production_catalog",
    "included": true,
    "location": {
      "raw": "Mountain View, California, 94041, United States",
      "city": "Mountain View",
      "region": null,
      "country": "United States",
      "is_remote": true,
      "confidence": 0.95
    },
    "countries": [
      "United States"
    ]
  },
  "remote_policy": "remote",
  "salary_period": null,
  "workplace_type": "remote",
  "salary_currency": null
}

{}

{
  "list_job": {
    "id": "26",
    "isRemote": null,
    "location": {
      "city": "Mountain View",
      "state": "California"
    },
    "atsLocation": {
      "city": null,
      "state": null,
      "country": null,
      "province": null
    },
    "departmentId": null,
    "locationType": "0",
    "jobOpeningName": "Principal Data Processing Engineer-OSS",
    "departmentLabel": null,
    "employmentStatusLabel": "Full-Time"
  },
  "detail_errors": [],
  "detail_job_opening": {
    "location": {
      "city": "Mountain View",
      "state": "California",
      "postalCode": "94041",
      "addressCountry": "United States"
    },
    "datePosted": "2024-09-25",
    "atsLocation": {
      "city": null,
      "state": null,
      "country": null,
      "countryId": null
    },
    "description": "<p><span style=\"font-family: arial, helvetica, sans-serif; font-weight: bold\">Principal Data Processing Engineer - OSS</span><br><span style=\"font-family: arial, helvetica, sans-serif\">Mountain View, CA </span></p>\n<p><br></p>\n<p><span style=\"color: rgb(72, 65, 63); font-family: Arial, sans-serif; font-size: 10pt; font-weight: bold\">About DataPelago:</span></p>\n<p><span style=\"color: rgb(72, 65, 63); font-family: Arial, sans-serif; font-size: 10pt\">DataPelago is at the forefront of revolutionizing data processing for traditional analytics and cutting-edge GenAI preprocessing. We are building an innovative data processing engine that is transforming how Apache Spark, Apache Flink, Ray, and others operate on diverse, large-scale data. Our team of engineers drive and adopt advances in hardware-accelerated computing, parallel processing of large-scale data, query optimization, distributed systems, compilers, machine learning, and cloud-native computing. We are looking for world-class engineers to join our team and shape the future of accelerated data processing.</span><span style=\"color: rgb(72, 65, 63); font-family: Arial, sans-serif; font-size: 10pt\"><br><br></span></p>\n<p><span style=\"color: rgb(72, 65, 63); font-family: Arial, sans-serif; font-size: 10pt; font-weight: bold\">The Role:</span></p>\n<p><span style=\"color: rgb(72, 65, 63); font-family: Arial, sans-serif; font-size: 10pt\">As a Principal Data Processing Engineer (OSS), you will be a key individual contributor in</span><span style=\"color: rgb(72, 65, 63); font-family: Arial, sans-serif; font-size: 10pt\"><br></span><span style=\"color: rgb(72, 65, 63); font-family: Arial, sans-serif; font-size: 10pt\">adopting and advancing the capabilities of open-source software (OSS) platforms such as Apache</span></p>\n<p><span style=\"color: rgb(72, 65, 63); font-family: Arial, sans-serif; font-size: 10pt\">Gluten, Velox, Apache Spark, and Apache Flink in the context of DataPelago’s data processing engine. You will enhance the functional breadth, performance, scale, and reliability of the DataPelago engine through downstream and upstream contributions. </span><span style=\"color: rgb(72, 65, 63); font-family: Arial, sans-serif; font-size: 10pt\">You will have the opportunity to engage with the community working on these platforms.</span><span style=\"color: rgb(72, 65, 63); font-family: Arial, sans-serif; font-size: 10pt\"> This is a unique opportunity to make a significant impact on a category-defining product and work with a talented team of engineers.</span><span style=\"color: rgb(72, 65, 63); font-family: Arial, sans-serif; font-size: 10pt\"><br><br></span></p>\n<p><span style=\"color: rgb(72, 65, 63); font-family: Arial, sans-serif; font-size: 10pt; font-weight: bold\">What You'll Do:</span></p>\n<ul>\n<li><span style=\"color: rgb(72, 65, 63); font-family: Arial, sans-serif; font-size: 10pt\">Influence the architecture of how our data processing engine interfaces with open-source platforms and engines.</span></li>\n<li><span style=\"color: rgb(72, 65, 63); font-family: Arial, sans-serif; font-size: 10pt\">Lead the design of functional and performance enhancements to open source platforms such as Apache Gluten and Velox, and their integration with our data processing engine.</span></li>\n<li><span style=\"color: rgb(72, 65, 63); font-family: Arial, sans-serif; font-size: 10pt\">Individually design, implement, test, optimize, and maintain </span><span style=\"color: rgb(72, 65, 63); font-family: Arial, sans-serif; font-size: 10pt\">components</span><span style=\"color: rgb(72, 65, 63); font-family: Arial, sans-serif; font-size: 10pt\"> of the data processing engine.</span></li>\n<li><span style=\"color: rgb(72, 65, 63); font-family: Arial, sans-serif; font-size: 10pt\">Analyze the technology roadmap of Apache Gluten, Velox, and equivalent platforms and identify opportunities for our engine to enhance technology and product leadership.</span></li>\n<li><span style=\"color: rgb(72, 65, 63); font-family: Arial, sans-serif; font-size: 10pt\">Collaboration: Partner with engineering, product management, the open-source community and customer success teams.</span></li>\n<li><span style=\"color: rgb(72, 65, 63); font-family: Arial, sans-serif; font-size: 10pt\">Foster best practices in design and code reviews, testing, CI/CD, and issue resolution to maintain the highest product quality, security, efficiency, and productivity.</span><span style=\"color: rgb(72, 65, 63); font-family: Arial, sans-serif; font-size: 10pt\"><br><br></span></li>\n</ul>\n<p><span style=\"color: rgb(72, 65, 63); font-family: Arial, sans-serif; font-size: 10pt; font-weight: bold\">What You'll Bring:</span></p>\n<ul>\n<li><span style=\"color: rgb(72, 65, 63); font-family: Arial, sans-serif; font-size: 10pt\">BS/MS in  Computer Science (or a related field) with 6+ years of relevant experience </span></li>\n<li><span style=\"color: rgb(72, 65, 63); font-family: Arial, sans-serif; font-size: 10pt\">3+ years of deep technical experience in instrumenting, analyzing, and optimizing the </span><span style=\"color: rgb(72, 65, 63); font-family: Arial, sans-serif; font-size: 10pt\">performance</span><span style=\"color: rgb(72, 65, 63); font-family: Arial, sans-serif; font-size: 10pt\"> of data processing engine components on benchmark and customer workloads.</span></li>\n<li><span style=\"color: rgb(72, 65, 63); font-family: Arial, sans-serif; font-size: 10pt\">Sound knowledge of the architecture and internal operation of one or more of Apache Spark,</span><span style=\"color: rgb(72, 65, 63); font-family: Arial, sans-serif; font-size: 10pt\"><br></span><span style=\"color: rgb(72, 65, 63); font-family: Arial, sans-serif; font-size: 10pt\">Apache Flink, Presto/Trino.</span></li>\n<li><span style=\"color: rgb(72, 65, 63); font-family: Arial, sans-serif; font-size: 10pt\">Demonstrated experience in the design, development, and successful release of </span><span style=\"color: rgb(72, 65, 63); font-family: Arial, sans-serif; font-size: 10pt\">high-performance</span><span style=\"color: rgb(72, 65, 63); font-family: Arial, sans-serif; font-size: 10pt\"> data processing engines for large production deployments.</span></li>\n<li><span style=\"color: rgb(72, 65, 63); font-family: Arial, sans-serif; font-size: 10pt\">Exceptional programming skills in C, C++, and Java.</span></li>\n<li><span style=\"color: rgb(72, 65, 63); font-family: Arial, sans-serif; font-size: 10pt\">Extensive development experience in Linux environments.</span></li>\n<li><span style=\"color: rgb(72, 65, 63); font-family: Arial, sans-serif; font-size: 10pt\">Excellent communication and collaboration skills, with the ability to articulate complex technical concepts to both technical and non-technical audiences.</span></li>\n<li><span style=\"color: rgb(72, 65, 63); font-family: Arial, sans-serif; font-size: 10pt\">Strong analytical and problem-solving skills with a passion for performance optimization.</span><span style=\"color: rgb(72, 65, 63); font-family: Arial, sans-serif; font-size: 10pt\"><br><br></span></li>\n</ul>\n<p><span style=\"color: rgb(72, 65, 63); font-family: Arial, sans-serif; font-size: 10pt; font-weight: bold\">Location Considerations:</span></p>\n<p><span style=\"color: rgb(72, 65, 63); font-family: Arial, sans-serif; font-size: 10pt\">We value face-to-face collaboration, but recognize that talent can be found anywhere. Our engineering team works at our headquarters in Mountain View, CA, at our India office in Hyderabad, and at remote locations.</span><span style=\"color: rgb(72, 65, 63); font-family: Arial, sans-serif; font-size: 10pt\"><br><br></span></p>\n<p><span style=\"color: rgb(72, 65, 63); font-family: Arial, sans-serif; font-size: 10pt; font-weight: bold\">Why Join DataPelago?</span></p>\n<ul>\n<li><span style=\"color: rgb(72, 65, 63); font-family: Arial, sans-serif; font-size: 10pt\">Technical Leadership: Take a leadership role in shaping the architecture and development of how our core engine works with open source data processing platforms</span></li>\n<li><span style=\"color: rgb(72, 65, 63); font-family: Arial, sans-serif; font-size: 10pt\">Cutting-Edge Innovation: Work on challenging problems at the forefront of accelerated</span><span style=\"color: rgb(72, 65, 63); font-family: Arial, sans-serif; font-size: 10pt\"><br></span><span style=\"color: rgb(72, 65, 63); font-family: Arial, sans-serif; font-size: 10pt\">computing and data processing.</span></li>\n<li><span style=\"color: rgb(72, 65, 63); font-family: Arial, sans-serif; font-size: 10pt\">Significant Impact: Your contributions will directly impact the performance and scalability of our mission-critical platform.</span></li>\n<li><span style=\"color: rgb(72, 65, 63); font-family: Arial, sans-serif; font-size: 10pt\">Mentorship and Growth: Mentor and guide other talented engineers while expanding your own technical expertise.</span></li>\n<li><span style=\"color: rgb(72, 65, 63); font-family: Arial, sans-serif; font-size: 10pt\">Competitive compensation, stock options, comprehensive benefits package, and leadership </span><span style=\"color: rgb(72, 65, 63); font-family: Arial, sans-serif; font-size: 10pt\">development</span><span style=\"color: rgb(72, 65, 63); font-family: Arial, sans-serif; font-size: 10pt\"> opportunities</span></li>\n</ul>",
    "compensation": null,
    "departmentId": null,
    "locationType": "0",
    "seekPromoted": false,
    "jobCategoryId": null,
    "jobOpeningName": "Principal Data Processing Engineer-OSS",
    "departmentLabel": "",
    "jobOpeningStatus": "Open",
    "minimumExperience": "Experienced",
    "jobOpeningShareUrl": "https://datapelago.bamboohr.com/careers/26",
    "employmentStatusLabel": "Full-Time"
  }
}