Software Engineer (Data Platform) - Merpay
Mercari, inc., Japan

JD in Japanese follows. JD

Circulate all forms of value to unleash the potential in all people

"What can I do to help society thrive with the finite resources we have?" The Mercari marketplace app was born in 2013 out of this thought by our founder Shintaro Yamada as he traveled the world. We believe that by circulating all forms of value, not just physical things and money, we can create opportunities for anyone to realize their dreams and contribute to society and the people around them. Mercari aims to use technology to connect people all over the world and create a world where anyone can unleash their potential.For more information about Mercari Group’s mission, see Mercari's Culture Doc.

Equal Opportunity Hiring

Here at Mercari, we work to realize a world in which no one’s potential is limited by their background and everyone has the opportunity to freely create value. We also firmly believe that a mindset of Diversity amp; Inclusion is essential for us to achieve our mission.

This, of course, extends to our hiring practices as well. Mercari is committed to eliminating discrimination based on age, gender, sexual orientation, race, religion, physical disability, and other such factors so that anyone who shares our mission and values can join us, regardless of their background. For more details, please read our Damp;I Statement.

• Mercari Group utilizes data for business decisions and other processes in a wide variety of areas, including marketing, machine learning, and Ramp;D. As part of Merpay’s Data Platform Team, you will design, develop and operate the data infrastructure/data pipelines which support data utilization for the entire Mercari Group, including not only Merpay, but the Mercari marketplace app. You will also be responsible for work on the Mercari Group’s data utilization as a whole.

• The Data Platform Team is an engineering organization which builds the systems necessary to a wide range of domains that utilize data. This work includes development of an ecosystem enabling data collection, Data Lake, DWH, use of collected data and other processes required by data utilization. The following describes the responsibilities of a software engineer on the Data Platform Team, as well as the environment and organization in which they work:

• Job Description
  • Collecting data from microservices
    • Developing batch-based data pipelines
    • Developing streaming-based data pipelines
    • Developing SDK for logging within microservices
  • Data Lake, DWH
    • Constructing Data Lakes for storing collected data
    • Managing partial permissions and resources on DWH
  • Data utilization
    • Developing tools to support use of data in Data Lakes and on DWH
    • Developing a platform to provide stream data processing function
  • Company-wide
    • Improving performance, aiming for low latency/high throughput according to application and middleware development, operation, and requirements
    • Developing tools to automate operations and/or lower costs
    • Identifying and solving technical issues with the system caused by engineering
• Tools Used/Dev Environment
  • Programming languages: Python, Java, Scala
  • Infrastructure: Google Cloud Platform (Compute Engine, Kubernetes Engine, Dataflow, Pub/Sub, Cloud Storage, BigQuery, Spanner, Cloud SQL, Cloud Logging etc.)
  • Distributed processing: Apache Flink / Apache Spark
  • Workflow: Apache Airflow
  • BI tools: Looker / Data Studio
  • Configuration management tools: Terraform
  • Monitoring: Stackdriver Monitoring / Datadog / PagerDuty
  • CI/CD: Circle CI / Spinnaker
  • Knowledge Tool: Crowi / Confluence
  • Other: GitHub / Slack
  • Development Method: Agile (Scrum, Kanban, etc.)
• Dev Organizations
  • As part of the Data amp; Machine Learning Team, you will work on development alongside members with a broad range of expertise from the following organizations.
  • Teams
    • Data amp; Machine Learning Platform
    • Data Management
    • Machine Learning (Credit)
    • Machine Learning (Anti-Fraud)

• Opportunity to build large-scale data infrastructure to support the massive amounts of data generated by Mercrai Group, including the Mercari Marketplace app with its over 20M monthly users.
• Work on development of an ecosystem to support data-driven business expansion by addressing the data utilization needs of many domains including analytics/decision-making, marketing, machine learning, and Ramp;D.
• Take on the entire data platform creation process, from design to developmen

Job Rewards and Benefits