Software Engineer, Backend (Data Engineering /Growth Platform) - Merpay
Mercari, inc., Japan

This position requires English language ability. JD is available in English only. JD

Introduction

"What can I do to help society thrive with the finite resources we have?" The Mercari marketplace app was born in 2013 out of this thought by our founder Shintaro Yamada as he traveled the world. We believe that by circulating all forms of value, not just physical things and money, we can create opportunities for anyone to realize their dreams and contribute to society and the people around them. Mercari aims to use technology to connect people all over the world and create a world where anyone can unleash their potential.For more information about Mercari Group’s mission, see Mercari's Culture Doc.

Here at Mercari, we work to realize a world in which no one’s potential is limited by their background and everyone has the opportunity to freely create value. We also firmly believe that a mindset of Diversity amp; Inclusion is essential for us to achieve our mission.

This, of course, extends to our hiring practices as well. Mercari is committed to eliminating discrimination based on age, gender, sexual orientation, race, religion, physical disability, and other such factors so that anyone who shares our mission and values can join us, regardless of their background. For more details, please read our Damp;I Statement.

Team mission:

• Develop and operate marketing tools, coupons, point rewarding systems, etc. used for the growth of not limited to Merpay but for the entire Mercari Group in the Japan Region (Mercari, Mercoin, Shops, etc.).
• In line with our group mission to "Circulate all forms of value to unleash the potential in all people", we aim to provide a platform for Growth teams to launch any type of campaign or growth experiment in the shortest possible time with maximum flexibility, seamlessly integrating our businesses, collaborating with other engineering teams, Data Analysts, and Machine Learning experts to implement smart solutions to large-scale problems

(Learn more about Mercari's mission and values here.)

As a Backend engineer with a focus on Data Engineering, specific work responsibilities in the Marketing amp; Growth Domain include the following:

• Design, develop, and operate data infrastructure/data pipelines to support large-scale campaigns; both in streaming-based and for large batches. This work includes the development of an ecosystem enabling data collection, Data Lake, DWH, use of collected data, and other processes required by data utilization
• Integrating internal and external systems. For example, in-house microservices and external marketing tools
• Interact with Data Platform engineers to apply and enhance reusable solutions
• Provide solutions for Data Analysts and ML experts for efficiently handling data related to the Marketing domain, including billions of data points per day generated by millions of active users
• QA automation for quality improvement
• Assist with operational tasks for systems, services, and campaigns

Work Responsibilities

• Collecting data from microservices
• Developing batch-based and streaming-based data pipelines
• Constructing Data Lakes and DWH for storing collected data
• Managing partial permissions and resources on DWH
• Manage data utilization and implement cost-reduction measures
• API/interface design for microservices using Protocol Buffers and gRPC
• Improving services and measuring effectiveness using quantitative and qualitative analysis on user behavior logs, etc.
• Design architecture and select middleware based on application requirements
• Microservice performance tuning, monitoring, and alert handling

Tools Used/Dev Environment

• Programming languages: Go, Python, Java, Scala
• Infrastructure: Google Cloud Platform (Compute Engine, Kubernetes Engine, Dataflow, Pub/Sub, Cloud Storage, BigQuery, Spanner, Cloud SQL, Cloud Logging etc.)
• Distributed processing: Apache Flink / Apache Spark
• Workflow: Apache Airflow
• BI tools: Looker / Data Studio
• Configuration management tools: Terraform
• Monitoring: Stackdriver Monitoring / Datadog / PagerDuty
• CI/CD: Circle CI / Spinnaker
• Knowledge Tool: Crowi / Confluence
• Other: GitHub / Slack
• Development Method: Agile (Scrum, Kanban, etc.)

• Growth Platform
  • With campaigns distributing millions of notifications and rewards per day, it is necessary to process billions of data points from many different services (internal and external)
  • Implement product features

Job Rewards and Benefits