Agentic Data Engineer
类别
技能
职位描述
We are seeking a highly motivated and skilled Agentic Data Engineer to join our dynamic team. This role is crucial in shaping the future of our data ecosystem by ensuring our infrastructure, data models, and pipelines are designed for seamless integration and collaboration with AI agents. You will not only perform traditional data engineering tasks but also pioneering the development of agentic capabilities for data health, monitoring, and recovery.
Key Responsibilities
Agentic Data Infrastructure & Development
Agent-Centric Design: Design, build, and optimize data infrastructure (on GCP) that inherently supports agentic AI integration, focusing on data model design and pipeline architecture for machine-readable and agent-actionable data.
AI Agent Development: Develop and deploy specific AI agents (leveraging Google Gemini/GCP AI services) for critical data engineering tasks, including:
Auto-Recovery Agents: Creating agents capable of autonomously detecting, diagnosing, and resolving common data quality, pipeline, or integration issues.
Proactive Monitoring Agents: Building agents to continuously monitor the health, performance, and integrity of data pipelines and interfaces, providing proactive alerts and insights.
Interface Integration: Ensure all data interfaces and APIs are architected to facilitate smooth, reliable, and secure interaction with autonomous AI agents.
Core Data Engineering
Pipeline Development: Build, maintain, and scale robust, high-performance ETL/ELT data pipelines using GCP technologies (e.g., Cloud Composer/Airflow, Dataflow, BigQuery) to ingest, transform, and load data from diverse sources.
Data Modeling: Implement and enforce robust data governance standards and best practices, focusing on developing scalable, optimized, and agent-friendly data models within BigQuery.
Performance and Optimization: Monitor data infrastructure performance and optimize pipelines and queries for cost-efficiency and speed.
Performance Marketing Data Focus
Integration Management: Maintain and enhance existing data integrations for performance marketing channels, including but not limited to Google Ads, The Trade Desk, Meta, TikTok, and Yahoo.
Data Quality Assurance: Ensure high data quality and fidelity for campaign measurement, optimization, automation, and personalization by establishing rigorous validation and reconciliation processes across all marketing data streams.
Stakeholder Collaboration: Work closely with Marketing, Analytics, and Data Science teams to understand data requirements and deliver reliable data solutions that drive end-to-end campaign execution.
Qualifications
Bachelor's degree in Computer Science, Engineering, or a related quantitative field.
Minimum of 3+ years of experience in Data Engineering, with hands-on experience building and maintaining production data pipelines.
Strong proficiency in Google Cloud Platform (GCP) data services (BigQuery, Cloud Composer/Airflow, Dataflow, Pub/Sub, Cloud Storage).
Expertise in SQL and at least one programming language (Python strongly preferred) for data manipulation and pipeline scripting.
Demonstrated experience or strong conceptual understanding of AI Agents, Generative AI (specifically Google Gemini), and Machine Learning operations (MLOps) as they apply to data infrastructure.
Experience with data integration from external marketing platforms (e.g., Google Ads API, Marketing APIs for Meta/TikTok, DSPs/DMPs like The Trade Desk).
Desired
Experience in developing automated data recovery mechanisms or proactive monitoring systems.
Familiarity with data governance, security best practices, and compliance (e.g., PII handling).
Experience with containerized application, GKE and CI/CD processes.
Excellent communication and collaboration skills, with the ability to articulate complex technical concepts to non-technical stakeholders.