KPG99 INC

FULL TIME: Senior Data Platform Engineer/Azure Lakehouse - Hybrid/Raleigh , NC (USC Only)

Raleigh, NC

Full-timeHybrid

Published Expires (UTC)

Your next opportunity

Create a job alert

Receive new jobs matching these criteria.

Data Warehousing SpecialistsRaleigh, NC · Within 15.5 mi · Data Science & Analytics, Hybrid Jobs, Full-Time Jobs
Free for job seekers. Always.
You’re in control

Manage your alerts or unsubscribe at any time.

Manage job alerts

About this opportunity

Hope you are doing well!

This is a full-time position, Candidates must be able to go full time from day one.

Requirement Notes (Candidate Job description below) : The Senior (10+ years with 15+ years preferred) Data Platform Engineer will manage, optimize, and evolve the cloud-native platforms used by the Data Office. The role combines deep data engineering capability with platform engineering accountability across Microsoft Azure, Databricks, Unity Catalog, integration and ingestion services, Collibra, and Immuta.

The engineer will be responsible for the day-to-day engineering and technical management of the platform ecosystem, keeping it secure, reliable, scalable, observable, and cost-effective. That includes platform health, availability, upgrades, configuration, automation, monitoring, access management, and incident resolution.

***** CANDIDATES NEED TO BE SENIOR (10+ YEARS) AZURE DATA ENGINEERS OR ARCHITECTS AND HAVE RECENT EXPERIENCE IN REGULATED ENVIRONMENTS. CANDIDATES MUST HAVE EXCELLENT COMMUNICATION SKILLS AND BE ABLE TO TALK TO THE TECHNICAL TEAM AND THE BUSINESS UNITS.

****CANDIDATES MUST HAVE GOOD TENURE AND/OR LONG PROJECTS. CANDIDATES WITH RECENT LOCAL PROJECTS REQUIRED. – NO RELOCATION

Required Candidate Location: Hybrid/ Raleigh , NC – 3days a week -NO RELOCATION

Type of Interview Required: Video

Candidate Visa’s Accepted: No Opt. No H1

*** All submittals must Include:

1. Driver’s license or State ID

2. Link to the candidates LinkedIn account.

Job Description:

The Senior Data Platform Engineer will manage, optimize, and evolve the cloud-native platforms used by the Data Office. The role combines deep data engineering capability with platform engineering accountability across Microsoft Azure, Databricks, Unity Catalog, integration and ingestion services, Collibra, and Immuta.

The engineer will be responsible for the day-to-day engineering and technical management of the platform ecosystem, keeping it secure, reliable, scalable, observable, and cost-effective. That includes platform health, availability, upgrades, configuration, automation, monitoring, access management, and incident resolution.

In parallel, the role delivers senior-level data engineering across ingestion, integration, transformation, orchestration, governance, and data access, building pipelines and lakehouse solutions that support analytics, business intelligence, AI, and machine learning. This is a hands-on role working with data architects, engineers, analysts, governance, security, and business stakeholders to deliver data products that are reliable, discoverable, controlled, and fit for purpose.

KEY RESPONSIBILITIES

Platform Management

· Manage the Data Office platform estate across Azure, Databricks, Unity Catalog, integration and ingestion services, Collibra, and Immuta, ensuring operational reliability and alignment with enterprise standards.

· Monitor and optimize platform performance, workloads, clusters, SQL warehouses, jobs, storage, and compute to improve reliability, scalability, user experience, and cost efficiency.

· Own capacity and workload management, including resource allocation, concurrency, job scheduling, autoscaling, workload isolation, and the assessment of serverless and cluster-based compute options.

· Administer platform configuration, workspaces, environments, service connections, secrets, policies, and technical integrations across development, test, and production.

Access and Governance

· Manage data access controls and entitlements across Azure, Databricks, Unity Catalog, Collibra, and Immuta, applying least-privilege, role-based, attribute-based, and policy-based access principles.

· Implement and maintain Unity Catalog structures, including catalogs, schemas, permissions, data objects, lineage, and controlled data-sharing patterns.

· Support Collibra integration and adoption for cataloging, metadata, business glossaries, lineage, ownership, stewardship, and governance workflows.

Data Engineering

· Design, build, and maintain secure, scalable lakehouse solutions using Azure Data Lake Storage, Databricks, Delta Lake, Spark, PySpark, Python, and SQL.

· Develop and optimize batch and streaming ingestion, integration, ETL, and ELT pipelines using Azure Data Factory, Databricks Workflows, APIs, event-based services, and appropriate orchestration tools.

· Engineer reusable integration and ingestion patterns for structured, semi-structured, and unstructured data from enterprise applications, files, databases, APIs, and event streams.

· Implement data quality, reconciliation, validation, exception handling, recovery, and audit controls to improve reliability and trust in enterprise data products.

Operations, Automation and Reliability

· Implement observability across platforms and pipelines, including monitoring, logging, alerting, performance metrics, usage reporting, cost reporting, failure analysis, and operational dashboards.

· Lead technical incident investigation, root-cause analysis, remediation, and problem management for platform, workload, access, and data pipeline issues.

· Automate platform deployment and configuration through infrastructure as code, CI/CD, testing, release management, and version control using tools such as Terraform, Git, Azure DevOps, or GitHub Actions.

· Plan and execute platform upgrades, feature adoption, migrations, configuration changes, resilience testing, backup and recovery activities, and lifecycle management.

· Maintain technical standards, runbooks, architecture diagrams, support procedures, data dictionaries, operating documentation, and knowledge articles.

Leadership and Collaboration

· Provide technical leadership, coaching, design review, and best-practice guidance to data engineers and other platform users.

· Collaborate with architecture, cloud operations, cyber security, risk, governance, analytics, and business teams to translate requirements into secure and sustainable engineering solutions.

REQUIREMENTS

Experience and Background

· 10+ years of hands-on experience in data engineering, data platform engineering, cloud engineering, or a closely related field.

· Proven experience managing and supporting enterprise data platforms in production, including performance, workloads, storage, compute, access, availability, and cost.

· Proven experience designing and implementing cloud data lake or lakehouse architectures at scale.

· Experience operating in a Microsoft Azure enterprise environment and working with cross-functional platform, security, governance, and engineering teams.

· Experience working in a regulated environment, preferably legal or financial services, with an understanding of security, privacy, audit, and compliance obligations.

Core Platform and Engineering Skills

· Advanced hands-on experience with Databricks, including Spark, PySpark, Delta Lake, SQL, Workflows, clusters, SQL warehouses, jobs, performance tuning, and cost optimization.

· Hands-on experience with Unity Catalog, including catalogs, schemas, permissions, lineage, secure data access, and governance administration.

· Strong experience with Microsoft Azure services, including Azure Data Lake Storage Gen2, Azure Data Factory, Azure Functions, Logic Apps, Key Vault, networking, identity, monitoring, and related services.

· Strong Python and SQL skills, with the ability to develop, review, troubleshoot, and optimize production-grade code and queries.

· Experience designing and supporting integration and ingestion patterns across databases, enterprise applications, files, APIs, and streaming or event-based sources.

· Experience with Collibra or a comparable data catalog and governance platform, including metadata, lineage, stewardship, ownership, and workflow integration.

· Experience with Immuta or a comparable data security platform, including policy-based access, sensitive data controls, auditability, and integration with Databricks.

· Knowledge of identity, access management, role-based access control, attribute-based access control, service principals, managed identities, secrets, and least-privilege design.

· Experience with platform monitoring, observability, capacity management, incident response, root-cause analysis, resilience, backup, and recovery.

· Experience with CI/CD, infrastructure as code, source control, automated testing, and release management using Terraform, Git, Azure DevOps, GitHub Actions, or equivalent tooling.

Professional Skills

· Strong analytical, problem-solving, troubleshooting, and root-cause analysis skills.

· Ability to balance platform stability and control with delivery speed, usability, and innovation.

· Clear verbal and written communication skills, with the ability to work effectively across technical and non-technical teams.

· Ability to prioritize operational work, engineering delivery, platform improvements, and technical debt.

· Proactive ownership, sound judgement, attention to detail, and commitment to continuous improvement and knowledge sharing.

PREFERRED QUALIFICATIONS

· Experience with dbt, Apache Airflow, Azure Event Hubs, Spark Structured Streaming, or comparable transformation and orchestration technologies.

· Experience with Azure Synapse Analytics, Microsoft Purview, containerization, Kubernetes, or enterprise API management.

· Knowledge of FinOps practices for data platforms, including attribution, budgeting, forecasting, chargeback or showback, and cost-control policies.

· Understanding of data quality, metadata management, lineage, retention, classification, privacy, and records-management controls.

· Familiarity with agile delivery and IT service management practices, including change, incident, problem, and release management.

Thanks and Regards,

Mamta Giri | Lead Recruiter | KPG99, INC

3240 E State St EXT, Hamilton, NJ 08619 | www.kpgtech.com

Your next step

Interested in this role?

Review the details, then choose the resume you want to share.

Apply for this job Free for job seekers. Always.

Share this job

About the employer

KPG99 INC

Represent this company?Claim this profile

Related opportunities

Explore current openings connected to this search.

Your next opportunity

Create a job alert

Receive new jobs matching these criteria.

Data Warehousing SpecialistsRaleigh, NC · Within 15.5 mi · Data Science & Analytics, Hybrid Jobs, Full-Time Jobs
Free for job seekers. Always.
You’re in control

Manage your alerts or unsubscribe at any time.

Manage job alerts
Your preferred location

Where would you like to work?

Choose a city, a state, or United States to search across the country.

Location is requested only when you choose this option. Precise coordinates are not stored.
JobbolJobbol

Your privacy. Your choice.

Choose optional cookies and data uses in this browser. Closing this window does not give consent. Limited Google measurement without optional cookies may run before your choice; recorded refusals and browser privacy signals are respected.

Essential services

Sign-in, secure forms, abuse prevention, language and the city or theme you explicitly select. These functions are needed to provide the service you request.

Always active

Do Not Sell or Share My Personal Information

Jobbol does not sell resumes or use your personal information for cross-context behavioral advertising. This protection is active for everyone. You can also turn off all optional processing below without signing in.

Protection active

You can change your choice at any time from the footer. Choices last up to 180 days in this browser. Account restrictions also apply when we can identify your signed-in account.

Notice version: 2026-11-12.ads1

Quick view