Sr. Data Engineer - Military veterans preferred

CAQH

Position Summary:

The Sr. Data Engineer will play a strategic role in designing, developing, and optimizing advanced data pipelines and infrastructure to support CAQH’s enterprise data modernization efforts. This role is critical to delivering scalable, secure, and high-performance data solutions that drive analytics, reporting, and AI/ML initiatives across the organization. The ideal candidate will bring deep technical experience with the Azure data ecosystem, demonstrate leadership in data architecture and governance, and proactively support the team’s growth and capability maturity.

The Sr. Data Engineer is a full-time, remote, exempt position and reports to the Sr. Director, Data and Analytics.

Base Salary: $130,000 - $155,000.

Specific Responsibilities:

• Lead the design and optimization of data pipelines in Databricks and Azure Data Factory for real-time and batch workflows.

• Architect and manage datasets in Delta Lake and OneLake, ensuring performance, scalability, and compliance.

• Oversee data ingestion, transformation, and delivery to Azure SQL Managed Instance, Synapse, and Snowflake.

• Develop, maintain, and review Power BI dashboards and data visualizations in support of key stakeholders.

• Partner with data scientists to prepare data for AI/ML modeling, training, and deployment.

• Write, review, and maintain robust, production-grade SQL and Python code for large-scale data workflows.

• Monitor, troubleshoot, and improve data pipeline reliability, latency, and quality.

• Champion metadata management, data cataloging, lineage tracking, and documentation efforts.

• Promote best practices for data governance, including data security, privacy, and quality frameworks.

• Provide guidance and mentorship to junior engineers and interns.

• Participate in Agile sprint planning and lead technical discussions during knowledge-sharing sessions.

Supervisory Responsibilities:

This position does not have direct reports. However, the Sr. Data Engineer may collaborate with contractors or external vendors on specific initiatives and is encouraged to provide technical guidance to interns or newer team members when appropriate.

Skills: 

• Advanced proficiency in SQL, Python, and PySpark.

• Deep experience with Databricks, Delta Lake, Azure Data Factory, and related Azure services.

• Familiarity with Microsoft Fabric, OneLake, and medallion architecture concepts.

• Strong Power BI development experience including DAX, row-level security, and performance optimization.

• Experience designing scalable cloud data architectures and implementing data quality frameworks.

• Working knowledge of Snowflake, Azure SQL, and Synapse Analytics.

• Exposure to AI/ML workflows, including MLOps, feature engineering, and data versioning.

• Strong understanding of data governance, PII/PHI compliance, and security best practices.

• Excellent communication skills in a remote, cross-functional team environment.

• Experience using Git, DevOps tools, and CI/CD pipelines for data engineering workflows.

Experience:

• 4–7 years of hands-on experience in a data engineering or analytics engineering role.

• Demonstrated success leading data modernization or migration initiatives in cloud environments.

• Prior experience working with healthcare or other regulated data environments is highly desirable.

Education:

• Bachelor’s degree in Computer Science, Information Systems, Data Engineering, or a related field.

• Azure Data Engineer Associate or related certification (preferred).

• Coursework or certification in AI/ML (preferred but not required).

WHO WE ARE

The Council for Affordable Quality Healthcare (CAQH) was formed by health plans who came together nearly 25 years ago with the desire to make healthcare work better. Today, we are trusted by stakeholders from across the healthcare ecosystem to improve business processes and offer the most comprehensive provider and member data in the United States.

WHAT YOU GET

CAQH recognizes that its most important asset is its growing team of smart, creative, collaborative, forward-thinking and passionate professionals – and that a comprehensive employee benefits package is an important factor for them in choosing where to work. CAQH offers competitive compensation along with an extensive benefits package for all full-time employees, including medical, dental and vision coverage, tuition assistance and a 401k. We offer full-time remote work to all staff from any location and maintain a physical office in downtown Washington, DC.

At CAQH, we are proud of our active commitment to equal opportunities for all. Our teams work diligently to foster a workplace where all individuals are valued, respected, and empowered. We embrace different perspectives and are dedicated to providing equal opportunities for everyone to thrive. Join us in our mission to transform healthcare through innovative technology solutions while making a positive impact on the lives of diverse communities.

CAQH is an equal opportunity employer. It is our policy to recruit, hire, train, and promote individuals, as well as administer any and all personnel actions, without regard to race, color, religion, sex, national origin or ancestry, age, marital status, disability, protected veteran status, personal appearance, sexual orientation, gender identity or expression, familial status, family responsibilities, matriculation, political affiliation, genetic information, source of income, place of residence, or any other characteristic protected by law. CAQH will not tolerate any unlawful discrimination, and any such conduct is strictly prohibited.

Applicants have rights under the Family Medical Leave Act (FMLA), Equal Employment Opportunity (EEO), and the Employee Polygraph Protection Act (EPPA). If you are interested in applying for employment with CAQH and need an accommodation to apply for a posted position, contact CAQH Human Resources at 202-517-0436.

#LI-Remote

PI274662216