Principal Data Engineer

jobs-near-me.org

<div class="isg-job-description“>

WHO WE ARE

We are the Metropolitan Council, the regional government for the seven-county Twin Cities metropolitan area. We plan 30 years ahead for the metropolitan area’s future and provide regional transportation, wastewater, and housing services. More information about us is on our website.
We are committed to supporting a diverse workforce that reflects the communities we serve.
Information Services (IS) is the central IT department within the Regional Administration (RA) division, supporting all divisions of the Metropolitan Council. Our 140 team members provide technology, practices, and innovative solutions that enable the Council’s core services.
How your work would contribute to our organization and the Twin Cities region:

The Principal Data Engineer works with business and IS leaders and stakeholders to establish data engineering strategies that support the organization’s goals and objectives. They lead and influence decisions around data architecture, data processing, and data governance, collaborating with IS and business units to create solutions that make data available to systems and/or data consumers, such as system users, developers, data scientists, or business analysts.

They create and socialize the data engineering technology roadmap and plan how to get there. They evaluate new technologies, tools, and techniques, and work closely with the IS architects to build data engineering solutions on top of the overall technology architecture. They help establish data governance policies, procedures, standards, and best practices to ensure compliance with industry regulations, security and data privacy laws, and information security requirements.

This role designs, develops, and maintains data solutions. This includes databases (DBs), data lakes, data warehouses, data marts, and data movement (data pipelines/ETL/ELT), as well as integrating analytics and data science outputs into existing business processes or systems. They administer, monitor, and maintain data solutions and tools, ensuring their security, performance, and availability. They also identify and resolve data quality issues.

The Principal Data Engineer requires the highest degree of expertise in database and data platform technologies, data modeling, structured and unstructured data, and strong programming skills. They often act as technical leads and work on initiatives ranging from small to large, advising and mentoring others. They play a critical role in ensuring the data is properly managed and leveraged to support strategic decision-making and to achieve business objectives.

Full Salary Range: $48.56 – $78.77 hourly/$101,005 – $163,842 yearly
This position is eligible for a hybrid telework arrangement (both remote and onsite). Candidate’s permanent residence must be in Minnesota or Wisconsin.

What you would do in this job

Partners with executives, architects, and technical leaders.
Guide to multi-team initiatives.
Represent data engineering in strategic planning.
Lead the design and architecture of enterprise data platforms.
Define engineering standards, frameworks, and best practices.
Drive modernization of data systems and adoption of cloud-native solutions.
Solve highly complex data engineering challenges across platforms.
Partner with leadership to define long-term data strategy.
Provide expert-level mentoring and oversight to engineering teams.
Develop data engineering strategies for operations, reporting, and analytics.
Create and socialize the data engineering roadmap to achieve the strategic plan.
Evaluate new technologies, tools, and techniques and make recommendations.
Develop data policies, procedures, standards, and best practices.
Mentor, influence, and advise data engineers on industry best practices.
May act as a technical lead on small to large data-related initiatives.

Data Operations

Oversee performance, reliability, and security of enterprise data systems.
Guide database and platform engineering teams.
Set and enforce access/security standards.
Define data governance and quality frameworks.
Ensure compliance across platforms.
Oversee classification, retention, and audit practices.
Lead root-cause analysis for major incidents.
Direct capacity planning and optimization.
Set operational standards and reliability targets.
Ensure data systems are available, scalable, secure, and supportable.
Administer, monitor, and support 24x7x365 operations, ensuring data security, performance, and availability.
Resolve incidents and meet Service Level Agreements (SLAs).
Ensure compliance with industry regulations, security, and data privacy laws.
Recommend performance improvements, optimizations, and automation.
Help fulfill data practices requests, complete audits, and respond to cybersecurity events.
Efficiently complete data service requests.
Identify and resolve data quality issues.

Data Development

Define technical architecture for Council-wide data solutions.
Design highly scalable pipelines and data models.
Guide to the adoption of new technologies.
Review and approve complex engineering designs.
Design, develop, and implement data solutions for operational, reporting, and analytical items.
Create solutions that make data available to systems and/or data consumers, including system users, developers, data scientists, and business analysts.
Build and leverage CI/CD development practices.
Integrate analytics and data outputs into business processes or systems.

Cross-Functional Collaboration

Partners with executives, architects, and technical leaders.
Guide to multi-team initiatives.
Represent data engineering in strategic planning.
Effectively collaborate with business teams on data needs and requirements.
Partner with IS teams to deliver available, secure, and supportable solutions for on-premises, hybrid, and cloud environments.
Mentor and train IS and business teams on data solutions.

What education and experience are required for this job (minimum qualifications)

Any of the following combinations of education (in Computer Science/Engineering, Information Technology, or related) and relevant experience:

Master’s degree and four (4) years of experience
Bachelor’s degree and six (6) years of experience
Associate’s degree and eight (8) years of experience
High school diploma/GED and ten (10) years of experience

What additional skills and experience would be helpful in this job (desired qualifications):

Experience with Azure, AWS, GCP, or other cloud service architecture
Experience working with CI/CD pipelines via GitHub or equivalent source control platform
Experience with Terraform or other Infrastructure as Code frameworks
Working knowledge of Spark or equivalent distributed computing engine
Experience as a technical lead on a team
Modern cloud data platform experience in Databricks or equivalent
Developing & supporting data integrations in Python or Scala
Reporting & data visualization experience

Knowledge, Skills, and Abilities Required:

Developed data strategies & roadmaps
Defined data standards, best practices, policies, & procedures
Developed data solutions for operations, reporting, & analytics
ITIL, Incident, Service, Change & Problem Management experience
Knowledge of & experience with data privacy laws, security, & industry regulations
Writing & tuning complex SQL queries
Extensive Experience with Databricks, Snowflake, or equivalent

What additional skills and experience would be helpful in this job (desired qualifications):

Experience with Azure, AWS, GCP, or other cloud service architecture
Experience working with CI/CD pipelines via GitHub or equivalent source control platform
Experience with Terraform or other Infrastructure as Code frameworks
Working knowledge of Spark or equivalent distributed computing engine
Experience as a technical lead on a team
Modern cloud data platform experience in Databricks or equivalent
Developing & supporting data integrations in Python or Scala
Reporting & data visualization experience

Knowledge of IS teams, roles & responsibilities, services, & general technologies.
Understand the data management team, group functions, & data technologies.
Knowledge of the various business units & the types of data each has and needs.
Understand & use incident, service, problem, & change management processes.
Strong interpersonal relationships within and across teams, to remove barriers, reduce risk, build trust and improve insights.
Skill and comfort in delivering strategic guidance to IS teams and leadership.

What you can expect from us:

We offer the opportunity to make a difference and positively influence the Twin Cities metropolitan area.
We encourage our employees to develop their skills through on-site training and tuition reimbursement.
We provide a competitive salary, excellent benefits, and a good work/life balance.

More about why you should join us

Additional information

Union/Grade: AFSCME/Grade J
FLSA Status: Exempt
Safety Sensitive: No
What your work environment would be:
You would perform your work in a standard office setting. Work may sometimes require travel between your primary work site and other sites.
If you are new to the Metropolitan Council, you must pass a background check, which verifies education, employment, and criminal history. If you have a criminal conviction, you do not automatically fail. The Metropolitan Council considers felony, gross misdemeanor, and misdemeanor convictions on a case-by-case basis, based on whether they are job-related and whether the candidate has demonstrated adequate rehabilitation.
If you are already an employee of the Metropolitan Council, you must pass a criminal background check if the job you’re applying for is in Information Services.
Security Policy:
This position involves direct access to Criminal Justice Information (CJI) as defined by the FBI CJIS (Criminal Justice Information Services) Security Policy. In accordance with section 5.12.1.1 of the FBI CJIS Security Policy, final candidates must agree to submit to a state-of-residence and national fingerprint-based record check. If the record check reveals criminal convictions, the Metropolitan Transit Police Department and/or the Minnesota Bureau of Criminal Apprehension will review the nature and circumstances of those convictions to determine whether access to Criminal Justice Information would be permissible. If it is determined that access to Criminal Justice Information would not be permissible, the candidate will no longer be eligible for this position.
IMPORTANT: If you make a false statement or withhold information, you may be barred from job consideration.
The Metropolitan Council is an Equal Opportunity, Affirmative Action, and veteran-friendly employer. The Council is committed to a workforce that reflects the region’s diversity and strongly encourages persons of color, members of the LGBTQ community, individuals with disabilities, women, and veterans to apply.
If you have a disability that requires accommodation during the selection process, please email HR-OCCHealth@metc.state.mn.us

To help us track our recruitment effort, please indicate in your cover/motivation letter where (usajobvacancies.com) you saw this job posting.

Next Helper »

Previous « Senior Project Coordinator

Principal Data Engineer

Maintenance Helper Maintenance Helper

Journeyman Millwright

BESS Technician – Southwest Region

Mechanical Engineering Internship – (August – December 2026) – Atlanta, GA

Production Supervisor

Shift Leader