DATA ENGINEER - 72020

Executive Service

This position is designated as hybrid.

Background Check: This position requires CJIS & FTI fingerprints and name based background checks.

Who we are and what we do:

The State of Tennessee is seeking a skilled Data Engineer to design, build, and optimize data pipelines and architectures in support of analytics, reporting, and machine learning initiatives. The ideal candidate will have deep expertise in data engineering, cloud-native services, and distributed data processing. This role requires strong problem-solving skills, hands-on technical knowledge, and the ability to work cross-functionally with stakeholders (business), data science, and engineering teams.

Key Responsibilities:

• Design, develop, and maintain scalable ETL/ELT pipelines and data integration processes on GCP.
• Build and optimize data warehouses, data lakes, and streaming data platforms using GCP services (BigQuery, Pub/Sub, Dataflow, Dataproc, etc.).
• Implement and maintain data governance, security, and compliance practices.
• Collaborate with stakeholders to understand business requirements and translate them into technical solutions.
• Ensure data quality, reliability, and performance across large-scale distributed systems.
• Monitor and troubleshoot pipeline performance, ensuring high availability and cost efficiency, perform GCP platform system administration (Security, IAM, patches, upgrades, networking, containerization etc.)
• Develop, secure, and manage APIs using API managers such as ApigeeX, MuleSoft etc., ensuring seamless data access and integration across platforms.
• Implement API-driven micro-services including security, versioning, monitoring, and lifecycle management using GCP/Terraform/Docker best practices.
• Support data scientists and analysts by providing clean, reliable, and well-modeled data sets.
• Contribute to best practices, standards, and documentation for data engineering workflows.

Minimum Qualifications:

Required Qualifications:

1. 3+ years of experience in data engineering or related field.
2. Strong proficiency in SQL and experience with relational and non-relational databases.
3. Hands-on experience with GCP services such as BigQuery, Pub/Sub, Dataflow, Dataproc, Cloud Storage, and Composer.
4. Proficiency in Python, SQL, Terraform and Docker.
5. Lead GCP projects end-to-end, from requirements gathering to deployment and support.
6. Experience with data modeling, ETL design patterns, and pipeline orchestration.
7. Experience with CI/CD, version control (Git), and containerization (Docker/Kubernetes).
8. Knowledge of data governance, security, and compliance frameworks.

Preferred Qualifications:

1. Google Cloud Professional Data Engineer certification or equivalent.

Pursuant to the State of Tennessee's Workplace Discrimination and Harassment policy, the State is firmly committed to the principle of fair and equal employment opportunities for its citizens and strives to protect the rights and opportunities of all people to seek, obtain, and hold employment without being subjected to illegal discrimination and harassment in the workplace. It is the State's policy to provide an environment free of discrimination and harassment of an individual because of that person's race, color, national origin, age (40 and over), sex, pregnancy, religion, creed, disability, veteran's status or any other category protected by state and/or federal civil rights laws.