Azure Data Engineer Job at Siri InfoSolutions Inc, Irving, TX

RFE4NlUwQlZvcUZNYkVaajB6QTViZzZY
  • Siri InfoSolutions Inc
  • Irving, TX

Job Description

Role: Azure Data Engineer

Location: Irving TX or Downtown Minneapolis (Day 1 Onsite)

Position Overview

We are seeking a Lead Developer with advanced skills in Python Apache Spark Azure Synapse and Azure Data Engineering services to develop and

manage ETL pipelines and data processing solutions that support AI/ML initiatives. The ideal candidate will have strong expertise in Azure Cloud

including SQL Data Factory SQL Pools Spark Pools and Data Warehousing. A background in CI/CD processes crossfunctional collaboration and

methodologies is essential. In addition knowledge of Data Science Java Kubernetes Azure Data Lake Storage (ADLS) Gen2 and API development is required.

Key Responsibilities

1. ETL and Data Pipeline Development

Design develop and optimize scalable ETL processes using Python Apache Spark and Azure Synapse.

Build and manage Azure Data Factory pipelines to orchestrate complex data workflows.

Use SQL Pools and Spark Pools within Synapse to manage and process large datasets efficiently.

Implement Data Warehousing solutions using Azure Synapse Analytics to provide structured and queryable data layers.

Ensure the data platform supports realtime and batch AI/ML data requirements.

2. Azure Cloud Development & CI/CD Deployment

Build configure and manage CI/CD pipelines on Azure DevOps for ETL and data processing tasks.

Automate infrastructure provisioning testing and deployment using InfrastructureasCode (IaC) tools like ARM templates or Terraform.

Optimize Azure Data Lake Storage (ADLS Gen2) to store and manage raw and processed data efficiently ensuring proper access control and data security.

3. CrossFunctional Collaboration

Collaborate with Data Scientists Data Engineers ML Engineers and Business Analysts to translate business requirements into data solutions.

Work with the DevOps and Security teams to ensure smooth and secure deployment of applications and pipelines.

Act as the technical lead in designing developing and implementing data solutions mentoring junior team members.

4. Data Engineering and API Development

Develop and integrate with external and internal APIs for data ingestion and data exchange.

Build test and deploy RESTful APIs for secure data access.

Use Kubernetes for containerizing and deploying data processing applications.

Manage data storage and transformation to support advanced Data Science and AI/ML models.

5. Agile Project Management

Participate in and lead Agile ceremonies such as sprint planning daily standups and retrospectives.

Collaborate with crossfunctional teams in iterative development to ensure highquality and timely feature delivery.

Adapt to changing project priorities and business needs in an Agile environment.

Required Skills and Qualifications

1. Technical Skills:

Expertise in Python and Apache Spark for largescale data processing.

Strong experience in Azure Synapse Analytics including SQL Pools and Spark Pools.

Advanced proficiency in Azure Data Factory for ETL pipeline orchestration and management.

Knowledge of Data Warehousing principles with handson experience building solutions on Azure.

Experience with SQL including complex queries optimization and performance tuning.

Familiarity with CI/CD tools like Azure DevOps and managing infrastructure in Azure Cloud.

Experience in Java for API integration and microservices architecture.

Handson knowledge of Kubernetes for containerized data processing environments.

Proficiency in working with Azure Data Lake Storage (ADLS) Gen2 for data storage and management.

Experience working with APIs (REST SOAP) and building APIbased data integrations.

2. Agile and CrossFunctional Skills:

Experience working in an Agile environment using Scrum or Kanban.

Ability to lead mentor and coach junior developers in the team.

Strong collaboration skills to work with data scientists analysts and crossfunctional teams to deliver endtoend data solutions.

3. Behavioral Skills:

Strong analytical and problemsolving skills with a passion for datadriven solutions.

Excellent communication and presentation skills able to explain complex technical concepts to nontechnical stakeholders.

Ability to work in a fastpaced dynamic environment with changing priorities.

Selfmotivated and resultsoriented with attention to detail.

Job Tags

Similar Jobs

Visiting Angels - Flagstaff

Sedona Caregiver - Help Someone in Your Neighborhood TODAY! Job at Visiting Angels - Flagstaff

 ...elder loved ones. A Visiting Angels Caregivers' purpose is to provide assistance with daily activities, light housekeeping, errands,...  ...dear clients. We do all of this to help our clients continue to live and age safely in their beloved home. Starting Pay $23.00 /... 

SystemsAccountants

NetSuite Principal Consultant Job at SystemsAccountants

 ...Do you have NetSuite consulting experience, and are you seeking a new role for a growing North American Alliance Partner? If so, we'd love to hear from you! Vursor is looking for a skilled NetSuite Principal Consultant, and the full-time role comes with an attractive... 

Worldwide Flight Services

Airline Ramp Agent Job at Worldwide Flight Services

 ...This job is covered by a collective bargaining agreement of the Transport Workers Union (TWU), a labor union which requires joining the...  ...Awardco Platform including gift cards and more!* Need quality medical care? Multiple options for both full and part-time employees!*... 

Bank of America Corporation

Audit Supervisor, BSA/AML Job at Bank of America Corporation

 ...along with the power to make a difference. Join us! This job is responsible for performing test work during moderately complex audit activities for Lines of Business (LOBs). Key responsibilities include designing and executing audit tests to evaluate the control environment... 

Halliburton

Service Operator I - Frac/Acid-196425 Job at Halliburton

 ...record with no DWI/DUI convictions in the past three years or any current DWI/DUI charges Completion of Operator Assistant II Frac/Acid competencies, or similar level of competence from previous industry experience. World Class Benefits Medical, dental, vision...