Job Description

We are looking for a Senior Data Engineer with expertise in Big Data, Spark, and Cloud technologies to join our team. If you have a strong background in designing, developing, and optimizing complex data pipelines , this role is for you!

Key Responsibilities:

• Design, develop, and support high-performance data pipelines for large-scale data processing.
• Work with Apache Spark (batch & streaming) to transform and analyze massive datasets.
• Utilize advanced SQL for data analysis and interaction.
• Build and optimize cloud-scale data lake solutions in Databricks, AWS EMR, Snowflake , and other Big Data environments.
• Implement Python-based scripting for automation and data processing.
• Ensure performance tuning and optimization of large datasets .
• Leverage GitHub for source control and adhere to best practices in development.
• Work with workflow scheduling tools like Airflow or Databricks Workflow .
• Collaborate with cross-functional teams, influencing stakeholders through effective communication.

Must-Have Skills:

• 5+ years of experience in designing and developing complex data pipelines .
• 5+ years of experience with Spark (batch & streaming mode) .
• 5+ years of experience with SQL for advanced analytics.
• 5+ years in Big Data environments (Databricks, AWS EMR, etc.).
• 3+ years of Python scripting experience.
• 3+ years of experience in AWS or other cloud platforms .
• Strong knowledge of solution & technical design for scalable data architectures.
• Experience with relational databases, Snowflake, and Data Warehouse concepts .
• Hands-on experience in workflow scheduling tools (Airflow, Databricks Workflow).
• Expertise in performance tuning of large datasets .

Nice-to-Have Skills:

• Familiarity with data governance and security best practices .
• Experience with CI/CD for data pipelines .

About BrickRed Systems:

BrickRed Systems is a global leader in next-generation technology, consulting, and business process service companies. We enable clients to navigate their digital transformation. BrickRed Systems delivers a range of consulting services to our clients across multiple industries around the world. Our practices employ highly skilled and experienced individuals with a client-centric passion for innovation and delivery excellence.

With ISO 27001 and ISO 9001 certification and over a decade of experience in managing the systems and workings of global enterprises, we harness the power of cognitive computing hyper-automation, robotics, cloud, analytics, and emerging technologies to help our clients adapt to the digital world and make them successful. Our always-on learning agenda drives their continuous improvement through building and transferring digital skills, expertise, and ideas from our innovation ecosystem.

Job Tags

Similar Jobs