Data Engineer

Data Engineer Dallas, TX (Remote) Data Engineer

• Should be able to create and maintain optimal data pipeline architecture.
• Bee able to work on large datasets and to meet functional and business requirements.
• Design infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL, Data Bricks, MS Azure Data Factory.
• Should be able to understand existing implementation and should enhance wherever needed.

• Create and maintain document and communicate standard methods and tools used.
• Team player and should work with other data engineers, data ingestion specialists, and experts across the team.
• Experience in performing root cause analysis on internal and external data and processes.
• Experienced using the following software/tools:
• Big data tools: Spark (Must), Hadoop (Not mandatory)

• Relational SQL and NoSQL databases, including COSMOS.
• Data pipeline and workflow management tools: DataBricks (Spark + Python), ADF, Dataflow
• Understanding and implementation knowledge on various Microsoft Azure Tools.
• Stream-processing systems: Streaming-Analytics, IoT Hub, Event Hub (Added advantage)
• Object-oriented/object function scripting languages: Python, SQL

• Knowledge and understanding of MS Azure DevOps.
• Should have experience in Agile methodology.