Big Data Engineer

Sky Solutions LLC
Columbia, MD
22 Oct 2024

Vacancy expired!

Summary: Sky Solutions is a technology staffing/consulting firm with over 10 years of experience in helping our clients with their needs on different technologies. Currently, we have a position for Big Data Engineer with one of our clients.

Position: Big Data Engineer

Location: Columbia, MD

Duration: 6+ Months Contract

Job Description:Preference for all candidates to work from our Columbia, MD office location. Hybrid approach: 2-3 days in the office; 2-3 days WFH;For a perfect candidate, full remote option may be possible; however, the candidate must live in MD, VA, or DC.

Responsibilities
  • Extract and integrate data from multiple data systems and organize data in a format that can be easily read by human or machine.
  • Transition of legacy ETLs with Java and Hive queries to Spark ETLs.
  • Design and develop data processing solutions and custom ETL pipelines for varied data formats like parquet and Avro.
  • Design, develop, test and release ETL mappings, mapplets, workflows using Streamsets, Java MapReduce, Spark and SQL.
  • Performance tuning of end-to-end ETL integration processes.
  • Analyze and recommend optimal approach for obtaining data from diverse source systems.
  • Work closely with the data architects and interface with business stakeholders to understand requirements and offer solutions.

Preferred Qualifications
  • BS in Information Technology, Computer Science, Software Engineering or related field.
  • Understanding of distributed computing principles and hands-on experience in Big Data Analytics and development.
  • Good knowledge of Hadoop and Spark ecosystems including HDFS, Hive, Spark, Yarn, MapReduce and Sqoop.
  • 4+ years of Big Data development experience.
  • 3+ years of working experience in ETL development and functional programming knowledge preferably with Scala, Spark, Java, Python, R.
  • 3+ years of experience tuning Spark/Java coding, SQL and No SQL.
  • AWS development using big data technologies preferred.
  • AWS cloud-certified, Databricks, and Snowflake experience a plus.

ADDITIONAL INFORMATION: Sky Solutions is a strategic consulting, staffing and technology services company headquartered in the Washington D.C. Metro Area. We deploy the optimal resources, expertise and technologies to help organizations improve their business performance. Sky Solutions is committed to creating innovative, flexible solutions for government and commercial clients. As a Small Disadvantaged Business (SDB) with Woman and Minority Owned certifications, we provide key staff to employers to meet their business-critical needs while promoting diversity and equal opportunity in employment.
  • ID: #21504827
  • State: Maryland Columbia 21044 Columbia USA
  • City: Columbia
  • Salary: Depends on Experience
  • Job type: Contract
  • Showed: 2021-10-22
  • Deadline: 2021-12-19
  • Category: Et cetera