Vacancy expired!
Title: Lead Databricks EngineerLocation: Sunrise, FLDuration: 9 monthsCompensation: $75-85/hrWork Requirements: US Citizen, GC Holders or Authorized to Work in the USThe Senior Databricks Engineer will help us build a solid, scalable data infrastructure and analytical environment. The ideal candidate enjoys data wrangling, optimizing data pipelines, and building solutions for data collection and analysis. Work assignments may cover requirements gathering, source-to-target mapping, data validation scripting and review, developing and monitoring ETL/ELT data pipelines, designing and implementing database schema/tables/views, and producing datasets as input to data science models and visualizations. The Senior data engineer will work directly with the Director of BI and Analytics to ensure consistency and compliance of deliverables to frameworks and governance processes. Duties and ResponsibilitiesDesign efficient and scalable data processing systems and pipelines on Databricks, APIs, and AWS Services.Design and implement tools to detect data anomalies. Ensure that data is accurate, complete, and across all platforms.Develop data models and mappings and build new data assets required by stakeholders. Perform exploratory analysis on existing products and datasets.Develop scalable and reusable frameworks for the ingestion and transformation of large datasets.Understand trends and the latest technologies. Evaluate the performance and applicability of potential tools for our requirements.Build the infrastructure required for optimal extraction, transformation, and loading of data from various data sources.Work with stakeholders, including the Executive, Product, Data, and Design teams, to assist with data-related technical issues and support their data infrastructure needs.Partner to deliver a modern data engineering model that follows Dev/Ops principles and standards for continuous integration/ continuous delivery (CI/CD) processes.Establish the documentation of reports and develops and maintain technical specification documentation for all reports and processes within Databricks.Collaborate with IT and business partners to define, manage and deliver innovative Data solutions to drive growth and adoption of capabilitiesMentor team members on best practices, CI/CD, and pipeline orchestration.Assist with the administrative tasks of the Databricks platform, such as job schedule monitoring, users access control, and performance management.Other duties as required.Required Qualifications Education/ Years of ExperienceBachelor's degree in computer science, information technology, or related field. Master's degree preferred.5+ years of experience in a data engineer role, including 2+ years designing and building Databricks data pipelines, is REQUIRED.3+ years of SQL experience, specifically to write complex, highly optimized queries across disparate data sets, is REQUIRED.2+ years of hands-on Python/PySpark/SparkSQL experience is REQUIRED; experience with Scala is a plus.2+ years of Spark experience (especially Databricks Spark and Delta Lake) is REQUIRED.2+ years of experience deploying code using a source control product like GitLab/GitHub is REQUIRED. 2+ years implementing data management life cycle, using data quality functions like standardization, transformation, rationalization, linking, and matching is REQUIRED.2+ years performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement is REQUIRED.2+ years of experience building and optimizing "big data pipelines, architectures, and data sets. In-depth knowledge of Model and Design of DB schemas for read-and-write performance.Professional Experience with cloud data warehouse solutions (e.g., Redshift, Snowflake, Databricks) is HIGHLY DESIRABLE.Experience with Databricks Administration is PREFERRED.Working knowledge of API or stream-based data extraction processes is PREFERRED. Knowledge, Skills, Abilities, and Other Characteristics (KSAOs)Strong project management and organizational skills.Experience supporting and working with cross-functional teams and enterprise-wide data programs.Exceptional written, verbal, and listening communication skillsProven analytical, problem-solving, decision-making, and prioritization skillsAble to contribute to the big picture, conceptualize outcomes, and identify trendsOur benefits package includes: Comprehensive medical benefits Competitive pay, 401(k) Retirement plan and much more!About INSPYR Solutions: As a leading information technology partner, we connect top IT talent with our clients to provide innovative business solutions through our IT Staffing, Professional Services, and Infrastructure Solutions divisions. We understand and value the unique needs of highly-skilled information technology professionals in the industry and always strive to stay above the curve. Our company was founded on the following core values: Be the Best, Understand the Urgency, Never Ever Give Up, Have the Courage to Excel, and Make a Contribution. We take pride in our business model and strive to create a positive workplace environment through an exemplary culture.INSPYR Solutions provides Equal Employment Opportunities (EEO) to all employees and applicants for employment without regard to race, color, religion, sex, national origin, age, disability, or genetics. In addition to federal law requirements, INSPYR complies with applicable state and local laws governing nondiscrimination in employment in every location in which the company has facilities.