Vacancy expired!
- You will work closely with key stakeholders across the organization to transform the way the company leverages and uses analytics to drive business decisions. You will be part of a team building an enterprise-class platform that delivers value by empowering data-driven decisions based on centralized, high-quality data.
- You will have the opportunity to be part of a team leading an enterprise IT and business transformational change to modernize the enterprise data and analytics platform using cutting-edge technology such as Databricks/Lakehouse, AWS, Spark and cloud-based data tools.
- Help to deliver a reality where insights are easily discoverable and where self-service data empowers the ability to leverage descriptive and predictive analytics to provide innovative strategies and solutions to support our members’ diverse communities.
- Be part of a team building and supporting an end to end data lake and data pipelines on AWS using Agile project management methodologies
- Design, implement and support an end to end Data Lake data pipelines using the technologies listed above as well as:
- Data integrity process using Databricks and related tools/processes
- Integrating Data Lake with 3rd party application APIs for downstream queries
- Access design and integrations for Data Lake users and applications
- Experience with AWS and related data technologies and concepts in the following areas is required:
- Python
- Spark (pySpark a plus)
- Delta Lake Formation
- IAM data lake security concepts/permissions
- Data Workflows & data automation
- ELT/ETL
- Athena
- Glue/Crawlers
- Triggers
- Lambda Functions
- Data Onboarding – create, deploy and operational support of data integration pipelines from a variety of data sources including historical data loads and daily synchronization updates
- Support of data systems create and manage integrations and pipeline design/deployment of new systems and software into the enterprise Data Lakehouse as required based on business and technology needs
- Cross-functional team support – partner, support and integrate Data Lakehouse with key analytical systems such as Alteryx, Tableau and Sagemaker to drive data-driven analytic business outcomes
- Scheduling systems – experience with scheduling systems such as Control-M or Airflow
- At least 3 years of data lake, database and data management experience.
- Databricks, Fivetran, AWS or other similar data development or certifications are highly preferred.
- Experience working in an Agile environment
- Experience working with large datasets and managing data quality and data availability at an enterprise level
- 4 year college degree in information technology or equivalent experience
- Understanding of cloud technologies such as IaaS and SaaS and cloud-based storage platforms
- Good communication (verbal, written, visualization and listening) skills
- Self-starter who works well in a team setting and as an individual with the ability to prioritize and organize competing work demands
- Detail-oriented, a critical thinker who has excellent analytical and problem-solving abilities and attention to detail
- Hands on technologist with ability to understand and drive/troubleshoot the target state architecture and work with others
- Ability to address technical challenges and research/identify solutions by adopting ways of embracing new technologies and techniques when needed
- Interest and ability to working across multiple teams and working with vendors to understand business needs to aid in development of the right solution
- Comfortable working with data APIs
- Good understanding data modeling and CDC will be a plus