Senior Python Data Engineer

Senior Python Data Engineer Raleigh, North Carolina (Hybrid) Phone+Skype Job Decription: Must Have: 5+ years experience as a data engineer or python engineer Strong python libraries- pandas (specifically), lambda Experience in SQL and Spark SQL and EMR A deep understanding of performance tuning AWS Cloud experience Plus: Experience in the finance industry Experience working with vendors like: JMP, Morning star, JPMC, or big fin vendors Day to Day: UAP is an initiative to create a common/unified acquisition platform to provide acquisition as a service through low code and config driven architecture for fidelity needs. Business Capabilities: 1) Self service capability to search and subscribe/request for existing dataset 2) Self service capability to request new vendor dataset 3) Integration and automation with MDD to accelerate new feed registration 4) Accelerators to acquire and onboard new feeds 5) Accelerators and capabilities to switch vendor 6) Drop zone to manage data distribution based on licensing/subscription 7) Registry and inventory to manage list of vendor feeds and their consumption patterns 8) Finops model for consumption 9) Finops model to derive ROI 10) Multi-tenant capability for applications to co-exists on the same infrastructure with right level of abstraction based on authorization. 11) Capability to provide reports comparing vendors. a) Data overlap, difference b) Coverage c) SLA and past performance interms of meeting SLA Technology Enablers: Integration and self service with MDD, Vendor management/Procurement and Governance Configuration driven feed acquisition Configuration driven transformation of vendor data into canonical format Capability to compare data across vendors and generate gap reports Registry to maintain feed metadata, contact, SLA, Owners Lineage to track usage (run time information on who consumes what data on a daily basis) Configuration driven distribution Self service capabilities for onboarding new feeds Ingestion adaptors for known file types Enable data for analytics and exploration usecase Reports (Quality, Coverage, Data Gaps, Data Catalogs, Feed metrics