Senior Data Engineer (Python/Spark)

Data Engineer Full time / Direct HireLocation - !00% Remote Requirements

• Strong background on database development with a minimum of 5 year experience
• Understanding OLTP vs OLAP, especially conceptual & logical data modeling on each side, 3NF vs Star Schema and normalization process with Armstrong Axiom's rule (Relational Algebra)
• Extensive hands-on experience in database performance tuning with execution plan and profiling blocking issue on locks
• Ability to deal with complex queries, recursion with CTE, Set operators (emphasizing on division operator) and Window Functions
• Handling with high volume of transaction management with multi-threaded environment;

• Understanding ACID property correctly and different isolation levels with pessimistic & optimistic locks
• 2PL Strict or Rigorous Schedule on conventional RDBMS vs MVCC on cloud based database
• Resolving deadlock issue with the best practice
  • Understanding Row store vs Column store and their pros & cons (NSM, DSM & PAX)
  • Acknowledging Data Warehousing methodology, Inmon, Kimball and Data Vault 2.0 and their pros & cons
  • Trending along with Data Lake, Data Hub, Data Mesh and Lakehouse and their pros & cons
  • Differentiating State Based vs Migration Based database migration approach with full CI/CD automation process
  • Hands-on experience with at least one of the following RDBMS; Oracle, MSSQL, Postgres, MySQL or Sybase
  • Hands-on experience with at least one of the following No-SQLs; HBase, Dynamo, CouchDB, MongoDB, Cassandra or Cosmos
  • Hands-on experience with at least one of the following cloud-native data warehouses; Redshift, BigQuery, Synapse or Snowflake
  • Hands-on experience of building data pipeline on at least one of the following cloud platforms; AWS, Azure or GCP
  • Python is preferable