Job Detail

Qualification

Any Graduation

Job description

● Understanding our data sets and how to bring them together. ● Working with our engineering team to support custom solutions offered to the product development.. ● Filling the gap between development, engineering and data ops. ● Creating, maintaining and documenting scripts to support ongoing custom solutions. ● Excellent organizational skills, including attention to precise details ● Strong multitasking skills and ability to work in a fast-paced environment ● 5+ years experience with Python to develop scripts. ● Know your way around RESTFUL APIs.[Able to integrate not necessary to publish] ● You are familiar with pulling and pushing files from SFTP and AWS S3. ● Experience with any Cloud solutions including GCP / AWS / OCI / Azure. ● Familiarity with SQL programming to query and transform data from relational Databases. ● Familiarity to work with Linux (and Linux work environment). ● Excellent written and verbal communication skills ● Extracting, transforming, and loading data into internal databases and Hadoop ● Optimizing our new and existing data pipelines for speed and reliability ● Deploying product build and product improvements ● Documenting and managing multiple repositories of code ● Experience with SQL and NoSQL databases (Casendra, MySQL) ● Hands-on experience in data pipelining and ETL. (Any of these frameworks/tools: Hadoop, Big Query, Red Shift, Athena) ● Hands-on experience in Airflow ● Understanding of best practices, common coding patterns and good practices around ● storing, partitioning, warehousing and indexing of data ● Experience in reading the data from Kafka topic (both live stream and offline) ● Experience in PySpark and Data frames

Desired Profile

● Collaborating across an agile team to continuously design, iterate, and develop big data systems. ● Extracting, transforming, and loading data into internal databases. ● Optimizing our new and existing data pipelines for speed and reliability. ● Deploying new products and product improvements. ● Documenting and managing multiple repositories of code..

Key Skills