Sunday, August 21, 2022

Data Engineering Series-Python skills for Data Engineers


            


Python skills required for Data Engineers

While interacting with data enthusiasts, one of the questions asked repeatedly is :
What level of python skill requires for a Data engineer?

Today we are going to see the most important python skills that require data engineers. To become a Data engineer, one should familiar with minimum python fundamentals and a few libraries used frequently. To know the important python libraries requires for a data engineer, please refer my previous post given in the first comment.

Python Basics:

Lists
Tuples
Directories & sets
Variables
Strings
Math expression
Loops
Python Functions
Error/File Handling
Unit Testing

To interact with Data sources(DBs)

Pycopg2
Mysql

Working with data:

Json
Jsonschema(validation)
Datetime
Numpy
Pandas

Connection to APIs:

Requests
logging

 






at

No comments:

Post a Comment

Subscribe to: Post Comments (Atom)

Spark- Window Function

  Window functions in Spark ================================================ -> Spark Window functions operate on a group of rows like pa...

  • SQL Interview Preparation
      SQL Interview Q& A Part- 1 1.Explain what is partitioning, hash partitioning and round robin partitioning? To improve performance, tra...
  • Databricks Interview Preparation
      Databricks Interview Q&A- Part 1 1.How to connect the azure storage account in the Databricks? To connect the azure blob storage in th...
  • Want to become a Data Architect?
      Are you looking for a resource for learning to become a Data Architect? Data Architect is one of the highest roles in the data world. To b...