What is the must-needed job and the must-have competencies for the world of data in 2021?
April 9th 2021
A friend of mine has been recruited as a Gendarmerie lieutenant thanks to his diploma in data science. That looks sexy but, unfortunately for him, the education focus mainly on basic infantry and law during the two first years, and an executive position for the first three years before joining back a department related to the data field.
By dint of doing interviews or contract opportunities I felt that the sexiest job in the 21st century wasn't data scientist anymore, at least for companies. I feel, without data yet to prove that, that the most requested position in the data world now is the Machine Learning Engineer. That is, the guy who puts into production algorithms.
Great but what do you need from that, apart from the usual language skills? The skills to master relate mostly to putting into production like with:
- Docker: Virtualization of app/algo/models at the operating system level to provide software in packages
But experience in Big data is also required:
- pySpark: Python API written in python to support Apache Spark. ... Apache Spark is written in Scala and can be integrated with Python, Scala, Java, R, SQL. Spark is a computational engine, which works with huge data sets by processing them in parallel and batch systems.
- ... (Kubernetes? Scrapy?)
But new tools for analytics are have just arrived as well such as:
- Databricks: a cloud-based data engineering tool used to process and transform massive amounts of data and explore data via machine learning models. Recently added to Azure, it is the latest Big Data tool for the Microsoft cloud.
- snowflake : looks like the same thing: to store and analyze data on the cloud.
So, do you have what it takes to be a Machine Learning Engineer?