Azure Databricks Developer at Luxoft

Про вакансію

Project description• We are seeking an experienced Azure Databricks Developer to join our dynamic team. The ideal candidate will have strong expertise in Azure Databricks and Azure DevOps to design and implement scalable cloud-based data solutions. You will work closely with other data engineers, architects, and business teams to develop and optimize data pipelines, real-time data streaming, and cloud-based analytics solutions. The applications primarily consist of data pipelines built in Azure Databricks. They either transform data and load it into downstream systems, or ingest data from upstream systems and transform it for further processing. You will be part of the SaaS Data Layer implementation team responsible for building and maintaining the client environments of the new trading platform. Luxoft will invest into your professional training and allow you to grow your professional career.ResponsibilitiesCloud expertise in Azure Databricks, Azure DevOps, Azure Log Analytics. Knowledge of Azure network concepts, Azure Service Bus, Azure Functions is a plus. Develop, implement, and maintain data pipelines and ETL processes using Azure Databricks. Design and manage Azure Databricks for optimized data storage and retrieval. Collaborate with cross-functional teams to define and implement data integration solutions across multiple sources and platforms. Implement best practices for data governance, security, and compliance in cloud-based solutions. Monitor and troubleshoot performance issues in Azure-based applications and data pipelines. Automate deployment and monitoring processes using CI/CD pipelines. Stay updated with emerging Azure technologies and provide recommendations for improvements. Producing Architectural design diagram/Technical DocumentationSKILLSMust have10+ years experience in development 3+ years of hands-on experience with Azure cloud services, including Databricks and DevOps. Strong proficiency Python and Pyspark or SQL for data transformation and