Here we learn a little more about Azure Synapse Analytics and Azure Machine Learning and how they help data scientists and others get the most out of their data without coding and machine learning expertise. We also build a roadmap of how we can approach using machine learning to analyze our data, in this case, home sales.
These days, it seems that every second business claims their product is powered by artificial intelligence (AI). Unfortunately, while every C-suite is chasing that buzzword, only a few understand the challenges.
A machine learning (ML) model is only as good as the data it ingests. To develop a model that delivers desired business outcomes, we need to use appropriate data to feed our model.
Some might argue that the meaning of "machine learning" is that the machine finds hidden patterns and learns independently. True, but how? Someone gives data to the machine for creating these "smart" algorithms and models. The responsibility of extracting, analyzing, and getting insights from data falls on data scientists.
Traditionally, it’s been a bit painful to include AI and ML in a data science pipeline. Data science and ML teams often work in separate silos. Sharing models and data, however necessary, can be time-consuming and inefficient.
Azure Synapse Analytics offers a smoother path for teams looking to supercharge their data science workflows with AI and ML. In this article, we’ll discuss Azure Synapse Analytics and Azure Machine Learning. In the following articles of the series, we’ll explore how to set up workspaces and ML and how to train and use our ML models and enhance the data.
What is Azure Synapse Analytics?
Azure Synapse Analytics is an integrated analytics service to extract meaningful insights across data lakes and data warehouses quickly. It processes, monitors, and secures data all in one place, and provides us data scientists the freedom to query data on our terms. The service also enables enterprise-wide descriptive, diagnostic, predictive, and prescriptive analytics.
Azure Synapse Analytics is the rebranded Azure SQL Data Warehouse (SQL DW) with improved performance and capabilities. It enables limitless pipelines for extract, transform, and load (ETL) and extract, load, and transform (ELT) operations. It bridges the gap between traditional SQL data warehousing, unstructured data stores, and Apache Spark pools, while fully supporting the continuously growing DevOps ecosystem.
Moreover, Azure Synapse Studio enables us to monitor, code, and secure the data and data pipelines using a single interface. Its deep integrations with other Azure services, like Azure Machine Learning (Azure ML) and Power BI, make it a perfect choice for any of your enterprise’s use cases.
The following image summarizes the tools and services Azure Synapse Analytics offers.
What is Azure Machine Learning?
In this series of articles, we’ll be exploring Azure Machine Learning integration. Azure Machine Learning creates and manages ML solutions. Microsoft designed the cloud-based service for data scientists and engineers to leverage their existing data processing capabilities and model development skills without learning a different framework or tool.
While Azure Machine Learning enables users to custom build and deploy ML models in the traditional manner, it requires significant domain knowledge and time to produce and compare dozens of models. So, to aid users of all levels of expertise, automated machine learning (AutoML) helps build highly scalable, efficient, and productive ML models.
Azure Synapse Analytics’ integration with AutoML enables us to enrich datasets and transform data into actionable baseline models without writing a single line of ML code.
The Azure Cognitive Services suite provides ML capabilities to solve general problems. These services include components like data, algorithms, and pre-trained models in a general ML solution. These offerings help users wrangle insights out of their data without needing much expertise or experience with ML or data science.
Our Data Project
We’ll explore Azure Synapse Analytics and demonstrate its end-to-end Azure Machine Learning and Cognitive Services integration in this series of articles. We'll learn to connect an Azure Synapse workspace to an Azure Machine Learning workspace using a linked service.
We’ll also discover how to enrich our data by training and evaluating a model using Azure Machine Learning. Besides pre-trained ML models, we’ll explore using Azure Cognitive Services to enrich our data in an SQL pool.
When our data is ready, we’ll use Azure Machine Learning to train a ML model for predictive analytics. In this case, we’ll analyze house sales data for King County, USA, and see what insights our model can find.
The project will follow this structure:
- Creating and configuring an Azure Synapse Analytics workspace
- Creating and configuring an Azure Machine Learning linked service
- Importing the data
- Enriching the data using a pretrained Azure ML Model
- Enriching the data using Azure Cognitive Services
- Training a prediction model using AutoML
For this tutorial, we assume you already have some SQL knowledge and are somewhat familiar with Azure Synapse Analytics. Therefore, we won’t go into details of individual features. However, you may want to explore some other articles first or register to view the Hands-on Training Series for Azure Synapse Analytics. Led by Microsoft engineering team members, this webinar series dives deep into Azure Synapse.
What’s Next?
You now know a little more about Azure Synapse Analytics and Azure Machine Learning and how they help data scientists and others get the most out of their data without coding and machine learning expertise. We also built a roadmap of how we can approach using machine learning to analyze our data, in this case, home sales.
In the next article of this series, we’ll jump right into setting up an Azure Synapse workspace and Azure Synapse Studio to prepare for our machine learning analysis in the third article.
