Essential Python Succinctly®
CHAPTER 12
Data Science and ML
Quick intro
Python is a powerful language in data science and machine learning, owing to its versatility and the rich ecosystem of libraries designed for data manipulation, scientific computing, and advanced machine learning techniques.
Libraries like Pandas, NumPy, scikit-learn, TensorFlow, and PyTorch are instrumental in helping data scientists and machine learning engineers process, analyze, and model data effectively.
Note: You can run the following code examples by opening the built-in terminal within VS Code and executing the command: python <script>.py. Replace <script> with the name of the respective Python file to execute.
Working with data using Pandas
Pandas provides powerful data manipulation capabilities, making it an essential library for handling large datasets, performing data analysis, and cleaning data.
Let’s look at a fundamental data manipulation example with Pandas.
Code Listing 12-a: sample_data.csv
column_name,category_column,value_column 45,Category A,150 67,Category B,200 89,Category A,300 34,Category C,400 78,Category B,250 56,Category A,350 23,Category C,500 49,Category B,180 66,Category A,270 92,Category C,330 |
Code Listing 12-b: basicpandas.py
# Importing Pandas import pandas as pd # Loading a CSV file data = pd.read_csv('sample_data.csv') # Displaying the first few rows of the dataset print(data.head()) # Descriptive statistics print(data.describe()) # Filtering data filtered_data = data[data['column_name'] > 50] # Aggregating data grouped_data = data.groupby('category_column')['value_column'].mean() print(grouped_data) |
Explanation:
· Loading data: pd.read_csv loads data from a CSV file into a DataFrame, Pandas’s primary data structure.
· Viewing data: data.head() shows the first few rows, giving you a quick preview.
· Descriptive statistics: data.describe() provides statistical summaries, like mean and standard deviation, for numerical columns.
· Filtering data: Filters rows where column_name values are greater than 50.
· Aggregation: Groups data by category_column and calculates the mean of value_column for each category.
Numerical computing with NumPy
NumPy is designed for numerical operations, supporting arrays, matrix operations, and high-level mathematical functions. Let’s consider the following code that performs basic array operations with NumPy.
Code Listing 12-c: basicnumpy.py
# Importing NumPy import numpy as np # Creating an array arr = np.array([1, 2, 3, 4, 5]) # Performing mathematical operations arr_squared = arr ** 2 # Creating a 2D array matrix = np.array([[1, 2], [3, 4]]) # Matrix multiplication result = np.dot(matrix, matrix) print("Matrix multiplication result:\n", result) |
Explanation:
· Array creation: np.array creates a NumPy array, which is more efficient than a regular Python list. The np.array function infers a data type that depends on the data passed to it, type np.int64 in this example.
· Element-wise operations: arr ** 2 squares each element in the array.
· 2D arrays and matrix multiplication: We create a 2D array (matrix) and use np.dot for matrix multiplication, an essential operation in linear algebra for machine learning.
Traditional machine learning with scikit-learn
scikit-learn provides a comprehensive suite of machine learning algorithms for classification, regression, clustering, and more, along with utilities for preprocessing and evaluation.
Let’s look at an example of how to build a simple linear regression model with scikit-learn.
Code Listing 12-d: regmodel.py
# Importing necessary modules from scikit-learn from sklearn.model_selection import train_test_split from sklearn.linear_model import LinearRegression from sklearn.metrics import mean_squared_error # Sample data X = [[1], [2], [3], [4], [5]] y = [2, 4, 6, 8, 10] # Splitting data into training and testing sets X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42) # Initializing and training the model model = LinearRegression() model.fit(X_train, y_train) # Making predictions predictions = model.predict(X_test) # Evaluating the model mse = mean_squared_error(y_test, predictions) print("Mean Squared Error:", mse) |
Explanation:
· Data splitting: train_test_split divides data into training and testing sets for model evaluation.
· Model initialization and training: We initialize and train a LinearRegression model using fit.
· Prediction and evaluation: predict generates forecasts on the test data, and mean_squared_error computes the error between actual and predicted values, providing insight into model accuracy.
Simple neural network with TensorFlow
TensorFlow and PyTorch offer sophisticated deep learning capabilities for more complex tasks, such as image recognition and natural language processing, enabling developers to create and train neural networks.
Let’s look at a super simple example of how to build a neural network with TensorFlow.
Code Listing 12-e: nn.py
# Importing TensorFlow import tensorflow as tf # Sample dataset: Simple XOR problem X_train = [[0, 0], [0, 1], [1, 0], [1, 1]] y_train = [[0], [1], [1], [0]] # Define a Sequential model model = tf.keras.Sequential([ tf.keras.layers.Dense(8, activation='relu', input_shape=(2,)), tf.keras.layers.Dense(1, activation='sigmoid') ]) # Compile the model model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy']) # Train the model model.fit(X_train, y_train, epochs=100, verbose=0) # Make predictions predictions = model.predict(X_train) print("Predictions:\n", predictions) |
Explanation:
· Data preparation: We define the XOR problem as a dataset for training.
· Model definition: A Sequential model is created with two layers—one with 8 neurons and rectified linear unit (relu) activation, and another with 1 neuron and sigmoid activation for binary output.
· Compilation and training: compile defines the optimizer, loss function, and metrics, and fit trains the model for 100 epochs.
· Prediction: predict provides model outputs for the training set, which should approximate XOR results after training.
Deep learning with PyTorch
Here’s an example of how to build the same XOR neural network model with PyTorch.
Code Listing 12-f: dl.py
# Importing PyTorch import torch import torch.nn as nn import torch.optim as optim # Sample data X_train = torch.tensor([[0, 0], [0, 1], [1, 0], [1, 1]], dtype=torch.float32) y_train = torch.tensor([[0], [1], [1], [0]], dtype=torch.float32) # Define the model class XORModel(nn.Module): def __init__(self): super(XORModel, self).__init__() self.layer1 = nn.Linear(2, 8) self.layer2 = nn.Linear(8, 1) def forward(self, x): x = torch.relu(self.layer1(x)) x = torch.sigmoid(self.layer2(x)) return x # Instantiate the model, define loss function and optimizer model = XORModel() criterion = nn.BCELoss() optimizer = optim.Adam(model.parameters(), lr=0.01) # Training loop for epoch in range(100): optimizer.zero_grad() # Reset gradients output = model(X_train) # Forward pass loss = criterion(output, y_train) # Compute loss loss.backward() # Backward pass optimizer.step() # Update weights # Make predictions with torch.no_grad(): predictions = model(X_train) print("Predictions:\n", predictions) |
Explanation:
· Data preparation: We define the XOR dataset with torch.tensor.
· Model definition: XORModel defines a two-layer neural network with relu and sigmoid activations.
· Loss and optimizer: Binary cross-entropy (BCE) is used as the loss function, while Adam optimizer updates model weights.
· Training loop: The loop performs forward and backward passes, updating weights with each epoch.
· Prediction: After training, predictions are made without tracking gradients (with torch.no_grad()), which should approximate the XOR pattern.
Recap
Python libraries like Pandas, NumPy, scikit-learn, TensorFlow, and PyTorch enable you to handle data efficiently, create machine learning models, and build deep learning architectures for complex tasks.
Mastering these libraries prepares you to work across the data science pipeline, from data preparation to model deployment, making Python an invaluable tool for data science and machine learning applications.
As these libraries are extensive and complex—and we’ve barely scratched the surface of what you can accomplish with them—I encourage you to dive deeper into each one if you are into data science and machine learning. The goal of this chapter was to give you a quick taste of their capabilities.
- 1800+ high-performance UI components.
- Includes popular controls such as Grid, Chart, Scheduler, and more.
- 24x5 unlimited support by developers.