Python Environment Setup

Installing Python

Python is the dominant language for data analysis. Before writing a single line of analysis code, you need a working Python installation. This chapter covers what to install, why the version matters, and how to verify everything works.

Why Python 3.11 or Later

Python 2 reached end-of-life in January 2020. Every modern data library — pandas, numpy, scikit-learn — targets Python 3. Install Python 3.11 or newer. Version 3.11 brought significant performance improvements (10-60% faster than 3.10 in many benchmarks), and 3.12 builds on that. If you install an older version, you will hit compatibility walls with current library releases.

Installation Steps

Windows: Download the installer from python.org. During installation, check the box that says "Add Python to PATH." This is the single most common setup failure — without it, the terminal cannot find the python command. After installation, open a new terminal and verify:

python --version
# Expected: Python 3.11.x or higher

macOS: Use Homebrew (brew install python@3.12) or the official installer. The system Python on macOS is managed by Apple and should not be modified.

Linux: Use your distribution's package manager. On Ubuntu: sudo apt install python3.12 python3.12-venv python3-pip.

Verifying the Installation

Run these commands to confirm Python is working correctly:

python --version
pip --version

Both should report version numbers without errors. If pip is missing on Linux, install it with sudo apt install python3-pip.

The Python REPL

Python ships with an interactive shell called the REPL (Read-Eval-Print Loop). Type python in your terminal and you will see the >>> prompt. This is useful for quick experiments:

>>> 2 + 2
4
>>> import sys
>>> sys.version
'3.11.5 (main, Sep 11 2023, ...)'

The REPL evaluates each line immediately. It is ideal for testing small code snippets before putting them in a script. For anything longer than a few lines, use a script file or Jupyter Notebook.

Choosing an Editor

You need a code editor. The two most popular choices for data analysis are:

• VS Code: Free, extensible, excellent Python support via the official extension. Install the Python extension, then open a .py file and click Run.
• JupyterLab: The next-generation Jupyter interface. Comes with the jupyterlab package. Provides a file browser, terminal, and notebook interface in one window.

Both work well. VS Code is better for writing scripts and modules. JupyterLab is better for interactive analysis. Many analysts use both.

Virtual Environments

A virtual environment is an isolated Python installation for a single project. It has its own packages, its own pip, and its own Python binary. When you install pandas in a virtual environment, it does not affect the system Python or any other project.

Why Isolation Matters

Without virtual environments, every project on your machine shares one global package set. This creates dependency conflicts. Project A needs pandas 1.5, Project B needs pandas 2.1. You cannot have both installed globally. Virtual environments solve this by giving each project its own package directory.

Beyond conflicts, virtual environments make your work reproducible. You can export a list of exact package versions (pip freeze > requirements.txt) and recreate the identical environment on another machine. This is essential for collaboration and deployment.

Creating a Virtual Environment

Python 3.3+ includes the venv module in the standard library. No additional installation is needed.

# Create a virtual environment in a folder named .venv
python -m venv .venv

This creates a .venv directory containing a copy of the Python binary and a site-packages folder where libraries will be installed.

Activating the Virtual Environment

Activation modifies your shell's PATH so that python and pip point to the virtual environment instead of the system Python.

Windows (PowerShell):

.venv\Scripts\Activate.ps1

Windows (Command Prompt):

.venv\Scripts\activate.bat

macOS/Linux:

source .venv/bin/activate

When activated, your prompt changes to show the environment name:

(.venv) E:\Amazon\AmazonPrep>

Installing Packages in the Virtual Environment

With the environment active, use pip to install packages:

pip install pandas numpy matplotlib

These packages install only in the virtual environment. The system Python remains clean.

Deactivating

When you are done working, deactivate the environment:

deactivate

This restores your shell to the system Python. You can re-activate the environment at any time by running the activation command again.

Best Practices

1. One environment per project. Do not share environments across projects.
2. Never commit .venv to version control. Add .venv/ to your .gitignore.
3. Pin your dependencies. Use pip freeze > requirements.txt to record exact versions.
4. Recreate from requirements. On a new machine, pip install -r requirements.txt reproduces the environment.

Jupyter Notebook

Jupyter Notebook is an interactive computing environment where you can write code, execute it, and see the output — including tables, charts, and formatted text — in a single document. It is the standard tool for exploratory data analysis.

Why Jupyter for Data Analysis

Traditional scripts run top-to-bottom and produce output at the end. Analysis is iterative: you load data, look at it, form a hypothesis, test it, look again. Jupyter supports this workflow by letting you execute code in small cells and inspect results immediately.

Each cell can be re-run independently. If you discover an error in your data cleaning logic, you fix that one cell and re-run it without re-executing the entire notebook. This tight feedback loop is why Jupyter dominates data analysis.

Installing and Launching

Install Jupyter in your virtual environment:

pip install jupyter

Launch it from your project directory:

jupyter notebook

This opens a browser window showing the file browser. Create a new notebook by clicking "New" and selecting your Python kernel.

Notebook Structure

A notebook is a sequence of cells. Each cell is either code or markdown.

Code cells contain Python. Run a cell with Shift+Enter (runs the cell and moves to the next) or Ctrl+Enter (runs and stays). Output appears directly below the cell.

import pandas as pd

df = pd.DataFrame({
    'product': ['A', 'B', 'C'],
    'sales': [150, 230, 180]
})

df

Markdown cells contain formatted text. Use them to document your analysis, explain your reasoning, and record findings. A notebook that mixes code, output, and explanation is a self-contained analysis report.

Keyboard Shortcuts

Learn these shortcuts — they dramatically speed up notebook work:

Best Practices for Notebooks

1. Restart and Run All before sharing. This ensures the notebook runs cleanly from top to bottom.
2. Use markdown cells to document. Explain what each section does and why.
3. Keep cells small. Each cell should do one thing.
4. Save outputs. If a chart or table is important, save it as an image or export the data.
5. Version control. Notebooks can be committed to git, but clear outputs first to avoid noise in diffs.

Installing Data Libraries

The Python data ecosystem is built on a small set of core libraries. This chapter covers what each one does and how to install them.

The Core Libraries

pandas is the primary data manipulation library. It provides DataFrame and Series objects for working with tabular data — the Python equivalent of a spreadsheet or SQL table. Every analysis workflow starts with pandas.

numpy is the numerical computing foundation. pandas is built on top of numpy. It provides fast array operations and mathematical functions. You will use numpy directly for numerical computations and indirectly through every pandas operation.

matplotlib is the plotting library. It creates static visualizations — line charts, bar charts, scatter plots, histograms. Seaborn (built on matplotlib) provides a higher-level interface for statistical plots.

Installation

With your virtual environment activated:

pip install pandas numpy matplotlib

This installs the latest stable versions of all three. pip resolves dependencies automatically — installing pandas will pull in numpy and python-dateutil as dependencies.

Verifying the Installation

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt

print(pd.__version__)
print(np.__version__)

If this runs without errors, your environment is ready.

Understanding the Dependency Chain

pandas does not implement its own array storage or mathematical functions. It delegates to numpy. When you create a pandas DataFrame, the underlying data is stored in numpy arrays. When you call df.mean(), pandas calls numpy's mean function.

This matters for two reasons:

1. Performance: numpy operations are implemented in C and are orders of magnitude faster than equivalent Python loops. When you use pandas vectorized operations (e.g., df['sales'] * 1.1), you are using numpy under the hood.
2. Compatibility: numpy arrays have a well-defined memory layout. This allows pandas to interface with other libraries (scikit-learn, TensorFlow) that expect numpy arrays as input.

Additional Libraries to Know

Beyond the core three, these libraries are commonly used in data analysis:

• scipy: Statistical tests, optimization, signal processing.
• scikit-learn: Machine learning algorithms and tools.
• statsmodels: Statistical modeling and hypothesis testing.
• openpyxl: Read and write Excel files.
• sqlalchemy: Connect to databases from Python.

Install them as needed. Do not install everything upfront — it creates confusion about which tool to use for a given task.

Keeping Packages Updated

Update packages regularly to get bug fixes and performance improvements:

pip install --upgrade pandas

Or update all packages listed in requirements.txt:

pip install --upgrade -r requirements.txt

Test your notebooks after updating. New versions can introduce breaking changes, especially in major version bumps (e.g., pandas 1.x to 2.x).