pandas Fundamentals (Data Science)

Series, DataFrame, and Inspection

A Series is a labeled 1-D array; a DataFrame is a dict of aligned Series sharing an index:

import pandas as pd

s = pd.Series([10, 20], index=["a", "b"], dtype="int64")
df = pd.DataFrame({"name": ["ann", "bo"], "score": [91, 84]})
df = pd.read_csv("scores.csv", usecols=["name", "score"])   # dtype inferred per column

df.head(); df.info(); df.describe()      # dtypes, nulls, five-number summary
df["score"]            # Series — always prefer bracket access
df.shape, df.columns, df.index

Column selection returns views of labels, not positions: df.loc["row1", "score"] is label-based, df.iloc[0, 1] is positional. Index alignment (a labeled Series keyed by id) is what makes vectorized joins between Series possible without manual matching.

Filter, Group, and Join

top = df[df["score"] > 85]
tagged = df.assign(passed=df["score"] >= 60)

summary = (df.groupby("class")              # split-apply-combine
             .agg(mean_score=("score", "mean"),
                  n=("score", "size"))
             .sort_values("mean_score", ascending=False))

merged = df.merge(roster, on="student_id", how="left")   # SQL-style join
df.query("score > 85 and class == 'A'")    # readable filter syntax

groupby follows three phases — split by key, apply an aggregation/transform/filter, combine results. how selects the join: inner, left, outer, .join aligns on index. Combine with sort_values, drop_duplicates, and value_counts() for the tabulations interviewers ask you to produce live.

Missing Data and Mutation Pitfalls

Missing values arrive as NaN (float columns) or pd.NA:

df.isna().sum()
df.dropna(subset=["score"])
df["score"] = df["score"].fillna(df["score"].median())

Chained assignment — df[df.a > 0]["b"] = 1 — may silently do nothing (SettingWithCopyWarning) because the filter produced a copy. Always assign through loc: df.loc[df["a"] > 0, "b"] = 1. Recent pandas (3.x) makes copy-on-write the default, so chained writes stop propagating instead of warning.

Index alignment: adding two Series pairs values by label — mismatches yield NaN. Reset with reset_index(drop=True) when labels are meaningless positions.

Interview angle: describe split-apply-combine, contrast merge (key join) with groupby (aggregate), and explain how copy-on-write removes an entire class of surprise mutations.