PANDAS:Selecting, filtering, and assigning

Columns by name, rows by condition

penguins["species"] returns a Series. penguins[["species", "island"]] returns a DataFrame, because the inner list means "these columns". One pair of brackets versus two is the most common mix-up in Pandas.

import pandas as pd

penguins = pd.read_csv("penguins.csv")
species = penguins["species"]
slim = penguins[["species", "body_mass_g"]]
print(type(species).__name__, type(slim).__name__)
print(slim.head(2))

Output:

Series DataFrame
  species  body_mass_g
0  Adelie       3750.0
1  Adelie       3800.0

Filter with a mask

The comparison penguins["species"] == "Gentoo" is a Series of True and False, the same idea as a NumPy mask. Put it in .loc to keep the matching rows.

import pandas as pd

penguins = pd.read_csv("penguins.csv")
gentoo = penguins.loc[penguins["species"] == "Gentoo", ["island", "body_mass_g"]]
print(gentoo.shape)
print(gentoo.head(2))

Output:

(124, 2)
    island  body_mass_g
152  Biscoe       4500.0
153  Biscoe       5700.0

.loc[rows, columns] takes a row selector and a column selector. Either side can be a mask, a list of labels, or a slice.

Combine conditions with & and |, each comparison in parentheses:

import pandas as pd

penguins = pd.read_csv("penguins.csv")
heavy_adelie = penguins.loc[
    (penguins["species"] == "Adelie") & (penguins["body_mass_g"] >= 4000)
]
print(len(heavy_adelie))

Output:

22

Assign a new column

import pandas as pd

penguins = pd.read_csv("penguins.csv")
penguins["body_mass_kg"] = penguins["body_mass_g"] / 1000
print(penguins[["species", "body_mass_kg"]].head(2))

Output:

  species  body_mass_kg
0  Adelie          3.75
1  Adelie          3.80

Assignment with [ writes the column on the original DataFrame. Chained assignment such as df[mask]["col"] = ... sometimes writes a copy and leaves the original unchanged. Prefer .loc[mask, "col"] = ... when you are filling values on a subset.

What to notice

• .loc uses labels. .iloc uses integer positions. If the index is the default 0..n-1 they look the same, until you filter and the index is no longer a sequence of positions.
• A filter does not delete rows from the original table unless you assign the result back.
• query("species == 'Gentoo'") is a readable alternative for simple filters. The mask form is what you should be able to read in other people's code.

Try this

From the penguins table, select Chinstrap penguins on Dream island and keep only bill_length_mm and sex. Print how many rows you kept.

Next: missing values, and why the dtype of a column sometimes changes under you.