{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 课堂练习：帕尔默群岛三种企鹅的形态数据\n",
    "\n",
    "数据：`../data/penguins.csv`（如果你只拿到了这一个文件夹，把 `penguins.csv` 放在本文件旁边即可）。\n",
    "这是南极帕尔默群岛 2007–2009 年测量的 344 只阿德利企鹅、帽带企鹅和巴布亚企鹅的形态数据，\n",
    "来自 Gorman、Williams 和 Fraser（2014，PLoS ONE 9: e90081），以 CC0 许可公开。\n",
    "\n",
    "练习分三部分，分别在第 1–3 周的上机课上完成，每部分约 80 分钟：\n",
    "\n",
    "| 部分 | 内容 | 对应周次 |\n",
    "|---|---|---|\n",
    "| A | 读数据、缺测值、样本构成、描述统计、分组作图 | 第 1 周 |\n",
    "| B | 两性体重差异：正态性检查、Welch t 检验、置信区间、Holm 校正、Mann–Whitney U | 第 2 周 |\n",
    "| C | 回归：物种内与合并拟合、辛普森悖论、把物种放进模型 | 第 3 周 |\n",
    "\n",
    "**怎么做**：从上到下逐块运行。看到 `...` 或 `TODO` 的地方补全代码；每题下面的“答题区”写数值结果和你的解释\n",
    "（数值保留到题目或论文表 1 的小数位，解释用完整的句子）。C6 是挑战题，不做不扣分。\n",
    "\n",
    "**自查**：每道题的代码后面有一个“自查”格。运行它，算对的项目打印 `✓ 通过`，不对的打印 `✗` 和一句提示，\n",
    "告诉你该回去看哪里。自查只检查数值，不检查解释和图的好坏；它不会告诉你正确答案是多少。\n",
    "自查格要用到上面“自查工具”格里的函数，所以那一格也要运行。\n",
    "\n",
    "**运行环境**：Python 3.10 以上，需要 pandas、matplotlib、scipy（`pip install pandas matplotlib scipy`）。\n",
    "在 Jupyter 中打开 `exercise.ipynb`，或在 VS Code / PyCharm / Spyder 中按 `# %%` 分块运行 `exercise.py`。\n",
    "作为脚本运行时，图保存在 `my_output/`。\n",
    "\n",
    "**提交**：每部分课后提交本文件（`.ipynb` 或 `.py`，要能从头到尾运行不报错）和生成的图。评分标准见课程主页。"
   ],
   "id": "cell-00"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "source": [
    "import sys\n",
    "from pathlib import Path\n",
    "\n",
    "import matplotlib\n",
    "\n",
    "if \"ipykernel\" not in sys.modules:  # 作为脚本运行时不弹窗口，只存图\n",
    "    matplotlib.use(\"Agg\")\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "from scipy import stats\n",
    "\n",
    "try:\n",
    "    HERE = Path(__file__).resolve().parent\n",
    "except NameError:  # Jupyter 中没有 __file__\n",
    "    HERE = Path.cwd()\n",
    "\n",
    "\n",
    "def find_data(name=\"penguins.csv\"):\n",
    "    \"\"\"依次在 ../data/、当前目录、./data/ 中找数据文件。\"\"\"\n",
    "    for p in (HERE.parent / \"data\" / name, HERE / name, HERE / \"data\" / name):\n",
    "        if p.exists():\n",
    "            return p\n",
    "    raise FileNotFoundError(f\"找不到 {name}，请把它放在本文件旁边或 ../data/ 中\")\n",
    "\n",
    "\n",
    "OUT = HERE / \"my_output\"\n",
    "OUT.mkdir(exist_ok=True)\n",
    "\n",
    "SPECIES = [\"Adelie\", \"Chinstrap\", \"Gentoo\"]\n",
    "MORPH = [\"bill_length_mm\", \"bill_depth_mm\", \"flipper_length_mm\", \"body_mass_g\"]\n",
    "COLORS = {\"Adelie\": \"#D55E00\", \"Chinstrap\": \"#0072B2\", \"Gentoo\": \"#009E73\"}  # 与论文图 1 相同\n",
    "LABEL = {\"Adelie\": \"Adélie\", \"Chinstrap\": \"Chinstrap\", \"Gentoo\": \"Gentoo\"}\n",
    "\n",
    "plt.rcParams.update({\n",
    "    \"figure.dpi\": 110, \"savefig.dpi\": 200, \"font.size\": 10,\n",
    "    \"axes.spines.top\": False, \"axes.spines.right\": False,\n",
    "})\n",
    "\n",
    "\n",
    "def finish(fig, name):\n",
    "    \"\"\"保存图；作为脚本运行时随即关闭，在 Jupyter 中留给 inline 后端显示。\"\"\"\n",
    "    fig.savefig(OUT / f\"{name}.png\", bbox_inches=\"tight\")\n",
    "    if \"ipykernel\" not in sys.modules:\n",
    "        plt.close(fig)"
   ],
   "execution_count": null,
   "outputs": [],
   "id": "cell-01"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "source": [
    "# 自查工具：运行这一格即可，不用读懂或修改。\n",
    "# 每题代码后面的“自查”格调用这里的 check()：数值算对了打印 ✓ 通过，不对就打印一句提示。\n",
    "# 正确答案只以哈希值保存在自查格里，看不到具体数字。\n",
    "import hashlib\n",
    "\n",
    "SELFCHECK_FAILED = []\n",
    "\n",
    "\n",
    "def _flat(v):\n",
    "    if hasattr(v, \"to_numpy\"):\n",
    "        v = v.to_numpy()\n",
    "    if isinstance(v, np.ndarray):\n",
    "        v = v.ravel().tolist()\n",
    "    if isinstance(v, (list, tuple)):\n",
    "        return [x for item in v for x in _flat(item)]\n",
    "    return [v]\n",
    "\n",
    "\n",
    "def _key(v, digits):\n",
    "    parts = []\n",
    "    for x in _flat(v):\n",
    "        if x is Ellipsis:\n",
    "            raise ValueError(\"ellipsis\")\n",
    "        s = f\"{float(x):.1e}\" if digits == \"e\" else f\"{float(x):.{digits}f}\"\n",
    "        parts.append(s[1:] if s.startswith(\"-\") and float(s) == 0 else s)\n",
    "    return hashlib.sha256(\"|\".join(parts).encode()).hexdigest()[:10]\n",
    "\n",
    "\n",
    "def check(qid, items):\n",
    "    print(f\"—— 自查 {qid} ——\")\n",
    "    for label, get, digits, good, hint, *known in items:\n",
    "        try:\n",
    "            k = _key(get(), digits)\n",
    "        except NameError as e:\n",
    "            msg = f\"还没有定义 {getattr(e, 'name', '')}：先运行上面的代码格\"\n",
    "        except Exception as e:\n",
    "            if \"ellipsis\" in str(e).lower():\n",
    "                msg = \"还没完成：上面的代码里还有 ... 没有填\"\n",
    "            else:\n",
    "                msg = f\"算不出来（{type(e).__name__}）：确认上面的代码已运行，变量名和结构与框架一致\"\n",
    "        else:\n",
    "            if k == good:\n",
    "                print(f\"✓ 通过　{label}\")\n",
    "                continue\n",
    "            msg = (known[0] if known else {}).get(k, hint)\n",
    "        SELFCHECK_FAILED.append(f\"{qid} {label}\")\n",
    "        print(f\"✗ {label}：{msg}\")\n",
    "\n",
    "\n",
    "def reflect(qid, prompts):\n",
    "    print(f\"—— 自查 {qid} ——（讨论题不自动判分，交之前对照检查）\")\n",
    "    for p in prompts:\n",
    "        print(\"□\", p)\n",
    "\n",
    "\n",
    "def _mf(sp):\n",
    "    \"\"\"B3、B4 自查用：sexed 中某个物种雄性、雌性的体重。\"\"\"\n",
    "    d = sexed[sexed[\"species\"] == sp]\n",
    "    return d.loc[d[\"sex\"] == \"male\", \"body_mass_g\"], d.loc[d[\"sex\"] == \"female\", \"body_mass_g\"]\n",
    "\n",
    "\n",
    "def _line_slopes(ax):\n",
    "    \"\"\"C3 自查用：图中每条直线的斜率，从小到大。\"\"\"\n",
    "    out = []\n",
    "    for ln in ax.get_lines():\n",
    "        x, y = np.asarray(ln.get_xdata(), float), np.asarray(ln.get_ydata(), float)\n",
    "        if len(x) >= 2 and x[-1] != x[0]:\n",
    "            out.append((y[-1] - y[0]) / (x[-1] - x[0]))\n",
    "    return sorted(out)"
   ],
   "execution_count": null,
   "outputs": [],
   "id": "cell-02"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## A 部分　认识数据（第 1 周）\n",
    "\n",
    "### A1　读入数据\n",
    "\n",
    "用 `pd.read_csv` 读入 `penguins.csv`（缺测值在文件里写作 `NA`），回答：\n",
    "\n",
    "1. 数据有几行、几列？每一行代表什么？\n",
    "2. 哪些变量是分类变量，哪些是数值变量？`year` 在文件里是整数，分析时应当把它当作哪一类？"
   ],
   "id": "cell-03"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "source": [
    "df = ...  # TODO：用 pd.read_csv 读入数据，路径用 find_data()，并指定 na_values=[\"NA\"]\n",
    "# TODO：查看行列数（.shape）、各列类型（.dtypes）和前几行（.head()）"
   ],
   "execution_count": null,
   "outputs": [],
   "id": "cell-04"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "source": [
    "# 自查（A1）：运行后 ✓ 表示通过，✗ 后面是提示。\n",
    "check('A1', [\n",
    "    ('行数和列数', lambda: df.shape, 0, '73e90472b8',\n",
    "     '对照 penguins.csv 检查：是不是读错了文件，或者把表头当成了数据？'),\n",
    "    ('四项形态测量是数值型', lambda: [df[c].dtype.kind == 'f' for c in MORPH], 0, 'bd85fe66f5',\n",
    "     '形态测量列应当是浮点数；如果是文本（object），检查 na_values 有没有写对'),\n",
    "])"
   ],
   "execution_count": null,
   "outputs": [],
   "id": "cell-05"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**答题区（A1）**\n",
    "\n",
    "（在这里写下数值结果和解释）"
   ],
   "id": "cell-06"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### A2　缺测值\n",
    "\n",
    "1. 每一列各缺多少个值？\n",
    "2. 有几条记录四项形态测量（喙长、喙深、鳍长、体重）全部缺失？分别是哪个物种？\n",
    "3. 有几条记录四项形态测量齐全、只缺性别？分别是哪个物种？有没有只缺一部分形态测量的记录？\n",
    "4. 如果一开始就写 `df = df.dropna()`，还剩几条记录？对“体重对鳍长的回归”来说，这样做有什么问题？"
   ],
   "id": "cell-07"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "source": [
    "# 1. TODO：每列缺测数（df.isna().sum()）\n",
    "\n",
    "no_morph = ...  # TODO：四项形态测量全部缺失的行（提示：df[MORPH].isna().all(axis=1)）\n",
    "sex_only = ...  # TODO：四项形态测量齐全、但缺性别的行\n",
    "# 2–3. TODO：分别数一数（.sum()），再按物种统计（df.loc[no_morph, \"species\"].value_counts()）\n",
    "\n",
    "# 4. TODO：len(df.dropna()) 是多少？\n",
    "\n",
    "# 后面几题要用的两个子集：\n",
    "morph = ...  # TODO：四项形态测量齐全的记录（df.dropna(subset=MORPH)）\n",
    "sexed = ...  # TODO：四项形态测量和性别都齐全的记录"
   ],
   "execution_count": null,
   "outputs": [],
   "id": "cell-08"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "source": [
    "# 自查（A2）：运行后 ✓ 表示通过，✗ 后面是提示。\n",
    "check('A2', [\n",
    "    ('四项全缺的记录数', lambda: no_morph.sum(), 0, 'd4735e3a26',\n",
    "     '四项要“全部”缺失才算：看看你对 df[MORPH].isna() 用的是 all 还是 any，axis 对不对',\n",
    "     {'4fc82b26ae': '你数的是“任何一列有缺测”的记录；这里只看四项形态测量是否全部缺失'}),\n",
    "    ('只缺性别的记录数', lambda: sex_only.sum(), 0, '19581e27de',\n",
    "     '两个条件要同时满足：sex 缺失，并且四项形态测量都不缺',\n",
    "     {'4fc82b26ae': '这是 sex 缺失的总数；其中有的记录连形态测量也没有，要排除'}),\n",
    "    ('morph 的记录数', lambda: len(morph), 0, '023849c389',\n",
    "     'morph 只要求四项形态测量齐全：看 dropna 的 subset',\n",
    "     {'556d7dc3a1': '你把缺性别的记录也删了：morph 不要求有性别'}),\n",
    "    ('sexed 的记录数', lambda: len(sexed), 0, '556d7dc3a1',\n",
    "     'sexed 要求四项形态测量和性别都齐全：看 dropna 的 subset',\n",
    "     {'023849c389': 'sexed 还要去掉没有性别的记录'}),\n",
    "])"
   ],
   "execution_count": null,
   "outputs": [],
   "id": "cell-09"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**答题区（A2）**\n",
    "\n",
    "（在这里写下数值结果和解释）"
   ],
   "id": "cell-10"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### A3　样本构成\n",
    "\n",
    "1. 做出物种 × 岛屿的交叉表（带行列合计），以及各年份的记录数。\n",
    "2. 能否用这批数据比较“岛屿”对体重的影响？如果能，只能在哪个物种内比较？"
   ],
   "id": "cell-11"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "source": [
    "tab = ...    # TODO：物种 × 岛屿交叉表（pd.crosstab(..., margins=True)）\n",
    "years = ...  # TODO：各年份的记录数（df[\"year\"].value_counts().sort_index()）\n",
    "print(tab)\n",
    "print(years)"
   ],
   "execution_count": null,
   "outputs": [],
   "id": "cell-12"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "source": [
    "# 自查（A3）：运行后 ✓ 表示通过，✗ 后面是提示。\n",
    "check('A3', [\n",
    "    ('物种 × 岛屿交叉表', lambda: tab.loc[SPECIES, ['Biscoe', 'Dream', 'Torgersen']], 0, '642ae2c4cb',\n",
    "     '行是物种、列是岛屿，用全部记录 df',\n",
    "     {'4e1ed1c889': '这是用 morph 做的；样本构成要用全部记录 df'}),\n",
    "    ('各年份记录数', lambda: years.loc[[2007, 2008, 2009]], 0, '7609773af2',\n",
    "     '对 df 的 year 列计数',\n",
    "     {'45d1bba5a2': '这是用 morph 数的；这里要用全部记录 df'}),\n",
    "])"
   ],
   "execution_count": null,
   "outputs": [],
   "id": "cell-13"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**答题区（A3）**\n",
    "\n",
    "（在这里写下数值结果和解释）"
   ],
   "id": "cell-14"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### A4　复现论文表 1\n",
    "\n",
    "按物种和性别分组，计算四项形态测量的均值和标准差（保留与表 1 相同的小数位），以及每组的只数 n。\n",
    "每个物种再加一行“全部”（含未记录性别的个体）。\n",
    "\n",
    "提示：`groupby(...).agg([\"mean\", \"std\", \"count\"])`。思考：`pandas` 的 `.std()` 和 `numpy` 的 `np.std()`\n",
    "默认结果为什么不一样？表 1 用的是哪一种？"
   ],
   "id": "cell-15"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "source": [
    "by_sex = ...      # TODO：按物种 × 性别分组，计算 MORPH 四列的 mean、std、count\n",
    "# 提示：morph.dropna(subset=[\"sex\"]).groupby([\"species\", \"sex\"])[MORPH].agg([...])\n",
    "by_species = ...  # TODO：只按物种分组（含未记录性别的个体）再算一遍\n",
    "print(by_sex.round(1))\n",
    "print(by_species.round(1))\n",
    "\n",
    "# 两种标准差：\n",
    "x = morph.loc[morph[\"species\"] == \"Adelie\", \"body_mass_g\"]\n",
    "# TODO：比较 x.std() 与 np.std(x.to_numpy())"
   ],
   "execution_count": null,
   "outputs": [],
   "id": "cell-16"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "source": [
    "# 自查（A4）：运行后 ✓ 表示通过，✗ 后面是提示。\n",
    "check('A4', [\n",
    "    ('雌雄各组的只数', lambda: [by_sex.loc[(sp, sx), ('body_mass_g', 'count')] for sp in SPECIES for sx in ('female', 'male')], 0, '17240ab62d',\n",
    "     '按 (物种, 性别) 分组，数据用形态测量和性别都齐全的记录'),\n",
    "    ('雌雄各组的均值', lambda: [by_sex.loc[(sp, sx), (v, 'mean')] for sp in SPECIES for sx in ('female', 'male') for v in MORPH], 1, '58b39d1444',\n",
    "     '检查分组（物种, 性别）和数据子集；agg 里要有 \"mean\"'),\n",
    "    ('雌雄各组的标准差', lambda: [by_sex.loc[(sp, sx), (v, 'std')] for sp in SPECIES for sx in ('female', 'male') for v in MORPH], 1, '779ea83633',\n",
    "     '检查分组和数据子集；agg 里要有 \"std\"',\n",
    "     {'937075dc48': '你算的是 ddof=0 的标准差（除以 n）；表 1 用的是样本标准差'}),\n",
    "    ('“全部”一行的只数', lambda: [by_species.loc[sp, ('body_mass_g', 'count')] for sp in SPECIES], 0, '38ffdbc141',\n",
    "     '“全部”按物种分组，用形态测量齐全的记录',\n",
    "     {'5cf662e89b': '你用的是有性别的记录；“全部”应当包含未记录性别的个体'}),\n",
    "    ('“全部”一行的均值和标准差', lambda: [by_species.loc[sp, (v, s)] for sp in SPECIES for v in MORPH for s in ('mean', 'std')], 1, 'd06d4e3cad',\n",
    "     '“全部”按物种分组，用形态测量齐全的记录，agg 里要有 \"mean\" 和 \"std\"',\n",
    "     {'1293a62d38': '你用的是有性别的记录；“全部”应当包含未记录性别的个体',\n",
    "      '52c26a0dab': '标准差用的是 ddof=0（除以 n）；表 1 用的是样本标准差'}),\n",
    "])"
   ],
   "execution_count": null,
   "outputs": [],
   "id": "cell-17"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**答题区（A4）**\n",
    "\n",
    "（在这里写下数值结果和解释）"
   ],
   "id": "cell-18"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### A5　体重的中位数与四分位距\n",
    "\n",
    "1. 计算每个物种体重的中位数和四分位距（第 1、第 3 四分位数）。\n",
    "2. 帽带企鹅的第 1 四分位数为什么不是一个实际测到的体重值？换一种分位数算法会不会得到不同的数？\n",
    "3. 巴布亚企鹅的体重中位数比阿德利企鹅高多少克？相当于阿德利企鹅中位数的百分之几？"
   ],
   "id": "cell-19"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "source": [
    "q = ...  # TODO：每个物种体重的 25%、50%、75% 分位数\n",
    "# 提示：morph.groupby(\"species\")[\"body_mass_g\"].quantile([0.25, 0.5, 0.75]).unstack()\n",
    "print(q)\n",
    "\n",
    "chin = morph.loc[morph[\"species\"] == \"Chinstrap\", \"body_mass_g\"].to_numpy()\n",
    "# TODO：比较 np.percentile(chin, 25, method=m) 在 m = \"linear\"、\"lower\"、\"midpoint\" 时的结果\n",
    "\n",
    "gap = ...      # TODO：巴布亚企鹅与阿德利企鹅体重中位数之差（g），从 q 中取\n",
    "gap_pct = ...  # TODO：gap 相当于阿德利企鹅中位数的百分之几"
   ],
   "execution_count": null,
   "outputs": [],
   "id": "cell-20"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "source": [
    "# 自查（A5）：运行后 ✓ 表示通过，✗ 后面是提示。\n",
    "check('A5', [\n",
    "    ('各物种的三个四分位数', lambda: q.loc[SPECIES, [0.25, 0.5, 0.75]], 1, '6d6ba64981',\n",
    "     '用 morph 按物种分组，对 body_mass_g 取 0.25、0.5、0.75 分位数（保持默认算法），再 unstack'),\n",
    "    ('中位数之差', lambda: gap, 0, 'a037a7dfaf',\n",
    "     '巴布亚企鹅的中位数减阿德利企鹅的中位数，都从 q 的 0.5 那一列取',\n",
    "     {'d818f221cb': '你用的是均值；题目问的是中位数'}),\n",
    "    ('差值占阿德利企鹅中位数的百分比', lambda: gap_pct, 1, '6cc32a9a06',\n",
    "     '差值除以阿德利企鹅的中位数，再乘 100',\n",
    "     {'80a753d4a4': '分母应当是阿德利企鹅的中位数'}),\n",
    "])"
   ],
   "execution_count": null,
   "outputs": [],
   "id": "cell-21"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**答题区（A5）**\n",
    "\n",
    "（在这里写下数值结果和解释）"
   ],
   "id": "cell-22"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### A6　分组作图\n",
    "\n",
    "1. 画出不分物种的体重直方图。\n",
    "2. 再按物种分开画（叠加直方图、分面直方图或箱线图加散点均可）。\n",
    "3. 对比两张图：不分物种时的分布形状说明了什么？用“全部 342 只的均值 ± 标准差”来概括体重合适吗？"
   ],
   "id": "cell-23"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "source": [
    "fig, axes = plt.subplots(4, 1, figsize=(6, 6.6), sharex=True, sharey=True)\n",
    "bins = np.arange(2600, 6500, 100)\n",
    "# TODO：axes[0]：不分物种的体重直方图（axes[0].hist(..., bins=bins)），加小标题\n",
    "# TODO：axes[1]–axes[3]：每个物种一张直方图（color=COLORS[sp]），小标题写物种名和只数\n",
    "axes[-1].set_xlabel(\"Body mass (g)\")\n",
    "fig.supylabel(\"Number of penguins\")\n",
    "fig.tight_layout()\n",
    "finish(fig, \"A6_body_mass_hist\")"
   ],
   "execution_count": null,
   "outputs": [],
   "id": "cell-24"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "source": [
    "# 自查（A6）：运行后 ✓ 表示通过，✗ 后面是提示。\n",
    "check('A6', [\n",
    "    ('四张直方图各画了多少只', lambda: [sum(p.get_height() for p in a.patches) for a in fig.axes], 0, '3a7a05b339',\n",
    "     '自上而下应分别是：形态齐全的全部企鹅、阿德利、帽带、巴布亚企鹅（顺序同 SPECIES）；数据用 morph 的 body_mass_g，bins 用框架里给的'),\n",
    "])"
   ],
   "execution_count": null,
   "outputs": [],
   "id": "cell-25"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**答题区（A6）**\n",
    "\n",
    "（在这里写下数值结果和解释）"
   ],
   "id": "cell-26"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## B 部分　两性体重差异（第 2 周）\n",
    "\n",
    "本部分只用形态测量和性别都齐全的 333 只企鹅（A2 中的 `sexed`）。\n",
    "\n",
    "### B1　描述两性差异\n",
    "\n",
    "对每个物种，计算雌、雄体重的均值、标准差和只数，雄雌均值差（雄 − 雌），\n",
    "以及雄性均值比雌性均值高出的百分比：（雄性均值 / 雌性均值 − 1）× 100%。\n",
    "三个物种中，哪个物种体重的两性差异最大？哪个最小？"
   ],
   "id": "cell-27"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "source": [
    "g = ...  # TODO：sexed 按物种 × 性别分组，计算 body_mass_g 的 mean、std、count\n",
    "print(g.round(0))\n",
    "\n",
    "diff, pct = {}, {}  # 每个物种的雄雌均值差（g）和雄性高出的百分比（%）\n",
    "for sp in SPECIES:\n",
    "    diff[sp] = ...  # TODO：雄性均值 − 雌性均值\n",
    "    pct[sp] = ...   # TODO：(雄性均值 / 雌性均值 − 1) × 100"
   ],
   "execution_count": null,
   "outputs": [],
   "id": "cell-28"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "source": [
    "# 自查（B1）：运行后 ✓ 表示通过，✗ 后面是提示。\n",
    "check('B1', [\n",
    "    ('雌雄体重的均值', lambda: [g.loc[(sp, sx), 'mean'] for sp in SPECIES for sx in ('female', 'male')], 0, 'e128bc7934',\n",
    "     '用 sexed 按 (物种, 性别) 分组，对 body_mass_g 求均值'),\n",
    "    ('雄雌均值差', lambda: [diff[sp] for sp in SPECIES], 0, '5a118a04d4',\n",
    "     '每个物种的雄性均值减雌性均值',\n",
    "     {'945f2ff772': '方向反了：题目要的是雄 − 雌'}),\n",
    "    ('雄性高出的百分比', lambda: [pct[sp] for sp in SPECIES], 1, 'cb20d77874',\n",
    "     '对照题目里的公式：雄性均值除以雌性均值，减 1，再乘 100',\n",
    "     {'be6c918128': '分母应当是雌性均值',\n",
    "      'e638c0f53f': '别忘了减 1：要的是“高出”的百分比'}),\n",
    "])"
   ],
   "execution_count": null,
   "outputs": [],
   "id": "cell-29"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**答题区（B1）**\n",
    "\n",
    "（在这里写下数值结果和解释）"
   ],
   "id": "cell-30"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### B2　检查正态性\n",
    "\n",
    "对 6 个物种 × 性别组分别做 Shapiro–Wilk 检验，并画出正态 Q–Q 图（`scipy.stats.probplot`）。\n",
    "Shapiro–Wilk 检验 P > 0.05 能不能说明“数据服从正态分布”？"
   ],
   "id": "cell-31"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "source": [
    "fig, axes = plt.subplots(2, 3, figsize=(9, 5.4))\n",
    "sw = []  # 6 组的 Shapiro–Wilk P 值\n",
    "for j, sp in enumerate(SPECIES):\n",
    "    for i, sx in enumerate([\"female\", \"male\"]):\n",
    "        y = ...  # TODO：该物种、该性别的体重，转成 numpy 数组\n",
    "        # TODO：p = stats.shapiro(y).pvalue，并 sw.append(p)\n",
    "        # TODO：stats.probplot(y, dist=\"norm\", plot=axes[i, j])，小标题写上物种、性别和 P 值\n",
    "fig.tight_layout()\n",
    "finish(fig, \"B2_qq\")"
   ],
   "execution_count": null,
   "outputs": [],
   "id": "cell-32"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "source": [
    "# 自查（B2）：运行后 ✓ 表示通过，✗ 后面是提示。\n",
    "check('B2', [\n",
    "    ('6 组 Shapiro–Wilk 检验的 P 值', lambda: sorted(sw), 2, '79978914f0',\n",
    "     '对 6 个物种 × 性别组的体重（来自 sexed）分别做 stats.shapiro，把 .pvalue 依次放进 sw'),\n",
    "])"
   ],
   "execution_count": null,
   "outputs": [],
   "id": "cell-33"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**答题区（B2）**\n",
    "\n",
    "（在这里写下数值结果和解释）"
   ],
   "id": "cell-34"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### B3　Welch t 检验与均值差的置信区间\n",
    "\n",
    "1. 对每个物种，用 `scipy.stats.ttest_ind(雄, 雌, equal_var=False)` 做双侧 Welch t 检验，记下 t、自由度和 P。\n",
    "2. 不调用现成函数，按公式自己算出 t、Welch–Satterthwaite 自由度和雄雌均值差的 95% 置信区间：\n",
    "   SE = √(s₁²/n₁ + s₂²/n₂)，t = (x̄₁ − x̄₂)/SE，\n",
    "   df = (s₁²/n₁ + s₂²/n₂)² / [(s₁²/n₁)²/(n₁−1) + (s₂²/n₂)²/(n₂−1)]，\n",
    "   95% CI = (x̄₁ − x̄₂) ± t₀.₉₇₅,df × SE。和第 1 步的结果对一下。\n",
    "3. 再用 `equal_var=True`（Student t 检验）算一遍。哪些物种的 t 值完全没变？为什么？"
   ],
   "id": "cell-35"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "source": [
    "def welch(a, b, level=0.95):\n",
    "    \"\"\"a − b 的 Welch t 检验：返回 t、自由度、双侧 P、均值差及其置信区间的下限和上限。\"\"\"\n",
    "    a, b = np.asarray(a, float), np.asarray(b, float)\n",
    "    n1, n2 = len(a), len(b)\n",
    "    va = a.var(ddof=1) / n1  # s₁²/n₁\n",
    "    vb = b.var(ddof=1) / n2  # s₂²/n₂\n",
    "    diff = a.mean() - b.mean()\n",
    "    se = ...    # 填空 1：均值差的标准误（对照题目里 SE 的公式，用 va、vb 表示）\n",
    "    df_w = ...  # 填空 2：Welch–Satterthwaite 自由度（对照题目里 df 的公式，用 va、vb、n1、n2 表示）\n",
    "    t = diff / se\n",
    "    p = 2 * stats.t.sf(abs(t), df_w)\n",
    "    half = ...  # 填空 3：置信区间的半宽 = t 分布的 (0.5 + level / 2) 分位数（自由度 df_w）× se\n",
    "    return t, df_w, p, diff, diff - half, diff + half\n",
    "\n",
    "\n",
    "p_welch = {}  # 每个物种的 Welch P，B4 要用\n",
    "for sp in SPECIES:\n",
    "    d = sexed[sexed[\"species\"] == sp]\n",
    "    m = d.loc[d[\"sex\"] == \"male\", \"body_mass_g\"]\n",
    "    f = d.loc[d[\"sex\"] == \"female\", \"body_mass_g\"]\n",
    "    # TODO：比较 welch(m, f) 与 stats.ttest_ind(m, f, equal_var=False)\n",
    "    # TODO：p_welch[sp] = ...\n",
    "    # TODO：再看 stats.ttest_ind(m, f, equal_var=True) 的 t 值"
   ],
   "execution_count": null,
   "outputs": [],
   "id": "cell-36"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "source": [
    "# 自查（B3）：运行后 ✓ 表示通过，✗ 后面是提示。\n",
    "check('B3', [\n",
    "    ('t 统计量', lambda: [welch(*_mf(sp))[0] for sp in SPECIES], 1, 'f84e3a44cf',\n",
    "     't = diff / se，t 不对说明填空 1 的标准误不对：对照题目里 SE 的公式',\n",
    "     {'f8477d58c6': '填空 1：标准误要开平方根',\n",
    "      '8e2663caec': '填空 1：先把两项 s²/n 相加，再开平方根'}),\n",
    "    ('自由度', lambda: [welch(*_mf(sp))[1] for sp in SPECIES], 1, '058e6b7330',\n",
    "     '检查填空 2：对照题目里 df 的公式，分子、分母各是什么',\n",
    "     {'d94c9f90e3': '这是 Student t 检验的自由度；Welch 检验要用题目里的 Welch–Satterthwaite 公式',\n",
    "      '567085cd6f': '填空 2：分母两项各自除以 n − 1，不是 n'}),\n",
    "    ('双侧 P 值', lambda: [welch(*_mf(sp))[2] for sp in SPECIES], 'e', '084cc6eb08',\n",
    "     'P 由 t 和自由度算出（这一行框架已写好），先让上面两项通过'),\n",
    "    ('均值差的 95% 置信区间', lambda: [welch(*_mf(sp))[4:6] for sp in SPECIES], 0, '8720a79fcc',\n",
    "     '检查填空 3：用的是哪个分布、哪个分位数、哪个自由度',\n",
    "     {'5c3f84de3a': '填空 3 用了正态分布的 1.96；这里要用 t 分布的分位数',\n",
    "      'e2f8a893ac': '95% 双侧区间用的是 0.975 分位数，不是 0.95'}),\n",
    "])"
   ],
   "execution_count": null,
   "outputs": [],
   "id": "cell-37"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**答题区（B3）**\n",
    "\n",
    "（在这里写下数值结果和解释）"
   ],
   "id": "cell-38"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### B4　多重比较：Holm 校正\n",
    "\n",
    "三个物种各做了一次检验。自己写一个函数实现 Holm 校正：把 m 个 P 值从小到大排序，\n",
    "第 k 小的乘以 (m − k + 1)，再从小到大取累计最大值，超过 1 的记为 1。\n",
    "校正后的 P 值是多少？和不校正、Bonferroni 校正（都乘以 m）相比有什么不同？"
   ],
   "id": "cell-39"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "source": [
    "def holm(pvals):\n",
    "    p = np.asarray(pvals, dtype=float)\n",
    "    m = len(p)\n",
    "    adj = np.empty(m)\n",
    "    # TODO：按 P 值从小到大处理（np.argsort(p)）：第 k 小的（k 从 0 数起）乘以 (m − k)，\n",
    "    #       与前面已得到的校正值取较大者（累计最大值），且不超过 1，存回 adj 的原位置\n",
    "    return adj\n",
    "\n",
    "\n",
    "# TODO：holm(list(p_welch.values()))，再与 Bonferroni（每个乘以 3，且不超过 1）比较"
   ],
   "execution_count": null,
   "outputs": [],
   "id": "cell-40"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "source": [
    "# 自查（B4）：运行后 ✓ 表示通过，✗ 后面是提示。\n",
    "check('B4', [\n",
    "    ('holm() 处理示例 [0.04, 0.01, 0.03]', lambda: holm([0.04, 0.01, 0.03]), 2, '96119003e7',\n",
    "     '对照题目里的步骤：排序、乘 (m − k)、取累计最大值、放回原来的位置',\n",
    "     {'b1a004b3ab': '漏了“累计最大值”：排在后面的校正 P 值不能比前面的小',\n",
    "      'ad999c02c7': '这是 Bonferroni（每个都乘 m）；Holm 的乘数随排名递减',\n",
    "      'dcd39a3549': '结果要放回每个 P 值原来的位置，不是按大小排好输出'}),\n",
    "    ('holm() 处理示例 [0.6, 0.7, 0.02]', lambda: holm([0.6, 0.7, 0.02]), 2, '6cb9a4c474',\n",
    "     '对照题目里的步骤逐一检查：乘数、累计最大值、上限 1、放回原来的位置',\n",
    "     {'567a478a2e': '校正后的 P 值不能超过 1'}),\n",
    "    ('三个物种的 Holm 校正 P 值', lambda: holm([stats.ttest_ind(*_mf(sp), equal_var=False).pvalue for sp in SPECIES]), 'e', 'a8e50abee3',\n",
    "     '上面两个示例通过后这一项应当也通过；不通过时检查 holm() 是否改动了传入的数组'),\n",
    "])"
   ],
   "execution_count": null,
   "outputs": [],
   "id": "cell-41"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**答题区（B4）**\n",
    "\n",
    "（在这里写下数值结果和解释）"
   ],
   "id": "cell-42"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### B5　非参数复核：Mann–Whitney U 检验\n",
    "\n",
    "对每个物种做双侧 Mann–Whitney U 检验，同样做 Holm 校正。结论和 Welch t 检验一样吗？"
   ],
   "id": "cell-43"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "source": [
    "pu = []        # 每个物种的 Mann–Whitney U 检验 P 值，按 SPECIES 的顺序\n",
    "# TODO：对每个物种做 stats.mannwhitneyu(雄, 雌, alternative=\"two-sided\")，把 .pvalue 放进 pu\n",
    "pu_holm = ...  # TODO：用 holm() 校正"
   ],
   "execution_count": null,
   "outputs": [],
   "id": "cell-44"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "source": [
    "# 自查（B5）：运行后 ✓ 表示通过，✗ 后面是提示。\n",
    "check('B5', [\n",
    "    ('Mann–Whitney U 检验的 P 值（未校正）', lambda: pu, 'e', 'c3cbceebb4',\n",
    "     '每个物种做 stats.mannwhitneyu(雄, 雌, alternative=\"two-sided\")，按 SPECIES 的顺序放进 pu'),\n",
    "    ('Holm 校正后的 P 值', lambda: pu_holm, 'e', 'a52be04055',\n",
    "     '把 pu 交给 B4 的 holm()',\n",
    "     {'c3cbceebb4': '还没做 Holm 校正'}),\n",
    "])"
   ],
   "execution_count": null,
   "outputs": [],
   "id": "cell-45"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**答题区（B5）**\n",
    "\n",
    "（在这里写下数值结果和解释）"
   ],
   "id": "cell-46"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### B6　讨论题（写在答题区，每题 2–4 句）\n",
    "\n",
    "1. 原始研究在每个巢中同时测量了两只亲鸟（通常一雌一雄）。这对 B3 的检验意味着什么？用这份数据能不能补救？\n",
    "2. 阿德利企鹅两性体重差异的 P 值小到 10⁻²⁵ 量级。“P 值很小”和“差异很大”是一回事吗？你会怎样报告这个结果？"
   ],
   "id": "cell-47"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "source": [
    "# 自查（B6）：讨论题不自动判分，运行后对照检查你的回答。\n",
    "reflect('B6', [\n",
    "    '第 1 题：说清楚了哪一条检验假设受影响吗？说明了这份数据能不能补救、为什么吗？',\n",
    "    '第 2 题：改写后的结论里，除了 P 值，有没有效应大小和它的置信区间？数字带单位了吗？',\n",
    "])"
   ],
   "execution_count": null,
   "outputs": [],
   "id": "cell-48"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**答题区（B6）**\n",
    "\n",
    "（在这里写下数值结果和解释）"
   ],
   "id": "cell-49"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## C 部分　回归与辛普森悖论（第 3 周）\n",
    "\n",
    "本部分用四项形态测量齐全的 342 只企鹅（A2 中的 `morph`）。\n",
    "\n",
    "### C1　喙深对喙长：不分物种\n",
    "\n",
    "用 `scipy.stats.linregress` 拟合喙深（y）对喙长（x）的普通最小二乘回归，报告斜率、R² 和斜率的 P 值。\n",
    "只看这个结果，你会怎样描述喙长和喙深的关系？"
   ],
   "id": "cell-50"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "source": [
    "pool = ...  # TODO：stats.linregress(x, y)，x 为 morph 的喙长，y 为喙深\n",
    "# TODO：打印 pool.slope、pool.rvalue**2、pool.pvalue"
   ],
   "execution_count": null,
   "outputs": [],
   "id": "cell-51"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "source": [
    "# 自查（C1）：运行后 ✓ 表示通过，✗ 后面是提示。\n",
    "check('C1', [\n",
    "    ('斜率和 R²', lambda: [pool.slope, pool.rvalue ** 2], 3, '02defb4e38',\n",
    "     'x 是喙长、y 是喙深，数据用 morph',\n",
    "     {'f61cf7e808': 'x 和 y 放反了：y 是喙深',\n",
    "      '254cffc076': '结果是 nan：数据里还有缺测值，要用 morph'}),\n",
    "    ('斜率的 P 值', lambda: pool.pvalue, 'e', 'ddc7f3a16f',\n",
    "     'P 值在 linregress 结果的 pvalue 里'),\n",
    "])"
   ],
   "execution_count": null,
   "outputs": [],
   "id": "cell-52"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**答题区（C1）**\n",
    "\n",
    "（在这里写下数值结果和解释）"
   ],
   "id": "cell-53"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### C2　喙深对喙长：分物种\n",
    "\n",
    "在每个物种内分别拟合，报告斜率、斜率的 95% 置信区间（t 分布，自由度 n − 2）、R² 和 P。\n",
    "提示：`linregress` 的结果里有斜率的标准误 `stderr`。"
   ],
   "id": "cell-54"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "source": [
    "def fit_table(data, x, y, by=\"species\"):\n",
    "    \"\"\"在 by 的每一组内拟合 y 对 x 的 OLS 回归，返回 n、斜率、截距、斜率的 95% CI、R² 和 P。\"\"\"\n",
    "    keys, out = [], []\n",
    "    for key, d in data.groupby(by):\n",
    "        r = stats.linregress(d[x], d[y])\n",
    "        half = ...  # TODO：t 分布 0.975 分位数（自由度 n − 2）× 斜率标准误 r.stderr\n",
    "        keys.append(key)\n",
    "        out.append(dict(n=len(d), slope=r.slope, intercept=r.intercept, lo=r.slope - half,\n",
    "                        hi=r.slope + half, r2=r.rvalue**2, p=r.pvalue))\n",
    "    return pd.DataFrame(out, index=pd.Index(keys))\n",
    "\n",
    "\n",
    "bill = ...  # TODO：fit_table(morph, \"bill_length_mm\", \"bill_depth_mm\")"
   ],
   "execution_count": null,
   "outputs": [],
   "id": "cell-55"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "source": [
    "# 自查（C2）：运行后 ✓ 表示通过，✗ 后面是提示。\n",
    "check('C2', [\n",
    "    ('三个物种的斜率', lambda: bill.loc[SPECIES, 'slope'], 2, 'fc536f32f6',\n",
    "     'fit_table 的 x 是喙长、y 是喙深，数据用 morph'),\n",
    "    ('斜率的 95% 置信区间', lambda: bill.loc[SPECIES, ['lo', 'hi']], 2, 'db6a3688ca',\n",
    "     '检查 fit_table 里的 half：用的是哪个分布、哪个分位数、哪个自由度',\n",
    "     {'8d00972235': '用了正态分布的 1.96；斜率的置信区间用 t 分布，自由度 n − 2',\n",
    "      '52c73f9031': '95% 双侧区间用的是 0.975 分位数，不是 0.95'}),\n",
    "    ('R²', lambda: bill.loc[SPECIES, 'r2'], 2, '90ee546298',\n",
    "     'R² 是 r.rvalue 的平方'),\n",
    "])"
   ],
   "execution_count": null,
   "outputs": [],
   "id": "cell-56"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**答题区（C2）**\n",
    "\n",
    "（在这里写下数值结果和解释）"
   ],
   "id": "cell-57"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### C3　把悖论画出来\n",
    "\n",
    "画喙深对喙长的散点图，点按物种着色；叠加三条物种内回归线（实线）和一条合并回归线（灰色虚线）。\n",
    "图中要有坐标轴标题（带单位）和图例。"
   ],
   "id": "cell-58"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "source": [
    "fig, ax = plt.subplots(figsize=(5.6, 4.2))\n",
    "for sp in SPECIES:\n",
    "    d = morph[morph[\"species\"] == sp]\n",
    "    # TODO：该物种的散点（color=COLORS[sp]，label=LABEL[sp]）\n",
    "    # TODO：该物种的回归线：y = bill.loc[sp, \"intercept\"] + bill.loc[sp, \"slope\"] * x\n",
    "# TODO：合并回归线（灰色虚线），用 C1 的 pool.intercept 和 pool.slope\n",
    "ax.set(xlabel=\"Bill length (mm)\", ylabel=\"Bill depth (mm)\")\n",
    "ax.legend(frameon=False, fontsize=9)\n",
    "fig.tight_layout()\n",
    "finish(fig, \"C3_simpson\")"
   ],
   "execution_count": null,
   "outputs": [],
   "id": "cell-59"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "source": [
    "# 自查（C3）：运行后 ✓ 表示通过，✗ 后面是提示。\n",
    "check('C3', [\n",
    "    ('图中直线的斜率', lambda: _line_slopes(ax), 2, '09be624873',\n",
    "     '先让 C1、C2 的自查通过；图中应有 4 条直线：三个物种各一条（截距、斜率取自 bill），一条合并回归线（取自 C1 的 pool）'),\n",
    "    ('散点的组数', lambda: len(ax.collections), 0, '4e07408562',\n",
    "     '每个物种画一组散点，共 3 组'),\n",
    "])"
   ],
   "execution_count": null,
   "outputs": [],
   "id": "cell-60"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**答题区（C3）**\n",
    "\n",
    "（在这里写下数值结果和解释）"
   ],
   "id": "cell-61"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### C4　体重对鳍长\n",
    "\n",
    "1. 在每个物种内拟合体重对鳍长的回归，报告斜率（95% CI）和 R²。\n",
    "2. 三个物种合并拟合，R² 是多少？合并后 R² 变高了，能不能说“合并分析时鳍长更能解释体重”？"
   ],
   "id": "cell-62"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "source": [
    "mass = ...    # TODO：fit_table(morph, \"flipper_length_mm\", \"body_mass_g\")\n",
    "pool_m = ...  # TODO：三个物种合并拟合的 stats.linregress"
   ],
   "execution_count": null,
   "outputs": [],
   "id": "cell-63"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "source": [
    "# 自查（C4）：运行后 ✓ 表示通过，✗ 后面是提示。\n",
    "check('C4', [\n",
    "    ('物种内斜率及其 95% 置信区间', lambda: mass.loc[SPECIES, ['slope', 'lo', 'hi']], 1, 'efacc57b86',\n",
    "     'fit_table 的 x 是鳍长、y 是体重，数据用 morph'),\n",
    "    ('物种内 R²', lambda: mass.loc[SPECIES, 'r2'], 2, '2cc3a45b7d',\n",
    "     'R² 来自 fit_table 的 r2 列'),\n",
    "    ('合并拟合的斜率', lambda: pool_m.slope, 1, 'fab41ffb8b',\n",
    "     '三个物种合并：对 morph 全体做 linregress，x 是鳍长、y 是体重'),\n",
    "    ('合并拟合的 R²', lambda: pool_m.rvalue ** 2, 2, '342a441fd8',\n",
    "     'R² 是 rvalue 的平方'),\n",
    "])"
   ],
   "execution_count": null,
   "outputs": [],
   "id": "cell-64"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**答题区（C4）**\n",
    "\n",
    "（在这里写下数值结果和解释）"
   ],
   "id": "cell-65"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### C5　把物种放进模型\n",
    "\n",
    "论文建议“至少应把物种作为分组变量或模型中的固定效应”。拟合多元回归\n",
    "喙深 = b₀ + b₁·喙长 + b₂·[帽带企鹅] + b₃·[巴布亚企鹅]（方括号为 0/1 指示变量，阿德利企鹅为参照），\n",
    "可以用 `np.linalg.lstsq`。b₁ 是多少？与 C1、C2 的斜率比较。"
   ],
   "id": "cell-66"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "source": [
    "X = np.column_stack([\n",
    "    np.ones(len(morph)),\n",
    "    morph[\"bill_length_mm\"],\n",
    "    ...,  # TODO：帽带企鹅指示变量（是为 1.0，否为 0.0）\n",
    "    ...,  # TODO：巴布亚企鹅指示变量\n",
    "])\n",
    "beta, *_ = np.linalg.lstsq(X, morph[\"bill_depth_mm\"].to_numpy(), rcond=None)\n",
    "# TODO：打印 beta[1]，与 C1、C2 的斜率比较\n",
    "# 选做：用 s²(XᵀX)⁻¹ 算出 beta[1] 的标准误和 95% 置信区间"
   ],
   "execution_count": null,
   "outputs": [],
   "id": "cell-67"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "source": [
    "# 自查（C5）：运行后 ✓ 表示通过，✗ 后面是提示。\n",
    "check('C5', [\n",
    "    ('b₁（喙长的系数）', lambda: beta[1], 2, '0e1a3f9422',\n",
    "     '设计矩阵应有 4 列：截距、喙长、两个物种指示变量；y 是喙深'),\n",
    "    ('b₂、b₃（两个指示变量的系数）', lambda: beta[2:4], 2, '87d97d75cf',\n",
    "     '第 3 列是帽带企鹅的指示变量，第 4 列是巴布亚企鹅；阿德利企鹅是参照组，不单独设列',\n",
    "     {'8d10187c0e': '两个指示变量的顺序反了：先帽带企鹅，再巴布亚企鹅'}),\n",
    "])"
   ],
   "execution_count": null,
   "outputs": [],
   "id": "cell-68"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**答题区（C5）**\n",
    "\n",
    "（在这里写下数值结果和解释）"
   ],
   "id": "cell-69"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### C6　挑战题：再把性别分开\n",
    "\n",
    "雄性在四个性状上都大于雌性，所以物种内的回归也混入了两性差异。在 6 个物种 × 性别组合内\n",
    "分别拟合体重对鳍长、喙深对喙长，回答：\n",
    "\n",
    "1. 体重对鳍长的斜率和 R² 落在什么范围？有几个组合的斜率和 R² 都低于该物种合并两性时的值？\n",
    "2. 喙深对喙长的关系在几个组合中 P < 0.05？阿德利企鹅雄性的斜率是多少？"
   ],
   "id": "cell-70"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "source": [
    "sx_mass = ...  # TODO：fit_table(sexed, \"flipper_length_mm\", \"body_mass_g\", by=[\"species\", \"sex\"])\n",
    "sx_bill = ...  # TODO：同上，喙深对喙长\n",
    "# TODO：与 C4 的 mass（两性合并）逐个物种比较斜率和 R²；数一数 sx_bill 中 P < 0.05 的组合"
   ],
   "execution_count": null,
   "outputs": [],
   "id": "cell-71"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "source": [
    "# 自查（C6）：运行后 ✓ 表示通过，✗ 后面是提示。\n",
    "check('C6', [\n",
    "    ('体重对鳍长：6 组斜率的最小值和最大值', lambda: [sx_mass['slope'].min(), sx_mass['slope'].max()], 1, '10e5686261',\n",
    "     '用 sexed，by=[\"species\", \"sex\"]，x 是鳍长、y 是体重'),\n",
    "    ('体重对鳍长：6 组 R² 的最小值和最大值', lambda: [sx_mass['r2'].min(), sx_mass['r2'].max()], 2, '751fb6f8df',\n",
    "     'R² 来自 sx_mass 的 r2 列'),\n",
    "    ('喙深对喙长：P < 0.05 的组合数', lambda: (sx_bill['p'] < 0.05).sum(), 0, '4e07408562',\n",
    "     'sx_bill 是喙深对喙长（x 是喙长）；数 p 列中小于 0.05 的行'),\n",
    "    ('阿德利企鹅雄性的喙部斜率', lambda: sx_bill.loc[('Adelie', 'male'), 'slope'], 3, '95619a4916',\n",
    "     '在 sx_bill 中找 (Adelie, male) 这一行'),\n",
    "])"
   ],
   "execution_count": null,
   "outputs": [],
   "id": "cell-72"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**答题区（C6）**\n",
    "\n",
    "（在这里写下数值结果和解释）"
   ],
   "id": "cell-73"
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
