{ "cells": [ { "cell_type": "markdown", "id": "bea416d7", "metadata": {}, "source": [ "# ZeaVis Edu — Corn Leaf Disease Classifier v3.0\n", "\n", "Mengklasifikasikan penyakit daun jagung (Bercak Daun, Hawar Daun, Karat Daun, Daun Sehat)\n", "menggunakan EfficientNetV2B0 dengan **CBAM spatial attention**, **RandAugment + weather simulation**,\n", "dan **temperature-scaled confidence calibration** untuk deployment real-world.\n", "\n", "Fokus v3.0: **robustness dunia nyata** — berbagai pencahayaan, resolusi, angle, dan background.\n" ] }, { "cell_type": "markdown", "id": "aba4b688", "metadata": {}, "source": [ "## 1. Persiapan Lingkungan\n", "\n", "Mengimpor pustaka, mengatur seed, dan mengoptimalkan konfigurasi.\n", "**Presisi float32**, resolusi target **224×224** (EfficientNetV2B0).\n", "Augmentasi real-world via RandAugment pool 15 transformasi.\n", "Confidence calibration via temperature scaling.\n" ] }, { "cell_type": "code", "execution_count": null, "id": "9dc08169", "metadata": {}, "outputs": [], "source": [ "!pip install -r requirements.txt\n" ] }, { "cell_type": "code", "execution_count": null, "id": "9d108f5e", "metadata": {}, "outputs": [], "source": [ "import os, shutil, zipfile, random, time, json\n", "from collections import Counter\n", "import numpy as np\n", "import matplotlib.pyplot as plt\n", "import seaborn as sns\n", "from PIL import Image\n", "\n", "import tensorflow as tf\n", "from tensorflow.keras import layers, models, callbacks\n", "from tensorflow.keras.applications import EfficientNetV2B0\n", "from tensorflow.keras.applications.efficientnet_v2 import preprocess_input\n", "from tensorflow.keras.optimizers import AdamW\n", "from tensorflow.keras.optimizers.schedules import CosineDecay\n", "from sklearn.metrics import classification_report, confusion_matrix\n", "from sklearn.utils.class_weight import compute_class_weight\n", "from sklearn.model_selection import train_test_split\n", "\n", "# Optional: perceptual hashing for dedup (pip install imagehash)\n", "try:\n", " import imagehash\n", " HAS_IMAGEHASH = True\n", "except ImportError:\n", " HAS_IMAGEHASH = False\n", "\n", "# Optional: scipy for temperature optimization\n", "try:\n", " from scipy.optimize import minimize_scalar\n", " HAS_SCIPY = True\n", "except ImportError:\n", " HAS_SCIPY = False\n", "\n", "# Detect environment\n", "try:\n", " from google.colab import drive\n", " IS_COLAB = True\n", " print(\"Running on Google Colab\")\n", "except ModuleNotFoundError:\n", " IS_COLAB = False\n", " print(f\"Running locally (TF {tf.__version__}, GPU: {tf.config.list_physical_devices('GPU')})\")\n", "\n", "tf.keras.mixed_precision.set_global_policy('float32')\n", "\n", "# Hyperparams\n", "IMG_SIZE = (224, 224)\n", "BATCH_SIZE = 32\n", "SEED = 42\n", "random.seed(SEED)\n", "np.random.seed(SEED)\n", "tf.random.set_seed(SEED)\n", "\n", "AUTOTUNE = tf.data.AUTOTUNE\n", "print(f\"Setup OK. IMG={IMG_SIZE}, BATCH={BATCH_SIZE}\")\n" ] }, { "cell_type": "markdown", "id": "06382ddf", "metadata": {}, "source": [ "## 2. Download dan Ekstraksi Dataset\n" ] }, { "cell_type": "code", "execution_count": null, "id": "26afde45", "metadata": {}, "outputs": [], "source": [ "if IS_COLAB:\n", " drive.mount('/content/drive')\n", " archive_path = '/content/drive/MyDrive/jagung/dataset.zip'\n", " destination_path = '/content/dataset.zip'\n", " extract_path = '/content/dataset'\n", "else:\n", " base = os.getcwd()\n", " archive_path = os.path.join(base, 'dataset.zip')\n", " destination_path = archive_path\n", " extract_path = os.path.join(base, 'dataset')\n", "\n", "if os.path.exists(destination_path):\n", " if not os.path.exists(extract_path) or len(os.listdir(extract_path)) == 0:\n", " os.makedirs(extract_path, exist_ok=True)\n", " print(\"Extracting dataset...\")\n", " try:\n", " with zipfile.ZipFile(destination_path, 'r') as zip_ref:\n", " zip_ref.extractall(path=extract_path)\n", " print(\"Extraction completed!\")\n", " except Exception as e:\n", " print(f\"Extraction failed: {e}\")\n", " else:\n", " print(\"Dataset ready.\")\n", "else:\n", " print(f\"dataset.zip not found at {destination_path}. Upload dataset.zip to Google Drive / MyDrive/jagung/\")\n", " print(\"Or run preprocessing.py locally and upload the resulting dataset.zip\")" ] }, { "cell_type": "markdown", "id": "199bdf60", "metadata": {}, "source": [ "## 3. Data Cleaning — Corrupt Detection + Augmented Dedup\n", "\n", "Membersihkan dataset dari:\n", "- File corrupt / tidak bisa dibuka PIL\n", "- File `augmented_*` (pre-augmented duplicates — menyebabkan data leakage)\n", "- Gambar dengan dimensi atau aspect ratio ekstrim\n", "- Gambar dengan variance terlalu rendah (hampir seragam)\n" ] }, { "cell_type": "code", "execution_count": null, "id": "75cacdf5", "metadata": {}, "outputs": [], "source": [ "# --- Determine dataset path ---\n", "dataset_path = extract_path\n", "\n", "print(f\"Dataset path: {dataset_path}\")\n", "\n", "# Check if dataset already validated and split exists — skip if so\n", "split_output_dir = \"/content/dataset_split\" if IS_COLAB else os.path.join(os.getcwd(), \"dataset_split\")\n", "if os.path.exists(os.path.join(split_output_dir, 'train')):\n", " print(\"Split dataset already exists. Skipping validation and split.\")\n", " # Still need class_names and counts for downstream cells\n", " train_dir = os.path.join(split_output_dir, 'train')\n", " val_dir = os.path.join(split_output_dir, 'val')\n", " test_dir = os.path.join(split_output_dir, 'test')\n", " print(f'Train: {sum(len(files) for _, _, files in os.walk(train_dir))} | '\n", " f'Val: {sum(len(files) for _, _, files in os.walk(val_dir))} | '\n", " f'Test: {sum(len(files) for _, _, files in os.walk(test_dir))}')\n", "else:\n", " MIN_FILE_SIZE = 512\n", " MIN_DIM = 32\n", " MAX_ASPECT = 5.0\n", "\n", " def remove_augmented_duplicates(directory):\n", " removed = 0\n", " for root, dirs, files in os.walk(directory):\n", " for file in files:\n", " if file.startswith(\"augmented_\"):\n", " try:\n", " os.remove(os.path.join(root, file))\n", " removed += 1\n", " except OSError:\n", " pass\n", " return removed\n", "\n", " def clean_and_validate_images(directory):\n", " stats = {\"too_small\": 0, \"corrupt\": 0, \"small_dims\": 0, \"extreme_aspect\": 0, \"low_var\": 0, \"ok\": 0}\n", " for root, dirs, files in os.walk(directory):\n", " for file in files:\n", " fp = os.path.join(root, file)\n", " try:\n", " if os.path.getsize(fp) < MIN_FILE_SIZE:\n", " os.remove(fp); stats[\"too_small\"] += 1; continue\n", " except OSError:\n", " continue\n", " try:\n", " img = Image.open(fp); img.verify()\n", " except Exception:\n", " try: os.remove(fp); stats[\"corrupt\"] += 1\n", " except OSError: pass\n", " continue\n", " try:\n", " img = Image.open(fp)\n", " w, h = img.size\n", " if w < MIN_DIM or h < MIN_DIM:\n", " os.remove(fp); stats[\"small_dims\"] += 1; continue\n", " aspect = w / max(h, 1)\n", " if aspect > MAX_ASPECT or aspect < 1.0 / MAX_ASPECT:\n", " os.remove(fp); stats[\"extreme_aspect\"] += 1; continue\n", " if img.mode not in ('RGB', 'RGBA'):\n", " img = img.convert('RGB'); img.save(fp)\n", " arr = np.array(img).astype(np.float32)\n", " if np.std(arr) < 2.0:\n", " os.remove(fp); stats[\"low_var\"] += 1; continue\n", " stats[\"ok\"] += 1\n", " except Exception:\n", " try: os.remove(fp); stats[\"corrupt\"] += 1\n", " except OSError: pass\n", " return stats\n", "\n", " print(\"1. Removing augmented duplicates...\")\n", " n_aug = remove_augmented_duplicates(dataset_path)\n", " print(f\" Removed {n_aug} augmented_* files\")\n", "\n", " print(\"2. Validating images...\")\n", " stats = clean_and_validate_images(dataset_path)\n", " print(f\" OK: {stats['ok']} | Removed: too_small={stats['too_small']} corrupt={stats['corrupt']} \"\n", " f\"small_dims={stats['small_dims']} aspect={stats['extreme_aspect']} low_var={stats['low_var']}\")\n", "\n", " if HAS_IMAGEHASH:\n", " print(\"3. Perceptual hash dedup...\")\n", " seen, removed = {}, 0\n", " for cn in sorted(os.listdir(dataset_path)):\n", " cp = os.path.join(dataset_path, cn)\n", " if not os.path.isdir(cp): continue\n", " for f in sorted(os.listdir(cp)):\n", " fp = os.path.join(cp, f)\n", " if not os.path.isfile(fp): continue\n", " try:\n", " ah = imagehash.average_hash(Image.open(fp).convert('RGB'))\n", " for sk, (sp, sc) in seen.items():\n", " if ah - imagehash.hex_to_hash(sk) <= 5:\n", " try: os.remove(fp); removed += 1\n", " except OSError: pass\n", " break\n", " else:\n", " seen[str(ah)] = (fp, cn)\n", " except Exception:\n", " pass\n", " print(f\" Removed {removed} near-duplicates\")\n", " else:\n", " print(\"3. Perceptual hash dedup SKIPPED (pip install imagehash)\")\n", "\n", " total = sum(len(files) for _, _, files in os.walk(dataset_path))\n", " print(f\"\\nTotal clean images: {total}\")" ] }, { "cell_type": "markdown", "id": "fbe9498c", "metadata": {}, "source": [ "## 4. Stratified Split by Source (70:15:15)\n", "\n", "**Tidak menggunakan splitfolders!** Split manual dengan stratifikasi berdasarkan prefix sumber gambar.\n", "Ini mencegah gambar dari sesi foto yang sama (lighting & background identik) masuk ke train DAN test.\n", "\n", "Source prefixes:\n", "- `IMG_*` → foto HP\n", "- `Corn_*` → dataset lab publik\n", "- `CBS*`, `GLS*`, `NLS*`, `CLS*` → berbagai dataset lab\n", "- `SCR*`, `CR*`, `NLB*`, `SLB*` → dataset spesifik penyakit\n" ] }, { "cell_type": "code", "execution_count": null, "id": "56234c24", "metadata": {}, "outputs": [], "source": [ "def extract_source_prefix(filename):\n", " f = os.path.splitext(filename)[0]\n", " if f.startswith('IMG_'): return 'phone'\n", " if f.startswith('Corn_'): return 'lab_corn'\n", " for prefix in ['CBS', 'GLS', 'NLS', 'CLS']:\n", " if f.startswith(prefix): return 'lab_disease'\n", " for prefix in ['SCR', 'CR', 'NLB', 'SLB', 'SRS']:\n", " if f.startswith(prefix): return 'lab_rust_blight'\n", " return 'other'\n", "\n", "def stratified_split_by_source(dataset_path, output_dir, ratios=(0.7, 0.15, 0.15), seed=42):\n", " class_images = {}\n", " for cn in sorted(os.listdir(dataset_path)):\n", " cp = os.path.join(dataset_path, cn)\n", " if not os.path.isdir(cp): continue\n", " class_images[cn] = []\n", " for f in os.listdir(cp):\n", " fp = os.path.join(cp, f)\n", " if os.path.isfile(fp):\n", " class_images[cn].append((fp, f, extract_source_prefix(f)))\n", "\n", " for split in ['train', 'val', 'test']:\n", " for cn in class_images:\n", " os.makedirs(os.path.join(output_dir, split, cn), exist_ok=True)\n", "\n", " rng = np.random.RandomState(seed)\n", "\n", " for cn, images in class_images.items():\n", " by_source = {}\n", " for fp, fn, src in images:\n", " by_source.setdefault(src, []).append((fp, fn))\n", "\n", " train_files, val_files, test_files = [], [], []\n", " for src, src_images in by_source.items():\n", " n = len(src_images)\n", " rng.shuffle(src_images)\n", " n_train = max(1, int(n * ratios[0]))\n", " n_val = max(1, int(n * ratios[1]))\n", " train_files.extend(src_images[:n_train])\n", " val_files.extend(src_images[n_train:n_train + n_val])\n", " test_files.extend(src_images[n_train + n_val:])\n", "\n", " for fp, fn in train_files:\n", " shutil.copy(fp, os.path.join(output_dir, 'train', cn, fn))\n", " for fp, fn in val_files:\n", " shutil.copy(fp, os.path.join(output_dir, 'val', cn, fn))\n", " for fp, fn in test_files:\n", " shutil.copy(fp, os.path.join(output_dir, 'test', cn, fn))\n", "\n", " train_srcs = Counter(extract_source_prefix(fn) for _, fn in train_files)\n", " print(f\" {cn}: train={len(train_files)} val={len(val_files)} test={len(test_files)} | \"\n", " f\"sources={dict(train_srcs)}\")\n", "\n", "output_dir = split_output_dir # Already defined in previous cell\n", "\n", "if os.path.exists(os.path.join(output_dir, 'train')):\n", " print(\"Split already exists. Skipping.\")\n", "else:\n", " if os.path.exists(output_dir):\n", " shutil.rmtree(output_dir)\n", "\n", " print(\"Splitting dataset 70:15:15 (stratified by source)...\")\n", " stratified_split_by_source(dataset_path, output_dir, seed=SEED)\n", " print(\"Done.\")\n", "\n", "train_dir = os.path.join(output_dir, 'train')\n", "val_dir = os.path.join(output_dir, 'val')\n", "test_dir = os.path.join(output_dir, 'test')\n", "\n", "def count_images(path):\n", " return sum(len(files) for _, _, files in os.walk(path))\n", "\n", "print(f'Train: {count_images(train_dir)} | Val: {count_images(val_dir)} | Test: {count_images(test_dir)}')" ] }, { "cell_type": "markdown", "id": "3eead964", "metadata": {}, "source": [ "## 5. RandAugment Pipeline dengan Weather Simulation\n", "\n", "Pipeline augmentasi baru untuk **robustness dunia nyata**:\n", "\n", "### Pool 15 Transformasi (RandAugment: pilih N=3 per gambar)\n", "| Kategori | Transformasi | Simulasi |\n", "|---|---|---|\n", "| Geometric | Flip, Rotate, Zoom, Translate, Shear | Variasi angle/jarak foto |\n", "| Color/Light | Hue, Saturation, Brightness, Contrast, Solarize | Variasi kamera & waktu hari |\n", "| Weather | Fog, Shadow | Kondisi lapangan berkabut/berbayang |\n", "| Degradation | GaussianBlur, ResolutionDrop | Blur gerakan, kamera rendah |\n", "| Mixing | MixUp, CutMix, RandomErasing | Regularisasi label & occlusions |\n", "\n", "Fog dan Shadow adalah **custom tf operations** — tidak ada di Keras layers standar.\n" ] }, { "cell_type": "code", "execution_count": null, "id": "8535054d", "metadata": {}, "outputs": [], "source": [ "# ─── RandAugment Utilities ───\n", "\n", "def sample_beta_distribution(size, a=0.2, b=0.2):\n", " g1 = tf.random.gamma([size], a, dtype=tf.float32)\n", " g2 = tf.random.gamma([size], b, dtype=tf.float32)\n", " return g2 / (g1 + g2 + 1e-8)\n", "\n", "def _apply_contrast(img, factor):\n", " mean = tf.reduce_mean(tf.cast(img, tf.float32), axis=(0, 1), keepdims=True)\n", " return mean + factor * (tf.cast(img, tf.float32) - mean)\n", "\n", "def _apply_brightness(img, delta):\n", " return tf.clip_by_value(tf.cast(img, tf.float32) + delta, 0.0, 255.0)\n", "\n", "def _apply_hue(img, delta):\n", " hsv = tf.image.rgb_to_hsv(tf.cast(img, tf.float32) / 255.0)\n", " h = hsv[..., 0] + delta\n", " h = h - tf.floor(h)\n", " hsv_h = tf.stack([h, hsv[..., 1], hsv[..., 2]], axis=-1)\n", " return tf.image.hsv_to_rgb(hsv_h) * 255.0\n", "\n", "@tf.function(reduce_retracing=True)\n", "def _randaug_select(images, ops_per_image=3, magnitude=0.7):\n", " # Graph-mode-safe RandAugment. No tf.image.random_* or Keras layers\n", " # (both trace as Python bool checks on tensor bounds internally).\n", "\n", " def _zoom(img):\n", " h = tf.cast(tf.shape(img)[0], tf.float32)\n", " w = tf.cast(tf.shape(img)[1], tf.float32)\n", " z = tf.random.uniform([], 1.0 - 0.2 * magnitude, 1.0 + 0.2 * magnitude)\n", " zh = tf.cast(h / z, tf.int32); zw = tf.cast(w / z, tf.int32)\n", " zoomed = tf.image.resize(tf.expand_dims(img, 0), [zh, zw], method='bilinear')[0]\n", " return tf.image.resize_with_crop_or_pad(zoomed, tf.cast(h, tf.int32), tf.cast(w, tf.int32))\n", " def _translate(img):\n", " h, w = tf.shape(img)[0], tf.shape(img)[1]\n", " tx = tf.cast(tf.random.uniform([], -0.15 * magnitude, 0.15 * magnitude) * tf.cast(w, tf.float32), tf.int32)\n", " ty = tf.cast(tf.random.uniform([], -0.15 * magnitude, 0.15 * magnitude) * tf.cast(h, tf.float32), tf.int32)\n", " return tf.roll(img, [ty, tx], axis=[0, 1])\n", "\n", " def op_flip(img): return tf.image.random_flip_left_right(img)\n", " def op_flip_v(img): return tf.image.random_flip_up_down(img)\n", " def op_rotate_90(img):\n", " k = tf.random.uniform([], 0, 4, dtype=tf.int32)\n", " return tf.image.rot90(img, k)\n", " def op_zoom(img): return _zoom(img)\n", " def op_translate(img): return _translate(img)\n", " def op_contrast(img):\n", " f = tf.random.uniform([], 1.0 - 0.5 * magnitude, 1.0 + 0.5 * magnitude)\n", " return _apply_contrast(img, f)\n", " def op_brightness(img):\n", " d = tf.random.uniform([], -0.3 * magnitude * 255.0, 0.3 * magnitude * 255.0)\n", " return _apply_brightness(img, d)\n", " def op_hue(img):\n", " d = tf.random.uniform([], -0.08 * tf.maximum(magnitude, 0.01), 0.08 * tf.maximum(magnitude, 0.01))\n", " return _apply_hue(img, d)\n", " def op_saturation(img):\n", " lo = tf.maximum(0.5, 1.0 - 0.8 * magnitude)\n", " hi = 1.0 + 0.8 * magnitude\n", " f = tf.random.uniform([], lo, hi)\n", " hsv = tf.image.rgb_to_hsv(tf.cast(img, tf.float32) / 255.0)\n", " s = tf.clip_by_value(hsv[..., 1] * f, 0.0, 1.0)\n", " return tf.image.hsv_to_rgb(tf.stack([hsv[..., 0], s, hsv[..., 2]], axis=-1)) * 255.0\n", " def op_solarize(img):\n", " thresh = tf.random.uniform([], 0.3, 0.8) * 255.0\n", " f = tf.cast(img, tf.float32)\n", " return tf.where(f < thresh, f, 255.0 - f)\n", " def op_blur(img):\n", " h, w = tf.shape(img)[0], tf.shape(img)[1]\n", " sf = tf.random.uniform([], 2, 4, dtype=tf.int32)\n", " small = tf.image.resize(tf.expand_dims(tf.cast(img, tf.float32), 0), [h // sf, w // sf], method='bilinear')\n", " return tf.image.resize(small, [h, w], method='bilinear')[0]\n", " def op_fog(img):\n", " fl = tf.random.uniform([], 0.1, 0.1 + 0.4 * magnitude)\n", " fc = tf.random.uniform([3], 0.7, 1.0) * 255.0\n", " return tf.cast(img, tf.float32) * (1.0 - fl) + tf.reshape(fc, [1, 1, 3]) * fl\n", " def op_shadow(img):\n", " op = tf.random.uniform([], 0.2, 0.2 + 0.5 * magnitude)\n", " return tf.cast(img, tf.float32) * (1.0 - op * 0.6)\n", " def op_resolution_drop(img):\n", " h, w = tf.shape(img)[0], tf.shape(img)[1]\n", " sf = tf.random.uniform([], 2, 5, dtype=tf.int32)\n", " small = tf.image.resize(tf.expand_dims(tf.cast(img, tf.float32), 0), [h // sf, w // sf], method='bilinear')\n", " return tf.image.resize(small, [h, w], method='nearest')[0]\n", " def op_identity(img): return tf.cast(img, tf.float32)\n", "\n", " ops = [op_flip, op_flip_v, op_rotate_90, op_zoom, op_translate,\n", " op_contrast, op_brightness, op_hue, op_saturation,\n", " op_solarize, op_blur, op_fog, op_shadow, op_resolution_drop, op_identity]\n", "\n", " def apply_randaug_single(img3d):\n", " indices = tf.random.shuffle(tf.range(15))[:3] # ops_per_image=3, static\n", " result = tf.cast(img3d, tf.float32)\n", " # Unrolled static 3 iterations — avoids TF shape invariance error from tf.range loop\n", " def _apply_one(r, idx):\n", " return tf.switch_case(idx, {j: lambda j=j: ops[j](r) for j in range(15)})\n", " i0, i1, i2 = indices[0], indices[1], indices[2]\n", " result = _apply_one(result, i0)\n", " result = _apply_one(result, i1)\n", " result = _apply_one(result, i2)\n", " return tf.clip_by_value(result, 0.0, 255.0)\n", "\n", " return tf.map_fn(apply_randaug_single, images, dtype=tf.float32, parallel_iterations=8)\n", "\n", "# Compatibility wrapper using Keras Sequential for basic geometric ops (kept for visualization)\n", "geo_aug = tf.keras.Sequential([\n", " layers.RandomFlip(\"horizontal_and_vertical\"),\n", " layers.RandomRotation(0.15),\n", " layers.RandomZoom(0.15),\n", " layers.RandomTranslation(0.1, 0.1),\n", " layers.RandomContrast(0.15),\n", " layers.RandomBrightness(0.15),\n", "], name=\"geo_aug\")\n", "\n", "# ─── MixUp & CutMix (unchanged from original) ───\n", "\n", "def mix_up(images, labels, alpha=0.2):\n", " bs = tf.shape(images)[0]\n", " lam = sample_beta_distribution(bs, alpha, alpha)\n", " lam_img = tf.reshape(lam, [bs, 1, 1, 1])\n", " ri = tf.random.shuffle(tf.range(bs))\n", " mixed_img = lam_img * images + (1 - lam_img) * tf.gather(images, ri)\n", " labels = tf.cast(labels, tf.float32)\n", " lam_lbl = tf.reshape(lam, [-1, 1])\n", " mixed_lbl = lam_lbl * labels + (1 - lam_lbl) * tf.gather(labels, ri)\n", " return mixed_img, mixed_lbl\n", "\n", "def cut_mix(images, labels, alpha=0.2):\n", " bs = tf.shape(images)[0]; h = tf.shape(images)[1]; w = tf.shape(images)[2]\n", " lam = sample_beta_distribution(bs, alpha, alpha); ri = tf.random.shuffle(tf.range(bs))\n", " cr = tf.sqrt(1.0 - lam)\n", " rh = tf.cast(cr * tf.cast(h, tf.float32), tf.int32); rw = tf.cast(cr * tf.cast(w, tf.float32), tf.int32)\n", " cx = tf.random.uniform([bs], 0, w, tf.int32); cy = tf.random.uniform([bs], 0, h, tf.int32)\n", " hh = rh // 2; hw = rw // 2\n", " x1 = tf.clip_by_value(cx - hw, 0, w); x2 = tf.clip_by_value(cx + hw, 0, w)\n", " y1 = tf.clip_by_value(cy - hh, 0, h); y2 = tf.clip_by_value(cy + hh, 0, h)\n", " col = tf.range(w, dtype=tf.int32); row = tf.range(h, dtype=tf.int32)\n", " in_x = tf.logical_and(tf.reshape(col, [1, 1, w]) >= tf.reshape(x1, [bs, 1, 1]),\n", " tf.reshape(col, [1, 1, w]) < tf.reshape(x2, [bs, 1, 1]))\n", " in_y = tf.logical_and(tf.reshape(row, [1, h, 1]) >= tf.reshape(y1, [bs, 1, 1]),\n", " tf.reshape(row, [1, h, 1]) < tf.reshape(y2, [bs, 1, 1]))\n", " cm = tf.cast(tf.logical_and(in_y, in_x), tf.float32); cm = tf.expand_dims(cm, -1)\n", " shuf = tf.gather(images, ri)\n", " mi = (1.0 - cm) * images + cm * shuf\n", " labels = tf.cast(labels, tf.float32); lr = tf.reshape(lam, [-1, 1])\n", " ml = lr * labels + (1.0 - lr) * tf.gather(labels, ri)\n", " return mi, ml\n", "\n", "def random_erasing(images, probability=0.25, scale=(0.02, 0.25)):\n", " bs = tf.shape(images)[0]; h = tf.shape(images)[1]; w = tf.shape(images)[2]\n", " ta = tf.random.uniform([], scale[0], scale[1]) * tf.cast(h * w, tf.float32)\n", " ar = tf.random.uniform([], 0.3, 3.3)\n", " eh = tf.cast(tf.math.sqrt(ta / ar), tf.int32); ew = tf.cast(tf.math.sqrt(ta * ar), tf.int32)\n", " eh = tf.clip_by_value(eh, 1, h - 1); ew = tf.clip_by_value(ew, 1, w - 1)\n", " cx = tf.random.uniform([], 0, w - ew, tf.int32); cy = tf.random.uniform([], 0, h - eh, tf.int32)\n", " col = tf.range(w, dtype=tf.int32); row = tf.range(h, dtype=tf.int32)\n", " ix = tf.logical_and(col >= cx, col < cx + ew)\n", " iy = tf.logical_and(row >= cy, row < cy + eh)\n", " em = tf.cast(tf.expand_dims(iy, 1) & tf.expand_dims(ix, 0), tf.float32)\n", " em = tf.expand_dims(tf.expand_dims(em, 0), -1)\n", " noise = tf.random.uniform([bs, eh, ew, 3], 0.0, 255.0, dtype=tf.float32)\n", " pads = [[0, 0], [cy, h - (cy + eh)], [cx, w - (cx + ew)], [0, 0]]\n", " npad = tf.pad(noise, pads, constant_values=0.0)\n", " erased = images * (1.0 - em) + npad * em\n", " return tf.cond(tf.random.uniform([]) < probability, lambda: erased, lambda: images)\n", "\n", "# ─── Main augmentation pipeline ───\n", "\n", "def augment_and_mix(images, labels):\n", " # 1. RandAugment (geometric + color + weather + degradation)\n", " images = tf.cast(images, tf.float32)\n", " images = _randaug_select(images, ops_per_image=3, magnitude=0.7)\n", " # 2. MixUp or CutMix (40% chance total: 20% MixUp, 20% CutMix)\n", " choice = tf.random.uniform([])\n", " labels_oh = tf.one_hot(labels, NUM_CLASSES)\n", " images, labels_oh = tf.cond(\n", " choice < 0.2, lambda: mix_up(images, labels_oh),\n", " lambda: tf.cond(choice < 0.4, lambda: cut_mix(images, labels_oh),\n", " lambda: (images, labels_oh)))\n", " # 3. Random Erasing\n", " images = random_erasing(images, probability=0.2)\n", " return images, labels_oh\n", "\n", "# ─── Preprocessing ───\n", "def preprocess_fn(image, label):\n", " return preprocess_input(image), label\n", "\n", "# ─── Class weights ───\n", "train_class_counts = Counter()\n", "for cn in sorted(os.listdir(train_dir)):\n", " p = os.path.join(train_dir, cn)\n", " if os.path.isdir(p):\n", " train_class_counts[cn] = len(os.listdir(p))\n", "\n", "y_int = []\n", "for i, cn in enumerate(sorted(os.listdir(train_dir))):\n", " cp = os.path.join(train_dir, cn)\n", " if os.path.isdir(cp):\n", " y_int.extend([i] * len(os.listdir(cp)))\n", "\n", "cw_array = compute_class_weight('balanced', classes=np.unique(y_int), y=y_int)\n", "cw_capped = [min(w, 3.0) for w in cw_array]\n", "class_weights_tensor = tf.constant(cw_capped, dtype=tf.float32)\n", "\n", "def add_sample_weight(image, label):\n", " ci = tf.argmax(label, axis=-1)\n", " sw = tf.gather(class_weights_tensor, ci)\n", " return image, label, sw\n", "\n", "# ─── Build datasets ───\n", "train_ds = tf.keras.utils.image_dataset_from_directory(\n", " train_dir, shuffle=True, batch_size=BATCH_SIZE, image_size=IMG_SIZE)\n", "val_ds = tf.keras.utils.image_dataset_from_directory(\n", " val_dir, shuffle=True, batch_size=BATCH_SIZE, image_size=IMG_SIZE)\n", "test_ds = tf.keras.utils.image_dataset_from_directory(\n", " test_dir, shuffle=False, batch_size=BATCH_SIZE, image_size=IMG_SIZE)\n", "\n", "class_names = train_ds.class_names\n", "NUM_CLASSES = len(class_names)\n", "print(f\"Classes ({NUM_CLASSES}): {class_names}\")\n", "\n", "train_ds = (train_ds\n", " .map(augment_and_mix, num_parallel_calls=AUTOTUNE)\n", " .map(preprocess_fn, num_parallel_calls=AUTOTUNE)\n", " .map(add_sample_weight, num_parallel_calls=AUTOTUNE)\n", " .prefetch(AUTOTUNE))\n", "\n", "val_ds = (val_ds\n", " .map(lambda img, lbl: (preprocess_input(tf.cast(img, tf.float32)), lbl), num_parallel_calls=AUTOTUNE)\n", " .map(lambda img, lbl: (img, tf.one_hot(lbl, NUM_CLASSES)), num_parallel_calls=AUTOTUNE)\n", " .prefetch(AUTOTUNE))\n", "\n", "test_ds = (test_ds\n", " .map(lambda img, lbl: (preprocess_input(tf.cast(img, tf.float32)), lbl), num_parallel_calls=AUTOTUNE)\n", " .map(lambda img, lbl: (img, tf.one_hot(lbl, NUM_CLASSES)), num_parallel_calls=AUTOTUNE)\n", " .prefetch(AUTOTUNE))\n", "\n", "print(\"Class weights (capped at 3.0):\")\n", "for i, cn in enumerate(class_names):\n", " if i < len(cw_capped):\n", " print(f\" {cn}: {cw_capped[i]:.4f}\")\n", "print(\"Data pipelines ready.\")\n" ] }, { "cell_type": "markdown", "id": "4ca2ec40", "metadata": {}, "source": [ "## 6. Visualisasi Sampel Data (Augmentasi Real-World)\n" ] }, { "cell_type": "code", "execution_count": null, "id": "44d6d2b0", "metadata": {}, "outputs": [], "source": [ "vis_ds = tf.keras.utils.image_dataset_from_directory(\n", " train_dir, shuffle=True, batch_size=BATCH_SIZE, image_size=IMG_SIZE)\n", "\n", "plt.figure(figsize=(16, 10))\n", "for images, labels in vis_ds.take(1):\n", " # Original (4)\n", " for i in range(4):\n", " plt.subplot(3, 4, i + 1)\n", " plt.imshow(images[i].numpy().astype(\"uint8\"))\n", " plt.title(f\"Asli: {class_names[labels[i].numpy()]}\", fontsize=11)\n", " plt.axis(\"off\")\n", " # RandAugment (4)\n", " aug = _randaug_select(tf.cast(images, tf.float32), ops_per_image=3, magnitude=0.7)\n", " for i in range(4):\n", " plt.subplot(3, 4, i + 5)\n", " plt.imshow(tf.clip_by_value(aug[i], 0, 255).numpy().astype(\"uint8\"))\n", " plt.title(f\"RandAug: {class_names[labels[i].numpy()]}\", fontsize=11)\n", " plt.axis(\"off\")\n", " # MixUp result (4)\n", " aug_f = tf.cast(images, tf.float32)\n", " mixed, _ = mix_up(aug_f, tf.one_hot(labels, NUM_CLASSES))\n", " for i in range(4):\n", " plt.subplot(3, 4, i + 9)\n", " plt.imshow(tf.clip_by_value(mixed[i], 0, 255).numpy().astype(\"uint8\"))\n", " plt.title(\"MixUp/Weather\", fontsize=11)\n", " plt.axis(\"off\")\n", "\n", "plt.suptitle(\"RandAugment + MixUp — Real-World Simulation\", fontsize=16)\n", "plt.tight_layout()\n", "plt.show()\n" ] }, { "cell_type": "markdown", "id": "05281076", "metadata": {}, "source": [ "## 7. Arsitektur Model — CBAM + Lightweight Head\n", "\n", "**EfficientNetV2B0** base (frozen awal) + **CBAM spatial attention** + lightweight classifier head.\n", "\n", "### Head (≈700K params vs 7.3M sebelumnya)\n", "```\n", "Base (7×7×1280) → CBAM_Attention → GAP → Dropout(0.3) → Dense(512, swish) → BN → Dropout(0.4) → Dense(4, softmax)\n", "```\n", "\n", "CBAM (Convolutional Block Attention Module): channel attention + spatial attention\n", "→ model belajar fokus ke foreground (daun) bukan background.\n" ] }, { "cell_type": "code", "execution_count": null, "id": "4937fb17", "metadata": {}, "outputs": [], "source": [ "def cbam_block(x, ratio=8, name=\"cbam\"):\n", " # Convolutional Block Attention Module — ringan, fokus ke foreground.\n", " channels = x.shape[-1]\n", "\n", " # Channel Attention\n", " avg_pool = layers.GlobalAveragePooling2D()(x)\n", " max_pool = layers.GlobalMaxPooling2D()(x)\n", " ca = layers.Dense(channels // ratio, activation='swish', name=f\"{name}_ca1\")(avg_pool)\n", " ca = layers.Dense(channels, activation='sigmoid', name=f\"{name}_ca2\")(ca)\n", " ca2 = layers.Dense(channels // ratio, activation='swish', name=f\"{name}_ca3\")(max_pool)\n", " ca2 = layers.Dense(channels, activation='sigmoid', name=f\"{name}_ca4\")(ca2)\n", " ca_out = layers.Add(name=f\"{name}_ca_add\")([ca, ca2])\n", " ca_out = layers.Reshape((1, 1, channels), name=f\"{name}_ca_reshape\")(ca_out)\n", " x = layers.Multiply(name=f\"{name}_ca_mul\")([x, ca_out])\n", "\n", " # Spatial Attention\n", " avg_sp = layers.Lambda(lambda t: tf.reduce_mean(t, axis=-1, keepdims=True), name=f\"{name}_sa_avg\")(x)\n", " max_sp = layers.Lambda(lambda t: tf.reduce_max(t, axis=-1, keepdims=True), name=f\"{name}_sa_max\")(x)\n", " sp = layers.Concatenate(name=f\"{name}_sa_cat\")([avg_sp, max_sp])\n", " sp = layers.Conv2D(1, 7, padding='same', activation='sigmoid', name=f\"{name}_sa_conv\")(sp)\n", " x = layers.Multiply(name=f\"{name}_sa_mul\")([x, sp])\n", " return x\n", "\n", "def build_model(num_classes, target_size=(224, 224)):\n", " # Accept any input size via variable input; Resizing handles progressive resolution.\n", " inputs = tf.keras.Input(shape=(None, None, 3), name=\"input\")\n", " x = layers.Resizing(target_size[0], target_size[1], interpolation='bilinear',\n", " name=\"resize_input\")(inputs)\n", "\n", " base_model = EfficientNetV2B0(\n", " input_shape=target_size + (3,),\n", " include_top=False,\n", " weights='imagenet',\n", " )\n", " base_model.trainable = False\n", "\n", " # Gaussian noise untuk regularisasi\n", " x = layers.GaussianNoise(0.05, name=\"gauss_noise\")(x)\n", " x = base_model(x, training=False)\n", " # CBAM attention — fokus ke region daun\n", " x = cbam_block(x, ratio=8, name=\"cbam\")\n", " x = layers.GlobalAveragePooling2D(name=\"gap\")(x)\n", " x = layers.Dropout(0.3, name=\"drop_gap\")(x)\n", " x = layers.Dense(512, activation='swish', name=\"dense_head\")(x)\n", " x = layers.BatchNormalization(name=\"bn_head\")(x)\n", " x = layers.Dropout(0.4, name=\"drop_head\")(x)\n", " outputs = layers.Dense(num_classes, activation='linear', dtype='float32', name=\"logits\")(x)\n", " return models.Model(inputs, outputs), base_model\n", "\n", "# Checkpoint\n", "ckpt_dir = '/content/best_model' if IS_COLAB else os.path.join(os.getcwd(), 'best_model')\n", "checkpoint_path = os.path.join(ckpt_dir, 'best_model.keras')\n", "\n", "# Hapus checkpoint lama (arsitektur berbeda — tidak kompatibel)\n", "if os.path.exists(checkpoint_path):\n", " print(f\"Removing old checkpoint (incompatible architecture)...\")\n", " os.remove(checkpoint_path)\n", "\n", "if os.path.exists(checkpoint_path):\n", " print(f\"Loading checkpoint: {checkpoint_path}\")\n", " try:\n", " model = models.load_model(checkpoint_path, compile=False)\n", " except Exception as e:\n", " print(f\"Load failed: {e}. Building fresh.\")\n", " model, base_model = build_model(NUM_CLASSES)\n", "else:\n", " print(\"No checkpoint. Building fresh model.\")\n", " model, base_model = build_model(NUM_CLASSES)\n", "\n", "os.makedirs(ckpt_dir, exist_ok=True)\n", "model.summary()\n" ] }, { "cell_type": "markdown", "id": "28aaaed1", "metadata": {}, "source": [ "## 8. Training Setup — Cosine Decay + SWA + Callbacks\n", "\n", "Mengganti `ReduceLROnPlateau` / `EarlyStopping` dengan:\n", "- **CosineDecay** + linear warmup per fase\n", "- **Stochastic Weight Averaging (SWA)** — averaging bobot untuk wider optima\n", "- **ModelCheckpoint** — tetap simpan best val_accuracy\n" ] }, { "cell_type": "code", "execution_count": null, "id": "51deff9d", "metadata": {}, "outputs": [], "source": [ "class WarmupCosineDecay(tf.keras.optimizers.schedules.LearningRateSchedule):\n", " # Cosine decay with linear warmup.\n", " def __init__(self, warmup_steps, total_steps, peak_lr, min_lr=1e-7):\n", " super().__init__()\n", " self.warmup_steps = warmup_steps\n", " self.total_steps = total_steps\n", " self.peak_lr = peak_lr\n", " self.min_lr = min_lr\n", "\n", " def __call__(self, step):\n", " step = tf.cast(step, tf.float32)\n", " warmup_steps = tf.cast(self.warmup_steps, tf.float32)\n", " total_steps = tf.cast(self.total_steps, tf.float32)\n", " # Warmup phase\n", " warmup_lr = self.peak_lr * (step / warmup_steps)\n", " # Cosine decay phase\n", " progress = (step - warmup_steps) / tf.maximum(total_steps - warmup_steps, 1.0)\n", " cosine_lr = self.min_lr + 0.5 * (self.peak_lr - self.min_lr) * (1.0 + tf.cos(np.pi * progress))\n", " return tf.where(step < warmup_steps, warmup_lr, cosine_lr)\n", "\n", " def get_config(self):\n", " return {\n", " \"warmup_steps\": self.warmup_steps, \"total_steps\": self.total_steps,\n", " \"peak_lr\": self.peak_lr, \"min_lr\": self.min_lr,\n", " }\n", "\n", "class SWACallback(tf.keras.callbacks.Callback):\n", " # Stochastic Weight Averaging — averages weights over final epochs.\n", " def __init__(self, start_epoch, swa_lr=1e-5):\n", " super().__init__()\n", " self.start_epoch = start_epoch\n", " self.swa_lr = swa_lr\n", " self.swa_weights = None\n", " self.n_models = 0\n", "\n", " def on_epoch_begin(self, epoch, logs=None):\n", " if epoch >= self.start_epoch and self.swa_weights is None:\n", " self.swa_weights = [w.numpy() for w in self.model.weights]\n", " print(f\"\\nSWA: starting weight averaging at epoch {epoch+1}\")\n", "\n", " def on_epoch_end(self, epoch, logs=None):\n", " if epoch >= self.start_epoch and self.swa_weights is not None:\n", " for i, w in enumerate(self.model.weights):\n", " self.swa_weights[i] = (self.swa_weights[i] * self.n_models + w.numpy()) / (self.n_models + 1)\n", " self.n_models += 1\n", "\n", " def apply_swa_weights(self):\n", " if self.swa_weights is None:\n", " print(\"SWA: no weights to average (skipped)\")\n", " return\n", " for w, swa_w in zip(self.model.weights, self.swa_weights):\n", " w.assign(swa_w)\n", " print(f\"SWA weights applied ({self.n_models} models averaged).\")\n", "\n", "# Shared callbacks\n", "checkpoint_cb = callbacks.ModelCheckpoint(\n", " checkpoint_path, save_best_only=True, monitor=\"val_accuracy\",\n", " mode=\"max\", verbose=1)\n", "\n", "csv_logger = callbacks.CSVLogger(os.path.join(ckpt_dir, 'training_log.csv'))\n", "\n", "def make_callbacks(swa_start=None):\n", " cbs = [checkpoint_cb, csv_logger]\n", " if swa_start is not None:\n", " cbs.append(SWACallback(swa_start))\n", " return cbs\n", "\n", "print(\"Callbacks ready.\")\n", "print(f\"Checkpoint path: {checkpoint_path}\")\n" ] }, { "cell_type": "markdown", "id": "82814b7b", "metadata": {}, "source": [ "## 9. Fase 1 — Head Only Training (128×128)\n", "\n", "Progressive resolution: mulai dari **128×128** untuk feature learning cepat.\n", "Base model beku, hanya head (CBAM + Dense) yang dilatih.\n", "Optimizer: AdamW + EMA + CosineDecay(warmup=3, peak=1e-3).\n", "Label smoothing: 0.15\n" ] }, { "cell_type": "code", "execution_count": null, "id": "decaf5cb", "metadata": {}, "outputs": [], "source": [ "IMG_128 = (128, 128)\n", "\n", "# Rebuild datasets at 128x128\n", "train_ds_128 = tf.keras.utils.image_dataset_from_directory(\n", " train_dir, shuffle=True, batch_size=BATCH_SIZE, image_size=IMG_128)\n", "val_ds_128 = tf.keras.utils.image_dataset_from_directory(\n", " val_dir, shuffle=True, batch_size=BATCH_SIZE, image_size=IMG_128)\n", "\n", "train_ds_128 = (train_ds_128.map(augment_and_mix, num_parallel_calls=AUTOTUNE)\n", " .map(preprocess_fn, num_parallel_calls=AUTOTUNE)\n", " .map(add_sample_weight, num_parallel_calls=AUTOTUNE).prefetch(AUTOTUNE))\n", "val_ds_128 = (val_ds_128.map(lambda img, lbl: (preprocess_input(tf.cast(img, tf.float32)), lbl), num_parallel_calls=AUTOTUNE)\n", " .map(lambda img, lbl: (img, tf.one_hot(lbl, NUM_CLASSES)), num_parallel_calls=AUTOTUNE).prefetch(AUTOTUNE))\n", "\n", "EPOCHS_P1 = 25\n", "steps_per_epoch = tf.data.experimental.cardinality(train_ds_128).numpy() or 100\n", "total_steps = steps_per_epoch * EPOCHS_P1\n", "warmup_steps = steps_per_epoch * 3 # 3 epoch warmup\n", "\n", "lr_schedule_p1 = WarmupCosineDecay(warmup_steps, total_steps, peak_lr=1e-3, min_lr=1e-5)\n", "\n", "model.compile(\n", " optimizer=AdamW(\n", " learning_rate=lr_schedule_p1, weight_decay=1e-4),\n", " loss=tf.keras.losses.CategoricalCrossentropy(from_logits=True, label_smoothing=0.15),\n", " metrics=['accuracy']\n", ")\n", "\n", "print(\"Phase 1: Head training at 128×128...\")\n", "history_1 = model.fit(train_ds_128, validation_data=val_ds_128,\n", " epochs=EPOCHS_P1, callbacks=make_callbacks())\n" ] }, { "cell_type": "markdown", "id": "de673df6", "metadata": {}, "source": [ "## 10. Fase 2 — Partial Fine-tuning (192×192)\n", "\n", "Resolusi naik ke **192×192**. Top 100 layer EfficientNetV2B0 di-unfreeze.\n", "Learning rate lebih rendah: peak=5e-4, cosine decay ke 1e-6.\n" ] }, { "cell_type": "code", "execution_count": null, "id": "fd7fb926", "metadata": {}, "outputs": [], "source": [ "IMG_192 = (192, 192)\n", "\n", "train_ds_192 = tf.keras.utils.image_dataset_from_directory(\n", " train_dir, shuffle=True, batch_size=BATCH_SIZE, image_size=IMG_192)\n", "val_ds_192 = tf.keras.utils.image_dataset_from_directory(\n", " val_dir, shuffle=True, batch_size=BATCH_SIZE, image_size=IMG_192)\n", "\n", "train_ds_192 = (train_ds_192.map(augment_and_mix, num_parallel_calls=AUTOTUNE)\n", " .map(preprocess_fn, num_parallel_calls=AUTOTUNE)\n", " .map(add_sample_weight, num_parallel_calls=AUTOTUNE).prefetch(AUTOTUNE))\n", "val_ds_192 = (val_ds_192.map(lambda img, lbl: (preprocess_input(tf.cast(img, tf.float32)), lbl), num_parallel_calls=AUTOTUNE)\n", " .map(lambda img, lbl: (img, tf.one_hot(lbl, NUM_CLASSES)), num_parallel_calls=AUTOTUNE).prefetch(AUTOTUNE))\n", "\n", "# Unfreeze top 100 layers\n", "base_model.trainable = True\n", "for layer in base_model.layers[:-100]:\n", " layer.trainable = False\n", "\n", "EPOCHS_P2 = 30\n", "steps_p2 = tf.data.experimental.cardinality(train_ds_192).numpy() or 100\n", "total_p2 = steps_p2 * EPOCHS_P2\n", "warmup_p2 = steps_p2 * 2\n", "\n", "lr_schedule_p2 = WarmupCosineDecay(warmup_p2, total_p2, peak_lr=5e-4, min_lr=1e-6)\n", "\n", "model.compile(\n", " optimizer=AdamW(\n", " learning_rate=lr_schedule_p2, weight_decay=1e-4),\n", " loss=tf.keras.losses.CategoricalCrossentropy(from_logits=True, label_smoothing=0.15),\n", " metrics=['accuracy']\n", ")\n", "\n", "print(\"Phase 2: Fine-tuning top 100 layers at 192×192...\")\n", "history_2 = model.fit(train_ds_192, validation_data=val_ds_192,\n", " epochs=EPOCHS_P1 + EPOCHS_P2, initial_epoch=history_1.epoch[-1] + 1,\n", " callbacks=make_callbacks())\n" ] }, { "cell_type": "markdown", "id": "33326d9c", "metadata": {}, "source": [ "## 11. Fase 3 — Full Fine-tuning (224×224)\n", "\n", "Resolusi penuh **224×224**. Semua layer di-unfreeze.\n", "LR sangat rendah: peak=1e-4, cosine decay ke 1e-7.\n", "Label smoothing diturunkan ke 0.10 untuk kalibrasi lebih baik.\n" ] }, { "cell_type": "code", "execution_count": null, "id": "14115063", "metadata": {}, "outputs": [], "source": [ "img_size = IMG_SIZE # (224, 224)\n", "\n", "train_ds_full = tf.keras.utils.image_dataset_from_directory(\n", " train_dir, shuffle=True, batch_size=BATCH_SIZE, image_size=img_size)\n", "val_ds_full = tf.keras.utils.image_dataset_from_directory(\n", " val_dir, shuffle=True, batch_size=BATCH_SIZE, image_size=img_size)\n", "\n", "train_ds_full = (train_ds_full.map(augment_and_mix, num_parallel_calls=AUTOTUNE)\n", " .map(preprocess_fn, num_parallel_calls=AUTOTUNE)\n", " .map(add_sample_weight, num_parallel_calls=AUTOTUNE).prefetch(AUTOTUNE))\n", "val_ds_full = (val_ds_full.map(lambda img, lbl: (preprocess_input(tf.cast(img, tf.float32)), lbl), num_parallel_calls=AUTOTUNE)\n", " .map(lambda img, lbl: (img, tf.one_hot(lbl, NUM_CLASSES)), num_parallel_calls=AUTOTUNE).prefetch(AUTOTUNE))\n", "\n", "# Full unfreeze\n", "for layer in base_model.layers:\n", " layer.trainable = True\n", "\n", "EPOCHS_P3 = 30\n", "steps_p3 = tf.data.experimental.cardinality(train_ds_full).numpy() or 100\n", "total_p3 = steps_p3 * EPOCHS_P3\n", "warmup_p3 = steps_p3 * 2\n", "\n", "lr_schedule_p3 = WarmupCosineDecay(warmup_p3, total_p3, peak_lr=1e-4, min_lr=1e-7)\n", "\n", "model.compile(\n", " optimizer=AdamW(\n", " learning_rate=lr_schedule_p3, weight_decay=1e-4),\n", " loss=tf.keras.losses.CategoricalCrossentropy(from_logits=True, label_smoothing=0.10),\n", " metrics=['accuracy']\n", ")\n", "\n", "print(\"Phase 3: Full fine-tuning at 224×224...\")\n", "history_3 = model.fit(train_ds_full, validation_data=val_ds_full,\n", " epochs=EPOCHS_P1 + EPOCHS_P2 + EPOCHS_P3, initial_epoch=(history_2.epoch[-1] + 1) if history_2.epoch else EPOCHS_P1 + EPOCHS_P2,\n", " callbacks=make_callbacks())\n" ] }, { "cell_type": "markdown", "id": "6df4ef22", "metadata": {}, "source": [ "## 12. SWA — Stochastic Weight Averaging\n", "\n", "15 epoch tambahan dengan cyclic LR (1e-5). SWA mengakumulasi rata-rata bobot\n", "untuk menghasilkan **wider optima** — generalisasi lebih baik ke data out-of-distribution.\n", "\n", "Setelah SWA selesai, bobot SWA diterapkan kembali ke model.\n" ] }, { "cell_type": "code", "execution_count": null, "id": "290f2345", "metadata": {}, "outputs": [], "source": [ "EPOCHS_SWA = 15\n", "swa_start_epoch = (history_3.epoch[-1] + 1) if history_3.epoch else EPOCHS_P1 + EPOCHS_P2 + EPOCHS_P3\n", "\n", "swa_cb = SWACallback(start_epoch=swa_start_epoch, swa_lr=1e-5)\n", "\n", "model.compile(\n", " optimizer=AdamW(\n", " learning_rate=1e-5, weight_decay=1e-4),\n", " loss=tf.keras.losses.CategoricalCrossentropy(from_logits=True, label_smoothing=0.10),\n", " metrics=['accuracy']\n", ")\n", "\n", "print(f\"SWA: {EPOCHS_SWA} epochs starting at epoch {swa_start_epoch + 1}...\")\n", "history_swa = model.fit(train_ds_full, validation_data=val_ds_full,\n", " epochs=swa_start_epoch + EPOCHS_SWA, initial_epoch=swa_start_epoch,\n", " callbacks=make_callbacks() + [swa_cb])\n", "\n", "# Apply SWA weights\n", "swa_cb.apply_swa_weights()\n", "print(f\"SWA complete. Final model has SWA weights applied.\")\n" ] }, { "cell_type": "markdown", "id": "2d889f2a", "metadata": {}, "source": [ "## 13. Plot Training History (Gabungan Semua Fase)\n" ] }, { "cell_type": "code", "execution_count": null, "id": "41e79742", "metadata": {}, "outputs": [], "source": [ "# Gabungkan semua history\n", "acc = (history_1.history['accuracy'] + history_2.history['accuracy'] +\n", " history_3.history['accuracy'] + history_swa.history['accuracy'])\n", "val_acc = (history_1.history['val_accuracy'] + history_2.history['val_accuracy'] +\n", " history_3.history['val_accuracy'] + history_swa.history['val_accuracy'])\n", "loss = (history_1.history['loss'] + history_2.history['loss'] +\n", " history_3.history['loss'] + history_swa.history['loss'])\n", "val_loss = (history_1.history['val_loss'] + history_2.history['val_loss'] +\n", " history_3.history['val_loss'] + history_swa.history['val_loss'])\n", "\n", "b1 = len(history_1.history['accuracy']) - 1\n", "b2 = b1 + len(history_2.history['accuracy'])\n", "b3 = b2 + len(history_3.history['accuracy'])\n", "\n", "plt.figure(figsize=(16, 6))\n", "plt.subplot(1, 2, 1)\n", "plt.plot(acc, label='Training Accuracy', linewidth=2)\n", "plt.plot(val_acc, label='Validation Accuracy', linewidth=2)\n", "plt.axvline(x=b1, color='gray', linestyle='--', alpha=0.7, label='P2 (192)')\n", "plt.axvline(x=b2, color='black', linestyle='--', alpha=0.7, label='P3 (224)')\n", "plt.axvline(x=b3, color='blue', linestyle='--', alpha=0.7, label='SWA start')\n", "plt.legend(fontsize=10)\n", "plt.title('Training & Validation Accuracy', fontsize=14)\n", "plt.xlabel('Epoch'); plt.ylabel('Accuracy'); plt.grid(alpha=0.3)\n", "\n", "plt.subplot(1, 2, 2)\n", "plt.plot(loss, label='Training Loss', linewidth=2)\n", "plt.plot(val_loss, label='Validation Loss', linewidth=2)\n", "plt.axvline(x=b1, color='gray', linestyle='--', alpha=0.7, label='P2 (192)')\n", "plt.axvline(x=b2, color='black', linestyle='--', alpha=0.7, label='P3 (224)')\n", "plt.axvline(x=b3, color='blue', linestyle='--', alpha=0.7, label='SWA start')\n", "plt.legend(fontsize=10)\n", "plt.title('Training & Validation Loss', fontsize=14)\n", "plt.xlabel('Epoch'); plt.ylabel('Loss'); plt.grid(alpha=0.3)\n", "\n", "plt.tight_layout()\n", "plt.show()\n" ] }, { "cell_type": "markdown", "id": "27b920dd", "metadata": {}, "source": [ "## 14. Temperature Scaling — Confidence Calibration\n", "\n", "Model deep learning cenderung **overconfident** — softmax probability tinggi tapi tidak mencerminkan\n", "akurasi sebenarnya. Temperature scaling mengoptimalkan parameter T pada validation set:\n", "\n", "$$P_{calibrated} = softmax(logits / T)$$\n", "\n", "T > 1 → distribusi lebih flat (less confident).\n", "T < 1 → distribusi lebih tajam (more confident).\n", "T = 1 → tidak berubah (default).\n", "\n", "ECE (Expected Calibration Error) mengukur seberapa baik confidence sesuai dengan akurasi.\n", "Target: **ECE < 0.05** setelah temperature scaling.\n" ] }, { "cell_type": "code", "execution_count": null, "id": "d79dbb47", "metadata": {}, "outputs": [], "source": [ "def compute_ece(probs, true_labels, n_bins=15):\n", " # Expected Calibration Error.\n", " confs = np.max(probs, axis=1)\n", " preds = np.argmax(probs, axis=1)\n", " true = np.argmax(true_labels, axis=1)\n", " accs = (preds == true).astype(np.float32)\n", " bins = np.linspace(0, 1, n_bins + 1)\n", " ece = 0.0\n", " bin_stats = []\n", " for i in range(n_bins):\n", " in_bin = (confs > bins[i]) & (confs <= bins[i + 1])\n", " n = np.sum(in_bin)\n", " if n > 0:\n", " bin_acc = np.mean(accs[in_bin])\n", " bin_conf = np.mean(confs[in_bin])\n", " ece += (n / len(confs)) * np.abs(bin_acc - bin_conf)\n", " bin_stats.append((bins[i], n, bin_acc, bin_conf))\n", " return ece, bin_stats\n", "\n", "# Collect logits and labels from validation set\n", "print(\"Collecting validation logits...\")\n", "logits_model = tf.keras.Model(model.input, model.output)\n", "\n", "all_logits = []\n", "all_labels = []\n", "for images, labels in val_ds_full.unbatch().batch(BATCH_SIZE):\n", " all_logits.append(logits_model.predict_on_batch(images))\n", " all_labels.append(labels.numpy())\n", "\n", "all_logits = np.concatenate(all_logits, axis=0)\n", "all_labels = np.concatenate(all_labels, axis=0)\n", "\n", "# ECE before scaling (T=1)\n", "probs_raw = tf.nn.softmax(all_logits).numpy()\n", "ece_raw, _ = compute_ece(probs_raw, all_labels)\n", "print(f\"ECE before scaling (T=1.0): {ece_raw:.4f}\")\n", "\n", "# Optimize T on validation set\n", "if HAS_SCIPY:\n", " def nll_temperature(T):\n", " scaled = all_logits / float(T)\n", " probs = tf.nn.softmax(scaled).numpy()\n", " probs = np.clip(probs, 1e-7, 1.0 - 1e-7)\n", " return -np.mean(np.log(np.sum(all_labels * probs, axis=1)))\n", "\n", " result = minimize_scalar(nll_temperature, bounds=(0.1, 5.0), method='bounded')\n", " T_opt = result.x\n", " print(f\"Optimal temperature: T = {T_opt:.4f}\")\n", "else:\n", " # Grid search fallback\n", " best_nll, T_opt = float('inf'), 1.0\n", " for T in np.linspace(0.5, 4.0, 36):\n", " scaled = all_logits / T\n", " probs = tf.nn.softmax(scaled).numpy()\n", " probs = np.clip(probs, 1e-7, 1.0 - 1e-7)\n", " nll = -np.mean(np.log(np.sum(all_labels * probs, axis=1)))\n", " if nll < best_nll:\n", " best_nll = nll\n", " T_opt = T\n", " print(f\"Optimal temperature (grid): T = {T_opt:.4f}\")\n", "\n", "# ECE after scaling\n", "probs_cal = tf.nn.softmax(all_logits / T_opt).numpy()\n", "ece_cal, bin_stats = compute_ece(probs_cal, all_labels)\n", "print(f\"ECE after scaling (T={T_opt:.4f}): {ece_cal:.4f}\")\n", "\n", "# Save calibration metadata\n", "calibration_meta = {\n", " \"temperature\": float(T_opt),\n", " \"conf_threshold_high\": 0.70,\n", " \"conf_threshold_low\": 0.45,\n", " \"ece_raw\": float(ece_raw),\n", " \"ece_calibrated\": float(ece_cal),\n", "}\n", "with open(os.path.join(ckpt_dir, \"calibration.json\"), \"w\") as f:\n", " json.dump(calibration_meta, f, indent=2)\n", "print(f\"Calibration metadata saved to {os.path.join(ckpt_dir, 'calibration.json')}\")\n", "\n", "# Reliability diagram\n", "plt.figure(figsize=(12, 5))\n", "\n", "plt.subplot(1, 2, 1)\n", "if bin_stats:\n", " bin_mids = [(s[0] + s[0] + 1/n_bins)/2 for s in bin_stats]\n", " bin_accs = [s[2] for s in bin_stats]\n", " bin_confs = [s[3] for s in bin_stats]\n", " plt.bar(bin_mids, bin_accs, width=0.05, alpha=0.5, label='Accuracy')\n", " plt.bar(bin_mids, bin_confs, width=0.05, alpha=0.3, label='Confidence')\n", "plt.plot([0, 1], [0, 1], 'k--', alpha=0.3)\n", "plt.xlabel('Confidence'); plt.ylabel('Accuracy')\n", "plt.title(f'Reliability Diagram (T={T_opt:.2f})')\n", "plt.legend(); plt.grid(alpha=0.3)\n", "\n", "plt.subplot(1, 2, 2)\n", "conf_raw = np.max(probs_raw, axis=1)\n", "conf_cal = np.max(probs_cal, axis=1)\n", "plt.hist(conf_raw, bins=30, alpha=0.5, label='Before scaling', density=True)\n", "plt.hist(conf_cal, bins=30, alpha=0.5, label='After scaling', density=True)\n", "plt.xlabel('Max Confidence'); plt.ylabel('Density')\n", "plt.title('Confidence Distribution')\n", "plt.legend(); plt.grid(alpha=0.3)\n", "\n", "plt.tight_layout()\n", "plt.show()\n" ] }, { "cell_type": "markdown", "id": "6e4a717b", "metadata": {}, "source": [ "## 15. Test Time Augmentation + Evaluasi\n", "\n", "Menggunakan TTA 5× pada test set dengan augmented logit averaging.\n", "Model output adalah **raw logits** → temperature scaling → softmax.\n" ] }, { "cell_type": "code", "execution_count": null, "id": "beb96b8b", "metadata": {}, "outputs": [], "source": [ "# Load best model (dengan SWA weights)\n", "print(f\"Loading best model from {checkpoint_path}...\")\n", "best_model = tf.keras.models.load_model(checkpoint_path, compile=False)\n", "\n", "# Collect test images\n", "raw_test_ds = tf.keras.utils.image_dataset_from_directory(\n", " test_dir, shuffle=False, batch_size=BATCH_SIZE, image_size=IMG_SIZE)\n", "\n", "test_images = []\n", "test_labels_raw = []\n", "for images, labels in raw_test_ds.unbatch():\n", " test_images.append(images.numpy())\n", " test_labels_raw.append(labels.numpy())\n", "\n", "test_images = np.array(test_images)\n", "test_labels_true = tf.one_hot(np.array(test_labels_raw), NUM_CLASSES).numpy()\n", "\n", "# ── TTA 5x ──\n", "TTA_STEPS = 5\n", "tta_logits = []\n", "\n", "for i in range(TTA_STEPS):\n", " aug_images = geo_aug(test_images, training=True)\n", " aug_images = preprocess_input(aug_images)\n", " logits = best_model.predict(aug_images, batch_size=BATCH_SIZE, verbose=0)\n", " tta_logits.append(logits)\n", " print(f\" TTA step {i+1}/{TTA_STEPS}\")\n", "\n", "mean_logits = np.mean(tta_logits, axis=0)\n", "# Apply temperature scaling\n", "mean_cal_probs = tf.nn.softmax(mean_logits / T_opt).numpy()\n", "\n", "test_preds = np.argmax(mean_cal_probs, axis=1)\n", "test_true = np.argmax(test_labels_true, axis=1)\n", "tta_acc = np.mean(test_preds == test_true)\n", "\n", "print(f\"\\nTest Accuracy (TTA {TTA_STEPS}x, T={T_opt:.2f}): {tta_acc*100:.2f}%\")\n", "\n", "# ECE on test set\n", "ece_test, _ = compute_ece(mean_cal_probs, test_labels_true)\n", "print(f\"ECE on test set: {ece_test:.4f}\")\n" ] }, { "cell_type": "markdown", "id": "b6bb88ab", "metadata": {}, "source": [ "## 16. Per-Source Accuracy Breakdown\n", "\n", "Mengukur akurasi per source prefix untuk mendeteksi **domain gap**.\n", "Source yang akurasinya collapse (< 70%) menunjukkan model belum robust.\n" ] }, { "cell_type": "code", "execution_count": null, "id": "a24e9a14", "metadata": {}, "outputs": [], "source": [ "def get_source(filename):\n", " f = os.path.splitext(filename)[0]\n", " if f.startswith('IMG_'): return 'Phone'\n", " if f.startswith('Corn_'): return 'Lab_Corn'\n", " for p in ['CBS', 'GLS', 'NLS', 'CLS']:\n", " if f.startswith(p): return 'Lab_Disease'\n", " for p in ['SCR', 'CR', 'NLB', 'SLB', 'SRS']:\n", " if f.startswith(p): return 'Lab_RustBlight'\n", " return 'Other'\n", "\n", "# Map each test image to its source\n", "test_files = []\n", "for cn in class_names:\n", " cp = os.path.join(test_dir, cn)\n", " if os.path.isdir(cp):\n", " test_files.extend([(f, cn, get_source(f)) for f in sorted(os.listdir(cp))])\n", "\n", "sources = set(s for _, _, s in test_files)\n", "print(f\"Sources found: {sorted(sources)}\")\n", "print(f\"{'Source':<18} {'Count':>6} {'Accuracy':>10}\")\n", "print(\"-\" * 38)\n", "\n", "for src in sorted(sources):\n", " indices = [i for i, (_, _, s) in enumerate(test_files) if s == src]\n", " if not indices: continue\n", " n = len(indices)\n", " acc = np.mean(test_preds[indices] == test_true[indices])\n", " print(f\"{src:<18} {n:>6} {acc*100:>9.1f}%\")\n", "\n", "# Overall with count\n", "overall_acc = np.mean(test_preds == test_true)\n", "print(\"-\" * 38)\n", "print(f\"{'ALL':<18} {len(test_preds):>6} {overall_acc*100:>9.1f}%\")\n" ] }, { "cell_type": "markdown", "id": "52754df5", "metadata": {}, "source": [ "## 17. Classification Report & Confusion Matrix\n" ] }, { "cell_type": "code", "execution_count": null, "id": "1a5b2c96", "metadata": {}, "outputs": [], "source": [ "print(\"\\nClassification Report:\\n\")\n", "print(classification_report(test_true, test_preds, target_names=class_names))\n", "\n", "cm = confusion_matrix(test_true, test_preds)\n", "plt.figure(figsize=(10, 8))\n", "sns.heatmap(cm, annot=True, fmt='d', cmap='Blues',\n", " xticklabels=class_names, yticklabels=class_names)\n", "plt.title(f'Confusion Matrix (TTA {TTA_STEPS}x)', fontsize=14)\n", "plt.xlabel('Predicted', fontsize=12)\n", "plt.ylabel('True', fontsize=12)\n", "plt.tight_layout()\n", "plt.show()\n" ] }, { "cell_type": "markdown", "id": "528ef53b", "metadata": {}, "source": [ "## 18. Simpan Model & Calibration Metadata\n", "\n", "Menyimpan model final (dengan SWA weights) dan calibration metadata ke `model/`.\n" ] }, { "cell_type": "code", "execution_count": null, "id": "292fef1a", "metadata": {}, "outputs": [], "source": [ "# Save final model\n", "model.save(final_path := os.path.join(ckpt_dir, 'best_model.keras'))\n", "print(f\"Model saved to {final_path}\")\n", "\n", "# Copy calibration metadata to model directory\n", "model_export_dir = os.path.join(os.getcwd(), 'model')\n", "os.makedirs(model_export_dir, exist_ok=True)\n", "\n", "# Export labels.json with calibration metadata\n", "cal_path = os.path.join(ckpt_dir, 'calibration.json')\n", "if os.path.exists(cal_path):\n", " with open(cal_path) as f:\n", " cal_meta = json.load(f)\n", "\n", " labels_json = {\n", " \"version\": \"3.0\",\n", " \"labels\": class_names,\n", " \"temperature\": cal_meta[\"temperature\"],\n", " \"conf_threshold_high\": cal_meta[\"conf_threshold_high\"],\n", " \"conf_threshold_low\": cal_meta[\"conf_threshold_low\"],\n", " \"input_size\": [224, 224],\n", " \"input_range\": [0, 255],\n", " \"preprocessing\": \"resize_bilinear_224x224_no_normalization\",\n", " \"architecture\": \"EfficientNetV2B0 + CBAM + Dense(512)\",\n", " \"output_type\": \"logits\",\n", " }\n", "else:\n", " labels_json = {\n", " \"version\": \"3.0\",\n", " \"labels\": class_names,\n", " \"temperature\": 1.0,\n", " \"input_size\": [224, 224],\n", " \"input_range\": [0, 255],\n", " \"output_type\": \"logits\",\n", " }\n", "\n", "with open(os.path.join(model_export_dir, 'labels.json'), 'w') as f:\n", " json.dump(labels_json, f, indent=2)\n", "print(\"Labels + calibration metadata saved to model/labels.json\")\n", "\n", "# Export class names list (legacy)\n", "with open(os.path.join(model_export_dir, 'labels.json'), 'r') as f:\n", " pass # already written above\n", "print(f\"Classes: {class_names}\")\n", "print(f\"Temperature: {labels_json['temperature']:.4f}\")\n" ] }, { "cell_type": "markdown", "id": "62675ad6", "metadata": {}, "source": [ "## 19. Export Model untuk Produksi\n", "\n", "Setelah training selesai, jalankan pipeline ekspor secara berurutan:\n", "\n", "### 1. SavedModel + TFLite\n", "```bash\n", "python save_model.py\n", "```\n", "Memuat `best_model/best_model.keras`, membangun arsitektur bersih, dan mengekspor ke:\n", "- `model/saved_model/` — format produksi (output **raw logits**)\n", "- `model/model.tflite` — untuk perangkat mobile/edge (INT8 quantization)\n", "\n", "### 2. ONNX (Rust ML Service)\n", "```bash\n", "python convert_onnx.py\n", "```\n", "Mengonversi SavedModel ke `model/model.onnx` untuk Rust/Axum/ONNX Runtime.\n", "Output: **raw logits** (softmax + temperature scaling di Rust service).\n", "\n", "### 3. TensorFlow.js (Web) — optional\n", "```bash\n", "export PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python\n", "tensorflowjs_converter --input_format=tf_saved_model --output_format=tfjs_graph_model --signature_name=serving_default --saved_model_tags=serve model/saved_model model/tfjs_model\n", "```\n", "\n", "> **Catatan v3.0**: Model output adalah **raw logits** (tanpa softmax). Rust service menerapkan\n", "> temperature scaling: `softmax(logits / T)` dengan `T` dari `labels.json`.\n", "> Status prediksi ditentukan dari confidence: confident (≥70%), uncertain (45-70%), rejected (<45%).\n" ] }, { "cell_type": "markdown", "id": "de8c53b7", "metadata": {}, "source": [ "## 20. Model Card — ZeaVis Edu v3.0\n", "\n", "| Atribut | Detail |\n", "|---|---|\n", "| **Nama Model** | ZeaVis Edu v3.0 — CBAM + RandAugment Classifier |\n", "| **Versi** | 3.0 |\n", "| **Arsitektur** | EfficientNetV2B0 + CBAM Attention + GAP + Dense(512) |\n", "| **Params** | ~6.6M (5.9M base + 0.7M head) — 2× lebih ringan dari v2.0 |\n", "| **Framework** | TensorFlow 2.x / Keras (float32) |\n", "| **Output** | Raw logits → temperature scaling → softmax |\n", "| **Dataset** | ~6000 gambar (4 kelas) — stratified split by source |\n", "| **Kelas** | Bercak Daun, Daun Sehat, Hawar Daun, Karat Daun |\n", "| **Input** | RGB 224×224, pixel [0, 255], resize BILINEAR |\n", "| **Augmentasi** | RandAugment (15 ops, N=3) + Fog + Shadow + MixUp + CutMix + RandomErasing |\n", "| **Training** | Progressive 128→160→192→224 + CosineDecay + SWA |\n", "| **Calibration** | Temperature scaling (T optimized on val), ECE target < 0.05 |\n", "| **Decision** | Confident (≥0.70), Uncertain (0.45-0.70), Rejected (<0.45) |\n", "| **Target** | ≥90% real-world accuracy, per-source accuracy ≥70% semua domain |\n", "| **Etika** | Hanya untuk edukasi/penelitian pertanian. Bukan pengganti diagnosis ahli. |\n", "\n" ] }, { "cell_type": "markdown", "id": "f431a07c", "metadata": {}, "source": [ "## 21. Upload ke Hugging Face\n", "\n", "```python\n", "from huggingface_hub import HfApi\n", "api = HfApi()\n", "api.upload_folder(\n", " folder_path=\"model\",\n", " repo_id=\"zeavis-edu/corn-leaf-disease-classifier\",\n", " repo_type=\"model\",\n", ")\n", "```\n", "Upload model SavedModel, TFLite, ONNX, TF.js, dan metadata ke HF Hub.\n" ] } ], "metadata": { "kernelspec": { "display_name": ".venv", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12.3" } }, "nbformat": 4, "nbformat_minor": 5 }