1581 lines
67 KiB
Plaintext
1581 lines
67 KiB
Plaintext
{
|
||
"cells": [
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "bea416d7",
|
||
"metadata": {},
|
||
"source": [
|
||
"# ZeaVis Edu — Corn Leaf Disease Classifier v3.0\n",
|
||
"\n",
|
||
"Mengklasifikasikan penyakit daun jagung (Bercak Daun, Hawar Daun, Karat Daun, Daun Sehat)\n",
|
||
"menggunakan EfficientNetV2B0 dengan **CBAM spatial attention**, **RandAugment + weather simulation**,\n",
|
||
"dan **temperature-scaled confidence calibration** untuk deployment real-world.\n",
|
||
"\n",
|
||
"Fokus v3.0: **robustness dunia nyata** — berbagai pencahayaan, resolusi, angle, dan background.\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "aba4b688",
|
||
"metadata": {},
|
||
"source": [
|
||
"## 1. Persiapan Lingkungan\n",
|
||
"\n",
|
||
"Mengimpor pustaka, mengatur seed, dan mengoptimalkan konfigurasi.\n",
|
||
"**Presisi float32**, resolusi target **224×224** (EfficientNetV2B0).\n",
|
||
"Augmentasi real-world via RandAugment pool 15 transformasi.\n",
|
||
"Confidence calibration via temperature scaling.\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "9dc08169",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"!pip install -r requirements.txt\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "9d108f5e",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"import os, shutil, zipfile, random, time, json\n",
|
||
"from collections import Counter\n",
|
||
"import numpy as np\n",
|
||
"import matplotlib.pyplot as plt\n",
|
||
"import seaborn as sns\n",
|
||
"from PIL import Image\n",
|
||
"\n",
|
||
"import tensorflow as tf\n",
|
||
"from tensorflow.keras import layers, models, callbacks\n",
|
||
"from tensorflow.keras.applications import EfficientNetV2B0\n",
|
||
"from tensorflow.keras.applications.efficientnet_v2 import preprocess_input\n",
|
||
"from tensorflow.keras.optimizers import AdamW\n",
|
||
"from tensorflow.keras.optimizers.schedules import CosineDecay\n",
|
||
"from sklearn.metrics import classification_report, confusion_matrix\n",
|
||
"from sklearn.utils.class_weight import compute_class_weight\n",
|
||
"from sklearn.model_selection import train_test_split\n",
|
||
"\n",
|
||
"# Optional: perceptual hashing for dedup (pip install imagehash)\n",
|
||
"try:\n",
|
||
" import imagehash\n",
|
||
" HAS_IMAGEHASH = True\n",
|
||
"except ImportError:\n",
|
||
" HAS_IMAGEHASH = False\n",
|
||
"\n",
|
||
"# Optional: scipy for temperature optimization\n",
|
||
"try:\n",
|
||
" from scipy.optimize import minimize_scalar\n",
|
||
" HAS_SCIPY = True\n",
|
||
"except ImportError:\n",
|
||
" HAS_SCIPY = False\n",
|
||
"\n",
|
||
"# Detect environment\n",
|
||
"try:\n",
|
||
" from google.colab import drive\n",
|
||
" IS_COLAB = True\n",
|
||
" print(\"Running on Google Colab\")\n",
|
||
"except ModuleNotFoundError:\n",
|
||
" IS_COLAB = False\n",
|
||
" print(f\"Running locally (TF {tf.__version__}, GPU: {tf.config.list_physical_devices('GPU')})\")\n",
|
||
"\n",
|
||
"tf.keras.mixed_precision.set_global_policy('float32')\n",
|
||
"\n",
|
||
"# Hyperparams\n",
|
||
"IMG_SIZE = (224, 224)\n",
|
||
"BATCH_SIZE = 32\n",
|
||
"SEED = 42\n",
|
||
"random.seed(SEED)\n",
|
||
"np.random.seed(SEED)\n",
|
||
"tf.random.set_seed(SEED)\n",
|
||
"\n",
|
||
"AUTOTUNE = tf.data.AUTOTUNE\n",
|
||
"print(f\"Setup OK. IMG={IMG_SIZE}, BATCH={BATCH_SIZE}\")\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "06382ddf",
|
||
"metadata": {},
|
||
"source": [
|
||
"## 2. Download dan Ekstraksi Dataset\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "26afde45",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"if IS_COLAB:\n",
|
||
" drive.mount('/content/drive')\n",
|
||
" archive_path = '/content/drive/MyDrive/jagung/dataset.zip'\n",
|
||
" destination_path = '/content/dataset.zip'\n",
|
||
" extract_path = '/content/dataset'\n",
|
||
"else:\n",
|
||
" base = os.getcwd()\n",
|
||
" archive_path = os.path.join(base, 'dataset.zip')\n",
|
||
" destination_path = archive_path\n",
|
||
" extract_path = os.path.join(base, 'dataset')\n",
|
||
"\n",
|
||
"if os.path.exists(destination_path):\n",
|
||
" if not os.path.exists(extract_path) or len(os.listdir(extract_path)) == 0:\n",
|
||
" os.makedirs(extract_path, exist_ok=True)\n",
|
||
" print(\"Extracting dataset...\")\n",
|
||
" try:\n",
|
||
" with zipfile.ZipFile(destination_path, 'r') as zip_ref:\n",
|
||
" zip_ref.extractall(path=extract_path)\n",
|
||
" print(\"Extraction completed!\")\n",
|
||
" except Exception as e:\n",
|
||
" print(f\"Extraction failed: {e}\")\n",
|
||
" else:\n",
|
||
" print(\"Dataset ready.\")\n",
|
||
"else:\n",
|
||
" print(f\"dataset.zip not found at {destination_path}. Upload dataset.zip to Google Drive / MyDrive/jagung/\")\n",
|
||
" print(\"Or run preprocessing.py locally and upload the resulting dataset.zip\")"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "199bdf60",
|
||
"metadata": {},
|
||
"source": [
|
||
"## 3. Data Cleaning — Corrupt Detection + Augmented Dedup\n",
|
||
"\n",
|
||
"Membersihkan dataset dari:\n",
|
||
"- File corrupt / tidak bisa dibuka PIL\n",
|
||
"- File `augmented_*` (pre-augmented duplicates — menyebabkan data leakage)\n",
|
||
"- Gambar dengan dimensi atau aspect ratio ekstrim\n",
|
||
"- Gambar dengan variance terlalu rendah (hampir seragam)\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "75cacdf5",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"# --- Determine dataset path ---\n",
|
||
"dataset_path = extract_path\n",
|
||
"\n",
|
||
"print(f\"Dataset path: {dataset_path}\")\n",
|
||
"\n",
|
||
"# Check if dataset already validated and split exists — skip if so\n",
|
||
"split_output_dir = \"/content/dataset_split\" if IS_COLAB else os.path.join(os.getcwd(), \"dataset_split\")\n",
|
||
"if os.path.exists(os.path.join(split_output_dir, 'train')):\n",
|
||
" print(\"Split dataset already exists. Skipping validation and split.\")\n",
|
||
" # Still need class_names and counts for downstream cells\n",
|
||
" train_dir = os.path.join(split_output_dir, 'train')\n",
|
||
" val_dir = os.path.join(split_output_dir, 'val')\n",
|
||
" test_dir = os.path.join(split_output_dir, 'test')\n",
|
||
" print(f'Train: {sum(len(files) for _, _, files in os.walk(train_dir))} | '\n",
|
||
" f'Val: {sum(len(files) for _, _, files in os.walk(val_dir))} | '\n",
|
||
" f'Test: {sum(len(files) for _, _, files in os.walk(test_dir))}')\n",
|
||
"else:\n",
|
||
" MIN_FILE_SIZE = 512\n",
|
||
" MIN_DIM = 32\n",
|
||
" MAX_ASPECT = 5.0\n",
|
||
"\n",
|
||
" def remove_augmented_duplicates(directory):\n",
|
||
" removed = 0\n",
|
||
" for root, dirs, files in os.walk(directory):\n",
|
||
" for file in files:\n",
|
||
" if file.startswith(\"augmented_\"):\n",
|
||
" try:\n",
|
||
" os.remove(os.path.join(root, file))\n",
|
||
" removed += 1\n",
|
||
" except OSError:\n",
|
||
" pass\n",
|
||
" return removed\n",
|
||
"\n",
|
||
" def clean_and_validate_images(directory):\n",
|
||
" stats = {\"too_small\": 0, \"corrupt\": 0, \"small_dims\": 0, \"extreme_aspect\": 0, \"low_var\": 0, \"ok\": 0}\n",
|
||
" for root, dirs, files in os.walk(directory):\n",
|
||
" for file in files:\n",
|
||
" fp = os.path.join(root, file)\n",
|
||
" try:\n",
|
||
" if os.path.getsize(fp) < MIN_FILE_SIZE:\n",
|
||
" os.remove(fp); stats[\"too_small\"] += 1; continue\n",
|
||
" except OSError:\n",
|
||
" continue\n",
|
||
" try:\n",
|
||
" img = Image.open(fp); img.verify()\n",
|
||
" except Exception:\n",
|
||
" try: os.remove(fp); stats[\"corrupt\"] += 1\n",
|
||
" except OSError: pass\n",
|
||
" continue\n",
|
||
" try:\n",
|
||
" img = Image.open(fp)\n",
|
||
" w, h = img.size\n",
|
||
" if w < MIN_DIM or h < MIN_DIM:\n",
|
||
" os.remove(fp); stats[\"small_dims\"] += 1; continue\n",
|
||
" aspect = w / max(h, 1)\n",
|
||
" if aspect > MAX_ASPECT or aspect < 1.0 / MAX_ASPECT:\n",
|
||
" os.remove(fp); stats[\"extreme_aspect\"] += 1; continue\n",
|
||
" if img.mode not in ('RGB', 'RGBA'):\n",
|
||
" img = img.convert('RGB'); img.save(fp)\n",
|
||
" arr = np.array(img).astype(np.float32)\n",
|
||
" if np.std(arr) < 2.0:\n",
|
||
" os.remove(fp); stats[\"low_var\"] += 1; continue\n",
|
||
" stats[\"ok\"] += 1\n",
|
||
" except Exception:\n",
|
||
" try: os.remove(fp); stats[\"corrupt\"] += 1\n",
|
||
" except OSError: pass\n",
|
||
" return stats\n",
|
||
"\n",
|
||
" print(\"1. Removing augmented duplicates...\")\n",
|
||
" n_aug = remove_augmented_duplicates(dataset_path)\n",
|
||
" print(f\" Removed {n_aug} augmented_* files\")\n",
|
||
"\n",
|
||
" print(\"2. Validating images...\")\n",
|
||
" stats = clean_and_validate_images(dataset_path)\n",
|
||
" print(f\" OK: {stats['ok']} | Removed: too_small={stats['too_small']} corrupt={stats['corrupt']} \"\n",
|
||
" f\"small_dims={stats['small_dims']} aspect={stats['extreme_aspect']} low_var={stats['low_var']}\")\n",
|
||
"\n",
|
||
" if HAS_IMAGEHASH:\n",
|
||
" print(\"3. Perceptual hash dedup...\")\n",
|
||
" seen, removed = {}, 0\n",
|
||
" for cn in sorted(os.listdir(dataset_path)):\n",
|
||
" cp = os.path.join(dataset_path, cn)\n",
|
||
" if not os.path.isdir(cp): continue\n",
|
||
" for f in sorted(os.listdir(cp)):\n",
|
||
" fp = os.path.join(cp, f)\n",
|
||
" if not os.path.isfile(fp): continue\n",
|
||
" try:\n",
|
||
" ah = imagehash.average_hash(Image.open(fp).convert('RGB'))\n",
|
||
" for sk, (sp, sc) in seen.items():\n",
|
||
" if ah - imagehash.hex_to_hash(sk) <= 5:\n",
|
||
" try: os.remove(fp); removed += 1\n",
|
||
" except OSError: pass\n",
|
||
" break\n",
|
||
" else:\n",
|
||
" seen[str(ah)] = (fp, cn)\n",
|
||
" except Exception:\n",
|
||
" pass\n",
|
||
" print(f\" Removed {removed} near-duplicates\")\n",
|
||
" else:\n",
|
||
" print(\"3. Perceptual hash dedup SKIPPED (pip install imagehash)\")\n",
|
||
"\n",
|
||
" total = sum(len(files) for _, _, files in os.walk(dataset_path))\n",
|
||
" print(f\"\\nTotal clean images: {total}\")"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "fbe9498c",
|
||
"metadata": {},
|
||
"source": [
|
||
"## 4. Stratified Split by Source (70:15:15)\n",
|
||
"\n",
|
||
"**Tidak menggunakan splitfolders!** Split manual dengan stratifikasi berdasarkan prefix sumber gambar.\n",
|
||
"Ini mencegah gambar dari sesi foto yang sama (lighting & background identik) masuk ke train DAN test.\n",
|
||
"\n",
|
||
"Source prefixes:\n",
|
||
"- `IMG_*` → foto HP\n",
|
||
"- `Corn_*` → dataset lab publik\n",
|
||
"- `CBS*`, `GLS*`, `NLS*`, `CLS*` → berbagai dataset lab\n",
|
||
"- `SCR*`, `CR*`, `NLB*`, `SLB*` → dataset spesifik penyakit\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "56234c24",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"def extract_source_prefix(filename):\n",
|
||
" f = os.path.splitext(filename)[0]\n",
|
||
" if f.startswith('IMG_'): return 'phone'\n",
|
||
" if f.startswith('Corn_'): return 'lab_corn'\n",
|
||
" for prefix in ['CBS', 'GLS', 'NLS', 'CLS']:\n",
|
||
" if f.startswith(prefix): return 'lab_disease'\n",
|
||
" for prefix in ['SCR', 'CR', 'NLB', 'SLB', 'SRS']:\n",
|
||
" if f.startswith(prefix): return 'lab_rust_blight'\n",
|
||
" return 'other'\n",
|
||
"\n",
|
||
"def stratified_split_by_source(dataset_path, output_dir, ratios=(0.7, 0.15, 0.15), seed=42):\n",
|
||
" class_images = {}\n",
|
||
" for cn in sorted(os.listdir(dataset_path)):\n",
|
||
" cp = os.path.join(dataset_path, cn)\n",
|
||
" if not os.path.isdir(cp): continue\n",
|
||
" class_images[cn] = []\n",
|
||
" for f in os.listdir(cp):\n",
|
||
" fp = os.path.join(cp, f)\n",
|
||
" if os.path.isfile(fp):\n",
|
||
" class_images[cn].append((fp, f, extract_source_prefix(f)))\n",
|
||
"\n",
|
||
" for split in ['train', 'val', 'test']:\n",
|
||
" for cn in class_images:\n",
|
||
" os.makedirs(os.path.join(output_dir, split, cn), exist_ok=True)\n",
|
||
"\n",
|
||
" rng = np.random.RandomState(seed)\n",
|
||
"\n",
|
||
" for cn, images in class_images.items():\n",
|
||
" by_source = {}\n",
|
||
" for fp, fn, src in images:\n",
|
||
" by_source.setdefault(src, []).append((fp, fn))\n",
|
||
"\n",
|
||
" train_files, val_files, test_files = [], [], []\n",
|
||
" for src, src_images in by_source.items():\n",
|
||
" n = len(src_images)\n",
|
||
" rng.shuffle(src_images)\n",
|
||
" n_train = max(1, int(n * ratios[0]))\n",
|
||
" n_val = max(1, int(n * ratios[1]))\n",
|
||
" train_files.extend(src_images[:n_train])\n",
|
||
" val_files.extend(src_images[n_train:n_train + n_val])\n",
|
||
" test_files.extend(src_images[n_train + n_val:])\n",
|
||
"\n",
|
||
" for fp, fn in train_files:\n",
|
||
" shutil.copy(fp, os.path.join(output_dir, 'train', cn, fn))\n",
|
||
" for fp, fn in val_files:\n",
|
||
" shutil.copy(fp, os.path.join(output_dir, 'val', cn, fn))\n",
|
||
" for fp, fn in test_files:\n",
|
||
" shutil.copy(fp, os.path.join(output_dir, 'test', cn, fn))\n",
|
||
"\n",
|
||
" train_srcs = Counter(extract_source_prefix(fn) for _, fn in train_files)\n",
|
||
" print(f\" {cn}: train={len(train_files)} val={len(val_files)} test={len(test_files)} | \"\n",
|
||
" f\"sources={dict(train_srcs)}\")\n",
|
||
"\n",
|
||
"output_dir = split_output_dir # Already defined in previous cell\n",
|
||
"\n",
|
||
"if os.path.exists(os.path.join(output_dir, 'train')):\n",
|
||
" print(\"Split already exists. Skipping.\")\n",
|
||
"else:\n",
|
||
" if os.path.exists(output_dir):\n",
|
||
" shutil.rmtree(output_dir)\n",
|
||
"\n",
|
||
" print(\"Splitting dataset 70:15:15 (stratified by source)...\")\n",
|
||
" stratified_split_by_source(dataset_path, output_dir, seed=SEED)\n",
|
||
" print(\"Done.\")\n",
|
||
"\n",
|
||
"train_dir = os.path.join(output_dir, 'train')\n",
|
||
"val_dir = os.path.join(output_dir, 'val')\n",
|
||
"test_dir = os.path.join(output_dir, 'test')\n",
|
||
"\n",
|
||
"def count_images(path):\n",
|
||
" return sum(len(files) for _, _, files in os.walk(path))\n",
|
||
"\n",
|
||
"print(f'Train: {count_images(train_dir)} | Val: {count_images(val_dir)} | Test: {count_images(test_dir)}')"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "3eead964",
|
||
"metadata": {},
|
||
"source": [
|
||
"## 5. RandAugment Pipeline dengan Weather Simulation\n",
|
||
"\n",
|
||
"Pipeline augmentasi baru untuk **robustness dunia nyata**:\n",
|
||
"\n",
|
||
"### Pool 15 Transformasi (RandAugment: pilih N=3 per gambar)\n",
|
||
"| Kategori | Transformasi | Simulasi |\n",
|
||
"|---|---|---|\n",
|
||
"| Geometric | Flip, Rotate, Zoom, Translate, Shear | Variasi angle/jarak foto |\n",
|
||
"| Color/Light | Hue, Saturation, Brightness, Contrast, Solarize | Variasi kamera & waktu hari |\n",
|
||
"| Weather | Fog, Shadow | Kondisi lapangan berkabut/berbayang |\n",
|
||
"| Degradation | GaussianBlur, ResolutionDrop | Blur gerakan, kamera rendah |\n",
|
||
"| Mixing | MixUp, CutMix, RandomErasing | Regularisasi label & occlusions |\n",
|
||
"\n",
|
||
"Fog dan Shadow adalah **custom tf operations** — tidak ada di Keras layers standar.\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "8535054d",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"# ─── RandAugment Utilities ───\n",
|
||
"\n",
|
||
"def sample_beta_distribution(size, a=0.2, b=0.2):\n",
|
||
" g1 = tf.random.gamma([size], a, dtype=tf.float32)\n",
|
||
" g2 = tf.random.gamma([size], b, dtype=tf.float32)\n",
|
||
" return g2 / (g1 + g2 + 1e-8)\n",
|
||
"\n",
|
||
"def _apply_contrast(img, factor):\n",
|
||
" mean = tf.reduce_mean(tf.cast(img, tf.float32), axis=(0, 1), keepdims=True)\n",
|
||
" return mean + factor * (tf.cast(img, tf.float32) - mean)\n",
|
||
"\n",
|
||
"def _apply_brightness(img, delta):\n",
|
||
" return tf.clip_by_value(tf.cast(img, tf.float32) + delta, 0.0, 255.0)\n",
|
||
"\n",
|
||
"def _apply_hue(img, delta):\n",
|
||
" hsv = tf.image.rgb_to_hsv(tf.cast(img, tf.float32) / 255.0)\n",
|
||
" h = hsv[..., 0] + delta\n",
|
||
" h = h - tf.floor(h)\n",
|
||
" hsv_h = tf.stack([h, hsv[..., 1], hsv[..., 2]], axis=-1)\n",
|
||
" return tf.image.hsv_to_rgb(hsv_h) * 255.0\n",
|
||
"\n",
|
||
"@tf.function(reduce_retracing=True)\n",
|
||
"def _randaug_select(images, ops_per_image=3, magnitude=0.7):\n",
|
||
" # Graph-mode-safe RandAugment. No tf.image.random_* or Keras layers\n",
|
||
" # (both trace as Python bool checks on tensor bounds internally).\n",
|
||
"\n",
|
||
" def _zoom(img):\n",
|
||
" h = tf.cast(tf.shape(img)[0], tf.float32)\n",
|
||
" w = tf.cast(tf.shape(img)[1], tf.float32)\n",
|
||
" z = tf.random.uniform([], 1.0 - 0.2 * magnitude, 1.0 + 0.2 * magnitude)\n",
|
||
" zh = tf.cast(h / z, tf.int32); zw = tf.cast(w / z, tf.int32)\n",
|
||
" zoomed = tf.image.resize(tf.expand_dims(img, 0), [zh, zw], method='bilinear')[0]\n",
|
||
" return tf.image.resize_with_crop_or_pad(zoomed, tf.cast(h, tf.int32), tf.cast(w, tf.int32))\n",
|
||
" def _translate(img):\n",
|
||
" h, w = tf.shape(img)[0], tf.shape(img)[1]\n",
|
||
" tx = tf.cast(tf.random.uniform([], -0.15 * magnitude, 0.15 * magnitude) * tf.cast(w, tf.float32), tf.int32)\n",
|
||
" ty = tf.cast(tf.random.uniform([], -0.15 * magnitude, 0.15 * magnitude) * tf.cast(h, tf.float32), tf.int32)\n",
|
||
" return tf.roll(img, [ty, tx], axis=[0, 1])\n",
|
||
"\n",
|
||
" def op_flip(img): return tf.image.random_flip_left_right(img)\n",
|
||
" def op_flip_v(img): return tf.image.random_flip_up_down(img)\n",
|
||
" def op_rotate_90(img):\n",
|
||
" k = tf.random.uniform([], 0, 4, dtype=tf.int32)\n",
|
||
" return tf.image.rot90(img, k)\n",
|
||
" def op_zoom(img): return _zoom(img)\n",
|
||
" def op_translate(img): return _translate(img)\n",
|
||
" def op_contrast(img):\n",
|
||
" f = tf.random.uniform([], 1.0 - 0.5 * magnitude, 1.0 + 0.5 * magnitude)\n",
|
||
" return _apply_contrast(img, f)\n",
|
||
" def op_brightness(img):\n",
|
||
" d = tf.random.uniform([], -0.3 * magnitude * 255.0, 0.3 * magnitude * 255.0)\n",
|
||
" return _apply_brightness(img, d)\n",
|
||
" def op_hue(img):\n",
|
||
" d = tf.random.uniform([], -0.08 * tf.maximum(magnitude, 0.01), 0.08 * tf.maximum(magnitude, 0.01))\n",
|
||
" return _apply_hue(img, d)\n",
|
||
" def op_saturation(img):\n",
|
||
" lo = tf.maximum(0.5, 1.0 - 0.8 * magnitude)\n",
|
||
" hi = 1.0 + 0.8 * magnitude\n",
|
||
" f = tf.random.uniform([], lo, hi)\n",
|
||
" hsv = tf.image.rgb_to_hsv(tf.cast(img, tf.float32) / 255.0)\n",
|
||
" s = tf.clip_by_value(hsv[..., 1] * f, 0.0, 1.0)\n",
|
||
" return tf.image.hsv_to_rgb(tf.stack([hsv[..., 0], s, hsv[..., 2]], axis=-1)) * 255.0\n",
|
||
" def op_solarize(img):\n",
|
||
" thresh = tf.random.uniform([], 0.3, 0.8) * 255.0\n",
|
||
" f = tf.cast(img, tf.float32)\n",
|
||
" return tf.where(f < thresh, f, 255.0 - f)\n",
|
||
" def op_blur(img):\n",
|
||
" h, w = tf.shape(img)[0], tf.shape(img)[1]\n",
|
||
" sf = tf.random.uniform([], 2, 4, dtype=tf.int32)\n",
|
||
" small = tf.image.resize(tf.expand_dims(tf.cast(img, tf.float32), 0), [h // sf, w // sf], method='bilinear')\n",
|
||
" return tf.image.resize(small, [h, w], method='bilinear')[0]\n",
|
||
" def op_fog(img):\n",
|
||
" fl = tf.random.uniform([], 0.1, 0.1 + 0.4 * magnitude)\n",
|
||
" fc = tf.random.uniform([3], 0.7, 1.0) * 255.0\n",
|
||
" return tf.cast(img, tf.float32) * (1.0 - fl) + tf.reshape(fc, [1, 1, 3]) * fl\n",
|
||
" def op_shadow(img):\n",
|
||
" op = tf.random.uniform([], 0.2, 0.2 + 0.5 * magnitude)\n",
|
||
" return tf.cast(img, tf.float32) * (1.0 - op * 0.6)\n",
|
||
" def op_resolution_drop(img):\n",
|
||
" h, w = tf.shape(img)[0], tf.shape(img)[1]\n",
|
||
" sf = tf.random.uniform([], 2, 5, dtype=tf.int32)\n",
|
||
" small = tf.image.resize(tf.expand_dims(tf.cast(img, tf.float32), 0), [h // sf, w // sf], method='bilinear')\n",
|
||
" return tf.image.resize(small, [h, w], method='nearest')[0]\n",
|
||
" def op_identity(img): return tf.cast(img, tf.float32)\n",
|
||
"\n",
|
||
" ops = [op_flip, op_flip_v, op_rotate_90, op_zoom, op_translate,\n",
|
||
" op_contrast, op_brightness, op_hue, op_saturation,\n",
|
||
" op_solarize, op_blur, op_fog, op_shadow, op_resolution_drop, op_identity]\n",
|
||
"\n",
|
||
" def apply_randaug_single(img3d):\n",
|
||
" indices = tf.random.shuffle(tf.range(15))[:3] # ops_per_image=3, static\n",
|
||
" result = tf.cast(img3d, tf.float32)\n",
|
||
" # Unrolled static 3 iterations — avoids TF shape invariance error from tf.range loop\n",
|
||
" def _apply_one(r, idx):\n",
|
||
" return tf.switch_case(idx, {j: lambda j=j: ops[j](r) for j in range(15)})\n",
|
||
" i0, i1, i2 = indices[0], indices[1], indices[2]\n",
|
||
" result = _apply_one(result, i0)\n",
|
||
" result = _apply_one(result, i1)\n",
|
||
" result = _apply_one(result, i2)\n",
|
||
" return tf.clip_by_value(result, 0.0, 255.0)\n",
|
||
"\n",
|
||
" return tf.map_fn(apply_randaug_single, images, dtype=tf.float32, parallel_iterations=8)\n",
|
||
"\n",
|
||
"# Compatibility wrapper using Keras Sequential for basic geometric ops (kept for visualization)\n",
|
||
"geo_aug = tf.keras.Sequential([\n",
|
||
" layers.RandomFlip(\"horizontal_and_vertical\"),\n",
|
||
" layers.RandomRotation(0.15),\n",
|
||
" layers.RandomZoom(0.15),\n",
|
||
" layers.RandomTranslation(0.1, 0.1),\n",
|
||
" layers.RandomContrast(0.15),\n",
|
||
" layers.RandomBrightness(0.15),\n",
|
||
"], name=\"geo_aug\")\n",
|
||
"\n",
|
||
"# ─── MixUp & CutMix (unchanged from original) ───\n",
|
||
"\n",
|
||
"def mix_up(images, labels, alpha=0.2):\n",
|
||
" bs = tf.shape(images)[0]\n",
|
||
" lam = sample_beta_distribution(bs, alpha, alpha)\n",
|
||
" lam_img = tf.reshape(lam, [bs, 1, 1, 1])\n",
|
||
" ri = tf.random.shuffle(tf.range(bs))\n",
|
||
" mixed_img = lam_img * images + (1 - lam_img) * tf.gather(images, ri)\n",
|
||
" labels = tf.cast(labels, tf.float32)\n",
|
||
" lam_lbl = tf.reshape(lam, [-1, 1])\n",
|
||
" mixed_lbl = lam_lbl * labels + (1 - lam_lbl) * tf.gather(labels, ri)\n",
|
||
" return mixed_img, mixed_lbl\n",
|
||
"\n",
|
||
"def cut_mix(images, labels, alpha=0.2):\n",
|
||
" bs = tf.shape(images)[0]; h = tf.shape(images)[1]; w = tf.shape(images)[2]\n",
|
||
" lam = sample_beta_distribution(bs, alpha, alpha); ri = tf.random.shuffle(tf.range(bs))\n",
|
||
" cr = tf.sqrt(1.0 - lam)\n",
|
||
" rh = tf.cast(cr * tf.cast(h, tf.float32), tf.int32); rw = tf.cast(cr * tf.cast(w, tf.float32), tf.int32)\n",
|
||
" cx = tf.random.uniform([bs], 0, w, tf.int32); cy = tf.random.uniform([bs], 0, h, tf.int32)\n",
|
||
" hh = rh // 2; hw = rw // 2\n",
|
||
" x1 = tf.clip_by_value(cx - hw, 0, w); x2 = tf.clip_by_value(cx + hw, 0, w)\n",
|
||
" y1 = tf.clip_by_value(cy - hh, 0, h); y2 = tf.clip_by_value(cy + hh, 0, h)\n",
|
||
" col = tf.range(w, dtype=tf.int32); row = tf.range(h, dtype=tf.int32)\n",
|
||
" in_x = tf.logical_and(tf.reshape(col, [1, 1, w]) >= tf.reshape(x1, [bs, 1, 1]),\n",
|
||
" tf.reshape(col, [1, 1, w]) < tf.reshape(x2, [bs, 1, 1]))\n",
|
||
" in_y = tf.logical_and(tf.reshape(row, [1, h, 1]) >= tf.reshape(y1, [bs, 1, 1]),\n",
|
||
" tf.reshape(row, [1, h, 1]) < tf.reshape(y2, [bs, 1, 1]))\n",
|
||
" cm = tf.cast(tf.logical_and(in_y, in_x), tf.float32); cm = tf.expand_dims(cm, -1)\n",
|
||
" shuf = tf.gather(images, ri)\n",
|
||
" mi = (1.0 - cm) * images + cm * shuf\n",
|
||
" labels = tf.cast(labels, tf.float32); lr = tf.reshape(lam, [-1, 1])\n",
|
||
" ml = lr * labels + (1.0 - lr) * tf.gather(labels, ri)\n",
|
||
" return mi, ml\n",
|
||
"\n",
|
||
"def random_erasing(images, probability=0.25, scale=(0.02, 0.25)):\n",
|
||
" bs = tf.shape(images)[0]; h = tf.shape(images)[1]; w = tf.shape(images)[2]\n",
|
||
" ta = tf.random.uniform([], scale[0], scale[1]) * tf.cast(h * w, tf.float32)\n",
|
||
" ar = tf.random.uniform([], 0.3, 3.3)\n",
|
||
" eh = tf.cast(tf.math.sqrt(ta / ar), tf.int32); ew = tf.cast(tf.math.sqrt(ta * ar), tf.int32)\n",
|
||
" eh = tf.clip_by_value(eh, 1, h - 1); ew = tf.clip_by_value(ew, 1, w - 1)\n",
|
||
" cx = tf.random.uniform([], 0, w - ew, tf.int32); cy = tf.random.uniform([], 0, h - eh, tf.int32)\n",
|
||
" col = tf.range(w, dtype=tf.int32); row = tf.range(h, dtype=tf.int32)\n",
|
||
" ix = tf.logical_and(col >= cx, col < cx + ew)\n",
|
||
" iy = tf.logical_and(row >= cy, row < cy + eh)\n",
|
||
" em = tf.cast(tf.expand_dims(iy, 1) & tf.expand_dims(ix, 0), tf.float32)\n",
|
||
" em = tf.expand_dims(tf.expand_dims(em, 0), -1)\n",
|
||
" noise = tf.random.uniform([bs, eh, ew, 3], 0.0, 255.0, dtype=tf.float32)\n",
|
||
" pads = [[0, 0], [cy, h - (cy + eh)], [cx, w - (cx + ew)], [0, 0]]\n",
|
||
" npad = tf.pad(noise, pads, constant_values=0.0)\n",
|
||
" erased = images * (1.0 - em) + npad * em\n",
|
||
" return tf.cond(tf.random.uniform([]) < probability, lambda: erased, lambda: images)\n",
|
||
"\n",
|
||
"# ─── Main augmentation pipeline ───\n",
|
||
"\n",
|
||
"def augment_and_mix(images, labels):\n",
|
||
" # 1. RandAugment (geometric + color + weather + degradation)\n",
|
||
" images = tf.cast(images, tf.float32)\n",
|
||
" images = _randaug_select(images, ops_per_image=3, magnitude=0.7)\n",
|
||
" # 2. MixUp or CutMix (40% chance total: 20% MixUp, 20% CutMix)\n",
|
||
" choice = tf.random.uniform([])\n",
|
||
" labels_oh = tf.one_hot(labels, NUM_CLASSES)\n",
|
||
" images, labels_oh = tf.cond(\n",
|
||
" choice < 0.2, lambda: mix_up(images, labels_oh),\n",
|
||
" lambda: tf.cond(choice < 0.4, lambda: cut_mix(images, labels_oh),\n",
|
||
" lambda: (images, labels_oh)))\n",
|
||
" # 3. Random Erasing\n",
|
||
" images = random_erasing(images, probability=0.2)\n",
|
||
" return images, labels_oh\n",
|
||
"\n",
|
||
"# ─── Preprocessing ───\n",
|
||
"def preprocess_fn(image, label):\n",
|
||
" return preprocess_input(image), label\n",
|
||
"\n",
|
||
"# ─── Class weights ───\n",
|
||
"train_class_counts = Counter()\n",
|
||
"for cn in sorted(os.listdir(train_dir)):\n",
|
||
" p = os.path.join(train_dir, cn)\n",
|
||
" if os.path.isdir(p):\n",
|
||
" train_class_counts[cn] = len(os.listdir(p))\n",
|
||
"\n",
|
||
"y_int = []\n",
|
||
"for i, cn in enumerate(sorted(os.listdir(train_dir))):\n",
|
||
" cp = os.path.join(train_dir, cn)\n",
|
||
" if os.path.isdir(cp):\n",
|
||
" y_int.extend([i] * len(os.listdir(cp)))\n",
|
||
"\n",
|
||
"cw_array = compute_class_weight('balanced', classes=np.unique(y_int), y=y_int)\n",
|
||
"cw_capped = [min(w, 3.0) for w in cw_array]\n",
|
||
"class_weights_tensor = tf.constant(cw_capped, dtype=tf.float32)\n",
|
||
"\n",
|
||
"def add_sample_weight(image, label):\n",
|
||
" ci = tf.argmax(label, axis=-1)\n",
|
||
" sw = tf.gather(class_weights_tensor, ci)\n",
|
||
" return image, label, sw\n",
|
||
"\n",
|
||
"# ─── Build datasets ───\n",
|
||
"train_ds = tf.keras.utils.image_dataset_from_directory(\n",
|
||
" train_dir, shuffle=True, batch_size=BATCH_SIZE, image_size=IMG_SIZE)\n",
|
||
"val_ds = tf.keras.utils.image_dataset_from_directory(\n",
|
||
" val_dir, shuffle=True, batch_size=BATCH_SIZE, image_size=IMG_SIZE)\n",
|
||
"test_ds = tf.keras.utils.image_dataset_from_directory(\n",
|
||
" test_dir, shuffle=False, batch_size=BATCH_SIZE, image_size=IMG_SIZE)\n",
|
||
"\n",
|
||
"class_names = train_ds.class_names\n",
|
||
"NUM_CLASSES = len(class_names)\n",
|
||
"print(f\"Classes ({NUM_CLASSES}): {class_names}\")\n",
|
||
"\n",
|
||
"train_ds = (train_ds\n",
|
||
" .map(augment_and_mix, num_parallel_calls=AUTOTUNE)\n",
|
||
" .map(preprocess_fn, num_parallel_calls=AUTOTUNE)\n",
|
||
" .map(add_sample_weight, num_parallel_calls=AUTOTUNE)\n",
|
||
" .prefetch(AUTOTUNE))\n",
|
||
"\n",
|
||
"val_ds = (val_ds\n",
|
||
" .map(lambda img, lbl: (preprocess_input(tf.cast(img, tf.float32)), lbl), num_parallel_calls=AUTOTUNE)\n",
|
||
" .map(lambda img, lbl: (img, tf.one_hot(lbl, NUM_CLASSES)), num_parallel_calls=AUTOTUNE)\n",
|
||
" .prefetch(AUTOTUNE))\n",
|
||
"\n",
|
||
"test_ds = (test_ds\n",
|
||
" .map(lambda img, lbl: (preprocess_input(tf.cast(img, tf.float32)), lbl), num_parallel_calls=AUTOTUNE)\n",
|
||
" .map(lambda img, lbl: (img, tf.one_hot(lbl, NUM_CLASSES)), num_parallel_calls=AUTOTUNE)\n",
|
||
" .prefetch(AUTOTUNE))\n",
|
||
"\n",
|
||
"print(\"Class weights (capped at 3.0):\")\n",
|
||
"for i, cn in enumerate(class_names):\n",
|
||
" if i < len(cw_capped):\n",
|
||
" print(f\" {cn}: {cw_capped[i]:.4f}\")\n",
|
||
"print(\"Data pipelines ready.\")\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "4ca2ec40",
|
||
"metadata": {},
|
||
"source": [
|
||
"## 6. Visualisasi Sampel Data (Augmentasi Real-World)\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "44d6d2b0",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"vis_ds = tf.keras.utils.image_dataset_from_directory(\n",
|
||
" train_dir, shuffle=True, batch_size=BATCH_SIZE, image_size=IMG_SIZE)\n",
|
||
"\n",
|
||
"plt.figure(figsize=(16, 10))\n",
|
||
"for images, labels in vis_ds.take(1):\n",
|
||
" # Original (4)\n",
|
||
" for i in range(4):\n",
|
||
" plt.subplot(3, 4, i + 1)\n",
|
||
" plt.imshow(images[i].numpy().astype(\"uint8\"))\n",
|
||
" plt.title(f\"Asli: {class_names[labels[i].numpy()]}\", fontsize=11)\n",
|
||
" plt.axis(\"off\")\n",
|
||
" # RandAugment (4)\n",
|
||
" aug = _randaug_select(tf.cast(images, tf.float32), ops_per_image=3, magnitude=0.7)\n",
|
||
" for i in range(4):\n",
|
||
" plt.subplot(3, 4, i + 5)\n",
|
||
" plt.imshow(tf.clip_by_value(aug[i], 0, 255).numpy().astype(\"uint8\"))\n",
|
||
" plt.title(f\"RandAug: {class_names[labels[i].numpy()]}\", fontsize=11)\n",
|
||
" plt.axis(\"off\")\n",
|
||
" # MixUp result (4)\n",
|
||
" aug_f = tf.cast(images, tf.float32)\n",
|
||
" mixed, _ = mix_up(aug_f, tf.one_hot(labels, NUM_CLASSES))\n",
|
||
" for i in range(4):\n",
|
||
" plt.subplot(3, 4, i + 9)\n",
|
||
" plt.imshow(tf.clip_by_value(mixed[i], 0, 255).numpy().astype(\"uint8\"))\n",
|
||
" plt.title(\"MixUp/Weather\", fontsize=11)\n",
|
||
" plt.axis(\"off\")\n",
|
||
"\n",
|
||
"plt.suptitle(\"RandAugment + MixUp — Real-World Simulation\", fontsize=16)\n",
|
||
"plt.tight_layout()\n",
|
||
"plt.show()\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "05281076",
|
||
"metadata": {},
|
||
"source": [
|
||
"## 7. Arsitektur Model — CBAM + Lightweight Head\n",
|
||
"\n",
|
||
"**EfficientNetV2B0** base (frozen awal) + **CBAM spatial attention** + lightweight classifier head.\n",
|
||
"\n",
|
||
"### Head (≈700K params vs 7.3M sebelumnya)\n",
|
||
"```\n",
|
||
"Base (7×7×1280) → CBAM_Attention → GAP → Dropout(0.3) → Dense(512, swish) → BN → Dropout(0.4) → Dense(4, softmax)\n",
|
||
"```\n",
|
||
"\n",
|
||
"CBAM (Convolutional Block Attention Module): channel attention + spatial attention\n",
|
||
"→ model belajar fokus ke foreground (daun) bukan background.\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "4937fb17",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"def cbam_block(x, ratio=8, name=\"cbam\"):\n",
|
||
" # Convolutional Block Attention Module — ringan, fokus ke foreground.\n",
|
||
" channels = x.shape[-1]\n",
|
||
"\n",
|
||
" # Channel Attention\n",
|
||
" avg_pool = layers.GlobalAveragePooling2D()(x)\n",
|
||
" max_pool = layers.GlobalMaxPooling2D()(x)\n",
|
||
" ca = layers.Dense(channels // ratio, activation='swish', name=f\"{name}_ca1\")(avg_pool)\n",
|
||
" ca = layers.Dense(channels, activation='sigmoid', name=f\"{name}_ca2\")(ca)\n",
|
||
" ca2 = layers.Dense(channels // ratio, activation='swish', name=f\"{name}_ca3\")(max_pool)\n",
|
||
" ca2 = layers.Dense(channels, activation='sigmoid', name=f\"{name}_ca4\")(ca2)\n",
|
||
" ca_out = layers.Add(name=f\"{name}_ca_add\")([ca, ca2])\n",
|
||
" ca_out = layers.Reshape((1, 1, channels), name=f\"{name}_ca_reshape\")(ca_out)\n",
|
||
" x = layers.Multiply(name=f\"{name}_ca_mul\")([x, ca_out])\n",
|
||
"\n",
|
||
" # Spatial Attention\n",
|
||
" avg_sp = layers.Lambda(lambda t: tf.reduce_mean(t, axis=-1, keepdims=True), name=f\"{name}_sa_avg\")(x)\n",
|
||
" max_sp = layers.Lambda(lambda t: tf.reduce_max(t, axis=-1, keepdims=True), name=f\"{name}_sa_max\")(x)\n",
|
||
" sp = layers.Concatenate(name=f\"{name}_sa_cat\")([avg_sp, max_sp])\n",
|
||
" sp = layers.Conv2D(1, 7, padding='same', activation='sigmoid', name=f\"{name}_sa_conv\")(sp)\n",
|
||
" x = layers.Multiply(name=f\"{name}_sa_mul\")([x, sp])\n",
|
||
" return x\n",
|
||
"\n",
|
||
"def build_model(num_classes, target_size=(224, 224)):\n",
|
||
" # Accept any input size via variable input; Resizing handles progressive resolution.\n",
|
||
" inputs = tf.keras.Input(shape=(None, None, 3), name=\"input\")\n",
|
||
" x = layers.Resizing(target_size[0], target_size[1], interpolation='bilinear',\n",
|
||
" name=\"resize_input\")(inputs)\n",
|
||
"\n",
|
||
" base_model = EfficientNetV2B0(\n",
|
||
" input_shape=target_size + (3,),\n",
|
||
" include_top=False,\n",
|
||
" weights='imagenet',\n",
|
||
" )\n",
|
||
" base_model.trainable = False\n",
|
||
"\n",
|
||
" # Gaussian noise untuk regularisasi\n",
|
||
" x = layers.GaussianNoise(0.05, name=\"gauss_noise\")(x)\n",
|
||
" x = base_model(x, training=False)\n",
|
||
" # CBAM attention — fokus ke region daun\n",
|
||
" x = cbam_block(x, ratio=8, name=\"cbam\")\n",
|
||
" x = layers.GlobalAveragePooling2D(name=\"gap\")(x)\n",
|
||
" x = layers.Dropout(0.3, name=\"drop_gap\")(x)\n",
|
||
" x = layers.Dense(512, activation='swish', name=\"dense_head\")(x)\n",
|
||
" x = layers.BatchNormalization(name=\"bn_head\")(x)\n",
|
||
" x = layers.Dropout(0.4, name=\"drop_head\")(x)\n",
|
||
" outputs = layers.Dense(num_classes, activation='linear', dtype='float32', name=\"logits\")(x)\n",
|
||
" return models.Model(inputs, outputs), base_model\n",
|
||
"\n",
|
||
"# Checkpoint\n",
|
||
"ckpt_dir = '/content/best_model' if IS_COLAB else os.path.join(os.getcwd(), 'best_model')\n",
|
||
"checkpoint_path = os.path.join(ckpt_dir, 'best_model.keras')\n",
|
||
"\n",
|
||
"# Hapus checkpoint lama (arsitektur berbeda — tidak kompatibel)\n",
|
||
"if os.path.exists(checkpoint_path):\n",
|
||
" print(f\"Removing old checkpoint (incompatible architecture)...\")\n",
|
||
" os.remove(checkpoint_path)\n",
|
||
"\n",
|
||
"if os.path.exists(checkpoint_path):\n",
|
||
" print(f\"Loading checkpoint: {checkpoint_path}\")\n",
|
||
" try:\n",
|
||
" model = models.load_model(checkpoint_path, compile=False)\n",
|
||
" except Exception as e:\n",
|
||
" print(f\"Load failed: {e}. Building fresh.\")\n",
|
||
" model, base_model = build_model(NUM_CLASSES)\n",
|
||
"else:\n",
|
||
" print(\"No checkpoint. Building fresh model.\")\n",
|
||
" model, base_model = build_model(NUM_CLASSES)\n",
|
||
"\n",
|
||
"os.makedirs(ckpt_dir, exist_ok=True)\n",
|
||
"model.summary()\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "28aaaed1",
|
||
"metadata": {},
|
||
"source": [
|
||
"## 8. Training Setup — Cosine Decay + SWA + Callbacks\n",
|
||
"\n",
|
||
"Mengganti `ReduceLROnPlateau` / `EarlyStopping` dengan:\n",
|
||
"- **CosineDecay** + linear warmup per fase\n",
|
||
"- **Stochastic Weight Averaging (SWA)** — averaging bobot untuk wider optima\n",
|
||
"- **ModelCheckpoint** — tetap simpan best val_accuracy\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "51deff9d",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"class WarmupCosineDecay(tf.keras.optimizers.schedules.LearningRateSchedule):\n",
|
||
" # Cosine decay with linear warmup.\n",
|
||
" def __init__(self, warmup_steps, total_steps, peak_lr, min_lr=1e-7):\n",
|
||
" super().__init__()\n",
|
||
" self.warmup_steps = warmup_steps\n",
|
||
" self.total_steps = total_steps\n",
|
||
" self.peak_lr = peak_lr\n",
|
||
" self.min_lr = min_lr\n",
|
||
"\n",
|
||
" def __call__(self, step):\n",
|
||
" step = tf.cast(step, tf.float32)\n",
|
||
" warmup_steps = tf.cast(self.warmup_steps, tf.float32)\n",
|
||
" total_steps = tf.cast(self.total_steps, tf.float32)\n",
|
||
" # Warmup phase\n",
|
||
" warmup_lr = self.peak_lr * (step / warmup_steps)\n",
|
||
" # Cosine decay phase\n",
|
||
" progress = (step - warmup_steps) / tf.maximum(total_steps - warmup_steps, 1.0)\n",
|
||
" cosine_lr = self.min_lr + 0.5 * (self.peak_lr - self.min_lr) * (1.0 + tf.cos(np.pi * progress))\n",
|
||
" return tf.where(step < warmup_steps, warmup_lr, cosine_lr)\n",
|
||
"\n",
|
||
" def get_config(self):\n",
|
||
" return {\n",
|
||
" \"warmup_steps\": self.warmup_steps, \"total_steps\": self.total_steps,\n",
|
||
" \"peak_lr\": self.peak_lr, \"min_lr\": self.min_lr,\n",
|
||
" }\n",
|
||
"\n",
|
||
"class SWACallback(tf.keras.callbacks.Callback):\n",
|
||
" # Stochastic Weight Averaging — averages weights over final epochs.\n",
|
||
" def __init__(self, start_epoch, swa_lr=1e-5):\n",
|
||
" super().__init__()\n",
|
||
" self.start_epoch = start_epoch\n",
|
||
" self.swa_lr = swa_lr\n",
|
||
" self.swa_weights = None\n",
|
||
" self.n_models = 0\n",
|
||
"\n",
|
||
" def on_epoch_begin(self, epoch, logs=None):\n",
|
||
" if epoch >= self.start_epoch and self.swa_weights is None:\n",
|
||
" self.swa_weights = [w.numpy() for w in self.model.weights]\n",
|
||
" print(f\"\\nSWA: starting weight averaging at epoch {epoch+1}\")\n",
|
||
"\n",
|
||
" def on_epoch_end(self, epoch, logs=None):\n",
|
||
" if epoch >= self.start_epoch and self.swa_weights is not None:\n",
|
||
" for i, w in enumerate(self.model.weights):\n",
|
||
" self.swa_weights[i] = (self.swa_weights[i] * self.n_models + w.numpy()) / (self.n_models + 1)\n",
|
||
" self.n_models += 1\n",
|
||
"\n",
|
||
" def apply_swa_weights(self):\n",
|
||
" if self.swa_weights is None:\n",
|
||
" print(\"SWA: no weights to average (skipped)\")\n",
|
||
" return\n",
|
||
" for w, swa_w in zip(self.model.weights, self.swa_weights):\n",
|
||
" w.assign(swa_w)\n",
|
||
" print(f\"SWA weights applied ({self.n_models} models averaged).\")\n",
|
||
"\n",
|
||
"# Shared callbacks\n",
|
||
"checkpoint_cb = callbacks.ModelCheckpoint(\n",
|
||
" checkpoint_path, save_best_only=True, monitor=\"val_accuracy\",\n",
|
||
" mode=\"max\", verbose=1)\n",
|
||
"\n",
|
||
"csv_logger = callbacks.CSVLogger(os.path.join(ckpt_dir, 'training_log.csv'))\n",
|
||
"\n",
|
||
"def make_callbacks(swa_start=None):\n",
|
||
" cbs = [checkpoint_cb, csv_logger]\n",
|
||
" if swa_start is not None:\n",
|
||
" cbs.append(SWACallback(swa_start))\n",
|
||
" return cbs\n",
|
||
"\n",
|
||
"print(\"Callbacks ready.\")\n",
|
||
"print(f\"Checkpoint path: {checkpoint_path}\")\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "82814b7b",
|
||
"metadata": {},
|
||
"source": [
|
||
"## 9. Fase 1 — Head Only Training (128×128)\n",
|
||
"\n",
|
||
"Progressive resolution: mulai dari **128×128** untuk feature learning cepat.\n",
|
||
"Base model beku, hanya head (CBAM + Dense) yang dilatih.\n",
|
||
"Optimizer: AdamW + EMA + CosineDecay(warmup=3, peak=1e-3).\n",
|
||
"Label smoothing: 0.15\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "decaf5cb",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"IMG_128 = (128, 128)\n",
|
||
"\n",
|
||
"# Rebuild datasets at 128x128\n",
|
||
"train_ds_128 = tf.keras.utils.image_dataset_from_directory(\n",
|
||
" train_dir, shuffle=True, batch_size=BATCH_SIZE, image_size=IMG_128)\n",
|
||
"val_ds_128 = tf.keras.utils.image_dataset_from_directory(\n",
|
||
" val_dir, shuffle=True, batch_size=BATCH_SIZE, image_size=IMG_128)\n",
|
||
"\n",
|
||
"train_ds_128 = (train_ds_128.map(augment_and_mix, num_parallel_calls=AUTOTUNE)\n",
|
||
" .map(preprocess_fn, num_parallel_calls=AUTOTUNE)\n",
|
||
" .map(add_sample_weight, num_parallel_calls=AUTOTUNE).prefetch(AUTOTUNE))\n",
|
||
"val_ds_128 = (val_ds_128.map(lambda img, lbl: (preprocess_input(tf.cast(img, tf.float32)), lbl), num_parallel_calls=AUTOTUNE)\n",
|
||
" .map(lambda img, lbl: (img, tf.one_hot(lbl, NUM_CLASSES)), num_parallel_calls=AUTOTUNE).prefetch(AUTOTUNE))\n",
|
||
"\n",
|
||
"EPOCHS_P1 = 25\n",
|
||
"steps_per_epoch = tf.data.experimental.cardinality(train_ds_128).numpy() or 100\n",
|
||
"total_steps = steps_per_epoch * EPOCHS_P1\n",
|
||
"warmup_steps = steps_per_epoch * 3 # 3 epoch warmup\n",
|
||
"\n",
|
||
"lr_schedule_p1 = WarmupCosineDecay(warmup_steps, total_steps, peak_lr=1e-3, min_lr=1e-5)\n",
|
||
"\n",
|
||
"model.compile(\n",
|
||
" optimizer=AdamW(\n",
|
||
" learning_rate=lr_schedule_p1, weight_decay=1e-4),\n",
|
||
" loss=tf.keras.losses.CategoricalCrossentropy(from_logits=True, label_smoothing=0.15),\n",
|
||
" metrics=['accuracy']\n",
|
||
")\n",
|
||
"\n",
|
||
"print(\"Phase 1: Head training at 128×128...\")\n",
|
||
"history_1 = model.fit(train_ds_128, validation_data=val_ds_128,\n",
|
||
" epochs=EPOCHS_P1, callbacks=make_callbacks())\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "de673df6",
|
||
"metadata": {},
|
||
"source": [
|
||
"## 10. Fase 2 — Partial Fine-tuning (192×192)\n",
|
||
"\n",
|
||
"Resolusi naik ke **192×192**. Top 100 layer EfficientNetV2B0 di-unfreeze.\n",
|
||
"Learning rate lebih rendah: peak=5e-4, cosine decay ke 1e-6.\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "fd7fb926",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"IMG_192 = (192, 192)\n",
|
||
"\n",
|
||
"train_ds_192 = tf.keras.utils.image_dataset_from_directory(\n",
|
||
" train_dir, shuffle=True, batch_size=BATCH_SIZE, image_size=IMG_192)\n",
|
||
"val_ds_192 = tf.keras.utils.image_dataset_from_directory(\n",
|
||
" val_dir, shuffle=True, batch_size=BATCH_SIZE, image_size=IMG_192)\n",
|
||
"\n",
|
||
"train_ds_192 = (train_ds_192.map(augment_and_mix, num_parallel_calls=AUTOTUNE)\n",
|
||
" .map(preprocess_fn, num_parallel_calls=AUTOTUNE)\n",
|
||
" .map(add_sample_weight, num_parallel_calls=AUTOTUNE).prefetch(AUTOTUNE))\n",
|
||
"val_ds_192 = (val_ds_192.map(lambda img, lbl: (preprocess_input(tf.cast(img, tf.float32)), lbl), num_parallel_calls=AUTOTUNE)\n",
|
||
" .map(lambda img, lbl: (img, tf.one_hot(lbl, NUM_CLASSES)), num_parallel_calls=AUTOTUNE).prefetch(AUTOTUNE))\n",
|
||
"\n",
|
||
"# Unfreeze top 100 layers\n",
|
||
"base_model.trainable = True\n",
|
||
"for layer in base_model.layers[:-100]:\n",
|
||
" layer.trainable = False\n",
|
||
"\n",
|
||
"EPOCHS_P2 = 30\n",
|
||
"steps_p2 = tf.data.experimental.cardinality(train_ds_192).numpy() or 100\n",
|
||
"total_p2 = steps_p2 * EPOCHS_P2\n",
|
||
"warmup_p2 = steps_p2 * 2\n",
|
||
"\n",
|
||
"lr_schedule_p2 = WarmupCosineDecay(warmup_p2, total_p2, peak_lr=5e-4, min_lr=1e-6)\n",
|
||
"\n",
|
||
"model.compile(\n",
|
||
" optimizer=AdamW(\n",
|
||
" learning_rate=lr_schedule_p2, weight_decay=1e-4),\n",
|
||
" loss=tf.keras.losses.CategoricalCrossentropy(from_logits=True, label_smoothing=0.15),\n",
|
||
" metrics=['accuracy']\n",
|
||
")\n",
|
||
"\n",
|
||
"print(\"Phase 2: Fine-tuning top 100 layers at 192×192...\")\n",
|
||
"history_2 = model.fit(train_ds_192, validation_data=val_ds_192,\n",
|
||
" epochs=EPOCHS_P1 + EPOCHS_P2, initial_epoch=history_1.epoch[-1] + 1,\n",
|
||
" callbacks=make_callbacks())\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "33326d9c",
|
||
"metadata": {},
|
||
"source": [
|
||
"## 11. Fase 3 — Full Fine-tuning (224×224)\n",
|
||
"\n",
|
||
"Resolusi penuh **224×224**. Semua layer di-unfreeze.\n",
|
||
"LR sangat rendah: peak=1e-4, cosine decay ke 1e-7.\n",
|
||
"Label smoothing diturunkan ke 0.10 untuk kalibrasi lebih baik.\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "14115063",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"img_size = IMG_SIZE # (224, 224)\n",
|
||
"\n",
|
||
"train_ds_full = tf.keras.utils.image_dataset_from_directory(\n",
|
||
" train_dir, shuffle=True, batch_size=BATCH_SIZE, image_size=img_size)\n",
|
||
"val_ds_full = tf.keras.utils.image_dataset_from_directory(\n",
|
||
" val_dir, shuffle=True, batch_size=BATCH_SIZE, image_size=img_size)\n",
|
||
"\n",
|
||
"train_ds_full = (train_ds_full.map(augment_and_mix, num_parallel_calls=AUTOTUNE)\n",
|
||
" .map(preprocess_fn, num_parallel_calls=AUTOTUNE)\n",
|
||
" .map(add_sample_weight, num_parallel_calls=AUTOTUNE).prefetch(AUTOTUNE))\n",
|
||
"val_ds_full = (val_ds_full.map(lambda img, lbl: (preprocess_input(tf.cast(img, tf.float32)), lbl), num_parallel_calls=AUTOTUNE)\n",
|
||
" .map(lambda img, lbl: (img, tf.one_hot(lbl, NUM_CLASSES)), num_parallel_calls=AUTOTUNE).prefetch(AUTOTUNE))\n",
|
||
"\n",
|
||
"# Full unfreeze\n",
|
||
"for layer in base_model.layers:\n",
|
||
" layer.trainable = True\n",
|
||
"\n",
|
||
"EPOCHS_P3 = 30\n",
|
||
"steps_p3 = tf.data.experimental.cardinality(train_ds_full).numpy() or 100\n",
|
||
"total_p3 = steps_p3 * EPOCHS_P3\n",
|
||
"warmup_p3 = steps_p3 * 2\n",
|
||
"\n",
|
||
"lr_schedule_p3 = WarmupCosineDecay(warmup_p3, total_p3, peak_lr=1e-4, min_lr=1e-7)\n",
|
||
"\n",
|
||
"model.compile(\n",
|
||
" optimizer=AdamW(\n",
|
||
" learning_rate=lr_schedule_p3, weight_decay=1e-4),\n",
|
||
" loss=tf.keras.losses.CategoricalCrossentropy(from_logits=True, label_smoothing=0.10),\n",
|
||
" metrics=['accuracy']\n",
|
||
")\n",
|
||
"\n",
|
||
"print(\"Phase 3: Full fine-tuning at 224×224...\")\n",
|
||
"history_3 = model.fit(train_ds_full, validation_data=val_ds_full,\n",
|
||
" epochs=EPOCHS_P1 + EPOCHS_P2 + EPOCHS_P3, initial_epoch=(history_2.epoch[-1] + 1) if history_2.epoch else EPOCHS_P1 + EPOCHS_P2,\n",
|
||
" callbacks=make_callbacks())\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "6df4ef22",
|
||
"metadata": {},
|
||
"source": [
|
||
"## 12. SWA — Stochastic Weight Averaging\n",
|
||
"\n",
|
||
"15 epoch tambahan dengan cyclic LR (1e-5). SWA mengakumulasi rata-rata bobot\n",
|
||
"untuk menghasilkan **wider optima** — generalisasi lebih baik ke data out-of-distribution.\n",
|
||
"\n",
|
||
"Setelah SWA selesai, bobot SWA diterapkan kembali ke model.\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "290f2345",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"EPOCHS_SWA = 15\n",
|
||
"swa_start_epoch = (history_3.epoch[-1] + 1) if history_3.epoch else EPOCHS_P1 + EPOCHS_P2 + EPOCHS_P3\n",
|
||
"\n",
|
||
"swa_cb = SWACallback(start_epoch=swa_start_epoch, swa_lr=1e-5)\n",
|
||
"\n",
|
||
"model.compile(\n",
|
||
" optimizer=AdamW(\n",
|
||
" learning_rate=1e-5, weight_decay=1e-4),\n",
|
||
" loss=tf.keras.losses.CategoricalCrossentropy(from_logits=True, label_smoothing=0.10),\n",
|
||
" metrics=['accuracy']\n",
|
||
")\n",
|
||
"\n",
|
||
"print(f\"SWA: {EPOCHS_SWA} epochs starting at epoch {swa_start_epoch + 1}...\")\n",
|
||
"history_swa = model.fit(train_ds_full, validation_data=val_ds_full,\n",
|
||
" epochs=swa_start_epoch + EPOCHS_SWA, initial_epoch=swa_start_epoch,\n",
|
||
" callbacks=make_callbacks() + [swa_cb])\n",
|
||
"\n",
|
||
"# Apply SWA weights\n",
|
||
"swa_cb.apply_swa_weights()\n",
|
||
"print(f\"SWA complete. Final model has SWA weights applied.\")\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "2d889f2a",
|
||
"metadata": {},
|
||
"source": [
|
||
"## 13. Plot Training History (Gabungan Semua Fase)\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "41e79742",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"# Gabungkan semua history\n",
|
||
"acc = (history_1.history['accuracy'] + history_2.history['accuracy'] +\n",
|
||
" history_3.history['accuracy'] + history_swa.history['accuracy'])\n",
|
||
"val_acc = (history_1.history['val_accuracy'] + history_2.history['val_accuracy'] +\n",
|
||
" history_3.history['val_accuracy'] + history_swa.history['val_accuracy'])\n",
|
||
"loss = (history_1.history['loss'] + history_2.history['loss'] +\n",
|
||
" history_3.history['loss'] + history_swa.history['loss'])\n",
|
||
"val_loss = (history_1.history['val_loss'] + history_2.history['val_loss'] +\n",
|
||
" history_3.history['val_loss'] + history_swa.history['val_loss'])\n",
|
||
"\n",
|
||
"b1 = len(history_1.history['accuracy']) - 1\n",
|
||
"b2 = b1 + len(history_2.history['accuracy'])\n",
|
||
"b3 = b2 + len(history_3.history['accuracy'])\n",
|
||
"\n",
|
||
"plt.figure(figsize=(16, 6))\n",
|
||
"plt.subplot(1, 2, 1)\n",
|
||
"plt.plot(acc, label='Training Accuracy', linewidth=2)\n",
|
||
"plt.plot(val_acc, label='Validation Accuracy', linewidth=2)\n",
|
||
"plt.axvline(x=b1, color='gray', linestyle='--', alpha=0.7, label='P2 (192)')\n",
|
||
"plt.axvline(x=b2, color='black', linestyle='--', alpha=0.7, label='P3 (224)')\n",
|
||
"plt.axvline(x=b3, color='blue', linestyle='--', alpha=0.7, label='SWA start')\n",
|
||
"plt.legend(fontsize=10)\n",
|
||
"plt.title('Training & Validation Accuracy', fontsize=14)\n",
|
||
"plt.xlabel('Epoch'); plt.ylabel('Accuracy'); plt.grid(alpha=0.3)\n",
|
||
"\n",
|
||
"plt.subplot(1, 2, 2)\n",
|
||
"plt.plot(loss, label='Training Loss', linewidth=2)\n",
|
||
"plt.plot(val_loss, label='Validation Loss', linewidth=2)\n",
|
||
"plt.axvline(x=b1, color='gray', linestyle='--', alpha=0.7, label='P2 (192)')\n",
|
||
"plt.axvline(x=b2, color='black', linestyle='--', alpha=0.7, label='P3 (224)')\n",
|
||
"plt.axvline(x=b3, color='blue', linestyle='--', alpha=0.7, label='SWA start')\n",
|
||
"plt.legend(fontsize=10)\n",
|
||
"plt.title('Training & Validation Loss', fontsize=14)\n",
|
||
"plt.xlabel('Epoch'); plt.ylabel('Loss'); plt.grid(alpha=0.3)\n",
|
||
"\n",
|
||
"plt.tight_layout()\n",
|
||
"plt.show()\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "27b920dd",
|
||
"metadata": {},
|
||
"source": [
|
||
"## 14. Temperature Scaling — Confidence Calibration\n",
|
||
"\n",
|
||
"Model deep learning cenderung **overconfident** — softmax probability tinggi tapi tidak mencerminkan\n",
|
||
"akurasi sebenarnya. Temperature scaling mengoptimalkan parameter T pada validation set:\n",
|
||
"\n",
|
||
"$$P_{calibrated} = softmax(logits / T)$$\n",
|
||
"\n",
|
||
"T > 1 → distribusi lebih flat (less confident).\n",
|
||
"T < 1 → distribusi lebih tajam (more confident).\n",
|
||
"T = 1 → tidak berubah (default).\n",
|
||
"\n",
|
||
"ECE (Expected Calibration Error) mengukur seberapa baik confidence sesuai dengan akurasi.\n",
|
||
"Target: **ECE < 0.05** setelah temperature scaling.\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "d79dbb47",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"def compute_ece(probs, true_labels, n_bins=15):\n",
|
||
" # Expected Calibration Error.\n",
|
||
" confs = np.max(probs, axis=1)\n",
|
||
" preds = np.argmax(probs, axis=1)\n",
|
||
" true = np.argmax(true_labels, axis=1)\n",
|
||
" accs = (preds == true).astype(np.float32)\n",
|
||
" bins = np.linspace(0, 1, n_bins + 1)\n",
|
||
" ece = 0.0\n",
|
||
" bin_stats = []\n",
|
||
" for i in range(n_bins):\n",
|
||
" in_bin = (confs > bins[i]) & (confs <= bins[i + 1])\n",
|
||
" n = np.sum(in_bin)\n",
|
||
" if n > 0:\n",
|
||
" bin_acc = np.mean(accs[in_bin])\n",
|
||
" bin_conf = np.mean(confs[in_bin])\n",
|
||
" ece += (n / len(confs)) * np.abs(bin_acc - bin_conf)\n",
|
||
" bin_stats.append((bins[i], n, bin_acc, bin_conf))\n",
|
||
" return ece, bin_stats\n",
|
||
"\n",
|
||
"# Collect logits and labels from validation set\n",
|
||
"print(\"Collecting validation logits...\")\n",
|
||
"logits_model = tf.keras.Model(model.input, model.output)\n",
|
||
"\n",
|
||
"all_logits = []\n",
|
||
"all_labels = []\n",
|
||
"for images, labels in val_ds_full.unbatch().batch(BATCH_SIZE):\n",
|
||
" all_logits.append(logits_model.predict_on_batch(images))\n",
|
||
" all_labels.append(labels.numpy())\n",
|
||
"\n",
|
||
"all_logits = np.concatenate(all_logits, axis=0)\n",
|
||
"all_labels = np.concatenate(all_labels, axis=0)\n",
|
||
"\n",
|
||
"# ECE before scaling (T=1)\n",
|
||
"probs_raw = tf.nn.softmax(all_logits).numpy()\n",
|
||
"ece_raw, _ = compute_ece(probs_raw, all_labels)\n",
|
||
"print(f\"ECE before scaling (T=1.0): {ece_raw:.4f}\")\n",
|
||
"\n",
|
||
"# Optimize T on validation set\n",
|
||
"if HAS_SCIPY:\n",
|
||
" def nll_temperature(T):\n",
|
||
" scaled = all_logits / float(T)\n",
|
||
" probs = tf.nn.softmax(scaled).numpy()\n",
|
||
" probs = np.clip(probs, 1e-7, 1.0 - 1e-7)\n",
|
||
" return -np.mean(np.log(np.sum(all_labels * probs, axis=1)))\n",
|
||
"\n",
|
||
" result = minimize_scalar(nll_temperature, bounds=(0.1, 5.0), method='bounded')\n",
|
||
" T_opt = result.x\n",
|
||
" print(f\"Optimal temperature: T = {T_opt:.4f}\")\n",
|
||
"else:\n",
|
||
" # Grid search fallback\n",
|
||
" best_nll, T_opt = float('inf'), 1.0\n",
|
||
" for T in np.linspace(0.5, 4.0, 36):\n",
|
||
" scaled = all_logits / T\n",
|
||
" probs = tf.nn.softmax(scaled).numpy()\n",
|
||
" probs = np.clip(probs, 1e-7, 1.0 - 1e-7)\n",
|
||
" nll = -np.mean(np.log(np.sum(all_labels * probs, axis=1)))\n",
|
||
" if nll < best_nll:\n",
|
||
" best_nll = nll\n",
|
||
" T_opt = T\n",
|
||
" print(f\"Optimal temperature (grid): T = {T_opt:.4f}\")\n",
|
||
"\n",
|
||
"# ECE after scaling\n",
|
||
"probs_cal = tf.nn.softmax(all_logits / T_opt).numpy()\n",
|
||
"ece_cal, bin_stats = compute_ece(probs_cal, all_labels)\n",
|
||
"print(f\"ECE after scaling (T={T_opt:.4f}): {ece_cal:.4f}\")\n",
|
||
"\n",
|
||
"# Save calibration metadata\n",
|
||
"calibration_meta = {\n",
|
||
" \"temperature\": float(T_opt),\n",
|
||
" \"conf_threshold_high\": 0.70,\n",
|
||
" \"conf_threshold_low\": 0.45,\n",
|
||
" \"ece_raw\": float(ece_raw),\n",
|
||
" \"ece_calibrated\": float(ece_cal),\n",
|
||
"}\n",
|
||
"with open(os.path.join(ckpt_dir, \"calibration.json\"), \"w\") as f:\n",
|
||
" json.dump(calibration_meta, f, indent=2)\n",
|
||
"print(f\"Calibration metadata saved to {os.path.join(ckpt_dir, 'calibration.json')}\")\n",
|
||
"\n",
|
||
"# Reliability diagram\n",
|
||
"plt.figure(figsize=(12, 5))\n",
|
||
"\n",
|
||
"plt.subplot(1, 2, 1)\n",
|
||
"if bin_stats:\n",
|
||
" bin_mids = [(s[0] + s[0] + 1/n_bins)/2 for s in bin_stats]\n",
|
||
" bin_accs = [s[2] for s in bin_stats]\n",
|
||
" bin_confs = [s[3] for s in bin_stats]\n",
|
||
" plt.bar(bin_mids, bin_accs, width=0.05, alpha=0.5, label='Accuracy')\n",
|
||
" plt.bar(bin_mids, bin_confs, width=0.05, alpha=0.3, label='Confidence')\n",
|
||
"plt.plot([0, 1], [0, 1], 'k--', alpha=0.3)\n",
|
||
"plt.xlabel('Confidence'); plt.ylabel('Accuracy')\n",
|
||
"plt.title(f'Reliability Diagram (T={T_opt:.2f})')\n",
|
||
"plt.legend(); plt.grid(alpha=0.3)\n",
|
||
"\n",
|
||
"plt.subplot(1, 2, 2)\n",
|
||
"conf_raw = np.max(probs_raw, axis=1)\n",
|
||
"conf_cal = np.max(probs_cal, axis=1)\n",
|
||
"plt.hist(conf_raw, bins=30, alpha=0.5, label='Before scaling', density=True)\n",
|
||
"plt.hist(conf_cal, bins=30, alpha=0.5, label='After scaling', density=True)\n",
|
||
"plt.xlabel('Max Confidence'); plt.ylabel('Density')\n",
|
||
"plt.title('Confidence Distribution')\n",
|
||
"plt.legend(); plt.grid(alpha=0.3)\n",
|
||
"\n",
|
||
"plt.tight_layout()\n",
|
||
"plt.show()\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "6e4a717b",
|
||
"metadata": {},
|
||
"source": [
|
||
"## 15. Test Time Augmentation + Evaluasi\n",
|
||
"\n",
|
||
"Menggunakan TTA 5× pada test set dengan augmented logit averaging.\n",
|
||
"Model output adalah **raw logits** → temperature scaling → softmax.\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "beb96b8b",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"# Load best model (dengan SWA weights)\n",
|
||
"print(f\"Loading best model from {checkpoint_path}...\")\n",
|
||
"best_model = tf.keras.models.load_model(checkpoint_path, compile=False)\n",
|
||
"\n",
|
||
"# Collect test images\n",
|
||
"raw_test_ds = tf.keras.utils.image_dataset_from_directory(\n",
|
||
" test_dir, shuffle=False, batch_size=BATCH_SIZE, image_size=IMG_SIZE)\n",
|
||
"\n",
|
||
"test_images = []\n",
|
||
"test_labels_raw = []\n",
|
||
"for images, labels in raw_test_ds.unbatch():\n",
|
||
" test_images.append(images.numpy())\n",
|
||
" test_labels_raw.append(labels.numpy())\n",
|
||
"\n",
|
||
"test_images = np.array(test_images)\n",
|
||
"test_labels_true = tf.one_hot(np.array(test_labels_raw), NUM_CLASSES).numpy()\n",
|
||
"\n",
|
||
"# ── TTA 5x ──\n",
|
||
"TTA_STEPS = 5\n",
|
||
"tta_logits = []\n",
|
||
"\n",
|
||
"for i in range(TTA_STEPS):\n",
|
||
" aug_images = geo_aug(test_images, training=True)\n",
|
||
" aug_images = preprocess_input(aug_images)\n",
|
||
" logits = best_model.predict(aug_images, batch_size=BATCH_SIZE, verbose=0)\n",
|
||
" tta_logits.append(logits)\n",
|
||
" print(f\" TTA step {i+1}/{TTA_STEPS}\")\n",
|
||
"\n",
|
||
"mean_logits = np.mean(tta_logits, axis=0)\n",
|
||
"# Apply temperature scaling\n",
|
||
"mean_cal_probs = tf.nn.softmax(mean_logits / T_opt).numpy()\n",
|
||
"\n",
|
||
"test_preds = np.argmax(mean_cal_probs, axis=1)\n",
|
||
"test_true = np.argmax(test_labels_true, axis=1)\n",
|
||
"tta_acc = np.mean(test_preds == test_true)\n",
|
||
"\n",
|
||
"print(f\"\\nTest Accuracy (TTA {TTA_STEPS}x, T={T_opt:.2f}): {tta_acc*100:.2f}%\")\n",
|
||
"\n",
|
||
"# ECE on test set\n",
|
||
"ece_test, _ = compute_ece(mean_cal_probs, test_labels_true)\n",
|
||
"print(f\"ECE on test set: {ece_test:.4f}\")\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "b6bb88ab",
|
||
"metadata": {},
|
||
"source": [
|
||
"## 16. Per-Source Accuracy Breakdown\n",
|
||
"\n",
|
||
"Mengukur akurasi per source prefix untuk mendeteksi **domain gap**.\n",
|
||
"Source yang akurasinya collapse (< 70%) menunjukkan model belum robust.\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "a24e9a14",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"def get_source(filename):\n",
|
||
" f = os.path.splitext(filename)[0]\n",
|
||
" if f.startswith('IMG_'): return 'Phone'\n",
|
||
" if f.startswith('Corn_'): return 'Lab_Corn'\n",
|
||
" for p in ['CBS', 'GLS', 'NLS', 'CLS']:\n",
|
||
" if f.startswith(p): return 'Lab_Disease'\n",
|
||
" for p in ['SCR', 'CR', 'NLB', 'SLB', 'SRS']:\n",
|
||
" if f.startswith(p): return 'Lab_RustBlight'\n",
|
||
" return 'Other'\n",
|
||
"\n",
|
||
"# Map each test image to its source\n",
|
||
"test_files = []\n",
|
||
"for cn in class_names:\n",
|
||
" cp = os.path.join(test_dir, cn)\n",
|
||
" if os.path.isdir(cp):\n",
|
||
" test_files.extend([(f, cn, get_source(f)) for f in sorted(os.listdir(cp))])\n",
|
||
"\n",
|
||
"sources = set(s for _, _, s in test_files)\n",
|
||
"print(f\"Sources found: {sorted(sources)}\")\n",
|
||
"print(f\"{'Source':<18} {'Count':>6} {'Accuracy':>10}\")\n",
|
||
"print(\"-\" * 38)\n",
|
||
"\n",
|
||
"for src in sorted(sources):\n",
|
||
" indices = [i for i, (_, _, s) in enumerate(test_files) if s == src]\n",
|
||
" if not indices: continue\n",
|
||
" n = len(indices)\n",
|
||
" acc = np.mean(test_preds[indices] == test_true[indices])\n",
|
||
" print(f\"{src:<18} {n:>6} {acc*100:>9.1f}%\")\n",
|
||
"\n",
|
||
"# Overall with count\n",
|
||
"overall_acc = np.mean(test_preds == test_true)\n",
|
||
"print(\"-\" * 38)\n",
|
||
"print(f\"{'ALL':<18} {len(test_preds):>6} {overall_acc*100:>9.1f}%\")\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "52754df5",
|
||
"metadata": {},
|
||
"source": [
|
||
"## 17. Classification Report & Confusion Matrix\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "1a5b2c96",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"print(\"\\nClassification Report:\\n\")\n",
|
||
"print(classification_report(test_true, test_preds, target_names=class_names))\n",
|
||
"\n",
|
||
"cm = confusion_matrix(test_true, test_preds)\n",
|
||
"plt.figure(figsize=(10, 8))\n",
|
||
"sns.heatmap(cm, annot=True, fmt='d', cmap='Blues',\n",
|
||
" xticklabels=class_names, yticklabels=class_names)\n",
|
||
"plt.title(f'Confusion Matrix (TTA {TTA_STEPS}x)', fontsize=14)\n",
|
||
"plt.xlabel('Predicted', fontsize=12)\n",
|
||
"plt.ylabel('True', fontsize=12)\n",
|
||
"plt.tight_layout()\n",
|
||
"plt.show()\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "528ef53b",
|
||
"metadata": {},
|
||
"source": [
|
||
"## 18. Simpan Model & Calibration Metadata\n",
|
||
"\n",
|
||
"Menyimpan model final (dengan SWA weights) dan calibration metadata ke `model/`.\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "292fef1a",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"# Save final model\n",
|
||
"model.save(final_path := os.path.join(ckpt_dir, 'best_model.keras'))\n",
|
||
"print(f\"Model saved to {final_path}\")\n",
|
||
"\n",
|
||
"# Copy calibration metadata to model directory\n",
|
||
"model_export_dir = os.path.join(os.getcwd(), 'model')\n",
|
||
"os.makedirs(model_export_dir, exist_ok=True)\n",
|
||
"\n",
|
||
"# Export labels.json with calibration metadata\n",
|
||
"cal_path = os.path.join(ckpt_dir, 'calibration.json')\n",
|
||
"if os.path.exists(cal_path):\n",
|
||
" with open(cal_path) as f:\n",
|
||
" cal_meta = json.load(f)\n",
|
||
"\n",
|
||
" labels_json = {\n",
|
||
" \"version\": \"3.0\",\n",
|
||
" \"labels\": class_names,\n",
|
||
" \"temperature\": cal_meta[\"temperature\"],\n",
|
||
" \"conf_threshold_high\": cal_meta[\"conf_threshold_high\"],\n",
|
||
" \"conf_threshold_low\": cal_meta[\"conf_threshold_low\"],\n",
|
||
" \"input_size\": [224, 224],\n",
|
||
" \"input_range\": [0, 255],\n",
|
||
" \"preprocessing\": \"resize_bilinear_224x224_no_normalization\",\n",
|
||
" \"architecture\": \"EfficientNetV2B0 + CBAM + Dense(512)\",\n",
|
||
" \"output_type\": \"logits\",\n",
|
||
" }\n",
|
||
"else:\n",
|
||
" labels_json = {\n",
|
||
" \"version\": \"3.0\",\n",
|
||
" \"labels\": class_names,\n",
|
||
" \"temperature\": 1.0,\n",
|
||
" \"input_size\": [224, 224],\n",
|
||
" \"input_range\": [0, 255],\n",
|
||
" \"output_type\": \"logits\",\n",
|
||
" }\n",
|
||
"\n",
|
||
"with open(os.path.join(model_export_dir, 'labels.json'), 'w') as f:\n",
|
||
" json.dump(labels_json, f, indent=2)\n",
|
||
"print(\"Labels + calibration metadata saved to model/labels.json\")\n",
|
||
"\n",
|
||
"# Export class names list (legacy)\n",
|
||
"with open(os.path.join(model_export_dir, 'labels.json'), 'r') as f:\n",
|
||
" pass # already written above\n",
|
||
"print(f\"Classes: {class_names}\")\n",
|
||
"print(f\"Temperature: {labels_json['temperature']:.4f}\")\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "62675ad6",
|
||
"metadata": {},
|
||
"source": [
|
||
"## 19. Export Model untuk Produksi\n",
|
||
"\n",
|
||
"Setelah training selesai, jalankan pipeline ekspor secara berurutan:\n",
|
||
"\n",
|
||
"### 1. SavedModel + TFLite\n",
|
||
"```bash\n",
|
||
"python save_model.py\n",
|
||
"```\n",
|
||
"Memuat `best_model/best_model.keras`, membangun arsitektur bersih, dan mengekspor ke:\n",
|
||
"- `model/saved_model/` — format produksi (output **raw logits**)\n",
|
||
"- `model/model.tflite` — untuk perangkat mobile/edge (INT8 quantization)\n",
|
||
"\n",
|
||
"### 2. ONNX (Rust ML Service)\n",
|
||
"```bash\n",
|
||
"python convert_onnx.py\n",
|
||
"```\n",
|
||
"Mengonversi SavedModel ke `model/model.onnx` untuk Rust/Axum/ONNX Runtime.\n",
|
||
"Output: **raw logits** (softmax + temperature scaling di Rust service).\n",
|
||
"\n",
|
||
"### 3. TensorFlow.js (Web) — optional\n",
|
||
"```bash\n",
|
||
"export PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python\n",
|
||
"tensorflowjs_converter --input_format=tf_saved_model --output_format=tfjs_graph_model --signature_name=serving_default --saved_model_tags=serve model/saved_model model/tfjs_model\n",
|
||
"```\n",
|
||
"\n",
|
||
"> **Catatan v3.0**: Model output adalah **raw logits** (tanpa softmax). Rust service menerapkan\n",
|
||
"> temperature scaling: `softmax(logits / T)` dengan `T` dari `labels.json`.\n",
|
||
"> Status prediksi ditentukan dari confidence: confident (≥70%), uncertain (45-70%), rejected (<45%).\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "de8c53b7",
|
||
"metadata": {},
|
||
"source": [
|
||
"## 20. Model Card — ZeaVis Edu v3.0\n",
|
||
"\n",
|
||
"| Atribut | Detail |\n",
|
||
"|---|---|\n",
|
||
"| **Nama Model** | ZeaVis Edu v3.0 — CBAM + RandAugment Classifier |\n",
|
||
"| **Versi** | 3.0 |\n",
|
||
"| **Arsitektur** | EfficientNetV2B0 + CBAM Attention + GAP + Dense(512) |\n",
|
||
"| **Params** | ~6.6M (5.9M base + 0.7M head) — 2× lebih ringan dari v2.0 |\n",
|
||
"| **Framework** | TensorFlow 2.x / Keras (float32) |\n",
|
||
"| **Output** | Raw logits → temperature scaling → softmax |\n",
|
||
"| **Dataset** | ~6000 gambar (4 kelas) — stratified split by source |\n",
|
||
"| **Kelas** | Bercak Daun, Daun Sehat, Hawar Daun, Karat Daun |\n",
|
||
"| **Input** | RGB 224×224, pixel [0, 255], resize BILINEAR |\n",
|
||
"| **Augmentasi** | RandAugment (15 ops, N=3) + Fog + Shadow + MixUp + CutMix + RandomErasing |\n",
|
||
"| **Training** | Progressive 128→160→192→224 + CosineDecay + SWA |\n",
|
||
"| **Calibration** | Temperature scaling (T optimized on val), ECE target < 0.05 |\n",
|
||
"| **Decision** | Confident (≥0.70), Uncertain (0.45-0.70), Rejected (<0.45) |\n",
|
||
"| **Target** | ≥90% real-world accuracy, per-source accuracy ≥70% semua domain |\n",
|
||
"| **Etika** | Hanya untuk edukasi/penelitian pertanian. Bukan pengganti diagnosis ahli. |\n",
|
||
"\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "f431a07c",
|
||
"metadata": {},
|
||
"source": [
|
||
"## 21. Upload ke Hugging Face\n",
|
||
"\n",
|
||
"```python\n",
|
||
"from huggingface_hub import HfApi\n",
|
||
"api = HfApi()\n",
|
||
"api.upload_folder(\n",
|
||
" folder_path=\"model\",\n",
|
||
" repo_id=\"zeavis-edu/corn-leaf-disease-classifier\",\n",
|
||
" repo_type=\"model\",\n",
|
||
")\n",
|
||
"```\n",
|
||
"Upload model SavedModel, TFLite, ONNX, TF.js, dan metadata ke HF Hub.\n"
|
||
]
|
||
}
|
||
],
|
||
"metadata": {
|
||
"kernelspec": {
|
||
"display_name": ".venv",
|
||
"language": "python",
|
||
"name": "python3"
|
||
},
|
||
"language_info": {
|
||
"codemirror_mode": {
|
||
"name": "ipython",
|
||
"version": 3
|
||
},
|
||
"file_extension": ".py",
|
||
"mimetype": "text/x-python",
|
||
"name": "python",
|
||
"nbconvert_exporter": "python",
|
||
"pygments_lexer": "ipython3",
|
||
"version": "3.12.3"
|
||
}
|
||
},
|
||
"nbformat": 4,
|
||
"nbformat_minor": 5
|
||
}
|