fix(ci): download model from Hugging Face instead of storing in repo

- Remove binary model artifacts tracked in repo (.tflite, saved_model, tfjs model.json)
- CI now downloads best_model.keras from Hugging Face Hub (MythEclipse2737/corn-leaf-disease-classifier)
- Add save_model.py step before convert_onnx.py in CI pipeline
- Remove LFS patterns from .gitattributes (no longer needed)
- Remove lfs:true from checkout action

The model is fetched at CI time using HF_TOKEN secret — no large
binary files stored in git.

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
MythEclipse
2026-06-11 20:05:55 +00:00
co-authored by Claude
parent 8137057186
commit 7f17c19dc8
8 changed files with 158 additions and 13 deletions
+1 -6
View File
@@ -1,6 +1 @@
best_model/best_model.keras filter=lfs diff=lfs merge=lfs -text # Model artifacts are downloaded from Hugging Face at CI time — not stored in this repo.
model/saved_model/variables/variables.data-00000-of-00001 filter=lfs diff=lfs merge=lfs -text
model/tfjs_model/*.bin filter=lfs diff=lfs merge=lfs -text
**/best_model.keras filter=lfs diff=lfs merge=lfs -text
**/variables.data* filter=lfs diff=lfs merge=lfs -text
**/tfjs_model/*.bin filter=lfs diff=lfs merge=lfs -text
+18 -5
View File
@@ -28,8 +28,6 @@ jobs:
steps: steps:
- name: Checkout repository - name: Checkout repository
uses: actions/checkout@v4 uses: actions/checkout@v4
with:
lfs: true
- name: Set image prefix - name: Set image prefix
run: echo "IMAGE_PREFIX=ghcr.io/${GITHUB_REPOSITORY,,}" >> "$GITHUB_ENV" run: echo "IMAGE_PREFIX=ghcr.io/${GITHUB_REPOSITORY,,}" >> "$GITHUB_ENV"
@@ -37,18 +35,33 @@ jobs:
- name: Set up Docker Buildx - name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3 uses: docker/setup-buildx-action@v3
- name: Set up Python for ONNX conversion - name: Set up Python for model download & export
if: matrix.service.name == 'ml' if: matrix.service.name == 'ml'
uses: actions/setup-python@v5 uses: actions/setup-python@v5
with: with:
python-version: '3.11' python-version: '3.11'
- name: Generate ONNX model artifact - name: Download model from Hugging Face & export ONNX
if: matrix.service.name == 'ml' if: matrix.service.name == 'ml'
working-directory: Machine_Learning working-directory: Machine_Learning
env:
HF_TOKEN: ${{ secrets.HF_TOKEN }}
run: | run: |
python -m pip install --upgrade pip python -m pip install --upgrade pip
python -m pip install 'tensorflow>=2.13.0' 'tf2onnx>=1.16.1' 'onnx>=1.16.0' python -m pip install 'tensorflow>=2.13.0' 'tf2onnx>=1.16.1' 'onnx>=1.16.0' 'huggingface_hub'
python -c "
from huggingface_hub import hf_hub_download
import os
os.makedirs('best_model', exist_ok=True)
path = hf_hub_download(
repo_id='MythEclipse2737/corn-leaf-disease-classifier',
filename='best_model.keras',
token=os.environ.get('HF_TOKEN'),
local_dir='best_model',
)
print(f'Model downloaded to {path}')
"
python save_model.py
python convert_onnx.py python convert_onnx.py
- name: Log in to GHCR - name: Log in to GHCR
Binary file not shown.
@@ -1 +0,0 @@
’ָחׁ›¡מY§¢¥ִּ™ֳֵא’–‎׃טׂקֶ־׸¥ִם—·(¡«‹…ֿמ»ך{2:39606749746474394
Binary file not shown.
File diff suppressed because one or more lines are too long
+139
View File
@@ -0,0 +1,139 @@
# 📊 Laporan Perubahan — `ddced09` → `8137057` (HEAD)
**Periode:** 11 Juni 2026, 15:17 — 19:42 UTC
**Branch:** `main`
**Total commit:** 6 (ddced09 tidak termasuk, itu adalah base)
---
## 📜 Daftar Commit
| # | Hash | Tanggal | Deskripsi |
|---|------|---------|-----------|
| 1 | `b83b2e1` | 15:18 | **feat**: download dataset dari Google Drive saat running locally |
| 2 | `7a7e5b5` | 15:19 | **fix**: update Google Drive file ID ke link dataset yang benar |
| 3 | `8c65e33` | 16:31 | **chore**: clear notebook outputs dan tambah `.gitignore` |
| 4 | `c733f3b` | 16:39 | *WIP commit* |
| 5 | `4377be1` | 16:49 | **chore**: update notebook execution count dan outputs |
| 6 | `8137057` | 19:42 | *WIP commit — hasil kerja sesi Hugging Face integration* |
---
## 📁 File yang Berubah (dari `ddced09` → HEAD)
| File | Perubahan | Keterangan |
|------|-----------|------------|
| `Machine_Learning/notebook.ipynb` | +3,857/-348 | Notebook dual-compatible (Colab + local), section 118 lengkap |
| `Machine_Learning/.gitignore` | +19 (new) | Ignore `dataset/`, `dataset_split/`, `best_model/`, `model/`, dll |
| `Machine_Learning/requirements.txt` | +2 | Tambah `gdown` + `huggingface_hub` |
| `Machine_Learning/save_model.py` | 1 line | Fix: `sigmoid``softmax` di output layer |
| `Machine_Learning/upload_hf.py` | +258 (new) | Script export pipeline + upload ke Hugging Face |
| `model/model.tflite` | binary | Regenerated (softmax fix) |
| `model/saved_model/` | binary | Regenerated |
**Total:** 9 file, +3,792 insertions, -348 deletions
---
## 🔍 Detail Perubahan per Area
### 1. Dataset Download (`b83b2e1`, `7a7e5b5`)
- Tambah dependency `gdown` ke `requirements.txt`
- Notebook sekarang auto-download dataset dari Google Drive (`file_id: 1s0H2l...`) saat running locally jika file ZIP belum ada
- Colab path tetap pakai `drive.mount()`
- Drive file ID diupdate di commit `7a7e5b5`
### 2. Notebook Restructuring (`ddced09` → `8c65e33` → `4377be1`)
- Notebook direstruktur dari numbering 1/2/4 menjadi section bernomor rapi 117 (kemudian 18)
- Semua section punya header markdown yang deskriptif
- Hyperparameter (seed, IMG_SIZE, BATCH_SIZE) ditambahkan
- Import diperluas: `AdamW`, `compute_class_weight`, `preprocess_input`, `Counter`, `random`
- Mixed precision policy `float32` eksplisit
- Cell outputs cleared di commit `8c65e33`
- Execution count dan outputs diupdate di `4377be1`
### 3. `.gitignore` (`8c65e33`)
File baru `Machine_Learning/.gitignore` mengabaikan:
- `dataset/`, `dataset_split/`, `dataset_jagung.zip`
- `best_model/`, `model/`
- Path development lainnya (`node_modules/`, `.bun/`, `.moon/cache/`, `.env`, `dist/`, `build/`, `coverage/`, `venv/`, `.claude/`, dll)
### 4. `save_model.py` Fix (`8137057`)
```diff
- outputs = layers.Dense(num_classes, activation='sigmoid', dtype='float32')(x)
+ outputs = layers.Dense(num_classes, activation='softmax', dtype='float32')(x)
```
Perbaikan kritis: output 4 kelas harus softmax, bukan sigmoid.
### 5. Hugging Face Upload (`8137057`) 🔥 **Fitur Baru**
**`Machine_Learning/upload_hf.py`** — 258 lines script mandiri:
```
Pipeline:
save_model.py → convert_onnx.py → TFJS converter (optional)
→ labels.json + README.md
→ upload ke Hugging Face Hub
```
- Repo: `MythEclipse2737/corn-leaf-disease-classifier`
- Auth via `HF_TOKEN` env var
- TFJS conversion graceful-fail (protobuf version conflict known)
- Upload semua: `best_model.keras` (213MB) + TFLite + ONNX + SavedModel + TFJS (partial) + labels.json + README.md
- **Hasil:** 11 files terupload ke https://huggingface.co/MythEclipse2737/corn-leaf-disease-classifier
**Notebook Section 18** — cell markdown + code untuk call `upload_hf.py` otomatis.
---
## 📈 Working Tree (Uncommitted Changes)
Saat ini **tidak ada uncommitted changes**`8137057` adalah commit terakhir yang mencakup semua hasil kerja.
---
## ✅ Ringkasan Outcome
| Goal | Status |
|------|--------|
| Notebook dual-compatible (Colab + local) | ✅ |
| Auto-download dataset dari Google Drive | ✅ |
| `.gitignore` untuk artifacts besar | ✅ |
| Fix `sigmoid``softmax` | ✅ |
| Hugging Face integration (upload script + notebook section) | ✅ |
| Model artifacts uploaded to HF Hub | ✅ 11 files |
| Temporarily pinned to `MythEclipse2737/` namespace | ⚠️ not `zeavis-edu/` (no org access) |
### Files on Hugging Face
```
MythEclipse2737/corn-leaf-disease-classifier
├── .gitattributes
├── README.md
├── best_model.keras (213 MB)
├── model/
│ ├── labels.json
│ ├── model.onnx
│ ├── model.tflite
│ ├── saved_model/
│ │ ├── fingerprint.pb
│ │ ├── saved_model.pb
│ │ └── variables/
│ │ ├── variables.data-00000-of-00001
│ │ └── variables.index
│ └── tfjs_model/
│ └── model.json (partial — no weight shards)
```
---
## ⚠️ Catatan
1. **TFJS converter gagal** karena protobuf version mismatch (`tensorflow_decision_forests``yggdrasil_decision_forests`). Model TF.js di HF hanya berisi `model.json` (metadata saja, tidak ada weight shards). Issue ini pre-existing dan tidak blocking.
2. **HF repo namespace**: pakai `MythEclipse2737/` karena token tidak punya write access ke `zeavis-edu/` org.
3. **Token HF terekspos** di chat — perlu di-rotate di https://huggingface.co/settings/tokens.