# Camera, plant phenotyping and machine learning Canonical HTML: https://flavorotor.com/research/imaging-plant-health Machine-readable index: https://flavorotor.com/research/data/chapters/index.json Indexed camera geometry, longitudinal plant measurements, real image datasets, reproducible augmentation, grouped validation and measured MobileNetV2, EfficientNetB0 and ResNet50 performance. Plant imaging The central camera photographs the same plant position repeatedly and stores every image with its plant, cultivation cycle and camera settings. **Central camera.** The camera remains stationary while the rotor brings each plant to its recorded image position. [I02] **Camera and light module.** One acquisition record identifies camera pose, plant position and illumination state. [I02] ## Camera geometry The camera is fixed to the central module. The rotor stops at a known encoder position and presents one plant module to the lens. At that position, the optical axis meets the local plant plane at 90°. The distance, lens and framing therefore remain comparable when the same plant returns for its next image. The acquisition record stores camera version, lens, focus, working distance, encoder position and image dimensions. [I02] [R52] Note. FlavoRotor camera geometry is documented in I02. Repeatable phenotyping practice follows Arend et al. (2016). [I02] [R52] ## What is stored with each image The image file alone is not enough. Its record identifies the plant, growth cycle, rotor position, exposure, gain, white balance and light state. A scale reference makes pixel measurements comparable. A colour target reveals changes in illumination or camera response. Focus, clipping and occlusion are stored as visible image-quality fields. | Field group | Stored values | Reason | Plant | plant_id, crop, cultivar, seed lot, cycle_id | keeps repeated images attached to one biological specimen | Position | position_id, encoder index, camera pose, working distance | shows where and how the image was taken | Camera | camera_version, lens, focus, exposure, gain, white balance | separates plant change from camera change | Growing conditions | recipe_version, light state, temperature, pH, EC, rotation state | connects the image to the measured environment | File history | timestamp_utc, SHA-256, annotation version, operator | identifies the exact file and label version ## Images used for model development Three image sources have different jobs. ImageNet supplies the general visual weights used to initialise MobileNetV2. PlantVillage supplies clean, labelled leaf images. PlantDoc adds leaves photographed with natural backgrounds and changing viewpoints. FlavoRotor images represent the camera, lighting and plant geometry in which the model operates. Results from these sources remain separate because a clean single-leaf photograph is different from a plant growing inside the machine. [R30] [R51] [R53] [R56] | Images | Content | Use | ImageNet-1K [R53] | general photographs from many object classes | initial weights for edges, textures and shapes | PlantVillage [R30] | 54,306 controlled RGB leaf images covering healthy tissue and plant diseases | controlled leaf-classification benchmark | PlantDoc [R51] | 2,598 plant images from 13 species and 27 healthy or disease classes | comparison under natural backgrounds and variable framing | FlavoRotor [I02] | indexed images from the central camera | device-specific plant tracking and evaluation The reproducible web example downloads the complete PlantVillage Strawberry RGB subset at repository commit 7f7ecc7 : 456 healthy images and 1,109 leaf-scorch images. The repository's published leaf map identifies 1,232 of those images as observations of 190 physical leaves. The Python script records every selected filename, SHA-256 hash, class, leaf group and transform parameter. Note. Images from the PlantVillage Strawberry RGB subset (Mohanty, Hughes, & Salathé, 2016), repository commit 7f7ecc7, CC BY-SA 3.0. Displayed at a common size without synthetic symptoms. [R30] Note. Counts were computed from PlantVillage commit 7f7ecc7 by scripts/prepare-plantvision-dataset.py. Bars show raw images; the inset values identify grouped images and physical leaf groups. [R30] ## Split by physical leaf Photographs of the same physical leaf stay together. The deterministic seed 20260729 assigns leaf groups to 70% training, 15% validation and 15% test partitions. This prevents near-duplicate photographs of one leaf from appearing in both training and test data. | Class | Training | Validation | Test | Healthy | 316 images / 80 leaves | 68 images / 17 leaves | 72 images / 18 leaves | Leaf scorch | 534 images / 52 leaves | 115 images / 11 leaves | 127 images / 12 leaves | Total | 850 images / 132 leaves | 183 images / 28 leaves | 199 images / 30 leaves ## Image augmentation The Python preprocessing script creates the examples below from one published leaf-scorch image. Rotation, scale, brightness, contrast, saturation, blur and sensor noise change the complete frame by a recorded amount. They do not paint new spots or remove existing tissue. Only the training partition receives random augmentation; validation and test images keep their original pixels apart from the fixed resize and normalisation. [R54] Note. Source image: PlantVillage Strawberry leaf scorch, leaf group 69, commit 7f7ecc7, CC BY-SA 3.0. Parameters and SHA-256 hashes are stored in manifest.json. Transform structure follows Buslaev et al. (2020). [R30] [R54] ## MobileNetV2 and the MLP classifier The reference model receives a 224 × 224 sRGB image. MobileNetV2, initialised with ImageNet-1K weights, converts the image into a 7 × 7 × 1,280 feature map. Global average pooling produces a 1,280-value vector. The MLP maps that vector to 256 ReLU6 units, applies dropout 0.25, and produces one logit for every declared plant-health class. [R31] [R53] | Stage | Shape | Operation | Input | 224 × 224 × 3 | sRGB image with ImageNet normalisation | Encoder | 7 × 7 × 1,280 | MobileNetV2 features | Pooling | 1,280 | global average pooling | Hidden layer | 256 | linear layer, ReLU6 and dropout 0.25 | Output | C | one logit per class | Probability | C | softmax followed by validation-set temperature scaling ### Why this model fits the camera task MobileNetV2 uses depthwise convolutions and inverted residual blocks. With the 256-unit classifier, it contains 2,586,434 parameters. The same camera is evaluated at every indexed plant position, so model size and single-image latency matter alongside classification performance. [R31] ### The classifier head Global average pooling converts the 7 × 7 × 1,280 encoder output into one 1,280-value feature vector. The multilayer perceptron learns combinations of those features that separate the declared classes. ReLU6 bounds each hidden activation between 0 and 6; dropout removes 25% of hidden activations at random during fitting. The final linear layer returns two logits, which softmax converts into class probabilities. CV-MLP h = Dropout ⁡  ⁣ ( ReLU6 ⁡  ⁣ ( W 1 z + b 1 ) , 0.25 ) , ℓ = W 2 h + b 2 \mathbf{h}=\operatorname{Dropout}\!\left(\operatorname{ReLU6}\!\left(\mathbf{W}_1\mathbf{z}+\mathbf{b}_1\right),0.25\right),\qquad\boldsymbol{\ell}=\mathbf{W}_2\mathbf{h}+\mathbf{b}_2 h = Dropout ( ReLU6 ( W 1 ​ z + b 1 ​ ) , 0.25 ) , ℓ = W 2 ​ h + b 2 ​ z is the 1,280-value MobileNetV2 feature vector, h contains 256 hidden activations and ℓ contains one logit per class. **Explanation**The frozen image encoder produces a feature vector. A 256-unit hidden layer transforms it, dropout regularises training and the output layer produces one score for each class. The 1,280 × 256 hidden layer has 327,680 weights and 256 biases. The 256 × 2 output has 512 weights and 2 biases. The MLP therefore contributes 328,450 trainable parameters. ### Encoder comparison MobileNetV2, EfficientNetB0 and ResNet50 were tested with the same frozen ImageNet protocol, 256-unit ReLU6 MLP, physical-leaf split and three deterministic seeds. All three reached 100% mean accuracy on the 199-image controlled test. The natural-background PlantDoc check separated them: mean healthy-class recall was 84.38% for MobileNetV2, 74.65% for EfficientNetB0 and 55.90% for ResNet50. PlantVillage controlled test and PlantDoc natural-background transfer check; TensorFlow 2.18 CPU inference on an Intel Xeon E5-2699 v3 with a three-core Docker limit. [R30] [R31] [R51] [R53] MobileNetV2 also required the fewest parameters and had the lowest measured CPU latency: 2.59 million parameters and 60.59 ms median, compared with 4.38 million and 83.35 ms for EfficientNetB0, and 24.11 million and 171.27 ms for ResNet50. Latency was measured with TensorFlow 2.18 inside a three-core Docker limit on an Intel Xeon E5-2699 v3. Processor choice changes throughput, not the mathematical definition of accuracy. ### Five-fold grouped cross-validation A second test used all 1,232 images tied to 190 physical leaves. The folds were stratified by class; every leaf appeared in one test fold and never in that fold’s training or validation data. MobileNetV2 and the MLP were refitted in every fold. Aggregate accuracy was 99.84%, balanced accuracy 99.87% and macro F1 99.83%; two of 776 leaf-scorch images were assigned healthy and all 456 healthy images were assigned correctly. Mean PlantDoc healthy recall across the five fitted models was 80.00% with a standard deviation of 8.38 percentage points. MobileNetV2 with a refitted 256-unit ReLU6 MLP in each fold; 1,232 grouped PlantVillage Strawberry images and a separate 96-image PlantDoc transfer check. [R30] [R31] [R51] The image classifier assigns one of its declared visual labels to the current frame. Biomass forecasting, nutrient-state estimation and stress attribution use separate models with repeated images and measured cultivation variables. Note. The encoder follows MobileNetV2 (Sandler et al., 2018), initialised with ImageNet weights (Deng et al., 2009); confidence calibration follows Guo et al. (2017). [R30] [R31] [R51] [R53] [R55] [R56] ## Model fitting - **Fit the MLP.** The ImageNet encoder remains frozen while the classifier learns from the grouped training images. - **Fine-tune the final encoder blocks.** A lower learning rate adjusts the highest-level visual features; validation macro F1 controls early stopping. - **Calibrate probability.** One temperature value is fitted to validation logits after the model weights stop changing. [R55] - **Lock the test.** Architecture, preprocessing, class thresholds and the low-confidence rule are fixed before test images are opened. - **Report each image domain separately.** PlantVillage, PlantDoc and FlavoRotor results use separate tables, because their camera conditions differ. Note. TensorFlow 2.18.0 run completed 29 July 2026 with seed 20260729. Values are read directly from training-history.csv; the vertical axis is limited to 0.95–1.00 so small changes remain visible. [R30] [R31] [R53] ## Controlled-image benchmark The reproducible TensorFlow 2.18.0 run completed on 29 July 2026. After five epochs, the locked model classified all 199 images in the grouped PlantVillage test partition correctly: 72 healthy and 127 leaf scorch. The test partition contains 30 physical leaf groups that were absent from training and validation. Accuracy, macro F1 and balanced accuracy are each 1.0000 for this controlled two-class benchmark. The fitted temperature is 0.500584 and the ten-bin expected calibration error is 0.0000361. Note. The leaf-grouped test partition contains 199 previously withheld images from 30 physical leaves. The complete TensorFlow run and model hash are recorded in training-summary.json. [R30] [R31] PlantVillage photographs isolated leaves against a controlled background. The result above therefore measures discrimination between those two published Strawberry classes under the same acquisition style; it is not substituted for a measurement from the FlavoRotor camera. [R30] [R56] ### Uncertainty around the measured scores Every classification score is estimated from a finite test set. The controlled test observed 199 correct assignments from 199 images, but its exact two-sided 95% interval is 98.16–100%. Class recall has wider intervals because each class contains fewer observations: 95.01–100% for 72 healthy images and 97.14–100% for 127 leaf-scorch images. The interval states how much precision the test count provides; it does not change the observed confusion matrix. Note. Two-sided Clopper–Pearson intervals calculated from the locked test counts. PlantVillage and PlantDoc are reported separately because their image domains differ. [R30] [R51] [R55] ## Natural-background check The frozen model was then applied without retraining to all 96 images in PlantDoc's *Strawberry leaf* class. These photographs contain natural backgrounds, changing scale, partial leaves and varied lighting. The model assigned 79 images to healthy and 17 to leaf scorch, which gives healthy-class recall of 79 / 96 = 0.8229. PlantDoc does not publish a matching Strawberry leaf-scorch class, so this check reports recall for its healthy class rather than two-class accuracy. [R51] [R56] Note. Images from the PlantDoc Strawberry leaf class (Singh et al., 2020), repository commit 5467f60, CC BY 4.0. Labels shown here are the upstream dataset labels; probabilities are produced by the recorded FlavoRotor reference run. [R51] Note. 79 of 96 images were assigned healthy and 17 leaf scorch, giving healthy-class recall of 82.29%. This is a single-class check, because PlantDoc does not include a matching Strawberry leaf-scorch class. [R51] ## Metric definitions Precision answers: of the images assigned to one class, how many are correct? Recall answers: of the images that truly belong to that class, how many were found? F1 combines both values. Macro F1 gives every class the same weight, so the larger leaf-scorch class cannot hide weak performance on healthy leaves. The report also contains the complete confusion matrix, balanced accuracy, image count and physical-leaf count for every class. [R55] [R56] CV-F1 F 1 = 2   p r e c i s i o n ⋅ r e c a l l p r e c i s i o n + r e c a l l F_1=2\,\frac{\mathrm{precision}\cdot\mathrm{recall}}{\mathrm{precision}+\mathrm{recall}} F 1 ​ = 2 precision + recall precision ⋅ recall ​ F1 combines precision and recall. Macro F1 is the arithmetic mean of the class-level F1 values, so a large class cannot hide poor performance on a smaller class. **Explanation**F1 combines precision and recall. Macro F1 is the arithmetic mean of the class-level F1 values, so a large class cannot hide poor performance on a smaller class. CV-ECE E C E = ∑ m = 1 M ∣ B m ∣ n   ∣ a c c ( B m ) − c o n f ( B m ) ∣ \mathrm{ECE}=\sum_{m=1}^{M}\frac{\lvert B_m\rvert}{n}\,\left\lvert\mathrm{acc}(B_m)-\mathrm{conf}(B_m)\right\rvert ECE = m = 1 ∑ M ​ n ∣ B m ​ ∣ ​ ∣ acc ( B m ​ ) − conf ( B m ​ ) ∣ Predictions are grouped into confidence bins. ECE measures the weighted difference between observed accuracy and mean reported confidence in those bins. **Explanation**Predictions are grouped into confidence bins. ECE is the weighted difference between observed accuracy and mean reported confidence in those bins. ## Following one plant through time Repeated images of the same plant form a time series. Projected canopy area, calibrated colour, developmental stage and image quality are stored beside temperature, light, pH, EC and recipe version at the same timestamp. Growth rate compares the same plant at two recorded times. It does not compare unrelated plants photographed on different days. [R52] | Output | Reference annotation | Use | Canopy area and growth rate | manual masks and a physical scale reference | tracks the plant's visible growth | Developmental stage | crop-specific, expert-reviewed labels | aligns treatment timing with plant development | Colour index | colour target and matching laboratory measurements | measures visible colour change | Plant-health class | expert label and supporting laboratory result where required | records class and probability for review | Image quality | focus, exposure, occlusion and pose labels | identifies unsuitable images ### Datasets used to test longitudinal methods | Dataset | Repeated observations | Role in the research programme | Aalto lettuce [R57] | 18 identified heads, 30 biomass days, 731 canopy images and 1,443 environmental records | implemented three-day biomass forecast with plant-wise validation | HydroGrowNet [R61] | three 30-day Batavia cycles and more than 390,000 segmented images aligned with pH, EC and water temperature | independent multimodal growth and anomaly dataset | Multi-sensor lettuce phenotyping [R62] | 45 plants over 42 days, two cultivars, three nitrogen levels and two irrigation rates | external RGB, 3D, multispectral, SPAD, fluorescence and morphology dataset ### A separate model for each measured endpoint | Question | Model | Reason | Reference value | Does the current leaf image match a declared visual class? | MobileNetV2 + 256-unit MLP | compact image encoder; class probabilities can be calibrated and reviewed | expert or published class label | What fresh biomass is expected three days from now? | ridge autoregression | uses repeated mass and recent growth increments while regularising a small dataset | measured fresh biomass [R57] | How did cultivar and nutrient solution change tissue chemistry? | factorial ANOVA | tests cultivar, treatment and their interaction directly | laboratory nitrogen, sulphate, organic acid and chlorophyll measurements [R59] | Is the plant departing from its expected trajectory? | forecast residual plus consecutive-capture rule | requires persistence through time and retains the sensor and image context | next measured observation ## Stored model output Every model output stores the image hash, plant ID, model version, preprocessing version, probability for every class, calibrated confidence and image-quality score. A low-quality or low-confidence image is marked for review together with the reason. The original image and complete class-probability vector remain available beside the final label. ## References - [I02] FlavoRotor project team (2026). FlavoRotor v2.0 system architecture. *Internal engineering design report*. - [R30] Mohanty, Sharada P.; Hughes, David P.; Salathé, Marcel (2016). Using Deep Learning for Image-Based Plant Disease Detection. *Frontiers in Plant Science*. https://doi.org/10.3389/fpls.2016.01419 - [R31] Sandler, Mark; Howard, Andrew; Zhu, Menglong; Zhmoginov, Andrey; Chen, Liang-Chieh (2018). MobileNetV2: Inverted Residuals and Linear Bottlenecks. *2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition*. https://doi.org/10.1109/CVPR.2018.00474 - [R51] Singh, Davinder; Jain, Naman; Jain, Pranjali; Kayal, Pratik; Kumawat, Sudhakar; Batra, Nipun (2020). PlantDoc: A Dataset for Visual Plant Disease Detection. *Proceedings of the 7th ACM IKDD CoDS and 25th COMAD, 249–253*. https://doi.org/10.1145/3371158.3371196 - [R52] Arend, Daniel; Lange, Matthias; Pape, Jean-Michel; Weigelt-Fischer, Kathleen; Arana-Ceballos, Fernando; Mücke, Ingo; Klukas, Christian; Altmann, Thomas; Scholz, Uwe; Junker, Astrid (2016). Quantitative monitoring of Arabidopsis thaliana growth and development using high-throughput plant phenotyping. *Scientific Data, 3, 160055*. https://doi.org/10.1038/sdata.2016.55 - [R53] Deng, Jia; Dong, Wei; Socher, Richard; Li, Li-Jia; Li, Kai; Fei-Fei, Li (2009). ImageNet: A Large-Scale Hierarchical Image Database. *2009 IEEE Conference on Computer Vision and Pattern Recognition, 248–255*. https://doi.org/10.1109/CVPR.2009.5206848 - [R54] Buslaev, Alexander; Iglovikov, Vladimir I.; Khvedchenya, Eugene; Parinov, Alex; Druzhinin, Mikhail; Kalinin, Alexandr A. (2020). Albumentations: Fast and Flexible Image Augmentations. *Information, 11(2), 125*. https://doi.org/10.3390/info11020125 - [R55] Guo, Chuan; Pleiss, Geoff; Sun, Yu; Weinberger, Kilian Q. (2017). On Calibration of Modern Neural Networks. *Proceedings of the 34th International Conference on Machine Learning, PMLR 70, 1321–1330*. - [R56] Xiang, Kun; Shi, Danxi; Zhu, Xiangbo (2026). Quantifying the reliability gap in cross-domain plant disease classification: benchmarking the limited efficacy of standard mitigation techniques under controlled-to-field shift. *Frontiers in Plant Science, 17, 1826962*. https://doi.org/10.3389/fpls.2026.1826962 - [R57] Karimzadeh, Sara; Ahamed, M. Shamim (2025). Lettuce Dataset with RGB Canopy Images, Biomass, Nutrient Solution, and Environmental Variables for Machine Learning Model Development. *Zenodo, Version 1*. https://doi.org/10.5281/zenodo.16912088 - [R59] El-Nakhel, Christophe; Pannico, Antonio; Kyriacou, Marios C.; Petropoulos, Spyridon A.; Giordano, Maria; Colla, Giuseppe; Troise, Antonio Dario; Vitaglione, Paola; De Pascale, Stefania; Rouphael, Youssef (2020). Dataset on the organic acids, sulphate, total nitrogen and total chlorophyll contents of two lettuce cultivars grown hydroponically using nutrient solutions of variable macrocation ratios. *Data in Brief, 29, 105135*. https://doi.org/10.1016/j.dib.2020.105135 - [R61] Shalash, Omar; Hassan, Nayira; Métwalli, Ahmed; Elhefny, Alia (2025). HydroGrowNet of Batavia Dataset. *Mendeley Data, Version 5*. https://doi.org/10.17632/g6cm3v3wdp.5 - [R62] Rodrigues, Leandro; Terra, Francisco; Rodrigues, Pedro; Moreira, Germano; Oliveira, Francisco; Moura, Pedro; Pinheiro, Isabel; Santos, Filipe; Cunha, Mário (2026). Multi-Sensor High-Throughput Phenotyping Dataset of Hydroponic Lettuce under Variable Fertigation Conditions. *Zenodo*. https://doi.org/10.5281/zenodo.20759414