Removed deepforest_docs folder

Use models from Huggingface Clean up the tests Update style for tests Switch to opencv-python-headless Using opencv-python was causing installation conflicts with albumentations which relies on the headless version. We were only using the GUI aspects of opencv in a single, debugging only, function. So this switches that function to matplotlib and updates the dependencies. Fixes #464. Don't install opencv via apt for testing We want to test the Python installs so we shouldn't be doing an apt install. This is particularly true since we've switched to opencv-python-headless as the dependency.
weecology · Aug 20, 2024 · 4ed6f76 · 4ed6f76
1 parent 1a7cef1
commit 4ed6f76
Show file tree

Hide file tree

Showing 62 changed files with 88 additions and 170 deletions.
diff --git a/docs/source/deepforest_docs/CONTRIBUTING.md → docs/CONTRIBUTING.md b/docs/source/deepforest_docs/CONTRIBUTING.md → docs/CONTRIBUTING.md
diff --git a/docs/Makefile b/docs/Makefile
@@ -5,7 +5,7 @@
 SPHINXOPTS    =
 SPHINXBUILD   = python -msphinx
 SPHINXPROJ    = deepforest
-SOURCEDIR     = source
+SOURCEDIR     = .
 BUILDDIR      = _build
 
 # Put it first so that "make" without argument is like "make help".

diff --git a/docs/source/_templates/navbar2.html → docs/_templates/navbar2.html b/docs/source/_templates/navbar2.html → docs/_templates/navbar2.html
diff --git a/...rest_docs/advanced_features/CropModels.md → docs/advanced_features/CropModels.md b/...rest_docs/advanced_features/CropModels.md → docs/advanced_features/CropModels.md
diff --git a/...docs/advanced_features/ExtendingModule.md → docs/advanced_features/ExtendingModule.md b/...docs/advanced_features/ExtendingModule.md → docs/advanced_features/ExtendingModule.md
diff --git a/...s/advanced_features/Model_Architecture.md → docs/advanced_features/Model_Architecture.md b/...s/advanced_features/Model_Architecture.md → docs/advanced_features/Model_Architecture.md
diff --git a/...epforest_docs/advanced_features/index.rst → docs/advanced_features/index.rst b/...epforest_docs/advanced_features/index.rst → docs/advanced_features/index.rst
diff --git a/...t_docs/advanced_features/multi_species.md → docs/advanced_features/multi_species.md b/...t_docs/advanced_features/multi_species.md → docs/advanced_features/multi_species.md
diff --git a/...pforest_docs/advanced_features/scaling.md → docs/advanced_features/scaling.md b/...pforest_docs/advanced_features/scaling.md → docs/advanced_features/scaling.md
diff --git a/..._docs/advanced_features/visualizations.md → docs/advanced_features/visualizations.md b/..._docs/advanced_features/visualizations.md → docs/advanced_features/visualizations.md
diff --git a/docs/source/conf.py → docs/conf.py b/docs/source/conf.py → docs/conf.py
@@ -25,7 +25,7 @@
 using [reticulate](https://rstudio.github.io/reticulate/) works.
 
 """
-file_obj = open('deepforest_docs/deepforestr.md', 'w')
+file_obj = open('deepforestr.md', 'w')
 readme_url = 'https://raw.githubusercontent.com/weecology/deepforestr/main/README.md'
 file_obj.write(deepforestr_title)
 
@@ -42,7 +42,7 @@
 
 # Create copy of CONTRIBUTING.md
 contributing_url = "https://raw.githubusercontent.com/weecology/DeepForest/main/CONTRIBUTING.md"
-contributing_source = "../../CONTRIBUTING.md"
+contributing_source = "../CONTRIBUTING.md"
 
 if not os.path.exists(contributing_source):
     with urllib.request.urlopen(contributing_url) as response:
@@ -52,7 +52,7 @@
 
 # reading from file1 and writing to file2
 with open(contributing_source, "r") as file1:
-    with open("deepforest_docs/CONTRIBUTING.md", "w") as file2:
+    with open("CONTRIBUTING.md", "w") as file2:
         file2.write(file1.read())
 
 needs_sphinx = "1.8"
@@ -78,7 +78,7 @@
 templates_path = ['_templates']
 
 # The master toctree document.
-master_doc = 'deepforest_docs/index'
+master_doc = 'index'
 
 # General information about the project.
 project = u'DeepForest'

diff --git a/...forest_docs/data_annotation/annotation.md → docs/data_annotation/annotation.md b/...forest_docs/data_annotation/annotation.md → docs/data_annotation/annotation.md
@@ -4,12 +4,12 @@ Annotation is the most important part of machine learning projects.  If you aren
 ## How should I annotate images?
 For quick annotations of a few images, we recommend using QGIS or ArcGIS. Either as project or unprojected data. Create a shapefile for each image.
 
-![QGISannotation](../../../../www/QGIS_annotation.png)
+![QGISannotation](../../www/QGIS_annotation.png)
 
 ### Label-studio
 For longer term projects, we recommend [label-studio](https://labelstud.io/) as an annotation platform. It has many useful features and is easy to set up.
 
-![QGISannotation](../../../../www/label_studio.png)
+![QGISannotation](../../www/label_studio.png)
 
 ## Do I need annotate all objects in my image?
 Yes! Object detection models use the non-annotated areas of an image as negative data. We know that it can be difficult to annotate all objects in an image, but non-annotation will cause the model *to ignore* objects that should be treated as positive samples, leading to poor model performance. 

diff --git a/...deepforest_docs/data_annotation/index.rst → docs/data_annotation/index.rst b/...deepforest_docs/data_annotation/index.rst → docs/data_annotation/index.rst
diff --git a/docs/source/deepforest_docs/deepforestr.md → docs/deepforestr.md b/docs/source/deepforest_docs/deepforestr.md → docs/deepforestr.md
diff --git a/docs/developer_resources/authors.rst b/docs/developer_resources/authors.rst
@@ -0,0 +1 @@
+.. include:: ../../AUTHORS.rst
diff --git a/...s/developer_resources/code_of_conduct.rst → docs/developer_resources/code_of_conduct.rst b/...s/developer_resources/code_of_conduct.rst → docs/developer_resources/code_of_conduct.rst
diff --git a/docs/developer_resources/history.rst b/docs/developer_resources/history.rst
@@ -0,0 +1 @@
+.. include:: ../../HISTORY.rst
diff --git a/...forest_docs/developer_resources/index.rst → docs/developer_resources/index.rst b/...forest_docs/developer_resources/index.rst → docs/developer_resources/index.rst
diff --git a/.../deepforest_docs/examples/Australia.ipynb → docs/examples/Australia.ipynb b/.../deepforest_docs/examples/Australia.ipynb → docs/examples/Australia.ipynb
diff --git a/...forest_docs/examples/nest_detection.ipynb → docs/examples/nest_detection.ipynb b/...forest_docs/examples/nest_detection.ipynb → docs/examples/nest_detection.ipynb
diff --git a/...urce/deepforest_docs/figures/Figure_1.png → docs/figures/Figure_1.png b/...urce/deepforest_docs/figures/Figure_1.png → docs/figures/Figure_1.png
diff --git a/.../deepforest_docs/figures/TrainingData.png → docs/figures/TrainingData.png b/.../deepforest_docs/figures/TrainingData.png → docs/figures/TrainingData.png
diff --git a/...ocs/figures/batch_classification_loss.svg → docs/figures/batch_classification_loss.svg b/...ocs/figures/batch_classification_loss.svg → docs/figures/batch_classification_loss.svg
diff --git a/...st_docs/figures/batch_regression_loss.svg → docs/figures/batch_regression_loss.svg b/...st_docs/figures/batch_regression_loss.svg → docs/figures/batch_regression_loss.svg
diff --git a/.../deepforest_docs/figures/iou_equation.png → docs/figures/iou_equation.png b/.../deepforest_docs/figures/iou_equation.png → docs/figures/iou_equation.png
diff --git a/...e/deepforest_docs/figures/output_17_1.png → docs/figures/output_17_1.png b/...e/deepforest_docs/figures/output_17_1.png → docs/figures/output_17_1.png
diff --git a/...e/deepforest_docs/figures/output_23_1.png → docs/figures/output_23_1.png b/...e/deepforest_docs/figures/output_23_1.png → docs/figures/output_23_1.png
diff --git a/...e/deepforest_docs/figures/output_28_0.png → docs/figures/output_28_0.png b/...e/deepforest_docs/figures/output_28_0.png → docs/figures/output_28_0.png
diff --git a/...e/deepforest_docs/figures/output_32_0.png → docs/figures/output_32_0.png b/...e/deepforest_docs/figures/output_32_0.png → docs/figures/output_32_0.png
diff --git a/...e/deepforest_docs/figures/output_65_0.png → docs/figures/output_65_0.png b/...e/deepforest_docs/figures/output_65_0.png → docs/figures/output_65_0.png
diff --git a/...e/deepforest_docs/figures/output_70_0.png → docs/figures/output_70_0.png b/...e/deepforest_docs/figures/output_70_0.png → docs/figures/output_70_0.png
diff --git a/...e/deepforest_docs/figures/output_71_0.png → docs/figures/output_71_0.png b/...e/deepforest_docs/figures/output_71_0.png → docs/figures/output_71_0.png
diff --git a/...igures/tree_predicted_bounding_boxes.jpeg → ...igures/tree_predicted_bounding_boxes.jpeg b/...igures/tree_predicted_bounding_boxes.jpeg → ...igures/tree_predicted_bounding_boxes.jpeg
diff --git a/...t_docs/figures/tree_predicted_labels.jpeg → docs/figures/tree_predicted_labels.jpeg b/...t_docs/figures/tree_predicted_labels.jpeg → docs/figures/tree_predicted_labels.jpeg
diff --git a/...cs/getting_started/Reading_and_Writing.md → docs/getting_started/Reading_and_Writing.md b/...cs/getting_started/Reading_and_Writing.md → docs/getting_started/Reading_and_Writing.md
diff --git a/...rest_docs/introduction/getting_started.md → docs/getting_started/getting_started.md b/...rest_docs/introduction/getting_started.md → docs/getting_started/getting_started.md
@@ -21,7 +21,7 @@ img = model.predict_image(path=sample_image_path, return_plot=True)
 plt.imshow(img[:,:,::-1])
 ```
 
-![](../../../../www/getting_started1.png)
+![](../../www/getting_started1.png)
 
 
 ** please note that this video was made before the deepforest-pytorch -> deepforest name change. **

diff --git a/...deepforest_docs/getting_started/index.rst → docs/getting_started/index.rst b/...deepforest_docs/getting_started/index.rst → docs/getting_started/index.rst
diff --git a/...pforest_docs/getting_started/sample.ipynb → docs/getting_started/sample.ipynb b/...pforest_docs/getting_started/sample.ipynb → docs/getting_started/sample.ipynb
diff --git a/docs/index.rst b/docs/index.rst
@@ -0,0 +1,9 @@
+:orphan:
+
+DeepForest
+========================
+
+.. toctree::
+   :maxdepth: 2
+
+   source/index
diff --git a/...stallation_and_setup/ConfigurationFile.md → ...stallation_and_setup/ConfigurationFile.md b/...stallation_and_setup/ConfigurationFile.md → ...stallation_and_setup/ConfigurationFile.md
diff --git a/...est_docs/installation_and_setup/index.rst → docs/installation_and_setup/index.rst b/...est_docs/installation_and_setup/index.rst → docs/installation_and_setup/index.rst
diff --git a/...cs/installation_and_setup/installation.md → docs/installation_and_setup/installation.md b/...cs/installation_and_setup/installation.md → docs/installation_and_setup/installation.md
diff --git a/...t_docs/installation_and_setup/prebuilt.md → docs/installation_and_setup/prebuilt.md b/...t_docs/installation_and_setup/prebuilt.md → docs/installation_and_setup/prebuilt.md
@@ -6,7 +6,7 @@ DeepForest has two prebuilt models: Bird Detection and Tree Crown Detection.
 
 The model was initially described in [Remote Sensing](https://www.mdpi.com/2072-4292/11/11/1309) on a single site. The prebuilt model uses a semi-supervised approach in which millions of moderate quality annotations are generated using a LiDAR unsupervised tree detection algorithm, followed by hand-annotations of RGB imagery from select sites. Comparisons among geographic sites were added to [Ecological Informatics](https://www.sciencedirect.com/science/article/pii/S157495412030011X). The model was further improved, and the Python package was released in [Methods in Ecology and Evolution](https://besjournals.onlinelibrary.wiley.com/doi/full/10.1111/2041-210X.13472).
 
-![image](../../../../www/MEE_Figure4.png)
+![image](../../www/MEE_Figure4.png)
 
 ### Citation
 > Weinstein, B.G.; Marconi, S.; Bohlman, S.; Zare, A.; White, E. Individual Tree-Crown Detection in RGB Imagery Using Semi-Supervised Deep Learning Neural Networks. Remote Sens. 2019, 11, 1309
@@ -23,15 +23,15 @@ The model was initially described in [Ecological Applications](https://esajourna
  ### Citation
 > Weinstein, B.G., Garner, L., Saccomanno, V.R., Steinkraus, A., Ortega, A., Brush, K., Yenni, G., McKellar, A.E., Converse, R., Lippitt, C.D., Wegmann, A., Holmes, N.D., Edney, A.J., Hart, T., Jessopp, M.J., Clarke, R.H., Marchowski, D., Senyondo, H., Dotson, R., White, E.P., Frederick, P. and Ernest, S.K.M. (2022), A general deep learning model for bird detection in high resolution airborne imagery. Ecological Applications. Accepted Author Manuscript e2694. https://doi-org.lp.hscl.ufl.edu/10.1002/eap.2694
 
-![image](../../../../www/example_predictions_small.png)
+![image](../../www/example_predictions_small.png)
 
 ```
 #Load deepforest model and set bird label
 m = main.deepforest(label_dict={"Bird":0})
 m.use_bird_release()
 ```
 
-![](../../../../www/bird_panel.jpg)
+![](../../www/bird_panel.jpg)
 
 We have created a [GPU colab tutorial](https://colab.research.google.com/drive/1e9_pZM0n_v3MkZpSjVRjm55-LuCE2IYE?usp=sharing
 ) to demonstrate the workflow for using the bird model.

diff --git a/...ce/deepforest_docs/introduction/index.rst → docs/introduction/index.rst b/...ce/deepforest_docs/introduction/index.rst → docs/introduction/index.rst
@@ -8,4 +8,3 @@ Welcome to the introduction section.
    :caption: Contents:
 
    landing
-   getting_started
diff --git a/...e/deepforest_docs/introduction/landing.md → docs/introduction/landing.md b/...e/deepforest_docs/introduction/landing.md → docs/introduction/landing.md
@@ -2,7 +2,7 @@
 
 DeepForest is a python package for training and predicting ecological objects in airborne imagery. DeepForest comes with models for immediate use and finetuning. Both are single class modules that can be extended to species classification based on new data. Users can extend these models by annotating and training custom models. 
 
-![](../../../../www/image.png)
+![](../../www/image.png)
 
 ## How does deepforest work?
 

diff --git a/docs/source/deepforest_docs/make.bat → docs/make.bat b/docs/source/deepforest_docs/make.bat → docs/make.bat
diff --git a/...ce/deepforest_docs/related_work/index.rst → docs/related_work/index.rst b/...ce/deepforest_docs/related_work/index.rst → docs/related_work/index.rst
diff --git a/...ource/deepforest_docs/related_work/use.md → docs/related_work/use.md b/...ource/deepforest_docs/related_work/use.md → docs/related_work/use.md
diff --git a/docs/source/deepforest_docs/requirements.txt → docs/requirements.txt b/docs/source/deepforest_docs/requirements.txt → docs/requirements.txt
diff --git a/...eepforest_docs/source/deepforest.data.rst → docs/source/deepforest.data.rst b/...eepforest_docs/source/deepforest.data.rst → docs/source/deepforest.data.rst
diff --git a/...rce/deepforest_docs/source/deepforest.rst → docs/source/deepforest.rst b/...rce/deepforest_docs/source/deepforest.rst → docs/source/deepforest.rst
diff --git a/docs/source/deepforest_docs/developer_resources/authors.rst b/docs/source/deepforest_docs/developer_resources/authors.rst
diff --git a/docs/source/deepforest_docs/developer_resources/history.rst b/docs/source/deepforest_docs/developer_resources/history.rst
diff --git a/docs/source/deepforest_docs/getting_started/getting_started.md b/docs/source/deepforest_docs/getting_started/getting_started.md
diff --git a/docs/source/deepforest_docs/index.rst b/docs/source/deepforest_docs/index.rst
diff --git a/docs/source/index.rst b/docs/source/index.rst
@@ -1,9 +1,61 @@
-:orphan:
+Welcome to DeepForest!
+**********************
 
-DeepForest
-========================
+DeepForest is a python package for airborne object detection and classification.
+
+**Tree crown prediction using DeepForest**
+
+.. image:: ../../www/OSBS_sample.png
+
+**Bird detection using DeepForest**
+
+.. image:: ../../www/bird_panel.jpg
+
+
+Why DeepForest?
+===============
+
+Observing the abundance and distribution of individual organisms is one of the foundations of ecology. Connecting broad-scale changes in organismal ecology, such as those associated with climate change, shifts in land use, and invasive species require fine-grained data on individuals at wide spatial and temporal extents.
+
+To capture these data, ecologists are turning to airborne data collection from uncrewed aerial vehicles, piloted aircraft, and earth-facing satellites. Computer vision, a type of image-based artificial intelligence, has become a reliable tool for converting images into ecological information by detecting and classifying ecological objects within airborne imagery.
+
+There have been many studies demonstrating that, with sufficient labeling, computer vision can yield near-human-level performance for ecological analysis. However, almost all of these studies rely on isolated computer vision models that require extensive technical expertise and human data labeling.
+In addition, the speed of innovation in computer vision makes it difficult for even experts to keep up with new innovations.
+
+To address these challenges, the next phase of ecological computer vision needs to reduce the technical barriers and move towards general models that can be applied across space, time, and taxa.
+
+DeepForest aims to be **simple**, **customizable**, and **modular**. DeepForest makes an effort to keep unnecessary complexity hidden from the ordinary user by developing straightforward functions like "predict_tile." The majority of machine learning projects actually fail due to poor data and project management, not clever models. DeepForest makes an effort to generate straightforward defaults, utilize already-existing labeling tools and interfaces, and minimize the effect of learning new APIs and code.
+
+Where can I get help, learn from others, and report bugs?
+=========================================================
+
+Given the enormous array of forest types and image acquisition environments, it is unlikely that your image will be perfectly predicted by a prebuilt model. Below are some tips and some general guidelines to improve predictions.
+
+Get suggestions on how to improve a model by using the [discussion board](https://github.com/weecology/DeepForest/discussions). Please be aware that only feature requests or bug reports should be posted on the issues page. The most helpful thing you can do is leave feedback on DeepForest `issue page`_. No feature or issue, or positive affirmation is too small. Please do it now!
+
+
+`Source code`_ is available on GitHub.
 
 .. toctree::
    :maxdepth: 2
 
-   deepforest_docs/index
+   ../introduction/index
+   ../installation_and_setup/index
+   ../getting_started/index
+   ../training_and_evaluation/index
+   ../data_annotation/index
+   ../advanced_features/index
+   ../developer_resources/index
+   ../deepforestr
+   modules.rst
+   ../related_work/index
+
+
+Indices and tables
+==================
+* :ref:`genindex`
+* :ref:`modindex`
+* :ref:`search`
+
+.. _issue page: https://github.com/weecology/DeepForest/issues
+.. _Source code: https://github.com/weecology/DeepForest.git
diff --git a/...source/deepforest_docs/source/modules.rst → docs/source/modules.rst b/...source/deepforest_docs/source/modules.rst → docs/source/modules.rst
diff --git a/...ocs/training_and_evaluation/Evaluation.md → docs/training_and_evaluation/Evaluation.md b/...ocs/training_and_evaluation/Evaluation.md → docs/training_and_evaluation/Evaluation.md
diff --git a/...st_docs/training_and_evaluation/better.md → docs/training_and_evaluation/better.md b/...st_docs/training_and_evaluation/better.md → docs/training_and_evaluation/better.md
@@ -13,14 +13,14 @@ The prebuilt model was trained on 10cm data at 400px crops. The model is sensiti
 tile = model.predict_tile("/Users/ben/Desktop/test.jpg",return_plot=True,patch_overlap=0,iou_threshold=0.05,patch_size=400)
 ```
 
-![](../../../../www/example_patch400.png)
+![](../../www/example_patch400.png)
 
 Acceptable, but not ideal.
 
 
 Here is 1000 px patches.
 
-![](../../../../www/example_patch1000.png)
+![](../../www/example_patch1000.png)
 
 
 improved.

diff --git a/...st_docs/training_and_evaluation/index.rst → docs/training_and_evaluation/index.rst b/...st_docs/training_and_evaluation/index.rst → docs/training_and_evaluation/index.rst
diff --git a/..._docs/training_and_evaluation/training.md → docs/training_and_evaluation/training.md b/..._docs/training_and_evaluation/training.md → docs/training_and_evaluation/training.md
diff --git a/setup.py b/setup.py
@@ -63,9 +63,11 @@
       license=LICENCE,
       packages=find_packages(),
       include_package_data=True,
-      install_requires=['albumentations>=1.0.0', 'aiolimiter', 'aiohttp', 'docformatter', 'huggingface_hub',
-                        'geopandas', 'matplotlib', 'nbqa', 'numpy', 'opencv-python-headless>=4.5.4', 'pandas', 'Pillow>6.2.0',
-                        'progressbar2', 'pycocotools', 'pydata-sphinx-theme', 'Pygments', 'pytorch-lightning>=1.5.8', 'rasterio',
-                        'recommonmark', 'rtree', 'scipy>1.5', 'six', 'slidingwindow', 'sphinx', 'supervision', 'torch',
-                        'torchvision>=0.13', 'tqdm', 'xmltodict', 'geopandas'],
+      install_requires=[
+          "albumentations>=1.0.0", "aiolimiter", "aiohttp", "docformatter", "huggingface_hub", "geopandas", "matplotlib", "nbqa", "numpy",
+          "'opencv-python-headless>=4.5.4'", "pandas", "Pillow>6.2.0", "progressbar2", "pycocotools", "pydata-sphinx-theme", "Pygments",
+          "pytorch-lightning>=1.5.8", "rasterio", "recommonmark", "rtree", "scipy>1.5",
+          "six", "slidingwindow", "sphinx", "supervision", "torch", "torchvision>=0.13", "tqdm",
+          "xmltodict","geopandas"
+      ],
       zip_safe=False)